Unable to Introspect Baremetal Nodes

Problem

  • Introspection fails as Dnsmasqfilter is stuck in 'uninitialized' state

2024-07-17 04:30:20.891 1289 ERROR ironic_inspector.node_cache [-] [node: 5f0ef0df-8131-43fc-32r3-2bb559c1ed14 state processing] Processing the error event because of an exception <class 'ironic_inspector.pxe_filter.base.InvalidFilterDriverState'>: The PXE filter driver DnsmasqFilter, state=uninitialized: my fsm encountered an exception: Can not transition from state 'uninitialized' on event 'sync' (no defined transition) raised by ironic_inspector.process._process_node: ironic_inspector.pxe_filter.base.InvalidFilterDriverState: The PXE filter driver DnsmasqFilter, state=uninitialized: my fsm encountered an exception: Can not transition from state 'uninitialized' on event 'sync' (no defined transition)meterValue: Cannot validate PXE bootloader. Some parameters were missing in node's driver_info and configuration. Missing are: ['deploy_ramdisk']
  • This causes baremetal hosts to get stuck into enroll state.

Environment

  • Platform9 Managed OpenStack - v5.0

Cause

  • The pxefilter driver is unable to initialize the DNSmasq filter. The process of synchronizing DHCP for introspection and hardware discovery in larger clusters can be time-consuming due to operational overhead. This can lead to processes remaining continuously busy, potentially impacting the successful introspection of recently added nodes.

Resolution

  • It is suggested to adjust the [pxe_filter]sync period setting to a larger value, allowing conductors more time between synchronization runs.

  • Create an ironic_override.conf file with permissions pf9:pf9group inside the /opt/pf9/etc/ironic-inspector/conf.d/ directory.

  • Add the below content to the override file.

[pxe_filter]
driver = dnsmasq
sync_period = 60
  • Finally restart the ironic services over the host

# systemctl restart pf9-ironic-inspector
# systemctl restart pf9-ironic-conductor