Unable to Introspect Baremetal Nodes

Problem

• Introspection fails as Dnsmasqfilter is stuck in 'uninitialized' state

2024-07-17 04:30:20.891 1289 ERROR ironic_inspector.node_cache [-] [node: 5f0ef0df-8131-43fc-32r3-2bb559c1ed14 state processing] Processing the error event because of an exception <class 'ironic_inspector.pxe_filter.base.InvalidFilterDriverState'>: The PXE filter driver DnsmasqFilter, state=uninitialized: my fsm encountered an exception: Can not transition from state 'uninitialized' on event 'sync' (no defined transition) raised by ironic_inspector.process._process_node: ironic_inspector.pxe_filter.base.InvalidFilterDriverState: The PXE filter driver DnsmasqFilter, state=uninitialized: my fsm encountered an exception: Can not transition from state 'uninitialized' on event 'sync' (no defined transition)meterValue: Cannot validate PXE bootloader. Some parameters were missing in node's driver_info and configuration. Missing are: ['deploy_ramdisk']

• This causes baremetal hosts to get stuck into enroll state.

Environment

• Platform9 Managed OpenStack - v5.0

Cause

• The pxefilter driver is unable to initialize the DNSmasq filter. The process of synchronizing DHCP for introspection and hardware discovery in larger clusters can be time-consuming due to operational overhead. This can lead to processes remaining continuously busy, potentially impacting the successful introspection of recently added nodes.

Resolution

• It is suggested to adjust the [pxe_filter]sync period setting to a larger value, allowing conductors more time between synchronization runs.
• Create an ironic_override.conf file with permissions pf9:pf9group inside the /opt/pf9/etc/ironic-inspector/conf.d/ directory.
• Add the below content to the override file.

xxxxxxxxxx

[pxe_filter] 

   driver = dnsmasq 

   sync_period = 60

• Finally restart the ironic services over the host

xxxxxxxxxx

# systemctl restart pf9-ironic-inspector 

# systemctl restart pf9-ironic-conductor