IPMISELStateCritical fires false positives on Entity Presence sensors (BIOS_POST_CMPLT, MAIN_POWER_PRS)

[ggouzi]

Description

IPMISELStateCritical alert fires false positives on every node reboot due to Entity Presence type SEL entries, specifically for System Firmware BIOS_POST_CMPLT and Power Module 1 MAIN_POWER_PRS sensors.

Current behavior

Every planned or unplanned server reboot seem to generate 2 IPMISELStateCritical alerts per node, one for each of these SEL entry names:

• System Firmware BIOS_POST_CMPLT (BIOS POST Completion): Transitions Device Absent → Device Present during POST
• Power Module 1 MAIN_POWER_PRS (Main Power Presence): Transitions Device Absent → Device Present during power cycle

Both are Entity Presence type sensors that reflect normal hardware state transitions during boot, not actual failures.
On a 9-node deployment, this results in 18 false positive critical alerts on every maintenance window involving reboots.

Charm revision

latest/stable rev 891

Juju version

3.6.23

Cloud

MAAS

Expected behaviour

Entity Presence type SEL entries should be ignored by IPMISELStateCritical, as these sensors only reflect physical presence detection and do not indicate hardware faults.

Reproduce / Test

juju deploy hardware-observer --channel=latest/stable --revision=891
juju relate hardware-observer <whatever baremetal machine charm>
juju relate hardware-observer:cos-agent opentelemetry-collector:cos-agent
juju relate <random machine app>:juju-info hardware-observer:general-info

Notes & References

Cisco baremetal machines.

Similar to #101 ?

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/SOLENG-2154.

This message was autogenerated

[jneo8]

The difficult here is we want to ignore the Entity Presence type SEL entries for expected reboot but also catch the unexpected reboot.

I am not sure if we can have some practice like maintenance mode to temporary disable all the related alert for this case. This will may be a possible direction.

[H-M-Quang-Ngo]

Until we find out a good solution, we can first document this in hardware-observer.

I think a rough operation workflow is:

• see IPMISELStateCritical firing
• check for recent reboot and confirm harmless Entity Presence
• silence the alert

For solutions, it might be a maintenance mode support, or simpler, an ack-reboot action similar to nrpe charm.