Feat: Add support for updating event capabilities on save

[niccoloalfredo]

Pull Request: Automatic event capabilities management for OpenStack providers

References

Closes #925

Note: This PR is submitted from my personal account (@niccoloalfredo). The original issue #925 was opened and discussed from my company account (@Niccolo-Alfredo).

Summary

This PR implements the callback-based solution discussed in #925 to automatically manage capabilities["events"] for OpenStack providers. The implementation adds an after_save callback that dynamically sets the event capability based on the actual configuration and availability of event brokers (AMQP, Ceilometer, or STF).

Problem

As detailed in #925, the verify_credentials method contains an early exit that prevented automatic setting of capabilities["events"] when credentials were verified with an explicit auth_type. This required manual intervention via Rails console to enable event catchers:

ems.capabilities['events'] = true
ems.save!

Solution

Instead of modifying the verify_credentials interface (which has architectural complexity as discussed with @kbrock), this PR implements an after_save callback in ManagerMixin that:

1. Automatically detects event broker configuration - Checks for active AMQP, Ceilometer, or STF endpoints/authentications
2. Verifies broker availability - Calls event_monitor_available? to confirm the broker is reachable
3. Updates capabilities dynamically - Sets capabilities["events"] based on actual configuration state
4. Handles all lifecycle scenarios - Creation, updates, additions, and removals of event configuration

Testing

Thoroughly tested all scenarios with AMQP (the other brokers follow the same code path via event_monitor_options and should behave identically):

• ✅ Create provider with AMQP → capabilities["events"] = true, EventCatchers start automatically
• ✅ Create provider without events → capabilities["events"] = false or nil
• ✅ Add AMQP to existing provider → capabilities["events"] becomes true
• ✅ Remove AMQP from provider → capabilities["events"] becomes false
• ✅ Multiple OpenStack providers → Each provider manages its own capability independently
• ✅ Broker unavailable → capabilities["events"] set to false, logged as warning

Implementation Details

The callback:

1. Executes only on CloudManager - Avoids duplicate execution on subproviders (CinderManager, NetworkManager)
2. Checks event monitor configuration - Via event_monitor_options which detects configured broker type (AMQP/Ceilometer/STF)
3. Tests broker availability - Via event_monitor_available? which performs actual connection test
4. Updates atomically - Only saves if capabilities["events"] value actually changed
5. Handles errors gracefully - Catches exceptions and sets capability to false with warning log

Broker Support

While testing was performed with AMQP, the implementation supports all three OpenStack event brokers:

• AMQP - Tested and working ✅
• Ceilometer - Uses same code path, should work identically
• STF - Uses same code path, should work identically

All three brokers are handled uniformly through event_monitor_options → event_monitor_available? → OpenstackEventMonitor.available?.

Additional Notes

This solution was preferred over modifying verify_credentials because:

• It doesn't require changes to the credential verification interface
• It works automatically for all event configuration changes (not just during credential verification)
• It's more maintainable and follows established patterns in the codebase

---

Ready for review. Please let me know if any adjustments are needed or if additional testing would be valuable before merge.

[Fryguy]

after_save might be expensive, because the manager gets updated rather frequently when we update the last_refreshed_at column. I'm not sure I understand why we wouldn't do it immediately after verification (I thought that's where we set capabilities for other providers). However @agrare may have other opinions.

[agrare]

Yeah after_save isn't going to work, event_monitor_available? hits the provider API and ExtManagementSystem save could be run from e.g. the ui/api which isn't guaranteed to have the ems_operations role in the right zone.

This should be run from verify_credentials, I need to dig into how that is called and why the auth_type guard was added in the first place.

[Fryguy]

Oh good point - I forgot that method needs to run on particular roles, which means it can't be in an after_save which could be run from anywhere.

[Fryguy]

update_event_capabilities is good in isolation though - we can call that from somwhere else - it's only the after_save part I'm concerned about.

[Fryguy]

I think you can set it anyway, and Rails will just do the right thing. That is, I think this is effectively equivalent:

Suggested change

	# Update only if the value changed
	if capabilities["events"] != expected_value
	capabilities["events"] = expected_value
	save! if changed?
	end
	capabilities["events"] = expected_value
	save! if changed?

[Fryguy]

@agrare Is event_monitor_available? the right method for capabilities? That is, if the event monitor goes down for some reason, say network issues, that doesn't me the provider no longer has that capability - it's just currently not working. Maybe I'm just confused on what the purpose of the capabilities is.

[Fryguy]

The more I think about it, we should set capabilities[:events] = opts[:events_monitor].present?. That is, if the user defined the events creds (regardless of if they are currently working on not), that is a statement of "yes, I expect events to be supported on this provider", and thus it has the capability (in lieu of us actually being able to detect the capability, which would be a better approach if possible)

[niccoloalfredo]

---

Hello @Fryguy and @agrare,

Thank you for your invaluable feedback and for highlighting the architectural issues with after_save.

I confirm your concerns: after_save is triggered on every provider refresh, making the event_monitor_available? check far too expensive and unsuitable for production.

---

Plan & Architecture Changes

I am now fully committed to moving this logic into verify_credentials or a similar appropriate hook to ensure it runs in the correct ems_operations context. I still need to refine the integration logic, though.

1. Capability Check Revision & Context

I want to clarify that I initially included the event_monitor_available? check because it was the logic used in the existing verify_credentials method within manageiq/providers/openstack/cloud_manager.rb, where it's the only place this capability is cited for OpenStack:

def verify_credentials(auth_type = nil, options = {})
  options[:service] ||= "Compute"
  ret = super
  return ret unless auth_type.nil?

  capabilities["events"] = !!event_monitor_available?
  save! if changed?

  true
end

I moved the implementation to manager_mixin.rb hoping to generalize it. Regarding the check itself: I am currently evaluating whether to eliminate this check entirely, which is tied to the availability verification performed by OpenstackEventMonitor.available?(event_monitor_options). I agree that capability should reflect configuration, not runtime availability, and the Event Catcher workers should ideally handle connection failures gracefully.

2. verify_credentials Coverage Challenge

I'm currently working to integrate the updated logic into verify_credentials. The most insidious case is reliably updating the capability to false when the user removes all event configurations (e.g., removing AMQP). Current placement attempts within verify_credentials aren't reliably covering this removal scenario, so I still need to refine this logic. I will also apply the suggested simplification to the save! logic.

3. Automated Spec Failures

Finally, I see that the automated specs have failed, particularly in the areas of AMQP credential validation and EventCatcher eligibility. I will review these failed tests and ensure the new logic, once moved from after_save into verify_credentials, properly integrates with and passes these existing expectations.

---

Related PR for Review

While I work on the re-implementation here, I would appreciate it if you could also take a look at PR #930, which fixes the network manager `event_target_parser.rb

[niccoloalfredo]

I see that the automated specs have failed, particularly in the areas of AMQP credential validation and EventCatcher eligibility. I will review these failed tests and ensure the new logic, once moved from after_save into verify_credentials, properly integrates with and passes these existing expectations.

[agrare]

We need something to indicate if the event endpoint is available not just configured. This is used by the EventCatcher worker to check if it is available to start.

We can't use the authentication status, because most of the event types re-use the default authentication record so marking the authentication invalid if ceilometer went down would stop refresh as well.

I agree capabilities["events"] should be if events have been configured not just if they are currently available. Maybe a new column on the Endpoint record to track if the "service" is available (this could conflict with authentication_status so we have to be careful)

[Fryguy]

We can't use the authentication status, because most of the event types re-use the default authentication record

Oh interesting I didn't realize the auth status was on the auth as opposed to the endpoint. It probably makes more sense in the endpoint because an auth is only valid or invalid with respect to each endpoint it's used on. Even so that's probably an invasive change. 🤔

[niccoloalfredo]

---

Hello @Fryguy and @agrare,
this is a quick update on the PR – I hope it clarifies the implementation and the current approach. I’ve included the automatic event capabilities logic for OpenStack providers, and I’m happy to get your feedback on whether this is the right path forward.

PR Update – Automatic Event Capabilities for OpenStack Providers

Summary

This PR introduces automatic management of capabilities["events"] for OpenStack providers by separating configuration detection from runtime availability checks.

Implementation

• ManagerMixin (manager_mixin.rb)
  Detects whether the provider has an event stream configured by inspecting endpoints and authentications for AMQP, Ceilometer, or STF. Updates capabilities["events"] based on the configuration state, ensuring it reflects expected support. This handles creation, updates, and removal of event streams automatically.

• CloudManager (cloud_manager.rb)
  Responsible for checking actual broker connectivity via verify_credentials. Updates capabilities["events_available"] to indicate if the Event Catcher can currently connect, without altering the configured capability. Non-event auth_types are skipped.

Advantages

• Clear separation between configuration and runtime availability.
• Avoids expensive API calls in after_save or during frequent provider refreshes.
• Ensures Event Catchers start correctly when events are configured and broker connectivity is available.
• Fully covers provider lifecycle changes (create, update, add/remove endpoints) and multiple providers independently.

Notes & Next Steps

• I have not yet added dedicated spec tests for this feature.
• I am open to feedback and suggestions on whether this approach is the right path for automatic event capability management.

---

Final note:
This change, and also #930, it’s needed to make the target refresh flow work properly, so that EventCatchers can start during our cloud infrastructure setup. For now, we’ll proceed with a forked version of radjabov-1 to address this immediate need.

I’m looking forward to contributing to a more complete version in the future, so that with the next update the full flow works seamlessly and this work can benefit the entire community.

[miq-bot]

Checked commit sourcesense@c2e5306 with ruby 3.1.7, rubocop 1.56.3, haml-lint 0.64.0, and yamllint
2 files checked, 3 offenses detected

app/models/manageiq/providers/openstack/cloud_manager.rb

• ❗ - Line 493, Col 7 - Rails/SkipsModelValidations - Avoid using update_columns because it skips validations.

app/models/manageiq/providers/openstack/manager_mixin.rb

• ❗ - Line 288, Col 5 - Rails/SkipsModelValidations - Avoid using update_columns because it skips validations.
• ❗ - Line 7, Col 61 - Style/HashSyntax - Use hash rockets syntax.

[Fryguy]

@agrare Please review. I think we missed the recent update on this one.

[agrare]

Closing in favor of #941

[miq-bot]

This pull request is not mergeable. Please rebase and repush.