Scheduled VMs does not change based on filtering

[WizardBit]

Description

In the OpenStack Compute dashboard there is a "Scheduled VMs" panel that does not update when filters are changed.

Charm revision

latest/stable rev59

Juju version

3.6.4

Cloud

MAAS

Expected behaviour

The OpenStack Compute dashboard should update the "Scheduled VMs" panel based on the filtering.

Reproduce / Test

Try to filter "Scheduled VMs" based on project, hypervisor, etc.

Notes & References

No response

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/SOLENG-1744.

This message was autogenerated

[Pjack]

What is the openstack environment? Is charmed-openstack or Sunbeam?

[Deezzir]

@WizardBit, I've looked into the current dashboard, and it seems that two filters are used on the metrics

1. hostname=~"$hypervisor.*" - by Hostname
2. project_name=~"$project" - by Project

The Scheduled VMs panel can currently only be filtered by project, and the metrics used are libvirt_domain_info_meta.

Can you share metrics from the deployment to confirm that the above metric includes the hostname label?

Also, Instances stats uses the same project filter (only) to aggregate stats. Can you also confirm that it does not react to the filters you choose? Thanks!

Reference:

• Instance stats
• Scheduled VMs

[WizardBit]

If I use my solution from #162 I find that I'm able to then filter Scheduled VMs by hypervisor, but then not projects.

While looking at libvirt_domain_info_meta in prometheus it does not have a hostname label. However, I found that we could possibly filter by dns_name instead. Which in the case of this environment I have is the same as the hypervisor name.

sum by(project_name) (libvirt_domain_info_meta{juju_application=~"$juju_application",juju_model=~"$juju_model",juju_model_uuid=~"$juju_model_uuid",juju_unit=~"$juju_unit",project_name=~"$project",dns_name=~"$hypervisor"})

Adding dns_name=~"$hypervisor" to libvirt_domain_info_meta, is this a viable solution for both Instance stats and Scheduled VMs?

[WizardBit]

Overall I think there needs to be consistency between the libvirt_domain_info_meta and openstack_nova_running_vms filtering options to make all the panels in the dashboard coherent or else there's remaining confusion when analyzing the dashboard in its current configuration.

[Deezzir]

@WizardBit Adding dns_name label as a filter sounds like a viable solution. Can we just confirm that it is consistently equal to hostname across different environments? Thanks for the insight!

Edit: I don't see that dns_name label in the libvirt-exporter docs, only the domain label

[WizardBit]

I can't find it in the libvirt-exporter docs, but we can see the charm shows the label being used here.

https://github.com/canonical/charm-prometheus-libvirt-exporter/blob/main/src/files/grafana-dashboards/libvirt.json

Also this part in cos-proxy is mentioned, though I'm not exactly sure how this relates.

https://github.com/canonical/cos-proxy-operator/blob/f186dd1c373550f3a1ad36d6586955fd26fdf1aa/src/metrics_endpoint_aggregator.py#L149-L151

From the environment I'm using here's one of the results of the libvirt_domain_info_meta{} query with information redacted.

libvirt_domain_info_meta{dns_name="mycompute", domain="instance-00000006", flavor="small", host="x", instance="x", instance_name="x, job="x", juju_application="prometheus-libvirt-exporter", juju_model="openstack", juju_model_uuid="x", juju_unit="prometheus-libvirt-exporter/0", project_name="x", project_uuid="x", user_name="admin", user_uuid="x", uuid="x"}

[Deezzir]

@WizardBit I guess, we can add filtering by dns_name to all panels and graphs - it should be non trivial. The main problem will still be there, as there is no way to filter openstack metrics by project. I see two possible solutions:

1. Remove the libvirt metrics from the dashboard and migrate to openstack only metrics - Migrate metric queries from libvirt to OpenStack #162
2. Remove filtering by projects and add dns_name/hostname as the only filter

In my opinion, the first solution makes more sense and we should leave everything as is for now and work on the migration. WDYT?

[WizardBit]

I agree with migrating from libvirt to OpenStack metrics, but this will affect the way admins currently use the dashboards and might have a need to filter by project. Not sure what the work around for that would be.

[Deezzir]

@WizardBit Can you confirm that project filter is a must-have for Managed Solutions or is it something that we can consider dropping if we decide to migrate? Thanks!

[marcusboden]

Well, we don't use it much ourselves, but some of our customers may use the project filtering. I would prefer not to drop it completely.

[WizardBit]

I've been thinking about this issue and decided to create a feature request here openstack-exporter/openstack-exporter#526. Hopefully this is an easy feature to implement

[Deezzir]

Thanks, @WizardBit. I think we can push for the feature to be implemented, so we can consider migrating to the OpenStack metrics. Let's wait for the upstream response, I'll will also keep an eye on the request, and provide some context if needed.