CloudStack agent trims Host.OS in the response to management server

[yadvr]

CloudStack agent trims the output for Host.OS from the version.sh (see below). In case of RHEL KVM hosts, CloudStack thinks they are "Red" OS and not "Red Hat Enterprise Linux", and doesn't allow the host to be added to an existing cluster.

# /usr/share/cloudstack-common/scripts/vm/hypervisor/versions.sh
Host.OS=Red Hat Enterprise Linux
Host.OS.Version=9.2
Host.OS.Kernel.Version=5.14.0-284.11.1.el9_2.x86_64

This causes NPE at:

2023-05-23 13:29:42,728 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-348:ctx-f5b5e351) (logid:36272aee) Failed to handle host connection:
java.lang.NullPointerException
        at com.cloud.hypervisor.kvm.discoverer.LibvirtServerDiscoverer.createHostVOForConnectedAgent(LibvirtServerDiscoverer.java:473)
        at com.cloud.resource.ResourceManagerImpl.dispatchToStateAdapters(ResourceManagerImpl.java:1997)

And the Host.OS in the cluster's first host's DB detail doesn't match that of the host being added:

https://github.com/apache/cloudstack/blob/4.18/server/src/main/java/com/cloud/hypervisor/kvm/discoverer/LibvirtServerDiscoverer.java#L471

Without DEBUG logs enabled in cloudstack-agent it may read as if libvirt is having issues.

ISSUE TYPE

• Bug Report

COMPONENT NAME

RHEL9 + ACS 4.18+

[NuxRo]

@DaanHoogland @rohityadavcloud version.sh has been getting things wrong for a while now and I bet there are users who have bogus entries in their host_details which will not match the correct entries of hypervisors added after this fix.

Not saying this shouldn't be fixed, of course it needs to, but may be worth mentioning something about it in the release notes at some point.

[DaanHoogland]

@DaanHoogland @rohityadavcloud version.sh has been getting things wrong for a while now and I bet there are users who have bogus entries in their host_details which will not match the correct entries of hypervisors added after this fix.

Not saying this shouldn't be fixed, of course it needs to, but may be worth mentioning something about it in the release notes at some point.

If we have examples (maybe the "Rocky" vs "Rocky Linux" from @weizhouapache 's test #7570 (comment)) we can add upgrade scripting for those.

Is this theoretical or can you reproduce issues @NuxRo ?

[DaanHoogland]

I am not sure if I should close this or leave it open per your comment @NuxRo. The PR is already merged. What do propose?

[NuxRo]

@DaanHoogland I've noticed in the past, at least with some RHEL versions that the entries in host_details are kind of broken, but because they are broken in the same way for all the hosts there is no problem.
This is not theoretical, I've seen it in the wild.

If people will be upgrading Cloudstack with this fix then add new HVs to their setup they might hit this problem, at least with some of the distros.

[weizhouapache]

@DaanHoogland @NuxRo
I restarted cloudstack-agent on a host, and got the exception

2023-06-09 14:49:01,657 DEBUG [c.c.a.m.AgentManagerImpl] (AgentConnectTaskPool-9:ctx-f3c5da4e) (logid:7f27c10b) Failed to handle host connection:
java.lang.IllegalArgumentException: Can't add host: 10.0.32.193 with hostOS: Rocky Linux into a cluster,in which there are Rocky hosts added
        at com.cloud.hypervisor.kvm.discoverer.LibvirtServerDiscoverer.createHostVOForConnectedAgent(LibvirtServerDiscoverer.java:474)
        at com.cloud.resource.ResourceManagerImpl.dispatchToStateAdapters(ResourceManagerImpl.java:1997)
        at com.cloud.resource.ResourceManagerImpl.createHostVO(ResourceManagerImpl.java:2224)
        at com.cloud.resource.ResourceManagerImpl.createHostVOForConnectedAgent(ResourceManagerImpl.java:2556)

I have created an issue #7616. we need to solve it.

[yadvr]

Fixed in #7570 I see a new #7616 so this issue can be closed