kvm: Agent does not honor control.cidr anymore

[wido]

ISSUE TYPE

• Bug Report

COMPONENT NAME

KVM Agent

CLOUDSTACK VERSION

4.18

CONFIGURATION

• Advanced Networking
• FRR BGP daemon with BGP Unnumbered and VXLAN

OS / ENVIRONMENT

Ubuntu 22.04

SUMMARY

The 'control.cidr' configuration value from the agent.properties is no longer honered and thus the subnet 169.254.0.0/16 is always used.

This conflicts with BGP Unnumbered and thus renders the network configuration broken.

STEPS TO REPRODUCE

• Install KVM Agent 4.18
• Set control.cidr to 169.254.240.0/20
• See the cloud0 still gets the address 169.254.0.1/255.255.0.0

EXPECTED RESULTS

I would expect the CIDR to be honered.

LOGS

May 31 12:18:09 hv-pod2-r1-21 java[5559]: DEBUG [utils.script.Script] (main:) (logid:) Executing: /bin/bash -c ip address add 169.254.0.1/255.255.0.0 dev cloud0
May 31 12:18:09 hv-pod2-r1-21 java[5559]: DEBUG [utils.script.Script] (main:) (logid:) Cannot find device "cloud0"
May 31 12:18:09 hv-pod2-r1-21 java[5559]: DEBUG [utils.script.Script] (main:) (logid:) Executing: /bin/bash -c ip route add 169.254.0.0/16 dev cloud0 src 169.254.0.1
May 31 12:18:09 hv-pod2-r1-21 java[5559]: DEBUG [utils.script.Script] (main:) (logid:) Cannot find device "cloud0"
May 31 12:18:09 hv-pod2-r1-21 java[5559]: DEBUG [utils.script.Script] (main:) (logid:) Executing: /bin/bash -c ip link add name cloud0 type bridge
May 31 12:18:09 hv-pod2-r1-21 java[5559]: DEBUG [utils.script.Script] (main:) (logid:) Executing: /bin/bash -c ip link set cloud0 up
May 31 12:18:09 hv-pod2-r1-21 systemd-networkd[3152]: cloud0: Link UP
May 31 12:18:09 hv-pod2-r1-21 java[5559]: DEBUG [utils.script.Script] (main:) (logid:) Executing: /bin/bash -c ip address add 169.254.0.1/255.255.0.0 dev cloud0

This can be observed in the logs of the KVM Agent and here we can see that the /16 subnet is still used.

This used to work in versions prior to 4.18 and is now broken.

[wido]

Downgrading to 4.17.2 on the KVM Agent while keeping the Management Server at 4.18 does seem to resolve this issue:

[root@hv-pod2-r1-21 ~]# ip addr show dev cloud0
7: cloud0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 72:55:25:fd:dc:02 brd ff:ff:ff:ff:ff:ff
    inet 169.254.240.1/20 scope global cloud0
       valid_lft forever preferred_lft forever
    inet6 fe80::7055:25ff:fefd:dc02/64 scope link 
       valid_lft forever preferred_lft forever
[root@hv-pod2-r1-21 ~]# 

This breaks anybody using BGP with VXLAN on their CloudStack environment as the Agent will break the BGP communication.

[DaanHoogland]

@wido, i noticed that the property control.cidr is also set in the Management Server. Can you change it there (as a workaround)?

I'll try and find the commit that broke this.

[wido]

@wido, i noticed that the property control.cidr is also set in the Management Server. Can you change it there (as a workaround)?

I'll try and find the commit that broke this.

Yes, but that's for when the mgmt server generates the Link Local IPv4 address for the SystemVMs.

That still seems to work, but it's the Agent which creates the local link wrongly.

It might be commit 0fe2e69 which broke it.

[DaanHoogland]

experimental code yet, but have a look at #7580 , @wido ?