[RFE] add additional QinQ modes to expand into automated support of LACP

[kambiz-aghaiepour]

Currently we have:

QinQ = 0 (default mode)
Isolated interfaces, each with separate QinQ enabled VLANs

QinQ = 1 (combined mode)
All private interfaces are placed within the same QinQ enabled VLAN

We should start thinking about ways to support LACP enabled modes. Here are some ideas:

QinQ = 2 (isolated QinQ vlans); "em1" and "em2" combined into an LACP configuration on the switch using
the QinQ VLAN intended for "em1"

QinQ = 3 (combined QinQ vlans); "em1" and "em2" combined into an LACP configuration on the switch. All
interfaces including LACP interface(s) use the QinQ VLAN intended for "em1"

QinQ = 4 (isolated QinQ vlans); "em3" and "em4" combined into an LACP configuration on the switch using
the QinQ VLAN intended for "em3"

QinQ = 5 (combined QinQ vlans); "em3" and "em4" combined into an LACP configuration on the switch. All
interfaces including LACP interface(s) use the QinQ VLAN intended for "em1"

QinQ = 6 (isolated QinQ vlans); "em1" and "em2" combined into an LACP configuration on the switch and
"em3" and "em4" combined into an LACP configuration on the switch. em1/em2 will use the QinQ
vlan used by em1. em3/em4 will use the QinQ vlan used by em3.

QinQ = 7 (combined QinQ vlans); "em1" and "em2" combined into an LACP configuration on the switch and
"em3" and "em4" combined into an LACP configuration on the switch. All interfaces including LACP
interface(s) use the QinQ VLAN indended for "em1"

In order to support this with Juniper switches, we have to ensure that the JunOS or JunOS-EVO configurations have the necessary configuration(s) and a way to track the assigned numbers for the aggregated interfaces. Specifically, the switch should have:

set chassis aggregated-devices ethernet device-count 128

Furthermore, we should reserve a range of aeXXX device names within each switch to be used with quads, and a way to track which interface are "in use" and which allocations they are associated with. Current use case is that the ToR switch uses ae0 or ae1 as the uplink to the core. As a matter of convention, we could reserve ae10 through ae99 to generate QUADS managed LACP interfaces. Here are some example configurations and how it might work.

Scenario

host1.example.com has two interfaces
em1 - attached to 192.168.1.101 on et-0/0/0:0
em2 - attached to 192.168.1.101 on et-0/0/0:1

host2.example.com has two interfaces
em1 - attached to 192.168.1.101 on et-0/0/0:2
em2 - attached to 192.168.1.101 on et-0/0/0:3

host3.example.com has two interfaces
em1 - attached to 192.168.1.102 on et-0/0/0:0
em2 - attached to 192.168.1.102 on et-0/0/0:1

Suppose the above 3 hosts were placed in an allocation (cloud02) using the new possible QinQ=2 mode (as described above), we would need to do a few things to ensure proper configurations are placed on each switch.

1. For each host attached to each switch, determine the first available aeXX interface to use, and reserve it for use with cloud02. This would likely be:

   ae10 on 192.168.1.101 - combine et-0/0/0:0 and et-0/0/0:1 and place in cloud02 in QinQ VLAN
   ae11 on 192.168.1.101 - combine et-0/0/0:2 and et-0/0/0:3 and place in cloud02 in QinQ VLAN
   ae10 on 192.168.1.102 - combine et-0/0/0:0 and et-0/0/0:1 and place in cloud02 in QinQ VLAN

2. Ensure that for each LACP configuration from 1, that the corresponding switch have LACP configurations. This may look like the following:

   On 192.168.1.101:

# first delete the interface configs
delete interfaces et-0/0/0:0
delete interfaces et-0/0/0:1
delete interfaces et-0/0/0:2
delete interfaces et-0/0/0:3

# May need to check the following (this assumes they are coming from cloud01)
delete vlans vlan1100 interface et-0/0/0:0
delete vlans vlan1101 interface et-0/0/0:1
delete vlans vlan1102 interface et-0/0/0:2
delete vlans vlan1103 interface et-0/0/0:3

# setup bonds
# host1.example.com
set interfaces et-0/0/0:0 ether-options 802.3ad ae10
set interfaces et-0/0/0:0 ether-options 802.3ad lacp force-up
set interfaces et-0/0/0:1 ether-options 802.3ad ae10
set interfaces et-0/0/0:1 ether-options 802.3ad lacp force-up

# host2.example.com
set interfaces et-0/0/0:2 ether-options 802.3ad ae11
set interfaces et-0/0/0:2 ether-options 802.3ad lacp force-up
set interfaces et-0/0/0:3 ether-options 802.3ad ae11
set interfaces et-0/0/0:3 ether-options 802.3ad lacp force-up

# ensure bonds are in the correct QinQ
set interfaces ae10 apply-groups QinQ_vl1110
set vlans vlan1110 interface ae10
set interfaces ae11 apply-groups QinQ_vl1110
set vlans vlan1190 interface ae102

Given that there are 20 or more hosts per rack, there needs to be very careful accounting of which aeXX interfaces are in use. The accounting needs to be done per switch. There is currently no "switch" metadata in QUADS, which may be an ideal place for tracking this type of information.

Furthermore adding this level of complexity means that additional checks will have to be done to ensure proper tear-down and build-up of switch configurations, as hosts are potentially going from a simple QinQ=0/1 configuration and going to/from a more complex LACP configuration.

[sadsfae]

Adding some relevant discussion here, it's determined the best way to accomplish this is augmenting quads --modify-switch-conf and have this done post-deployment.

At the same time, inclusive with this change should be making move_and_rebuild.py to know how to revert the configuration so things don't break when they move out of a finishing allocation.