strongswan-ipsec: Azure-optimal defaults, automatic VAES kernel, pref…

README.md

@@ -157,6 +157,10 @@ Lets pods on Crusoe Managed Kubernetes assume AWS IAM roles directly via IRSA, u
 
 ### Networking
 
+[Bandwidth Test](./bandwidth-test/)
+
+An Ansible-driven iperf3 harness that generates traffic from one-or-many source hosts to one-or-many destination hosts and collects the results from the receivers, so it measures correctly even when the path in between is a managed appliance you don't control — an Azure VPN Gateway, a GCP HA VPN, a customer firewall. Supports paired, many-to-one and full-mesh topologies, TCP or UDP, and a public-IP baseline mode for comparing a tunnel against the raw path.
+
 [/etc/hosts Pin](./etchosts-pin/README.md)
 
 A daemon that resolves a hostname on a fixed interval and keeps the resulting A/AAAA records in `/etc/hosts`. Works around undesirable TTL cache values from intermediate DNS resolvers
@@ -165,9 +169,13 @@ A daemon that resolves a hostname on a fixed interval and keeps the resulting A/
 
 A Helm chart that establishes a highly-available IPSec VPN between a remote site and a CMK cluster using paired StrongSwan deployments with BGP-based dynamic route sharing, so that pod, node, and service IPs on both sides are mutually reachable. Includes an example Terraform module for standing up a matching Google Cloud VPN endpoint.
 
+[Crusoe Site-to-Site VPN (AWS / GCP)](./site-to-site-vpn/)
+
+A hardened, redundant route-based IPsec (IKEv2) VPN terminating on one or two Ubuntu VMs running strongSwan and FRR, with BGP dynamic routing and automatic tunnel failover, provisioned entirely by Terraform from a single params file. Pairs with AWS Site-to-Site VPN or GCP HA VPN; the GCP path and dual-VM HA mode are validated end to end.
+
 [StrongSwan Site-to-Site VPN for Crusoe Cloud](./strongswan-ipsec/)
 
-An Ansible-managed, encrypted IPsec site-to-site VPN between a Crusoe Cloud region and a remote site — another Crusoe region, or Azure/GCP/AWS — with VMs on both sides communicating via their real (non-NAT'd) IP addresses. Uses GRE-over-FOU on the Crusoe side to work around SDN port-security source-IP checks, while the remote cloud side relies on native IP-forwarding; supports adding VMs incrementally via inventory changes.
+An Ansible-managed, encrypted IPsec site-to-site VPN between a Crusoe Cloud region and a remote site — another Crusoe region, or Azure/GCP/AWS — with VMs on both sides communicating via their real (non-NAT'd) IP addresses over a GRE-over-FOU overlay or plain routes. You fill in an inventory and five values, and one command preflights connectivity, installs a VAES-capable kernel where it pays, and configures gateways and clients with defaults tuned for a managed cloud peer. Measured at 2.4 Gbps with one gateway per side rising to 20.7 Gbps with five, and 8.5 Gbps through a single VM holding two tunnels; also configures managed Kubernetes nodes via a DaemonSet.
 
 ## Contributing
 

bandwidth-test/.gitignore

@@ -0,0 +1,10 @@
+# Fetched iperf3 results land here (bw_results_dir, relative to ansible/).
+# They are per-run measurements, not part of the solution.
+ansible/results/
+results/
+
+# Ansible
+*.retry
+
+# Editor/OS
+.DS_Store

bandwidth-test/README.md

@@ -0,0 +1,121 @@
+# Bandwidth Test
+
+Point-to-point and many-to-many throughput measurement with iperf3, driven by
+Ansible. Generates traffic from one-or-many source hosts to one-or-many
+destination hosts and **collects the results from the servers**.
+
+Use it to measure a VPN tunnel, a peering link, a managed cloud gateway
+(Azure ↔ GCP), or just two VMs in the same subnet for a baseline. It does not
+care what is in the middle.
+
+## Why server-side collection
+
+The receiving end is the authoritative measurement — it counts what actually
+arrived. It is also the only end that still reports usefully when the path in
+between is an appliance you do not control: a managed VPN gateway, a customer
+firewall, a carrier link. If the path degrades badly enough, a client's own
+summary can be truncated or lost; the server's file is already on disk.
+
+Every flow gets its own port and its own one-shot server (`iperf3 -s -1`), so
+each result is a clean, complete JSON object rather than an appended stream.
+
+## Quick start
+
+```bash
+cd ansible
+vim inventory.ini          # source_hosts and dest_hosts, with bw_private_ip
+ansible-playbook -i inventory.ini site.yml
+```
+
+Output:
+
+```
+======================================================================
+BANDWIDTH TEST - 20 flow(s) measured at the receiver
+======================================================================
+  flow                                Gbps   secs   retrans
+  000-src-1-to-dst-1                  2.11   30.0       412
+  001-src-2-to-dst-2                  2.08   30.0       380
+  ...
+----------------------------------------------------------------------
+  TOTAL                              42.41 Gbps across 20 flow(s)
+  mean per flow                       2.12 Gbps
+  total retransmits                 26,417,060
+```
+
+## Topologies
+
+| `bw_mode` | Meaning |
+|---|---|
+| `paired` (default) | `source[i]` → `dest[i]`. Groups must be the same size. |
+| `many_to_one` | every source → `dest[0]`. Load-tests one receiver. |
+| `mesh` | every source → every destination. Flows = sources × destinations. |
+
+```bash
+ansible-playbook -i inventory.ini site.yml -e bw_mode=many_to_one
+```
+
+## Common runs
+
+```bash
+# baseline: straight over the public IPs, bypassing whatever you are testing
+ansible-playbook -i inventory.ini site.yml -e bw_dest_address=public
+
+# UDP, to separate packet loss from TCP's reaction to it
+ansible-playbook -i inventory.ini site.yml -e bw_protocol=udp -e bw_udp_bitrate=2G
+
+# the other direction
+ansible-playbook -i inventory.ini site.yml -e bw_reverse=true
+
+# longer, more streams
+ansible-playbook -i inventory.ini site.yml -e bw_duration=120 -e bw_streams=16
+```
+
+## Reading the results
+
+- **Run a baseline first** (`bw_dest_address=public`). Without it you cannot
+  tell "the tunnel is slow" from "the path is slow" or "the hosts are small".
+- **Never judge on a single stream.** One TCP flow pins to one path and one
+  CPU core by design. Use `bw_streams` ≥ 8, or several flows.
+- **Watch retransmits, not just Gbps.** A large retransmit count means the
+  path is being driven past its knee. Throughput can look good while the link
+  is thrashing.
+- **If TCP is slow but UDP is clean**, you are looking at MTU, MSS, or an
+  offload problem, not capacity. Re-run with `bw_protocol=udp` at a few packet
+  sizes to confirm.
+- **Compare like with like.** Stream count and duration both change the
+  number; keep them fixed across the runs you intend to compare.
+
+## Requirements
+
+- SSH access to every host, with `become` for the package install
+- `iperf3` (installed automatically unless `bw_install: false`)
+- The chosen port range open between sources and destinations:
+  `bw_port_base` .. `bw_port_base + flows - 1` (default 5201+)
+- Python 3 on the controller for the summary
+
+## Tunables
+
+Everything lives in `ansible/roles/bandwidth_test/defaults/main.yml`:
+
+| Variable | Default | Purpose |
+|---|---|---|
+| `bw_mode` | `paired` | `paired`, `many_to_one`, `mesh` |
+| `bw_dest_address` | `private` | `private` through the path, `public` for a baseline |
+| `bw_duration` | `30` | seconds per flow |
+| `bw_streams` | `8` | parallel TCP streams per flow |
+| `bw_omit` | `5` | seconds of slow start discarded |
+| `bw_protocol` | `tcp` | `tcp` or `udp` |
+| `bw_udp_bitrate` | `1G` | per flow, UDP only |
+| `bw_reverse` | `false` | measure destination → source |
+| `bw_port_base` | `5201` | first port; one per flow |
+| `bw_results_dir` | `results` | where fetched JSON lands |
+| `bw_install` | `true` | install iperf3 if missing |
+
+## Limitations
+
+- One flow per source/destination pair per run. For more concurrency per pair,
+  raise `bw_streams` or use `mesh`.
+- No traffic shaping or scheduling — every flow starts at once.
+- Results are raw iperf3 JSON. The aggregator summarises; it does not chart.
+- IPv4 only, as written.

bandwidth-test/ansible/ansible.cfg

@@ -0,0 +1,9 @@
+[defaults]
+roles_path = roles
+host_key_checking = False
+forks = 50
+gathering = smart
+
+[ssh_connection]
+pipelining = True
+ssh_args = -o ControlMaster=auto -o ControlPersist=300s -o StrictHostKeyChecking=no

bandwidth-test/ansible/group_vars/all.yml

@@ -0,0 +1,11 @@
+---
+# The inventory groups are FIXED: [source_hosts] and [dest_hosts]. Ansible
+# resolves play targets before variables load, so site.yml names them
+# literally and they cannot be renamed here.
+
+# See roles/bandwidth_test/defaults/main.yml for every tunable.
+# bw_mode: "paired"        # paired | many_to_one | mesh
+# bw_duration: 30
+# bw_streams: 8
+# bw_protocol: "tcp"       # tcp | udp
+# bw_dest_address: "private"

bandwidth-test/ansible/inventory.ini

@@ -0,0 +1,24 @@
+# ===========================================================================
+# Bandwidth test inventory
+#
+# Two groups. Sources generate traffic, destinations receive and measure it.
+# They can be anywhere - either side of a VPN, either side of a managed
+# gateway, or the same subnet for a baseline.
+#
+# bw_private_ip is the address the sources should connect to (the one that
+# routes through the path you want to measure). ansible_host is used only for
+# SSH, and for the "public" baseline mode.
+# ===========================================================================
+
+[source_hosts]
+src-1  ansible_host=<public-ip>  bw_private_ip=<private-ip>
+src-2  ansible_host=<public-ip>  bw_private_ip=<private-ip>
+
+[dest_hosts]
+dst-1  ansible_host=<public-ip>  bw_private_ip=<private-ip>
+dst-2  ansible_host=<public-ip>  bw_private_ip=<private-ip>
+
+[all:vars]
+ansible_user=ubuntu
+ansible_become=true
+ansible_ssh_common_args=-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null

bandwidth-test/ansible/roles/bandwidth_test/defaults/main.yml

@@ -0,0 +1,65 @@
+---
+# ===========================================================================
+# bandwidth_test role
+#
+# Generates iperf3 traffic from one-or-many source hosts to one-or-many
+# destination hosts and collects the results FROM THE SERVERS.
+#
+# Server-side collection is the point. The receiving end is the authoritative
+# measurement, and it is the only end that still reports usefully when the
+# path in between is a managed appliance you do not control - an Azure VPN
+# Gateway, a GCP HA VPN, a customer firewall. A client's own summary can be
+# lost or truncated when the path degrades; the server's is not.
+# ===========================================================================
+
+# ---------------------------------------------------------------------------
+# What talks to what
+# ---------------------------------------------------------------------------
+#   paired       source[i] -> dest[i]        (equal-length groups; the default)
+#   many_to_one  every source -> dest[0]
+#   mesh         every source -> every dest  (sources x dests flows)
+bw_mode: "paired"
+
+# Which address on each destination the sources connect to.
+#   private  the destination's private IP  (through the tunnel / VPN)
+#   public   the destination's public IP   (direct, for a baseline)
+# Set bw_dest_ip on a host to override it explicitly.
+bw_dest_address: "private"
+
+# ---------------------------------------------------------------------------
+# Test parameters
+# ---------------------------------------------------------------------------
+bw_duration: 30
+bw_streams: 8
+bw_omit: 5                  # seconds of slow start to discard
+bw_protocol: "tcp"          # tcp | udp
+bw_udp_bitrate: "1G"        # per flow, udp only
+bw_reverse: false           # measure destination -> source instead
+bw_port_base: 5201          # each flow gets its own port: base + index
+
+# ---------------------------------------------------------------------------
+# Inventory groups
+# ---------------------------------------------------------------------------
+# The role enumerates sources and destinations from these two groups. They are
+# NOT free to rename: site.yml targets its plays with the literal group names
+# [source_hosts] and [dest_hosts], because Ansible resolves play targets before
+# any variable is loaded. Change all three together or not at all.
+bw_source_group: "source_hosts"
+bw_dest_group: "dest_hosts"
+
+# ---------------------------------------------------------------------------
+# Collection
+# ---------------------------------------------------------------------------
+# Each server runs one-shot ("iperf3 -s -1") and writes its own JSON, so the
+# file ON THE SERVER is the measurement. Results are fetched to bw_results_dir.
+bw_remote_dir: "/tmp/bandwidth-test"
+# Absolute on purpose: fetch resolves a relative dest against the playbook
+# directory, but the Summarise play runs a command from wherever you are.
+bw_results_dir: "{{ playbook_dir }}/results"
+
+# Extra seconds to wait past bw_duration before giving up on a server.
+bw_grace: 45
+
+# Install iperf3 if missing. Set false if the image already has it, or if you
+# have no package access on these hosts.
+bw_install: true