Skip to content

strongswan-ipsec: Azure-optimal defaults, automatic VAES kernel, pref… - #103

Merged
chinmaybaikar merged 1 commit into
crusoecloud:mainfrom
c0dyhi11:chill/strongswan-10g-defaults
Sep 3, 2026
Merged

strongswan-ipsec: Azure-optimal defaults, automatic VAES kernel, pref…#103
chinmaybaikar merged 1 commit into
crusoecloud:mainfrom
c0dyhi11:chill/strongswan-10g-defaults

Conversation

@c0dyhi11

@c0dyhi11 c0dyhi11 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

One command now deploys a correctly performing IPsec gateway against a managed cloud peer. The customer edits two files - inventory.ini and group_vars/all.yml (five values) - and site.yml does the rest: preflight, kernel, gateways, clients.

Defaults (were single-tunnel / conservative, now tuned for Azure/GCP/AWS):

  • gcmaes256 IKE+ESP; one tunnel per peer address (vpn_tunnel_count 0 = auto, so listing an Azure active-active gateway's two IPs yields two tunnels)
  • IKE identities derived from ansible_host (the public IP the peer matches on)
  • stateless datapath: conntrack off, no SNAT, so multi-gateway ECMP survives a failover; host tuning on; RFS off (measured: 176x fewer retransmits)
  • clients ECMP across every host in the vpn_gateways group automatically

Kernel play (tasks/kernel.yml): installs a VAES-capable kernel (6.11) where the CPU has vaes and the bound AES-GCM driver does not - measured +23% per tunnel (4.67 -> 5.75 Gbps) and +16..50% on the 1-5 gateway curve. Verifies the initramfs was actually built and the GRUB default (incl. pinned GRUB_DEFAULT / saved_entry) before spending a reboot. Reboots only a host with no VPN on it yet; a live gateway gets the kernel installed and a note (vpn_kernel_reboot_live opts in). Never reboots a host already on the requested kernel, so it cannot loop. Refuses 7.0 by documentation: same crypto, measured 2.4x slower overall.

Preflight play (tasks/preflight.yml) replaces scripts/preflight.sh and gates everything with any_errors_fatal: the five values (checked before any SSH), apt-based OS with reachable mirrors (retried past the unattended-upgrades lock), required kernel modules, uplink MTU >= 1500, every client inside a gateway's vpn_client_cidr, ansible_host is an IPv4 literal, and UDP 500/4500 in BOTH directions between gateways in the run (skipped, with the reason, for a cloud peer, an established SA, or a peer excluded by --limit). Skipped for --tags teardown so a broken deployment stays removable.

Datapath fixes: stale xfrm devices and detached ip rules purged (by pref) when the tunnel count shrinks; health timers forget their memo whenever the route is reinstalled; IP identities never indexed; replay_window validated 0-4096 (32768 fails to install). Probe state lives root-only under /run with O_NOFOLLOW writes; pkill patterns audited so none can match its own shell.

bandwidth-test/: new sibling solution that measures at the RECEIVER, one iperf3 process per flow, so it works through a managed gateway on either end. Fixes from review: summary play sees bw_results_dir, stale servers killed, client JSON cleared (iperf3 --logfile appends), directories 0755.

Docs: DEPLOYING-10G.md (measured 1-5 gateway curve on both kernels, what Azure does NOT do by default and why it costs 3.3x, where the bottleneck sits before and after), README rewritten for the two-file flow with corrected defaults table, root README entries for bandwidth-test and site-to-site-vpn (the latter was failing the structure check before this change).

Tested on Crusoe eu-iceland2-a <-> eu-norway1-a, 20 client pairs per side: 1/2/3/4/5 gateways = 2.37/5.60/9.42/16.16/20.66 Gbps on 6.11 (2.05/4.38/7.38/ 10.78/14.79 on 6.8); one 8 vCPU VM with two tunnels = 8.46 Gbps; kernel play verified to leave a live gateway on 6.8 un-rebooted and to reboot it into 6.11 with vaes when asked; UDP probe verified PASS both ways and skipped correctly under --limit; bandwidth-test verified end to end through the tunnel. Reviewed by 8 independent reviewers + 3 skeptics per finding: 49 confirmed findings fixed, all re-tested. Not validated: a real Azure peer, BGP/FRR against a real peer, the "route" transport.

…light play; add bandwidth-test

One command now deploys a correctly performing IPsec gateway against a managed
cloud peer. The customer edits two files - inventory.ini and group_vars/all.yml
(five values) - and site.yml does the rest: preflight, kernel, gateways, clients.

Defaults (were single-tunnel / conservative, now tuned for Azure/GCP/AWS):
  * gcmaes256 IKE+ESP; one tunnel per peer address (vpn_tunnel_count 0 = auto,
    so listing an Azure active-active gateway's two IPs yields two tunnels)
  * IKE identities derived from ansible_host (the public IP the peer matches on)
  * stateless datapath: conntrack off, no SNAT, so multi-gateway ECMP survives
    a failover; host tuning on; RFS off (measured: 176x fewer retransmits)
  * clients ECMP across every host in the vpn_gateways group automatically

Kernel play (tasks/kernel.yml): installs a VAES-capable kernel (6.11) where the
CPU has vaes and the bound AES-GCM driver does not - measured +23% per tunnel
(4.67 -> 5.75 Gbps) and +16..50% on the 1-5 gateway curve. Verifies the
initramfs was actually built and the GRUB default (incl. pinned GRUB_DEFAULT /
saved_entry) before spending a reboot. Reboots only a host with no VPN on it
yet; a live gateway gets the kernel installed and a note (vpn_kernel_reboot_live
opts in). Never reboots a host already on the requested kernel, so it cannot
loop. Refuses 7.0 by documentation: same crypto, measured 2.4x slower overall.

Preflight play (tasks/preflight.yml) replaces scripts/preflight.sh and gates
everything with any_errors_fatal: the five values (checked before any SSH),
apt-based OS with reachable mirrors (retried past the unattended-upgrades lock),
required kernel modules, uplink MTU >= 1500, every client inside a gateway's
vpn_client_cidr, ansible_host is an IPv4 literal, and UDP 500/4500 in BOTH
directions between gateways in the run (skipped, with the reason, for a cloud
peer, an established SA, or a peer excluded by --limit). Skipped for --tags
teardown so a broken deployment stays removable.

Datapath fixes: stale xfrm devices and detached ip rules purged (by pref) when
the tunnel count shrinks; health timers forget their memo whenever the route is
reinstalled; IP identities never indexed; replay_window validated 0-4096
(32768 fails to install). Probe state lives root-only under /run with
O_NOFOLLOW writes; pkill patterns audited so none can match its own shell.

bandwidth-test/: new sibling solution that measures at the RECEIVER, one iperf3
process per flow, so it works through a managed gateway on either end. Fixes
from review: summary play sees bw_results_dir, stale servers killed, client
JSON cleared (iperf3 --logfile appends), directories 0755.

Docs: DEPLOYING-10G.md (measured 1-5 gateway curve on both kernels, what Azure
does NOT do by default and why it costs 3.3x, where the bottleneck sits before
and after), README rewritten for the two-file flow with corrected defaults
table, root README entries for bandwidth-test and site-to-site-vpn (the latter
was failing the structure check before this change).

Tested on Crusoe eu-iceland2-a <-> eu-norway1-a, 20 client pairs per side:
1/2/3/4/5 gateways = 2.37/5.60/9.42/16.16/20.66 Gbps on 6.11 (2.05/4.38/7.38/
10.78/14.79 on 6.8); one 8 vCPU VM with two tunnels = 8.46 Gbps; kernel play
verified to leave a live gateway on 6.8 un-rebooted and to reboot it into 6.11
with vaes when asked; UDP probe verified PASS both ways and skipped correctly
under --limit; bandwidth-test verified end to end through the tunnel.
Reviewed by 8 independent reviewers + 3 skeptics per finding: 49 confirmed
findings fixed, all re-tested. Not validated: a real Azure peer, BGP/FRR
against a real peer, the "route" transport.
@chinmaybaikar
chinmaybaikar merged commit 3149e92 into crusoecloud:main Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants