Skip to content

feat(wireguard-gateway): Add WireGuard gateway for tenant VMs - #289

Closed
20vikash wants to merge 69 commits into
frappe:developfrom
20vikash:wireguard
Closed

20vikash wants to merge 69 commits into
frappe:developfrom
20vikash:wireguard

Conversation

@20vikash

@20vikash 20vikash commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

WireGuard Gateway Server

Customers reach their own tenant's private fdaa::/16 VMs through a WireGuard gateway. The client holds an fdac address from the gateway's own /48, the gateway forwards it without address translation, and WG Mesh carries the packet to the VM.

Why

  • One public address serves any number of customers; tenant VMs need no public address of their own.
  • The peer list lives inside the gateway: Atlas installs a daemon and hands Central a URL, and never carries peer state itself.

What changed

  • Gateway component (services/wg-gateway/): setup.sh installs wireguard-tools and the eBPF toolchain, assigns the gateway its fdac:<region>:<gateway ID>::1/128 on wg0 with its /48 route, compiles bpf/gateway.c, and attaches the tenant filter to the wg0 and eth0 ingress paths. systemd/ re-applies the interface, addresses, routes, and filters on boot.
  • Client addresses: fdac | region 16 | gateway ID 16 | tenant ID 32 | client ID 32 | zero 16. Each gateway owns one /48.
  • Gateway API daemon (services/wg-gateway/daemon/): serves peers, config, and health on the gateway mesh address at port 80. Owns peers.json; every change rewrites peers.conf and applies it with wg setconf.
  • Tenant isolation (gateway.bpf.o): wg0 ingress admits only this gateway's client sources toward same-region, same-tenant fdaa destinations; eth0 ingress admits only matching replies to fdac. Mismatches drop; the client source is preserved end to end.
  • Atlas (atlas/service/doctype/wireguard_gateway_server/): one wg-gateway-NNN record owns the VM and the daemon credential. Creation needs an enabled Available System image, a reserved tenant-0 IPv4 allocation, a listen port, and an Active Proxy Server, and returns the daemon URL, token audience, public IPv4, listen port, and region ID. Provisioning enables the network gateway role and installs with the numeric record suffix as GATEWAY_ID. Archive removes the proxy route and terminates the VM.
  • Return routes: each Active gateway's /48 is installed, scoped wireguard-gateway, only in VMs with Accessible via WireGuard Gateway enabled. The scope keeps the route in the VM namespace on the host; guest metadata never lists it. The existing server sync converges the set; Edit Routes never touches it.
  • Authentication: Ed25519 JWT for the atlas-wg-gateway:<region> audience, validated against the Atlas JWKS. The key ID prefix selects the issuer, a tenant claim is refused, and the scopes are *, peers:*, peers:read, peers:update, gateway:read. No shared secret; Central reaches the daemon through the <gateway>.<wildcard-domain> proxy route.

nftables-based gateway that SNATs customer fdac addresses into the
tenant-0 mesh. No custom eBPF; conntrack handles the return path.
fdac address layout, peer render and sync over SSH, and the gateway
provisioner. Tenant-wide access only.
Gateway lifecycle with Desk create, archive, and peer resync actions.
Peer records are managed through the gateway API so every change syncs.
Publish the wg-gateway service package, add its Atlas Settings fields,
and retry pending gateway provisioning every minute.
parse_ndp added an unbounded packet length to a packet pointer. The
byteswap wipes the register range, so kernels like 6.8 reject the
object with 'math between pkt pointer and register with unbounded
min value'. Cap the payload at the minimum IPv6 MTU before the
pointer arithmetic. NDP lives on a link and never exceeds it.

Verified with bpftool prog load on 6.8.0-88-generic and by running
the rebuilt gateway on a live metal host.
Atlas boots the guest kernel from outside the root file system, so the
image can miss these modules. Install them for the running kernel and
persist them across reboots.
The gateway uses kernel WireGuard and nftables only. No BPF programs
are attached anywhere in this path.
# Conflicts:
#	atlas/service/SPEC.md
#	services/wg-mesh/bpf/mesh.h
Frappe has no Sidebar doctype and no importer for <app>/<app>/sidebar/, so the
fixture never loaded. The Service sidebar lives as a Workspace Sidebar record
created directly during development.
Frappe 16 has no Sidebar doctype and no importer for <app>/<app>/sidebar/, so
atlas/atlas/sidebar/atlas/atlas.json never loaded. The Service sidebar is a
Workspace Sidebar record instead.
…ge test

get_download_url now refuses a File without a URL, which the test never
provided, so install_package raised before it could take the early return.
The gateway SPEC still pointed at the deleted wg_gateway/address.py and claimed
Atlas owns the peer list, which the daemon took over. The repository, app, and
operator docs never listed the component at all.
@20vikash 20vikash changed the title feat(wireguard-gateway): Add wireguard gateway for customers to connect to their tenant scoped private VMs # feat(wireguard-gateway): Add WireGuard gateway for tenant VMs Sep 25, 2026
@20vikash 20vikash changed the title # feat(wireguard-gateway): Add WireGuard gateway for tenant VMs feat(wireguard-gateway): Add WireGuard gateway for tenant VMs Sep 25, 2026
@20vikash

Copy link
Copy Markdown
Contributor Author

@greptile review

20vikash and others added 12 commits October 1, 2026 12:51
Add the wireguard-gateway route scope. Metal converges a scoped route in
the VM namespace on the host but keeps it out of the guest metadata, so
the routes inside the VM never change. The host sync reports each VM
desired routes.
Give each gateway its own fdac /48 with the gateway ID in the address
layout. Replace the nftables tenant filter and SNAT with tc ingress
programs on wg0 and eth0 that check the tenant without address
translation.
Enable the network gateway role so WG Mesh carries client source
addresses without translation, and pass the gateway ID to the
installer. Send guest IPv6 traffic to the host by default, and describe
the gateway lifecycle and return routes.
Converge the scoped return routes of every Active gateway inside the
existing server sync, using the per-VM routes the host reports. The
sync owns the scoped subset and skips the normal routes, and the Edit
Routes action preserves the scoped subset.
Derive the gateway access flag from the scoped routes Metal holds, and
show the scoped routes in a separate gateway routes field. Toggle access
through an action that installs or removes the gateway routes, like Edit
Routes.
@tanmoysrt
tanmoysrt marked this pull request as draft October 2, 2026 08:54
@tanmoysrt

Copy link
Copy Markdown
Member

There are couple of changes for region lockdown.
I will raise this as seperate pr on the stack - #311 and do necessary changes.

@tanmoysrt

Copy link
Copy Markdown
Member

Moved here : #314

@tanmoysrt tanmoysrt closed this Oct 2, 2026
@tanmoysrt tanmoysrt mentioned this pull request Oct 2, 2026
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants