feat(sponsord): add the sponsord role, standalone or beside a node

.github/workflows/molecule.yml

@@ -27,6 +27,7 @@ jobs:
     runs-on: ubuntu-latest      # Docker is preinstalled
     # ~4m41s at JOBS=3 with 8 scenarios. os-matrix (3 containers) and lifecycle took the
     # suite to ~10m at JOBS=3 on a local 16-thread box; re-measure from CI's first run.
+    # sponsord added a 12th scenario (2 containers, Debian 12 + Ubuntu 24.04).
     # A bound, not a target: without one a wedged privileged systemd container burns
     # the 360-minute default, and with cancel-in-progress above, some branch-protection
     # setups read the resulting cancelled check as "not failed" rather than as a failure.

.github/workflows/upstream-drift.yml

@@ -3,7 +3,8 @@ name: Upstream drift
 
 # Three things in this repo mirror decdn/decdn and go stale when upstream moves:
 #   - roles/decdn_node/vars/main/networks.yml  (contract addresses; upstream
-#     redeploys change every one of them at once)
+#     redeploys change every one of them at once) and its subset
+#     roles/sponsord/vars/main/networks.yml (the same script and check)
 #   - charts/decdn-node/files/monitoring/      (dashboards + alert rules)
 #   - ansible/molecule/schema/files/schema-keys.txt  (the node.toml key inventory)
 # This job regenerates each from upstream main and fails when a committed copy

AGENTS.md

@@ -54,8 +54,8 @@ mirror (`vars/main/networks.yml`), never hand-copied into inventory or docs.
 
 ```
 ansible/                # the deployment project (DevSec-hardened, lean roles)
-  playbooks/            # site.yml (decdn node), backup.yml, decommission.yml
-  roles/                # baseline, decdn_node, grafana_alloy
+  playbooks/            # site.yml (decdn node + sponsord), sponsord.yml, backup.yml, decommission.yml
+  roles/                # baseline, decdn_node, grafana_alloy, sponsord
   inventory/ galaxy/ molecule/    # see ansible/README.md
 cloud-init/             # user-data.yaml + on-host bootstrap.sh; pinned ansible-core/collections (see its README.md)
 compose/                # Docker Compose deploy path for a single host (see its README.md)
@@ -69,7 +69,8 @@ scripts/                # upstream-mirror generators + the release gate
 ```
 
 **Generated mirrors of upstream — regenerate, never hand-edit:**
-`ansible/roles/decdn_node/vars/main/networks.yml` (`scripts/sync-network-profiles.py`),
+`ansible/roles/decdn_node/vars/main/networks.yml` and its subset
+`ansible/roles/sponsord/vars/main/networks.yml` (both `scripts/sync-network-profiles.py`),
 `charts/decdn-node/files/monitoring/` (`scripts/sync-monitoring.sh`) and
 `ansible/molecule/schema/files/schema-keys.txt` (`gen-schema-keys.py`). The weekly
 `upstream-drift` workflow flags staleness. These are the only protocol facts (contract
@@ -125,6 +126,36 @@ addresses) the repo carries, and they carry their upstream commit.
   everything, so only the real pinned binary proves the rendered config loads. See
   `ansible/roles/grafana_alloy/README.md`.
 
+- **`ansible/roles/sponsord`** — the deCDN onboarding sponsor (`decdn/sponsord`): the
+  treasury wallet's signer for capped capabilities plus the PaymentPool keeper. It is
+  independent of the node, so `playbooks/sponsord.yml` (`make deploy-sponsord`, also
+  imported by `site.yml`) targets its own `sponsord_hosts` group, standalone or
+  co-located with a node.
+  - **Install:** manual binary by default, or a GPG-verified `sponsord-v*` release
+    (none cut yet).
+  - **Unit:** `DynamicUser`; the API token (generated on the host, never replaced),
+    treasury keystore and password (operator-provisioned) are `LoadCredential=`
+    credentials, never env. The keystore is re-copied at 0600 into the unit's
+    `RuntimeDirectory` by `ExecStartPre`, because newer systemd (255) writes credentials 0440
+    and upstream rejects a group-readable keystore. Debian 12's systemd 252 hides
+    this (0400); the `molecule/sponsord` Ubuntu 24.04 platform catches it, because
+    the stub enforces upstream's keystore mode check.
+  - **RPC URL:** in `0600 /etc/sponsord/secret.env`, with the same dual-home and
+    provenance guards as `decdn_rpc_url`. A host-provisioned file may carry
+    `SPONSORD_RPC_URL` only (an EnvironmentFile overrides every other setting).
+  - **Listener:** loopback only (asserted), no firewall hole.
+  - **Config gate:** sponsord has no `config validate` and only env/flag config, so
+    `/healthz` after start is the gate, and it is **fatal**. The daemon binds only
+    after the keystore decrypts and the on-chain pool-owner check passes.
+  - **Alloy toggles:** `playbooks/group_vars/all.yml` derives
+    `grafana_alloy_node_enabled` / `grafana_alloy_sponsord_enabled` from group
+    membership. They are host-scoped so a co-located host's two plays render one
+    config.
+  - **Molecule:** under docker, `/` must be `--make-rshared` (see
+    `molecule/sponsord/prepare.yml`) or every credential directory is empty.
+  - **Not in cloud-init:** its lint only allows `decdn_nodes`.
+  - The public `sponsord-onramp` is not deployed yet.
+
 - **`cloud-init/`** — the Ansible path with no control machine. `user-data.yaml` carries
   only public material (the lint refuses secret-looking keys, credentials in URLs and
   unknown `bootstrap.env` keys) and a stage-1 `decdn-bootstrap`. That script clones this
@@ -190,18 +221,20 @@ make security         # KICS over ansible/, the rendered chart and compose/
 # Ansible — run from ansible/
 make deps                          # vendor pinned Galaxy collections
 make check / deploy                # site.yml; fleet-wide unless LIMIT=<host>; INVENTORY=<overlay>
+make check-sponsord / deploy-sponsord   # sponsord.yml only (sponsord_hosts)
 make backup / decommission LIMIT=… # lifecycle playbooks (decommission requires LIMIT)
 make build / galaxy-check          # the decdn.node collection
 ```
 
 **Inventory is private; the firewall hole is not.** This repo is public, so
 `ansible/inventory/hosts.yml` is git-ignored and a real fleet lives in a private overlay.
-Inventory-adjacent group_vars do not load for an overlay, so anything every node needs regardless of inventory (today only the udp/4433
-`baseline_extra_inbound` hole) lives in `ansible/playbooks/group_vars/decdn_nodes.yml`.
+Inventory-adjacent group_vars do not load for an overlay, so anything every node needs regardless of inventory (today the udp/4433
+`baseline_extra_inbound` hole) lives in `ansible/playbooks/group_vars/decdn_nodes.yml`, and
+the Alloy per-daemon toggles every host needs in `ansible/playbooks/group_vars/all.yml`.
 Don't move it back under `inventory/`.
 
-**Galaxy collection (`decdn.node`).** The three roles (`baseline` + `decdn_node` +
-`grafana_alloy`) ship as a
+**Galaxy collection (`decdn.node`).** The four roles (`baseline` + `decdn_node` +
+`grafana_alloy` + `sponsord`) ship as a
 distributable collection. The overlay lives in `ansible/galaxy/` and is staged into a clean
 collection tree by `galaxy/build.sh` — there is **no** `galaxy.yml` at the `ansible/` root
 (that would make ansible-lint treat the deploy project as a collection). Build/validate with

README.md

@@ -97,7 +97,7 @@ This is the canonical statement; the per-path READMEs add only what is specific
 
 | Path | What it is |
 |------|------------|
-| [`ansible/`](ansible/README.md) | The Ansible project: `inventory/`, `playbooks/` (`site.yml`, `backup.yml`, `decommission.yml`), `roles/` (`baseline`, `decdn_node`, `grafana_alloy`), `galaxy/` (the `decdn.node` collection), `molecule/`. |
+| [`ansible/`](ansible/README.md) | The Ansible project: `inventory/`, `playbooks/` (`site.yml`, `sponsord.yml`, `backup.yml`, `decommission.yml`), `roles/` (`baseline`, `decdn_node`, `grafana_alloy`, `sponsord`), `galaxy/` (the `decdn.node` collection), `molecule/`. |
 | [`cloud-init/`](cloud-init/README.md) | The cloud-init deploy path: `user-data.yaml`, the on-host `bootstrap.sh`, and the pinned ansible-core and collections it installs. |
 | [`compose/`](compose/README.md) | The Docker Compose deploy path. |
 | [`charts/decdn-node/`](charts/decdn-node/README.md) | The Helm chart, with vendored dashboards and alert rules in `files/monitoring/`. |

ansible/Makefile

@@ -1,6 +1,6 @@
 # Convenience targets for the deCDN Ansible project.
 # Always run from the ansible/ directory.
-.PHONY: deps lint check deploy backup decommission molecule molecule-serial molecule-fanout molecule-all build galaxy-check
+.PHONY: deps lint check deploy check-sponsord deploy-sponsord backup decommission molecule molecule-serial molecule-fanout molecule-all build galaxy-check
 SHELL := /bin/bash
 
 # Install the required Galaxy collections (>= constraints in requirements.yml)
@@ -43,20 +43,20 @@ INVENTORY ?= inventory/hosts.yml
 # A missing or unparseable inventory fails the run. Ansible's default only warns,
 # skips the `hosts: decdn_nodes` play and exits 0, so a mistyped INVENTORY= would
 # report a successful deploy that did nothing. This applies to the playbook targets
-# (check/deploy/backup/decommission) only:
+# (check/deploy/check-sponsord/deploy-sponsord/backup/decommission) only:
 # ansible.cfg can't set it globally, because ansible-lint's own syntax checks run
 # with no inventory present.
 strict_inventory = ANSIBLE_INVENTORY_UNPARSED_FAILED=True
 
 # Fail loud on the two ways a run can look scoped but silently be fleet-wide. Scoped to
-# the playbook targets (check/deploy/backup/decommission) via MAKECMDGOALS so an
+# the playbook targets (check/deploy[-sponsord]/backup/decommission) via MAKECMDGOALS so an
 # unrelated `make lint` is never blocked. NOTE: this
 # does not catch an argument that is well-formed but selects nothing — `--tags` with a
 # typo (decdn-node vs decdn_node) runs zero tasks and still exits 0. See ansible/README.md.
-ifneq ($(filter check deploy backup decommission,$(MAKECMDGOALS)),)
+ifneq ($(filter check deploy check-sponsord deploy-sponsord backup decommission,$(MAKECMDGOALS)),)
   # Echo back the goal the operator actually typed, so the hint never nudges a
   # `make check` dry-run toward the mutating `make deploy`.
-  scoped_goal := $(firstword $(filter check deploy backup decommission,$(MAKECMDGOALS)))
+  scoped_goal := $(firstword $(filter check deploy check-sponsord deploy-sponsord backup decommission,$(MAKECMDGOALS)))
   ifeq ($(origin LIMIT),environment)
     $(error LIMIT is set in your environment ($(LIMIT)) and would silently scope this run. Unset it or pass it on the command line: make $(scoped_goal) LIMIT=<host>)
   endif
@@ -92,6 +92,14 @@ check:
 deploy:
 	$(strict_inventory) ansible-playbook -i '$(INVENTORY)' playbooks/site.yml $(limit_arg) $(ANSIBLE_ARGS)
 
+# sponsord alone (playbooks/sponsord.yml, hosts in sponsord_hosts). check/deploy
+# above already include it via site.yml; these leave every decdn-node untouched.
+check-sponsord:
+	$(strict_inventory) ansible-playbook -i '$(INVENTORY)' playbooks/sponsord.yml --check --diff $(limit_arg) $(ANSIBLE_ARGS)
+
+deploy-sponsord:
+	$(strict_inventory) ansible-playbook -i '$(INVENTORY)' playbooks/sponsord.yml $(limit_arg) $(ANSIBLE_ARGS)
+
 # Encrypted backup of each node (identity by default; -e decdn_backup_scope=full for
 # the whole data dir, which stops the node for the copy). One host at a time.
 # Needs decdn_backup_age_recipients. See docs/lifecycle.md.
@@ -104,16 +112,17 @@ decommission:
 	$(strict_inventory) ansible-playbook -i '$(INVENTORY)' playbooks/decommission.yml $(limit_arg) $(ANSIBLE_ARGS)
 
 # --- Tests -------------------------------------------------------------------
-# Containerised converge + idempotence + verify of the decdn_node role against a
-# stub daemon (needs Docker; a privileged systemd container). See molecule/.
+# Containerised converge + idempotence + verify of the roles against stub daemons
+# (needs Docker; privileged systemd containers). See molecule/.
 # Scenarios: default (rendered values + idempotence), schema (config key-set drift
-# against the upstream field list), validation (bad knobs, both roles, must be
+# against the upstream field list), validation (bad knobs, every role, must be
 # rejected by their own asserts), generate-keystore (opt-in host-side wallet),
 # host-env (host-provisioned /etc/decdn/decdn.env), slow-readiness (advisory
 # /metrics probe timeout), grafana-cloud and grafana-cloud-token (opt-in Grafana
 # Cloud observability, host-provisioned vs inventory token), os-matrix (default's
-# plays on Debian 13, Ubuntu 24.04 and 26.04) and lifecycle (network profile,
-# backup + decommission). The glob below is the authority; this list is a guide.
+# plays on Debian 13, Ubuntu 24.04 and 26.04), lifecycle (network profile,
+# backup + decommission) and sponsord (the sponsord role on a host with no node,
+# through playbooks/sponsord.yml, on Debian 12 and Ubuntu 24.04). The glob below is the authority; this list is a guide.
 #
 # The scenarios run concurrently: distinct container names, no published host ports,
 # per-scenario ephemeral dirs, and what they share on the control machine (the
@@ -200,8 +209,8 @@ molecule-all: deps
 	molecule test --all
 
 # --- Galaxy collection (decdn.node) ------------------------------------------
-# Stage baseline + decdn_node + grafana_alloy into a clean collection tree and build the
-# artifact under build/. Only the three roles ship; see galaxy/README.md. Publishing
+# Stage baseline + decdn_node + grafana_alloy + sponsord into a clean collection tree and
+# build the artifact under build/. Only the four roles ship; see galaxy/README.md. Publishing
 # runs from a vX.Y.Z tag in .github/workflows/release.yml (see RELEASING.md).
 build:
 	./galaxy/build.sh

ansible/README.md

@@ -8,13 +8,16 @@ hardened host baseline. The roles also ship as the `decdn.node` Galaxy collectio
 | **`site.yml`** | Harden the host and deploy the public **deCDN node** (`decdn-node`). | `make check` / `make deploy` |
 | `backup.yml` | Encrypted backup of a node's identity or full state. | `make backup` |
 | `decommission.yml` | Stop a node and remove its service (keeps the identity; no on-chain steps). | `make decommission` |
+| `sponsord.yml` | Harden the host and deploy **sponsord**, the onboarding sponsor, on hosts in `sponsord_hosts`, with or without a node. Also run by `site.yml`. | `make check-sponsord` / `make deploy-sponsord` |
 
 ```
 baseline        host hardening: DevSec os/ssh, nftables default-deny inbound,
    │            fail2ban, unattended-upgrades, chrony, an admin sudo user
    ├─ grafana_alloy   opt-in Grafana Cloud agent (metrics, journald, traces), loopback-only
-   └─ decdn_node      public QUIC udp/4433; metrics + admin loopback; signed-tarball or
-                      local-build install; hardened systemd unit; backup/decommission
+   ├─ decdn_node      public QUIC udp/4433; metrics + admin loopback; signed-tarball or
+   │                  local-build install; hardened systemd unit; backup/decommission
+   └─ sponsord        treasury signer + PaymentPool keeper; loopback API; DynamicUser
+                      unit with systemd credentials; standalone or beside a node
 ```
 
 ## Security model
@@ -245,6 +248,25 @@ guardrails and rollback: [`roles/grafana_alloy/README.md`](roles/grafana_alloy/R
 To import upstream's decdn dashboards and alert rules into that stack, see
 [`charts/decdn-node/files/monitoring/`](../charts/decdn-node/files/monitoring/README.md).
 
+### sponsord (optional)
+
+The fourth role, `sponsord`, deploys the deCDN onboarding sponsor. It holds the
+treasury wallet that owns a PaymentPool pool, signs capabilities for its onramp and
+keeps the pool topped up. It needs no node. Put its host in the `sponsord_hosts` group:
+a dedicated host or a node host listed in both groups. Then run
+`make deploy-sponsord`; `make deploy` covers it too.
+
+- **You provision** the treasury keystore and password on the host. The role never
+  creates that wallet.
+- **The API token** is generated on the host.
+- **The RPC URL** is dual-homed like `decdn_rpc_url`.
+- **Observability:** with Grafana Cloud on, Alloy scrapes sponsord on hosts in
+  `sponsord_hosts` and the node on hosts in `decdn_nodes`
+  (`playbooks/group_vars/all.yml`).
+
+Setup and variables: [`roles/sponsord/README.md`](roles/sponsord/README.md). The public
+`sponsord-onramp` is not deployed by this repo yet.
+
 ### Backup and decommission
 
 ```bash
@@ -273,23 +295,26 @@ make lint-alloy     # render grafana_alloy's templates, then `alloy validate` th
 
 `make molecule` runs every scenario under `molecule/`: **`default`** (described below),
 `schema` (config key-set drift against the upstream field list), `validation` (bad knobs,
-for both roles, must be rejected by their own asserts), `generate-keystore` (opt-in
+for every role, must be rejected by their own asserts), `generate-keystore` (opt-in
 host-side wallet), `host-env` (host-provisioned `/etc/decdn/decdn.env`),
 `slow-readiness` (advisory `/metrics` probe timeout), `grafana-cloud` (the opt-in
 observability wiring — see [Grafana Cloud observability](#grafana-cloud-observability-opt-in))
 `grafana-cloud-token` (the same wiring with the API token carried through
 git-ignored inventory instead: role-authored env file + provenance record),
 `os-matrix` (default's plays on Debian 13, Ubuntu 24.04 and Ubuntu 26.04; `default`
-itself is Debian 12) and `lifecycle` (a `decdn_network` profile with an override, both
-backup scopes decrypted and checked, a rejected and a real decommission).
+itself is Debian 12), `lifecycle` (a `decdn_network` profile with an override, both
+backup scopes decrypted and checked, a rejected and a real decommission) and
+`sponsord` (`playbooks/sponsord.yml` on a host with no node: generated token, credential
+rotation restart, the fatal `/healthz` gate, Alloy scraping sponsord and not the node;
+`grafana-cloud` covers sponsord co-located with a node).
 They are independent, so they run concurrently, and each line of output is prefixed with its scenario name
 because the runs interleave. `make molecule-serial` is the escape hatch when that
 interleaving gets in the way of reading a failure.
 
 `grafana-cloud` runs against a *stub* Alloy that exits 0 for every subcommand, so it
 proves the role's plumbing but cannot prove the rendered `config.alloy` is loadable.
 That gap is closed by `make lint-alloy` (repo root; CI job `alloy-config`), which renders
-the templates in eight variable combinations — including one with every observability
+the templates in twelve variable combinations — including one with every observability
 sub-knob off, which must reproduce the pre-machine-monitoring pipeline exactly — and runs
 the **real**, digest-pinned Alloy
 binary's `alloy validate` over them plus a check that every `ExecStart` flag actually
@@ -335,12 +360,12 @@ the RPC URL when it is not provisioned on the host instead). Highlights:
 
 ## Packaging as a Galaxy collection (`decdn.node`)
 
-The three roles (`baseline`, `decdn_node`, `grafana_alloy`) are also packaged as the
+The four roles (`baseline`, `decdn_node`, `grafana_alloy`, `sponsord`) are also packaged as the
 distributable **`decdn.node`** collection, for operators who bring their own playbooks.
 
 The collection overlay lives in [`galaxy/`](galaxy/) (`galaxy.yml`, the collection
 `README.md`/`CHANGELOG.md`, `meta/runtime.yml`, `build.sh`). It is deliberately **not**
-a `galaxy.yml` at the project root: `galaxy/build.sh` stages only the three roles into a
+a `galaxy.yml` at the project root: `galaxy/build.sh` stages only the four roles into a
 clean `ansible_collections/decdn/node/` tree and builds the artifact, so this project
 stays a plain Ansible project (the `make deploy`/`lint` flow is unchanged).