diff --git a/.gitignore b/.gitignore index b5a996b..68b932e 100644 --- a/.gitignore +++ b/.gitignore @@ -12,6 +12,10 @@ !**/.env.example # An operator's local Compose override (compose/README.md); decdn-compose includes it compose/compose.override.yaml +# A hosts.yml filled in inside this checkout: real host IPs never belong in this public +# repo. The fleet example's own .gitignore must not ignore it (a private copy commits +# it), and this file does not travel with `cp -R examples/collection-fleet`. +/examples/collection-fleet/inventory/hosts.y*ml # Private keys & certs (defense-in-depth; real keystores live under /etc on the host) **/cert.pem diff --git a/AGENTS.md b/AGENTS.md index 0219c43..ad2f87c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -120,7 +120,7 @@ monitoring/ # Grafana dashboards + Prometheus alert rules (maintaine sponsord/ # sponsord's (+ promtool unit tests), Ansible/Compose only iroh-relay/ # the relay's (+ promtool unit tests, exported-metrics.txt), Ansible and Compose docs/ # cross-path operator docs: node-operators.md + publishers.md (the front doors), requirements.md, lifecycle.md -examples/collection-fleet/ # a consumer project for the Galaxy collections: requirements.yml, ansible.cfg, .gitignore (the inventory comes from ansible/inventory/'s examples) +examples/collection-fleet/ # a consumer project for the Galaxy collections: requirements.yml, ansible.cfg, .gitignore, inventory/ (ansible/inventory/'s templates: every tracked file identical but the hosts-*.yml.example headers, which end at the first blank line and hold only comments; make test-scripts checks, and that both parse to the same inventory, so change both. The root .gitignore keeps a hosts.yml filled in there out of this public repo) scripts/ # upstream-mirror generators, the compose Alloy config generator (render-compose-alloy.sh), the decdn bump (open-decdn-bump.sh, bump_decdn.py), the release script (release.sh, git-cliff: ../cliff.toml) and its gate (+ its Artifact Hub changes generator), the molecule driver (molecule.sh), ansible/Makefile's LIMIT preflight (limit-guard.sh) ``` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index edea5cc..3d00d42 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -42,7 +42,7 @@ targets, so a local pass means a CI pass. Deploy targets live in | `make lint-alloy` | renders `roles/grafana_alloy`'s templates and validates them with the **real** digest-pinned Alloy binary. The molecule stub exits 0 for everything, so this is the only gate that proves the config loads. It also checks that `compose/alloy/config.alloy` (the Compose `alloy` profile's render of the same template) is current, and runs it in Alloy: every component healthy, each daemon scraped only under the profiles that start it. `ALLOY_BIN=` skips the download. | | `make lint-compose` | reads `compose/compose.yaml` as written (`yq`) against `compose/tests/bind-sources.jq` (every bind mount's `create_host_path`), then renders it with every profile on (with its example env, without env files, with an empty `.env`) and asserts `compose/tests/invariants.jq`, `inline-env.jq` and `fail-closed.jq` | | `make lint-cloud-init` | `cloud-init schema` on `cloud-init/user-data-node.yaml` and `cloud-init/user-data-publisher.yaml`, then `cloud-init/tests/lint.py` on each: only the templates' top-level modules and no YAML anchors, no secrets (only the bootstrap's own files, once each, plain `content` only, no secret-looking keys or assignments), no hardening skip, `release` installs verified against the vendored keys with a host-generated node wallet (trust knobs only in their own group's `vars`, no moved keyring or secret path), only localhost, in `decdn_nodes` and/or `sponsord_hosts` (an origin only beside `decdn_nodes` and with a backend, every backend key only in `decdn_origin_nodes.vars`, the onramp only beside sponsord), a keyed admin account, `runcmd` exactly stage 1, shellcheck-clean scripts, and a collection lock that covers `ansible/requirements.yml` (needs `cloud-init`, `shellcheck`, `yq`). `CLOUD_INIT_FILE=` checks your own filled-in copy. | -| `make test-scripts` | `tests/scripts-test.sh`: the `ansible/Makefile` scoping guards (dry runs, and `limit-guard.sh` on a fixture inventory), the molecule driver's selection guards and locks (no containers), the split scenarios' shared inventories, the release gate, `scripts/release.sh` on a fixture repo and `pr-title.yml`'s check (the former skipped without `git-cliff` or `ssh-keygen` on PATH, failed in CI), the negative cases of `lint-compose` and `lint-cloud-init` (the latter skipped without `cloud-init` on PATH), `compose/decdn-compose`'s unit tests (`compose/tests/test_decdn_compose.py`), `scripts/bump_decdn.py`'s (`scripts/tests/test_bump_decdn.py`), that `compose/alloy/config.alloy` is a fresh render of the `grafana_alloy` template (`scripts/render-compose-alloy.sh --check`; needs ansible-core, failed in CI without it) and that Compose's `grafana/alloy` image is `grafana_alloy_version`, and the cloud-init bootstrap's contracts: `baseline-plays.sh` on fixtures and on the real playbooks (every node.yml, origin.yml and sponsord.yml play runs `baseline` tagged `baseline`), the two templates' shared stage 1, login hint and `final_message`, `bootstrap.sh`'s groups against lint.py's, and lint.py's `FORBIDDEN_VARS` against the roles' keyrings and secret paths; and the Galaxy collections' shipped playbooks: every playbook in a `playbooks.txt`, `galaxy/stage-playbooks.py` on both collections and its refusals, that it changes only role names, `decdn.node`'s `backup.yml`/`decommission.yml` against the fleet-wide node plays in both directions (needs yq, failed in CI without it), and the two roles' shared `*_backup_local_dir` default. `UPSTREAM=` adds the sync generators' exit codes. | +| `make test-scripts` | `tests/scripts-test.sh`: the `ansible/Makefile` scoping guards (dry runs, and `limit-guard.sh` on a fixture inventory), the molecule driver's selection guards and locks (no containers), the split scenarios' shared inventories, the release gate, `scripts/release.sh` on a fixture repo and `pr-title.yml`'s check (the former skipped without `git-cliff` or `ssh-keygen` on PATH, failed in CI), the negative cases of `lint-compose` and `lint-cloud-init` (the latter skipped without `cloud-init` on PATH), `compose/decdn-compose`'s unit tests (`compose/tests/test_decdn_compose.py`), `scripts/bump_decdn.py`'s (`scripts/tests/test_bump_decdn.py`), that `compose/alloy/config.alloy` is a fresh render of the `grafana_alloy` template (`scripts/render-compose-alloy.sh --check`; needs ansible-core, failed in CI without it) and that Compose's `grafana/alloy` image is `grafana_alloy_version`, and the cloud-init bootstrap's contracts: `baseline-plays.sh` on fixtures and on the real playbooks (every node.yml, origin.yml and sponsord.yml play runs `baseline` tagged `baseline`), the two templates' shared stage 1, login hint and `final_message`, `bootstrap.sh`'s groups against lint.py's, and lint.py's `FORBIDDEN_VARS` against the roles' keyrings and secret paths; and the Galaxy collections' shipped playbooks: every playbook in a `playbooks.txt`, `galaxy/stage-playbooks.py` on both collections and its refusals, that it changes only role names, `decdn.node`'s `backup.yml`/`decommission.yml` against the fleet-wide node plays in both directions (needs yq, failed in CI without it), and the two roles' shared `*_backup_local_dir` default; and `examples/collection-fleet/inventory/` against `ansible/inventory/` (the same tracked files, byte for byte but for the hosts examples' comment headers; both hosts examples parse to the same inventory, needs ansible-core, failed in CI without it), and the ignore rules that keep a `secret.*` or a filled-in `hosts.yml` there out of git, with negative cases on a fixture. `UPSTREAM=` adds the sync generators' exit codes. | | `make security` | KICS IaC scan of `ansible/`, the rendered chart and `compose/` (digest-pinned engine, fail on HIGH) | | `make galaxy-check` | build the `decdn.node` and `decdn.publisher` collections, run galaxy-importer's checks on each, then install each and check its playbooks by FQCN, firewall holes included (`ansible/galaxy/check-playbooks.sh`; `make -C ansible galaxy-check-node` / `galaxy-check-publisher` for one) | diff --git a/README.md b/README.md index a299c78..12b2752 100644 --- a/README.md +++ b/README.md @@ -93,15 +93,14 @@ one VM); on Kubernetes, the [Helm chart](charts/decdn-node/README.md). From your workstation (ansible-core ≥ 2.15, e.g. `pipx install ansible-core`) with an SSH key in `~/.ssh/`, and a fresh Debian or Ubuntu host. The collections bring the roles -and playbooks; you clone this repository once, only to copy the example project and -inventory templates into a private repository of your own: +and playbooks; you clone this repository once, only to copy the example project (its +inventory templates included) into a private repository of your own: ```bash git clone --depth 1 https://github.com/decdn/devops.git cp -R devops/examples/collection-fleet my-fleet && cd my-fleet # make this your private repo ansible-galaxy collection install -r requirements.yml -p collections -mkdir inventory && cp ../devops/ansible/inventory/hosts-node.yml.example inventory/hosts.yml -cp -R ../devops/ansible/inventory/{group_vars,host_vars} inventory/ +cp inventory/hosts-node.yml.example inventory/hosts.yml cp inventory/host_vars/decdn-node-1/secret.yml.example inventory/host_vars/decdn-node-1/secret.yml $EDITOR inventory/hosts.yml # ansible_host: the node's IP $EDITOR inventory/host_vars/decdn-node-1/main.yml # decdn_region; add decdn_node_generate_keystore: true (the node makes its wallet) diff --git a/ansible/README.md b/ansible/README.md index ff6a85f..0979bdf 100644 --- a/ansible/README.md +++ b/ansible/README.md @@ -132,7 +132,9 @@ Ansible loads `group_vars/` and `host_vars/` from beside whichever `hosts.yml` y the overlay carries its own: create them next to its `hosts.yml`, starting from the public `inventory/group_vars/` and `inventory/host_vars/` templates. Git-ignore `host_vars/*/secret.*` in the private repo, so it never tracks a per-node secret. The public -`inventory/group_vars/` and `inventory/host_vars/` are **not** loaded for it. Add child groups +`inventory/group_vars/` and `inventory/host_vars/` are **not** loaded for it. (Deploying from +the Galaxy collections instead? [`examples/collection-fleet/`](../examples/collection-fleet/README.md) +is a ready-made private repository for that path, inventory templates and `.gitignore` included.) Add child groups of your own when hosts differ by group (disk size, an egress cap on metered bandwidth), each with a `group_vars/.yml`. Origins go in `decdn_origin_nodes`, as a child of `decdn_nodes` (see `inventory/hosts-publisher.yml.example`). Settings every host needs regardless of diff --git a/ansible/galaxy/node/README.md b/ansible/galaxy/node/README.md index b7bed00..875f357 100644 --- a/ansible/galaxy/node/README.md +++ b/ansible/galaxy/node/README.md @@ -76,8 +76,10 @@ ansible-playbook -i inventory/hosts.yml decdn.node.decommission -l ``` Start the inventory from -[`hosts-node.yml.example`](https://github.com/decdn/devops/blob/main/ansible/inventory/hosts-node.yml.example) -and the `group_vars/` and `host_vars/` beside it. The playbooks open the node's QUIC +[`hosts-node.yml.example`](https://github.com/decdn/devops/blob/main/examples/collection-fleet/inventory/hosts-node.yml.example) +and the `group_vars/` and `host_vars/` beside it, in +[`examples/collection-fleet/`](https://github.com/decdn/devops/tree/main/examples/collection-fleet), +a ready-made project for these collections. The playbooks open the node's QUIC port in the baseline firewall themselves (`baseline_extra_inbound`, from their own `group_vars/`, which ship beside them), and `decommission` refuses more hosts than `decdn_decommission_max_hosts` (default 1). diff --git a/ansible/galaxy/publisher/README.md b/ansible/galaxy/publisher/README.md index 69f7e25..c0a648b 100644 --- a/ansible/galaxy/publisher/README.md +++ b/ansible/galaxy/publisher/README.md @@ -88,8 +88,10 @@ ansible-playbook -i inventory/hosts.yml decdn.publisher.decommission -l ``` Start the inventory from -[`hosts-publisher.yml.example`](https://github.com/decdn/devops/blob/main/ansible/inventory/hosts-publisher.yml.example) -and the `group_vars/` and `host_vars/` beside it. Beyond the roles, the playbooks +[`hosts-publisher.yml.example`](https://github.com/decdn/devops/blob/main/examples/collection-fleet/inventory/hosts-publisher.yml.example) +and the `group_vars/` and `host_vars/` beside it, in +[`examples/collection-fleet/`](https://github.com/decdn/devops/tree/main/examples/collection-fleet), +a ready-made project for these collections. Beyond the roles, the playbooks bring: - **the firewall holes**, from the groups a host is in: the node's udp/4433, Caddy's diff --git a/ansible/inventory/group_vars/decdn_nodes.yml b/ansible/inventory/group_vars/decdn_nodes.yml index 2cc0cb5..c77132b 100644 --- a/ansible/inventory/group_vars/decdn_nodes.yml +++ b/ansible/inventory/group_vars/decdn_nodes.yml @@ -6,9 +6,9 @@ # The sensitive rpc_url is provisioned as /etc/decdn/decdn.env on the host, or (if # you would rather the role authored that file) in a git-ignored secret.yml. -# The node's public QUIC firewall hole (udp/4433) is NOT set here. It lives in -# playbooks/group_vars/ (all.yml builds it) so it applies to every inventory, -# including a private fleet overlay passed with INVENTORY=. +# The node's public QUIC firewall hole (udp/4433) is NOT set here. The playbooks set +# it from their own group_vars/ (all.yml builds it), which ship beside them in +# decdn/devops and in its Galaxy collections, so it applies to any inventory. # # Settings every node shares (decdn_network, decdn_node_install_method, …) can move # here from host_vars//main.yml once you run more than one node. host_vars @@ -25,9 +25,10 @@ # --- Cache sizing ----------------------------------------------------------------- # Size to each host's disk and to the largest object you expect to serve; see -# docs/requirements.md for the disk formula. A node rejects any blob larger than -# decdn_max_blob_size_mb with BlobTooLarge, and decdn_cache_size_mb must be at -# least that. Override per host in host_vars when disks differ. +# https://github.com/decdn/devops/blob/main/docs/requirements.md for the disk formula. +# A node rejects any blob larger than decdn_max_blob_size_mb with BlobTooLarge, and +# decdn_cache_size_mb must be at least that. Override per host in host_vars when +# disks differ. # decdn_max_blob_size_mb: 1024 # role default # decdn_cache_size_mb: 10240 # role default (10 GiB) diff --git a/ansible/inventory/host_vars/decdn-node-1/main.yml b/ansible/inventory/host_vars/decdn-node-1/main.yml index 1d2350e..fa4838e 100644 --- a/ansible/inventory/host_vars/decdn-node-1/main.yml +++ b/ansible/inventory/host_vars/decdn-node-1/main.yml @@ -9,7 +9,7 @@ # # CONTRACT ADDRESSES ARE NOT INVENTED HERE, and not copied here either: # decdn_network below selects the role's mirror of upstream's deployment manifest -# (roles/decdn_node/vars/main/networks.yml, regenerated by +# (roles/decdn_node/vars/main/networks.yml, regenerated by decdn/devops's # scripts/sync-network-profiles.py from contracts/deployments/.json in # decdn/decdn). # It is the same manifest `decdn config init --chain arbitrum-sepolia` bakes in, and diff --git a/ansible/inventory/host_vars/origin-1/main.yml b/ansible/inventory/host_vars/origin-1/main.yml index 54b7373..05bbbf5 100644 --- a/ansible/inventory/host_vars/origin-1/main.yml +++ b/ansible/inventory/host_vars/origin-1/main.yml @@ -11,7 +11,7 @@ # # CONTRACT ADDRESSES ARE NOT INVENTED HERE, and not copied here either: # decdn_network below selects the role's mirror of upstream's deployment manifest -# (roles/decdn_node/vars/main/networks.yml, regenerated by +# (roles/decdn_node/vars/main/networks.yml, regenerated by decdn/devops's # scripts/sync-network-profiles.py from contracts/deployments/.json in # decdn/decdn). # It is the same manifest `decdn config init --chain arbitrum-sepolia` bakes in, and diff --git a/ansible/inventory/hosts-node.yml.example b/ansible/inventory/hosts-node.yml.example index 52947f8..a9d7715 100644 --- a/ansible/inventory/hosts-node.yml.example +++ b/ansible/inventory/hosts-node.yml.example @@ -30,13 +30,13 @@ # affected host: `sudo --version` reports "sudo-rs" and/or /usr/bin/sudo.ws exists. Leave # it commented on Debian bookworm / Ubuntu noble — that path does not exist there. -# Public deCDN cache nodes — provisioned by playbooks/node.yml (`make deploy-node`) -# and by site.yml (`make deploy`). A cache node has no origin backend: it fills a -# miss from other nodes (node-to-node pull-through). Origins are a publisher's -# (hosts-publisher.yml.example). +# Public deCDN cache nodes — provisioned by the node.yml playbook (which site.yml +# imports too). A cache node has no origin backend: it fills a miss from other nodes +# (node-to-node pull-through). Origins are a publisher's (hosts-publisher.yml.example). # Per-node config (contract addresses, region, version, …) goes in the committed -# inventory/host_vars//main.yml; the one secret, rpc_url, goes in a git-ignored -# secret.yml beside it (copy the shipped secret.yml.example there). +# inventory/host_vars//main.yml; secrets (rpc_url, and any other +# secret.yml.example lists) go in a git-ignored secret.yml beside it (copy the shipped +# secret.yml.example there). decdn_nodes: hosts: decdn-node-1: diff --git a/ansible/inventory/hosts-publisher.yml.example b/ansible/inventory/hosts-publisher.yml.example index 453d6ba..7258c81 100644 --- a/ansible/inventory/hosts-publisher.yml.example +++ b/ansible/inventory/hosts-publisher.yml.example @@ -18,16 +18,16 @@ # reads every node's ID over SSH, and under -u root it would try the nodes, whose # root login baseline has already turned off, as root. -# Origin nodes — provisioned by playbooks/origin.yml (`make deploy-origin`) and by -# publisher.yml. An origin is a deCDN node with an origin backend: the canonical +# Origin nodes — provisioned by the origin.yml playbook and by publisher.yml (which +# site.yml imports). An origin is a deCDN node with an origin backend: the canonical # source of its namespace's content. decdn_origin_nodes is a child of decdn_nodes, so # an origin gets a node's firewall hole, backup, decommission and observability; # origin.yml refuses one outside decdn_nodes or without a backend. The chain # recognises it as the origin only once you, the namespace's publisher, seat its # operator with OriginAssignment.addOrigin; until then it serves its bytes as a cache # does. The backend goes in group_vars/decdn_origin_nodes.yml (shared by every origin) -# or host_vars//main.yml (see host_vars/origin-1/), with the region and the -# rest of the per-origin config. +# or host_vars//main.yml (see host_vars/origin-1/), with the region and the rest +# of the per-origin config. decdn_nodes: children: decdn_origin_nodes: @@ -40,11 +40,11 @@ decdn_nodes: # ansible_host: REPLACE_WITH_ORIGIN_VPS_IP_OR_DNS # OPTIONAL: sponsord, the onboarding sponsor (treasury signer + PaymentPool keeper), -# which pays for your users' downloads — provisioned by playbooks/sponsord.yml (`make -# deploy-sponsord`) and by publisher.yml. It needs no node: give it its own host, or -# list an origin host here as well to co-locate them. Its knobs are in -# group_vars/sponsord_hosts.yml; the treasury keystore and password are provisioned on -# the host (roles/sponsord/README.md). +# which pays for your users' downloads — provisioned by the sponsord.yml playbook and +# by publisher.yml. It needs no node: give it its own host, or list an origin host +# here as well to co-locate them. Its knobs are in group_vars/sponsord_hosts.yml; the +# treasury keystore and password are provisioned on the host (the sponsord role's +# README). # sponsord_hosts: # hosts: # sponsord-1: @@ -54,20 +54,20 @@ decdn_nodes: # capability API), behind Caddy on tcp/80 + tcp/443. List a sponsord host # again here; it must be in sponsord_hosts too. Its knobs are in # group_vars/sponsord_onramp_hosts.yml; the Turnstile secret is provisioned on the -# host (roles/sponsord_onramp/README.md). +# host (the sponsord_onramp role's README). # sponsord_onramp_hosts: # hosts: # sponsord-1: # # OPTIONAL: self-hosted iroh relays, the fallback path for peers that cannot -# hole-punch — provisioned by playbooks/iroh_relay.yml (`make deploy-relay`) and by -# publisher.yml. Public by design: each terminates its own Let's Encrypt TLS on +# hole-punch — provisioned by the iroh_relay.yml playbook and by publisher.yml. +# Public by design: each terminates its own Let's Encrypt TLS on # tcp/80 + tcp/443 and serves QUIC address discovery on udp/7842, so it cannot share a # host with sponsord-onramp. Deploy at least two before pointing nodes' # decdn_relay_urls at them; by default a relay admits only this inventory's nodes # (iroh_relay_access: allowlist), so list other operators' node IDs in # iroh_relay_allowlist or use everyone. Its knobs are in -# group_vars/iroh_relay_hosts.yml (roles/iroh_relay/README.md). +# group_vars/iroh_relay_hosts.yml (the iroh_relay role's README). # iroh_relay_hosts: # hosts: # relay-1: @@ -76,12 +76,12 @@ decdn_nodes: # # OPTIONAL: a self-hosted iroh DNS server, the pkarr relay nodes publish their # address records to and the DNS server peers resolve them through (in place of -# n0's dns.iroh.link) — provisioned by playbooks/iroh_dns_server.yml (`make -# deploy-dns`) and by publisher.yml. Public by design: it terminates its own Let's -# Encrypt TLS on tcp/443 and answers DNS on udp+tcp/53, so it cannot share a host -# with an iroh relay or sponsord-onramp. The parent zone must delegate its hostname -# to it (NS + glue). Its knobs are in group_vars/iroh_dns_server_hosts.yml -# (roles/iroh_dns_server/README.md). +# n0's dns.iroh.link) — provisioned by the iroh_dns_server.yml playbook and by +# publisher.yml. Public by design: it terminates its own Let's Encrypt TLS on tcp/443 +# and answers DNS on udp+tcp/53, so it cannot share a host with an iroh relay or +# sponsord-onramp. The parent zone must delegate its hostname to it (NS + glue). Its +# knobs are in group_vars/iroh_dns_server_hosts.yml (the iroh_dns_server role's +# README). # iroh_dns_server_hosts: # hosts: # dns-1: diff --git a/docs/node-operators.md b/docs/node-operators.md index f74340a..59ca7a1 100644 --- a/docs/node-operators.md +++ b/docs/node-operators.md @@ -19,7 +19,7 @@ loopback, and the chain's contract addresses taken from upstream's deployment ma | Path | Start here | Entry point | |------|------------|-------------| | **Docker Compose** (one Docker host), recommended | [`compose/README.md`](../compose/README.md) | `decdn-compose init node` | -| **Ansible** (VMs, bare metal, a fleet) | [below](#deploy-with-ansible-from-galaxy), [`ansible/README.md`](../ansible/README.md) | `ansible/inventory/hosts-node.yml.example`, `ansible-playbook decdn.node.node` (or `make deploy-node` from a clone) | +| **Ansible** (VMs, bare metal, a fleet) | [below](#deploy-with-ansible-from-galaxy), [`ansible/README.md`](../ansible/README.md) | `examples/collection-fleet/inventory/hosts-node.yml.example`, `ansible-playbook decdn.node.node` (or `ansible/inventory/hosts-node.yml.example` and `make deploy-node` from a clone) | | **cloud-init** (one VM, no control machine) | [`cloud-init/README.md`](../cloud-init/README.md) | `cloud-init/user-data-node.yaml` | | **Helm** (Kubernetes, one release per node) | [`charts/decdn-node/README.md`](../charts/decdn-node/README.md) | `oci://ghcr.io/decdn/charts/decdn-node`, with no `config.cache.origin` | @@ -42,8 +42,8 @@ The roles and the node playbooks are the [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/node/), so you run them from a private repository that holds only your inventory. Set it up once with the [root README's quickstart](../README.md#quickstart-ansible-from-galaxy) (it clones this -repository only to copy [`examples/collection-fleet/`](../examples/collection-fleet/README.md) -and the inventory templates), then: +repository only to copy [`examples/collection-fleet/`](../examples/collection-fleet/README.md), +inventory templates included), then: ```bash ansible-playbook decdn.node.node -l -u root # a fresh host's first run (or -u ubuntu, …) diff --git a/examples/collection-fleet/README.md b/examples/collection-fleet/README.md index da38305..2c44a99 100644 --- a/examples/collection-fleet/README.md +++ b/examples/collection-fleet/README.md @@ -9,30 +9,33 @@ lifecycle guards included), and this repository holds only your inventory. | File | What it is | |------|------------| | `requirements.yml` | the collections, from Ansible Galaxy (delete `decdn.publisher` for cache nodes only) | -| `ansible.cfg` | this repo's `ansible/ansible.cfg`, minus `roles_path`, failing on an inventory that does not parse | +| `ansible.cfg` | decdn/devops's `ansible/ansible.cfg`, minus `roles_path`, failing on an inventory that does not parse | | `.gitignore` | keeps the installed collections, per-host secrets and backups out of git | +| `inventory/` | the inventory templates: `hosts-node.yml.example` and `hosts-publisher.yml.example`, with their `group_vars/` and `host_vars/` | + +The `inventory/` templates are decdn/devops's `ansible/inventory/` ones, with the hosts +examples' headers written for this layout (decdn/devops's `make test-scripts` keeps +everything else identical). Their comments name files by their path in decdn/devops: +`roles//…` and `playbooks/…` ship in the collections, under +`collections/ansible_collections/decdn/{node,publisher}/` once installed. ## Set up -Copy this directory into a new private repository, then add an inventory, starting -from this repository's examples: +Copy this directory into a new private repository, then, in that copy (never in a +decdn/devops checkout, which is public), start your inventory from the template for +your role: ```bash ansible-galaxy collection install -r requirements.yml -p collections -mkdir -p inventory -cp /ansible/inventory/hosts-node.yml.example inventory/hosts.yml # or hosts-publisher.yml.example -cp -R /ansible/inventory/group_vars /ansible/inventory/host_vars inventory/ +cp inventory/hosts-node.yml.example inventory/hosts.yml # or hosts-publisher.yml.example ``` Fill in `inventory/hosts.yml` and its `group_vars/` and `host_vars/` (each file says what it needs; [`ansible/README.md`](../../ansible/README.md) has the variables and the -on-host secrets). In `hosts-publisher.yml.example` only the origin is live: uncomment -the groups you run. Rename `host_vars/decdn-node-1/` (and `origin-1/`) to your hosts, -and keep secrets in the git-ignored `host_vars//secret.yml` (copied from its -`secret.yml.example`). The copied files are written for this public repository: they -name its make targets ([From `make` to `ansible-playbook`](#from-make-to-ansible-playbook) -translates them), and where they say `hosts.yml` is git-ignored, in your private -repository you commit it. +on-host secrets), and commit them. In `hosts-publisher.yml.example` only the origin is +live: uncomment the groups you run. Rename `host_vars/decdn-node-1/` (and `origin-1/`) +to your hosts, and keep secrets in the git-ignored `host_vars//secret.yml` +(copied from its `secret.yml.example`). ## Run diff --git a/examples/collection-fleet/inventory/group_vars/all.yml b/examples/collection-fleet/inventory/group_vars/all.yml new file mode 100644 index 0000000..810e38a --- /dev/null +++ b/examples/collection-fleet/inventory/group_vars/all.yml @@ -0,0 +1,35 @@ +--- +# Deployment overrides for the deCDN node. +# +# Sane defaults for every knob live in the roles (roles/*/defaults/main.yml) so +# the roles are self-contained. Set anything here to override per host/group. +# Secrets are generated ON THE HOST by the roles and never live here. + +# --- Admin SSH access: ONE operator list (deploy as yourself by default) ----- +# Admin/operator accounts + their keys are installed BEFORE ssh_hardening disables root +# + password login. There is no separate admin knob: the runner (this control box's +# $USER + its ~/.ssh key) is auto-detected and prepended as the head, so the person +# running the playbook is provisioned without being committed here. Add other admins +# below; list yourself (same name) to OVERRIDE the auto-detected head with explicit +# key(s). The baseline role aborts before hardening unless the resolved set has a +# NON-ROOT account with a key, so you can't lock yourself out. +# NOTE: under `sudo ansible-playbook`, cron, or CI the $USER/$HOME — and thus the +# autodetected key — may not be yours; set baseline_sudo_autodetect_runner: false and +# list admins explicitly here. +# baseline_sudo_users: +# - name: alice +# keys: +# - "ssh-ed25519 AAAA... alice@laptop" +# - name: bot # optional per-entry passwordless override +# keys: ["ssh-ed25519 AAAA... bot@ci"] +# passwordless: false +baseline_sudo_users: [] +# baseline_sudo_autodetect_runner: true # false = provision ONLY the explicit list above +# baseline_sudo_passwordless: true # default sudo mode: NOPASSWD sudo + locked password + +# Optional inbound-SSH source allowlist (CIDRs). Empty = accept from any source. +ssh_allow_cidrs: [] + +# --- Common overrides (uncomment to change; see roles/*/defaults for all) ---- +# See roles/decdn_node/defaults/main.yml for the node's tunable knobs, and set +# per-node config in inventory/host_vars//main.yml. diff --git a/examples/collection-fleet/inventory/group_vars/decdn_nodes.yml b/examples/collection-fleet/inventory/group_vars/decdn_nodes.yml new file mode 100644 index 0000000..c77132b --- /dev/null +++ b/examples/collection-fleet/inventory/group_vars/decdn_nodes.yml @@ -0,0 +1,42 @@ +--- +# Shared config for public deCDN node hosts, cache nodes (playbooks/node.yml) and a +# publisher's origins alike (playbooks/origin.yml; group_vars/decdn_origin_nodes.yml +# wins over this file for them). +# Per-node values live in host_vars//: addresses/region/version in main.yml. +# The sensitive rpc_url is provisioned as /etc/decdn/decdn.env on the host, or (if +# you would rather the role authored that file) in a git-ignored secret.yml. + +# The node's public QUIC firewall hole (udp/4433) is NOT set here. The playbooks set +# it from their own group_vars/ (all.yml builds it), which ship beside them in +# decdn/devops and in its Galaxy collections, so it applies to any inventory. +# +# Settings every node shares (decdn_network, decdn_node_install_method, …) can move +# here from host_vars//main.yml once you run more than one node. host_vars +# still win for a node that differs. + +# --- Binaries (fleet-wide) ------------------------------------------------------ +# With decdn_node_install_method: manual, rather than the two per-host +# *_manual_bin_src paths in host_vars, point every node at one decdn +# checkout's target/release dir (cross-compile for aarch64 hosts; the role derives +# each host's target from its architecture and checks the ELF before shipping): +# cargo build --release -p decdn-node -p decdn-cli +# A host that sets BOTH explicit paths ignores this dir. +# decdn_release_target_dir: "" # control-machine path to decdn's target/release + +# --- Cache sizing ----------------------------------------------------------------- +# Size to each host's disk and to the largest object you expect to serve; see +# https://github.com/decdn/devops/blob/main/docs/requirements.md for the disk formula. +# A node rejects any blob larger than decdn_max_blob_size_mb with BlobTooLarge, and +# decdn_cache_size_mb must be at least that. Override per host in host_vars when +# disks differ. +# decdn_max_blob_size_mb: 1024 # role default +# decdn_cache_size_mb: 10240 # role default (10 GiB) + +# --- Grafana Cloud observability (opt-in) -------------------------------------- +# One mirrored flag drives BOTH roles (grafana_alloy install/config AND the +# otlp_endpoint injection into node.toml). Uncomment after provisioning the +# credential file on each host — see roles/grafana_alloy/files/grafana-alloy.env.example: +# +# decdn_grafana_cloud_enabled: true +# grafana_alloy_region: "{{ decdn_region }}" # override only when needed +# grafana_alloy_deployment_environment: production diff --git a/examples/collection-fleet/inventory/group_vars/decdn_origin_nodes.yml b/examples/collection-fleet/inventory/group_vars/decdn_origin_nodes.yml new file mode 100644 index 0000000..3d04ada --- /dev/null +++ b/examples/collection-fleet/inventory/group_vars/decdn_origin_nodes.yml @@ -0,0 +1,25 @@ +--- +# Shared config for a publisher's origin nodes (playbooks/origin.yml, also run by +# publisher.yml and site.yml). decdn_origin_nodes is a child of decdn_nodes, so +# group_vars/decdn_nodes.yml applies to origins too, and this file wins over it. +# Per-origin values live in host_vars//main.yml (see host_vars/origin-1/). +# +# Every origin needs an origin backend: decdn_cache_origin_kind (http, fs or s3) and +# that kind's fields, or decdn_cache_origins (an ordered fallback list); origin.yml +# refuses a host without one. Set it here when every origin reads the same store, in +# host_vars when they differ. Full list: roles/decdn_node/defaults/main.yml. +# +# decdn_cache_origin_kind: s3 +# decdn_cache_origin_s3_bucket: my-namespace-content +# decdn_cache_origin_s3_region: eu-central-1 +# decdn_cache_origin_s3_endpoint_url: "" # R2/B2/MinIO; omit for AWS +# decdn_cache_origin_s3_use_default_chain: true # keys in /etc/decdn/decdn.env + +# --- Origin behaviour ---------------------------------------------------------------- +# With a backend set, two defaults follow: decdn_relay_foreign_namespaces false (the +# daemon's: the node serves only what its backend holds and declines the rest, even a +# blob already in its cache; ADR 002 § Retrieval by namespace, ADR 037) and +# decdn_node_to_node_pull_through_enabled false (the role's: misses fill from the +# backend). Set either explicitly only to change that. +# decdn_fs_rescan_interval_sec: 60 # fs backend: how often the root is rescanned +# decdn_pinned_hashes: [] # blobs (lowercase BLAKE3) never evicted diff --git a/examples/collection-fleet/inventory/group_vars/iroh_dns_server_hosts.yml b/examples/collection-fleet/inventory/group_vars/iroh_dns_server_hosts.yml new file mode 100644 index 0000000..d36a176 --- /dev/null +++ b/examples/collection-fleet/inventory/group_vars/iroh_dns_server_hosts.yml @@ -0,0 +1,39 @@ +--- +# Shared config for iroh DNS server hosts (playbooks/iroh_dns_server.yml, also run +# by publisher.yml). Uncomment and fill in; per-host values (the hostname, at least) go +# under the host in hosts.yml (as hosts-publisher.yml.example shows) or in +# host_vars//main.yml. +# Full variable list: roles/iroh_dns_server/defaults/main.yml and its README.md. +# Nothing to provision on the host: the server generates its ACME account key and +# TLS key itself, in its 0700 state directory. + +# --- Public name + certificate (REQUIRED) ------------------------------------------ +# Per host. The parent zone delegates the hostname to this host (NS . +# plus a glue A record), and tcp/443 must reach it from the internet, before Let's +# Encrypt issues the certificate. +# iroh_dns_server_hostname: dns.example.org +# iroh_dns_server_acme_contact: ops@example.org +# A trial run on Let's Encrypt's staging CA (untrusted certificates): +# iroh_dns_server_acme_staging: true + +# --- DNS zone ---------------------------------------------------------------------- +# The public IPv4 the glue record names. Defaults to the DNS bind address when that +# is a public one; a host behind 1:1 NAT (most clouds) must set it. +# iroh_dns_server_rr_a: 203.0.113.10 +# iroh_dns_server_rr_aaaa: "2001:db8::10" +# More zones served beside the hostname: +# iroh_dns_server_origins: [dns.example.org, discovery.example.net] + +# --- Listeners --------------------------------------------------------------------- +# DNS binds ONE address, by default the host's default IPv4 address, which leaves +# systemd-resolved's 127.0.0.53 alone. "::" also serves IPv6, once nothing else holds +# port 53 (DNSStubListener=no). On a host behind NAT, the private address the +# provider forwards to is the right one. +# iroh_dns_server_dns_bind_address: "::" +# iroh_dns_server_https_bind_address: "0.0.0.0" + +# --- The deploy's certificate check ------------------------------------------------ +# "warn" (default) reports a server without a trusted certificate yet, and fails once +# Let's Encrypt has issued one before; "fail" always fails without one, for once the +# delegation and tcp/443 are known to be in place. +# iroh_dns_server_certificate_check: fail diff --git a/examples/collection-fleet/inventory/group_vars/iroh_relay_hosts.yml b/examples/collection-fleet/inventory/group_vars/iroh_relay_hosts.yml new file mode 100644 index 0000000..3212200 --- /dev/null +++ b/examples/collection-fleet/inventory/group_vars/iroh_relay_hosts.yml @@ -0,0 +1,45 @@ +--- +# Shared config for iroh relay hosts (playbooks/iroh_relay.yml, also run by +# publisher.yml). Uncomment and fill in; per-host values (the hostname, at least) go +# under the host in hosts.yml (as hosts-publisher.yml.example shows) or in +# host_vars//main.yml. +# Full variable list: roles/iroh_relay/defaults/main.yml and its README.md. +# Nothing to provision on the host: the relay generates its ACME account key and +# TLS key itself, in its 0700 state directory. + +# --- Public name + certificate (REQUIRED) ------------------------------------------ +# Per host. Its A/AAAA records must point at the host, and tcp/443 must reach it from +# the internet, before Let's Encrypt issues the certificate. +# iroh_relay_hostname: relay1.example.org +# iroh_relay_acme_contact: ops@example.org +# A trial run on Let's Encrypt's staging CA (untrusted certificates): +# iroh_relay_acme_staging: true + +# --- Listeners --------------------------------------------------------------------- +# "::" binds every address (IPv4 too on a dual-stack host); "0.0.0.0" for an +# IPv4-only host. QUIC address discovery (udp/7842) is on; set it here, not in a +# play, because the firewall hole reads it. +# iroh_relay_bind_address: "0.0.0.0" +# iroh_relay_enable_quic_addr_discovery: false + +# --- Access + limits --------------------------------------------------------------- +# By default only listed endpoint IDs may relay: every decdn_nodes host's ID (read on +# the node by playbooks/iroh_relay.yml, so a relay deploy needs SSH to every node) +# plus these. decdn clients use a fresh key per fetch and cannot be listed: they +# reach a node homed here only if the node is directly reachable, so nodes behind +# NAT that serve clients need relays in "everyone" mode. IDs are 64 lowercase hex, +# as `decdn whoami` prints them. Re-deploy the relays after adding a node. +# iroh_relay_allowlist: [] +# Or serve everyone, or everyone but some: +# iroh_relay_access: everyone # "allowlist" | "denylist" | "everyone" +# iroh_relay_denylist: [] +# Per-client receive rate (bytes/s) and burst; server-wide new-connection rate: +# iroh_relay_client_rx_bytes_per_second: 4194304 +# iroh_relay_client_rx_max_burst_bytes: 8388608 +# iroh_relay_accept_conn_limit: 50 +# iroh_relay_accept_conn_burst: 100 + +# --- The deploy's certificate check ------------------------------------------------ +# "warn" (default) reports a relay without a trusted certificate yet; "fail" fails +# the deploy once DNS and tcp/443 are known to be in place. +# iroh_relay_certificate_check: fail diff --git a/examples/collection-fleet/inventory/group_vars/sponsord_hosts.yml b/examples/collection-fleet/inventory/group_vars/sponsord_hosts.yml new file mode 100644 index 0000000..6680a40 --- /dev/null +++ b/examples/collection-fleet/inventory/group_vars/sponsord_hosts.yml @@ -0,0 +1,43 @@ +--- +# Shared config for sponsord hosts (playbooks/sponsord.yml, also run by publisher.yml). +# Uncomment and fill in; per-host values can move to host_vars//main.yml. +# Full variable list: roles/sponsord/defaults/main.yml and roles/sponsord/README.md. +# +# Before the first deploy, on the host as root (the role never creates the wallet): +# /etc/sponsord/treasury-keystore.json the treasury keystore (decdn key-gen), 0600 +# /etc/sponsord/treasury-password its password, 0600 +# /etc/sponsord/secret.env SPONSORD_RPC_URL=… (or sponsord_rpc_url in +# a git-ignored host_vars//secret.yml) +# The API token is generated on the host at /etc/sponsord/api-token. + +# --- Binary ------------------------------------------------------------------------ +# The default method, "release", installs the decdn/sponsord release the role pins +# (tag v); set it only to hold the host on another one, together +# with sponsord_onramp_version: +# sponsord_version: "0.0.4" +# Or build on the host from a git ref (tag, branch or SHA; sponsord_version is not +# checked in this mode): +# sponsord_install_method: source +# sponsord_source_ref: v0.0.4 +# Or copy a control-machine build: a decdn/sponsord checkout's target dir +# (`cargo build --release -p sponsord`; the role checks the ELF matches the host). +# sponsord_install_method: manual +# sponsord_release_target_dir: /path/to/sponsord/target/release + +# --- Chain + pool ------------------------------------------------------------------ +# chain_id and the PaymentPool address from upstream's deployment manifest. +# sponsord_network: arbitrum-sepolia +# The pool `decdn pool open` printed when the treasury wallet opened it (REQUIRED). +# sponsord_pool_id: "0x…" + +# --- Limits ("" = the daemon's default) --------------------------------------------- +# sponsord_max_spending_cap_micro_usdc: "" # [5000000 = $5] +# sponsord_max_ttl_secs: "" # [172800 = 48h] +# sponsord_pool_low_water_micro_usdc: "" # [20000000 = $20] +# sponsord_pool_refill_micro_usdc: "" # [100000000 = $100] + +# --- Grafana Cloud (opt-in) -------------------------------------------------------- +# Same agent as the node's. On a sponsord-only host the region label has no +# decdn_region to fall back on, so set it here. +# decdn_grafana_cloud_enabled: true +# grafana_alloy_region: US diff --git a/examples/collection-fleet/inventory/group_vars/sponsord_onramp_hosts.yml b/examples/collection-fleet/inventory/group_vars/sponsord_onramp_hosts.yml new file mode 100644 index 0000000..21a22f8 --- /dev/null +++ b/examples/collection-fleet/inventory/group_vars/sponsord_onramp_hosts.yml @@ -0,0 +1,39 @@ +--- +# Shared config for sponsord-onramp hosts (the second play of playbooks/sponsord.yml, +# also run by publisher.yml). Each host must be in sponsord_hosts too. Uncomment and fill +# in; per-host values can move to host_vars//main.yml. +# Full variable list: roles/sponsord_onramp/defaults/main.yml and its README.md. +# +# The Turnstile widget's secret key: sponsord_onramp_turnstile_secret in the +# git-ignored host_vars//secret.yml (the role writes it to the host), or +# /etc/sponsord/turnstile-secret (0600) written on the host by you. +# The daemon's API token (/etc/sponsord/api-token) is the sponsord role's. + +# --- Binary ------------------------------------------------------------------------ +# The same three methods as sponsord. "release" (default) installs the same release +# as the sponsord role; set both versions together to hold the host on another: +# sponsord_onramp_version: "0.0.4" +# Or build on the host (its own checkout, so it may pin a different ref): +# sponsord_onramp_install_method: source +# sponsord_onramp_source_ref: v0.0.4 +# Or manual, which follows sponsord_release_target_dir (`cargo build --release` in +# a decdn/sponsord checkout builds both binaries): +# sponsord_onramp_install_method: manual + +# --- Public address ---------------------------------------------------------------- +# Caddy terminates TLS for this name (ACME). Its A/AAAA records must point at the +# host; the playbooks open tcp/80 + tcp/443. To bring your own proxy instead, set +# sponsord_onramp_proxy: none (HERE, not in a play: the firewall holes read it) and +# sponsord_onramp_public_url. +# sponsord_onramp_domain: downloads.example.org +# sponsord_onramp_acme_email: ops@example.org + +# --- What users get (REQUIRED) ----------------------------------------------------- +# PUBLIC: served to every user. A keyless endpoint, never sponsord_rpc_url. +# sponsord_onramp_rpc_url: https://sepolia-rollup.arbitrum.io/rpc +# sponsord_onramp_turnstile_sitekey: "0x…" +# The decdn release the installers install defaults to the role's pin (decdn/decdn +# v0.0.4), with the SHA-256 of its SHA256SUMS. To serve another, set both: +# sponsord_onramp_decdn_release: v0.0.4 +# sponsord_onramp_decdn_sums_sha256: "…" +# CapacityBond and SlashJudge come from sponsord_network (group_vars/sponsord_hosts.yml). diff --git a/examples/collection-fleet/inventory/host_vars/decdn-node-1/main.yml b/examples/collection-fleet/inventory/host_vars/decdn-node-1/main.yml new file mode 100644 index 0000000..fa4838e --- /dev/null +++ b/examples/collection-fleet/inventory/host_vars/decdn-node-1/main.yml @@ -0,0 +1,61 @@ +--- +# Per-node deployment values for decdn-node-1 (COMMITTED — non-secret config only). +# Ansible merges every *.yml in this directory for host decdn-node-1. The sensitive +# values (decdn_rpc_url, and any environment-borne secrets) live OFF this file: +# either in a 0600 /etc/decdn/decdn.env provisioned on the target host (preferred — +# nothing sensitive on the control machine), or in the sibling git-ignored +# secret.yml (copy secret.yml.example). The role refuses to start the node until +# this file is set and ONE of those two secret sources exists. +# +# CONTRACT ADDRESSES ARE NOT INVENTED HERE, and not copied here either: +# decdn_network below selects the role's mirror of upstream's deployment manifest +# (roles/decdn_node/vars/main/networks.yml, regenerated by decdn/devops's +# scripts/sync-network-profiles.py from contracts/deployments/.json in +# decdn/decdn). +# It is the same manifest `decdn config init --chain arbitrum-sepolia` bakes in, and +# upstream redeploys change every address at once, so one mirror beats a hand copy +# per inventory. To target a different deployment, clear decdn_network and set +# decdn_chain_id plus the decdn_*_address variables from that deployment's manifest. +# MUST-EDIT lines are yours to fill. + +# --- Which binaries to install ------------------------------------------------ +# "release" (the default): the v GitHub Release tarballs, verified against +# the GPG-signed SHA256SUMS manifest. The version defaults to the decdn/decdn release +# the role pins; set it only to hold a node on another one. +decdn_node_install_method: release +# decdn_node_version: "0.0.4" +# +# Alternative: build on the node from a git ref (any tag, branch or SHA), as an +# unprivileged build user with a pinned toolchain (decdn_node_version is not checked +# in this mode): +# decdn_node_install_method: source +# decdn_node_source_ref: v0.0.4 +# Alternative: copy binaries built on this control machine +# (`cargo build --release -p decdn-node -p decdn-cli` in a decdn/decdn checkout): +# decdn_node_install_method: manual +# decdn_node_manual_bin_src: /path/to/decdn/target/release/decdn-node +# decdn_cli_manual_bin_src: /path/to/decdn/target/release/decdn + +# --- Chain (Arbitrum Sepolia, chain 421614) ----------------------------------- +# The RPC endpoint is the one sensitive value — provision it as DECDN_RPC_URL in +# /etc/decdn/decdn.env on the host, or set decdn_rpc_url in secret.yml. +# The profile supplies decdn_chain_id and all eight contract addresses (the four +# required ones, SlashAppeal, OriginAssignment, PublisherRegistry and USDC). Set any +# decdn_*_address here only to override it deliberately; the role reports overrides. +decdn_network: arbitrum-sepolia + +# --- Cache ------------------------------------------------------------------- +# A cache node has no origin backend (decdn_cache_origin_kind stays unset): it fills +# a cache miss by pulling from other nodes (node-to-node pull-through, on by default +# when no origin is set) and relays every namespace. An origin backend makes a node a +# publisher's origin instead: see host_vars/origin-1/ and hosts-publisher.yml.example. + +# --- Node identity / locale --------------------------------------------------- +decdn_region: "US" # MUST-EDIT — ISO 3166-1 alpha-2 of the node's physical location +# decdn_relay_urls: [] # optional iroh relays for NAT traversal ([] => n0 defaults) + +# --- Economics ---------------------------------------------------------------- +# Seller-side quote: this node's asking price, advertised verbatim and never clamped +# to the delivery floor by the daemon (ADR 003 §Rate-floor enforcement, ADR 019 +# §Step 3.3 — the ADRs, not this file, govern that behaviour). +decdn_rate_per_mb: 10 # USDC base units (6 decimals); 10 = $0.00001/MB diff --git a/examples/collection-fleet/inventory/host_vars/decdn-node-1/secret.yml.example b/examples/collection-fleet/inventory/host_vars/decdn-node-1/secret.yml.example new file mode 100644 index 0000000..b8846b7 --- /dev/null +++ b/examples/collection-fleet/inventory/host_vars/decdn-node-1/secret.yml.example @@ -0,0 +1,56 @@ +--- +# Per-node SECRETS for decdn-node-1 — the INVENTORY path, which is optional. +# +# You only need this file if you want the role to author /etc/decdn/decdn.env for +# you. The preferred posture is the other way round: provision that 0600 file on +# the target host (see roles/decdn_node/files/decdn.env.example) and skip this +# file entirely — the secret then never sits in plaintext on your control machine +# and never transits it on a converge. The role fails loud if NEITHER exists. +# +# If you do use it: copy to secret.yml (same directory, git-ignored) and fill in. +# Ansible merges it with main.yml for this host. These are the ONLY per-node +# values kept out of git — everything else lives in the committed main.yml. +# +# SENSITIVE (may embed an API key). Setting this makes the role REWRITE +# /etc/decdn/decdn.env from these values on every run, discarding any lines added +# on the host. The public endpoint below works for light use; run your own or use +# a provider for production reliability. +decdn_rpc_url: "https://sepolia-rollup.arbitrum.io/rpc" + +# Optional, and only meaningful alongside decdn_rpc_url above (the role writes the +# env file all-or-nothing, and rejects extra_env against a host-provisioned file +# rather than silently dropping it). Extra KEY: value pairs appended to the 0600 +# /etc/decdn/decdn.env that systemd hands the daemon — most notably the AWS +# credentials backing an S3 cache origin, which the role deliberately does NOT +# write into node.toml (that file is 0640 and diffable): +# +# decdn_extra_env: +# AWS_ACCESS_KEY_ID: "AKIA..." +# AWS_SECRET_ACCESS_KEY: "..." +# AWS_SESSION_TOKEN: "..." # only for assume-role / SSO flows +# +# Pair that with decdn_cache_origin_s3_use_default_chain: true in main.yml. + +# --- Grafana Cloud (only when decdn_grafana_cloud_enabled is on) --------------- +# +# The opt-in observability agent accepts its API token through the same channel: +# set grafana_alloy_api_token here and the role authors /etc/grafana-alloy.env on +# the host for you (token-only, root 0600), skipping the hand-provisioning shown +# in roles/grafana_alloy/files/grafana-alloy.env.example. SENSITIVE — leave it +# commented out to keep the operator-provisioned host path instead. +# +# grafana_alloy_api_token: "glc_example_token_replace_me" + +# --- sponsord (only on a host in sponsord_hosts) -------------------------------- +# +# Same choice for sponsord's RPC endpoint: set it here and the role authors +# /etc/sponsord/secret.env (root 0600), or leave it commented out and provision that +# file on the host (roles/sponsord/files/secret.env.example). SENSITIVE. +# +# sponsord_rpc_url: "https://sepolia-rollup.arbitrum.io/rpc" +# +# And for the onramp (a host in sponsord_onramp_hosts): the Turnstile widget's +# secret key. Set it here and the role writes /etc/sponsord/turnstile-secret (root +# 0600), or leave it commented out and write that file on the host. SENSITIVE. +# +# sponsord_onramp_turnstile_secret: "0x4AAAAAAA…" diff --git a/examples/collection-fleet/inventory/host_vars/origin-1/main.yml b/examples/collection-fleet/inventory/host_vars/origin-1/main.yml new file mode 100644 index 0000000..05bbbf5 --- /dev/null +++ b/examples/collection-fleet/inventory/host_vars/origin-1/main.yml @@ -0,0 +1,73 @@ +--- +# Per-node deployment values for origin-1, a publisher's origin node (COMMITTED — +# non-secret config only). Ansible merges every *.yml in this directory for host +# origin-1. An origin is a node like decdn-node-1 (see that host's main.yml) plus +# the origin backend below. The sensitive +# values (decdn_rpc_url, and any environment-borne secrets) live OFF this file: +# either in a 0600 /etc/decdn/decdn.env provisioned on the target host (preferred — +# nothing sensitive on the control machine), or in the sibling git-ignored +# secret.yml (copy secret.yml.example). The role refuses to start the node until +# this file is set and ONE of those two secret sources exists. +# +# CONTRACT ADDRESSES ARE NOT INVENTED HERE, and not copied here either: +# decdn_network below selects the role's mirror of upstream's deployment manifest +# (roles/decdn_node/vars/main/networks.yml, regenerated by decdn/devops's +# scripts/sync-network-profiles.py from contracts/deployments/.json in +# decdn/decdn). +# It is the same manifest `decdn config init --chain arbitrum-sepolia` bakes in, and +# upstream redeploys change every address at once, so one mirror beats a hand copy +# per inventory. To target a different deployment, clear decdn_network and set +# decdn_chain_id plus the decdn_*_address variables from that deployment's manifest. +# MUST-EDIT lines are yours to fill. + +# --- Which binaries to install ------------------------------------------------ +# "release" (the default): the v GitHub Release tarballs, verified against +# the GPG-signed SHA256SUMS manifest. The version defaults to the decdn/decdn release +# the role pins; set it only to hold a node on another one. +decdn_node_install_method: release +# decdn_node_version: "0.0.4" +# +# Alternative: build on the node from a git ref (any tag, branch or SHA), as an +# unprivileged build user with a pinned toolchain (decdn_node_version is not checked +# in this mode): +# decdn_node_install_method: source +# decdn_node_source_ref: v0.0.4 +# Alternative: copy binaries built on this control machine +# (`cargo build --release -p decdn-node -p decdn-cli` in a decdn/decdn checkout): +# decdn_node_install_method: manual +# decdn_node_manual_bin_src: /path/to/decdn/target/release/decdn-node +# decdn_cli_manual_bin_src: /path/to/decdn/target/release/decdn + +# --- Chain (Arbitrum Sepolia, chain 421614) ----------------------------------- +# The RPC endpoint is the one sensitive value — provision it as DECDN_RPC_URL in +# /etc/decdn/decdn.env on the host, or set decdn_rpc_url in secret.yml. +# The profile supplies decdn_chain_id and all eight contract addresses (the four +# required ones, SlashAppeal, OriginAssignment, PublisherRegistry and USDC). Set any +# decdn_*_address here only to override it deliberately; the role reports overrides. +decdn_network: arbitrum-sepolia + +# --- Origin backend (the canonical source of the namespace's content) --------- +# MUST-EDIT — where this origin reads the content it serves (NOT a chain fact). +# http shown; fs (a local disk or an NFS mount: decdn_cache_origin_path) and s3 +# (S3/R2/B2/MinIO: decdn_cache_origin_s3_*) are in roles/decdn_node/defaults/main.yml, +# and decdn_cache_origins takes an ordered fallback list. S3 access keys never go +# here: put them in /etc/decdn/decdn.env on the host (AWS_ACCESS_KEY_ID, …) with +# decdn_cache_origin_s3_use_default_chain: true, or in secret.yml's decdn_extra_env. +# playbooks/origin.yml refuses a decdn_origin_nodes host without a backend. +decdn_cache_origin_kind: "http" +decdn_cache_origin_url: "https://your-origin.example/" +# Being configured is not being recognised: the namespace's publisher seats this +# node's operator with OriginAssignment.addOrigin (decdn/adr/011-content-takedown.md, +# Origin Assignment Authority). With a backend set, the node serves only what the +# backend holds (decdn_relay_foreign_namespaces defaults to false) and does not pull +# from other nodes (decdn_node_to_node_pull_through_enabled). + +# --- Node identity / locale --------------------------------------------------- +decdn_region: "US" # MUST-EDIT — ISO 3166-1 alpha-2 of the node's physical location +# decdn_relay_urls: [] # optional iroh relays for NAT traversal ([] => n0 defaults) + +# --- Economics ---------------------------------------------------------------- +# Seller-side quote: this node's asking price, advertised verbatim and never clamped +# to the delivery floor by the daemon (ADR 003 §Rate-floor enforcement, ADR 019 +# §Step 3.3 — the ADRs, not this file, govern that behaviour). +decdn_rate_per_mb: 10 # USDC base units (6 decimals); 10 = $0.00001/MB diff --git a/examples/collection-fleet/inventory/host_vars/origin-1/secret.yml.example b/examples/collection-fleet/inventory/host_vars/origin-1/secret.yml.example new file mode 100644 index 0000000..c511b8c --- /dev/null +++ b/examples/collection-fleet/inventory/host_vars/origin-1/secret.yml.example @@ -0,0 +1,56 @@ +--- +# Per-node SECRETS for origin-1 — the INVENTORY path, which is optional. +# +# You only need this file if you want the role to author /etc/decdn/decdn.env for +# you. The preferred posture is the other way round: provision that 0600 file on +# the target host (see roles/decdn_node/files/decdn.env.example) and skip this +# file entirely — the secret then never sits in plaintext on your control machine +# and never transits it on a converge. The role fails loud if NEITHER exists. +# +# If you do use it: copy to secret.yml (same directory, git-ignored) and fill in. +# Ansible merges it with main.yml for this host. These are the ONLY per-node +# values kept out of git — everything else lives in the committed main.yml. +# +# SENSITIVE (may embed an API key). Setting this makes the role REWRITE +# /etc/decdn/decdn.env from these values on every run, discarding any lines added +# on the host. The public endpoint below works for light use; run your own or use +# a provider for production reliability. +decdn_rpc_url: "https://sepolia-rollup.arbitrum.io/rpc" + +# Optional, and only meaningful alongside decdn_rpc_url above (the role writes the +# env file all-or-nothing, and rejects extra_env against a host-provisioned file +# rather than silently dropping it). Extra KEY: value pairs appended to the 0600 +# /etc/decdn/decdn.env that systemd hands the daemon — most notably the AWS +# credentials backing an S3 cache origin, which the role deliberately does NOT +# write into node.toml (that file is 0640 and diffable): +# +# decdn_extra_env: +# AWS_ACCESS_KEY_ID: "AKIA..." +# AWS_SECRET_ACCESS_KEY: "..." +# AWS_SESSION_TOKEN: "..." # only for assume-role / SSO flows +# +# Pair that with decdn_cache_origin_s3_use_default_chain: true in main.yml. + +# --- Grafana Cloud (only when decdn_grafana_cloud_enabled is on) --------------- +# +# The opt-in observability agent accepts its API token through the same channel: +# set grafana_alloy_api_token here and the role authors /etc/grafana-alloy.env on +# the host for you (token-only, root 0600), skipping the hand-provisioning shown +# in roles/grafana_alloy/files/grafana-alloy.env.example. SENSITIVE — leave it +# commented out to keep the operator-provisioned host path instead. +# +# grafana_alloy_api_token: "glc_example_token_replace_me" + +# --- sponsord (only on a host in sponsord_hosts) -------------------------------- +# +# Same choice for sponsord's RPC endpoint: set it here and the role authors +# /etc/sponsord/secret.env (root 0600), or leave it commented out and provision that +# file on the host (roles/sponsord/files/secret.env.example). SENSITIVE. +# +# sponsord_rpc_url: "https://sepolia-rollup.arbitrum.io/rpc" +# +# And for the onramp (a host in sponsord_onramp_hosts): the Turnstile widget's +# secret key. Set it here and the role writes /etc/sponsord/turnstile-secret (root +# 0600), or leave it commented out and write that file on the host. SENSITIVE. +# +# sponsord_onramp_turnstile_secret: "0x4AAAAAAA…" diff --git a/examples/collection-fleet/inventory/hosts-node.yml.example b/examples/collection-fleet/inventory/hosts-node.yml.example new file mode 100644 index 0000000..3fa14cf --- /dev/null +++ b/examples/collection-fleet/inventory/hosts-node.yml.example @@ -0,0 +1,55 @@ +--- +# NODE OPERATORS: once this directory is your own private repository, copy this to +# inventory/hosts.yml, fill in your hosts and commit it (ansible.cfg reads it). A +# publisher starts from hosts-publisher.yml.example instead. +# +# No ansible_user by default: Ansible connects as your local $USER, which is the admin +# account the baseline role creates (the auto-detected runner). A fresh box doesn't have +# it yet, so bootstrap each new host ONCE as a user that can sudo (root, or the image's +# default user such as ubuntu): +# ansible-playbook decdn.node.node -l -u root # swap root for your image's bootstrap user +# After that first converge ssh_hardening disables root login, and a plain +# `ansible-playbook decdn.node.node -l ` connects as $USER. Set a per-host +# ansible_user only if the admin account's name differs from your $USER (you listed +# another name in baseline_sudo_users) or you set baseline_sudo_autodetect_runner: +# false. Never set it to the bootstrap user: an inventory ansible_user overrides -u, and +# root login is gone after the first converge. For the same reason, add a per-host +# ansible_user only AFTER that host's bootstrap run. +# +# Ubuntu sudo-rs workaround: 25.10 (Questing) and 26.04 ship sudo-rs as the default +# /usr/bin/sudo. Ansible's sudo become plugin passes a custom `-p` prompt sentinel and +# waits for it before sending the become password; sudo-rs does not honor that custom +# `-p`, so `--ask-become-pass` / -K hangs with "Timeout waiting for privilege +# escalation prompt". This only bites when a become PASSWORD is actually sent: with the +# default `baseline_sudo_passwordless: true` the admin user has NOPASSWD sudo, so +# Ansible sends no become password and the hang never occurs. It resurfaces only if you +# set `baseline_sudo_passwordless: false` (classic password sudo). Fix: classic sudo is +# still installed at /usr/bin/sudo.ws and honors the -p prompt Ansible expects — point +# become at it with the per-host `ansible_become_exe` line below. Only uncomment on an +# affected host: `sudo --version` reports "sudo-rs" and/or /usr/bin/sudo.ws exists. Leave +# it commented on Debian bookworm / Ubuntu noble — that path does not exist there. + +# Public deCDN cache nodes — provisioned by the node.yml playbook (which site.yml +# imports too). A cache node has no origin backend: it fills a miss from other nodes +# (node-to-node pull-through). Origins are a publisher's (hosts-publisher.yml.example). +# Per-node config (contract addresses, region, version, …) goes in the committed +# inventory/host_vars//main.yml; secrets (rpc_url, and any other +# secret.yml.example lists) go in a git-ignored secret.yml beside it (copy the shipped +# secret.yml.example there). +decdn_nodes: + hosts: + decdn-node-1: + ansible_host: REPLACE_WITH_NODE_VPS_IP_OR_DNS + # ansible_user: alice # post-bootstrap, only if the admin account's name ≠ your $USER — see note above + # ansible_become_exe: /usr/bin/sudo.ws # Ubuntu sudo-rs workaround — see note above + # More nodes: one entry each, with its own host_vars//main.yml + # (decdn_region is REQUIRED per host). + # decdn-node-2: + # ansible_host: REPLACE_WITH_NODE_VPS_IP_OR_DNS + # Child groups, for hosts that differ in a way that is easier to express per group + # than per host (disk size, an egress cap on pay-per-TB bandwidth). Give + # each one a group_vars/.yml; Ansible merges it over decdn_nodes'. + # children: + # big_disk: + # hosts: + # decdn-node-2: diff --git a/examples/collection-fleet/inventory/hosts-publisher.yml.example b/examples/collection-fleet/inventory/hosts-publisher.yml.example new file mode 100644 index 0000000..df1b3e2 --- /dev/null +++ b/examples/collection-fleet/inventory/hosts-publisher.yml.example @@ -0,0 +1,87 @@ +--- +# PUBLISHERS: once this directory is your own private repository, copy this to +# inventory/hosts.yml, fill in your hosts and commit it (ansible.cfg reads it). A node +# operator starts from hosts-node.yml.example instead. +# +# A publisher runs its origin nodes and, optionally, the services around them. Each +# group below has its own playbook in decdn.publisher; decdn.publisher.publisher runs +# them all, and decdn.publisher.site also runs any cache nodes listed here. +# +# Bootstrapping a fresh host, ansible_user and the Ubuntu sudo-rs workaround work as +# in hosts-node.yml.example: read its header before the first run. In short, the first +# run per host is baseline only, +# ansible-playbook decdn.publisher.publisher -l --tags baseline -u root +# (your image's bootstrap user in place of root); after it, a plain +# `ansible-playbook decdn.publisher.publisher -l `. Baseline only, because a +# relay's full run reads every node's ID over SSH, and under -u root it would try the +# nodes, whose root login baseline has already turned off, as root. + +# Origin nodes — provisioned by the origin.yml playbook and by publisher.yml (which +# site.yml imports). An origin is a deCDN node with an origin backend: the canonical +# source of its namespace's content. decdn_origin_nodes is a child of decdn_nodes, so +# an origin gets a node's firewall hole, backup, decommission and observability; +# origin.yml refuses one outside decdn_nodes or without a backend. The chain +# recognises it as the origin only once you, the namespace's publisher, seat its +# operator with OriginAssignment.addOrigin; until then it serves its bytes as a cache +# does. The backend goes in group_vars/decdn_origin_nodes.yml (shared by every origin) +# or host_vars//main.yml (see host_vars/origin-1/), with the region and the rest +# of the per-origin config. +decdn_nodes: + children: + decdn_origin_nodes: + hosts: + origin-1: + ansible_host: REPLACE_WITH_ORIGIN_VPS_IP_OR_DNS + # More origins: one entry each, with its own host_vars//main.yml + # (decdn_region and the origin backend are REQUIRED per host). + # origin-2: + # ansible_host: REPLACE_WITH_ORIGIN_VPS_IP_OR_DNS + +# OPTIONAL: sponsord, the onboarding sponsor (treasury signer + PaymentPool keeper), +# which pays for your users' downloads — provisioned by the sponsord.yml playbook and +# by publisher.yml. It needs no node: give it its own host, or list an origin host +# here as well to co-locate them. Its knobs are in group_vars/sponsord_hosts.yml; the +# treasury keystore and password are provisioned on the host (the sponsord role's +# README). +# sponsord_hosts: +# hosts: +# sponsord-1: +# ansible_host: REPLACE_WITH_SPONSORD_VPS_IP_OR_DNS +# +# OPTIONAL: sponsord-onramp, sponsord's public side (Turnstile gate, installers, the +# capability API), behind Caddy on tcp/80 + tcp/443. List a sponsord host +# again here; it must be in sponsord_hosts too. Its knobs are in +# group_vars/sponsord_onramp_hosts.yml; the Turnstile secret is provisioned on the +# host (the sponsord_onramp role's README). +# sponsord_onramp_hosts: +# hosts: +# sponsord-1: +# +# OPTIONAL: self-hosted iroh relays, the fallback path for peers that cannot +# hole-punch — provisioned by the iroh_relay.yml playbook and by publisher.yml. +# Public by design: each terminates its own Let's Encrypt TLS on +# tcp/80 + tcp/443 and serves QUIC address discovery on udp/7842, so it cannot share a +# host with sponsord-onramp. Deploy at least two before pointing nodes' +# decdn_relay_urls at them; by default a relay admits only this inventory's nodes +# (iroh_relay_access: allowlist), so list other operators' node IDs in +# iroh_relay_allowlist or use everyone. Its knobs are in +# group_vars/iroh_relay_hosts.yml (the iroh_relay role's README). +# iroh_relay_hosts: +# hosts: +# relay-1: +# ansible_host: REPLACE_WITH_RELAY_VPS_IP_OR_DNS +# iroh_relay_hostname: relay1.example.org +# +# OPTIONAL: a self-hosted iroh DNS server, the pkarr relay nodes publish their +# address records to and the DNS server peers resolve them through (in place of +# n0's dns.iroh.link) — provisioned by the iroh_dns_server.yml playbook and by +# publisher.yml. Public by design: it terminates its own Let's Encrypt TLS on tcp/443 +# and answers DNS on udp+tcp/53, so it cannot share a host with an iroh relay or +# sponsord-onramp. The parent zone must delegate its hostname to it (NS + glue). Its +# knobs are in group_vars/iroh_dns_server_hosts.yml (the iroh_dns_server role's +# README). +# iroh_dns_server_hosts: +# hosts: +# dns-1: +# ansible_host: REPLACE_WITH_DNS_VPS_IP_OR_DNS +# iroh_dns_server_hostname: dns.example.org diff --git a/tests/scripts-test.sh b/tests/scripts-test.sh index 5f0c5a6..148d9c6 100755 --- a/tests/scripts-test.sh +++ b/tests/scripts-test.sh @@ -4,7 +4,8 @@ # and locks (scripts/molecule.sh), the release gate and scripts/release.sh, the # lint-compose and lint-cloud-init invariants (negative cases), compose/decdn-compose's # and scripts/bump_decdn.py's unit tests, the firewall holes -# playbooks/group_vars/ derives per host, ci.yml's helm path filter, and — with +# playbooks/group_vars/ derives per host, ci.yml's helm path filter, +# examples/collection-fleet's inventory copy, and — with # UPSTREAM= — the upstream-mirror generators' exit codes. # `make test-scripts` runs it; CI's `scripts` job does too. Needs make, docker # (compose v2), jq, flock, gpg; the cloud-init cases also need cloud-init, shellcheck and yq, @@ -342,6 +343,95 @@ else skipped+=("node lifecycle playbook check (needs yq)") fi +# --- examples/collection-fleet's inventory ------------------------------------------- +# The fleet example ships its own copy of ansible/inventory/'s templates, so a Galaxy +# user copies one directory. fleet_inventory_check : the copy tracks exactly the +# same files, byte for byte but for the two hosts examples, whose header (every line up +# to the first blank one, comments only: make targets and a git-ignored hosts.yml in +# ansible/inventory/, ansible-playbook and a committed one in the fleet) is the one +# place they differ. Tracked files only, so a local secret.yml never counts. Each git +# runs on its own line, so a failing one fails the check instead of comparing two +# empty lists. Then the ignore rules: a secret.* (the fleet's .gitignore) or a filled-in +# hosts.yml (the root one: this repo is public) never reaches git, and the +# secret.*.example templates still ship. +fleet_inventory_check() { + local root="$1" inv=ansible/inventory fleet=examples/collection-fleet/inventory + local mine theirs f side hdr p rc n=0 + local -A body + mine="$(git -C "$root/$inv" ls-files)" || fail "git ls-files failed in $inv/ (not a git checkout?)" + theirs="$(git -C "$root/$fleet" ls-files)" || fail "git ls-files failed in $fleet/ (not a git checkout?)" + { grep -qx group_vars/all.yml <<<"$mine" && grep -qx hosts-node.yml.example <<<"$mine"; } \ + || fail "$inv/: git ls-files lists none of the templates: ${mine:-nothing}" + [[ $mine == "$theirs" ]] \ + || { diff <(echo "$mine") <(echo "$theirs") >&2 || true; fail "$fleet/ and $inv/ track different files: change both"; } + while IFS= read -r f; do + case $f in + hosts-*.yml.example) + for side in "$inv" "$fleet"; do + hdr="$(sed -n '1,/^$/p' "$root/$side/$f")" || fail "cannot read $side/$f" + ! grep -qvE '^(---|#.*)?$' <<<"$hdr" \ + || fail "$side/$f: its header (up to the first blank line) holds more than comments" + body[$side]="$(sed '1,/^$/d' "$root/$side/$f")" + [[ -n ${body[$side]} ]] || fail "$side/$f: nothing below its header" + done + [[ ${body[$inv]} == "${body[$fleet]}" ]] || fail "$fleet/$f: below its header, it differs from $inv/$f: change both" + ;; + *) + cmp "$root/$inv/$f" "$root/$fleet/$f" >/dev/null || fail "$fleet/$f differs from $inv/$f (or is missing): change both" + ;; + esac + n=$((n + 1)) + done <<<"$mine" + for p in host_vars/h/secret.yml host_vars/h/secret.yaml host_vars/h/secret.json group_vars/g/secret.yml hosts.yml hosts.yaml; do + git -C "$root" check-ignore -q --no-index "$fleet/$p" || fail "$fleet/$p is not git-ignored" + done + rc=0; git -C "$root" check-ignore -q --no-index "$fleet/host_vars/h/secret.yml.example" || rc=$? + [[ $rc == 1 ]] || fail "$fleet/host_vars/h/secret.yml.example is git-ignored (or git failed, exit $rc): the template would not ship" + pass "examples/collection-fleet/inventory/ is ansible/inventory/ but for the hosts examples' headers ($n files), and keeps secrets and hosts.yml out of git" +} +fleet_inventory_check "$repo" +# Negative cases, on a git copy of the files the check reads, rebuilt for each. +fleet_fixture() { + local fx="$work/fleet" f + rm -rf "$fx" && mkdir "$fx" + while IFS= read -r f; do + mkdir -p "$fx/$(dirname "$f")" && cp "$repo/$f" "$fx/$f" + done < <(git -C "$repo" ls-files .gitignore ansible/inventory examples/collection-fleet) + [[ -f $fx/ansible/inventory/group_vars/all.yml ]] || fail "could not copy the inventory templates into the fleet fixture" + git -C "$fx" init -q && git -C "$fx" add -A +} +fleet_check_fixture() { (fleet_inventory_check "$work/fleet"); } +# fleet_refuses : the check fails on the fixture, saying why. +fleet_refuses() { + expect 1 "fleet inventory check refuses $1" fleet_check_fixture + grep -qF -- "$2" "$work/out" || { cat "$work/out" >&2; fail "fleet inventory check: $1 failed without \"$2\""; } +} +fleet_fixture +expect 0 "fleet inventory check passes on its fixture" fleet_check_fixture +fleet_inv="$work/fleet/examples/collection-fleet/inventory" fleet_src="$work/fleet/ansible/inventory" +fleet_fixture; echo "# drift" >>"$fleet_inv/group_vars/all.yml" +fleet_refuses "an edited group_vars file" "group_vars/all.yml differs" +fleet_fixture; echo "x: 1" >"$fleet_inv/notes.yml"; git -C "$work/fleet" add -A +fleet_refuses "a file tracked on one side only" "track different files" +fleet_fixture; sed -i '1a all: {vars: {ansible_user: root}}' "$fleet_inv/hosts-node.yml.example" +fleet_refuses "YAML in a hosts example's header" "holds more than comments" +fleet_fixture; sed -i '0,/^$/{/^$/d}' "$fleet_src/hosts-publisher.yml.example" "$fleet_inv/hosts-publisher.yml.example" +fleet_refuses "hosts examples with no blank line after the header" "holds more than comments" +fleet_fixture; sed -i 's/origin-2:/origin-3:/' "$fleet_inv/hosts-publisher.yml.example" +fleet_refuses "a hosts example's body edited on one side" "below its header, it differs" +fleet_fixture; echo "# fleet only" >>"$fleet_inv/hosts-node.yml.example"; sed -i '2s/^# NODE OPERATORS/# NODE OPERATORS!/' "$fleet_inv/hosts-node.yml.example" +fleet_refuses "a body edit beside a header edit" "below its header, it differs" +fleet_fixture; rm -rf "$work/fleet/.git" +fleet_refuses "a tree that is not a git checkout" "git ls-files failed" +fleet_fixture; sed -i '/^!inventory\/host_vars/d' "$work/fleet/examples/collection-fleet/.gitignore" +fleet_refuses "a .gitignore that ignores the secret.yml.example templates" "the template would not ship" +fleet_fixture; sed -i '/^inventory\/host_vars\/\*\/secret\.\*$/d' "$work/fleet/examples/collection-fleet/.gitignore" +fleet_refuses "a .gitignore that lets a secret.yml through" "host_vars/h/secret.yml is not git-ignored" +fleet_fixture; sed -i '/collection-fleet\/inventory\/hosts/d' "$work/fleet/.gitignore" +fleet_refuses "a filled-in hosts.yml git would track" "hosts.yml is not git-ignored" +fleet_fixture; echo "# header only" | cat - "$fleet_inv/hosts-node.yml.example" >"$work/h" && mv "$work/h" "$fleet_inv/hosts-node.yml.example" +expect 0 "fleet inventory check allows a header-only edit" fleet_check_fixture + # --- release gate ---------------------------------------------------------------------- gate="$repo/scripts/check-release-version.sh" expect 2 "release gate rejects a stray argument" "$gate" decdn-node-0.1.0 notes.md @@ -1806,6 +1896,29 @@ fi if command -v ansible >/dev/null; then expect 0 "firewall holes resolve per host shape, co-located included" \ "$repo/ansible/tests/firewall-holes/check.sh" + # Both copies of the inventory templates parse, each hosts example with the + # group_vars/ and host_vars/ beside it, to the same inventory with its host in it: + # the fleet check above compares text, this is what Ansible makes of it. Tracked + # files only (a local secret.yml would change the result), from a directory without + # ansible.cfg. + for ex in hosts-node:decdn-node-1 hosts-publisher:origin-1; do + host="${ex#*:}" ex="${ex%%:*}" + for side in ansible/inventory examples/collection-fleet/inventory; do + d="$work/parse/$side" + rm -rf "$d" && mkdir -p "$d" + while IFS= read -r f; do + mkdir -p "$d/$(dirname "$f")" && cp "$repo/$side/$f" "$d/$f" + done < <(git -C "$repo/$side" ls-files group_vars host_vars) + cp "$repo/$side/$ex.yml.example" "$d/hosts.yml" + (cd "$work" && ANSIBLE_INVENTORY_UNPARSED_FAILED=True ansible-inventory -i "$d/hosts.yml" --list) \ + >"$d/list.json" 2>"$work/out" || { cat "$work/out" >&2; fail "$side/$ex.yml.example does not parse"; } + jq -e --arg h "$host" '._meta.hostvars[$h].decdn_region' "$d/list.json" >/dev/null \ + || fail "$side/$ex.yml.example: $host has no decdn_region from its host_vars" + done + [[ "$(jq -S . "$work/parse/ansible/inventory/list.json")" == "$(jq -S . "$work/parse/examples/collection-fleet/inventory/list.json")" ]] \ + || fail "$ex.yml.example: the fleet copy parses to a different inventory than ansible/inventory/'s" + done + pass "both copies of hosts-node/hosts-publisher.yml.example parse, to the same inventory" # playbooks/sponsord.yml refuses an onramp host outside sponsord_hosts. Its guard # play is lifted out with yq and run alone (running the whole playbook would need # the roles' collections just to parse), from a directory without ansible.cfg,