Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ jobs:
- name: Lint (yamllint + ansible-lint)
run: make lint
- name: Syntax-check playbooks
# Dummy inventory: the real inventory/hosts.yml is git-ignored, and
# --syntax-check only parses, it never connects.
# Dummy inventory: no real hosts.yml is committed upstream (only hosts.yml.example),
# and --syntax-check only parses, it never connects.
run: |
for p in playbooks/site.yml playbooks/anvil.yml playbooks/add-dev-user.yml; do
echo "::group::syntax-check $p"
Expand Down
5 changes: 3 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# ─────────────────────────────────────────────────────────────────────────────
# Secrets & generated state — NEVER commit these.
# The repo ships only *.example templates; the real files are generated on the
# VPS by bin/bootstrap.sh / bin/install.sh and live under /etc and /var/lib.
# The repo ships *.example templates for secret files; the real secrets are generated
# on the host by the Ansible role tasks and live under /etc and /var/lib. Non-secret
# config (e.g. ansible host_vars/<node>/main.yml) is committed directly.
# These patterns are defense-in-depth in case anyone copies generated files in.
# ─────────────────────────────────────────────────────────────────────────────

Expand Down
5 changes: 3 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,9 @@ economic claims — those live in `decdn/adr/`. If something here states a proto
1. **Never commit secrets.** No mnemonics, passwords, bcrypt hashes, private keys, API
tokens, or tunnel credentials in any tracked file. Secrets are *generated on the
target host* by the role tasks (e.g. basic-auth creds) and stored under `/etc/<svc>/` with `chmod 600`
and a dedicated owner. The repo ships `*.example` templates only. The root
`.gitignore` is a backstop — do not rely on it; keep secrets out by design.
and a dedicated owner. The repo ships `*.example` templates for secret files only
(non-secret config may be committed directly). The root `.gitignore` is a backstop —
do not rely on it; keep secrets out by design.
2. **Localhost-only by default.** Service daemons (anvil, etc.) bind `127.0.0.1`. The
*only* sanctioned public path is an explicit reverse proxy with auth + TLS in front
(e.g. the anvil devnet's public-HTTPS Caddy, or that same proxy behind an outbound
Expand Down
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@
Two checks run on every change: **pre-commit** locally and **GitHub Actions** on
push / PR. The repo's #1 rule still stands — **never commit secrets** (see
[`CLAUDE.md`](CLAUDE.md)); secrets are generated on the target host and the repo
ships `*.example` templates only. There is no dedicated secret-scanner in the
ships `*.example` templates for them (never the real thing; non-secret config is
committed directly). There is no dedicated secret-scanner in the
pipeline — keep secrets out by design (and rely on GitHub's push protection).

## One-time setup
Expand Down
14 changes: 9 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,12 @@ automated here — the node serves paid traffic only after it is staked and regi

```bash
cd ansible
make deps # vendor pinned Galaxy collections into ./collections
make deps # vendor pinned Galaxy collections into ./collections
cp inventory/hosts.yml.example inventory/hosts.yml
$EDITOR inventory/hosts.yml # set hosts for decdn_nodes and/or anvil_devnet
$EDITOR inventory/group_vars/all.yml # set ssh_admin_pubkey (REQUIRED — prevents lockout)
$EDITOR inventory/hosts.yml # set hosts for decdn_nodes and/or anvil_devnet
$EDITOR inventory/group_vars/all.yml # set ssh_admin_pubkey (REQUIRED — prevents lockout)
cp inventory/host_vars/decdn-node-1/secret.yml.example inventory/host_vars/decdn-node-1/secret.yml
$EDITOR inventory/host_vars/decdn-node-1/secret.yml # set decdn_rpc_url (per-node config is in main.yml)
make check # dry run (--check --diff)
make deploy # provision the deCDN node
```
Expand All @@ -71,8 +73,10 @@ and the anvil devnet flow.

- **Nothing secret is committed.** Mnemonics, basic-auth credentials, and eth keystores
are **generated on — or operator-provisioned to — the target host**, never the repo.
Ansible roles render them on the host (`no_log`, `0600`); the repo ships `*.example`
templates only, with the root `.gitignore` as a backstop.
Ansible roles render them on the host (`no_log`, `0600`); the repo ships a `*.example`
template for the one node secret (`rpc_url`) plus a `hosts.yml.example` starter, and commits
non-secret per-node config directly (`host_vars/<node>/main.yml`), with the root `.gitignore`
as a backstop.
- **Localhost-only by default.** Backends bind `127.0.0.1`; the only sanctioned public
path is an explicit reverse proxy / tunnel with auth in front. The node host opens one
extra hole (udp/4433 QUIC); everything else (anvil 8545, caddy 8080, node metrics 9090,
Expand Down
13 changes: 7 additions & 6 deletions ansible/.gitignore
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Galaxy collections vendored by `make deps`
collections/

# Real inventory (only the .example is committed — never a real host/IP)
inventory/hosts.yml

# Per-node deployment values incl. the sensitive rpc_url (only .example committed)
inventory/host_vars/*.yml
!inventory/host_vars/*.yml.example
# Per-node RPC secret (may embed an API key). Ignore secret.* under any host dir so a
# copy named secret.yaml or secret.json (Ansible loads .yml/.yaml/.json all the same)
# can't leak; the committed secret.*.example template is re-included. Everything else in
# host_vars — main.yml (contract addresses, region, cache origin, …) — is committable
# template config. hosts.yml is committable too (your fork's inventory is yours to track).
inventory/host_vars/*/secret.*
!inventory/host_vars/*/secret.*.example

# Ansible / molecule scratch
*.retry
Expand Down
25 changes: 18 additions & 7 deletions ansible/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ baseline host hardening — DevSec os/ssh, nftables default-deny inbound,
- **No secrets in the repo.** anvil's mnemonic + caddy basic-auth are **generated on the
host** (stat-guarded, `no_log`, revealed once). The node's eth keystore is
**operator-provisioned** and never generated here; its `rpc_url` (which may embed an API
key) lives in git-ignored `host_vars` and is rendered to a `0600` config.
key) lives in a git-ignored `host_vars/<node>/secret.yml` (the rest of `host_vars` is
committed, non-secret config) and is rendered to a `0600` config.
- **Host hardening via DevSec** (`os_hardening` + `ssh_hardening`): key-only SSH, no root
login, kernel/sysctl/PAM hardening — applied last, after the admin key is in place.

Expand All @@ -47,8 +48,14 @@ make deps # vendor pinned collections into ./
cp inventory/hosts.yml.example inventory/hosts.yml
$EDITOR inventory/hosts.yml # set hosts for decdn_nodes and/or anvil_devnet
$EDITOR inventory/group_vars/all.yml # optional: override admin user/keys, allowlists
# Per-node RPC secret (the rest of host_vars/<node>/main.yml is committed config):
cp inventory/host_vars/decdn-node-1/secret.yml.example inventory/host_vars/decdn-node-1/secret.yml
$EDITOR inventory/host_vars/decdn-node-1/secret.yml # set decdn_rpc_url
```

Your `hosts.yml` is no longer force-ignored — commit it in your fork if you want, or keep
it local.

By default baseline **deploys you as yourself**: an empty `ssh_admin_user` resolves to your
control-machine `$USER`, and an empty `ssh_admin_pubkey` is autodetected from `~/.ssh`
(`id_ed25519` > `ecdsa` > `rsa`). Add teammates' keys via `ssh_admin_extra_pubkeys`. Set
Expand All @@ -65,10 +72,13 @@ one).
**Prerequisites** (see `roles/decdn_node/README.md` for the full flow):

1. A published **`v<version>` release** exists (the role downloads the release tarball).
2. Per-node `inventory/host_vars/<node>.yml` (git-ignored) with `decdn_node_version`,
`decdn_rpc_url`, the three contract addresses, and `decdn_region`. Copy the shipped
`host_vars/decdn-node-1.yml.example`. Contract addresses are protocol facts — source
them from the deployment / an ADR, never guess.
2. Per-node config in `inventory/host_vars/<node>/main.yml` (committed) —
`decdn_node_version`, the three contract addresses, `decdn_region`, cache origin, … — plus
the one secret, `decdn_rpc_url`, in a sibling git-ignored `secret.yml` (copy the shipped
`host_vars/decdn-node-1/secret.yml.example`). The committed `main.yml` already carries the
Arbitrum Sepolia genesis contract addresses; edit `decdn_region` + cache origin for your
node. Contract addresses are protocol facts — source them from the deployment / an ADR,
never guess.
3. The **eth keystore + password file** provisioned on the host (operator step — the wallet
must be funded + staked per `decdn/adr/019`). Generate with, as the `decdn` user:
`decdn key-gen --output-dir /var/lib/decdn --password-file /etc/decdn/keystore.password`.
Expand Down Expand Up @@ -129,15 +139,16 @@ covers it as a non-mutating dry run.
## Configuration

Defaults live in each role (`roles/*/defaults/main.yml`); override in `group_vars`
(shared) or `host_vars` (per node, git-ignored). Highlights:
(shared) or `host_vars` (per node; `main.yml` committed config, `secret.yml` git-ignored for
the RPC URL). Highlights:

| Var | Default | Notes |
|-----|---------|-------|
| `ssh_admin_user` / `ssh_admin_pubkey` | `""` / `""` | Empty = local `$USER` + autodetected `~/.ssh` key; admin created before SSH hardening. |
| `ssh_admin_extra_pubkeys` | `[]` | Extra authorized keys for the admin user (teammates). |
| `baseline_extra_inbound` | `[]` | public inbound ports; `decdn_nodes` opens udp/4433. |
| `decdn_node_version` | `""` | **required**; a `v<version>` release must exist. |
| `decdn_rpc_url` + 3 contract addresses | `""` | **required** per node (host_vars); sourced from an ADR/deployment. |
| `decdn_rpc_url` + 3 contract addresses | `""` | **required** per node — `rpc_url` in `host_vars/<node>/secret.yml`, addresses in `main.yml`; sourced from an ADR/deployment. |
| `decdn_region` / `decdn_bind_port` / `decdn_rate_per_mb` | `""` / `4433` / `10` | node identity, QUIC port, USDC base units/MB. |
| `anvil_chain_id` … `rpc_hostname` | see `roles/anvil`,`roles/caddy` | internal devnet knobs. |
| `caddy_fail2ban` (+ `_maxretry`/`_findtime`/`_bantime`) | `true` (5 / 10m / 1h) | RPC basic-auth brute-force jail; public listener only. |
Expand Down
3 changes: 2 additions & 1 deletion ansible/inventory/group_vars/decdn_nodes.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
# Shared config for public deCDN node hosts (playbooks/site.yml).
# Per-node values (rpc_url, addresses, region, version) live in host_vars.
# Per-node values live in host_vars/<node>/: addresses/region/version in main.yml,
# the sensitive rpc_url in a git-ignored secret.yml.

# Open the node's public QUIC listener in the baseline firewall. Metrics (9090)
# and admin RPC (9191) stay loopback and get NO hole.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
# Per-node deployment values. Copy to inventory/host_vars/<node-name>.yml
# (git-ignored) and fill in. The node refuses to start until these are set.
# Per-node deployment values for decdn-node-1 (COMMITTED — non-secret config only).
# Ansible merges every *.yml in this directory for host decdn-node-1. The one
# sensitive value, decdn_rpc_url, lives in the sibling secret.yml (git-ignored) —
# copy secret.yml.example to secret.yml and fill it in. The node refuses to start
# until both this file and secret.yml are set.
#
# The contract addresses below are the canonical Arbitrum Sepolia (chain 421614)
# v0.1.0 genesis deploy, cited to decdn/contracts/deployments/421614.json
Expand All @@ -14,9 +17,7 @@ decdn_node_version: "0.1.0"
decdn_node_sha256: ""

# --- Chain (Arbitrum Sepolia, chain 421614) -----------------------------------
# MUST-EDIT — SENSITIVE (may embed an API key). The public endpoint below works
# for light use; run your own or use a provider for production reliability.
decdn_rpc_url: "https://sepolia-rollup.arbitrum.io/rpc"
# The RPC endpoint (decdn_rpc_url) is the one sensitive value — set it in secret.yml.
decdn_chain_id: 421614 # Arbitrum Sepolia
# Required contracts (deployments/421614.json).
decdn_payment_channel_address: "0xb4bcA0AbF679212708164dCAb98eFa621Fa0F4d3" # PaymentChannel
Expand Down
8 changes: 8 additions & 0 deletions ansible/inventory/host_vars/decdn-node-1/secret.yml.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# Per-node SECRET for decdn-node-1. Copy to secret.yml (same directory, git-ignored)
# and fill in. Ansible merges it with main.yml for this host. This is the ONLY
# per-node value kept out of git — everything else lives in the committed main.yml.
#
# MUST-EDIT — SENSITIVE (may embed an API key). The public endpoint below works
# for light use; run your own or use a provider for production reliability.
decdn_rpc_url: "https://sepolia-rollup.arbitrum.io/rpc"
10 changes: 6 additions & 4 deletions ansible/inventory/hosts.yml.example
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
---
# Copy to inventory/hosts.yml and fill in your hosts. The real file is git-ignored —
# no host or IP is ever committed.
# Copy to inventory/hosts.yml and fill in your hosts. This .example is the shipped
# template; your real hosts.yml is yours — commit it in your fork if you like, or keep
# it local. (Just never push real host IPs to the upstream template.)
#
# First run: connect as a user that can sudo (often root) to bootstrap the box.
# After the baseline role creates {{ ssh_admin_user }} and ssh_hardening disables
# root login, switch ansible_user to that admin account.

# PRIMARY: public deCDN node(s) — provisioned by playbooks/site.yml.
# Per-node secrets/knobs (rpc_url, contract addresses, region, version) go in
# inventory/host_vars/<name>.yml (git-ignored; see the .example there).
# Per-node config (contract addresses, region, version, …) goes in the committed
# inventory/host_vars/<name>/main.yml; the one secret, rpc_url, goes in a git-ignored
# secret.yml beside it (copy the shipped secret.yml.example there).
decdn_nodes:
hosts:
decdn-node-1:
Expand Down
5 changes: 3 additions & 2 deletions ansible/roles/decdn_node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,10 @@ Per `decdn/adr/019-node-onboarding.md`, a node only serves paid traffic after
The role **refuses to start** until the keystore + password file exist — it
never generates wallet material itself.

## Required variables (set in `host_vars/<node>.yml`)
## Required variables (set in `host_vars/<node>/`)

`decdn_node_version`, `decdn_rpc_url` (sensitive — may embed an API key),
`decdn_node_version`, `decdn_rpc_url` (sensitive — may embed an API key; goes in the
git-ignored `secret.yml`, everything else in the committed `main.yml`),
`decdn_payment_channel_address`, `decdn_capacity_bond_address`,
`decdn_slash_judge_address` (all `0x`+40-hex; SlashJudge non-zero),
`decdn_region` (ISO 3166-1 alpha-2). Contract addresses/chain-id are protocol
Expand Down
5 changes: 3 additions & 2 deletions ansible/roles/decdn_node/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
# decdn-node role defaults. Per-node deployment values (rpc_url, contract
# addresses, region, version) belong in host_vars; see host_vars/*.yml.example.
# decdn-node role defaults. Per-node deployment values belong in host_vars/<node>/:
# non-secret config (contract addresses, region, version) in the committed main.yml,
# and the sensitive rpc_url in a git-ignored secret.yml (see secret.yml.example).

# --- Release to install (REQUIRED — a v<version> GitHub Release must exist) ---
decdn_node_version: "" # e.g. "0.1.0" (asserted non-empty)
Expand Down
Loading