Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -131,8 +131,8 @@ jobs:
name: kics-results
path: kics-results/
if-no-files-found: ignore
# If GitHub Advanced Security is enabled on this private repo, surface KICS
# findings in the Security tab by un-commenting the block below (add
# To surface KICS findings in the repo's Security tab, enable GitHub code
# scanning (free on public repos) and un-comment the block below (add
# `security-events: write` to this job's permissions):
# - name: Upload SARIF to code scanning
# if: always()
Expand Down
39 changes: 39 additions & 0 deletions .github/workflows/molecule.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
name: Molecule

# Containerised converge + idempotence + verify for the decdn_node role.
# Heavy (privileged systemd Docker container) — scoped to ansible/ changes and
# blocking. Mark it a required status check in branch protection once proven.
on:
pull_request:
paths: ['ansible/**']
push:
branches: [main]
paths: ['ansible/**']

permissions:
contents: read

concurrency:
group: molecule-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
molecule:
runs-on: ubuntu-latest # Docker is preinstalled
defaults:
run:
working-directory: ansible
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
with:
python-version: '3.12'
- name: Install molecule + Ansible
run: |
python -m pip install --upgrade \
molecule "molecule-plugins[docker]" ansible ansible-lint docker
- name: Install Galaxy collections
Comment thread
thiras marked this conversation as resolved.
run: make deps
- name: molecule test
run: molecule test
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,7 @@ kics-results/
ansible/build/
*.tar.gz
ansible/importer_result.json

# Python bytecode (e.g. from the molecule stub daemon or any local tooling)
__pycache__/
*.pyc
37 changes: 13 additions & 24 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,16 @@
# AGENTS.md — decdn-devops

Guidance for AI coding agents (Claude Code, Codex, Cursor, …) working in the deCDN
DevOps monorepo.
DevOps repo.

## What this repo is

This is the deCDN team's **DevOps monorepo**: infrastructure, deployment, and
operational tooling. It is one of the independent git repositories inside the
`/home/thiras/dev/decdn/` workspace (see the workspace-level `CLAUDE.md`). Today the
whole repo is **Ansible-driven**: `ansible/` is the deployment project. `services/` is a
reserved convention for future imperative bash + systemd units and is currently empty.
The official Ansible project for deploying a **deCDN node**: infrastructure, deployment,
and operational tooling. The whole repo is **Ansible-driven** — `ansible/` is the
deployment project.

This repo is **infrastructure only**. It is *not* a source of truth for protocol or
economic claims — those live in `decdn/adr/`. If something here states a protocol fact
economic claims — those trace to the deCDN ADRs. If something here states a protocol fact
(chain-id, token address, fee split), it must trace back to an ADR, not invent one.

## Hard rules
Expand All @@ -28,9 +26,9 @@ economic claims — those live in `decdn/adr/`. If something here states a proto
node's QUIC udp/4433) via `baseline_extra_inbound`; if a service ever needs an HTTP-facing
public path, front it with an explicit reverse proxy that terminates auth + TLS. Never
bind a *backend* to `0.0.0.0` or expose its raw port.
3. **`etc/` mirrors the target filesystem** (the `services/` convention, currently unused —
see Layout). Put a config where it installs:
`services/<svc>/etc/systemd/system/foo.service` → `/etc/systemd/system/foo.service`.
3. **Role templates render to their target paths.** Ansible roles template config directly
onto the host (e.g. `roles/decdn_node/templates/decdn-node.service.j2` →
`/etc/systemd/system/`), with secrets generated on the host at `0600`.
4. **Scripts are idempotent and fail loud.** `set -euo pipefail`, re-runnable, refuse to
overwrite existing secrets, and require typed confirmation before destructive ops.
5. **Show before installing.** When building or changing infra, present the files; the
Expand All @@ -39,34 +37,22 @@ economic claims — those live in `decdn/adr/`. If something here states a proto

## Layout

The repo's one active unit is the Ansible project; `services/` is a reserved convention:

```
ansible/ # the deployment project (DevSec-hardened, lean roles)
playbooks/ # site.yml (decdn node)
roles/ # baseline, decdn_node
inventory/ galaxy/ # see ansible/README.md

services/<name>/ # reserved: future imperative bash + systemd units (currently empty)
README.md bin/ etc/ (filesystem-mirrored) contracts/
inventory/ galaxy/ molecule/ # see ansible/README.md
```

For Ansible-managed services the `etc/`-mirror convention (hard-rule #3) does not apply —
role **templates** render to their target paths instead. Hard-rules #1 (no committed
secrets; generated on host) and #2 (localhost-only by default) hold for both conventions.

## Current services

- **`ansible/`** — the team's declarative deployment project. **The public deCDN node**
- **`ansible/`** — the declarative deployment project. **The public deCDN node**
(`playbooks/site.yml` → baseline + `decdn-node`), installed from a pinned GitHub release
tarball under a hardened systemd unit; public QUIC udp/4433, loopback metrics/admin,
operator-provisioned eth keystore, required chain knobs (no baked protocol facts — sourced
from ADRs), over a shared DevSec-hardened `baseline`. See `ansible/README.md`. (On-chain
node stake/registration, ADR 019 Phase 2, is an operator step, not automated.)

`services/` remains the documented convention for any future imperative bash + systemd
unit, but currently holds none.

## Commands

Two Makefiles: the **root** is the hygiene/security/CI mirror; **`ansible/`** drives
Expand All @@ -78,6 +64,7 @@ make hooks # one-time: install pre-commit git hook (pip install pre-c
make lint # all pre-commit hooks on all files (hygiene, shellcheck, yamllint, markdown)
make lint-ansible # vendor collections + full ansible-lint (production profile)
make security # KICS IaC scan of ansible/ (pinned engine image)
make molecule # containerised converge/verify of the decdn_node role (needs Docker)

# Ansible deploys — run from ansible/ (see ansible/README.md for the full flow)
cd ansible
Expand All @@ -97,3 +84,5 @@ collection tree by `galaxy/build.sh` — there is **no** `galaxy.yml` at the `an
`make hooks`/`make lint` on your machine, **not** in CI. CI (`.github/workflows/`) is the
blocking gate and runs `ansible-lint` + KICS + `galaxy-build` (on `ansible/**`) + `actionlint`. `ansible-lint`
is **not** a per-commit hook (it needs collections vendored) — run `make lint-ansible`.
A separate `molecule.yml` workflow runs the containerised converge/verify in CI too, so
`make molecule` is not purely local.
5 changes: 4 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Two checks run on every change: **pre-commit** locally and **GitHub Actions** on
push / PR. The repo's #1 rule still stands — **never commit secrets** (see
[`CLAUDE.md`](CLAUDE.md)); secrets are generated on the target host and the repo
[`AGENTS.md`](AGENTS.md)); secrets are generated on the target host and the repo
ships `*.example` templates for them (never the real thing; non-secret config is
committed directly). There is no dedicated secret-scanner in the
pipeline — keep secrets out by design (and rely on GitHub's push protection).
Expand Down Expand Up @@ -33,6 +33,9 @@ Run it on demand with `make lint-ansible`, or `pre-commit run ansible-lint --hoo
- **`ci.yml`** — `ansible-lint` + `galaxy-build` + `kics` (on `ansible/**`) and
`actionlint`. Bash-only PRs skip the Ansible jobs. Hygiene/shellcheck/markdownlint
run via **pre-commit locally only** (`make hooks` / `make lint`), not in CI.
- **`molecule.yml`** — containerised converge + idempotence + verify of the `decdn_node`
role (privileged systemd Docker container; scoped to `ansible/**`). Run locally with
`make molecule` (needs Docker).

## Supply-chain / pinning rules

Expand Down
5 changes: 4 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Convenience targets for the deCDN DevOps monorepo.
# Run from the repo root. Ansible-specific work is delegated to ansible/Makefile.
.PHONY: help hooks lint lint-ansible security galaxy-build galaxy-check
.PHONY: help hooks lint lint-ansible security molecule galaxy-build galaxy-check
SHELL := /bin/bash

# Local KICS runs use the engine image pinned by digest. CI runs the official
Expand Down Expand Up @@ -32,6 +32,9 @@ security: ## KICS IaC security scan of ansible/ (CI runs the official
--report-formats json --output-path /repo/kics-results \
--no-progress --fail-on high

molecule: ## containerised converge/verify of the decdn_node role (needs Docker)
$(MAKE) -C ansible molecule

galaxy-build: ## stage + build the decdn.node Galaxy collection artifact
$(MAKE) -C ansible build

Expand Down
28 changes: 15 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,13 @@
[![shellcheck](https://img.shields.io/badge/shellcheck-passing-brightgreen)](https://www.shellcheck.net/)
[![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://www.conventionalcommits.org)

The deCDN team's **DevOps monorepo** — infrastructure, deployment, and operational
tooling for the deCDN project, driven by a single declarative [Ansible](ansible/README.md)
project.
The official **DevOps repo** for deploying a deCDN node — infrastructure, deployment, and
operational tooling, driven by a single declarative [Ansible](ansible/README.md) project.

This repo is **infrastructure only**. It is *not* a source of truth for protocol or
economic facts (chain-id, token addresses, fee splits) — those live in `decdn/adr/` (see
the workspace `CLAUDE.md`). Anything here that states a protocol fact traces back to an
ADR; nothing is invented in this repo.
economic facts (chain-id, token addresses, fee splits) — those trace to the deCDN ADRs.
Anything here that states a protocol fact traces back to an ADR; nothing is invented in
this repo.

## What it deploys

Expand All @@ -42,10 +41,10 @@ automated here — the node serves paid traffic only after it is staked and regi

| Path | What it is |
|------|------------|
| [`ansible/`](ansible/README.md) | The team's **declarative deployment project** — `inventory/`, `playbooks/`, `roles/` (baseline, decdn_node). The whole deploy surface lives here. |
| [`ansible/`](ansible/README.md) | The **declarative deployment project** — `inventory/`, `playbooks/`, `roles/` (baseline, decdn_node). The whole deploy surface lives here. |
| `Makefile` | Root hygiene/security/CI mirror — runs the same lint + IaC scans CI does. |
| `ansible/Makefile` | The deploy driver — `make deps/check/deploy`. |
| `.github/workflows/` | The blocking CI gate (`ansible-lint` + KICS + `galaxy-build` + `actionlint`). |
| `.github/workflows/` | The blocking CI gate (`ansible-lint` + KICS + `galaxy-build` + `molecule` + `actionlint`). |

## Quickstart

Expand Down Expand Up @@ -101,7 +100,8 @@ make check / deploy # deCDN node (site.yml): dry-run / provision

**Gotcha — pre-commit is local-only.** Hygiene/shellcheck/yamllint/markdown run via
`make hooks`/`make lint` on your machine, **not** in CI. The blocking gate is
`.github/workflows/` (`ansible-lint` + KICS on `ansible/**` + `actionlint`). `ansible-lint`
`.github/workflows/` (`ansible-lint` + KICS + `galaxy-build` + `molecule` on `ansible/**`,
plus `actionlint`). `ansible-lint`
is not a per-commit hook (it needs collections vendored) — run `make lint-ansible`.

## CI & quality gates
Expand All @@ -111,12 +111,14 @@ is not a per-commit hook (it needs collections vendored) — run `make lint-ansi
collection and runs galaxy-importer's checks), **KICS** IaC scan (fail on HIGH), and
`actionlint` on the workflows themselves. The KICS engine is pinned by digest and every
third-party action by full commit SHA (a re-pointed tag can ship malicious code).
- **`molecule.yml`** — a containerised converge + idempotence + verify of the `decdn_node`
role in a privileged systemd Docker container (scoped to `ansible/**` changes).

## Conventions & source of truth

- **ADRs are the only source of truth for protocol facts.** They live in `decdn/adr/`
(a separate repo in the workspace) — e.g. payments (ADR 003), node onboarding
(ADR 019), tokenomics (ADR 026). If a doc here contradicts an ADR, fix the doc.
- **ADRs are the only source of truth for protocol facts.** The deCDN ADRs cover, e.g.,
payments (ADR 003), node onboarding (ADR 019), and tokenomics (ADR 026). If a doc here
contradicts an ADR, fix the doc.
- **Role templates render to their target paths.** Ansible roles template config directly
onto the host (e.g. `roles/decdn_node/templates/decdn-node.service.j2` →
`/etc/systemd/system/`), with secrets generated on the host at `0600`.
Expand All @@ -126,4 +128,4 @@ is not a per-commit hook (it needs collections vendored) — run `make lint-ansi
- [`ansible/README.md`](ansible/README.md) — full setup, security model, and deploy steps
- [`ansible/roles/decdn_node/README.md`](ansible/roles/decdn_node/README.md) — the deCDN node role
- [`CONTRIBUTING.md`](CONTRIBUTING.md) — the local + CI check workflow
- workspace `CLAUDE.md` — repo hard rules and the single-source-of-truth policy
- [`AGENTS.md`](AGENTS.md) — repo hard rules and conventions (for humans and AI agents)
8 changes: 7 additions & 1 deletion ansible/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Convenience targets for the deCDN Ansible project.
# Always run from the ansible/ directory.
.PHONY: deps lint check deploy build galaxy-check
.PHONY: deps lint check deploy molecule build galaxy-check
SHELL := /bin/bash

# Install the required Galaxy collections (>= constraints in requirements.yml)
Expand All @@ -22,6 +22,12 @@ check:
deploy:
ansible-playbook playbooks/site.yml

# --- Tests -------------------------------------------------------------------
# Containerised converge + idempotence + verify of the decdn_node role against a
# stub daemon (needs Docker; a privileged systemd container). See molecule/.
molecule:
molecule test

# --- Galaxy collection (decdn.node) ------------------------------------------
# Stage baseline + decdn_node into a clean collection tree and build the artifact
# under build/. Only those two roles ship; see galaxy/README.md. Publishing stays
Expand Down
37 changes: 25 additions & 12 deletions ansible/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ansible — deCDN deployment

Declarative Ansible project for the deCDN team. One deployment over a shared host
Declarative Ansible project for deploying a deCDN node. One deployment over a shared host
baseline:

| Playbook | Purpose | Exposure |
Expand Down Expand Up @@ -56,7 +56,7 @@ it local.

By default baseline **deploys you as yourself**: an empty `ssh_admin_user` resolves to your
control-machine `$USER`, and an empty `ssh_admin_pubkey` is autodetected from `~/.ssh`
(`id_ed25519` > `ecdsa` > `rsa`). Add teammates' keys via `ssh_admin_extra_pubkeys`. Set
(`id_ed25519` > `ecdsa` > `rsa`). Add additional operators' keys via `ssh_admin_extra_pubkeys`. Set
`ssh_admin_user`/`ssh_admin_pubkey` explicitly to override (e.g. a shared `deploy` account,
or when deploying from CI). baseline **asserts a key resolves** before `ssh_hardening`
disables root + password login, so you can't lock yourself out. After the first deploy,
Expand All @@ -69,16 +69,22 @@ one).

**Prerequisites** (see `roles/decdn_node/README.md` for the full flow):

1. A published **`v<version>` release** exists (the role downloads the release tarball).
1. A published **`v<version>` release** exists and is **publicly downloadable** — the role
fetches the release tarball from `decdn_node_release_base`
(default `https://github.com/decdn/decdn/releases/download`). To install from a mirror,
override `decdn_node_release_base`; to deploy a locally-built binary with no release at
all, use the **`manual`** install method (`decdn_node_install_method: manual` +
`decdn_node_manual_bin_src` / `decdn_cli_manual_bin_src`).
2. Per-node config in `inventory/host_vars/<node>/main.yml` (committed) —
`decdn_node_version`, the three contract addresses, `decdn_region`, cache origin, … — plus
the one secret, `decdn_rpc_url`, in a sibling git-ignored `secret.yml` (copy the shipped
`host_vars/decdn-node-1/secret.yml.example`). The committed `main.yml` already carries the
Arbitrum Sepolia genesis contract addresses; edit `decdn_region` + cache origin for your
node. Contract addresses are protocol facts — source them from the deployment / an ADR,
never guess.
node. Contract addresses are protocol facts — source them from the deCDN contract
deployment / an ADR, never guess.
3. The **eth keystore + password file** provisioned on the host (operator step — the wallet
must be funded + staked per `decdn/adr/019`). Generate with, as the `decdn` user:
must be funded + staked per the deCDN node-onboarding ADR, 019). Generate with, as the
`decdn` user:
`decdn key-gen --output-dir /var/lib/decdn --password-file /etc/decdn/keystore.password`.

```bash
Expand Down Expand Up @@ -107,13 +113,20 @@ with no turnkey CLI yet (see `roles/decdn_node/README.md`).
```bash
make lint # yamllint + ansible-lint (production profile)
ansible-playbook playbooks/site.yml --syntax-check
make molecule # containerised converge + idempotence + verify (needs Docker)
```

The **node** role is verified statically pre-release (syntax-check, `systemd-analyze verify`
on the rendered unit, TOML validity, fail-loud asserts); a live deploy follows once a
`v<version>` release is published (no real chain runs in CI). `baseline` is not exercised in
a container (its `ssh_hardening` would sever the connection); `make check` covers it as a
non-mutating dry run.
`make molecule` converges the **`decdn_node`** role in a privileged systemd container
against a stub daemon (`molecule/default/`): it installs via the `manual` method (no
published release needed), stages a placeholder keystore, renders `node.toml` + the
hardened unit, starts the service, and passes the role's own `/metrics` readiness probe;
`verify.yml` then asserts the node user, valid TOML, a valid systemd unit, loopback-only
metrics binding, and the `0600` secret env file. It does **not** exercise real node logic
or a live chain — full paid-traffic readiness still needs on-chain registration and a real
release. `baseline` is not exercised in a container — the scenario connects over Docker
(not SSH), and baseline's host-level hardening (nftables default-deny, DevSec os/ssh
hardening, fail2ban) isn't meaningful in a throwaway container; `make check` covers it as
a non-mutating dry run.

## Configuration

Expand All @@ -124,7 +137,7 @@ the RPC URL). Highlights:
| Var | Default | Notes |
|-----|---------|-------|
| `ssh_admin_user` / `ssh_admin_pubkey` | `""` / `""` | Empty = local `$USER` + autodetected `~/.ssh` key; admin created before SSH hardening. |
| `ssh_admin_extra_pubkeys` | `[]` | Extra authorized keys for the admin user (teammates). |
| `ssh_admin_extra_pubkeys` | `[]` | Extra authorized keys for the admin user (additional operators). |
| `baseline_extra_inbound` | `[]` | public inbound ports; `decdn_nodes` opens udp/4433. |
| `decdn_node_version` | `""` | **required**; a `v<version>` release must exist. |
| `decdn_rpc_url` + 3 contract addresses | `""` | **required** per node — `rpc_url` in `host_vars/<node>/secret.yml`, addresses in `main.yml`; sourced from an ADR/deployment. |
Expand Down
Loading
Loading