Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -738,10 +738,12 @@ artifact's tag (`cliff.toml`), and pushes a signed `chore(release)` commit and t
no release PR. The repo squash-merges with the PR title as the subject, so the PR title
is the changelog entry (`pr-title.yml` checks it is conventional); release.sh refuses a
commit that touched the artifact and that git-cliff cannot parse as conventional, unless
`--allow-unconventional`. Until an artifact's first release, its manifest stays at
the `0.0.0` placeholder and its changelog changes are logged by hand under
`[Unreleased]`, which becomes the first release's section; after it, do not edit the
changelogs by hand.
`--allow-unconventional`. Every artifact is released and publishing is on
(`PUBLISH_ENABLED` is `true`): `decdn.node` and `decdn.publisher` are on Galaxy, the
chart on `oci://ghcr.io/decdn/charts` and Artifact Hub. So every changelog section is
generated: never edit the changelogs by hand or add an `[Unreleased]` section (release.sh
refuses one). release.sh's `0.0.0`-placeholder path is only for a new artifact's first
release.

**CI.** `ci.yml` is the blocking gate: `pre-commit`, `scripts` and `actionlint` on every PR, the
Ansible, chart, compose and cloud-init jobs path-filtered, KICS on the first three (KICS has
Expand Down
6 changes: 3 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ the subjects into the collections' and the chart's changelogs at release (see
[RELEASING.md](RELEASING.md)). Write a title an operator can read as a changelog entry;
the `pr-title.yml` workflow refuses a title that is not conventional, with a lowercase
type, one of `feat` `fix` `perf` `refactor` `docs` `chore` `revert` `security` `ci`
`test` `style` `build`. Until an
artifact's first release, also add an entry under `[Unreleased]` in its changelog by
hand: that section becomes the first release's notes.
`test` `style` `build`. Never edit a released artifact's changelog by hand: every
section after its first is generated. Only a new artifact's first section is written by
hand, under `[Unreleased]` (see [RELEASING.md](RELEASING.md)).

## One-time setup

Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

[![CI](https://github.com/decdn/devops/actions/workflows/ci.yml/badge.svg)](https://github.com/decdn/devops/actions/workflows/ci.yml)
[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/decdn-node)](https://artifacthub.io/packages/search?repo=decdn-node)
[![decdn.node on Galaxy](https://img.shields.io/ansible/collection/v/decdn/node?label=decdn.node)](https://galaxy.ansible.com/ui/repo/published/decdn/node/)
[![decdn.publisher on Galaxy](https://img.shields.io/ansible/collection/v/decdn/publisher?label=decdn.publisher)](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/)
[![Ansible](https://img.shields.io/badge/Ansible-%E2%89%A5%202.15-1A1918?logo=ansible&logoColor=white)](https://docs.ansible.com/)
[![ansible-lint: production](https://img.shields.io/badge/ansible--lint-production-blue)](https://ansible.readthedocs.io/projects/lint/)
[![IaC scan: KICS](https://img.shields.io/badge/IaC%20scan-KICS-7B61FF)](https://kics.io/)
Expand All @@ -22,9 +24,9 @@ Four ways to deploy the node, cache or origin:
| Path | For | Start here |
|------|-----|------------|
| **Docker Compose** (`compose/`), recommended | One host that already runs Docker. The quickest start, [below](#quickstart-docker-compose). Profiles for a cache node or an origin, the onboarding sponsor (`sponsord` + its onramp), an iroh relay, an iroh DNS server and Grafana Alloy, driven by the `decdn-compose` wrapper (set-up, preflight, guarded restarts). | [`compose/README.md`](compose/README.md) |
| **Ansible** (`ansible/`) | VMs and bare metal, one node or a fleet. Hardens the host too (firewall, SSH, patching). Also published as the `decdn.node` and `decdn.publisher` Galaxy collections. | [`ansible/README.md`](ansible/README.md) |
| **Ansible** (`ansible/`) | VMs and bare metal, one node or a fleet. Hardens the host too (firewall, SSH, patching). Also published on Ansible Galaxy as the [`decdn.node`](https://galaxy.ansible.com/ui/repo/published/decdn/node/) and [`decdn.publisher`](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/) collections. | [`ansible/README.md`](ansible/README.md) |
| **cloud-init** (`cloud-init/`) | One VM, no control machine: paste the user-data into your provider's "create server" form. It runs the Ansible playbook on the host itself, hardening included. One template per kind of operator: a cache node, or a publisher's origin with the onboarding sponsor (`sponsord` + its onramp). | [`cloud-init/README.md`](cloud-init/README.md) |
| **Helm** (`charts/decdn-node/`) | Kubernetes, one release per node, cache or origin. | [`charts/decdn-node/README.md`](charts/decdn-node/README.md) |
| **Helm** (`charts/decdn-node/`) | Kubernetes, one release per node, cache or origin. Published to `oci://ghcr.io/decdn/charts/decdn-node`, signed, and listed on [Artifact Hub](https://artifacthub.io/packages/helm/decdn-node/decdn-node). | [`charts/decdn-node/README.md`](charts/decdn-node/README.md) |

The iroh relay and DNS server a publisher may run deploy with Ansible or Compose.

Expand Down
28 changes: 16 additions & 12 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ The repo publishes three artifacts, each on its own version and released by its

Release an artifact when it has changes; the others are not touched. Their versions are
independent. The one dependency is Galaxy's: `decdn.publisher` requires `decdn.node`
(its playbooks run `decdn.node`'s baseline, Alloy and origin nodes), so the publisher
collection's first release comes after the node collection's. `scripts/release.sh`
(its playbooks run `decdn.node`'s baseline, Alloy and origin nodes), so a publisher
release that raises its `decdn.node` floor comes after that node release. `scripts/release.sh`
refuses any publisher release whose `decdn.node: ">=X.Y.Z"` no `node-collection-v*`
tag on origin satisfies yet. A tag is not a Galaxy upload, so the publish job also
asks Galaxy, before uploading a `decdn.publisher`, for a `decdn.node` that satisfies
Expand All @@ -26,8 +26,8 @@ Each workflow's `build` job runs on every matching tag push. It gates the tag
(`scripts/check-release-version.sh`), re-runs that artifact's checks (`make
galaxy-check-node`, `make galaxy-check-publisher` or `make lint-helm`), packages it,
and uploads it with a `SHA256SUMS` as a workflow artifact. The `publish` job pushes it to Galaxy or ghcr.io and creates the GitHub
Release, but **only** when the repository variable `PUBLISH_ENABLED` is `true`. Until
then, a tag push is a dry run. So is a manual *Run workflow* (`workflow_dispatch`) with an
Release, but **only** when the repository variable `PUBLISH_ENABLED` is `true`, as it
is now; set it to anything else and a tag push is a dry run. So is a manual *Run workflow* (`workflow_dispatch`) with an
existing tag. Before a tag exists, the rehearsal is `scripts/release.sh`'s dry run and the
artifact's make target (below). The repo's *Latest* release is the highest `decdn.node`
version, the node operator's collection: a node collection release is marked Latest only
Expand Down Expand Up @@ -114,7 +114,7 @@ change lands in several, run the script once for each.
**Prerequisites.**

- [git-cliff](https://git-cliff.org) 2.9 or later, for `require_conventional`
(`cargo install git-cliff`; CI tests with 2.14.1). The first release does not use it.
(`cargo install git-cliff`; CI tests with 2.14.1).
- Git configured to sign, with `user.signingkey` set explicitly. The script signs a
throwaway commit object first, so a key that cannot sign fails before the checks.
- A clean `main` equal to `origin/main`, and no local tag of the artifact above
Expand All @@ -126,13 +126,13 @@ change lands in several, run the script once for each.
as the squash commit's subject (*Settings → General → Pull Requests*, and the `main`
ruleset's allowed merge methods), and `pr-title` a required status check.

**The first release** of each artifact starts from the `0.0.0` placeholder in its
manifest and names its level: `scripts/release.sh chart minor --execute` cuts `0.1.0`.
Without a tag, git-cliff would render the whole history, so the first release does not
use it. It renames the changelog's hand-written `## [Unreleased]` section to the dated
version instead, so until then, changes are logged there by hand. After that, no
`[Unreleased]` section is kept: the script refuses one, because every later section is
generated.
**A first release.** All three artifacts are past theirs. A new artifact starts from
the `0.0.0` placeholder in its manifest and names its level:
`scripts/release.sh <artifact> minor --execute` cuts `0.1.0`. Without a tag, git-cliff
would render the whole history, so the first release does not use it. It renames the
changelog's hand-written `## [Unreleased]` section to the dated version instead, so
until then, changes are logged there by hand. After that, no `[Unreleased]` section is
kept: the script refuses one, because every later section is generated.

The chart's workflow file name (`release-chart.yml`) and tag shape (`decdn-node-X.Y.Z`)
are both part of its cosign certificate identity and of its attestations' signer
Expand All @@ -141,6 +141,10 @@ either once a chart is published.

## First release only

Done for all three artifacts: the Galaxy namespace and key, the `release` environment,
`PUBLISH_ENABLED`, the public GHCR package and the Artifact Hub listing with its
`repositoryID`. Kept for a new artifact, or to rebuild the setup.

Before the first publish:

- **Galaxy namespace.** The `decdn` namespace must exist on galaxy.ansible.com and the
Expand Down
13 changes: 7 additions & 6 deletions ansible/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
# ansible — deCDN deployment

Declarative Ansible project for deploying deCDN nodes, one or a fleet, over a
hardened host baseline. The roles also ship as two Galaxy collections: `decdn.node`
(what a node operator runs) and `decdn.publisher` (what a publisher runs beside its
hardened host baseline. The roles also ship as two collections on Ansible Galaxy:
[`decdn.node`](https://galaxy.ansible.com/ui/repo/published/decdn/node/) (what a node operator runs) and
[`decdn.publisher`](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/) (what a publisher runs beside its
origin nodes).

| Playbook | Purpose | Make target |
Expand Down Expand Up @@ -513,8 +514,9 @@ the RPC URL when it is not provisioned on the host instead). Highlights:

## Packaging as Galaxy collections (`decdn.node`, `decdn.publisher`)

The seven roles are also packaged as two distributable collections, for operators who
bring their own playbooks:
The seven roles are also packaged as two collections, published on Ansible Galaxy, for
operators who bring their own playbooks (`ansible-galaxy collection install
decdn.node` or `decdn.publisher`):

| Collection | Roles | For |
|---|---|---|
Expand Down Expand Up @@ -542,5 +544,4 @@ each collection on its own version (the Helm chart is released separately), cut
`main` by `scripts/release.sh`: see [RELEASING.md](../RELEASING.md). Each
`galaxy/<collection>/CHANGELOG.md` section is generated at release from the
conventional commit subjects that touched that collection's roles, its overlay or
`galaxy/build.sh`, so give the PR a title that reads as a changelog entry. Until a
collection's first release, also log the change under its `[Unreleased]` by hand.
`galaxy/build.sh`, so give the PR a title that reads as a changelog entry.
26 changes: 15 additions & 11 deletions ansible/galaxy/node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,17 +25,7 @@ collection, which depends on this one.

## Install

> **Not on Galaxy yet.** The first release (`node-collection-v0.1.0`) has not been
> cut; see [RELEASING.md](https://github.com/decdn/devops/blob/main/RELEASING.md).
> Until then, build and install it from a checkout:
>
> ```bash
> git clone https://github.com/decdn/devops && cd devops/ansible
> make build-node
> ansible-galaxy collection install build/decdn-node-*.tar.gz
> ```

Once published:
From [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/node/):

```bash
ansible-galaxy collection install decdn.node
Expand All @@ -49,6 +39,20 @@ collections:
version: ">=0.1.0"
```

Each release is listed in the
[CHANGELOG](https://github.com/decdn/devops/blob/main/ansible/galaxy/node/CHANGELOG.md)
and as a `node-collection-vX.Y.Z`
[GitHub Release](https://github.com/decdn/devops/releases) with the tarball and its
`SHA256SUMS`. To run changes not released yet, build it from a checkout. The build
carries the last release's version until the next one, so `--force` replaces an
installed release:

```bash
git clone https://github.com/decdn/devops && cd devops/ansible
make build-node
ansible-galaxy collection install --force build/decdn-node-*.tar.gz
```

## Usage

A minimal node playbook — baseline first (so the admin key lands before SSH
Expand Down
32 changes: 17 additions & 15 deletions ansible/galaxy/publisher/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,21 +28,8 @@ repository — four roles and nothing else:

## Install

> **Not on Galaxy yet.** The first release (`publisher-collection-v0.1.0`) has not
> been cut, and it follows `decdn.node`'s; see
> [RELEASING.md](https://github.com/decdn/devops/blob/main/RELEASING.md). Until then,
> build and install both from a checkout. A checkout's builds are both `0.0.0`, below
> the `decdn.node >=0.1.0` this collection declares, so install the node collection
> first (which pulls its own dependencies) and this one without dependency resolution:
>
> ```bash
> git clone https://github.com/decdn/devops && cd devops/ansible
> make build
> ansible-galaxy collection install build/decdn-node-*.tar.gz
> ansible-galaxy collection install --no-deps build/decdn-publisher-*.tar.gz
> ```

Once published:
From [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/),
which also installs `decdn.node`:

```bash
ansible-galaxy collection install decdn.publisher
Expand All @@ -56,6 +43,21 @@ collections:
version: ">=0.1.0"
```

Each release is listed in the
[CHANGELOG](https://github.com/decdn/devops/blob/main/ansible/galaxy/publisher/CHANGELOG.md)
and as a `publisher-collection-vX.Y.Z`
[GitHub Release](https://github.com/decdn/devops/releases) with the tarball and its
`SHA256SUMS`. To run changes not released yet, build both collections from a checkout
and install the node collection first. The builds carry the last releases' versions
until the next ones, so `--force` replaces installed releases:

```bash
git clone https://github.com/decdn/devops && cd devops/ansible
make build
ansible-galaxy collection install --force build/decdn-node-*.tar.gz
ansible-galaxy collection install --force build/decdn-publisher-*.tar.gz
```

## Usage

A minimal sponsor playbook — `decdn.node`'s baseline first (so the admin key lands
Expand Down
13 changes: 9 additions & 4 deletions charts/decdn-node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,14 +158,19 @@ config:
```

```bash
# From a checkout of this repo:
helm install decdn-node-1 charts/decdn-node -n decdn -f values-node-1.yaml

# From the OCI registry, at a released version (CHANGELOG.md):
# From the OCI registry, at a released version:
helm install decdn-node-1 oci://ghcr.io/decdn/charts/decdn-node --version X.Y.Z \
-n decdn -f values-node-1.yaml

# Changes not released yet, from a checkout of this repo:
helm install decdn-node-1 charts/decdn-node -n decdn -f values-node-1.yaml
```

The chart is listed on
[Artifact Hub](https://artifacthub.io/packages/helm/decdn-node/decdn-node), and each
version's changes are in its
[CHANGELOG](https://github.com/decdn/devops/blob/main/charts/decdn-node/CHANGELOG.md).

Published charts are signed with cosign, and every release after 0.1.0 also carries
build-provenance attestations;
[RELEASING.md](https://github.com/decdn/devops/blob/main/RELEASING.md#verifying-a-release)
Expand Down
7 changes: 4 additions & 3 deletions docs/node-operators.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,12 @@ loopback, and the chain's contract addresses taken from upstream's deployment ma
| **Docker Compose** (one Docker host), recommended | [`compose/README.md`](../compose/README.md) | `decdn-compose init node` |
| **Ansible** (VMs, bare metal, a fleet) | [`ansible/README.md`](../ansible/README.md) | `inventory/hosts-node.yml.example`, `make deploy-node` (`playbooks/node.yml`) |
| **cloud-init** (one VM, no control machine) | [`cloud-init/README.md`](../cloud-init/README.md) | `cloud-init/user-data-node.yaml` |
| **Helm** (Kubernetes, one release per node) | [`charts/decdn-node/README.md`](../charts/decdn-node/README.md) | the chart, with no `config.cache.origin` |
| **Helm** (Kubernetes, one release per node) | [`charts/decdn-node/README.md`](../charts/decdn-node/README.md) | `oci://ghcr.io/decdn/charts/decdn-node`, with no `config.cache.origin` |

[`requirements.md`](requirements.md) has the hardware, network and platform requirements,
and compares the paths. The Galaxy collection for your own playbooks is
[`decdn.node`](../ansible/galaxy/node/README.md).
and compares the paths. For your own playbooks, the roles are the
[`decdn.node`](../ansible/galaxy/node/README.md) collection on
[Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/node/) (`ansible-galaxy collection install decdn.node`).

A node without an origin backend fills a miss by pulling from other nodes
(`node_to_node_pull_through_enabled`). The Ansible role and the chart turn that on for
Expand Down
7 changes: 4 additions & 3 deletions docs/publishers.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,10 @@ Only the origin is required. With Ansible, start from
`ansible/inventory/hosts-publisher.yml.example`; `make deploy-publisher`
(`playbooks/publisher.yml`) deploys all four in that order, and each has its own target
(`deploy-origin`, `deploy-sponsord`, `deploy-relay`, `deploy-dns`). The roles also ship as
the [`decdn.publisher`](../ansible/galaxy/publisher/README.md) Galaxy collection, which
depends on [`decdn.node`](../ansible/galaxy/node/README.md) for the baseline, observability
and the origin node itself.
the [`decdn.publisher`](../ansible/galaxy/publisher/README.md) collection on
[Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/decdn/publisher/) (`ansible-galaxy collection install decdn.publisher`),
which depends on [`decdn.node`](../ansible/galaxy/node/README.md) for the baseline,
observability and the origin node itself.

## Origin nodes

Expand Down
Loading