Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions .github/actions/image-tag-cleanup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
name: Image tag cleanup
description: Delete ephemeral (commit-tagged) image tags/versions older than a retention window, always keeping a minimum number, and never touching anything that doesn't match the ephemeral prefix.

inputs:
registry:
description: "Which registry to clean (dockerhub | ghcr)"
required: true
image_name:
description: "Docker Hub image name (e.g. sisqueslabs/beacon-api). Required when registry=dockerhub."
required: false
default: ""
ghcr_image_name:
description: "GHCR image name (e.g. ghcr.io/sisques-labs/beacon-api). Required when registry=ghcr."
required: false
default: ""
ephemeral_tag_prefix:
description: "Only tags starting with this prefix are ever eligible for deletion. Everything else (stable releases, latest, edge, legacy alpha/beta/rc, ...) is always protected."
required: false
default: "sha-"
retention_days:
description: "Delete eligible tags/versions older than this many days"
required: false
default: "14"
keep_min:
description: "Always keep at least this many of the most recently updated eligible tags/versions, regardless of age"
required: false
default: "5"
dry_run:
description: "Log what would be deleted without deleting anything"
required: false
default: "false"
dockerhub_username:
description: "Required when registry=dockerhub"
required: false
default: ""
dockerhub_token:
description: "Required when registry=dockerhub. Needs Read, Write, Delete scope — Read & Write is NOT enough to delete tags."
required: false
default: ""
github_token:
description: "Required when registry=ghcr. Needs packages:write."
required: false
default: ""

runs:
using: composite
steps:
- name: Clean Docker Hub tags
if: inputs.registry == 'dockerhub'
shell: bash
env:
DOCKERHUB_USERNAME: ${{ inputs.dockerhub_username }}
DOCKERHUB_TOKEN: ${{ inputs.dockerhub_token }}
run: |
bash "${{ github.action_path }}/cleanup-dockerhub.sh" \
"${{ inputs.image_name }}" "${{ inputs.ephemeral_tag_prefix }}" \
"${{ inputs.retention_days }}" "${{ inputs.keep_min }}" "${{ inputs.dry_run }}"

- name: Clean GHCR versions
if: inputs.registry == 'ghcr'
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.github_token }}
run: |
bash "${{ github.action_path }}/cleanup-ghcr.sh" \
"${{ inputs.ghcr_image_name }}" "${{ inputs.ephemeral_tag_prefix }}" \
"${{ inputs.retention_days }}" "${{ inputs.keep_min }}" "${{ inputs.dry_run }}"
52 changes: 52 additions & 0 deletions .github/actions/image-tag-cleanup/cleanup-dockerhub.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#!/usr/bin/env bash
set -euo pipefail

# Deletes ephemeral tags (see select-deletions.sh) from a Docker Hub
# repository. Fetches the real tag list, delegates the decision of what's
# safe to delete to select-deletions.sh, then deletes only those.

IMAGE_NAME="${1:?image_name required, e.g. sisqueslabs/beacon-api}"
EPHEMERAL_PREFIX="${2:?ephemeral tag prefix required}"
RETENTION_DAYS="${3:?retention_days required}"
KEEP_MIN="${4:?keep_min required}"
DRY_RUN="${5:-true}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

DOCKERHUB_USERNAME="${DOCKERHUB_USERNAME:?}"
DOCKERHUB_TOKEN="${DOCKERHUB_TOKEN:?}"

TOKEN=$(curl -sf -X POST "https://hub.docker.com/v2/users/login/" \
-H "Content-Type: application/json" \
-d "{\"username\": \"${DOCKERHUB_USERNAME}\", \"password\": \"${DOCKERHUB_TOKEN}\"}" \
| jq -r '.token')

if [ -z "$TOKEN" ] || [ "$TOKEN" = "null" ]; then
echo "::error::Failed to authenticate with Docker Hub." >&2
exit 1
fi

CANDIDATES="[]"
URL="https://hub.docker.com/v2/repositories/${IMAGE_NAME}/tags?page_size=100&ordering=last_updated"
while [ -n "$URL" ] && [ "$URL" != "null" ]; do
PAGE=$(curl -sf -H "Authorization: JWT ${TOKEN}" "$URL")
PAGE_ENTRIES=$(echo "$PAGE" | jq -c '[.results[] | {id: .name, tags: [.name], updated_at: .last_updated}]')
CANDIDATES=$(jq -c -n --argjson a "$CANDIDATES" --argjson b "$PAGE_ENTRIES" '$a + $b')
URL=$(echo "$PAGE" | jq -r '.next')
done

TO_DELETE=$(echo "$CANDIDATES" | bash "${SCRIPT_DIR}/select-deletions.sh" "$EPHEMERAL_PREFIX" "$RETENTION_DAYS" "$KEEP_MIN")
TOTAL_COUNT=$(echo "$CANDIDATES" | jq 'length')
DELETE_COUNT=$(echo "$TO_DELETE" | jq 'length')

echo "Docker Hub (${IMAGE_NAME}): ${DELETE_COUNT} tag(s) selected for deletion out of ${TOTAL_COUNT} total tags scanned."

echo "$TO_DELETE" | jq -r '.[]' | while read -r TAG; do
if [ "$DRY_RUN" = "true" ]; then
echo "[dry-run] would delete ${IMAGE_NAME}:${TAG}"
else
echo "Deleting ${IMAGE_NAME}:${TAG}"
curl -sf -X DELETE -H "Authorization: JWT ${TOKEN}" \
"https://hub.docker.com/v2/repositories/${IMAGE_NAME}/tags/${TAG}/" \
|| echo "::warning::Failed to delete ${IMAGE_NAME}:${TAG}"
fi
done
56 changes: 56 additions & 0 deletions .github/actions/image-tag-cleanup/cleanup-ghcr.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
set -euo pipefail

# Deletes ephemeral package versions (see select-deletions.sh) from GHCR.
# A GHCR "version" is one image manifest and can carry several tags at
# once, so an entry is only a deletion candidate when EVERY tag on that
# version matches the ephemeral prefix — a version co-tagged with a stable
# release is always protected, same guarantee as Docker Hub's per-tag check.

GHCR_IMAGE_NAME="${1:?ghcr_image_name required, e.g. ghcr.io/sisques-labs/beacon-api}"
EPHEMERAL_PREFIX="${2:?ephemeral tag prefix required}"
RETENTION_DAYS="${3:?retention_days required}"
KEEP_MIN="${4:?keep_min required}"
DRY_RUN="${5:-true}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

GITHUB_TOKEN="${GITHUB_TOKEN:?}"

WITHOUT_HOST="${GHCR_IMAGE_NAME#ghcr.io/}"
ORG="${WITHOUT_HOST%%/*}"
PACKAGE="${WITHOUT_HOST#*/}"
ENCODED_PACKAGE=$(jq -rn --arg v "$PACKAGE" '$v|@uri')

CANDIDATES="[]"
PAGE=1
while :; do
RESP=$(curl -sf \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/orgs/${ORG}/packages/container/${ENCODED_PACKAGE}/versions?per_page=100&page=${PAGE}")
COUNT=$(echo "$RESP" | jq 'length')
[ "$COUNT" = "0" ] && break
PAGE_ENTRIES=$(echo "$RESP" | jq -c '[.[] | {id: (.id|tostring), tags: (.metadata.container.tags // []), updated_at: .created_at}]')
CANDIDATES=$(jq -c -n --argjson a "$CANDIDATES" --argjson b "$PAGE_ENTRIES" '$a + $b')
[ "$COUNT" -lt 100 ] && break
PAGE=$((PAGE + 1))
done

TO_DELETE=$(echo "$CANDIDATES" | bash "${SCRIPT_DIR}/select-deletions.sh" "$EPHEMERAL_PREFIX" "$RETENTION_DAYS" "$KEEP_MIN")
TOTAL_COUNT=$(echo "$CANDIDATES" | jq 'length')
DELETE_COUNT=$(echo "$TO_DELETE" | jq 'length')

echo "GHCR (${GHCR_IMAGE_NAME}): ${DELETE_COUNT} version(s) selected for deletion out of ${TOTAL_COUNT} total versions scanned."

echo "$TO_DELETE" | jq -r '.[]' | while read -r VERSION_ID; do
if [ "$DRY_RUN" = "true" ]; then
echo "[dry-run] would delete package version ${VERSION_ID}"
else
echo "Deleting package version ${VERSION_ID}"
curl -sf -X DELETE \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/orgs/${ORG}/packages/container/${ENCODED_PACKAGE}/versions/${VERSION_ID}" \
|| echo "::warning::Failed to delete package version ${VERSION_ID}"
fi
done
92 changes: 92 additions & 0 deletions .github/workflows/image-cleanup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
name: Image Tag Cleanup

# Deletes old ephemeral (commit-tagged) image tags — the ones trunk-ci-cd.yml
# publishes on every merge to main (:sha-<shortsha>) — so they don't
# accumulate forever in the registry. Never touches anything that doesn't
# match ephemeral_tag_prefix: stable releases (:X.Y.Z, :latest), :edge, and
# legacy release-train tags (:alpha, :beta, :X.Y.Z-alpha.N, ...) are always
# safe regardless of the prefix a given repo uses.
# See openspec/changes/trunk-based-ci-cd/design.md.
#
# This workflow only runs when called — the consumer repo owns the
# schedule (see README "Image Tag Cleanup" usage example).

on:
workflow_call:
inputs:
image_name:
description: "Docker Hub image name (e.g. sisqueslabs/beacon-api)"
required: true
type: string
ghcr_image_name:
description: "GHCR image name (e.g. ghcr.io/sisques-labs/beacon-api). Required when push_ghcr=true."
required: false
type: string
default: ""
push_ghcr:
description: "Also clean up GHCR (mirrors the flag used to publish there)"
required: false
type: boolean
default: false
ephemeral_tag_prefix:
description: "Only tags starting with this prefix are ever eligible for deletion"
required: false
type: string
default: "sha-"
retention_days:
description: "Delete eligible tags older than this many days"
required: false
type: number
default: 14
keep_min:
description: "Always keep at least this many of the most recent eligible tags, regardless of age"
required: false
type: number
default: 5
dry_run:
description: "Log what would be deleted without deleting anything — use this to validate on a new repo before trusting it"
required: false
type: boolean
default: false
secrets:
DOCKERHUB_USERNAME:
required: true
DOCKERHUB_TOKEN:
description: "Needs Read, Write, Delete scope — Read & Write is NOT enough to delete tags"
required: true

permissions:
packages: write

jobs:
dockerhub:
name: Clean Docker Hub
runs-on: ubuntu-latest
steps:
- name: Clean
uses: sisques-labs/workflows/.github/actions/image-tag-cleanup@main
with:
registry: dockerhub
image_name: ${{ inputs.image_name }}
ephemeral_tag_prefix: ${{ inputs.ephemeral_tag_prefix }}
retention_days: ${{ inputs.retention_days }}
keep_min: ${{ inputs.keep_min }}
dry_run: ${{ inputs.dry_run }}
dockerhub_username: ${{ secrets.DOCKERHUB_USERNAME }}
dockerhub_token: ${{ secrets.DOCKERHUB_TOKEN }}

ghcr:
name: Clean GHCR
if: inputs.push_ghcr
runs-on: ubuntu-latest
steps:
- name: Clean
uses: sisques-labs/workflows/.github/actions/image-tag-cleanup@main
with:
registry: ghcr
ghcr_image_name: ${{ inputs.ghcr_image_name }}
ephemeral_tag_prefix: ${{ inputs.ephemeral_tag_prefix }}
retention_days: ${{ inputs.retention_days }}
keep_min: ${{ inputs.keep_min }}
dry_run: ${{ inputs.dry_run }}
github_token: ${{ github.token }}
56 changes: 55 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This repository contains reusable GitHub Actions workflows and composite actions
│ ├── trunk-ci-cd.yml # Continuous build + dev/pre deploy for trunk-based repos
│ ├── docker-release.yml # Version bump + Docker build & publish (or promote, for trunk-based repos)
│ ├── docker-smoke-build.yml # PR-time Dockerfile build + blocking vuln scan
│ ├── image-cleanup.yml # Delete old ephemeral image tags (dockerhub + ghcr)
│ ├── codeql.yml # CodeQL security analysis (init + analyze)
│ ├── pr-labeler.yml # Auto-label PRs by changed files
│ ├── coolify-deploy.yml # Trigger a Coolify deploy via its API
Expand All @@ -21,7 +22,8 @@ This repository contains reusable GitHub Actions workflows and composite actions
│ ├── setup/ # Common setup (Node.js, pnpm, checkout)
│ ├── install/ # Install dependencies with pnpm
│ ├── trivy-scan/ # Scan a local image, report always, block optionally
│ └── release-train-detect/ # Compute the exact next version from git tags
│ ├── release-train-detect/ # Compute the exact next version from git tags
│ └── image-tag-cleanup/ # Decide + delete ephemeral tags older than a retention window
└── tests/ # Test suites for the scripts in this repo
```

Expand Down Expand Up @@ -495,6 +497,58 @@ This has not yet been validated against a real multi-arch image — confirm
it works before relying on `bump_mode: promote` for an actual production
release.

### Image Tag Cleanup

`trunk-ci-cd.yml` publishes a new, uniquely-tagged image (`:sha-<shortsha>`)
on **every** merge to `main` — with no branch-per-channel model to bound how
many accumulate, that tag count only ever goes up. `image-cleanup.yml`
deletes the old ones on a schedule you own, on both Docker Hub and GHCR.

**Safety guarantee, not a suggestion:** an image is only ever a deletion
*candidate* when **every** tag it carries starts with `ephemeral_tag_prefix`
(default `sha-`). A stable release (`:X.Y.Z`), `:latest`, `:edge`, and every
legacy release-train tag (`:alpha`, `:beta`, `:X.Y.Z-alpha.N`, ...) never
match that prefix, so they are **never even considered** — this holds
regardless of `retention_days`/`keep_min`, and is covered by
`tests/image-tag-cleanup-select.test.sh`. Among the tags that *do* match,
the `keep_min` most recently published are always kept regardless of age;
the rest are deleted once older than `retention_days`.

**Usage (consumer repository)** — this workflow owns no schedule itself, so
the consumer's `on:` trigger decides the cadence:

```yaml
name: Image Tag Cleanup

on:
schedule:
- cron: "0 3 * * 1" # weekly, Monday 03:00 UTC
workflow_dispatch: # lets you trigger it manually too

jobs:
cleanup:
uses: sisques-labs/workflows/.github/workflows/image-cleanup.yml@main
with:
image_name: sisqueslabs/my-app
ghcr_image_name: ghcr.io/sisques-labs/my-app
push_ghcr: true
# dry_run: true # uncomment to validate on a new repo before trusting it
secrets:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
```

**⚠️ Token permissions:** exactly like the Docker Hub description sync
above, `DOCKERHUB_TOKEN` needs **`Read, Write, Delete`** scope — a token
scoped only for `docker login` + push (`Read & Write`) can authenticate but
every delete call will fail with `401`/`403`. GHCR cleanup uses the
workflow's own `github.token` (needs `packages: write`, already declared at
the workflow level) — no extra secret required.

**First run on any repo:** pass `dry_run: true` once to see what the
selection logic *would* delete (logged, nothing touched), then remove it
once you've confirmed the output looks right.

### Branch sync after a stable release

After a stable release (a push to `main` that graduates a release), both
Expand Down
12 changes: 12 additions & 0 deletions openspec/changes/trunk-based-ci-cd/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,22 @@ Fixed: `promote` now computes both automatically —

This makes cutting a release a genuine one-click action (`workflow_dispatch` with no required fields), matching the spirit of the whole migration: humans decide *when* to release, the pipeline decides *what* the release contains.

### D9 — Ephemeral tag retention (discovered during the beacon-api pilot)

Every merge to `main` publishes `:sha-<shortsha>` (D1) with no expiry — unlike the old branch-per-channel model, there is no `develop`/`staging` boundary to bound how many of these accumulate, so left alone the tag count grows forever. This is a real gap, not a hypothetical: the old model had the identical problem (`release-train.yml` never pruned `alpha.N`/`beta.N` tags either) but bringing back environment branches to "fix" it would reintroduce exactly what this migration removes — a rebuild-per-branch model that breaks the "build once, promote the same artifact" guarantee (D2). The fix has to be orthogonal to branch topology.

New reusable workflow `image-cleanup.yml` + composite action `image-tag-cleanup`, invoked on a schedule the *consumer* repo owns (this shared workflow has no `on:` trigger of its own). Design:

- **Never touches an official tag.** An image/tag is only a deletion candidate when **every** tag it carries matches `ephemeral_tag_prefix` (default `sha-`). A stable release, `:latest`, `:edge`, or a legacy `:alpha`/`:beta`/`:X.Y.Z-alpha.N` tag never matches that prefix, so it's never even considered — this is a hard filter, not a heuristic, and it's the one piece of this design covered by a pure, offline-testable unit test (`select-deletions.sh` + `tests/image-tag-cleanup-select.test.sh`), independent of any live registry call.
- **`keep_min` is a floor, `retention_days` is the trigger.** The `keep_min` most recently published ephemeral tags are always kept regardless of age (so there's always something to promote even during a slow release cadence); only entries beyond that floor are deleted once older than `retention_days`.
- **Generic across registries and repos.** The same selection logic runs against Docker Hub (per-tag) and GHCR (per-package-version, which can carry multiple tags — a version is skipped entirely if any of its tags is non-ephemeral). `ephemeral_tag_prefix` is configurable per consumer, so a repo using a different continuous-build tag scheme isn't hardcoded to `sha-`.
- **`dry_run` is the required first step on any new repo** — logs what would be deleted without touching the registry.

## Risks / Trade-offs

- **[Risk] Multi-arch digest promotion is unproven** → Validate `imagetools create` against a real multi-platform image before beacon-api's pilot relies on it for an actual prod release.
- **[Risk] Downstream consumers pinned to `:alpha`/`:beta` tags** → Each repo must audit this before migrating; out of scope for this shared-workflow change.
- **[Risk] Cleanup deletes a tag mid-promotion** → Extremely narrow window (between `trunk-ci-cd.yml` publishing a digest and a human running `release.yml` against it); mitigated by `keep_min` defaulting to 5, which comfortably covers any realistic gap between "build validated in pre" and "someone clicks release."
- **[Trade-off] `deploy-dev`/`deploy-pre` are placeholders** → The pipeline shape ships now; real deploy logic is deferred until a repo has infrastructure to target.
- **[Trade-off] Org-wide standardization without a forced cutover** → Slower convergence (repos migrate on their own schedule) in exchange for zero blast radius on this change.

Expand Down
Loading