Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .github/workflows/maintenance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,12 @@ jobs:
# Unset = ubuntu-latest. A malformed value stops this job reporting any
# check at all -- see docs/setup/runners.md and ci.yml's header.
runs-on: ${{ fromJSON(vars.RUNNER_LABELS || '["ubuntu-latest"]') }}
timeout-minutes: 10
# 30, not 10: `make check-tool-versions` looks each pin up in turn with a
# 90 s fetch limit — the five kit tools plus any adopter rows in
# scripts/tool-pins.extra — so a slow registry day must still end in the
# script's own ERROR report, not in the job being cancelled first. A normal
# run takes about a minute; this weekly job costs nothing extra to allow.
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand Down Expand Up @@ -52,7 +57,7 @@ jobs:
# and it is three GitHub API calls plus two registry fetches per week.
- name: Run maintenance checks
env:
# Lets check-tool-versions.sh use `gh` (1,000 req/hr per repo) rather
# Lets check-tool-versions.sh authenticate its GitHub lookups (1,000 req/hr per repo) rather
# than the shared unauthenticated 60/hr-per-IP pool. Read-only public
# metadata; the job keeps its top-level `contents: read` permission.
GH_TOKEN: ${{ github.token }}
Expand Down
2 changes: 1 addition & 1 deletion docs/template/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Why each piece of this repository exists, and what it costs to keep. A component
| `scripts/bootstrap.sh` | Applies everything a template can't ship as files; idempotent sync | Highest-cost component — E2E-verified each release (below) |
| `scripts/check-*.sh` | Self-consistency: skills index, local-md hygiene, licence marker, label values repeated in forms and the labeler, the `scripts/*.test.js` suites | Near zero |
| `scripts/install-ci-tools.sh` | Checksum-verified CI tool installs; single home for all five tool version pins, shared by `ci.yml` and `maintenance.yml` via `make ci-tools`; bounds `RUNNER_LABELS` to linux x86_64 | Hand-bump a pin when the drift check flags it |
| `scripts/check-tool-versions.sh` | Diffs those pins against upstream weekly and fails on drift — Dependabot cannot see them, so nothing else would | Near zero; add a row when a tool is added |
| `scripts/check-tool-versions.sh` | Diffs those pins, plus any adopter rows in the optional `scripts/tool-pins.extra`, against upstream weekly and fails on drift — Dependabot cannot see them, so nothing else would | Near zero; add a row when a kit tool is added (adopter tools go in `tool-pins.extra`) |
| `docs/adr/` | Decision records; the "why" layer | Grows slowly by trigger criteria |
| `docs/setup/` | Bootstrap, licensing and runner-selection reference, plus manual fallbacks | Update alongside `bootstrap.sh` and the workflows |
| `docs/template/` | Template-product meta (this dir); deleted on adoption | Only exists upstream |
Expand Down
99 changes: 85 additions & 14 deletions scripts/check-tool-versions.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,15 @@
set -euo pipefail

PINS_FILE="${PINS_FILE:-scripts/install-ci-tools.sh}"
# Optional, adopter-owned: pins for tools the template does not ship (syft,
# uv, pnpm, ...), one row per tool — `<tool> <version> <source>` — where the
# source is github:<owner>/<repo>, npm:<package> or pypi:<package>. The
# template ships no such file; upstream syncs never touch it, so adding a tool
# here needs no edit to this script or to install-ci-tools.sh. Blank lines and
# `#` comments are ignored; any other malformed row is an ERROR, never a skip.
# Install those tools wherever you install them, reading the version from
# this file (e.g. `awk '$1 == "syft" { print $2 }' scripts/tool-pins.extra`).
EXTRA_PINS_FILE="${EXTRA_PINS_FILE:-scripts/tool-pins.extra}"

# --- deliberate holds --------------------------------------------------------
# If a newer upstream release is knowingly NOT adopted (it breaks something, or
Expand Down Expand Up @@ -57,9 +66,17 @@ read_pin() {
}

# --- upstream lookups --------------------------------------------------------
# 90 s, not less: PyPI's JSON for a large project is several MB (uv's is ~4.9 MB)
# and a cold CDN fetch of it has taken 45-55 s; this weekly job is not latency
# bound, and a timeout here is reported as an ERROR, never as a pass.
fetch_json_field() {
local url="$1" expr="$2" body
body="$(curl -fsSL --max-time 30 "$url")" || return 1
local url="$1" expr="$2" auth="${3:-}" body
if [ -n "$auth" ]; then
# header from a file descriptor, so the token is not in curl's argv
body="$(curl -fsSL --max-time 90 -H @<(printf 'Authorization: Bearer %s\n' "$auth") "$url")" || return 1
else
body="$(curl -fsSL --max-time 90 "$url")" || return 1
fi
printf '%s' "$body" | python3 -c "
import json,sys
try:
Expand All @@ -69,18 +86,24 @@ except Exception:
" || return 1
}

# Prefer gh so Actions runs authenticate with GITHUB_TOKEN (1,000 req/hr per
# repo) instead of sharing the unauthenticated 60/hr-per-IP pool. Falls back to
# the public REST endpoint so this still works without gh installed.
latest_github_tag() {
local repo="$1" tag=""
if command -v gh >/dev/null 2>&1; then
tag="$(gh api "repos/${repo}/releases/latest" --jq '.tag_name' 2>/dev/null)" || tag=""
fi
if [ -z "$tag" ]; then
tag="$(fetch_json_field "https://api.github.com/repos/${repo}/releases/latest" \
"json.load(sys.stdin)['tag_name']")" || return 1
# Authenticate when a token is at hand so Actions runs use GITHUB_TOKEN (1,000
# req/hr per repo) instead of the unauthenticated 60/hr-per-IP pool: GH_TOKEN
# or GITHUB_TOKEN from the environment, else a logged-in gh's token (a local
# read, no network). The request itself always goes through curl, so it has
# the same 90 s limit as every other lookup — `gh api` has no timeout, and a
# stalled call would otherwise outlast the job.
github_token() {
if [ -n "${GH_TOKEN:-}" ]; then printf '%s' "$GH_TOKEN"
elif [ -n "${GITHUB_TOKEN:-}" ]; then printf '%s' "$GITHUB_TOKEN"
elif command -v gh >/dev/null 2>&1; then gh auth token 2>/dev/null || true
fi
}

latest_github_tag() {
local repo="$1" tag token
token="$(github_token)"
tag="$(fetch_json_field "https://api.github.com/repos/${repo}/releases/latest" \
"json.load(sys.stdin)['tag_name']" "$token")" || return 1
[ -n "$tag" ] || return 1
printf '%s' "$tag"
}
Expand Down Expand Up @@ -108,6 +131,26 @@ latest_upstream() {
esac
}

# An extra pin's upstream, by source scheme. GitHub tags lose any non-numeric
# prefix (`v`, `lychee-v`, ...) so they compare directly against the pin.
latest_extra() {
case "$1" in
github:*/*) latest_github_tag "${1#github:}" | sed 's/^[^0-9]*//' ;;
Comment thread
TzuH-Hsu marked this conversation as resolved.
Resolved
npm:?*)
fetch_json_field "https://registry.npmjs.org/${1#npm:}/latest" \
"json.load(sys.stdin)['version']"
;;
pypi:?*)
fetch_json_field "https://pypi.org/pypi/${1#pypi:}/json" \
"json.load(sys.stdin)['info']['version']"
;;
*)
echo "error: unknown source '$1' (use github:<owner>/<repo>, npm:<pkg> or pypi:<pkg>)" >&2
return 1
;;
esac
}

# --- main --------------------------------------------------------------------
if [ ! -f "$PINS_FILE" ]; then
echo "error: pins file not found: ${PINS_FILE}" >&2
Expand Down Expand Up @@ -156,6 +199,34 @@ done <<EOF
$TOOLS
EOF

if [ -f "$EXTRA_PINS_FILE" ]; then
while IFS= read -r row || [ -n "$row" ]; do
row="${row%%#*}"
# shellcheck disable=SC2086 # split the row into its whitespace-separated fields
set -- $row
[ "$#" -gt 0 ] || continue
if [ "$#" -ne 3 ]; then
printf '%-20s %-12s %-12s %s\n' "${1:-?}" "?" "?" "ERROR (${EXTRA_PINS_FILE}: expected '<tool> <version> <source>')"
errors=$((errors + 1))
continue
fi
tool="$1" pinned="$2" source="$3"
if ! latest="$(latest_extra "$source")" || [ -z "$latest" ]; then
Comment thread
TzuH-Hsu marked this conversation as resolved.
Resolved
printf '%-20s %-12s %-12s %s\n' "$tool" "$pinned" "?" "ERROR (lookup failed: ${source})"
errors=$((errors + 1))
continue
fi
if [ "$pinned" = "$latest" ]; then
printf '%-20s %-12s %-12s %s\n' "$tool" "$pinned" "$latest" "ok"
elif reason="$(hold_reason "${tool}:${latest}")"; then
printf '%-20s %-12s %-12s %s\n' "$tool" "$pinned" "$latest" "HELD - ${reason}"
else
printf '%-20s %-12s %-12s %s\n' "$tool" "$pinned" "$latest" "DRIFT"
drift=$((drift + 1))
fi
done < "$EXTRA_PINS_FILE"
fi

echo

if [ "$errors" -gt 0 ]; then
Expand All @@ -166,7 +237,7 @@ fi

if [ "$drift" -gt 0 ]; then
echo "FAIL: ${drift} tool pin(s) behind upstream." >&2
echo " Bump the pin in ${PINS_FILE}, run 'make verify', open a PR." >&2
echo " Bump the pin in ${PINS_FILE} (or ${EXTRA_PINS_FILE}), run 'make verify', open a PR." >&2
echo " To knowingly stay behind, add a hold_reason entry in this script." >&2
exit 1
fi
Expand Down
2 changes: 1 addition & 1 deletion skills/github-actions-hygiene/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Workflows are the highest-privilege, least-reviewed code in most repositories
7. **Destructive or irreversible dispatch workflows require a typed-confirmation guard** — the operator must type an exact phrase (e.g. the repo name) as a `workflow_dispatch` input, checked before anything destructive runs.
8. **Checksum-verify any binary downloaded and installed in a workflow** (release tools, linters) — download the checksums file alongside the asset and verify before `chmod +x` / `install`.
9. **Anti-sprawl**: a new workflow file needs a justification that no existing workflow already covers. This template's target set is four: CI (verification), issue labeling, link maintenance, release PRs. Adding a fifth is a deliberate decision, not a default — prefer adding a job or a `make` target to an existing workflow first.
10. **Every pinned tool version lives in one file, and something watches it.** SHA-pinning actions (rule 3) only covers `uses:`. Dependabot reads manifests, so a version pinned inside a `run:` step or an install script is invisible to it and will rot silently — no alert, no PR, no signal at all. Keep those pins in a single file (`scripts/install-ci-tools.sh` here) and run a scheduled check that diffs them against upstream (`make check-tool-versions`, a step in the weekly maintenance workflow). A pin nobody is watching is a pin that stays put for years.
10. **Every pinned tool version lives in one file, and something watches it.** SHA-pinning actions (rule 3) only covers `uses:`. Dependabot reads manifests, so a version pinned inside a `run:` step or an install script is invisible to it and will rot silently — no alert, no PR, no signal at all. Keep those pins in a single file (`scripts/install-ci-tools.sh` here) and run a scheduled check that diffs them against upstream (`make check-tool-versions`, a step in the weekly maintenance workflow). A pin nobody is watching is a pin that stays put for years. Tools the template does not ship go in the optional `scripts/tool-pins.extra` (`<tool> <version> github:<owner>/<repo>|npm:<pkg>|pypi:<pkg>`), which the same check reads — so adding one edits neither kit script.

## How

Expand Down
Loading