Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
required: true
type: string
html-dir:
description: Directory the build writes the site to (packed as docs.zip's html/).
description: Directory the build writes the site to (packed as docs.zip's single root dir, html/).
required: false
default: docs/_build/html
type: string
Expand All @@ -19,14 +19,14 @@ on:

# REUSABLE docs build — the UNPRIVILEGED half, shared across MyST projects. Builds
# the docs at the versioned BASE_URL and uploads this build's `docs` artifact
# (docs.zip, bare html/ root). It runs for every event — PRs (including forks),
# (docs.zip, a single html/ root dir). It runs for every event — PRs (including forks),
# pushes to the default branch, and tags — but NEVER publishes. publish.yml (the
# privileged half) deploys, called separately by the consumer's ci.yml on internal
# events, injecting THIS build's artifact for the current version name.
#
# This workflow OWNS the build/publish contract so it cannot drift: the version
# name, the BASE_URL, the docs.zip bare-html/ root, and the `docs` artifact name all
# live here. The consumer supplies only HOW to turn sources into HTML, via
# name, the BASE_URL, the docs.zip single-root-dir shape, and the `docs` artifact name
# all live here. The consumer supplies only HOW to turn sources into HTML, via
# `build-command`.
#
# Environment given to build-command: Node/npx (preinstalled on the runner) and uv
Expand Down Expand Up @@ -98,10 +98,13 @@ jobs:
set -euo pipefail
eval "$BUILD_COMMAND"

# Pack the site as docs.zip with a bare html/ root — the durable contract both
# publish.yml's gather and release.yml's asset rely on. Stage into html/
# first so any html-dir name yields the same root.
- name: Pack docs.zip (bare html/ root)
# Pack the site as docs.zip with a single root directory — the durable contract
# both publish.yml's gather and release.yml's asset rely on. Stage into html/
# first so any html-dir name yields the same root. publish.yml's extract keys on
# there being exactly ONE root dir, not on it being called html/ (release assets
# are durable and may have been packed by another pipeline), but everything this
# workflow produces is html/-rooted.
- name: Pack docs.zip (single html/ root)
env:
HTML_DIR: ${{ inputs.html-dir }}
run: |
Expand Down
29 changes: 25 additions & 4 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -255,16 +255,37 @@ jobs:

# Unzip every gathered zip into SITE/<version-name>.
# Persist the default-branch zip into SITE/_sources/ for future recovery.
#
# THE ZIP-SHAPE CONTRACT: a docs.zip must unzip to exactly ONE top-level
# directory, whose CONTENTS become site/<version-name>. The directory's NAME is
# ignored. docs.yml packs `html/`, but release assets are durable and long-lived
# — a repo migrating from python-copier-template's `_release.yml` has immutable
# releases carrying zips rooted at the TAG NAME (`mv html $GITHUB_REF_NAME;
# zip -r docs.zip $GITHUB_REF_NAME`), which cannot be re-cut. Keying on `html/`
# silently dropped every one of those releases from the switcher. Anything else
# — no entries, several entries, or files loose at the zip root — is malformed
# (nothing is known to produce it) and warns + skips rather than guessing.
- name: Extract artifacts into site
run: |
set -euo pipefail
for f in "$RUNNER_TEMP/gather"/*.zip; do
[ -f "$f" ] || continue
name=$(basename "$f" .zip)
tmp=$(mktemp -d)
unzip -q "$f" 'html/*' -d "$tmp" 2>/dev/null && [ -d "$tmp/html" ] \
&& { rm -rf "$RUNNER_TEMP/site/$name"; mv "$tmp/html" "$RUNNER_TEMP/site/$name"; } \
|| echo "::warning::$name: no html/ root in zip — skipping"
if ! unzip -q "$f" -d "$tmp" 2>/dev/null; then
echo "::warning::$name: could not unzip docs.zip — skipping"
rm -rf "$tmp"; continue
fi
# -print0/-d '' so the count is exact whatever the entries are named,
# and dotfiles at the root count too (they would be a malformed zip).
roots=(); mapfile -d '' -t roots < <(find "$tmp" -mindepth 1 -maxdepth 1 -print0)
if [ "${#roots[@]}" -eq 1 ] && [ -d "${roots[0]}" ]; then
rm -rf "$RUNNER_TEMP/site/$name"
mv "${roots[0]}" "$RUNNER_TEMP/site/$name"
else
found=$(printf '%s ' "${roots[@]##*/}")
echo "::warning::$name: docs.zip must unzip to exactly one top-level directory — found [${found% }] — skipping"
fi
rm -rf "$tmp"
done
dzip="$RUNNER_TEMP/gather/$DEFAULT.zip"
Expand All @@ -275,7 +296,7 @@ jobs:
# `generate` orders versions, writes switcher.json + index.html, and prints the
# stable-alias source dir (the newest release), or "" when no release is deployed yet.
# Hard-fails if the default branch is absent (Ensure step guarantees the zip, but
# the extract could still fail if the zip had no html/ root).
# the extract could still fail if the zip was malformed).
- name: Generate switcher.json and stable alias
run: |
set -euo pipefail
Expand Down
18 changes: 13 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,14 @@ IO (gather + extract); the JS kernel does the pure logic (`generate` + the folde
sanitisation**: version names are clean by construction — `main`, `pr-<number>`, or
a tag without `/` (the `tags: ['*']` trigger never builds `/`-tags).

The extract step's zip-shape rule is deliberately **name-agnostic**: a `docs.zip` must
unzip to exactly one top-level directory, whose *contents* become `site/<version-name>`
— the directory's *name* is ignored. `docs.yml` packs `html/`, but release assets are
durable and may predate this pipeline (python-copier-template's `_release.yml` roots
its zip at the tag name, and an immutable release can't be re-cut), so keying on
`html/` silently dropped every such release from the switcher. Zero entries, several
entries, or files loose at the zip root are malformed — warn and skip, never guess.

### Self-referencing assemble.mjs (no separate action)
`publish.yml` sparse-checks-out **this** repo's `assemble/` at `job.workflow_repository`
+ `job.workflow_sha` — the `job` context resolves to the file that defines the running
Expand Down Expand Up @@ -177,14 +185,14 @@ Sub-workflows of `ci.yml`:
version name (`pr-<n>` / default-branch / tag) → run `build-command` (required
input) with `BASE_URL` + `VERSION_NAME` set (the latter for builds that need the
bare token, e.g. a Sphinx conf.py setting pydata's switcher `version_match`) →
pack `docs.zip` (bare `html/`, staged so any `html-dir` works) → upload the `docs`
pack `docs.zip` (single root dir `html/`, staged so any `html-dir` works) → upload the `docs`
artifact. No deploy; `contents: read` only (the fork-PR hint lives in publish.yml's
`warn` job). Installs uv unconditionally and relies on the runner's
preinstalled Node, so `build-command` can be `make docs` / `npx … myst build` /
`tox -e docs` regardless of project. This repo passes `npm ci && npm run docs`. It
OWNS the build↔publish contract (version name, BASE_URL, docs.zip `html/` root,
`docs` artifact name — publish.yml gathers cross-run artifacts by that NAME via the
artifacts API, never by workflow filename) so consumers only choose a command.
OWNS the build↔publish contract (version name, BASE_URL, docs.zip's single root
dir, `docs` artifact name — publish.yml gathers cross-run artifacts by that NAME via
the artifacts API, never by workflow filename) so consumers only choose a command.
- `release.yml` — **PUBLIC reusable, tag-only.** Downloads every artifact in the run
and attaches them to the tag's GitHub Release via `gh` — `gh release create` if no
Release exists yet (draft→upload→publish atomically, so immutable-safe), else
Expand Down Expand Up @@ -295,7 +303,7 @@ a shim missing the input rejects that dispatch (breaking every tagged release's
re-publish, not just the wedged-origin retry it exists for).

Add a `release` job that `uses:` this repo's `release.yml@<tag>` (with `permissions:
contents: write`) to attach each tag's built `docs.zip` (bare `html/` root) as a
contents: write`) to attach each tag's built `docs.zip` (single `html/` root dir) as a
Release asset so `assemble` can reconstruct released versions, and the
`publish-dispatch.yml` wrapper around `publish.yml` — the single caller of the engine
and the one place you pin `publish.yml@<tag>`. The wrapper owns all
Expand Down
5 changes: 5 additions & 0 deletions docs/explanations/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,11 @@ The `stable` segment name is a fixed convention, hardcoded in the widget.
redirect → `main/`. Graceful; no release required.
- **Release without `docs.zip`** (cut before this scheme): not selected by the
releases query (it filters on a `docs.zip` asset) → skipped, no hard failure.
- **Release with a differently-rooted `docs.zip`**: served fine. The extract takes
the zip's single top-level directory whatever it is named, so an immutable release
asset packed by another pipeline (python-copier-template's `_release.yml` roots
its zip at the tag name, not `html/`) still deploys. A zip with *no* single root
directory is malformed → warning, skipped.
- **Default branch missing**: if `main` has no recent successful CI artifact, no durable
`_sources/main.zip` in the live site, and no migration seed release, the deploy
**hard-fails** rather than publish a site missing it.
Expand Down
6 changes: 5 additions & 1 deletion docs/how-to/migrate-from-gh-pages.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,11 @@ Two consequences drive the procedure:
**backfill** of `docs.zip` onto those Releases — not just a config flip. (A tag with
a `gh-pages` directory but **no Release at all** gets one *created* with the
`docs.zip` — covering repos that tag without releasing, and fork rehearsals, since
forking copies tags but not Releases.)
forking copies tags but not Releases.) If your Releases *already* carry a `docs.zip`
from a previous pipeline, the backfill leaves them alone and they are used as-is —
the extract takes the zip's single root directory whatever it is named, so
python-copier-template's tag-name-rooted zips (`1.2.3/` rather than `html/`) need no
re-cutting, which matters because an immutable release cannot be re-cut anyway.
2. **The default branch has no permanent source until a publish captures it.** Each
deploy persists the branch's `docs.zip` into the site (`_sources/<branch>.zip`) and
restores from there when the CI artifact expires — but before the *first* publish
Expand Down
17 changes: 11 additions & 6 deletions docs/reference/workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,15 @@ directly; it is not a separately consumed action.

## `docs.yml` — build (unprivileged)

Builds the docs at the versioned `BASE_URL`, packs `docs.zip` (bare `html/` root),
Builds the docs at the versioned `BASE_URL`, packs `docs.zip` (single `html/` root dir),
and uploads the `docs` artifact. Declares `contents: read` only —
it never holds a write token. Installs `uv` unconditionally and relies on the runner's
preinstalled Node, so `build-command` can be `make` / `npx` / `tox` / `npm` driven.

| input | required | default | meaning |
|---|---|---|---|
| `build-command` | **yes** | — | Command that builds the HTML site into `html-dir` at `$BASE_URL`. Fold any project setup (`cp CONFIG`, `npm ci`, apt deps) behind it. |
| `html-dir` | no | `docs/_build/html` | Directory the build writes the site to; staged into `docs.zip`'s `html/`. |
| `html-dir` | no | `docs/_build/html` | Directory the build writes the site to; staged into `docs.zip` as its single root directory, `html/`. |

| output | meaning |
|---|---|
Expand Down Expand Up @@ -119,10 +119,15 @@ fresher source always wins when names collide.
Two contracts let the build (`docs.yml`) and the reconstruction (`assemble`) agree
without coordination — `docs.yml` owns both:

- **`docs.zip` is one zip with a bare `html/` root.** `docs.yml` packs it once and it
is delivered the *same file* two ways: uploaded verbatim as the `docs` artifact
(every run), and attached verbatim as the `docs.zip` Release asset on tags. Both the
release gather and the branch/PR gather unzip the same `html/` shape.
- **`docs.zip` unzips to exactly one top-level directory.** `docs.yml` packs it once,
rooted at `html/`, and it is delivered the *same file* two ways: uploaded verbatim as
the `docs` artifact (every run), and attached verbatim as the `docs.zip` Release asset
on tags. The extract side is deliberately looser than the pack side: it takes that
single directory's **contents** and ignores its **name**, because release assets are
durable and may predate this pipeline — python-copier-template's `_release.yml` roots
its zip at the tag name (`1.2.3/`), and an immutable release cannot be re-cut. Zero
entries, several entries, or files loose at the zip root are malformed: that version
is skipped with a warning rather than guessed at.
- **The version name is the site sub-dir *and* the `BASE_URL`.** It is `pr-<n>` for
PRs, else the ref name (the default branch, or a tag without `/`). `docs.yml` sets
`BASE_URL=/REPO/<version-name>` and `assemble` files the artifact at
Expand Down