From fc0c2255b99f0b530607d2773d51630973d57cb7 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 25 Jun 2026 09:20:54 +0000 Subject: [PATCH 1/8] migrate: open the github-pages env policy; fix the ordering how-to MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit scripts/migrate.sh: after flipping the Pages source, set the github-pages environment's deployment_branch_policy to null ("no restriction") so deploys from PR/tag refs — which run under the nested-publish model — aren't rejected by the environment. Previously a manual step. docs/how-to/migrate-from-gh-pages.md: correct the migration ordering. - Flipping the Pages source to GitHub Actions is non-destructive: the last gh-pages deployment keeps serving until the first Actions deploy supersedes it (no downtime). Drop the earlier "blank site" / "PR-first, red publish window" framing. - The clean universal order is seed-first: seed the default branch before any publish runs (a publish replaces the whole site), run the cutover before merging the pipeline PR, and let that PR's CI be the first deploy. This makes every publish safe — including on repos already serving Pages from Actions, which is no longer a special case. - Document the new env-policy step and the legacy-switcher gh-pages-deletion caveat; correct the --delete-gh-pages guard wording (_sources/.zip is 200). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- docs/how-to/migrate-from-gh-pages.md | 64 +++++++++++++++++++++------- scripts/migrate.sh | 15 ++++++- 2 files changed, 62 insertions(+), 17 deletions(-) diff --git a/docs/how-to/migrate-from-gh-pages.md b/docs/how-to/migrate-from-gh-pages.md index 3711f23..0864fe2 100644 --- a/docs/how-to/migrate-from-gh-pages.md +++ b/docs/how-to/migrate-from-gh-pages.md @@ -41,16 +41,33 @@ Two consequences drive the whole procedure: - `gh` authenticated with **repo-admin** on the target repo (flipping the Pages source needs admin; a CI token can't — which is why this is a local script). -- Land the new pipeline + `myst.yml` changes from the - [tutorial](../tutorials/adding-to-a-fresh-repo.md) as **one PR, and merge it into - your default branch.** Merging is safe: `gh-pages` keeps serving until you flip the - Pages source, so the only effect is that the default branch's CI now produces a - `docs.zip` artifact — which is exactly the precondition the deletion later checks - for. +- Prepare the new pipeline + `myst.yml` changes from the + [tutorial](../tutorials/adding-to-a-fresh-repo.md) as **one PR**, but **don't merge it + (or let its `publish` deploy) until the cutover has seeded the default branch** — the + cutover (Step 2) seeds first, then uses that PR's CI as the first Actions deploy, and + you merge afterwards. See the ordering note below. - Run the script from **inside a clone of the target repo**: it reads the repo's tags and `gh-pages` tree from the working directory (it fetches `origin/gh-pages` for you). Running it from anywhere else will find no tags/branches. +> **Ordering — seed before any publish.** Flipping the Pages source to GitHub Actions +> is **non-destructive**: the last `gh-pages` deployment keeps serving until the first +> Actions deploy supersedes it ([confirmed +> here](https://github.com/orgs/community/discussions/158055)), so there is no downtime +> and no "blank site" window. The one rule that matters is **seed the default branch +> before any v0.7.0 `publish` runs** — a `publish` reassembles and *replaces the whole +> site*, so if it runs before a durable `main` exists it drops `//`. The +> cutover does exactly that (seed first, deploy second), which is why you run it +> **before merging the pipeline PR**, and let that PR's CI be the first deploy. +> +> This matters most if your repo **already serves Pages from GitHub Actions** (you are +> upgrading an earlier Actions deploy, not a classic `gh-pages` branch): there an +> internal PR's `publish` deploys live the moment it runs, so an un-seeded publish would +> drop `//` immediately. Seeding first — with `guard-default-branch` at its +> default `true` — makes every publish safe. (On a classic `gh-pages` repo a too-early +> publish instead just fails closed, since `deploy-pages` won't publish until the flip; +> seeding first keeps it green regardless.) + ## Step 1 — always dry-run `scripts/migrate.sh` drives the sequence. Start with `--dry-run`: it prints the @@ -81,14 +98,20 @@ It executes, in order, then **stops with `gh-pages` intact**: `gh-pages` directory and whose Release lacks a `docs.zip`, zip that directory as a bare `html/` root and attach it as `docs.zip`. Tags containing `/` are skipped. This is first: the reconstructed site is built from these assets. -2. **Flip the Pages source → GitHub Actions** (`gh api PUT …/pages`). `deploy-pages` - refuses to publish unless the source is already "GitHub Actions", so this must - precede the deploy. -3. **Trigger a deploy.** Pass `--deploy-workflow ` to dispatch one (your - `publish.yml` `workflow_dispatch` works: the default branch is *gathered* from its - latest CI artifact, so a pure dispatch still reconstructs `main` + the backfilled - releases), or push to the default branch / re-run its CI when prompted. -4. **Verify.** The script fetches `switcher.json` and checks every listed version URL +2. **Seed the default branch.** Capture the gh-pages `/` tree as a published + `pages-default-seed` release, so the default branch is durable *before* any publish + runs (the whole point of the ordering note above). +3. **Flip the Pages source → GitHub Actions** and **open the `github-pages` + environment's deployment-branch policy** (to "no restriction", so deploys from + PR/tag refs — which run under the nested-publish model — aren't rejected). Both are + done for you. The flip is non-destructive: the last `gh-pages` deployment keeps + serving until the first Actions deploy; `deploy-pages` only publishes once the source + is "GitHub Actions", so this precedes the deploy. +4. **Trigger the first Actions deploy.** Pass `--deploy-workflow ` to dispatch + one, or — the recommended path — when prompted, open/merge the pipeline PR (or push + the default branch) so its CI's `publish` runs. It gathers the default branch from + the seed and persists `_sources/.zip`. +5. **Verify.** The script fetches `switcher.json` and checks every listed version URL returns `200`. Probes are cache-busted, so a too-early probe can't pin a cached `404` at the CDN. @@ -113,6 +136,17 @@ the new model can reconstruct `/main/` without `gh-pages`. It then re-verifies t site, asks you to type the repo name, deletes `gh-pages`, **and deletes the seed release** (the in-site `_sources` copy supersedes it). After this, the rollback is gone. +> **Caveat — old pages that reference `gh-pages` at runtime.** Docs built under the old +> model sometimes embed a hardcoded version switcher that reads `gh-pages` live — via +> the GitHub *contents API* (`…/contents?ref=gh-pages`) or by loading assets from a +> `gh-pages` URL. Those pages are reconstructed verbatim from their `docs.zip`, so the +> references remain. After deletion, a switcher that only *queries the API* degrades +> harmlessly: the request `404`s, its populate script throws an uncaught promise +> (console-only), and the version list simply empties — the page itself is intact. +> Anything that *loads assets* (CSS/JS/images) from `gh-pages`, though, will break. +> `grep` your old release pages for `gh-pages` before finalizing and accept (the +> switcher emptying is usually fine) or fix what you find. + ## Rollback Between the cutover (step 2) and the deletion (step 3), `gh-pages` is no longer @@ -126,7 +160,7 @@ step. | flag | effect | |---|---| | `--dry-run` | Print the backfill plan + probe the current site only; upload nothing; skip flip/deploy. | -| `--delete-gh-pages` | **The only mode that deletes.** Guard that the default branch publishes `docs.zip`, re-verify, then delete `gh-pages`. | +| `--delete-gh-pages` | **The only mode that deletes.** Guard that `_sources/.zip` is live (`200`), re-verify, then delete `gh-pages` **and the seed release**. | | `--pages-ref ` | `gh-pages` ref to read (default `origin/gh-pages`). | | `--deploy-workflow ` | `gh workflow run ` to trigger the cutover deploy; omit to be prompted to deploy by hand. | | `--yes` | Skip the typed confirmation before deleting `gh-pages` (with `--delete-gh-pages`; use with care). | diff --git a/scripts/migrate.sh b/scripts/migrate.sh index 4d7f69f..0914871 100755 --- a/scripts/migrate.sh +++ b/scripts/migrate.sh @@ -17,8 +17,8 @@ # the durable source; gh-pages is no longer needed and deleting it is split off behind # a guard: # -# cutover (default) backfill → seed → flip Pages → deploy → verify, STOP -# (gh-pages retained as the rollback) +# cutover (default) backfill → seed → flip Pages + open env policy → +# deploy → verify, STOP (gh-pages retained as the rollback) # finalize (--delete-gh-pages) guard _sources/.zip is live → re-verify → # delete gh-pages + the seed release (rollback dies here) # @@ -256,6 +256,17 @@ echo " (gh-pages still exists after this and is the rollback: set the source" echo " back to 'Deploy from a branch' to restore serving with no data lost.)" gh api --method PUT "repos/$REPO/pages" -f build_type=workflow +# --- Allow deploys from any ref in the github-pages environment ------------- +# Under the nested-publish model, internal PRs and tags deploy from THEIR OWN ref +# (publish runs inside their CI run), so the github-pages environment's +# deployment-branch policy must allow those refs or deploy-pages is rejected by the +# environment. Set it to "no restriction" (deployment_branch_policy: null) so any +# branch/tag can deploy. Idempotent; PUT creates the environment if absent. +echo +echo "-- 2b. Allowing github-pages deploys from any branch/tag --" +echo '{"deployment_branch_policy":null}' \ + | gh api --method PUT "repos/$REPO/environments/github-pages" --input - >/dev/null + # --- Trigger a deploy ------------------------------------------------------- echo echo "-- 3. Triggering a deploy --" From 3bf8d61cd65671c6980014f2453ef1bc0dabbd57 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 25 Jun 2026 09:32:53 +0000 Subject: [PATCH 2/8] migrate: split into prepare + finalize runs; PR does the publish MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Restructure the migration into two script runs with the pipeline PR's own publish doing the deploy in between, instead of the prepare run triggering and verifying a deploy itself: - prepare (default run): backfill + seed + flip Pages source + open the env policy, then STOP. Uploads and flips only — no deploy, no verify. (Flipping is non-destructive: the last gh-pages deployment keeps serving until the first Actions deploy.) Removes the interactive "trigger a deploy / press Enter / verify" steps and the --deploy-workflow flag. - the pipeline PR's CI runs the first publish (reads the seed, persists _sources/.zip). - finalize (--delete-gh-pages): guard _sources is live, verify the live site, delete gh-pages + the seed. Rewrite the how-to around the three steps (prepare run → PR publishes → finalize run) and the "why this order" (non-destructive flip; seed before any publish). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- docs/how-to/migrate-from-gh-pages.md | 207 +++++++++++++-------------- scripts/migrate.sh | 97 +++++-------- 2 files changed, 139 insertions(+), 165 deletions(-) diff --git a/docs/how-to/migrate-from-gh-pages.md b/docs/how-to/migrate-from-gh-pages.md index 0864fe2..d3504ae 100644 --- a/docs/how-to/migrate-from-gh-pages.md +++ b/docs/how-to/migrate-from-gh-pages.md @@ -1,140 +1,135 @@ # How-to: migrate from an existing `gh-pages` site If your docs already publish to a `gh-pages` branch (the `keep_files` model), this -moves you onto the reconstruct-from-durable-sources model **in a single PR into your -default branch**, **without losing any served version**, and with an instant -rollback until the very last step. +moves you onto the reconstruct-from-durable-sources model **in a single pipeline PR**, +**without losing any served version**, and with an instant rollback until the very last +step. It is **two local `migrate.sh` runs, with your pipeline PR doing the deploy in +between**: + +1. **`migrate.sh ORG/REPO`** — backfill releases, seed the default branch, flip the + Pages source to Actions + open the environment policy. *Uploads and flips only — no + deploy.* +2. **Open + merge your pipeline PR** — its CI runs the first publish, which reads the + seed and persists the default branch durably into the site. +3. **`migrate.sh ORG/REPO --delete-gh-pages`** — verify the live site, then delete + `gh-pages` + the seed release. ## What you are migrating onto (read this first) Every deploy reconstructs the whole site from sources with **very different -durability** — and the safety of the migration hinges entirely on that difference -(see the [architecture explanation](../explanations/architecture.md)): +durability** — and the safety of the migration hinges on that difference (see the +[architecture explanation](../explanations/architecture.md)): | version | source under the new model | durable? | |---|---|---| | released tags | a `docs.zip` asset on each **GitHub Release** | **yes, permanent** — but only once attached | -| default branch (`main`) | persisted each deploy into the site at `_sources/.zip` | **yes** — once one deploy has captured it (seeded at cutover) | +| default branch (`main`) | persisted each deploy into the site at `_sources/.zip` | **yes** — once one deploy has captured it (seeded in run 1) | | open PRs (`pr-`) | each PR's build artifact | no — drops on merge/close | -Two consequences drive the whole procedure: +Two consequences drive the procedure: 1. **Your old releases are not durable yet.** Their docs exist only as directories on - `gh-pages`; the matching Releases have no `docs.zip`. So migration includes a - one-time **backfill** of `docs.zip` onto those Releases — not just a config flip. -2. **The default branch has no permanent source until a deploy captures it.** Each - deploy persists the branch's `docs.zip` into the published site - (`_sources/.zip`) and restores it from there when the CI artifact expires — - but before the *first* such capture there is nothing in the site, and `gh-pages` is - the only recoverable copy of `/main/`. So the cutover **seeds** it: it captures the - gh-pages `/` tree as a published seed release, which the first deploy reads - and persists to `_sources/.zip`. After that the in-site copy carries `/main/` - — *before* the default branch ever builds docs itself — which is what lets a repo cut - over to the reusable workflow in **one PR**, ahead of `docs→main`. + `gh-pages`; the matching Releases have no `docs.zip`. So run 1 includes a one-time + **backfill** of `docs.zip` onto those Releases — not just a config flip. +2. **The default branch has no permanent source until a publish captures it.** Each + deploy persists the branch's `docs.zip` into the site (`_sources/.zip`) and + restores from there when the CI artifact expires — but before the *first* publish + there is nothing in the site. So run 1 **seeds** it: it captures the gh-pages + `/` tree as a published seed release, which the first publish reads and + persists to `_sources/.zip`. After that the in-site copy carries `//` + — *before* the default branch ever builds docs itself. + +## Why this order + +- **Flipping the Pages source to GitHub Actions is non-destructive.** The last + `gh-pages` deployment keeps serving until the first Actions deploy supersedes it + ([community discussion #158055](https://github.com/orgs/community/discussions/158055)), + so run 1 can flip up front with no downtime and no blank window. +- **Seed the default branch before any publish runs.** A publish reassembles and + *replaces the entire site*, so a publish that runs before a durable `` exists + would drop `//`. Run 1 seeds first; only then does the pipeline PR publish. + This is what makes the pipeline PR's own publish safe — including on a repo that + **already serves Pages from GitHub Actions**, where a publish deploys live the moment + it runs (with `guard-default-branch` at its default `true`, an un-seeded publish would + otherwise fail rather than silently drop the branch). > **The load-bearing rule:** *keep `gh-pages` until `_sources/.zip` is live in -> the deployed site* (the cutover seed gets you there on the first deploy; the -> `--delete-gh-pages` guard probes exactly this). Deleting `gh-pages` is a **separate, -> gated step**, never part of the cutover — and it removes the seed release too. +> the deployed site.* Run 1's seed gets you there on the first publish; the +> `--delete-gh-pages` guard probes exactly this. Deleting `gh-pages` is the **separate, +> gated final run** — never part of run 1 — and it removes the seed release too. ## Before you start -- `gh` authenticated with **repo-admin** on the target repo (flipping the Pages - source needs admin; a CI token can't — which is why this is a local script). +- `gh` authenticated with **repo-admin** on the target repo (flipping the Pages source + and setting the environment policy need admin; a CI token can't — which is why this is + a local script). - Prepare the new pipeline + `myst.yml` changes from the - [tutorial](../tutorials/adding-to-a-fresh-repo.md) as **one PR**, but **don't merge it - (or let its `publish` deploy) until the cutover has seeded the default branch** — the - cutover (Step 2) seeds first, then uses that PR's CI as the first Actions deploy, and - you merge afterwards. See the ordering note below. + [tutorial](../tutorials/adding-to-a-fresh-repo.md) as **one PR** — but **don't merge + it until run 1 has seeded the default branch** (you merge it as step 3 below). - Run the script from **inside a clone of the target repo**: it reads the repo's tags - and `gh-pages` tree from the working directory (it fetches `origin/gh-pages` for - you). Running it from anywhere else will find no tags/branches. - -> **Ordering — seed before any publish.** Flipping the Pages source to GitHub Actions -> is **non-destructive**: the last `gh-pages` deployment keeps serving until the first -> Actions deploy supersedes it ([confirmed -> here](https://github.com/orgs/community/discussions/158055)), so there is no downtime -> and no "blank site" window. The one rule that matters is **seed the default branch -> before any v0.7.0 `publish` runs** — a `publish` reassembles and *replaces the whole -> site*, so if it runs before a durable `main` exists it drops `//`. The -> cutover does exactly that (seed first, deploy second), which is why you run it -> **before merging the pipeline PR**, and let that PR's CI be the first deploy. -> -> This matters most if your repo **already serves Pages from GitHub Actions** (you are -> upgrading an earlier Actions deploy, not a classic `gh-pages` branch): there an -> internal PR's `publish` deploys live the moment it runs, so an un-seeded publish would -> drop `//` immediately. Seeding first — with `guard-default-branch` at its -> default `true` — makes every publish safe. (On a classic `gh-pages` repo a too-early -> publish instead just fails closed, since `deploy-pages` won't publish until the flip; -> seeding first keeps it green regardless.) - -## Step 1 — always dry-run - -`scripts/migrate.sh` drives the sequence. Start with `--dry-run`: it prints the -backfill plan and probes the current site, uploading nothing and skipping every -destructive step. + and `gh-pages` tree from the working directory (it fetches `origin/gh-pages` for you). + Running it from anywhere else will find no tags/branches. + +## Step 1 — dry-run (recommended) ```bash scripts/migrate.sh ORG/REPO --dry-run ``` -Compare the **backfill plan** against the **probe list**: any version the live site -serves that is *not* in the backfill plan is a `gh-pages` directory with **no -matching GitHub Release** — it will be **dropped** from the reconstructed site (it -stays on `gh-pages`, so it is recoverable, but it won't be served). Decide whether -that is acceptable, or cut real Releases for the versions you want to keep first. - -## Step 2 — cutover (reversible) +It prints the backfill + seed plan and probes the current site, uploading nothing and +flipping nothing. Compare the **backfill plan** against the **probe list**: any version +the live site serves that is *not* in the backfill plan is a `gh-pages` directory with +**no matching GitHub Release** — it will be **dropped** from the reconstructed site (it +stays on `gh-pages`, so it is recoverable, but won't be served). Cut real Releases for +any such versions you want to keep before proceeding. -Run it for real: +## Step 2 — prepare (uploads + flips; no deploy) ```bash scripts/migrate.sh ORG/REPO ``` -It executes, in order, then **stops with `gh-pages` intact**: - -1. **Backfill (non-destructive, idempotent).** For each release tag that is a - `gh-pages` directory and whose Release lacks a `docs.zip`, zip that directory as a - bare `html/` root and attach it as `docs.zip`. Tags containing `/` are skipped. - This is first: the reconstructed site is built from these assets. -2. **Seed the default branch.** Capture the gh-pages `/` tree as a published - `pages-default-seed` release, so the default branch is durable *before* any publish - runs (the whole point of the ordering note above). -3. **Flip the Pages source → GitHub Actions** and **open the `github-pages` - environment's deployment-branch policy** (to "no restriction", so deploys from - PR/tag refs — which run under the nested-publish model — aren't rejected). Both are - done for you. The flip is non-destructive: the last `gh-pages` deployment keeps - serving until the first Actions deploy; `deploy-pages` only publishes once the source - is "GitHub Actions", so this precedes the deploy. -4. **Trigger the first Actions deploy.** Pass `--deploy-workflow ` to dispatch - one, or — the recommended path — when prompted, open/merge the pipeline PR (or push - the default branch) so its CI's `publish` runs. It gathers the default branch from - the seed and persists `_sources/.zip`. -5. **Verify.** The script fetches `switcher.json` and checks every listed version URL - returns `200`. Probes are cache-busted, so a too-early probe can't pin a cached - `404` at the CDN. - -It does **not** delete `gh-pages`. You are now live on the new model. The cutover -deploy should have **seeded** the default branch — captured its gh-pages content as a -seed release that this deploy persisted to `_sources/.zip` — so `/main/` is -now served from the in-site copy. `gh-pages` is retained only as the rollback. - -## Step 3 — finalize (irreversible), once `_sources/.zip` is live - -When the deployed site serves `_sources/.zip` (the cutover seed gets you there; -or, later, the default branch building `docs.zip` itself keeps it fresh), delete -`gh-pages`: +It does the following, then **stops with `gh-pages` intact and still serving**: + +1. **Backfill (non-destructive, idempotent).** For each release tag that is a `gh-pages` + directory and whose Release lacks a `docs.zip`, zip that directory (bare `html/` + root) and attach it as `docs.zip`. Tags containing `/` are skipped. +2. **Seed the default branch.** Capture the gh-pages `/` tree as the published + `pages-default-seed` release, so the default branch is durable before any publish. +3. **Flip the Pages source → GitHub Actions** and **open the `github-pages` environment's + `deployment_branch_policy`** to "no restriction" (so deploys from PR/tag refs — which + run under the nested-publish model — aren't rejected by the environment). The flip is + non-destructive; the site keeps serving the last `gh-pages` deployment until step 3 + publishes. + +No deploy is triggered here — that is your pipeline PR's job. + +## Step 3 — publish, via your pipeline PR + +Open and merge the pipeline PR. Its CI runs the first **publish**: with the source on +Actions, the seed present, and the env policy open, it reconstructs the whole site +(default branch from the seed, the backfilled releases, any open PRs) and deploys it, +**persisting `_sources/.zip`** into the published site. Merging then has the +default branch build its own docs, so `_sources/.zip` refreshes with real +content and the seed becomes redundant. + +Confirm the site is live on the new model (visit `https://ORG.github.io/REPO/` and the +switcher) before finalizing. + +## Step 4 — finalize (irreversible) + +Once the publish has deployed and `_sources/.zip` is live: ```bash scripts/migrate.sh ORG/REPO --delete-gh-pages ``` -This **guards** the deletion: it refuses unless `https://ORG.github.io/REPO/_sources/.zip` -returns `200` — i.e. the deployed site holds a durable copy of the default branch, so -the new model can reconstruct `/main/` without `gh-pages`. It then re-verifies the live -site, asks you to type the repo name, deletes `gh-pages`, **and deletes the seed -release** (the in-site `_sources` copy supersedes it). After this, the rollback is gone. +This **guards** the deletion: it refuses unless +`https://ORG.github.io/REPO/_sources/.zip` returns `200` — i.e. the deployed +site holds a durable copy of the default branch, so the new model can reconstruct +`//` without `gh-pages`. It then verifies the live site, asks you to type the +repo name, deletes `gh-pages`, **and deletes the seed release** (the in-site `_sources` +copy supersedes it). After this, the rollback is gone. > **Caveat — old pages that reference `gh-pages` at runtime.** Docs built under the old > model sometimes embed a hardcoded version switcher that reads `gh-pages` live — via @@ -149,18 +144,16 @@ release** (the in-site `_sources` copy supersedes it). After this, the rollback ## Rollback -Between the cutover (step 2) and the deletion (step 3), `gh-pages` is no longer -*served* but still *exists* — your rollback. If anything is wrong, flip the Pages -source back to **Deploy from a branch / `gh-pages`** and serving is restored -instantly, with nothing lost. That is exactly why the deletion is a separate, gated -step. +Between run 1 (step 2) and the deletion (step 4), `gh-pages` is no longer *served* but +still *exists* — your rollback. If anything is wrong, flip the Pages source back to +**Deploy from a branch / `gh-pages`** and serving is restored instantly, with nothing +lost. That is exactly why the deletion is a separate, gated run. ## Flags | flag | effect | |---|---| -| `--dry-run` | Print the backfill plan + probe the current site only; upload nothing; skip flip/deploy. | -| `--delete-gh-pages` | **The only mode that deletes.** Guard that `_sources/.zip` is live (`200`), re-verify, then delete `gh-pages` **and the seed release**. | +| `--dry-run` | Print the backfill + seed plan + probe the current site only; upload nothing; skip the flip. | +| `--delete-gh-pages` | **The only mode that deletes.** Guard that `_sources/.zip` is live (`200`), verify the live site, then delete `gh-pages` **and the seed release**. | | `--pages-ref ` | `gh-pages` ref to read (default `origin/gh-pages`). | -| `--deploy-workflow ` | `gh workflow run ` to trigger the cutover deploy; omit to be prompted to deploy by hand. | | `--yes` | Skip the typed confirmation before deleting `gh-pages` (with `--delete-gh-pages`; use with care). | diff --git a/scripts/migrate.sh b/scripts/migrate.sh index 0914871..228492f 100755 --- a/scripts/migrate.sh +++ b/scripts/migrate.sh @@ -9,48 +9,49 @@ # - the destructive step wants a human watching with their own `gh auth`; # - it leaves no workflow_dispatch stub behind in each consumer repo. # -# TWO PHASES, because the new model's sources differ in durability. Releases become -# permanent once their docs.zip is attached (backfill). The DEFAULT BRANCH has no -# permanent source until it builds docs.zip itself — so we SEED it: capture the -# gh-pages / tree as a published seed release, which the cutover deploy reads -# and persists to _sources/.zip in the site. After that the in-site copy is -# the durable source; gh-pages is no longer needed and deleting it is split off behind -# a guard: +# TWO RUNS, with your pipeline PR doing the deploy in between, because the new model's +# sources differ in durability. Releases become permanent once their docs.zip is +# attached (backfill). The DEFAULT BRANCH has no permanent source until it builds +# docs.zip itself — so we SEED it: capture the gh-pages / tree as a published +# seed release, which the first publish reads and persists to _sources/.zip in +# the site. After that the in-site copy is the durable source; gh-pages is no longer +# needed and deleting it is split off behind a guard: # -# cutover (default) backfill → seed → flip Pages + open env policy → -# deploy → verify, STOP (gh-pages retained as the rollback) -# finalize (--delete-gh-pages) guard _sources/.zip is live → re-verify → -# delete gh-pages + the seed release (rollback dies here) +# 1. prepare (default) backfill releases → seed default branch → flip Pages +# source to Actions + open the env policy, then STOP. +# Uploads + flips only; no deploy. (Flipping is +# non-destructive: the last gh-pages deployment keeps +# serving until the first Actions deploy supersedes it.) +# 2. (you) open + merge your pipeline PR — its CI does the first publish, which reads +# the seed and persists _sources/.zip. +# 3. finalize (--delete-gh-pages) guard _sources/.zip is live → verify the +# live site → delete gh-pages + the seed release. # -# Until the finalize step, gh-pages still EXISTS (just unserved) and is the rollback: -# flip the Pages source back to "Deploy from a branch" to restore serving instantly. +# Between runs 1 and 3, gh-pages still EXISTS (just unserved) and is the rollback: flip +# the Pages source back to "Deploy from a branch" to restore serving instantly. # # Usage: # scripts/migrate.sh [--dry-run] [--pages-ref ] -# [--deploy-workflow ] # scripts/migrate.sh --delete-gh-pages [--pages-ref ] [--yes] # # --dry-run print the backfill + seed plan + probe the current site only; -# upload nothing, skip the flip / deploy. -# --delete-gh-pages finalize: guard _sources/.zip is live, re-verify, -# then delete gh-pages + the seed release. The only mode that -# deletes. +# upload nothing, skip the flip. +# --delete-gh-pages finalize: guard _sources/.zip is live, verify the live +# site, then delete gh-pages + the seed release. The only mode +# that deletes. # --pages-ref gh-pages ref to read (default: origin/gh-pages) -# --deploy-workflow `gh workflow run ` to trigger the cutover deploy; -# omit to be prompted to trigger a deploy by hand. # --yes skip the typed confirmation before deleting gh-pages. set -euo pipefail REPO="" PAGES_REF="origin/gh-pages" -DEPLOY_WORKFLOW="" DRY_RUN=false DELETE_GH_PAGES=false ASSUME_YES=false SEED_TAG="pages-default-seed" # published seed release holding the default branch's docs usage() { - echo "usage: scripts/migrate.sh [--dry-run] [--pages-ref ] [--deploy-workflow ]" >&2 + echo "usage: scripts/migrate.sh [--dry-run] [--pages-ref ]" >&2 echo " scripts/migrate.sh --delete-gh-pages [--pages-ref ] [--yes]" >&2 } @@ -60,7 +61,6 @@ while [ $# -gt 0 ]; do --delete-gh-pages) DELETE_GH_PAGES=true; shift ;; --yes) ASSUME_YES=true; shift ;; --pages-ref) PAGES_REF="$2"; shift 2 ;; - --deploy-workflow) DEPLOY_WORKFLOW="$2"; shift 2 ;; -h|--help) usage; exit 0 ;; -*) echo "unknown flag: $1" >&2; exit 2 ;; *) if [ -z "$REPO" ]; then REPO="$1"; else echo "unexpected arg: $1" >&2; exit 2; fi; shift ;; @@ -195,8 +195,8 @@ guard_default_durable() { if [ "$code" != "200" ]; then echo "::error::REFUSING to delete gh-pages: $url is not live ($code)." >&2 echo " The deployed site has no durable copy of '$default' yet, so gh-pages is still" >&2 - echo " the only recoverable copy. Run a deploy (seed, or the default branch's own CI)" >&2 - echo " so assemble persists _sources/$default.zip, then retry." >&2 + echo " the only recoverable copy. Open/merge the pipeline PR so its publish persists" >&2 + echo " _sources/$default.zip into the site, then retry." >&2 exit 1 fi echo " ok: $url is live — '$default' is durable independently of gh-pages" @@ -205,13 +205,13 @@ guard_default_durable() { echo "== migrate: $REPO (pages-ref=$PAGES_REF, dry-run=$DRY_RUN, delete=$DELETE_GH_PAGES) ==" # ============================================================================ -# Finalize: --delete-gh-pages. Guard → re-verify → delete. Nothing else. +# Finalize: --delete-gh-pages. Guard → verify → delete. Nothing else. # ============================================================================ if $DELETE_GH_PAGES; then echo guard_default_durable echo - echo "-- Re-verifying the live site before deleting gh-pages --" + echo "-- Verifying the live site before deleting gh-pages --" if ! verify; then echo echo "Verification FAILED — NOT deleting gh-pages. Flip the Pages source back to" >&2 @@ -235,7 +235,8 @@ if $DELETE_GH_PAGES; then fi # ============================================================================ -# Cutover: backfill → seed → flip → deploy → verify, then STOP (gh-pages retained). +# Prepare: backfill → seed → flip + open env policy, then STOP (gh-pages retained). +# The first deploy is your pipeline PR's own publish; verification is the 2nd run. # ============================================================================ backfill seed_default_branch @@ -245,7 +246,7 @@ if $DRY_RUN; then echo "-- (dry-run) probing current site --" verify || echo "(dry-run probe failed — expected if the site isn't deployed yet)" echo - echo "Dry run complete: backfill + seed plan shown (nothing uploaded); flip/deploy skipped." + echo "Dry run complete: backfill + seed plan shown (nothing uploaded); flip skipped." exit 0 fi @@ -267,34 +268,14 @@ echo "-- 2b. Allowing github-pages deploys from any branch/tag --" echo '{"deployment_branch_policy":null}' \ | gh api --method PUT "repos/$REPO/environments/github-pages" --input - >/dev/null -# --- Trigger a deploy ------------------------------------------------------- echo -echo "-- 3. Triggering a deploy --" -if [ -n "$DEPLOY_WORKFLOW" ]; then - gh workflow run "$DEPLOY_WORKFLOW" --repo "$REPO" - echo " dispatched $DEPLOY_WORKFLOW; wait for it to finish before continuing." -else - echo " No --deploy-workflow given. Trigger a deploy now (push to the default" - echo " branch / re-run the latest CI), wait for it to finish, then continue." -fi -read -r -p " Press Enter once the deploy has completed… " _ - -# --- Verify ----------------------------------------------------------------- -echo -echo "-- 4. Verifying the reconstructed site --" -if ! verify; then - echo - echo "Verification FAILED. gh-pages is intact — flip the Pages source back to" >&2 - echo "'Deploy from a branch / gh-pages' to roll back." >&2 - exit 1 -fi - -echo -echo "Cutover complete. The site is served from GitHub Actions; gh-pages is RETAINED" -echo "as the rollback. This deploy should have persisted the default branch durably to" -echo "_sources/.zip (from the seed); once that is live, gh-pages is no longer" -echo "the only copy." +echo "Prepared. Releases are backfilled, the default branch is seeded ('$SEED_TAG')," +echo "and Pages is served from GitHub Actions (gh-pages RETAINED as the rollback —" +echo "flip the source back to 'Deploy from a branch' to restore it instantly)." echo -echo "Finalize (deletes gh-pages AND the seed release) once _sources/.zip is" -echo "live in the site — the finalize guard checks exactly that:" -echo " scripts/migrate.sh $REPO --delete-gh-pages" +echo "Next:" +echo " 1. Open + merge your pipeline PR. Its CI runs the first publish, which reads" +echo " the seed and persists _sources/.zip into the deployed site." +echo " 2. Once that publish has deployed, finalize (verify + delete gh-pages and the" +echo " seed). The guard checks _sources/.zip is live first:" +echo " scripts/migrate.sh $REPO --delete-gh-pages" From 75b6bf22309ef765d9cc5ba61aec3416aacdad97 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 25 Jun 2026 09:44:24 +0000 Subject: [PATCH 3/8] docs: pin favicon to the DLS logo + set logo_text The docs set no favicon, so `myst build` fetched MyST's default favicon from mystmd.org on every build; when that remote fetch flakes ("Premature close") the build fails. Pin the favicon to the bundled dls-logo.svg so the build is self-contained, and set logo_text to the repo name. (Same fix applied to the PandABlocks consumers.) Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- docs/myst.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/myst.yml b/docs/myst.yml index 601bd11..902d7de 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -42,4 +42,9 @@ site: # default flattened /directive. folders: true logo: images/dls-logo.svg + logo_text: myst-version-switcher-plugin + # Pin the favicon to the bundled DLS logo so `myst build` never fetches the + # default favicon from mystmd.org (a flaky remote fetch that intermittently + # fails the build). + favicon: images/dls-logo.svg github_url: https://github.com/DiamondLightSource/myst-version-switcher-plugin From ea3b8acdfbc4ec89df23df54a69f6de268dd573e Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 25 Jun 2026 10:09:32 +0000 Subject: [PATCH 4/8] docs: keep the migration how-to task-focused (move the "why" to architecture) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - migrate-from-gh-pages: drop the "Why this order" explanation section (a how-to shouldn't carry it) and link to the architecture explanation instead. Move the "prepare the pipeline PR" prerequisite out of "Before you start" into Step 3, where it's actually used. - architecture: add a "Migrating from gh-pages" subsection holding the moved rationale (non-destructive source flip; whole-site-replace ⇒ seed before any publish), the anchor the how-to now links to. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- docs/explanations/architecture.md | 19 +++++++++++++++++++ docs/how-to/migrate-from-gh-pages.md | 25 +++++++------------------ 2 files changed, 26 insertions(+), 18 deletions(-) diff --git a/docs/explanations/architecture.md b/docs/explanations/architecture.md index f93ad47..272594e 100644 --- a/docs/explanations/architecture.md +++ b/docs/explanations/architecture.md @@ -53,6 +53,25 @@ whole site makes deletion self-healing: a merged PR or a deleted release simply isn't gathered next time, so it disappears — no `keep_files` drift, no branch to prune. +### Migrating from gh-pages + +Two facts make the [cutover](../how-to/migrate-from-gh-pages.md) safe and fix its +ordering: + +- **Flipping the Pages source from a branch to GitHub Actions is non-destructive.** + The last `gh-pages` deployment keeps serving until the first Actions deploy + supersedes it ([community + discussion #158055](https://github.com/orgs/community/discussions/158055)) — so the + source can be flipped up front, with no downtime and no blank window. +- **A publish replaces the *whole* site, so the default branch must be durable before + any publish runs.** A publish that runs before `_sources/.zip` exists would + drop `//`. The migration therefore *seeds* the default branch (a published + `pages-default-seed` release captured from the old gh-pages tree) before the first + publish — which is the pipeline PR's own CI. This makes that first publish safe even + when the repo already serves Pages from Actions (where a publish deploys live + immediately); with `guard-default-branch` at its default `true`, an un-seeded publish + fails loudly rather than silently dropping the branch. + ## The `docs.zip` and version-token contracts Two small contracts let the build (in CI) and the reconstruction (in `assemble`) diff --git a/docs/how-to/migrate-from-gh-pages.md b/docs/how-to/migrate-from-gh-pages.md index d3504ae..a8c5d85 100644 --- a/docs/how-to/migrate-from-gh-pages.md +++ b/docs/how-to/migrate-from-gh-pages.md @@ -39,19 +39,8 @@ Two consequences drive the procedure: persists to `_sources/.zip`. After that the in-site copy carries `//` — *before* the default branch ever builds docs itself. -## Why this order - -- **Flipping the Pages source to GitHub Actions is non-destructive.** The last - `gh-pages` deployment keeps serving until the first Actions deploy supersedes it - ([community discussion #158055](https://github.com/orgs/community/discussions/158055)), - so run 1 can flip up front with no downtime and no blank window. -- **Seed the default branch before any publish runs.** A publish reassembles and - *replaces the entire site*, so a publish that runs before a durable `` exists - would drop `//`. Run 1 seeds first; only then does the pipeline PR publish. - This is what makes the pipeline PR's own publish safe — including on a repo that - **already serves Pages from GitHub Actions**, where a publish deploys live the moment - it runs (with `guard-default-branch` at its default `true`, an un-seeded publish would - otherwise fail rather than silently drop the branch). +For *why* this ordering is safe — the non-destructive source flip and the +seed-before-publish rule — see the [architecture explanation](../explanations/architecture.md#migrating-from-gh-pages). > **The load-bearing rule:** *keep `gh-pages` until `_sources/.zip` is live in > the deployed site.* Run 1's seed gets you there on the first publish; the @@ -63,9 +52,6 @@ Two consequences drive the procedure: - `gh` authenticated with **repo-admin** on the target repo (flipping the Pages source and setting the environment policy need admin; a CI token can't — which is why this is a local script). -- Prepare the new pipeline + `myst.yml` changes from the - [tutorial](../tutorials/adding-to-a-fresh-repo.md) as **one PR** — but **don't merge - it until run 1 has seeded the default branch** (you merge it as step 3 below). - Run the script from **inside a clone of the target repo**: it reads the repo's tags and `gh-pages` tree from the working directory (it fetches `origin/gh-pages` for you). Running it from anywhere else will find no tags/branches. @@ -106,8 +92,11 @@ No deploy is triggered here — that is your pipeline PR's job. ## Step 3 — publish, via your pipeline PR -Open and merge the pipeline PR. Its CI runs the first **publish**: with the source on -Actions, the seed present, and the env policy open, it reconstructs the whole site +Prepare the new pipeline + `myst.yml` changes from the +[tutorial](../tutorials/adding-to-a-fresh-repo.md) on a branch, and **open the PR only +after run 1 has seeded the default branch** (so its first publish is safe). Then open +and merge it: its CI runs the first **publish** — with the source on Actions, the seed +present, and the env policy open, it reconstructs the whole site (default branch from the seed, the backfilled releases, any open PRs) and deploys it, **persisting `_sources/.zip`** into the published site. Merging then has the default branch build its own docs, so `_sources/.zip` refreshes with real From aa39182ed7ba831c65cefbc1f61f5e37df91f014 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 25 Jun 2026 11:10:31 +0000 Subject: [PATCH 5/8] docs: make the release job required + literalinclude it from _release.yml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The release job is required, not optional: assemble reconstructs each released version from a docs.zip asset on its GitHub Release, and tags get no _sources/ copy — so without it a tag's docs drop on the next deploy. The tutorial said "optional" and pointed at a non-existent "step 7". - Tutorial: new Step 6 "Add the release job (required)" that literalinclude's the generic steps (download docs.zip + attach to the Release) straight from this repo's _release.yml, so it can't drift; Step 7 is now "Cut your first release". - _release.yml: bracket those two steps with docs-literalinclude markers and a note to keep them; the version-switcher.mjs checkout+cp stays outside the markers (plugin-specific). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- .github/workflows/_release.yml | 11 ++++++++-- docs/tutorials/adding-to-a-fresh-repo.md | 27 +++++++++++++++++++----- 2 files changed, 31 insertions(+), 7 deletions(-) diff --git a/.github/workflows/_release.yml b/.github/workflows/_release.yml index b50703c..881229e 100644 --- a/.github/workflows/_release.yml +++ b/.github/workflows/_release.yml @@ -16,7 +16,13 @@ jobs: sparse-checkout: plugins/version-switcher.mjs - run: cp .src/plugins/version-switcher.mjs . - # All other release artifacts from other stages (docs.zip). + # The two steps below are the generic release job a consumer needs (download the + # built docs.zip, attach it to the Release). They are literalinclude'd into + # docs/tutorials/adding-to-a-fresh-repo.md between these markers — keep the markers + # when editing. The checkout + cp above is plugin-specific (it stages + # version-switcher.mjs) and is deliberately left outside the markers. + # docs-literalinclude:release-steps:start + # Download every artifact produced earlier in the run (here: docs.zip). - name: Download artifacts uses: actions/download-artifact@v5 with: @@ -28,7 +34,8 @@ jobs: uses: softprops/action-gh-release@6da8fa9354ddfdc4aeace5fc48d7f679b5214090 # v2.4.1 with: prerelease: ${{ contains(github.ref_name, 'a') || contains(github.ref_name, 'b') || contains(github.ref_name, 'rc') }} - files: "*" + files: "*" # everything staged in the cwd — here, docs.zip generate_release_notes: true env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + # docs-literalinclude:release-steps:end diff --git a/docs/tutorials/adding-to-a-fresh-repo.md b/docs/tutorials/adding-to-a-fresh-repo.md index 26b01fe..30081d7 100644 --- a/docs/tutorials/adding-to-a-fresh-repo.md +++ b/docs/tutorials/adding-to-a-fresh-repo.md @@ -68,7 +68,7 @@ jobs: # uv and Node are preinstalled, so this can be make / tox / npx / npm. build-command: myst build --html # e.g. make docs · tox -e docs · npm ci && npm run docs - # (optional) a tag-only release job attaching docs.zip + version-switcher.mjs — step 7. + # a tag-only release job attaching docs.zip — REQUIRED once you cut releases (step 6). publish: needs: [docs] @@ -140,11 +140,28 @@ Push to `main`. CI builds `main`, the `publish` job assembles a single-entry showing one entry. (The single-entry first deploy is graceful by design; no release is required.) -## 6. Cut your first release +## 6. Add the release job (required for versioned releases) + +`assemble` reconstructs each released version from a **`docs.zip` asset on its GitHub +Release** — that asset is the only durable source for a tag (unlike the default branch, +tags get no `_sources/` copy). Without a job that attaches it, a tag's docs appear only +on its *own* deploy and **drop on the next** unrelated deploy. So add a tag-only job — +gated with `if: github.ref_type == 'tag'`, `runs-on: ubuntu-latest`, and +`permissions: { contents: write }` — whose steps download the build's `docs` artifact +and attach `docs.zip` to the Release. Those steps are exactly what this repo's +`_release.yml` runs (minus its plugin-specific `version-switcher.mjs` handling): + +```{literalinclude} ../../.github/workflows/_release.yml +:language: yaml +:start-after: docs-literalinclude:release-steps:start +:end-before: docs-literalinclude:release-steps:end +``` + +`actions/download-artifact` pulls the `docs` artifact `docs.yml` uploaded (the +`docs.zip`); `action-gh-release`'s `files: "*"` then attaches everything in the working +directory — here, just that `docs.zip` — to the tag's Release. -Tag a release and attach the built docs as a `docs.zip` asset (bare `html/` root) — -the easiest way is a tag-only job in `ci.yml` that downloads the `docs` artifact and -uploads it verbatim, alongside `version-switcher.mjs`. +## 7. Cut your first release ```bash git tag v1.0.0 && git push origin v1.0.0 From 138d84785a9261aed3d11e9a26c663f6da5fb936 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 25 Jun 2026 11:17:29 +0000 Subject: [PATCH 6/8] docs: drop logo_text (book-theme navbar looks better with just the logo) Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- docs/myst.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/myst.yml b/docs/myst.yml index 902d7de..daa1895 100644 --- a/docs/myst.yml +++ b/docs/myst.yml @@ -42,7 +42,6 @@ site: # default flattened /directive. folders: true logo: images/dls-logo.svg - logo_text: myst-version-switcher-plugin # Pin the favicon to the bundled DLS logo so `myst build` never fetches the # default favicon from mystmd.org (a flaky remote fetch that intermittently # fails the build). From 70a923d23d13dac3a5593811693bfa10c8a2273c Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 25 Jun 2026 11:25:40 +0000 Subject: [PATCH 7/8] docs: fold the release job into the ci.yml step; reframe the tutorial around a PR MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Merge the old "add the release job" step into Step 3, so the reader writes ci.yml once (docs + release + publish), with the release steps still literalinclude'd from _release.yml. - Reframe the deploy steps around the normal PR flow: Step 5 "push your branch and open a PR" explains what the docs/publish checks do and the /pr-/ preview (plus the one-time fresh-repo bootstrap where publish is red until main exists); Step 6 "merge to main — your first deploy"; Step 7 "cut your first release". Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01RBtsHDzshkwFgTJtrDVveU --- docs/tutorials/adding-to-a-fresh-repo.md | 98 +++++++++++++++--------- 1 file changed, 61 insertions(+), 37 deletions(-) diff --git a/docs/tutorials/adding-to-a-fresh-repo.md b/docs/tutorials/adding-to-a-fresh-repo.md index 30081d7..2f2df05 100644 --- a/docs/tutorials/adding-to-a-fresh-repo.md +++ b/docs/tutorials/adding-to-a-fresh-repo.md @@ -44,14 +44,22 @@ will generate it on your first deploy. In **Settings → Pages**, set **Source** to **GitHub Actions** (not "Deploy from a branch"). `deploy-pages` refuses to publish otherwise. -## 3. Add a `ci.yml` that calls the shared workflows - -You don't copy any workflows into your repo — you **call** this project's two reusable -workflows by full path, pinned to ``. `docs.yml` builds and uploads the `docs` -artifact for every event (including fork PRs); `publish.yml` reconstructs the whole -versioned site and deploys it, nested as a job that runs **only for internal events** -on your repo (the canonical-repo + non-fork guard). Fork PRs build to verify but never -reach a write token. +## 3. Add the `ci.yml` (build · release · publish) + +This is the one workflow file you add. You don't copy the build/publish logic into your +repo — you **call** this project's two reusable workflows by full path, pinned to +``. Three jobs: + +- **`docs`** — `docs.yml` builds your site at the versioned `BASE_URL` and uploads the + `docs` artifact, for every event (fork PRs included). +- **`release`** — a small tag-only job that attaches the built `docs.zip` to the GitHub + Release. **Required:** that asset is a tag's *only* durable source (tags get no + `_sources/` copy, unlike the default branch), so without it a release drops on the + next deploy. +- **`publish`** — `publish.yml` reconstructs the whole versioned site and deploys it, + nested so its status shows on the PR/commit, but it runs **only for internal events** + (the canonical-repo + non-fork guard). Fork PRs build to verify but never reach a + write token. ```yaml # .github/workflows/ci.yml @@ -68,7 +76,14 @@ jobs: # uv and Node are preinstalled, so this can be make / tox / npx / npm. build-command: myst build --html # e.g. make docs · tox -e docs · npm ci && npm run docs - # a tag-only release job attaching docs.zip — REQUIRED once you cut releases (step 6). + release: + needs: [docs] + if: github.ref_type == 'tag' # tag pushes only + runs-on: ubuntu-latest + permissions: + contents: write # create the Release + attach assets + steps: + # ↓ paste the two steps from the literalinclude below ↓ publish: needs: [docs] @@ -87,6 +102,17 @@ jobs: statuses: write ``` +The `release` job's `steps:` are exactly what this repo's own `_release.yml` runs (minus +its plugin-specific `version-switcher.mjs` handling) — drop these in under `steps:` +above (`download-artifact` pulls the `docs` artifact; `action-gh-release`'s `files: "*"` +attaches everything in the working directory, here just that `docs.zip`): + +```{literalinclude} ../../.github/workflows/_release.yml +:language: yaml +:start-after: docs-literalinclude:release-steps:start +:end-before: docs-literalinclude:release-steps:end +``` + That's the whole integration. `build-command` is the only thing most repos customise: `docs.yml` owns the versioning contract (the `BASE_URL`, the `docs.zip` bare-`html/` root, the `docs` artifact name), and `publish.yml` runs the site reconstruction @@ -132,34 +158,31 @@ runs inside their CI run), the `github-pages` environment's deployment policy mu allow those refs. In **Settings → Environments → github-pages**, it is recommended to set **Deployment branches and tags** to **No restriction**. -## 5. First deploy +## 5. Push your branch and open a PR -Push to `main`. CI builds `main`, the `publish` job assembles a single-entry -`switcher.json` and an `index.html` redirecting to `main/`, and deploys. Visit -`https://ORG.github.io/REPO/` — the redirect lands you on `main/` with the switcher -showing one entry. (The single-entry first deploy is graceful by design; no release -is required.) +Steps 2 and 4 are repo settings (done once in the UI); steps 1 and 3 are file changes — +commit them on a branch and open a PR. On the PR you'll see: -## 6. Add the release job (required for versioned releases) +- **`docs / build`** go green — it builds your docs at the versioned `BASE_URL` and + uploads the `docs` artifact. This runs for every PR, forks included. +- **`publish / publish`** runs for an *internal* PR (a branch in your own repo) and + deploys a preview of just this PR at `https://ORG.github.io/REPO/pr-/`, linked from + the PR's checks/Deployments. A **fork** PR builds but never auto-publishes (its token + is read-only) — use the opt-in from step 3 to preview one. -`assemble` reconstructs each released version from a **`docs.zip` asset on its GitHub -Release** — that asset is the only durable source for a tag (unlike the default branch, -tags get no `_sources/` copy). Without a job that attaches it, a tag's docs appear only -on its *own* deploy and **drop on the next** unrelated deploy. So add a tag-only job — -gated with `if: github.ref_type == 'tag'`, `runs-on: ubuntu-latest`, and -`permissions: { contents: write }` — whose steps download the build's `docs` artifact -and attach `docs.zip` to the Release. Those steps are exactly what this repo's -`_release.yml` runs (minus its plugin-specific `version-switcher.mjs` handling): +> **First-time exception:** on a brand-new repo the `publish` check is **red on this +> setup PR** — there's no `main` build yet for the versioned site to anchor on, and the +> default-branch guard refuses to publish a site missing it. It clears the moment you +> merge (step 6). Every PR after that previews normally. -```{literalinclude} ../../.github/workflows/_release.yml -:language: yaml -:start-after: docs-literalinclude:release-steps:start -:end-before: docs-literalinclude:release-steps:end -``` +## 6. Merge to main — your first deploy -`actions/download-artifact` pulls the `docs` artifact `docs.yml` uploaded (the -`docs.zip`); `action-gh-release`'s `files: "*"` then attaches everything in the working -directory — here, just that `docs.zip` — to the tag's Release. +Merging pushes to `main`, which builds `main` and runs `publish`: it assembles a +single-entry `switcher.json` and an `index.html` redirecting to `main/`, and deploys. +Visit `https://ORG.github.io/REPO/` — the redirect lands you on `main/` with the +switcher showing one entry. (The single-entry first deploy is graceful by design; no +release required.) From here on, every push to `main` redeploys and every internal PR +gets its own `/pr-/` preview. ## 7. Cut your first release @@ -167,11 +190,12 @@ directory — here, just that `docs.zip` — to the tag's Release. git tag v1.0.0 && git push origin v1.0.0 ``` -On the next deploy, `assemble` gathers that release, flags it `preferred` (★), -creates the `stable/` alias pointing at it, and the root redirect now targets the -constant `stable/` URL. Your switcher now lists `main` and `1.0.0`, and -`https://ORG.github.io/REPO/stable/` always resolves to the latest release — a -stable URL for cross-project `objects.inv` references. +The tag build runs, the **`release`** job attaches its `docs.zip` to the GitHub Release, +and the next deploy's `assemble` gathers that release, flags it `preferred` (★), creates +the `stable/` alias pointing at it, and points the root redirect at the constant +`stable/` URL. Your switcher now lists `main` and `1.0.0`, and +`https://ORG.github.io/REPO/stable/` always resolves to the latest release — a stable +URL for cross-project `objects.inv` references. ## Where next From 4c550c3baa5b241f9494102035f49b6e1e35d13d Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Thu, 25 Jun 2026 11:38:45 +0000 Subject: [PATCH 8/8] publish: instrument the origin verify with header diagnostics MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Log etag/last-modified/age/x-served-by/x-cache each poll and stretch the window 60s→180s so a wedged Pages origin (etag pinned while age climbs) is unmistakable vs slow CDN propagation. Temporary, to diagnose the tag-after-main publish wedge; revert once we have the data. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/publish.yml | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index be1b7ee..c19f4ae 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -170,20 +170,30 @@ jobs: # comparison is robust to a curl/file trailing-newline difference. want=$(cat "$local_json") url="${PAGE_URL%/}/switcher.json" - deadline=$(( $(date +%s) + 60 )) + # DIAGNOSTIC: log the origin's response headers each poll. A frozen origin + # keeps the SAME etag/last-modified across the whole window while `age` + # climbs (the served artifact never changes), distinguishing a real wedge + # from slow CDN propagation. Window stretched 60→180s purely to make that + # signature unmistakable on this release; revert once we have the data. + hdr() { curl -sI "$url?cb=$(date +%s%N)" \ + | grep -iE '^(HTTP/|etag|last-modified|age|x-served-by|x-cache|x-github-request-id)' \ + | tr -d '\r' | paste -sd' ' -; } + echo "::group::origin baseline headers (before verify loop)"; hdr; echo "::endgroup::" + deadline=$(( $(date +%s) + 180 )) attempt=0 while :; do attempt=$((attempt + 1)) + echo "[attempt $attempt @ $(date -u +%H:%M:%S)] $(hdr)" got=$(curl -fsS "$url?cb=$(date +%s%N)" || true) # cache-bust → hit the origin if [ -n "$got" ] && [ "$got" = "$want" ]; then echo "origin matches the assembled switcher.json (attempt $attempt)" exit 0 fi if [ "$(date +%s)" -ge "$deadline" ]; then - echo "::error::live switcher.json still does not match the assembled one after 60s — Pages origin likely stale/wedged" + echo "::error::live switcher.json still does not match the assembled one after 180s — Pages origin likely stale/wedged" echo "--- assembled ($local_json) ---"; printf '%s\n' "$want" echo "--- live ($url) ---"; printf '%s\n' "$got" exit 1 fi - sleep 1 + sleep 3 done