From 2dd7468fe23269e8f7d7c21f3bc5e06bd61a132f Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Fri, 26 Jun 2026 09:39:42 +0000 Subject: [PATCH 1/2] publish: extract publish-dispatch.yml wrapper; make publish.yml a workflow_call engine publish.yml was both the assemble+deploy engine AND a dispatch entry point (workflow_call + workflow_dispatch with the fork-preview/tag logic), and ci.yml reached it two different ways (nested workflow_call for non-tags, gh-workflow-run of publish.yml for the tag trampoline). That split the dispatch surface oddly and meant this repo dispatched publish.yml directly while consumers used a wrapper. Unify on one wrapper: - publish.yml becomes a pure workflow_call ENGINE (drop on: workflow_dispatch). Its only caller is publish-dispatch.yml. version-name is now optional (default ""). - add publish-dispatch.yml: on {workflow_call, workflow_dispatch}, both just `uses: ./.github/workflows/publish.yml` and thread version-name/guard/pr through, carrying the privileged perms. This repo dogfoods the exact wrapper consumers add (theirs pins publish.yml@; ours uses the local path). - ci.yml: the `publish` job calls publish-dispatch.yml (workflow_call); `publish-tag` does `gh workflow run publish-dispatch.yml`. Both route through the wrapper, never publish.yml directly. Mechanics unchanged: inline non-tag deploys serve fine (first of their SHA); tags deploy via the dispatch trampoline (re-serves a same-SHA deploy). Docs updated (tutorial wrapper now workflow_call+dispatch and ci.yml routes through it; architecture, workflow reference, CLAUDE.md). Co-Authored-By: Claude Opus 4.8 --- .github/workflows/ci.yml | 39 +++++++-------- .github/workflows/publish-dispatch.yml | 60 ++++++++++++++++++++++++ .github/workflows/publish.yml | 53 +++++++++------------ CLAUDE.md | 53 ++++++++++++--------- docs/explanations/architecture.md | 17 ++++--- docs/reference/workflows.md | 37 ++++++++++----- docs/tutorials/adding-to-a-fresh-repo.md | 50 +++++++++++--------- 7 files changed, 197 insertions(+), 112 deletions(-) create mode 100644 .github/workflows/publish-dispatch.yml diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f214072..68d0aae 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -4,13 +4,13 @@ name: CI # forks), pushes to main, and tags — lint, test, build the docs, and on tags # publish the release assets. It uploads each build's `docs` artifact. # -# Publishing is nested here (the `publish` job → publish.yml) so its status is -# visible on the PR / commit, but ONLY for internal NON-TAG events: a fork PR's build -# runs with a read-only token and must never deploy, so the `publish` job is skipped -# for it (the `docs` reusable workflow points a maintainer at the manual opt-in -# instead). TAGS deploy via the `publish-tag` trampoline (workflow_dispatch) instead -# of inline — a same-SHA tag deploy would otherwise be silently dropped by Pages. See -# the architecture explanation in docs/. +# Publishing is nested here (the `publish` job → publish-dispatch.yml → publish.yml) +# so its status is visible on the PR / commit, but ONLY for internal NON-TAG events: a +# fork PR's build runs with a read-only token and must never deploy, so the `publish` +# job is skipped for it (the `docs` reusable workflow points a maintainer at the manual +# opt-in instead). TAGS deploy via the `publish-tag` trampoline (which dispatches +# publish-dispatch.yml) instead of inline — a same-SHA tag deploy would otherwise be +# silently dropped by Pages. See the architecture explanation in docs/. on: pull_request: push: @@ -40,9 +40,10 @@ jobs: # Internal NON-TAG events (an internal PR or a push to main): a same-repo build we # can trust + deploy inline, with the publish status visible on the PR/commit. These - # are the FIRST deploy of their SHA, so the Pages backend serves them fine. - # publish.yml reconstructs the whole site and injects this build's `docs` artifact - # for `docs.version-name`. Fork PRs (head repo ≠ this repo) are excluded — docs.yml's + # are the FIRST deploy of their SHA, so the Pages backend serves them fine. We call + # the publish-dispatch.yml wrapper (its workflow_call path), which nests publish.yml + # to reconstruct the whole site and inject this build's `docs` artifact for + # `docs.version-name`. Fork PRs (head repo ≠ this repo) are excluded — docs.yml's # build job warns them instead. TAGS are excluded too: see publish-tag below. publish: needs: [lint, test, docs] @@ -51,7 +52,7 @@ jobs: github.ref_type != 'tag' && ( github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository ) - uses: ./.github/workflows/publish.yml + uses: ./.github/workflows/publish-dispatch.yml with: version-name: ${{ needs.docs.outputs.version-name }} permissions: @@ -65,12 +66,12 @@ jobs: # the default branch's just-deployed SHA — and the Pages backend silently DROPS a # second same-SHA deploy on a non-`workflow_dispatch` event (reports success, keeps # serving the old artifact; see publish.yml). The escape hatch is that a - # `workflow_dispatch` deploy of that SHA forces a re-serve. So instead of nesting - # publish.yml inline, re-trigger it as a workflow_dispatch (after `release`, so the - # dispatched run gathers the new release's docs.zip). The dispatched run does the - # deploy + verify; its status shows on its own run rather than this tag's CI, which - # is fine — a tag has no PR to annotate. (Consumers do the same by dispatching their - # fork-preview wrapper instead — see the how-to.) + # `workflow_dispatch` deploy of that SHA forces a re-serve. So instead of nesting the + # deploy inline, re-trigger publish-dispatch.yml as a workflow_dispatch (after + # `release`, so the dispatched run gathers the new release's docs.zip). The dispatched + # run does the deploy + verify; its status shows on its own run rather than this tag's + # CI, which is fine — a tag has no PR to annotate. (Consumers do the same by + # dispatching their own publish-dispatch.yml — see the how-to.) publish-tag: needs: [lint, test, docs, release] if: >- @@ -78,11 +79,11 @@ jobs: github.ref_type == 'tag' runs-on: ubuntu-latest permissions: - actions: write # dispatch publish.yml + actions: write # dispatch publish-dispatch.yml steps: - name: Re-deploy via workflow_dispatch (tag trampoline) env: GH_TOKEN: ${{ github.token }} DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} run: | - gh workflow run publish.yml --repo "$GITHUB_REPOSITORY" --ref "$DEFAULT_BRANCH" + gh workflow run publish-dispatch.yml --repo "$GITHUB_REPOSITORY" --ref "$DEFAULT_BRANCH" diff --git a/.github/workflows/publish-dispatch.yml b/.github/workflows/publish-dispatch.yml new file mode 100644 index 0000000..1769169 --- /dev/null +++ b/.github/workflows/publish-dispatch.yml @@ -0,0 +1,60 @@ +name: Publish (dispatch) + +# The ONE wrapper around publish.yml (the assemble + deploy engine). publish.yml is a +# pure `workflow_call` engine; this file is what exposes it, two ways: +# +# workflow_dispatch — the tag trampoline (ci.yml's `publish-tag` does +# `gh workflow run publish-dispatch.yml`), the fork-PR preview +# opt-in (`pr`), and any manual re-deploy. A deploy that runs on +# a `workflow_dispatch` event is what lets a release tag's +# same-SHA deploy actually re-serve, instead of being silently +# dropped by Pages (see publish.yml / the architecture doc). +# workflow_call — ci.yml's `publish` job nests this for internal NON-TAG events +# (an internal PR or a push to the default branch), passing the +# build's `version-name` so the deploy is a visible check on the +# PR/commit. Those are the first deploy of their SHA, so they +# serve fine inline. +# +# This repo DOGFOODS the exact wrapper a consumer adds — theirs differs only in the +# `uses:` target (DiamondLightSource/myst-version-switcher-plugin/.github/workflows/ +# publish.yml@) and is the single place they pin the engine's . +on: + workflow_call: + inputs: + version-name: + description: Version name of an in-run build to inject (pr- | | ); "" → pure durable gather. + required: false + default: "" + type: string + guard-default-branch: + description: "'false' to skip the default-branch-present guard (e.g. mid gh-pages migration)." + required: false + default: "true" + type: string + pr: + description: External fork PR number to approve + preview. + required: false + default: "" + type: string + workflow_dispatch: + inputs: + pr: + description: External fork PR number to approve (pins its head SHA) and preview. Leave empty to just re-deploy. + required: false + default: "" + +jobs: + publish: + # Grant the engine the privileged scopes it needs. On workflow_dispatch these are + # the run's token scopes; on workflow_call they're capped by ci.yml's grant. + permissions: + contents: read # checkout + read release assets + actions: read # gh run download (this run's + cross-run docs artifacts) + pages: write # deploy to Pages + id-token: write # deploy-pages OIDC + statuses: write # set the preview-approved status on a fork PR head SHA + uses: ./.github/workflows/publish.yml + with: + version-name: ${{ inputs.version-name }} + guard-default-branch: ${{ inputs.guard-default-branch || 'true' }} + pr: ${{ inputs.pr }} diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 8ba57c1..96f9e6d 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -3,21 +3,19 @@ name: Publish # Reconstruct the WHOLE versioned docs site from durable sources (the consumer's # default-branch build, release docs.zip assets, open-PR build artifacts) and # deploy it directly to GitHub Pages — no gh-pages branch. This is the PRIVILEGED -# half (pages/id-token/statuses write), kept in its own reusable workflow so it can -# only run two trusted ways: +# half (pages/id-token/statuses write) and a pure ENGINE: its only entry point is +# `workflow_call`, and the ONLY thing that calls it is a `publish-dispatch.yml` +# wrapper (this repo's own, or a consumer's pinned at ). That wrapper exposes the +# engine as both `workflow_dispatch` (the tag trampoline + fork-PR opt-in + manual +# re-deploy) and `workflow_call` (ci.yml's inline publish for internal non-tag +# events). Keeping the dispatch surface in the wrapper means this engine has a single +# trusted caller and no event-type branching of its own. # -# workflow_call — invoked by the consumer's ci.yml AFTER a successful build, for -# INTERNAL events only (internal PRs, pushes to the default -# branch/tags). The caller never calls it for a fork PR, so -# untrusted fork code never reaches a write token. The current -# build isn't a *completed* CI run yet, so the caller passes its -# `version-name`; this run downloads the same-run `docs` artifact -# and stages it directly (otherwise a default-branch/tag push -# would publish the PREVIOUS build). A consumer's manual fork-PR -# preview also comes through here — a workflow_dispatch wrapper in -# their repo calls this with `pr: ` (and version-name: ""). -# workflow_dispatch — this repo's own maintainer opt-in to preview an EXTERNAL fork -# PR (the local, non-cross-repo path). +# `version-name` (set by ci.yml's inline path, threaded through the wrapper) names an +# in-run build to stage directly — the run isn't a *completed* CI run yet, so the +# gather can't find it (or would find a stale prior build); this run downloads that +# run's `docs` artifact and stages it. The dispatch paths pass version-name "" → pure +# durable gather. `pr` (fork opt-in) pins + previews a fork PR's head SHA. # # The assemble logic is THIS repo's assemble/assemble.sh (+ assemble.mjs); we run it # directly rather than wrap it in a composite action. To version-match the scripts @@ -34,8 +32,9 @@ on: workflow_call: inputs: version-name: - description: Version name (pr- | | ) of the in-run build to inject. - required: true + description: Version name (pr- | | ) of the in-run build to inject; "" → pure durable gather. + required: false + default: "" type: string guard-default-branch: description: >- @@ -48,16 +47,10 @@ on: pr: description: >- External fork PR number to approve (pins its head SHA) and preview. Set this - (with version-name: "") from a consumer's workflow_dispatch wrapper, since a - reusable workflow can't be workflow_dispatch'd cross-repo directly. + (with version-name: "") from the publish-dispatch.yml wrapper's dispatch path. required: false default: "" type: string - workflow_dispatch: - inputs: - pr: - description: External fork PR number to approve (pins its head SHA) and preview. - required: false permissions: contents: read # checkout + read release assets @@ -72,10 +65,9 @@ concurrency: jobs: publish: - # The canonical-repo guard lives in the caller (its publish job's `if`), not - # here, so this reusable workflow stays generic. Both entry points are trusted: - # the caller only calls it for internal events, and workflow_dispatch needs write - # access (a fork dispatching it deploys to the fork's own Pages). + # The canonical-repo guard lives upstream (ci.yml's publish job `if`, and the + # wrapper's dispatch needs write access), not here, so this engine stays generic. + # Its single caller is publish-dispatch.yml, which only reaches it on trusted paths. runs-on: ubuntu-latest environment: name: github-pages @@ -101,9 +93,8 @@ jobs: # Fork opt-in: pin the PR's head commit as approved. assemble gathers an external # PR only when its head SHA carries this status, so a later push (new SHA) silently - # drops the preview until re-approved. Gated on `pr` alone (not event_name) so it - # works both from this repo's own workflow_dispatch and from a consumer's dispatch - # wrapper that passes `pr` through workflow_call. + # drops the preview until re-approved. Gated on `pr` alone, which the + # publish-dispatch.yml wrapper threads in from its dispatch input. - name: Approve fork PR head SHA if: inputs.pr != '' env: @@ -120,7 +111,7 @@ jobs: # When publishing within the build's own run (workflow_call), download this # run's `docs` artifact so assemble can stage it directly as version-name — # the run isn't a completed success yet, so the gather can't find it (or would - # find a stale prior build). Skipped on workflow_dispatch (version-name empty). + # find a stale prior build). Skipped when version-name is "" (the dispatch paths). - name: Download this run's docs artifact if: inputs.version-name != '' uses: actions/download-artifact@v8 diff --git a/CLAUDE.md b/CLAUDE.md index b1c6c0e..578935f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -16,8 +16,9 @@ scripts/migrate.sh # operator gh-pages → durable-s test/ # npm test suite (node, no framework) docs/ # this repo's own docs (dogfoods the plugin) .github/workflows/docs.yml # PUBLIC reusable: build at the versioned BASE_URL → pack docs.zip → upload `docs` artifact (workflow_call; build-command input) -.github/workflows/publish.yml # PUBLIC reusable: run assemble/ (checked out at job.workflow_sha) + deploy to Pages (PRIVILEGED): workflow_call (internal non-tag) + workflow_dispatch (tag trampoline + fork-PR opt-in) -.github/workflows/ci.yml # this repo's own entry: _lint / _test / docs.yml / _release, then nests publish.yml for internal non-tag events (tags trampoline via publish-tag → workflow_dispatch) +.github/workflows/publish.yml # PUBLIC reusable ENGINE: run assemble/ (checked out at job.workflow_sha) + deploy to Pages (PRIVILEGED). workflow_call ONLY; its sole caller is publish-dispatch.yml +.github/workflows/publish-dispatch.yml # PUBLIC wrapper around publish.yml: workflow_call (ci.yml's inline publish) + workflow_dispatch (tag trampoline + fork-PR opt-in + manual re-deploy). Dogfoods the wrapper consumers add +.github/workflows/ci.yml # this repo's own entry: _lint / _test / docs.yml / _release, then nests publish-dispatch.yml for internal non-tag events (tags trampoline via publish-tag → gh workflow run publish-dispatch.yml) ``` ## Two halves, different lifecycles @@ -153,22 +154,27 @@ One entry workflow (`ci.yml`) that nests the privileged publish: fork-PR warning is a step inside `docs.yml`'s build job (not a separate ci.yml job), keeping the top-level workflow lean. The `publish` job's `if` also excludes **tags** (`ref_type != 'tag'`) — a separate `publish-tag` job (tags only, `needs: - [_release]`, `actions: write`) re-triggers `publish.yml` via - `gh workflow run publish.yml --ref ` (`workflow_dispatch`), because - a same-SHA tag deploy is silently dropped by Pages unless it's a dispatch event - (deploy-pages#383). -- `publish.yml` — **assemble + deploy, privileged.** `workflow_call` (the nested - internal non-tag path) + `workflow_dispatch` (the tag trampoline + the fork-PR - opt-in). Sparse-checks-out this - repo's `assemble/` at `job.workflow_sha` (so the scripts match the ref the consumer - pinned `publish.yml@` to — see "Self-referencing the assemble scripts" below), - runs `assemble.sh`, then `upload-pages-artifact` + `deploy-pages`, carrying the + [_release]`, `actions: write`) re-triggers the deploy via + `gh workflow run publish-dispatch.yml --ref ` (`workflow_dispatch`), + because a same-SHA tag deploy is silently dropped by Pages unless it's a dispatch + event (deploy-pages#383). Both the inline `publish` job and `publish-tag` route + through `publish-dispatch.yml`, never `publish.yml` directly. +- `publish-dispatch.yml` — **the one wrapper around the engine.** `workflow_call` + (ci.yml's inline `publish` job) + `workflow_dispatch` (the tag trampoline + fork-PR + opt-in + manual re-deploy); both just `uses: publish.yml` and pass `version-name` / + `pr` / `guard-default-branch` through, carrying the privileged perms. This repo + dogfoods the exact wrapper consumers add (theirs pins `publish.yml@`; this one + uses the local path). +- `publish.yml` — **assemble + deploy ENGINE, privileged.** `workflow_call` ONLY + (sole caller: `publish-dispatch.yml`). Sparse-checks-out this repo's `assemble/` at + `job.workflow_sha` (so the scripts match the ref the consumer pinned + `publish.yml@` to — see "Self-referencing the assemble scripts" below), runs + `assemble.sh`, then `upload-pages-artifact` + `deploy-pages`, carrying the `github-pages` environment + `pages`/`id-token`/`statuses` perms + `concurrency`. - Its job has **no `if`** — the canonical-repo guard lives in ci.yml's `publish` job, - and both entry points are trusted (ci.yml gates the call; dispatch needs write - access). - On `workflow_call` it injects the in-run build via `artifact-version-name` (see the - split design decision). See the architecture explanation in docs/. + Its job has **no `if`** — the canonical-repo guard lives upstream in ci.yml's + `publish` job + the wrapper's dispatch (which needs write access). + On `workflow_call` with a non-empty `version-name` it injects the in-run build (see + the split design decision). See the architecture explanation in docs/. Sub-workflows of `ci.yml`: - `_lint.yml` — biome @@ -271,7 +277,7 @@ jobs: github.ref_type != 'tag' && ( github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository ) - uses: DiamondLightSource/myst-version-switcher-plugin/.github/workflows/publish.yml@ + uses: ./.github/workflows/publish-dispatch.yml # your wrapper; it pins publish.yml@ with: version-name: ${{ needs.docs.outputs.version-name }} permissions: { pages: write, id-token: write, contents: read, actions: read, statuses: write } @@ -287,11 +293,12 @@ jobs: Add a `_release.yml`-style job that attaches each tag's built `docs.zip` (bare `html/` root) + the plugin as Release assets so `assemble` can reconstruct released -versions, and a small `publish-dispatch.yml` `workflow_dispatch` wrapper around -`publish.yml` (reusable workflows can't be dispatched cross-repo). The wrapper is the -**tag trampoline** target (`publish-tag` dispatches it so the deploy runs as a -`workflow_dispatch` — required for a release's same-SHA deploy to actually go live) -**and** the fork-PR preview opt-in (pass `pr`). See the full how-to + snippets in +versions, and a small `publish-dispatch.yml` wrapper around `publish.yml` — the single +caller of the engine and the one place you pin `publish.yml@`. It's +`workflow_call` (your `publish` job nests it for the inline non-tag deploy) **and** +`workflow_dispatch` (the **tag trampoline** target — `publish-tag` dispatches it so the +deploy runs as a `workflow_dispatch`, required for a release's same-SHA deploy to go +live — plus the fork-PR preview opt-in via `pr`). See the full how-to + snippets in `docs/`. ## Upstreaming diff --git a/docs/explanations/architecture.md b/docs/explanations/architecture.md index 73aa910..bf3f110 100644 --- a/docs/explanations/architecture.md +++ b/docs/explanations/architecture.md @@ -107,14 +107,19 @@ So a fork's build can never reach a write token; only trusted code deploys. ### Why publish is *nested* in CI (for internal events) The deploy is surfaced as a **job inside the CI run** (`ci.yml`'s `publish` job → -`publish.yml` via `workflow_call`) so its status and URL are visible on the PR / -commit — rather than running invisibly after the fact. But this is gated to -**internal events only**: the `publish` job's `if` excludes fork PRs +`publish-dispatch.yml` → `publish.yml` via `workflow_call`) so its status and URL are +visible on the PR / commit — rather than running invisibly after the fact. But this is +gated to **internal events only**: the `publish` job's `if` excludes fork PRs (`head.repo.full_name != github.repository`). A fork PR's build instead emits a warning (a step in the build job) that the preview was not published, linking the -manual opt-in. The privileged `publish.yml` is therefore reachable two ways, both -trusted: `workflow_call` (nested, internal) and `workflow_dispatch` (the maintainer -fork opt-in). +manual opt-in. + +`publish.yml` itself is a pure `workflow_call` **engine** with a single caller — +`publish-dispatch.yml`, the one wrapper that exposes it as both `workflow_call` +(ci.yml's inline publish) and `workflow_dispatch` (the tag trampoline, the fork-PR +opt-in, and manual re-deploys). Keeping the dispatch surface in the wrapper means the +engine has no event-type branching of its own, and this repo dogfoods the exact wrapper +a consumer adds (theirs just pins `publish.yml@`). The cost of nesting is an environment-policy change: because internal PRs now deploy from **their own ref**, the `github-pages` environment's deployment-branch policy must diff --git a/docs/reference/workflows.md b/docs/reference/workflows.md index 2b3e25c..c7154f9 100644 --- a/docs/reference/workflows.md +++ b/docs/reference/workflows.md @@ -37,20 +37,13 @@ preinstalled Node, so `build-command` can be `make` / `npx` / `tox` / `npm` driv |---|---| | `version-name` | The version this build was served at (`pr-` \| default-branch \| ``) — pass to `publish.yml`. | -## `publish.yml` — assemble + deploy (privileged) +## `publish.yml` — assemble + deploy engine (privileged) Reconstructs the whole site and deploys it to Pages. Carries the `github-pages` environment, `concurrency: pages`, and `pages`/`id-token`/`statuses` write permissions. -Reachable two ways: `workflow_call` (nested by the consumer's `ci.yml` for internal -**non-tag** events) and `workflow_dispatch`. The dispatch path covers both the tag -trampoline and the fork-PR opt-in — **tags must deploy via `workflow_dispatch`**, since -GitHub Pages silently drops a second deploy of an already-deployed SHA (a release tag -shares the merge commit's SHA) unless the event is `workflow_dispatch` -([`actions/deploy-pages#383`](https://github.com/actions/deploy-pages/issues/383); see -the [architecture explanation](../explanations/architecture.md)). Consumers dispatch a -small `publish-dispatch.yml` wrapper (a reusable workflow can't be dispatched -cross-repo). The canonical-repo guard lives in the **caller**, so the workflow stays -generic. +It's a pure **`workflow_call` engine** with a single caller: the `publish-dispatch.yml` +wrapper (below). The canonical-repo guard lives upstream (in `ci.yml` / the wrapper's +dispatch), so the engine stays generic. It self-checks-out this repo's `assemble/` scripts at `job.workflow_sha` / `job.workflow_repository` (the `job` context resolves to the reusable file, not the @@ -58,9 +51,29 @@ caller), so the scripts match the `` the consumer pinned — automatically, version bump. The consumer's repo stays checked out at the root so `assemble.mjs`'s `git tag` lists *their* versions. +## `publish-dispatch.yml` — the wrapper (one per repo) + +The only thing that calls `publish.yml`, and the single place you pin +`publish.yml@`. It exposes the engine two ways: + +- `workflow_call` — `ci.yml`'s `publish` job nests it for internal **non-tag** events + (inline deploy, status on the PR/commit), threading `version-name` through. +- `workflow_dispatch` — the **tag trampoline** (`ci.yml`'s `publish-tag` does + `gh workflow run publish-dispatch.yml`), the fork-PR opt-in (`pr`), and manual + re-deploys. + +**Tags must deploy via `workflow_dispatch`**: GitHub Pages silently drops a second +deploy of an already-deployed SHA (a release tag shares the merge commit's SHA) unless +the event is `workflow_dispatch` +([`actions/deploy-pages#383`](https://github.com/actions/deploy-pages/issues/383); see +the [architecture explanation](../explanations/architecture.md)). A reusable workflow +can't be `workflow_dispatch`'d cross-repo, which is why the wrapper lives in each repo. + +### `publish.yml` inputs (threaded through by the wrapper) + | input | required | default | meaning | |---|---|---|---| -| `version-name` | yes (`workflow_call`) | — | Version name of **this run's** `docs` artifact to stage directly, instead of gathering it from durable sources. Used when publishing inside the build's own run (the run isn't a completed success yet, so the gather can't discover it — or would find a stale previous build). Empty → pure durable gather. | +| `version-name` | no | `""` | Version name of **this run's** `docs` artifact to stage directly, instead of gathering it from durable sources. Set by `ci.yml`'s inline publish (the run isn't a completed success yet, so the gather can't discover it — or would find a stale previous build). Empty → pure durable gather (the dispatch paths). | | `guard-default-branch` | no | `true` | When `true`, hard-fail if the consumer's default branch is absent from the site (its build artifact expired). Set `false` while a repo's default branch isn't yet publishing `docs.zip` (mid-migration). | ## What `publish.yml` gathers diff --git a/docs/tutorials/adding-to-a-fresh-repo.md b/docs/tutorials/adding-to-a-fresh-repo.md index 9489446..6091301 100644 --- a/docs/tutorials/adding-to-a-fresh-repo.md +++ b/docs/tutorials/adding-to-a-fresh-repo.md @@ -97,7 +97,7 @@ jobs: github.ref_type != 'tag' && ( github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository ) - uses: DiamondLightSource/myst-version-switcher-plugin/.github/workflows/publish.yml@ + uses: ./.github/workflows/publish-dispatch.yml # your wrapper (below); it pins publish.yml@ with: version-name: ${{ needs.docs.outputs.version-name }} permissions: @@ -139,33 +139,40 @@ internally — it checks out this project's `assemble` scripts at the same ``. It exposes the engine two ways and so covers all three publish paths: + +- **Inline publish (`workflow_call`)** — your `publish` job above nests it for internal + non-tag events, passing `version-name`, so the deploy is a visible check on the + PR/commit. +- **Tag trampoline (`workflow_dispatch`, no `pr`)** — `publish-tag` above dispatches it + on a tag. Deploying as a `workflow_dispatch` event is what makes a same-SHA tag + deploy actually go live (an inline tag deploy is silently dropped — see the + [architecture explanation](../explanations/architecture.md)). **This is why the + wrapper is required, not optional.** +- **Fork-PR preview (`workflow_dispatch`, with `pr`)** — internal PRs publish + automatically; fork PRs build and verify but never auto-deploy. A maintainer runs it + from the Actions tab with a PR number to approve that fork's head SHA and deploy a + preview (a later push to the PR drops it until re-run). ```yaml # .github/workflows/publish-dispatch.yml name: Publish (dispatch) on: - workflow_dispatch: + workflow_call: # ci.yml's inline `publish` job + inputs: + version-name: { required: false, default: "", type: string } + pr: { required: false, default: "", type: string } + workflow_dispatch: # tag trampoline + fork-PR preview + manual re-deploy inputs: pr: { description: "Fork PR number to approve + preview (leave empty to just re-deploy)", required: false, default: "" } jobs: publish: uses: DiamondLightSource/myst-version-switcher-plugin/.github/workflows/publish.yml@ with: - version-name: "" # no in-run build to inject → pure durable gather - pr: ${{ inputs.pr }} # empty → plain re-deploy; set → pin that fork head SHA as preview-approved + version-name: ${{ inputs.version-name }} # "" on dispatch → pure durable gather + pr: ${{ inputs.pr }} # empty → plain re-deploy; set → pin that fork head SHA permissions: contents: read actions: read @@ -179,10 +186,11 @@ If you point `docs.yml`'s fork-preview warning at this file, also set ## 4. Allow the deploying refs in the `github-pages` environment -Because internal PRs and tags now deploy from **their own ref** (the publish job -runs inside their CI run), the `github-pages` environment's deployment policy must -allow those refs. In **Settings → Environments → github-pages**, it is recommended -to set **Deployment branches and tags** to **No restriction**. +Because internal PRs deploy from **their own ref** (the publish job runs inside their +CI run; tags deploy from the default-branch ref via the trampoline), the +`github-pages` environment's deployment policy must allow those refs. In **Settings → +Environments → github-pages**, it is recommended to set **Deployment branches and +tags** to **No restriction**. ## 5. Push your branch and open a PR From 28195fba4e54baa681b467d66c38526226af3269 Mon Sep 17 00:00:00 2001 From: Tom Cobb Date: Fri, 26 Jun 2026 09:54:49 +0000 Subject: [PATCH 2/2] =?UTF-8?q?publish-dispatch:=20own=20all=20branching?= =?UTF-8?q?=20(tag=20trampoline=20+=20fork=20warn)=20=E2=80=94=20slim=20ci?= =?UTF-8?q?.yml=20&=20docs.yml?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fold the tag trampoline and the fork-PR preview hint into the wrapper so a consumer's ci.yml is a single `publish` job and docs.yml carries no fork logic. publish-dispatch.yml now branches on the originating event into three jobs: - deploy — internal PR / default-branch push (inline, uses publish.yml), or any workflow_dispatch (the trampoline's re-dispatch / fork preview / manual). - trampoline — a tag push: wait for the parallel release job to attach docs.zip, then `gh workflow run publish-dispatch.yml` so the deploy is a workflow_dispatch that re-serves (a same-SHA tag deploy is otherwise dropped — #383). - warn — a fork PR: read-only, never deploys, posts the manual-opt-in hint (moved here from docs.yml's build job). - ci.yml: ONE `publish` job (`if: repository == …`) for every event, `actions: write` added for the trampoline; drop the separate `publish-tag` job and the ref/fork guards. - docs.yml: drop the fork-preview warning step and the `preview-workflow` input. Security boundary preserved: a fork PR's token is read-only and the `deploy` job is if-excluded for forks, so untrusted code never reaches a write token — only `warn` runs. Docs updated (tutorial: one publish job + full wrapper; architecture; reference; CLAUDE.md). Co-Authored-By: Claude Opus 4.8 --- .github/workflows/ci.yml | 61 +++-------- .github/workflows/docs.yml | 23 +---- .github/workflows/publish-dispatch.yml | 89 ++++++++++++---- CLAUDE.md | 123 ++++++++++------------- docs/explanations/architecture.md | 29 +++--- docs/reference/workflows.md | 20 ++-- docs/tutorials/adding-to-a-fresh-repo.md | 123 ++++++++++++----------- 7 files changed, 230 insertions(+), 238 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 68d0aae..1ac7d19 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -4,13 +4,12 @@ name: CI # forks), pushes to main, and tags — lint, test, build the docs, and on tags # publish the release assets. It uploads each build's `docs` artifact. # -# Publishing is nested here (the `publish` job → publish-dispatch.yml → publish.yml) -# so its status is visible on the PR / commit, but ONLY for internal NON-TAG events: a -# fork PR's build runs with a read-only token and must never deploy, so the `publish` -# job is skipped for it (the `docs` reusable workflow points a maintainer at the manual -# opt-in instead). TAGS deploy via the `publish-tag` trampoline (which dispatches -# publish-dispatch.yml) instead of inline — a same-SHA tag deploy would otherwise be -# silently dropped by Pages. See the architecture explanation in docs/. +# Publishing is nested here (the `publish` job → publish-dispatch.yml → publish.yml) so +# its status is visible on the PR / commit. A single `publish` job covers every event: +# publish-dispatch.yml branches internally into deploy (internal PR / main push), tag +# trampoline (a same-SHA tag deploy would be silently dropped by Pages unless dispatched), +# or a fork-PR warning (forks build with a read-only token and never deploy). See the +# architecture explanation in docs/. on: pull_request: push: @@ -38,52 +37,22 @@ jobs: permissions: contents: write # create the GitHub Release + attach assets - # Internal NON-TAG events (an internal PR or a push to main): a same-repo build we - # can trust + deploy inline, with the publish status visible on the PR/commit. These - # are the FIRST deploy of their SHA, so the Pages backend serves them fine. We call - # the publish-dispatch.yml wrapper (its workflow_call path), which nests publish.yml - # to reconstruct the whole site and inject this build's `docs` artifact for - # `docs.version-name`. Fork PRs (head repo ≠ this repo) are excluded — docs.yml's - # build job warns them instead. TAGS are excluded too: see publish-tag below. + # One publish job for every event — publish-dispatch.yml owns the branching (deploy + # inline / tag trampoline / fork-PR warn), so there's no separate tag or fork job and + # no ref/fork guards here. It runs for fork PRs too, but only to surface the opt-in + # hint: the wrapper's `deploy` job excludes forks and a fork PR's token is read-only, + # so untrusted code never reaches a write token. `actions: write` lets the wrapper's + # tag trampoline re-dispatch itself; `version-name` is the in-run build for the inline + # path (ignored by the trampoline/warn branches). publish: needs: [lint, test, docs] - if: >- - github.repository == 'DiamondLightSource/myst-version-switcher-plugin' && - github.ref_type != 'tag' && - ( github.event_name != 'pull_request' || - github.event.pull_request.head.repo.full_name == github.repository ) + if: github.repository == 'DiamondLightSource/myst-version-switcher-plugin' uses: ./.github/workflows/publish-dispatch.yml with: version-name: ${{ needs.docs.outputs.version-name }} permissions: contents: read - actions: read + actions: write # the wrapper's tag trampoline dispatches via gh workflow run pages: write id-token: write statuses: write - - # Tags can't deploy inline. A release tag is cut on the merge commit, so it shares - # the default branch's just-deployed SHA — and the Pages backend silently DROPS a - # second same-SHA deploy on a non-`workflow_dispatch` event (reports success, keeps - # serving the old artifact; see publish.yml). The escape hatch is that a - # `workflow_dispatch` deploy of that SHA forces a re-serve. So instead of nesting the - # deploy inline, re-trigger publish-dispatch.yml as a workflow_dispatch (after - # `release`, so the dispatched run gathers the new release's docs.zip). The dispatched - # run does the deploy + verify; its status shows on its own run rather than this tag's - # CI, which is fine — a tag has no PR to annotate. (Consumers do the same by - # dispatching their own publish-dispatch.yml — see the how-to.) - publish-tag: - needs: [lint, test, docs, release] - if: >- - github.repository == 'DiamondLightSource/myst-version-switcher-plugin' && - github.ref_type == 'tag' - runs-on: ubuntu-latest - permissions: - actions: write # dispatch publish-dispatch.yml - steps: - - name: Re-deploy via workflow_dispatch (tag trampoline) - env: - GH_TOKEN: ${{ github.token }} - DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} - run: | - gh workflow run publish-dispatch.yml --repo "$GITHUB_REPOSITORY" --ref "$DEFAULT_BRANCH" diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 14a8972..35f0f78 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -13,11 +13,6 @@ on: required: false default: docs/_build/html type: string - preview-workflow: - description: Consumer's dispatchable publish workflow file, for the fork-preview hint. - required: false - default: publish.yml - type: string outputs: version-name: description: The version name this build was served at (pr- | | ). @@ -109,18 +104,6 @@ jobs: path: ${{ runner.temp }}/docs.zip compression-level: 0 - # Fork PRs build + verify here but do not auto-publish (the consumer's publish - # job excludes them — the security boundary). Surface that on the PR with a link - # to the manual preview opt-in, rather than silently publishing nothing. - - name: Explain the fork-preview opt-in - if: >- - github.event_name == 'pull_request' && - github.event.pull_request.head.repo.full_name != github.repository - env: - PREVIEW_WORKFLOW: ${{ inputs.preview-workflow }} - run: | - echo "::warning title=Docs preview not published::This is a fork PR, so the \ - versioned docs site is NOT auto-published (fork builds run with a read-only \ - token). A maintainer can publish a preview by running the $PREVIEW_WORKFLOW \ - workflow for PR #${{ github.event.pull_request.number }}: \ - https://github.com/${{ github.repository }}/actions/workflows/$PREVIEW_WORKFLOW" + # NOTE: the fork-PR "preview not published" hint now lives in publish-dispatch.yml + # (its `warn` job), which runs for fork PRs too — so this build job stays purely + # about building + uploading the docs artifact. diff --git a/.github/workflows/publish-dispatch.yml b/.github/workflows/publish-dispatch.yml index 1769169..981451f 100644 --- a/.github/workflows/publish-dispatch.yml +++ b/.github/workflows/publish-dispatch.yml @@ -1,23 +1,29 @@ name: Publish (dispatch) -# The ONE wrapper around publish.yml (the assemble + deploy engine). publish.yml is a -# pure `workflow_call` engine; this file is what exposes it, two ways: +# The ONE wrapper around publish.yml (the assemble + deploy engine) — it owns ALL the +# publish decisions so a consumer's ci.yml needs a single `publish` job (no separate +# tag or fork-warning job) and docs.yml needs no fork-preview hint. publish.yml is a +# pure `workflow_call` engine; this file branches on the originating event into one of +# three jobs: # -# workflow_dispatch — the tag trampoline (ci.yml's `publish-tag` does -# `gh workflow run publish-dispatch.yml`), the fork-PR preview -# opt-in (`pr`), and any manual re-deploy. A deploy that runs on -# a `workflow_dispatch` event is what lets a release tag's -# same-SHA deploy actually re-serve, instead of being silently -# dropped by Pages (see publish.yml / the architecture doc). -# workflow_call — ci.yml's `publish` job nests this for internal NON-TAG events -# (an internal PR or a push to the default branch), passing the -# build's `version-name` so the deploy is a visible check on the -# PR/commit. Those are the first deploy of their SHA, so they -# serve fine inline. +# deploy — internal PR / default-branch push (via ci.yml's workflow_call), or any +# workflow_dispatch (the trampoline's re-dispatch, a fork-PR preview, a +# manual re-deploy). Nests publish.yml; for the inline path it injects the +# in-run build via `version-name`, otherwise pure durable gather. +# trampoline — a TAG push (via workflow_call). An inline tag deploy would be silently +# DROPPED by Pages (it reuses the default branch's just-deployed SHA on a +# non-dispatch event — deploy-pages#383), so re-dispatch THIS workflow as +# `workflow_dispatch`; that event forces a re-serve. Waits for ci.yml's +# parallel release job to attach the tag's docs.zip first (the dispatched +# run gathers the tag from that durable asset). +# warn — a FORK PR. Fork builds run with a read-only token and must never deploy, +# so just surface the manual-opt-in hint on the PR. (The token is +# read-only here regardless — `deploy` is `if`-excluded for forks too, and +# the workflow file itself comes from the trusted base, not the fork.) # # This repo DOGFOODS the exact wrapper a consumer adds — theirs differs only in the -# `uses:` target (DiamondLightSource/myst-version-switcher-plugin/.github/workflows/ -# publish.yml@) and is the single place they pin the engine's . +# `deploy` job's `uses:` target (DiamondLightSource/myst-version-switcher-plugin/ +# .github/workflows/publish.yml@), the single place they pin the engine's . on: workflow_call: inputs: @@ -44,9 +50,13 @@ on: default: "" jobs: - publish: - # Grant the engine the privileged scopes it needs. On workflow_dispatch these are - # the run's token scopes; on workflow_call they're capped by ci.yml's grant. + # Internal inline (PR / default-branch push) or any workflow_dispatch → deploy. + deploy: + if: >- + github.event_name == 'workflow_dispatch' || + (github.event_name == 'push' && github.ref_type != 'tag') || + (github.event_name == 'pull_request' && + github.event.pull_request.head.repo.full_name == github.repository) permissions: contents: read # checkout + read release assets actions: read # gh run download (this run's + cross-run docs artifacts) @@ -58,3 +68,46 @@ jobs: version-name: ${{ inputs.version-name }} guard-default-branch: ${{ inputs.guard-default-branch || 'true' }} pr: ${{ inputs.pr }} + + # A tag can't deploy inline — re-dispatch this workflow so the deploy runs on a + # workflow_dispatch event (which forces Pages to re-serve a same-SHA deploy). + trampoline: + if: github.event_name == 'push' && github.ref_type == 'tag' + runs-on: ubuntu-latest + permissions: + contents: read # gh release view + actions: write # gh workflow run (re-dispatch self) + steps: + - name: Wait for the release, then re-deploy via workflow_dispatch + env: + GH_TOKEN: ${{ github.token }} + REPO: ${{ github.repository }} + TAG: ${{ github.ref_name }} + DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} + run: | + set -euo pipefail + # ci.yml's release job attaches the tag's docs.zip in parallel; the dispatched + # run gathers the tag from that asset, so don't dispatch until it exists. + for _ in $(seq 1 36); do + gh release view "$TAG" --repo "$REPO" >/dev/null 2>&1 && break + sleep 5 + done + gh release view "$TAG" --repo "$REPO" >/dev/null 2>&1 \ + || { echo "::error::release $TAG not found after 3 min — did the release job fail?"; exit 1; } + gh workflow run publish-dispatch.yml --repo "$REPO" --ref "$DEFAULT_BRANCH" + + # Fork PR: never deploys (read-only token). Surface the manual-opt-in hint on the PR. + warn: + if: >- + github.event_name == 'pull_request' && + github.event.pull_request.head.repo.full_name != github.repository + runs-on: ubuntu-latest + permissions: {} + steps: + - name: Explain the fork-preview opt-in + run: | + echo "::warning title=Docs preview not published::This is a fork PR, so the \ + versioned docs site is NOT auto-published (fork builds run with a read-only \ + token). A maintainer can publish a preview by running the publish-dispatch.yml \ + workflow for PR #${{ github.event.pull_request.number }}: \ + https://github.com/${{ github.repository }}/actions/workflows/publish-dispatch.yml" diff --git a/CLAUDE.md b/CLAUDE.md index 578935f..834ab95 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -18,7 +18,7 @@ docs/ # this repo's own docs (dogfoods .github/workflows/docs.yml # PUBLIC reusable: build at the versioned BASE_URL → pack docs.zip → upload `docs` artifact (workflow_call; build-command input) .github/workflows/publish.yml # PUBLIC reusable ENGINE: run assemble/ (checked out at job.workflow_sha) + deploy to Pages (PRIVILEGED). workflow_call ONLY; its sole caller is publish-dispatch.yml .github/workflows/publish-dispatch.yml # PUBLIC wrapper around publish.yml: workflow_call (ci.yml's inline publish) + workflow_dispatch (tag trampoline + fork-PR opt-in + manual re-deploy). Dogfoods the wrapper consumers add -.github/workflows/ci.yml # this repo's own entry: _lint / _test / docs.yml / _release, then nests publish-dispatch.yml for internal non-tag events (tags trampoline via publish-tag → gh workflow run publish-dispatch.yml) +.github/workflows/ci.yml # this repo's own entry: _lint / _test / docs.yml / _release, then ONE publish job nesting publish-dispatch.yml for every event (the wrapper branches: deploy / tag-trampoline / fork-warn) ``` ## Two halves, different lifecycles @@ -66,17 +66,15 @@ gathered (a merged/closed PR, a deleted release) is correctly dropped — no ### Split build (unprivileged) from publish (privileged), but nest publish for visibility `ci.yml` builds + uploads the `docs` artifact for every event including fork PRs. -It then nests the privileged `publish.yml` (`workflow_call`) as a `publish` job — -but **only for internal, non-tag events** (an internal PR, or a push to main): the -job's `if` excludes fork PRs (`head.repo.full_name != github.repository`), whose -builds run with a read-only token and must never deploy. So publish status is -**visible on the PR/commit** for internal work, while untrusted fork-PR code still -never reaches a write token. A fork PR instead gets a warning step in `docs.yml`'s -build job that links the manual opt-in (`publish.yml`'s `workflow_dispatch`). -**Tags don't nest** — a tag shares the merge commit's SHA and GitHub Pages drops a -second same-SHA deploy unless it's a `workflow_dispatch` (deploy-pages#383), so the -`publish-tag` job re-triggers `publish.yml` via `workflow_dispatch` instead (see -the Publish flow below). +It then nests a single privileged `publish` job (`uses: publish-dispatch.yml`) for +**every** event in the canonical repo. `publish-dispatch.yml` branches: an internal +PR / main push deploys inline (status **visible on the PR/commit**); a fork PR hits a +read-only **warn** job (untrusted fork-PR code never reaches a write token — the +`deploy` job is `if`-excluded for forks and the token is read-only anyway); a **tag** +hits the **trampoline** (a tag shares the merge commit's SHA and Pages drops a second +same-SHA deploy unless it's a `workflow_dispatch` — deploy-pages#383 — so it +re-dispatches `publish-dispatch.yml`). One `publish` job, no ref/fork guards in +`ci.yml`. Because the inline publish runs **inside the build's own CI run**, the current build isn't a *completed* successful run the gather can discover (and for a main push the @@ -146,25 +144,22 @@ One entry workflow (`ci.yml`) that nests the privileged publish: - `ci.yml` — **build + verify, then publish on internal events.** Triggers on `pull_request` + push to `main`/tags (no other-branch pushes; `*` excludes `/`-tags). Orchestrates `_lint` / `_test` / `_docs` / `_release`, runs for forks, - and uploads the `docs` artifact. It then nests the `publish` job - (`uses: publish.yml`) — runs only for the **canonical repo on internal** events - (the `if` is `repository == '…' && (event != pull_request || - head.repo.full_name == github.repository)`); grants `pages`/`id-token`/`statuses` - perms to the call and passes `version-name: needs.docs.outputs.version-name`. The - fork-PR warning is a step inside `docs.yml`'s build job (not a separate ci.yml - job), keeping the top-level workflow lean. The `publish` job's `if` also excludes - **tags** (`ref_type != 'tag'`) — a separate `publish-tag` job (tags only, `needs: - [_release]`, `actions: write`) re-triggers the deploy via - `gh workflow run publish-dispatch.yml --ref ` (`workflow_dispatch`), - because a same-SHA tag deploy is silently dropped by Pages unless it's a dispatch - event (deploy-pages#383). Both the inline `publish` job and `publish-tag` route - through `publish-dispatch.yml`, never `publish.yml` directly. -- `publish-dispatch.yml` — **the one wrapper around the engine.** `workflow_call` - (ci.yml's inline `publish` job) + `workflow_dispatch` (the tag trampoline + fork-PR - opt-in + manual re-deploy); both just `uses: publish.yml` and pass `version-name` / - `pr` / `guard-default-branch` through, carrying the privileged perms. This repo - dogfoods the exact wrapper consumers add (theirs pins `publish.yml@`; this one - uses the local path). + and uploads the `docs` artifact. It then nests a single `publish` job + (`uses: publish-dispatch.yml`) for **every** event in the canonical repo (the `if` + is just `repository == '…'`); grants `pages`/`id-token`/`statuses`/`actions:write` + perms to the call and passes `version-name: needs.docs.outputs.version-name`. No + ref/fork guards and no separate tag job — `publish-dispatch.yml` does all the + branching. +- `publish-dispatch.yml` — **the one wrapper around the engine**, owning all publish + branching. `workflow_call` (ci.yml's `publish` job, every event) + `workflow_dispatch` + (the tag trampoline's re-dispatch + fork-PR opt-in + manual re-deploy). Three jobs by + originating event: **deploy** (internal PR / default-branch push inline, or any + dispatch → `uses: publish.yml`), **trampoline** (a tag — waits for `_release`, then + `gh workflow run publish-dispatch.yml` so the deploy is a `workflow_dispatch` that + re-serves; a same-SHA tag deploy is otherwise dropped by Pages — deploy-pages#383), + and **warn** (a fork PR — read-only, never deploys, posts the manual-opt-in hint; + moved here from `docs.yml`'s build job). This repo dogfoods the exact wrapper + consumers add (theirs pins `publish.yml@`; this one uses the local path). - `publish.yml` — **assemble + deploy ENGINE, privileged.** `workflow_call` ONLY (sole caller: `publish-dispatch.yml`). Sparse-checks-out this repo's `assemble/` at `job.workflow_sha` (so the scripts match the ref the consumer pinned @@ -191,25 +186,23 @@ Sub-workflows of `ci.yml`: - `_release.yml` — attaches `version-switcher.mjs` + the tag's `docs.zip` (the `docs` artifact, verbatim — no repack) as Release assets (tag-only). -**Publish flow.** Internal PRs and `main` publish as part of the same CI run once -lint/test/docs pass (`ci.yml`'s `publish` job → `publish.yml` injects this build + -`assemble` gathers `main`, releases, and every other open PR → deploy), so the deploy -is a visible check on the PR/commit. **Tags are the exception:** a release tag shares -the merge commit's SHA, and GitHub Pages silently drops a second deploy of an -already-deployed SHA *unless* the event is `workflow_dispatch` +**Publish flow.** The single `publish` job (→ `publish-dispatch.yml`) runs for every +event and the wrapper branches. Internal PRs and `main` deploy inline as part of the +same CI run once lint/test/docs pass (`publish.yml` injects this build + `assemble` +gathers `main`, releases, and every other open PR → deploy), a visible check on the +PR/commit. **Tags** hit the wrapper's `trampoline`: a release tag shares the merge +commit's SHA, and GitHub Pages silently drops a second deploy of an already-deployed +SHA *unless* the event is `workflow_dispatch` ([`actions/deploy-pages#383`](https://github.com/actions/deploy-pages/issues/383)) — -so the `publish-tag` job (after `_release`) re-triggers `publish.yml` via -`gh workflow run` (`workflow_dispatch`), which re-gathers from durable sources (incl. -the new release) and deploys with the origin actually updating. The dispatched run -carries the deploy status (not the tag's CI run). An **external fork PR** builds + -verifies in CI but does **not** auto-publish (its `publish` job is skipped); a -maintainer runs `publish.yml` with the PR number (`workflow_dispatch` → `pr`), -which sets a `preview-approved` commit status pinned to that **head SHA** and -assembles — so a later push to the PR (new SHA) drops the preview until re-approved. -`assemble` gathers an open PR's artifact via its head SHA; internal PRs always, -fork PRs only when the SHA carries that status. Consumers can't dispatch `publish.yml` -cross-repo, so they trampoline via a small `publish-dispatch.yml` wrapper (which also -serves the fork-preview opt-in) — see the how-to. +so it waits for `_release`, then `gh workflow run publish-dispatch.yml` +(`workflow_dispatch`), which re-gathers from durable sources (incl. the new release) +and deploys with the origin actually updating (the dispatched run carries the deploy +status, not the tag's CI run). An **external fork PR** hits the wrapper's `warn` job +(read-only, no deploy); a maintainer dispatches `publish-dispatch.yml` with the PR +number (`pr`), which sets a `preview-approved` commit status pinned to that **head +SHA** and assembles — so a later push to the PR (new SHA) drops the preview until +re-approved. `assemble` gathers an open PR's artifact via its head SHA; internal PRs +always, fork PRs only when the SHA carries that status. `mystmd` is pinned at `1.10.1` (not `latest`). @@ -270,36 +263,24 @@ jobs: uses: DiamondLightSource/myst-version-switcher-plugin/.github/workflows/docs.yml@ with: build-command: make docs # or: tox -e docs / npx … myst build / npm ci && npm run docs - publish: # internal, non-tag events (fork PRs + tags excluded by the if) + publish: # every event — the wrapper branches needs: [docs] - if: >- - github.repository == 'ORG/REPO' && - github.ref_type != 'tag' && - ( github.event_name != 'pull_request' || - github.event.pull_request.head.repo.full_name == github.repository ) + if: github.repository == 'ORG/REPO' uses: ./.github/workflows/publish-dispatch.yml # your wrapper; it pins publish.yml@ with: version-name: ${{ needs.docs.outputs.version-name }} - permissions: { pages: write, id-token: write, contents: read, actions: read, statuses: write } - publish-tag: # tags: re-deploy via workflow_dispatch (see how-to) - needs: [docs, release] - if: github.repository == 'ORG/REPO' && github.ref_type == 'tag' - runs-on: ubuntu-latest - permissions: { actions: write } - steps: - - env: { GH_TOKEN: "${{ github.token }}", DEFAULT_BRANCH: "${{ github.event.repository.default_branch }}" } - run: gh workflow run publish-dispatch.yml --repo "$GITHUB_REPOSITORY" --ref "$DEFAULT_BRANCH" + permissions: { pages: write, id-token: write, contents: read, actions: write, statuses: write } ``` Add a `_release.yml`-style job that attaches each tag's built `docs.zip` (bare `html/` root) + the plugin as Release assets so `assemble` can reconstruct released -versions, and a small `publish-dispatch.yml` wrapper around `publish.yml` — the single -caller of the engine and the one place you pin `publish.yml@`. It's -`workflow_call` (your `publish` job nests it for the inline non-tag deploy) **and** -`workflow_dispatch` (the **tag trampoline** target — `publish-tag` dispatches it so the -deploy runs as a `workflow_dispatch`, required for a release's same-SHA deploy to go -live — plus the fork-PR preview opt-in via `pr`). See the full how-to + snippets in -`docs/`. +versions, and the `publish-dispatch.yml` wrapper around `publish.yml` — the single +caller of the engine and the one place you pin `publish.yml@`. The wrapper owns all +the branching: **deploy** (inline non-tag, or a dispatch), **trampoline** (a tag +re-dispatches itself so the deploy is a `workflow_dispatch` — required for a release's +same-SHA deploy to go live), and **warn** (a fork PR, read-only, posts the preview +opt-in hint; a maintainer dispatches the wrapper with `pr` to actually preview). See the +full how-to + snippets in `docs/`. ## Upstreaming diff --git a/docs/explanations/architecture.md b/docs/explanations/architecture.md index bf3f110..e4af3ca 100644 --- a/docs/explanations/architecture.md +++ b/docs/explanations/architecture.md @@ -108,18 +108,20 @@ So a fork's build can never reach a write token; only trusted code deploys. The deploy is surfaced as a **job inside the CI run** (`ci.yml`'s `publish` job → `publish-dispatch.yml` → `publish.yml` via `workflow_call`) so its status and URL are -visible on the PR / commit — rather than running invisibly after the fact. But this is -gated to **internal events only**: the `publish` job's `if` excludes fork PRs -(`head.repo.full_name != github.repository`). A fork PR's build instead emits a -warning (a step in the build job) that the preview was not published, linking the -manual opt-in. +visible on the PR / commit — rather than running invisibly after the fact. A single +`publish` job runs for **every** event; `publish-dispatch.yml` does the branching. `publish.yml` itself is a pure `workflow_call` **engine** with a single caller — -`publish-dispatch.yml`, the one wrapper that exposes it as both `workflow_call` -(ci.yml's inline publish) and `workflow_dispatch` (the tag trampoline, the fork-PR -opt-in, and manual re-deploys). Keeping the dispatch surface in the wrapper means the -engine has no event-type branching of its own, and this repo dogfoods the exact wrapper -a consumer adds (theirs just pins `publish.yml@`). +`publish-dispatch.yml`, the one wrapper that owns all the publish branching and is +reached by `workflow_call` (the inline `publish` job) and `workflow_dispatch` (the +trampoline's re-dispatch, the fork-PR opt-in, manual re-deploys). It routes each event +to one of three jobs: **deploy** (internal PR / default-branch push, or a dispatch), +**trampoline** (a tag — below), or **warn** (a fork PR — read-only, never deploys, just +posts the manual-opt-in hint; this replaced the old warning step in the build job). A +fork PR can't reach a write token (its `GITHUB_TOKEN` is read-only and `deploy` is +`if`-excluded for forks), so the security boundary holds while the hint now lives next +to the rest of the publish logic. This repo dogfoods the exact wrapper a consumer adds +(theirs just pins `publish.yml@`). The cost of nesting is an environment-policy change: because internal PRs now deploy from **their own ref**, the `github-pages` environment's deployment-branch policy must @@ -141,9 +143,10 @@ record active, but the origin keeps serving the *first* artifact. So an inline t deploy would "succeed" while the site stayed on the pre-tag build. The one documented escape hatch: a `workflow_dispatch` deploy of that same SHA -*forces* a re-serve. So tags don't deploy inline. `ci.yml`'s `publish-tag` job (after -`release`, so the new `docs.zip` asset exists) re-triggers `publish.yml` as a -`workflow_dispatch`; that run re-gathers from durable sources (now including the new +*forces* a re-serve. So tags don't deploy inline — the wrapper's `trampoline` job +waits for `ci.yml`'s parallel `release` job to attach the new `docs.zip`, then +re-dispatches `publish-dispatch.yml` as a `workflow_dispatch`; that run re-gathers from +durable sources (now including the new release) and deploys, and because its event is `workflow_dispatch` the origin updates. `main` and internal-PR deploys are the *first* of their SHA, so they serve fine inline and keep their PR/commit visibility. Consumers do the same by dispatching their own diff --git a/docs/reference/workflows.md b/docs/reference/workflows.md index c7154f9..0b82d78 100644 --- a/docs/reference/workflows.md +++ b/docs/reference/workflows.md @@ -31,7 +31,6 @@ preinstalled Node, so `build-command` can be `make` / `npx` / `tox` / `npm` driv |---|---|---|---| | `build-command` | no | `make docs` | Command that builds the HTML site into `html-dir` at `$BASE_URL`. Fold any project setup (`cp CONFIG`, `npm ci`, apt deps) behind it. | | `html-dir` | no | `docs/_build/html` | Directory the build writes the site to; staged into `docs.zip`'s `html/`. | -| `preview-workflow` | no | `publish.yml` | Consumer's dispatchable publish workflow file, named in the fork-preview warning. | | output | meaning | |---|---| @@ -53,14 +52,17 @@ version bump. The consumer's repo stays checked out at the root so `assemble.mjs ## `publish-dispatch.yml` — the wrapper (one per repo) -The only thing that calls `publish.yml`, and the single place you pin -`publish.yml@`. It exposes the engine two ways: - -- `workflow_call` — `ci.yml`'s `publish` job nests it for internal **non-tag** events - (inline deploy, status on the PR/commit), threading `version-name` through. -- `workflow_dispatch` — the **tag trampoline** (`ci.yml`'s `publish-tag` does - `gh workflow run publish-dispatch.yml`), the fork-PR opt-in (`pr`), and manual - re-deploys. +The only thing that calls `publish.yml`, the single place you pin `publish.yml@`, +and the owner of **all** publish branching — so `ci.yml` needs just one `publish` job +(called for every event) and `docs.yml` needs no fork hint. Reached via `workflow_call` +(ci.yml's `publish`) and `workflow_dispatch` (the trampoline's re-dispatch, a fork-PR +preview, a manual re-deploy), it routes each event to one of three jobs: + +- **`deploy`** — internal PR / default-branch push (inline), or any `workflow_dispatch`. + Nests `publish.yml`; the inline path injects the in-run build via `version-name`. +- **`trampoline`** — a tag push. Waits for the release, then re-dispatches this workflow + as `workflow_dispatch` so the deploy re-serves (see below). +- **`warn`** — a fork PR. Read-only, never deploys; just posts the manual-opt-in hint. **Tags must deploy via `workflow_dispatch`**: GitHub Pages silently drops a second deploy of an already-deployed SHA (a release tag shares the merge commit's SHA) unless diff --git a/docs/tutorials/adding-to-a-fresh-repo.md b/docs/tutorials/adding-to-a-fresh-repo.md index 6091301..3228434 100644 --- a/docs/tutorials/adding-to-a-fresh-repo.md +++ b/docs/tutorials/adding-to-a-fresh-repo.md @@ -56,15 +56,10 @@ repo — you **call** this project's two reusable workflows by full path, pinned Release. **Required:** that asset is a tag's *only* durable source (tags get no `_sources/` copy, unlike the default branch), so without it a release drops on the next deploy. -- **`publish`** — `publish.yml` reconstructs the whole versioned site and deploys it, - nested so its status shows on the PR/commit, but it runs **only for internal, - non-tag events** (the canonical-repo + non-fork guard). Fork PRs build to verify but - never reach a write token. -- **`publish-tag`** — tags can't deploy inline (a release tag shares the merge commit's - SHA, and GitHub Pages silently drops a second same-SHA deploy unless it's a - `workflow_dispatch`). So this tag-only job re-triggers the deploy as a dispatch of - your `publish-dispatch.yml` wrapper (below), which forces a re-serve. **Required for - releases to go live.** +- **`publish`** — calls your `publish-dispatch.yml` wrapper (below) for **every** event; + the wrapper decides what to do (deploy a PR/`main` preview, trampoline a tag, or just + warn on a fork PR), so this is one small job with no ref/fork guards. Its status shows + on the PR/commit. Fork PRs build + verify but never reach a write token. ```yaml # .github/workflows/ci.yml @@ -90,34 +85,18 @@ jobs: steps: # ↓ paste the two steps from the literalinclude below ↓ - publish: # internal, non-tag events only + publish: # every event — the wrapper branches needs: [docs] - if: >- - github.repository == 'ORG/REPO' && - github.ref_type != 'tag' && - ( github.event_name != 'pull_request' || - github.event.pull_request.head.repo.full_name == github.repository ) + if: github.repository == 'ORG/REPO' uses: ./.github/workflows/publish-dispatch.yml # your wrapper (below); it pins publish.yml@ with: version-name: ${{ needs.docs.outputs.version-name }} permissions: contents: read - actions: read + actions: write # the wrapper's tag trampoline dispatches via gh workflow run pages: write id-token: write statuses: write - - publish-tag: # tags: re-deploy via workflow_dispatch - needs: [docs, release] - if: github.repository == 'ORG/REPO' && github.ref_type == 'tag' - runs-on: ubuntu-latest - permissions: - actions: write # dispatch the wrapper - steps: - - env: - GH_TOKEN: ${{ github.token }} - DEFAULT_BRANCH: ${{ github.event.repository.default_branch }} - run: gh workflow run publish-dispatch.yml --repo "$GITHUB_REPOSITORY" --ref "$DEFAULT_BRANCH" ``` The `release` job's `steps:` are exactly what this repo's own `_release.yml` runs (minus @@ -140,49 +119,70 @@ internally — it checks out this project's `assemble` scripts at the same ``. It exposes the engine two ways and so covers all three publish paths: - -- **Inline publish (`workflow_call`)** — your `publish` job above nests it for internal - non-tag events, passing `version-name`, so the deploy is a visible check on the - PR/commit. -- **Tag trampoline (`workflow_dispatch`, no `pr`)** — `publish-tag` above dispatches it - on a tag. Deploying as a `workflow_dispatch` event is what makes a same-SHA tag - deploy actually go live (an inline tag deploy is silently dropped — see the - [architecture explanation](../explanations/architecture.md)). **This is why the - wrapper is required, not optional.** -- **Fork-PR preview (`workflow_dispatch`, with `pr`)** — internal PRs publish - automatically; fork PRs build and verify but never auto-deploy. A maintainer runs it - from the Actions tab with a PR number to approve that fork's head SHA and deploy a - preview (a later push to the PR drops it until re-run). +**only** thing that calls it — also the single place you pin the engine's `@`. It +owns all the branching, so your `ci.yml` `publish` job stays a one-liner. Copy it +verbatim, changing only ``: ```yaml # .github/workflows/publish-dispatch.yml name: Publish (dispatch) on: - workflow_call: # ci.yml's inline `publish` job + workflow_call: # ci.yml's `publish` job, for every event inputs: - version-name: { required: false, default: "", type: string } - pr: { required: false, default: "", type: string } - workflow_dispatch: # tag trampoline + fork-PR preview + manual re-deploy + version-name: { required: false, default: "", type: string } + guard-default-branch: { required: false, default: "true", type: string } + pr: { required: false, default: "", type: string } + workflow_dispatch: # tag trampoline's re-dispatch + fork-PR preview + manual re-deploy inputs: pr: { description: "Fork PR number to approve + preview (leave empty to just re-deploy)", required: false, default: "" } jobs: - publish: + # Internal PR / default-branch push (inline), or any workflow_dispatch → deploy. + deploy: + if: >- + github.event_name == 'workflow_dispatch' || + (github.event_name == 'push' && github.ref_type != 'tag') || + (github.event_name == 'pull_request' && + github.event.pull_request.head.repo.full_name == github.repository) + permissions: { contents: read, actions: read, pages: write, id-token: write, statuses: write } uses: DiamondLightSource/myst-version-switcher-plugin/.github/workflows/publish.yml@ with: - version-name: ${{ inputs.version-name }} # "" on dispatch → pure durable gather - pr: ${{ inputs.pr }} # empty → plain re-deploy; set → pin that fork head SHA - permissions: - contents: read - actions: read - pages: write - id-token: write - statuses: write + version-name: ${{ inputs.version-name }} # "" on dispatch → pure durable gather + guard-default-branch: ${{ inputs.guard-default-branch || 'true' }} + pr: ${{ inputs.pr }} # set → pin that fork head SHA + + # A tag can't deploy inline (same SHA as the default-branch push → Pages drops it + # unless the event is workflow_dispatch). Re-dispatch this workflow after the release. + trampoline: + if: github.event_name == 'push' && github.ref_type == 'tag' + runs-on: ubuntu-latest + permissions: { contents: read, actions: write } + steps: + - env: { GH_TOKEN: "${{ github.token }}", REPO: "${{ github.repository }}", TAG: "${{ github.ref_name }}", DEFAULT_BRANCH: "${{ github.event.repository.default_branch }}" } + run: | + set -euo pipefail + for _ in $(seq 1 36); do gh release view "$TAG" --repo "$REPO" >/dev/null 2>&1 && break; sleep 5; done + gh release view "$TAG" --repo "$REPO" >/dev/null 2>&1 || { echo "::error::release $TAG not found"; exit 1; } + gh workflow run publish-dispatch.yml --repo "$REPO" --ref "$DEFAULT_BRANCH" + + # Fork PR: read-only token, never deploys — just surface the manual-opt-in hint. + warn: + if: >- + github.event_name == 'pull_request' && + github.event.pull_request.head.repo.full_name != github.repository + runs-on: ubuntu-latest + permissions: {} + steps: + - run: | + echo "::warning title=Docs preview not published::Fork PR — not auto-published. A maintainer \ + can preview it by running publish-dispatch.yml with pr=${{ github.event.pull_request.number }}: \ + https://github.com/${{ github.repository }}/actions/workflows/publish-dispatch.yml" ``` -If you point `docs.yml`'s fork-preview warning at this file, also set -`preview-workflow: publish-dispatch.yml` on the `docs` job. +The three branches: **deploy** (internal PR / `main` push inline, or any dispatch), +**trampoline** (a tag re-dispatches itself so the deploy runs as a `workflow_dispatch` +— [why](../explanations/architecture.md)), and **warn** (a fork PR, read-only, never +deploys). A maintainer publishes a fork preview by running this workflow from the +Actions tab with the `pr` number. ## 4. Allow the deploying refs in the `github-pages` environment @@ -199,10 +199,11 @@ commit them on a branch and open a PR. On the PR you'll see: - **`docs / build`** go green — it builds your docs at the versioned `BASE_URL` and uploads the `docs` artifact. This runs for every PR, forks included. -- **`publish / publish`** runs for an *internal* PR (a branch in your own repo) and +- **`publish / deploy`** runs for an *internal* PR (a branch in your own repo) and deploys a preview of just this PR at `https://ORG.github.io/REPO/pr-/`, linked from - the PR's checks/Deployments. A **fork** PR builds but never auto-publishes (its token - is read-only) — use the opt-in from step 3 to preview one. + the PR's checks/Deployments. A **fork** PR instead gets **`publish / warn`** (a + read-only hint — forks never auto-publish); a maintainer dispatches the wrapper with + the `pr` number to preview one. > **First-time exception:** on a brand-new repo the `publish` check is **red on this > setup PR** — there's no `main` build yet for the versioned site to anchor on, and the