Skip to content

fix: wait for the tag's docs.zip asset before re-dispatching - #62

Closed
coretl wants to merge 1 commit into
mainfrom
fix/re-dispatch-waits-for-release-asset
Closed

coretl wants to merge 1 commit into
mainfrom
fix/re-dispatch-waits-for-release-asset

Conversation

@coretl

@coretl coretl commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The bug

publish.yml's re-dispatch job waited for the release record
(gh release view --json publishedAt), but the deploy's gather selects releases by their
docs.zip asset.

Consumers who publish a release from the GitHub UI create the record first —
publishing creates the tag, which is what starts CI — so the wait is a no-op and the
dispatched deploy lists releases minutes before release.yml attaches the asset.

Found in blueapi, where the site is
permanently one release behind. Timeline for 1.18.1:

time (UTC) event
09:34:14 release 1.18.1 published (UI)
09:34:16 tag CI run 32122292712 starts
09:34:50–54 re-dispatch sees the release, dispatches at once
09:35:02 dispatched deploy 32122349268 lists releases — 1.18.1 has no assets yet
09:35:57 release / release attaches docs.zip — 55 s too late

Not a one-off: 1.18.0's own deploy logged Sorted versions: ["main","1.17.0",…] — it
missed itself too, and only reached the site 11 days later carried in by 1.18.1's deploy.

This repo was immune, which is why it went unnoticed: release.yml creates the
release draft → upload → publish atomically, so publishedAt already implies the asset.

The fix

re-dispatch polls the tag's assets for docs.zip:

  • skips the wait entirely when no release in the repo has ever carried one (consumers not
    using release.yml);
  • 30 min budget — the release job is usually gated on a full test matrix;
  • if the asset never lands: loud ::error::, dispatch anyway so the rest of the site
    refreshes, then exit 1;
  • retry-until is anchored on the asset's arrival, not publishedAt — which a
    UI-published release stamps before CI even starts, burning the budget before the first
    deploy attempt.

No consumer change is required — no new shim input.

Making the next post-mortem minutes, not hours

The old logs never said why a release was skipped. Now a deploy explains itself:

  • a context header — event, ref, every input, which assemble.mjs, timestamps;
  • a per-release decision table (gathered / skip: no docs.zip asset /
    DOWNLOAD FAILED) plus a ::warning:: when a release published in the last 30 min has
    no asset — this bug's exact signature, which would have shown it in seconds;
  • artifact gathers name the artifact, run and SHA, and split the old catch-all
    artifact unavailable into not found / download failed / contains no docs.zip
    (the last is blueapi's six stale PRs — artifacts predating the docs.zip packing
    contract, needing a CI re-run, not a bug);
  • per-zip extract lines, and a job-summary table of every deployed version with the
    source it came from;
  • a missing from live: version diff on verify failure, separating a stale origin from a
    bad gather.

Docs

reference/workflows.md and explanations/architecture.md explain what the re-dispatch
waits for and why the UI-publish flow breaks the record wait. New
docs/how-to/choose-publish-events.md covers which events should publish — including why
gating publish on tags alone removes self-healing entirely, which is the second half of
the blueapi failure (their ci.yml has if: … && github.ref_type == 'tag', so every PR
and main push shows publish → skipped and nothing ever corrects a bad deploy).

Verification

Each run: block was extracted and exercised against real blueapi data with a stubbed gh:

  • the race — with the asset arriving late, re-dispatch waits and only then
    dispatches (old code dispatched immediately); also covered: no-asset repo → no wait,
    release missing → error, asset never arrives → error + dispatch + exit 1;
  • release gather over all 188 real blueapi releases, plus a synthesised
    "published now, no asset" fixture that produces the warning;
  • artifact gather reproducing all four blueapi cases distinctly; extract over
    html/-rooted, tag-rooted and malformed zips; summary rendering.

npm test (24 checks) and npm run docs --strict pass; all workflows parse and every
run: block passes bash -n.

Two silent traps the testing caught, both noted in comments:

  • unzip -q a.zip member -d dir exits 0 with only a "caution: filename not matched"
    note when the member is absent — the check must be for the file, not the exit status
    (relying on the status let a bare mv fail under set -e and kill the whole deploy job);
  • tab is IFS whitespace, so read collapses consecutive tabs — a null published_at
    shifted every column and could have made a good release read as asset-less; jq now
    emits "-" rather than an empty field.

🤖 Generated with Claude Code

The re-dispatch job waited for the release RECORD (`gh release view --json
publishedAt`), but the deploy's gather selects releases by their `docs.zip`
ASSET. Consumers who publish the release from the GitHub UI create the record
first — publishing creates the tag, which is what starts CI — so the wait was a
no-op and the dispatched deploy listed releases before `release.yml` attached
the asset. Every one of those releases shipped a site missing itself, healed
only when the next release's deploy carried it in.

This repo was immune because release.yml creates the release draft → upload →
publish atomically, so publishedAt already implies the asset.

re-dispatch now polls the tag's assets for docs.zip (30 min, enough for a
release job gated on a full test matrix), skips the wait entirely when no
release in the repo has ever carried one, and — if the asset never lands —
dispatches anyway so the rest of the site refreshes, then fails loudly.
retry-until is anchored on the asset's arrival rather than publishedAt, which a
UI-published release stamps before CI even starts.

Also make a deploy explain itself, so the next post-mortem is minutes not hours:

- a context header (event, ref, inputs, which assemble.mjs, timestamps)
- a per-release decision table — gathered / skip: no docs.zip asset / download
  failed — plus a warning when a release published in the last 30 min has no
  asset, which is this bug's signature
- artifact gathers name the artifact, run and SHA, and split the old catch-all
  "artifact unavailable" into not-found / download-failed / contains-no-docs.zip
- per-zip extract lines, and a job-summary table of every deployed version with
  the source it came from
- a version diff on verify failure, separating a stale origin from a bad gather

Note `unzip -q a.zip member -d dir` exits 0 with only a "filename not matched"
caution when the member is absent, so the check is for the file, not the status.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coretl

coretl commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

Closing without merging, but keeping the fix/re-dispatch-waits-for-release-asset branch — the fix may come back.

Context: the wider question is whether publish should move off the inline ci.yml path onto workflow_run. If it does, this race disappears on its own (workflow_run fires after the tag's CI run — including release.yml — has completed, so docs.zip is already attached and there is nothing to wait for).

That hinges on whether a workflow_run-triggered Pages deploy re-serves at an already-deployed SHA. workflow_run pins GITHUB_SHA to the default branch's HEAD, but a repeat workflow_dispatch at the same SHA demonstrably does re-serve, so the same-SHA rule can't just be assumed to apply. Settling it with a live deploy experiment on this repo.

If the experiment says workflow_run is unusable, this commit gets cherry-picked onto the follow-up branch.

@coretl coretl closed this Aug 21, 2026
coretl added a commit that referenced this pull request Aug 21, 2026
…site (#64)

* feat(assemble): select which releases the site publishes, with a cap

The deploy uploads the WHOLE site as a single Pages artifact, and Pages caps that
at 1 GB. blueapi is at 452,549,958 bytes across 131 released docs.zips and climbing
~5 MB per release — roughly 110 releases of headroom before deploys hard-fail, with
no warning on the way. Every deploy also re-downloads all 453 MB.

Adds `selectReleases`, which decides what each listed release contributes, and a
`select-releases` subcommand that runs BEFORE the gather so publish.yml downloads
only what it will actually publish. The rules were previously an untested bash
`case` in publish.yml; moving them here matches the split the repo already uses —
bash does the IO, the kernel does the logic and is unit-tested.

Ranking is on `created_at`, not `published_at`. published_at records when the
release record was last published, so re-publishing an old one makes it look new:
blueapi's 1.3.2-a9 was created 2025-10-01 but published 2026-07-30, and under
published_at it outranked the genuinely newer 1.11.3. No version-number parsing is
involved either way — tags are too inconsistent across repos to parse safely, and
the API already knows the order.

The seed release is exempt from the cap: it stands in for the default branch rather
than being a version. Skipped releases don't consume a cap slot either.

Verified against blueapi's real 188-release payload: 131 selected unlimited, and
`--max-releases 20` keeps the 20 most recent (73.1 MB, down from 453 MB).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* perf(publish): cache, cap and parallelise the gather

On blueapi the deploy job took 654 s against a 27–45 s docs build, and the cost was
independent of what changed — every event reconstructed the whole site. Three of
those steps are addressed here.

Gather release artifacts (114 s): releases were listed and downloaded one at a time,
all 131 of them, on every deploy. Now assemble.mjs picks the set first, an
actions/cache entry keyed on exactly that set of asset ids supplies the zips already
seen, and only the misses are fetched — in parallel. Release assets are immutable, so
steady state is a full hit and a new release downloads exactly one zip. Cached files
are named by asset id rather than tag, so a re-cut release can never be served stale,
and the cache is pruned to the published set so a capped site keeps a capped cache.
Saving happens only on the default branch: a cache written on a PR ref is scoped to
that PR and would just churn the 10 GB quota.

Gather branch CI artifacts (175 s): the artifacts API was re-paginated once per open
PR — 27 identical passes over thousands of artifacts for blueapi. Now it paginates
once, indexes newest-artifact-per-head-SHA, and downloads in parallel.

max-releases input: the site deploys as ONE Pages artifact against a hard 1 GB cap.
blueapi is at 452 MB over 131 releases and grows ~5 MB per release, so it has a few
years before deploys simply start failing. The cap is opt-in (0 = unlimited, so
nothing disappears from an existing site on upgrade) and set in the shim's `with:`
so it applies on every path rather than only the inline one. A warning past 700 MB
means the ceiling can no longer arrive unannounced.

Also adds test/workflow-harness: these gather steps were previously verifiable only
by merging and watching a real deploy. The harness loads them out of the YAML — so
the tests cannot drift from the workflow — and runs them against a mock gh, covering
the cap, cache hit/miss/prune, the artifact index, fork approval, and each of the
four failure modes. It caught two real bugs while being written.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* style: use a template literal for the select-releases TSV row

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor(publish): keep the default branch's durable docs.zip in the Actions cache

Releases are durable and PRs are ephemeral by design, but the default branch had no
permanent source — it is gathered from its latest CI artifact, which expires, after
which it drops out and the guard hard-fails. That was solved by publishing the zip
into the site itself as _sources/<default>.zip: permanent, but it shipped a
multi-megabyte zip inside every Pages artifact forever, on a site already pressing
against the 1 GB cap.

The Actions cache holds it instead, content-addressed so an unchanged default branch
adds no new entry, and saved only from the default branch (a cache written on a PR
ref is scoped to that PR and unreadable by anything else).

The trade is deliberate and worse: cache entries are evicted after 7 days without a
read and by LRU past 10 GB, so a repo that goes quiet for over a week AND whose main
CI artifact has expired loses this rung, where _sources/ was forever. Accepted — the
common case is a repo that deploys more often than weekly, and the payoff is a
smaller site on every single deploy.

_sources/ is no longer written. Its READ survives as a deprecated rung so a site last
deployed by an older publish.yml still has a source on its first upgraded deploy —
one-shot, since that deploy republishes the whole tree without it.

Harness extended to the full fallback chain (fresh / cache / deprecated in-site /
hard-fail), the no-op-save case, and the extract step's zip-shape contract and size
guard.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: the size ceiling, the release cap, and the cache-backed default branch

Documents what the preceding commits changed, and fixes the one place they broke
something: scripts/migrate.sh gated the irreversible gh-pages delete on
_sources/<default>.zip returning 200 from the live site, and that file is no longer
published.

The replacement guard is stronger than the one it replaces. It now requires both that
/<default>/ is live in the new Actions-deployed site AND that a non-expired `docs`
artifact exists for the default branch — i.e. its CI really builds docs.zip now. The
old check passed while /<default>/ was still coming from the migration seed, which is
exactly the state where gh-pages remains the only real copy; it was only safe because
_sources was permanent. With a cache-backed (evictable) durable copy it would not have
been, so the guard waits for the condition it always should have used.

Also recovers docs/how-to/choose-publish-events.md, which was written for the closed
PR #62 and is independent of it: the "publish only on tags" gate is what leaves a site
unable to self-heal, and blueapi needs that written down. Its claim about a job-summary
table is corrected to describe what this branch's deploy actually prints.

New docs/how-to/keep-the-site-small.md covers max-releases, what the cap does and does
not delete, and what is exempt. architecture.md gains a "why the site has a size limit"
section; the tutorial's shim ships max-releases: 20; both new pages are added to the toc
(myst.yml has an explicit one, so a new file is otherwise silently unpublished).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: untrack blueapi-issue.md

Handover text for another repo, swept in by a `git add -A`. It was deliberately
untracked; keep it that way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: drop local working-tree changes swept into the perf commit

`git add -A` picked up two uncommitted local changes that are not part of this
work: a devcontainer --device=/dev/net/tun tweak and a package-lock edit. Restored
to origin/main; both are back in the working tree, uncommitted, as they were.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore: revert an unrelated renovate.json reformat, ignore __pycache__

`biome check --write .` reformatted renovate.json (it is 2-space in main, tabs under
biome's config) and `git add -A` shipped it along with the harness's .pyc files.
Neither belongs in this branch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(publish): measure the size guard against the packed artifact, not the tree

The guard compared `du -sb` of the site directory against the 1 GB Pages cap, but
that cap applies to the compressed artifact. Caught on this branch's own deploy
(run 32471577932): the tree was 734 MiB and the warning fired, while
upload-pages-artifact reported 233,824,999 bytes — 22% of the limit, not 73%. HTML
and JS pack about 3x, so `du` cries wolf at a third of real usage.

Now the tree size is reported for information, and the artifact is actually packed
and measured only once the tree passes SIZE_PROBE_BYTES (1.5 GiB) — packing costs
real seconds, so it is not worth doing on every deploy of a small site. `tar -ch`
matches upload-pages-artifact's --dereference, which inflates the `stable/` symlink
into a full copy of a release; measuring it as a symlink would under-read by a whole
version.

Both thresholds come from job env, which makes the warning path testable without a
multi-gigabyte fixture — the harness now covers probe/no-probe, warn/no-warn.

blueapi's 452 MB figure was already the artifact size, so that claim stands; the docs
that implied `du` was the measure are corrected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(publish): move the cache actions to v6, explain this repo's max-releases

The real deploy warned that actions/cache/restore@v4 targets deprecated Node 20 and
is being forced onto Node 24. v6 is only an ESM/packages migration — same inputs and
outputs — and matches the recency of everything else pinned here (checkout@v7,
download-artifact@v8, deploy-pages@v5).

Also records why this repo's shim says max-releases: "0" while the tutorial ships
"20": our site is ~223 MB packed across 24 releases, a fifth of the cap, so capping
would drop old versions for nothing. Worth stating, because CLAUDE.md claims this
repo dogfoods the shim consumers copy and this is a deliberate divergence in
behaviour rather than just the `uses:` path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

This branch was previously deployed

1 inactive deployment
github-pages — f483589f Deployed Aug 20, 2026 by coretl via publish / publish / deploy #201
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant