Repository navigation
Conversation
The re-dispatch job waited for the release RECORD (`gh release view --json publishedAt`), but the deploy's gather selects releases by their `docs.zip` ASSET. Consumers who publish the release from the GitHub UI create the record first — publishing creates the tag, which is what starts CI — so the wait was a no-op and the dispatched deploy listed releases before `release.yml` attached the asset. Every one of those releases shipped a site missing itself, healed only when the next release's deploy carried it in. This repo was immune because release.yml creates the release draft → upload → publish atomically, so publishedAt already implies the asset. re-dispatch now polls the tag's assets for docs.zip (30 min, enough for a release job gated on a full test matrix), skips the wait entirely when no release in the repo has ever carried one, and — if the asset never lands — dispatches anyway so the rest of the site refreshes, then fails loudly. retry-until is anchored on the asset's arrival rather than publishedAt, which a UI-published release stamps before CI even starts. Also make a deploy explain itself, so the next post-mortem is minutes not hours: - a context header (event, ref, inputs, which assemble.mjs, timestamps) - a per-release decision table — gathered / skip: no docs.zip asset / download failed — plus a warning when a release published in the last 30 min has no asset, which is this bug's signature - artifact gathers name the artifact, run and SHA, and split the old catch-all "artifact unavailable" into not-found / download-failed / contains-no-docs.zip - per-zip extract lines, and a job-summary table of every deployed version with the source it came from - a version diff on verify failure, separating a stale origin from a bad gather Note `unzip -q a.zip member -d dir` exits 0 with only a "filename not matched" caution when the member is absent, so the check is for the file, not the status. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Closing without merging, but keeping the Context: the wider question is whether publish should move off the inline That hinges on whether a If the experiment says |
…site (#64) * feat(assemble): select which releases the site publishes, with a cap The deploy uploads the WHOLE site as a single Pages artifact, and Pages caps that at 1 GB. blueapi is at 452,549,958 bytes across 131 released docs.zips and climbing ~5 MB per release — roughly 110 releases of headroom before deploys hard-fail, with no warning on the way. Every deploy also re-downloads all 453 MB. Adds `selectReleases`, which decides what each listed release contributes, and a `select-releases` subcommand that runs BEFORE the gather so publish.yml downloads only what it will actually publish. The rules were previously an untested bash `case` in publish.yml; moving them here matches the split the repo already uses — bash does the IO, the kernel does the logic and is unit-tested. Ranking is on `created_at`, not `published_at`. published_at records when the release record was last published, so re-publishing an old one makes it look new: blueapi's 1.3.2-a9 was created 2025-10-01 but published 2026-07-30, and under published_at it outranked the genuinely newer 1.11.3. No version-number parsing is involved either way — tags are too inconsistent across repos to parse safely, and the API already knows the order. The seed release is exempt from the cap: it stands in for the default branch rather than being a version. Skipped releases don't consume a cap slot either. Verified against blueapi's real 188-release payload: 131 selected unlimited, and `--max-releases 20` keeps the 20 most recent (73.1 MB, down from 453 MB). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf(publish): cache, cap and parallelise the gather On blueapi the deploy job took 654 s against a 27–45 s docs build, and the cost was independent of what changed — every event reconstructed the whole site. Three of those steps are addressed here. Gather release artifacts (114 s): releases were listed and downloaded one at a time, all 131 of them, on every deploy. Now assemble.mjs picks the set first, an actions/cache entry keyed on exactly that set of asset ids supplies the zips already seen, and only the misses are fetched — in parallel. Release assets are immutable, so steady state is a full hit and a new release downloads exactly one zip. Cached files are named by asset id rather than tag, so a re-cut release can never be served stale, and the cache is pruned to the published set so a capped site keeps a capped cache. Saving happens only on the default branch: a cache written on a PR ref is scoped to that PR and would just churn the 10 GB quota. Gather branch CI artifacts (175 s): the artifacts API was re-paginated once per open PR — 27 identical passes over thousands of artifacts for blueapi. Now it paginates once, indexes newest-artifact-per-head-SHA, and downloads in parallel. max-releases input: the site deploys as ONE Pages artifact against a hard 1 GB cap. blueapi is at 452 MB over 131 releases and grows ~5 MB per release, so it has a few years before deploys simply start failing. The cap is opt-in (0 = unlimited, so nothing disappears from an existing site on upgrade) and set in the shim's `with:` so it applies on every path rather than only the inline one. A warning past 700 MB means the ceiling can no longer arrive unannounced. Also adds test/workflow-harness: these gather steps were previously verifiable only by merging and watching a real deploy. The harness loads them out of the YAML — so the tests cannot drift from the workflow — and runs them against a mock gh, covering the cap, cache hit/miss/prune, the artifact index, fork approval, and each of the four failure modes. It caught two real bugs while being written. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * style: use a template literal for the select-releases TSV row Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * refactor(publish): keep the default branch's durable docs.zip in the Actions cache Releases are durable and PRs are ephemeral by design, but the default branch had no permanent source — it is gathered from its latest CI artifact, which expires, after which it drops out and the guard hard-fails. That was solved by publishing the zip into the site itself as _sources/<default>.zip: permanent, but it shipped a multi-megabyte zip inside every Pages artifact forever, on a site already pressing against the 1 GB cap. The Actions cache holds it instead, content-addressed so an unchanged default branch adds no new entry, and saved only from the default branch (a cache written on a PR ref is scoped to that PR and unreadable by anything else). The trade is deliberate and worse: cache entries are evicted after 7 days without a read and by LRU past 10 GB, so a repo that goes quiet for over a week AND whose main CI artifact has expired loses this rung, where _sources/ was forever. Accepted — the common case is a repo that deploys more often than weekly, and the payoff is a smaller site on every single deploy. _sources/ is no longer written. Its READ survives as a deprecated rung so a site last deployed by an older publish.yml still has a source on its first upgraded deploy — one-shot, since that deploy republishes the whole tree without it. Harness extended to the full fallback chain (fresh / cache / deprecated in-site / hard-fail), the no-op-save case, and the extract step's zip-shape contract and size guard. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: the size ceiling, the release cap, and the cache-backed default branch Documents what the preceding commits changed, and fixes the one place they broke something: scripts/migrate.sh gated the irreversible gh-pages delete on _sources/<default>.zip returning 200 from the live site, and that file is no longer published. The replacement guard is stronger than the one it replaces. It now requires both that /<default>/ is live in the new Actions-deployed site AND that a non-expired `docs` artifact exists for the default branch — i.e. its CI really builds docs.zip now. The old check passed while /<default>/ was still coming from the migration seed, which is exactly the state where gh-pages remains the only real copy; it was only safe because _sources was permanent. With a cache-backed (evictable) durable copy it would not have been, so the guard waits for the condition it always should have used. Also recovers docs/how-to/choose-publish-events.md, which was written for the closed PR #62 and is independent of it: the "publish only on tags" gate is what leaves a site unable to self-heal, and blueapi needs that written down. Its claim about a job-summary table is corrected to describe what this branch's deploy actually prints. New docs/how-to/keep-the-site-small.md covers max-releases, what the cap does and does not delete, and what is exempt. architecture.md gains a "why the site has a size limit" section; the tutorial's shim ships max-releases: 20; both new pages are added to the toc (myst.yml has an explicit one, so a new file is otherwise silently unpublished). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: untrack blueapi-issue.md Handover text for another repo, swept in by a `git add -A`. It was deliberately untracked; keep it that way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: drop local working-tree changes swept into the perf commit `git add -A` picked up two uncommitted local changes that are not part of this work: a devcontainer --device=/dev/net/tun tweak and a package-lock edit. Restored to origin/main; both are back in the working tree, uncommitted, as they were. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: revert an unrelated renovate.json reformat, ignore __pycache__ `biome check --write .` reformatted renovate.json (it is 2-space in main, tabs under biome's config) and `git add -A` shipped it along with the harness's .pyc files. Neither belongs in this branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(publish): measure the size guard against the packed artifact, not the tree The guard compared `du -sb` of the site directory against the 1 GB Pages cap, but that cap applies to the compressed artifact. Caught on this branch's own deploy (run 32471577932): the tree was 734 MiB and the warning fired, while upload-pages-artifact reported 233,824,999 bytes — 22% of the limit, not 73%. HTML and JS pack about 3x, so `du` cries wolf at a third of real usage. Now the tree size is reported for information, and the artifact is actually packed and measured only once the tree passes SIZE_PROBE_BYTES (1.5 GiB) — packing costs real seconds, so it is not worth doing on every deploy of a small site. `tar -ch` matches upload-pages-artifact's --dereference, which inflates the `stable/` symlink into a full copy of a release; measuring it as a symlink would under-read by a whole version. Both thresholds come from job env, which makes the warning path testable without a multi-gigabyte fixture — the harness now covers probe/no-probe, warn/no-warn. blueapi's 452 MB figure was already the artifact size, so that claim stands; the docs that implied `du` was the measure are corrected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(publish): move the cache actions to v6, explain this repo's max-releases The real deploy warned that actions/cache/restore@v4 targets deprecated Node 20 and is being forced onto Node 24. v6 is only an ESM/packages migration — same inputs and outputs — and matches the recency of everything else pinned here (checkout@v7, download-artifact@v8, deploy-pages@v5). Also records why this repo's shim says max-releases: "0" while the tutorial ships "20": our site is ~223 MB packed across 24 releases, a fifth of the cap, so capping would drop old versions for nothing. Worth stating, because CLAUDE.md claims this repo dogfoods the shim consumers copy and this is a deliberate divergence in behaviour rather than just the `uses:` path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The bug
publish.yml'sre-dispatchjob waited for the release record(
gh release view --json publishedAt), but the deploy's gather selects releases by theirdocs.zipasset.Consumers who publish a release from the GitHub UI create the record first —
publishing creates the tag, which is what starts CI — so the wait is a no-op and the
dispatched deploy lists releases minutes before
release.ymlattaches the asset.Found in blueapi, where the site is
permanently one release behind. Timeline for 1.18.1:
re-dispatchsees the release, dispatches at oncerelease / releaseattachesdocs.zip— 55 s too lateNot a one-off: 1.18.0's own deploy logged
Sorted versions: ["main","1.17.0",…]— itmissed itself too, and only reached the site 11 days later carried in by 1.18.1's deploy.
This repo was immune, which is why it went unnoticed:
release.ymlcreates therelease draft → upload → publish atomically, so
publishedAtalready implies the asset.The fix
re-dispatchpolls the tag's assets fordocs.zip:using
release.yml);releasejob is usually gated on a full test matrix;::error::, dispatch anyway so the rest of the siterefreshes, then exit 1;
retry-untilis anchored on the asset's arrival, notpublishedAt— which aUI-published release stamps before CI even starts, burning the budget before the first
deploy attempt.
No consumer change is required — no new shim input.
Making the next post-mortem minutes, not hours
The old logs never said why a release was skipped. Now a deploy explains itself:
assemble.mjs, timestamps;gathered/skip: no docs.zip asset/DOWNLOAD FAILED) plus a::warning::when a release published in the last 30 min hasno asset — this bug's exact signature, which would have shown it in seconds;
artifact unavailableinto not found / download failed / contains no docs.zip(the last is blueapi's six stale PRs — artifacts predating the
docs.zippackingcontract, needing a CI re-run, not a bug);
source it came from;
missing from live:version diff on verify failure, separating a stale origin from abad gather.
Docs
reference/workflows.mdandexplanations/architecture.mdexplain what the re-dispatchwaits for and why the UI-publish flow breaks the record wait. New
docs/how-to/choose-publish-events.mdcovers which events should publish — including whygating
publishon tags alone removes self-healing entirely, which is the second half ofthe blueapi failure (their
ci.ymlhasif: … && github.ref_type == 'tag', so every PRand
mainpush showspublish → skippedand nothing ever corrects a bad deploy).Verification
Each
run:block was extracted and exercised against real blueapi data with a stubbedgh:re-dispatchwaits and only thendispatches (old code dispatched immediately); also covered: no-asset repo → no wait,
release missing → error, asset never arrives → error + dispatch + exit 1;
"published now, no asset" fixture that produces the warning;
html/-rooted, tag-rooted and malformed zips; summary rendering.npm test(24 checks) andnpm run docs --strictpass; all workflows parse and everyrun:block passesbash -n.Two silent traps the testing caught, both noted in comments:
unzip -q a.zip member -d direxits 0 with only a "caution: filename not matched"note when the member is absent — the check must be for the file, not the exit status
(relying on the status let a bare
mvfail underset -eand kill the whole deploy job);readcollapses consecutive tabs — a nullpublished_atshifted every column and could have made a good release read as asset-less;
jqnowemits
"-"rather than an empty field.🤖 Generated with Claude Code