Run the data build from main only, sync from the lock, publish one release per build - #98
Merged
Merged
Conversation
Build DB could be dispatched from any branch, and a branch run publishes its DB as the latest release and pushes to main. Its first step now fails the job with an error unless the ref is refs/heads/main: a failure shows up in the Actions list, a skipped job would not. Build site's workflow_run trigger also requires head_branch == main, so a branch run of Build DB can never deploy the site. uv sync takes --locked in all three workflows. statusui is a git dependency with no rev in pyproject.toml, so uv.lock is its only pin; a pyproject/uv.lock mismatch now fails the job instead of re-resolving statusui to whatever its default branch holds. uv lock --check passes on this commit. astral-sh/setup-uv is pinned to the commit of v9.0.0 (a lightweight tag, resolved with git ls-remote). It is the only third-party action; the actions/* ones are left on their tags. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ
The second build of a day re-publishes to the release the first one created, with gh release upload --clobber. --clobber deletes the old asset and then uploads, so an upload that fails (a 34 MB transfer is the step most likely to) left the latest release with no uisce.db, and every later Build DB and Build site run then failed at its download. The upload now goes to uisce.db.next, which is never read while uisce.db exists. Only once it has landed is the old uisce.db deleted and uisce.db.next renamed to uisce.db (PATCH on the release asset). A failed upload therefore touches nothing a reader uses. What is left is the gap between the DELETE and the PATCH, two small API calls: GitHub has no atomic replace, and two assets cannot share a name, so some gap is unavoidable while the name stays uisce.db. It is covered twice: - Both downloads fall back to uisce.db.next when the latest release has no uisce.db. In that state uisce.db.next is always the complete new DB, because the old one is only deleted after the upload succeeds. - The next Publish DB finishes an interrupted swap (renames a lone uisce.db.next to uisce.db) before it stages its own upload, so its --clobber on the staging name can never delete the only copy. A first build of the day still uses gh release create, which uploads into a draft and publishes it only once the asset is there, so it never exposed an empty release. Rejected: a new tag per build (the replay writes the tag into closed_at as a date, and the release count would double) and readers walking back to an older release (it would build on a DB that lacks the earlier build's observations, and archive a stale DB as the day's). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ
Dry run of uisce-replay-closed-at over all 75 release snapshots (2026-06-30 to 2026-09-23) against a copy of the 2026-09-23 release: 6,961 transitions found, every one already stamped with the same date, 0 rows to stamp and 0 that would change. The replay_closed_at input is left in place for the owner to decide on; its --limit 100 runs out around 2026-10-19. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ
The dry run over all 75 releases found nothing left to recover, so the input and its step go (owner decision). The script stays for a DB restored from an older release, run by hand. The roadmap entry is closed and the outcome recorded in data-quality.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ
From a code review of this PR: - The swap only repaired today's release, so a run dying mid-swap on its day's last build stranded that release with only uisce.db.next. publish-db.sh repairs the latest published release first. - releases/tags/<tag> 404s on a draft that `gh release view` finds, which wedged the day after a killed create. The script reads assets through `gh release view` (apiUrl) and replaces a leftover draft. - It deleted uisce.db without checking the staged upload was listed; it now stops first. A 5xx on the view was read as "no release" and then failed on create; only "release not found" means absent now. - fetch-db.sh is the one download, with the .next fallback, used by both workflows, the README, build.py's hint and the replay docstring (whose loop now lists up to 1000 releases). - The main-only refusal is its own job and the build-db concurrency group moved onto the build job, so a refused dispatch cannot displace a queued scheduled run. - uv is pinned (0.8.17) beside the setup-uv SHA, so --locked cannot fail on a new uv release. CLAUDE.md gains the replay row. Both scripts were run against a fake gh through: no release, second build, stranded yesterday, stranded today, leftover draft, a 502 on the view, and a lost upload; actionlint and shellcheck pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ
A second code review of the swap found three more ways it could strand a release or quietly lose a build's DB, all from juggling two asset names inside one release. Owner decision: every build publishes its own release, tagged YYYY-MM-DD-HHMM (UTC). gh creates it as a draft, uploads and publishes, so the latest release always holds a complete uisce.db. - publish-db.sh is a single `gh release create --latest`, staging the file as uisce.db whatever it is called (the `#` suffix is only a display label). fetch-db.sh is a plain download with no fallback, so its errors are its own. - replay_closed_at stamps the tag's date part and sorts by tag, since "2026-09-24-1845.db" sorts before "2026-09-24.db" by file name; the documented loop excludes drafts. - uv is pinned at 0.12.18, not 0.8.17, which only offers CPython 3.14.0rc2; 0.12.18 passes `uv lock --check` and installs 3.14.7. - pipeline-dependencies.md's recipe uses fetch-db.sh; data-quality.md records the decision and the rejected swap; CLAUDE.md gains a row. Checked: shellcheck, actionlint, a fake gh (publish, same-minute refusal, a non-uisce file name, fetch), `uvx uv@0.12.18 lock --check`, ruff, pytest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR G of the fixes from the retroactive review: CI safeguards, a publish that can't leave the latest release without a DB, and the removal of the closed_at replay input.
Changes
mainis refused by its own job, which fails visibly. Thebuild-dbconcurrency group sits on the build job, so a refused dispatch can't displace a queued scheduled run.pages.ymlalso requiresworkflow_run.head_branch == 'main'.uv sync --lockedin all three workflows, with uv pinned to 0.12.18 next to thesetup-uvcommit SHA.uv lock --checkon the current lock and installs CPython 3.14.7.YYYY-MM-DD-HHMM(UTC), withgh release create --latest.uisce.db. That is about two releases a day instead of one.gh release upload --clobber, which deleted the old asset before uploading, so a failure in between left the day's release with no DB.scripts/publish-db.shandscripts/fetch-db.share now the only way to publish or download. Both workflows, the README,pipeline-dependencies.mdandbuild.py's refresh hint use them.astral-sh/setup-uvpinned to its commit (c771a70, tag v9.0.0). It runs in the job withcontents: write.replay_closed_atdispatch input and its step are removed (owner decision).Review
First round, 10 findings. The first design replaced the asset inside a per-day release by uploading under a temporary name and renaming. The review found four failure paths in that swap, and those fixes led to the swap being moved into scripts.
Second round, 10 findings. It found three more states the swap could strand:
All three came from juggling two names inside one release, so the owner chose one release per build. That removes the swap and the fallback, and with them 7 of the 10 findings.
The other three:
#suffix ingh release createis only a display label, per gh's source);notes/pipeline-dependencies.mdhad been missed.Also found while testing: snapshot file names sort
2026-09-24-1845.dbbefore2026-09-24.db, so the replay now sorts by tag, and a test covers it.Verification
uvx uv@0.12.18 lock --checkpasses.ghexercised: a publish (the previous release untouched), a second publish in the same minute (refused), a DB file under another name (still published asuisce.db), and a fetch.uv run ruff checkanduv run pytestpass: 687 tests, merged with currentmain.Notes
data-quality.md:CLAUDE.md: rows for the replay input and for one release per build.🤖 Generated with Claude Code
https://claude.ai/code/session_0168hLW2X3mkV3Jhm26LJSwQ