feat: the completeness evidence gets a second road that a full artifact quota cannot close (#74) - #85
Merged
Merged
Conversation
…ct quota cannot close (#74) `Every unit ran, exactly once` is a CORRECTNESS claim — this file's own comment says only such a claim may fail a suite — and it is the one thing here that cannot be skipped when its input goes missing. A full artifact storage quota took the artifact road out and the gate with it, and a fleet of green suites read red for six days. Cache is the transport that clears every constraint this workflow has: job outputs matrix legs OVERWRITE each other Checks/Actions API need `checks: read` / `actions: read`, and a reusable workflow may not request more than its CALLER granted, so asking makes every consumer fail to START a new action a new action and its first reference cannot land together (#84) cache none of the above, and a quota of its own MEASURED FIRST, on a probe branch, 8 legs and a collector, with a negative control that asks for a key nobody saved: never-saved key hit='' s1..s8 hit='true' files=3 distinct legs recovered: 8/8, 480 bytes THE PATH MUST MATCH THE SAVE'S EXACTLY, and that is not obvious: a cache entry is identified by its key AND by a version derived from the `path` inputs. The probe's first run saved `evidence` and restored into `parts/sN` and read 'Cache saved with key: X' in the leg against 'Cache not found for input keys: X' in the collector, same run, same key. So the collector restores to the shard's own path and moves the files afterwards. The cache road feeds ONLY the gate. The records merge keeps reading `parts` alone: two roads delivering the same `records-*.jsonl` would double every record, while the gate's three lists are `sort -u`'d and are therefore idempotent under duplication. A LADDER rather than a loop, because `uses:` cannot be looped and the shard count is an input. Sixteen covers twice the default; a run with more shards keeps today's behaviour. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
📚 Docs preview: https://codes.sota-shimozono.com/TestShards.jl/previews/PR85/ (updates on each push to this PR) |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…low twice This repo's own CI does exactly that: `ci` with 8 shards and `solo` with 1. A key of `run_id + sid` alone made solo's collector restore ci's shard s1, and solo's gate reported '3 unit(s) ran twice: 1 on local and s1' — its own evidence is labelled `local`, what it picked up was ci's. The probe could not have found this: it had one caller. The self-test did, because it has two. `artifact-prefix` is the discriminator this workflow already uses for the same reason on the artifact road. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Additive: the artifact road is unchanged and every new step is `continue-on-error: true`, so the second road cannot fail a suite by existing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every unit ran, exactly onceis a correctness claim — this file's own comment says only such a claim may fail a suite — and it is the one thing here that cannot simply be skipped when its input goes missing. A full artifact storage quota took the artifact road out and the gate with it, and a fleet of green suites read red for six days.Why cache, and not the others
outputschecks: read/actions: read, and a reusable workflow may not request more than its caller granted — asking makes every consumer fail to start, with no logPayload measured on a real consumer: ~373 B per file, three files, ~1.1 kB a shard.
Measured first, on a probe branch
8 legs and a collector, with a negative control that asks for a key nobody saved — so a restore that always succeeded would be caught before any count could be read as evidence:
The trap that run found, and it is not obvious
A cache entry is identified by its key AND by a version derived from the
pathinputs. The probe's first attempt savedevidenceand restored intoparts/sN, and read— same run, same key. So the collector here restores to the shard's own path and moves the files afterwards. Anyone reading only the key would have shipped a silent miss.
(The probe also spent two runs at zero jobs:
${{ … }}inside a{ … }YAML flow mapping closes it on its own}}. That isstartup_failure, not a result.)Scope
The cache road feeds only the gate. The records merge keeps reading
partsalone: two roads delivering the samerecords-*.jsonlwould double every record, while the gate's three lists aresort -u'd and are therefore idempotent under duplication.A ladder rather than a loop, because
uses:cannot be looped and the shard count is an input. Sixteen covers twice the default; a run with more shards keeps today's behaviour. Every step iscontinue-on-error: true— the second road must never be able to fail a suite by existing.What this does not change
The artifact road stays exactly as it is. Coverage still travels by artifact, which is fine since #75 removed the 85.7 % of each counter file that was source text already present at the destination. This PR only makes the correctness gate survive a road being closed.
🤖 Generated with Claude Code