fix(ci): three uploads ignored the input that already gates their consumer — so an artifact-free run is now actually artifact-free (#78) - #79
Conversation
…sumer (#78) Every artifact upload here is `continue-on-error: true` (#76/#77), so a full artifact quota no longer fails a green suite. What it still does is ATTEMPT six uploads per run, three of which are produced for consumers that may be switched off — and on an account at its quota that is six failed API calls a run and a log full of errors nobody can act on. Three inputs already exist and already gate the consuming side. None gated the producing side: `record-timings` gates the `timings` JOB (line 874) that consumes the history — but not the upload that produces it (331) `diagnose` gates every step that READS the records (681/685/851) — but not the per-shard upload (645) nor the merged one (840) So `diagnose: false` switched off the readers and left the writers running. This adds the missing three; `coverage` and `prebuild` already gated theirs. `always()` on the per-shard records is load-bearing and stays: a FAILED shard is exactly when its records are worth having. It becomes `always() && inputs.diagnose`, which keeps that and adds the switch. WITH THIS, `coverage: false, diagnose: false, record-timings: false` UPLOADS NOTHING, which is the point. Measured on FunctionMeasures with `coverage: false` alone, against an account whose artifact quota was FULL: `Shard labels` went green and all eight shards ran — where before every run died there with zero shards executed. A suite that creates no artifacts does not care that artifact storage is full. That is a stronger position than keeping storage below the limit, because it does not depend on anyone remembering to prune. Not the whole of #78 — coverage is still an artifact round-trip when it is on, and sending each shard to Codecov directly would remove that too. This is the part that needs no design decision.
|
📚 Docs preview: https://codes.sota-shimozono.com/TestShards.jl/previews/PR79/ (updates on each push to this PR) |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Closing: this does not fix the failure it was opened against, and I said it did. The three guards are correct in themselves —
So the PR body's claim that it restores the fleet was wrong. What it actually buys is that an explicit Reopening this as part of the real requirement in #80: the split has to work on a free plan, on self-hosted runners, with no artifacts and no hosted runner, while keeping completeness, coverage aggregation and timing history intact. A guard-only change cannot reach that. |
Closes part of #78.
What is wrong
Every upload in
sharded-tests.ymliscontinue-on-error: trueafter #76/#77, so a full artifact quota no longer turns a green suite red. What it still does is attempt six uploads per run — and on an account at its quota that is six failed API calls and a log full of errors nobody can act on.Three of the six are produced for consumers that can be switched off, and the inputs that switch those consumers off do not reach the producers:
record-timingstimingsjob (L874)diagnosecoverageprebuildSo
diagnose: falseswitched off the readers and left the writers running.What this changes
Three
if:guards, using the input names that already exist. No new inputs, no new concepts.always()on the per-shard records is load-bearing and kept: a failed shard is exactly when its records are worth having. It gains the switch rather than losing the semantics.Why it matters — measured
With this,
coverage: false, diagnose: false, record-timings: falseuploads nothing at all.That was worth establishing empirically, so it was. On
FunctionMeasures.jl, withcoverage: falsealone, against an account whose artifact quota was full:Before, every run in the fleet died at
Shard labelswith zero shards executed —main, feature branches and CompatHelper's own branches alike, for eighteen hours.A suite that creates no artifacts does not care that artifact storage is full. That is a stronger position than keeping storage under the limit, because it does not depend on anyone remembering to prune, and it takes effect immediately — GitHub recalculates quota usage only every 6–12 hours, so deleting artifacts does not unblock anything today.
Scope
This is the part of #78 that needs no design decision. Not included:
collectalready has checked out #74 by removing the payload rather than trimming it.collectjob's download (L671) is still unguarded. Withcoverage: falsethe steps that consumeparts/are all skipped, so it is inert rather than harmful — but it is the one remaining unguarded artifact operation.-prebuild/ depot caching belongs with The depot cache has a 0% hit rate: N shards save the SAME key, each uploading the whole depot, and the repo quota evicts everything #72, which reports the depot cache at a 0 % hit rate; moving anything else into that cache before The depot cache has a 0% hit rate: N shards save the SAME key, each uploading the whole depot, and the repo quota evicts everything #72 is fixed would be moving it somewhere already thrashed.