Three defects in one day shared a cause: this repository is not a consumer of itself in the ways that matter.
| defect |
why CI missed it |
actions/upload-coverage failed to load — an expression in an input's description |
nothing invoked the action |
| a secret did not reach the reusable |
dogfooded through a local ./ reference, never cross-repository |
errexit killed the absent-ci-timings path (#54) |
this repo HAS a ci-timings branch, so ls-remote returns 0 and the failing branch is unreachable |
The third is the sharpest: the code handling "no timing history yet" could not be reached by having no timing history, and it shipped. It surfaced on the first adopter that happened to be fresh — and every repository that has never been sharded is fresh, so the next sixteen adopters would all have hit it.
What would have caught them
action-smoke already covers the first, added when it bit. The other two need the same treatment: a job that takes the path a new consumer takes.
Concretely for the third: extract the history read into a script the workflow calls, so a CI job can run that script — not a copy of it — against a deliberately absent branch under bash -e and assert it survives. A copied snippet would not catch a reintroduced out=$(...); rc=$?, which is the actual regression to prevent.
The general shape: for each state a consumer can be in that this repository is never in — no timing history, no Codecov token, a private caller, a first run — there should be a job that is in it.
Not urgent, but not optional
Each of the three cost a red run in someone else's repository and a bisect from the wrong end. The fix for each was one line; finding it was not.
Three defects in one day shared a cause: this repository is not a consumer of itself in the ways that matter.
actions/upload-coveragefailed to load — an expression in an input'sdescription./reference, never cross-repositoryerrexitkilled the absent-ci-timingspath (#54)ci-timingsbranch, sols-remotereturns 0 and the failing branch is unreachableThe third is the sharpest: the code handling "no timing history yet" could not be reached by having no timing history, and it shipped. It surfaced on the first adopter that happened to be fresh — and every repository that has never been sharded is fresh, so the next sixteen adopters would all have hit it.
What would have caught them
action-smokealready covers the first, added when it bit. The other two need the same treatment: a job that takes the path a new consumer takes.Concretely for the third: extract the history read into a script the workflow calls, so a CI job can run that script — not a copy of it — against a deliberately absent branch under
bash -eand assert it survives. A copied snippet would not catch a reintroducedout=$(...); rc=$?, which is the actual regression to prevent.The general shape: for each state a consumer can be in that this repository is never in — no timing history, no Codecov token, a private caller, a first run — there should be a job that is in it.
Not urgent, but not optional
Each of the three cost a red run in someone else's repository and a bisect from the wrong end. The fix for each was one line; finding it was not.