Coverage counters can travel without the source they are printed against (#74) - #75
Merged
Merged
Conversation
…nst (#74) Julia writes `Foo.jl.<pid>.cov` as a 9-character counter column, one space, and a VERBATIM COPY OF THE SOURCE LINE. Measured on one real 48-file shard payload from `lab-sotashimozono/ParaLinearAlgebra.jl`: total 965,565 bytes counter columns 136,629 bytes (14.3 %) embedded source text 815,592 bytes (85.7 %) lines with a counter 2,134 of 15,181 (14.1 %) Every shard emits counters for the WHOLE tree, not the files it touched, so an 8-shard run ships eight near-identical copies of the package's source through artifact storage. That filled an organisation's Actions storage quota today, and a full quota fails the run at `Shard labels` — before a single test executes. The text never needed to travel: `collect` runs `actions/checkout` BEFORE `download-artifact` (it must — Codecov builds the file network from the tree), so the source is already at the destination, at the same paths and the same revision. Verified on that artifact: the embedded text is byte-identical to the repo file at the run's `head_sha` for 48 of 48 files. `counter_index(cov)` keeps only the counted lines, as `<line>,<hits>`, under a `# lines <n>` header. `restore_counters` rebuilds the `.cov` from the checkout. Measured end to end on the same payload: 965,565 -> 15,614 bytes, 61.8x smaller, with all 48 files rebuilt BYTE FOR BYTE. A zero counter is kept — "reached and never taken" is not "not executable", and dropping it would inflate coverage. THE MISMATCH CHECK IS THE FAILURE MODE THIS INTRODUCES, so it refuses by name. A short or shifted `.cov` does not look broken to CoverageTools: it reports FEWER covered lines, which reads as a coverage drop rather than as a bug. That is the same shape as the silent 54.5 %-for-94.8 % this file's other docstring records, so the index carries the line count the shard saw and a checkout that disagrees — or lacks the file — stops the merge with a message naming the cause. Additive on purpose: a `parts/` tree holding real `.cov` files still restores by copy, so this can be registered BEFORE the workflow starts producing indexes. The workflow side is a separate PR for exactly that reason — `sharded-tests.yml@main` is live for every consumer the moment it merges, while `collect` installs the REGISTERED package, so flipping both at once would break coverage in every repo for the length of the registration lag.
Contributor
|
📚 Docs preview: https://codes.sota-shimozono.com/TestShards.jl/previews/PR75/ (updates on each push to this PR) |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This was referenced Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes half of #74 — the package half. The workflow half is a separate PR on purpose; see the last section.
What the artifacts actually contain
Julia writes
Foo.jl.<pid>.covas a 9-character counter column, one space, and a verbatim copy of the source line. Measured on one livetestshards-coverage-s1artifact fromlab-sotashimozono/ParaLinearAlgebra.jl(48 files):and only 2,134 of 15,181 lines (14.1 %) carry a counter at all. Every shard emits counters for the whole tree, not the files it touched, so an 8-shard run ships eight near-identical copies of the package's source through artifact storage.
That filled an organisation's Actions storage quota today. A full quota fails the run at
Shard labels:— which is before a single test executes. Live artifact totals in that one repo: coverage 170.84 MB against lcov 4.63 MB, records 21.80 MB, out 28.46 MB.
The text never needed to travel
collectrunsactions/checkoutbeforedownload-artifact— it must, since Codecov builds the file network from the tree — so the source is already at the destination, at the same paths and the same revision. Verified against that artifact: the text embedded in the.covis byte-identical to the repo file at the run'shead_sha, for 48 of 48 files.The change
counter_index(cov)keeps only the counted lines,<line>,<hits>, under a# lines <n>header.restore_countersrebuilds the.covfrom the checkout. End to end on the same real payload:Everything downstream of
restore_counters— CoverageTools, the lcov merge, the Codecov upload — is untouched, because the rebuilt file is the original file.A zero counter is kept. "Reached and never taken" is not "not executable", and dropping it would inflate coverage; there is a test for that line specifically.
The mismatch check is the failure mode this introduces
A short or shifted
.covdoes not look broken to CoverageTools — it reports fewer covered lines, which reads as a coverage drop rather than as a bug, and nothing announces it. That is the same shape as the silent 54.5 %-for-94.8 %restore_counters' own docstring records. So the index carries the line count the shard saw, and reconstruction refuses by name when the checkout disagrees or lacks the file. Both refusals are asserted, with the message text.This does not re-open the reason the conversion lives in
collectsharded-tests.ymlrecords it: doing the conversion per shard "cost 3-25 s per shard, the variance being a 20 s JuliaSyntax precompile". That cost is Julia startup, not the conversion — the producing side is a line filter (awk, no Julia, no depot), and the merge still happens exactly once, incollect.Why the workflow is a separate PR
sharded-tests.yml@mainis live for every consumer the moment it merges, whilecollectinstalls the registered package (testshards-specdefaults toname="TestShards"). Flipping both at once would have every repo uploading indexes that the registeredrestore_counterscannot read — coverage broken everywhere for the length of the registration lag.So this PR is additive only: a
parts/tree holding real.covfiles still restores by copy, and there is a test for that (a raw .cov still restores by copy, so a mid-upgrade run is not broken). Register this, then flip the workflow.0.3.34 → 0.3.35, patch.