Repository navigation
Conversation
pawan's main-branch protection requires exactly one status check, context `Coverage + CRAP`, and no job on main has ever produced it -- so every merge into main has been blocked. This workflow supplies that check. One job, `Coverage + CRAP`, one instrumented run feeding two gates: - cargo-tarpaulin 0.37.5 (--engine Llvm) runs the whole suite and writes lcov.info, gated with --fail-under 58. - cargo-crap 0.6.1 scores CRAP(m) = CC^2 * (1 - cov)^3 + CC over every function of the five pawan crates from that same lcov.info, ceiling 510, with the two 0%-by-design binary entry points (**/main.rs) excluded. Thresholds are measured, not estimated: the exact command was run in an isolated worktree on origin/main (60.13%, 9168/15248) and on the PR #140 head 90f8a53 (60.76%, 9530/15685); the floor is ~1pp below the runner's own 59.73% on the origin/main base, which absorbs the measured 0.40pp runner-vs-build-host delta. Worst CRAP is 431.04 on both bases (handle_session_browser_key, CC 22, 1/22 lines); the ceiling is 510 because CC^3 + CC = 506 is what that function scores if its single covered line is measured uncovered. Both gate directions were proven before the thresholds were committed: --fail-under 99 exits 1 ("Coverage is below the failure threshold 60.09% < 99.00%") and --threshold 400 exits 1, while 58 and 510 exit 0. ci.yml is untouched. tools are pinned release assets (actions-rs/tarpaulin is archived) with in-job version assertions.
|
Ready for review — this is the gate pawan's protection has always required (it never existed). What to review: The numbers (all measured, all with the same command on the same bases):
Tooling (pinned release assets, in-job version assertions): cargo-tarpaulin 0.37.5 (2026-09-27) and cargo-crap 0.6.1 (2026-09-29) — a real maintained crate, not a bespoke script; Resource note: Ratchet path (per the owner's directive): raise |
What this adds
One new workflow,
.github/workflows/coverage.yml, containing one job whosecheck-run context is exactly
Coverage + CRAP— the single status check pawan'smain-branch protection has always required and that has never existed.
ci.ymlis byte-identical afterwards (no existing job is modified, reordered, orre-run). No workspace dependency is added.
Two gates, one instrumented run:
--engine Llvm) runs the whole suiteunder instrumentation and writes
lcov.info, gated with--fail-under 58.writes:
cargo crap --lcov lcov.info --workspace --exclude '**/main.rs' --threshold 510 --fail-above. cargo-crap never runs tests itself — it parsessource with
synand computesCRAP(m) = CC² · (1 − cov)³ + CC. With nocoverage feed every function scores
cov = 0and the ceiling measures nothing,which is why both gates share one run.
Why this is the check protection has always required
Coverage + CRAPis in the required-contexts list, but no workflow or job onmainhas ever produced a check with that name — so the check has beenpermanently unsatisfied and every merge into
mainhas been blocked by it.This workflow supplies it. The check-run name was confirmed empirically after
this workflow's first run by reading
gh api repos/dirmacs/pawan/commits/<sha>/check-runsand comparing thenamefield character-for-character (the context is the job
name:, not the workflowname and not the job id).
First run, observed
On this PR's head (
95b30c3), the check-runs API reports exactly one check namedCoverage + CRAP, byte-for-byte (15 bytes:C o v e r a g e ␣ + ␣ C R A P),conclusion success, from run
https://github.com/dirmacs/pawan/actions/runs/38147255596/job/114493108545.
All ten steps
success; the suite ran green under instrumentation; the CRAP stepcompleted under
--threshold 510 --fail-above. That 59.75% on the runner is thesame 0.40pp-below-the-build-host delta measured on the
origin/mainbase, whichis exactly what the floor's margin is sized for.
Measured coverage on both bases
Measured, not estimated. Each number comes from running the workflow's exact
command —
cargo tarpaulin --engine Llvm --workspace --locked --out Lcov --timeout 600withRUSTFLAGS=-C debuginfo=line-tables-only, stable,x86_64 — in a clean isolated worktree checked out at that base.
origin/maine15a5d2guardrail/appa-integration(PR #140 head)90f8a53Per crate (
origin/main→90f8a53):Both suites ran green under instrumentation (0 failed tests on either base), so
these are full-suite numbers, not partial ones.
Threshold and margin
COVERAGE_FLOOR = 58.The lower of the two measured bases is 60.13% (
origin/main). The margin isapplied to the number the gate itself will see on that base rather than to the
build host's: a GitHub-hosted
ubuntu-latestrunner measured 59.73%(9107/15248) on the
origin/mainbase — 0.40pp below this build host with anidentical line denominator. So:
which absorbs the runner delta plus normal run-to-run noise on a floating
@stabletoolchain. Both bases clear it, on the runner and on the build host.Crate scope
--workspace— all five pawan crates, nothing hand-picked. pawan-core is 58.8%of the surface and the reason the gate is worth having (71.3%), but pawan-cli is
38.6% of it, and excluding it would leave the largest surface unguarded.
pawan-api (223 lines), pawan-mcp (136) and pawan-aegis (24) are small enough that
including them costs nothing and keeps the denominator whole.
One disclosed scope artifact: tarpaulin's tree walk also lists the separate
grind/workspace (its ownCargo.lock, run bymise grind-test, never builtor run by this job) at 489 lines / 0%. That is a constant −3.2pp drag on the
number tarpaulin reports. It is disclosed rather than silently excluded; the run
summary prints both the gated number and the crates-only number so the ratchet has
the cleaner metric available.
Pinned tool versions
actions-rs/tarpaulinis archived (the actions-rs org was archived 2023-10-13, last push 2023-02-04)dtolnay/rust-toolchain@stable+llvm-tools-previewci.yml; llvm-tools backs theLlvmengineBoth versions are asserted inside the job
(
cargo-tarpaulin --version | grep -qx "tarpaulin 0.37.5"and the cargo-crapequivalent) so two runs cannot disagree because one host had a newer tool. Both
measured baselines above were produced with exactly these versions.
Release-asset install rather than
cargo installin CI:cargo installwouldrecompile tarpaulin from source on every run for a binary whose contents are
already reproducible in the pinned release, and the release is what the
thresholds were measured against.
CRAP approach
A real maintained crate, not a bespoke script: cargo-crap 0.6.1, fed by this
run's
lcov.info.The measured worst on both bases is the same function at the same score:
431.04 =
handle_session_browser_key(
crates/pawan-cli/src/tui/events/default.rs:36, CC 22, 1/22 lines covered =5.45%). The ceiling is 510 rather than 431 because that score is knife-edged:
the same function scores
CC³ + CC = 506if the llvm engine measures its singlecovered line as uncovered, so 510 clears both outcomes on both bases.
--exclude '**/main.rs'keeps the two binary entry points out of the ceiling:argument parsing, process wiring and setup, 0% covered by design (worst measured
CC 42 ⇒ CRAP 1806). Gate those numbers and the ceiling measures nothing; silence
them and it is meaningful.
The ceiling caps the single worst function's score. It is not a cap on how
many functions exceed a lower bar: 60 of 1053 scored functions on
origin/main(1135 on
90f8a53) sit above the tool's default 30, 44 above 50, 20 above 100 —identically on both bases.
The ceiling's teeth were proven before this PR was opened: the same LCOV scores
rc = 1at--threshold 400(worst 431.04) andrc = 0at--threshold 510.Ratchet plan
runner margin whenever a PR improves coverage. Only ever upward, and only via a
PR that measured the new number in this job.
handle_session_browser_keygains real coverage,re-measure and drop the ceiling toward the new worst + margin. Then walk the 44
functions above 50 and the 20 above 100.
currently exceed it) once the worst-offender ceiling has been tightened.
ENOSPC / OOM remediation
RUSTFLAGS: -C debuginfo=line-tables-onlyis baked into the job's env: full debuginfo is what fills the disk and the linker's RSS on coverage runs, and line tables
carry every mapping tarpaulin needs. The job deliberately does not carry
-D warnings— lint enforcement isci.yml'sCheck & Lintjob, and this job'sexit code has to mean "gate", not "lint".
Notes
90f8a53) is what it unblocks, and it is measured above.ci.ymlas a job namedCoverage + CRAP(and its run measured 59.73% on theorigin/mainbase, whichis where the runner-vs-build-host delta quoted above comes from). This PR takes
the separate-workflow route so
ci.ymlstays byte-identical. Whichever landsfirst should be kept and the other closed, so that two checks named
Coverage + CRAPnever coexist.