[vyncint]

What is wrong

Every guard under scripts/ is wired twice: a step in a workflow, and a line in the Justfile's check-guards recipe. The Justfile states that as a contract —

check-guards covers the status-guard, naming-guard and cargo-deny workflows in full

— and nothing enforced it. It was already false. check-book-catalog-stamp.sh landed in #1106 wired into status-guard.yml and never added to check-guards, so twelve of the thirteen guards ran locally and "in full" was wrong.

That direction of drift is the one that happens, because forgetting the Justfile line breaks nothing visible: CI still runs the guard, just check still passes, and the only symptom is that a contributor who runs the documented local gate before pushing does not run it — exactly the person the recipe exists for. The opposite direction is worse and just as quiet: a guard in check-guards but in no workflow reads as enforced and never runs on a pull request.

The change

scripts/check-guard-coverage.sh compares three sets — the check-*.sh scripts on disk, the ones named in the recipe body, and the ones named under .github/workflows/ — and fails on any asymmetry, with a different message per case. Adding the missing recipe line makes it pass.

smoketest.sh and debug-smoketest.sh are deliberately out of scope: the first is the example suite with its own CI job and just smoketest recipe, the second a developer tool. Only the check-* family is a guard.

Verification

mutation	message
drop check-book-catalog-stamp.sh from the recipe	runs in CI but not in just check-guards
name a script in the recipe that no workflow runs	runs in check-guards but in no workflow — plus "invoked but does not exist"
a real guard drops out of status-guard.yml	runs in check-guards but in no workflow, naming it
rename the check-guards: recipe	refuses: the recipe body was not found, fix the script
a new check-*.sh nobody wired	exists but nothing runs it

Unmutated: OK: all 14 guard scripts run in both a workflow and just check-guards, and all fourteen still pass individually, the new one included. Reads three text files — no cargo, no network, no toolchain. bash -n clean, SPDX header present, mode 755 like its siblings.

Note for review

#1102 also edits the Justfile, in the test-cuda recipe around line 105. This change is inside check-guards around line 235, so the two do not overlap textually — I verified the hunks. Happy to rebase if you would rather it land after.

[nihalpasham]

• Superseded by the formal changes-requested review on the current head.

[nihalpasham]

• Blocker: the guard is still a text search, so it can pass when the real PR job no longer runs just guard.
• Counterexample: remove the reachable workflow step, then place the same run: text inside a YAML block scalar or an if: false step. The guard stays green even though no PR path executes it. A reusable workflow that is never called has the same problem.
• Required fix: parse workflow YAML structurally and walk the PR-reachable workflow graph, including workflow_call, conditions, and continue-on-error. The Justfile check needs the same structural treatment instead of matching dumped text.
• The local guard passes on these counterexamples, so its green result is not enforcement evidence yet.

[vyncint]

Rebuilt as a structural check. Force-pushed as a single commit; the text search is gone.

Workflows are a graph now. The YAML is parsed and reachability computed: a workflow is an entry point only if it triggers on pull_request/pull_request_target; a reusable workflow counts only if a reachable job uses: it, resolved to a fixpoint; a job or step proven if: false is dropped; and anything that may continue-on-error is not credited, since a swallowed failure is not enforcement.

A run: script is parsed into commands, and a guard counts only in command position — echo "bash scripts/x.sh" runs echo. Leading VAR=value assignments and wrappers (bash, sh, env, ., source, exec, time, sudo) are stepped over, comment lines dropped, and &&/||/;/| split a line so a guard mid-pipeline still counts.

The Justfile gets the same treatment. Recipes and dependencies are a graph walked from check, so a guard moved into a recipe check does not reach stops counting. A body line prefixed with - is just's continue-on-error and does not enforce. Same command-position rule, so a recipe that echoes an invocation is not running it.

Your three counterexamples, each now failing:

defeat attempt	caught as
run: text only echoed	executed by just check but by no pull-request workflow
step behind if: false	same
only in a reusable workflow nothing calls	same
step with continue-on-error: true	same
job with continue-on-error: true	same
moved to a workflow with no pull_request trigger	same
Justfile line prefixed with -	executed by a workflow but not by just check
moved to a recipe check does not reach	same
the Justfile echoing it	same

And seven refusals, so a broken parse cannot read as clean: nothing triggering on pull_request, either recipe renamed, check no longer depending on check-guards, a uses: target lacking workflow_call, a uses: target that does not exist, and unparseable YAML.

Reachability is conservative in one direction only, deliberately: an if: that is not literally false is treated as reachable, because github.event_name == 'pull_request' cannot be evaluated statically. The guard never calls a live step dead. It is strict the other way — a step that may be skipped or may swallow its failure is not coverage. If you want the stricter reading (treat a non-literal if: as unproven and refuse), that is a one-line change in statically_false.

One subtlety that decides whether any of it works, recorded in the script: YAML 1.1 reads a bare on as the boolean true, so PyYAML returns {True: ...} and document.get("on") is None on all eleven workflows here. Both keys are looked up, and a self-test refuses if no workflow appears to trigger on pull_request — the shape that would otherwise leave the reachable set silently empty.

One environmental assumption to flag: structural parsing needs PyYAML. The script probes python3, /usr/bin/python3 and python in turn, because the first python3 on PATH is not always the one that has it (where Homebrew shadows the system Python, it is not), and refuses with the install command if none does. /usr/bin/python3 on ubuntu-latest carries it, but if your runners differ this job will fail loudly rather than silently — say so and I will add an install step.

0.09s, no cargo, no network, no toolchain. All 14 guards still pass individually, and the branch merges cleanly with the rest of the open batch.

[vyncint]

Two changes since your review, both from things I should have resolved before asking.

The PyYAML dependency is gone

I had left it as an "environmental assumption" for you to rule on. That was wrong of me — checking it settles it: PyYAML is not in the actions/runner-images preinstalled manifest. Depending on it risked the job failing on merge, and no other guard in this family needs a third-party package, on a repo that already works around macOS's bash 3.2 and BSD grep for the same reason.

The YAML is now read by a strict-subset reader in the script. It covers exactly what these eleven files use — block mappings and sequences, plain and quoted scalars, literal | and folded > block scalars with chomping, flow sequences, comments, one leading document marker — and refuses on anything else: a tab, an anchor, an alias, a tag, a flow mapping, a second document. That is the property that separates it from the text search it replaces: it cannot silently misread, only decline to read.

Folding is implemented rather than approximated, because book.yml folds one command across two lines and treating it as literal would invent a second command whose first token is cuda-oxide-book.

Where PyYAML is installed the script uses it as an oracle — comparing its own parse against PyYAML's on every workflow, over exactly the fields the walk reads, refusing on any disagreement. Locally that reports subset YAML reader agrees with PyYAML on all 11 workflows. So the reader is checked rather than trusted, without being required.

The nine defeats are now tests, not a comment

I had verified your counterexamples by hand-mutating the repository and pasting the results here. That is neither repeatable nor visible in CI, and under the tightened criteria it is not the evidence it should be. They are now fixtures the guard asserts about itself on every run, alongside controls that catch a rule grown too strict (a live step, a guard mid-pipeline, a called reusable workflow, @-suppressed Justfile lines) and four reader refusals.

Each was confirmed to bite by breaking the implementation it guards:

implementation broken	self-test that failed
step if/continue-on-error check removed	a step behind if: false was counted
job continue-on-error check removed	a job with continue-on-error was counted
command position → any token	the invocation only echoed was counted
reusable-workflow fixpoint removed	a called reusable workflow was not counted
Justfile - skip removed	a - prefixed script path was counted
Justfile reachability walk removed	a recipe check does not reach was counted
reader's tab refusal removed	the reader accepted a tab
reader's flow-mapping refusal removed	the reader accepted a flow mapping

That last-but-three row is worth a note: my first - fixture used -bash scripts/x.sh, which the guard ignores anyway because -bash is not a recognised command — so the test passed for the wrong reason and the explicit - skip was not actually covered. The spelling that needs it is -scripts/check-x.sh, where basename strips the prefix and would otherwise credit it. Both spellings are now fixtures.

All nine end-to-end defeats still fail against the real tree after the refactor. All fourteen guards pass, 0.02s, python3 only.

One thing to push back on if you want

At 835 lines this is now the largest guard in the family by a good margin, and that is what structural parsing without a dependency costs. If you would rather have the dependency and a much shorter script, say so and I will swap it — the oracle code becomes the parser and most of the file goes away.

There is also a small ASCII flow of the reachability walk in the header now.

[vyncint]

One more commit, from reading the PRs closed on 2026-08-24 back against this branch.

#1167 was closed partly because "the cargo-oxide compatibility finalization path remains unguarded", and #1169 because its instruction table "omits" families — neither for an observed failure, both for a path the change did not enumerate. This guard had one of those.

workflow_coverage read literal bash scripts/check-*.sh commands out of a run: script. A step that runs just check-guards executes every guard in that recipe, and the walk saw only the command just. So a workflow written that way would have had every one of its guards reported as absent from CI — a false failure that blocks the run, with the verdict exactly backwards.

Nothing under .github/workflows/ invokes just today, which is why the corpus never showed it. That is rather the point: the guard's job is to be right about the wiring, not about this week's wiring.

A just <recipe> command is now resolved through the Justfile recipe graph the script already parses — the authoritative list in that case, rather than a second one derived here — and the guards that recipe reaches are credited to the step. - still does not enforce inside a recipe reached that way, and the recipe named on the command line is its own entry point, so reachability from check does not apply to it. recipes_reachable_from and recipe_guards are now shared by both halves instead of the walk existing twice.

Three fixtures, each failing before the change: a step running just check-guards credits a guard nested one dependency deep; the same step credits nothing when no Justfile is supplied; and a - prefixed line inside the recipe just runs is still not counted.

Everything else holds — all fourteen guards pass, the nine end-to-end defeats still fail, the subset reader still agrees with PyYAML on all 11 workflows, 0.02s.

[nihalpasham]

Closing this PR because the hand-built workflow, Just, and shell interpreter still cannot establish that a pull request actually executes a guard and propagates its failure.

• The latest c5e95c6b commit adds just <recipe> resolution, but it does not resolve the execution-model blocker.
• On the clean merge tree, the checker remained green after seven independent defeats: a push-only job condition, guard || true, a Just pipeline to cat, an unsatisfied needs dependency, a pull-request paths filter, pull_request_target, and checkout of the base SHA.
• Every mutation exited 0 while reporting all 14 guards covered.

Extending this 919-line parser until it models GitHub Actions reachability, shell failure propagation, checkout provenance, and Just semantics is substantial new architecture rather than a small maintainer correction. A replacement should make CI and local checks consume one authoritative execution path or manifest, with fail-before/pass-after mutations and an unaffected control.

Please align on that design first under the acceptance criteria shared in Discord.