The release-rehearsal pin check passes over zero uses: lines, and nothing counts them

[vladimirrott]

tests/release/release-rehearsal.test.sh:88-92 states the strongest supply-chain
invariant in this repo:

# Positive invariant: EVERY `uses:` in EVERY workflow MUST pin a full 40-hex
# commit SHA. This catches every mutable form (semver tags like @v6.1.0,
# @stable, @main, per-tool tags like @cargo-nextest, and short SHAs), across
# all workflows — not just the publishing one — for a uniform supply-chain
# posture that cannot silently drift.

It enforces that over whatever grep hands it, and never asks whether grep
handed it anything:

$ sed -n '93,108p' tests/release/release-rehearsal.test.sh
for workflow in "${repo_root}"/.github/workflows/*.yml; do
    while IFS= read -r uses_line; do
        ...
    done < <(grep -E '^[[:space:]]*(-[[:space:]]+)?uses:' "$workflow")
done

The file contains no count assertion. grep -nE 'seen|count|-eq 0|-gt 0'
returns nothing.

Measured

Throwaway clone of 83d893d, one mutation at a time, git checkout -- between
each, tree confirmed clean at the end.

$ bash tests/release/release-rehearsal.test.sh; echo "rc=$?"
Release rehearsal contract passed.
rc=0

M1, the calibration. Take the SHA off one action and the check works, names
the workflow, and names the line:

$ sed -i '0,/uses: \([^@ ]*\)@[0-9a-f]\{40\}/s//uses: \1@main/' .github/workflows/docs.yml
$ grep -n 'uses:.*@main' .github/workflows/docs.yml
38:      - uses: actions/checkout@main  # v7
$ bash tests/release/release-rehearsal.test.sh; echo "rc=$?"
FAIL: docs.yml action is not pinned to a 40-hex SHA:       - uses: actions/checkout@main  # v7
rc=1

M2. Now break the extraction instead of the workflows, leaving every real
pin in place:

$ sed -i "s/grep -E '\^\[\[:space:\]\]\*(-\[\[:space:\]\]+)?uses:'/grep -E 'ZZZ_NEVER_MATCHES'/" \
      tests/release/release-rehearsal.test.sh
$ bash tests/release/release-rehearsal.test.sh; echo "rc=$?"
Release rehearsal contract passed.
rc=0

Zero uses: lines inspected, and the check reports the invariant holding across
every workflow. Restored, rc=0 again over a clean tree.

Why the extraction is the part that drifts

Nothing in M2 is hypothetical. That regex has to track YAML's two spellings of a
sequence item, and I shipped the version that only matched uses: at the start
of a line, in a pin checker in another repository of mine, last week. It skipped
every - uses: entry and reported a clean sweep. The same silence arrives if the
workflow directory is renamed, if the glob stops matching, or if a workflow moves
to .yaml. In each of those the check keeps printing Release rehearsal contract passed.

The glob is the second door. for workflow in "${repo_root}"/.github/workflows/*.yml
with no match leaves the literal unexpanded path in $workflow, grep fails on
it, the loop body never runs, and the result is the same green line.

Scope

• Count the uses: lines the extraction found across all workflows, and refuse
  below a floor rather than at zero. Zero is one failure mode; three when the
  tree has thirty is the same failure caught earlier.
• Assert the glob matched at least one file before entering the loop.
• The negative twin belongs in the test's own coverage: a fixture workflow
  spelling a pin in a form the extraction misses should turn the check red, not
  leave it green.
• scripts/check_public_claims.sh:27 and tests/release/node-eol.test.sh:86
  both already carry this guard, in the array form and the counter form. Either
  shape is fine; use one of them rather than a third.

Difficulty

Easy to fix, and the value is in the second bullet of the scope rather than the
first. The interesting work is picking a floor that catches a partial
extraction, not only an empty one.

[vladimirrott]

@vsolano9 offered this to you on #402 as you were merging, and repeating it here so it is on the issue rather than buried in a pull request thread.

This is #295 again, and #313 and #317: a check that grades nothing. tests/release/release-rehearsal.test.sh writes down the strongest supply-chain rule in the repo, that every uses: in every workflow pins a full 40-hex SHA, then enforces it over whatever grep hands it and never asks whether grep handed it anything. Break the extraction instead of the workflows, leaving every real pin in place, and it prints Release rehearsal contract passed.

The issue carries both mutations, including the calibration that shows the check does bite when it is fed anything at all. The part I find more interesting than the empty case is the second scope bullet: a floor catches a partial extraction, and zero is only the loudest way for one to go wrong.

Yours if you want it. Say so and I will label and assign it.

[vsolano9]

Taking this. I will add a floor on extracted uses: lines, assert the workflow glob matched, and add a negative case for an extraction miss so the pin check cannot pass over an empty set.