Repository navigation
ci: run the conformance gate on every PR, ratcheted by a baseline - #86
Merged
Merged
Conversation
Closes the remaining half of #81. The gate landed in #82 but nothing ran it, so it could only catch defects for whoever remembered to invoke it by hand. Blocking rather than advisory. An advisory job that is permanently red is noise people learn to scroll past, and the gate currently reports real findings in three of four members. So known findings are waived in family-conformance-baseline.json and the job fails only on a NEW divergence. Two rules make the waiver list a ratchet instead of a place defects go to die: - every waiver must name the issue tracking it. A waiver without a ticket is just a hidden defect with extra steps. - a waiver that no longer fires FAILS the gate. Fixing a defect and leaving its waiver behind would silently pre-accept the next regression, so fixes have to shrink the file. Verified in all three directions, because a gate that cannot fail is worthless: known findings waived -> exit 0 a waived finding un-waived -> exit 1, reported as not in the baseline a waiver that no longer fires -> exit 1, reported as STALE BASELINE srdcheck itself passes with no waivers. The baseline's sibling entries are consulted only when those repos are audited, so this job reports nothing for them; the sibling CI jobs come next and will read the same canonical file rather than copying it, per FAMILY.md's pin-by-link rule. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the remaining half of #81. The gate landed in #82, but nothing ran it — it could only catch defects for whoever remembered to invoke it by hand, which is the same failure mode as
SKILL.mdbeing prose nobody executes.Blocking, not advisory
The obvious move is a non-blocking advisory job. That produces a permanently-red check people learn to scroll past — the gate currently reports real findings in three of four members, so it would be red from day one and stay red.
Instead: known findings are waived in
family-conformance-baseline.json, and the job fails only on a new divergence. That makes it blocking and useful immediately.Two rules keep the waiver list a ratchet rather than a place defects go to die:
Verified in all three directions
A gate that cannot fail is worthless, so I tested it failing:
0 not in the baseline1 not in the baselineSTALE BASELINE: … Remove it from the baselinesrdcheck itself passes with no waivers of its own.
Scope
This job audits this repo. The baseline's sibling entries are consulted only when those repos are actually audited, so this job reports nothing for them and cannot go stale on their behalf.
Sibling CI jobs come next — three small PRs, each checking out srdcheck and running the same canonical gate and baseline against itself rather than copying either, per FAMILY.md's pin-by-link rule.
Note
tests/test_event_apply_validation.py::test_deep_json_parse_boundaries_do_not_crash_cli_or_mcp_stdiofails on local Python 3.14 (-32600 == -32700). That is #72, pre-existing and verified identical on untouchedmain; CI's 3.10–3.13 matrix is unaffected. Full suite otherwise: 711 passed.🤖 Generated with Claude Code