Add a family conformance gate that executes each SKILL.md documented exit contract

[chaoz23]

Every member's SKILL.md passed the clause-7 acceptance test ("a fresh-context agent, given only this file, can produce a well-formed invocation") when the front doors landed. All four still misstate their own exit-code contract.

That test checked invocability. Nothing checked truth. This issue proposes the missing gate.

What it does

scripts/family_conformance.py (stdlib-only, no deps) executes a member's CLI with read-only probes and diffs observed behaviour against what its SKILL.md claims:

• UNDOCUMENTED_EXIT_CODE — the CLI returns a code SKILL.md never documents
• HONEST_LANE_OVERLOAD — SKILL.md reserves a code for cannot-adjudicate with do-not-retry guidance, but a plain usage error returns the same code
• STREAM_MISMATCH — SKILL.md says failures print to one stream; they print to the other
• MISSING_PIPE / SCHEMA_NOT_FLAG — clause 7 flags codified by name that the CLI rejects

It obeys the contract it enforces: exit 0 conformant / 1 findings / 2 cannot-adjudicate, plus --pipe, --schema, and an action field on the honest lane. Pointed at a repo with no SKILL.md it exits 2 rather than guessing.

Current results

repo	verdict
srdcheck	PASS
charactercheck	FAIL — honest-lane overload
dmcheck	FAIL — missing --pipe, honest-lane overload, stream mismatch
table-kit	FAIL — missing --pipe, --schema not a flag, honest-lane overload

It reproduces 4 of the 8 independently-confirmed audit findings plus part of a fifth, with zero per-repo configuration.

Known blind spots, stated up front

• no MCP-surface probe (misses table-kit's missing server)
• no worked-example runner (misses SKILL.md examples that don't reproduce)
• srdcheck's specific exit-2 semantic gap is unreachable by the generic bad-flag probe, since that probe returns 3, which srdcheck correctly documents

Adoption

It currently fails 3 of 4 repos, so land it non-blocking first, fix, then flip to blocking.

Before adoption: downgrade HONEST_LANE_OVERLOAD from high to medium (it fires on 3 of 4 members, and a rule that flags most of the family at high trains you to ignore it).

One caveat worth recording: the first version of this gate passed dmcheck clean. The stream claim wraps across two source lines ("Failures print JSON on" / "stderr, never a traceback.") and a line-based matcher saw neither half. A checker that reports PASS because it structurally cannot perceive the defect is the exact failure class this gate exists to catch. Fixed by matching at paragraph level — and the reason it was validated against known-bad repos rather than trusted on a green result.

---

Found by a conformance audit of the check-family four (srdcheck, charactercheck, dmcheck, table-kit) against FAMILY.md v1 on 2026-08-17. Audited at working-tree HEAD, which was 1–3 commits ahead of origin/main and 0 behind, so absences hold for main. Every claim below was verified by execution or by git grep across all local heads, then independently re-derived by a second pass instructed to refute it.

[chaoz23]

Done. The gate exists, runs everywhere, and is blocking.

gate	#82
stream-claim false positive fixed	#84
class-aware (FAMILY.md v2.1)	#85
CI + baseline ratchet, this repo	#86
CI in siblings	chaoz23/charactercheck#41 · chaoz23/dmcheck#19 · chaoz23/table-kit#28

Landed blocking rather than advisory, contrary to the plan in this issue's original description. An always-red advisory check is noise people scroll past, and the gate reports real findings in three of four members. Known findings are waived in family-conformance-baseline.json, so it fails only on a new divergence — every waiver names the issue tracking it, and a waiver that stops firing also fails, so fixes shrink the file rather than leaving stale permission behind.

Siblings read the gate and baseline from this repo pinned by SHA rather than vendoring copies, matching the cross-repo convention already in table-kit.

Current state — every finding maps to an open issue, no false positives:

PASS  srdcheck        class=verdict
FAIL  charactercheck  class=verdict     HONEST_LANE_OVERLOAD   -> chaoz23/charactercheck#18
FAIL  dmcheck         class=verdict     MISSING_PIPE           -> chaoz23/dmcheck#14
                                        HONEST_LANE_OVERLOAD   -> chaoz23/dmcheck#15
FAIL  tablekit        class=transport   SCHEMA_NOT_FLAG        -> chaoz23/table-kit#23
                                        HONEST_LANE_OVERLOAD   -> chaoz23/table-kit#1

The baseline is now the work queue. Two blind spots remain and are recorded in #82: no MCP-surface probe, and no worked-example runner.