From 922e23015d1c2dd34823e39441545dabe220ca6e Mon Sep 17 00:00:00 2001 From: chaoz23 Date: Tue, 25 Aug 2026 14:15:42 -0700 Subject: [PATCH 1/2] feat: FAMILY.md v2.2 harmonises exit 3; gate can see codes above 3 Two halves of one problem, surfaced by chaoz23/charactercheck#18. CONTRACT. v2.1 said "srdcheck's exit 3 is the family precedent", which was too weak to settle what a member should do when 3 is already spent. v2.2 states the pattern outright: 0/1/2 are the universal verdict contract, >= 3 is the no-verdict taxonomy, and 3 is usage error in EVERY member. A malformed call is the one non-verdict outcome every tool has, so harmonising it means an agent that mis-invokes any member gets the same answer. Further outcomes take 4+. charactercheck therefore moves could-not-retrieve from 3 to 4 rather than taking a different usage code. Decided deliberately over the cheaper option of letting it keep 3 and putting usage on 4: that would have left the family with two spellings for the concept an agent hits most often. GATE. parse_skill capped recorded codes at 0..3, so a documented exit 4 was silently dropped -- and the CLI returning 4 then surfaced as a bogus UNDOCUMENTED_EXIT_CODE against a tool that had documented itself correctly. The gate would have rejected the very refactor the contract now requires. That is the third time this parser has been the defect rather than the tools it checks (line-vs-paragraph in #82, stream-word matching in #84). The shape repeats: the checker silently cannot perceive the thing it is checking. Verified both directions: a synthetic 0/1/2/3/4 SKILL.md now parses all five with honest-lane=2 and usage=4, and all four real SKILL.md files parse unchanged. Co-Authored-By: Claude Opus 5 --- FAMILY.md | 34 ++++++++++++++++++++++++++-------- scripts/family_conformance.py | 9 +++++++-- 2 files changed, 33 insertions(+), 10 deletions(-) diff --git a/FAMILY.md b/FAMILY.md index 4dddcf0..e9ed8ce 100644 --- a/FAMILY.md +++ b/FAMILY.md @@ -1,4 +1,4 @@ -# The check family — shared contract (v2.1) +# The check family — shared contract (v2.2) Canonical copy: this file, in [srdcheck](https://github.com/chaoz23/srdcheck) (the family's reference implementation). Sibling repos link here. @@ -39,13 +39,25 @@ Members differ by what they produce, and the contract binds them accordingly. cannot-adjudicate answer meaning the question is outside the tool's codified jurisdiction, or that the tool cannot answer it from the evidence it has (unknown content, discretion, ambiguity, incomplete coverage) — is the - contract of the verdict class: srdcheck and dmcheck use exit `2`; - charactercheck uses exit `2` for unhandled content and `3` for could-not- - retrieve. Consuming agents route the honest lane to a human; never retry or - guess. Usage errors are **not** the honest lane and should not share its exit - code; srdcheck's exit `3` is the family precedent. Each SKILL.md states its - own tool's exact contract — read that, not this table, when invoking, and see - clause 7 on how that statement is kept true. + contract of the verdict class: srdcheck, dmcheck and charactercheck all use + exit `2` (for charactercheck, unhandled content). Consuming agents route the + honest lane to a human; never retry or guess. Usage errors are **not** the + honest lane and must not share its exit code. + + **`0`, `1` and `2` are the universal verdict contract.** Codes `3` and above + are the *no-verdict-was-produced* taxonomy: a consuming agent branches on + 0/1/2 and treats anything `>= 3` as "this is not a verdict — read the + payload". + + **`3` is usage error in every member.** A malformed call is the one + non-verdict outcome every tool has, so it is worth harmonising rather than + leaving each tool to pick: an agent that mis-invokes any member gets the same + answer. Further non-verdict outcomes take `4` and up, named in the tool's own + SKILL.md — charactercheck uses `4` for could-not-retrieve. A new tool takes + `3` for usage errors and the next free code for anything else. + + Each SKILL.md states its own tool's exact contract — read that, not this + table, when invoking, and see clause 7 on how that statement is kept true. 2. **Deterministic verdict paths.** No model invocation, no network, no randomness anywhere in a verdict path. A model call to compute what a lookup or formula can decide is a defect. Same input, same verdict, every time. @@ -122,6 +134,12 @@ repos pin by link, not by copy. ### Changelog +- **v2.2** — Stated the exit-code pattern explicitly: 0/1/2 are the universal + verdict contract and `>= 3` is the no-verdict taxonomy, with `3` = usage error + harmonised across every member. v2.1 called srdcheck's exit 3 "the family + precedent", which was too weak to settle what a member should do when 3 was + already spent — charactercheck moves could-not-retrieve to `4` rather than + taking a different usage code (chaoz23/charactercheck#18). - **v2.1** — Members declare their class machine-readably as `family_class` in `tool.json`. v2 introduced classes but left them stated only in this file's members table, so the conformance gate had no way to apply clause 7 per class diff --git a/scripts/family_conformance.py b/scripts/family_conformance.py index 9e1e500..b684fbc 100755 --- a/scripts/family_conformance.py +++ b/scripts/family_conformance.py @@ -214,7 +214,12 @@ def parse_skill(skill: Path) -> dict: tight: dict[int, list[str]] = {} def record(code: int, ctx: str, is_tight: bool = False) -> None: - if not 0 <= code <= 3: + # 0/1/2 are the universal verdict contract; codes >=3 are each tool's own + # "no verdict was produced" taxonomy. charactercheck already spends 3 on + # could-not-retrieve, so its usage-error code has to go higher. Capping at + # 3 silently dropped anything above it, which then surfaced as a bogus + # UNDOCUMENTED_EXIT_CODE against a tool that documented itself correctly. + if not 0 <= code <= 9: return ctx = " ".join(ctx.split())[:220] documented.setdefault(code, []).append(ctx) @@ -233,7 +238,7 @@ def record(code: int, ctx: str, is_tight: bool = False) -> None: if "exit" not in head and "exit" not in para.lower(): continue # "`0` = clean" / "0 passes the named scope" / "· 2 cannot-adjudicate" - for m in re.finditer(r"(?:^|[·(\s])`?([0-3])`?\s*(?:=|\s)\s*([A-Za-z][^·|]{2,90})", para): + for m in re.finditer(r"(?:^|[·(\s])`?([0-9])`?\s*(?:=|\s)\s*([A-Za-z][^·|]{2,90})", para): record(int(m.group(1)), f"{m.group(1)} {m.group(2)}", is_tight=True) honest, usage = set(), set() From 632b24c3ded91b808a5ab6e8ad96d24655d541f0 Mon Sep 17 00:00:00 2001 From: chaoz23 Date: Tue, 25 Aug 2026 14:19:02 -0700 Subject: [PATCH 2/2] chore: drop charactercheck's HONEST_LANE_OVERLOAD waiver charactercheck is harmonising onto exit 3 for usage errors (chaoz23/charactercheck#18). The waiver comes out before the fix lands, as the ratchet requires; the SHA pin keeps charactercheck's CI green in the meantime. Co-Authored-By: Claude Opus 5 --- family-conformance-baseline.json | 3 --- 1 file changed, 3 deletions(-) diff --git a/family-conformance-baseline.json b/family-conformance-baseline.json index fd34162..9db10cd 100644 --- a/family-conformance-baseline.json +++ b/family-conformance-baseline.json @@ -1,9 +1,6 @@ { "_comment": "Findings accepted for now. Each MUST name the issue tracking it — a waiver without a ticket is just a hidden defect. A waiver that stops firing fails the gate, so fixes shrink this file.", "accepted": { - "charactercheck": { - "HONEST_LANE_OVERLOAD": "chaoz23/charactercheck#18" - }, "dmcheck": { "MISSING_PIPE": "chaoz23/dmcheck#14" }