Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 26 additions & 8 deletions FAMILY.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# The check family — shared contract (v2.1)
# The check family — shared contract (v2.2)

Canonical copy: this file, in [srdcheck](https://github.com/chaoz23/srdcheck)
(the family's reference implementation). Sibling repos link here.
Expand Down Expand Up @@ -39,13 +39,25 @@ Members differ by what they produce, and the contract binds them accordingly.
cannot-adjudicate answer meaning the question is outside the tool's codified
jurisdiction, or that the tool cannot answer it from the evidence it has
(unknown content, discretion, ambiguity, incomplete coverage) — is the
contract of the verdict class: srdcheck and dmcheck use exit `2`;
charactercheck uses exit `2` for unhandled content and `3` for could-not-
retrieve. Consuming agents route the honest lane to a human; never retry or
guess. Usage errors are **not** the honest lane and should not share its exit
code; srdcheck's exit `3` is the family precedent. Each SKILL.md states its
own tool's exact contract — read that, not this table, when invoking, and see
clause 7 on how that statement is kept true.
contract of the verdict class: srdcheck, dmcheck and charactercheck all use
exit `2` (for charactercheck, unhandled content). Consuming agents route the
honest lane to a human; never retry or guess. Usage errors are **not** the
honest lane and must not share its exit code.

**`0`, `1` and `2` are the universal verdict contract.** Codes `3` and above
are the *no-verdict-was-produced* taxonomy: a consuming agent branches on
0/1/2 and treats anything `>= 3` as "this is not a verdict — read the
payload".

**`3` is usage error in every member.** A malformed call is the one
non-verdict outcome every tool has, so it is worth harmonising rather than
leaving each tool to pick: an agent that mis-invokes any member gets the same
answer. Further non-verdict outcomes take `4` and up, named in the tool's own
SKILL.md — charactercheck uses `4` for could-not-retrieve. A new tool takes
`3` for usage errors and the next free code for anything else.

Each SKILL.md states its own tool's exact contract — read that, not this
table, when invoking, and see clause 7 on how that statement is kept true.
2. **Deterministic verdict paths.** No model invocation, no network, no
randomness anywhere in a verdict path. A model call to compute what a lookup
or formula can decide is a defect. Same input, same verdict, every time.
Expand Down Expand Up @@ -122,6 +134,12 @@ repos pin by link, not by copy.

### Changelog

- **v2.2** — Stated the exit-code pattern explicitly: 0/1/2 are the universal
verdict contract and `>= 3` is the no-verdict taxonomy, with `3` = usage error
harmonised across every member. v2.1 called srdcheck's exit 3 "the family
precedent", which was too weak to settle what a member should do when 3 was
already spent — charactercheck moves could-not-retrieve to `4` rather than
taking a different usage code (chaoz23/charactercheck#18).
- **v2.1** — Members declare their class machine-readably as `family_class` in
`tool.json`. v2 introduced classes but left them stated only in this file's
members table, so the conformance gate had no way to apply clause 7 per class
Expand Down
3 changes: 0 additions & 3 deletions family-conformance-baseline.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,6 @@
{
"_comment": "Findings accepted for now. Each MUST name the issue tracking it — a waiver without a ticket is just a hidden defect. A waiver that stops firing fails the gate, so fixes shrink this file.",
"accepted": {
"charactercheck": {
"HONEST_LANE_OVERLOAD": "chaoz23/charactercheck#18"
},
"dmcheck": {
"MISSING_PIPE": "chaoz23/dmcheck#14"
}
Expand Down
9 changes: 7 additions & 2 deletions scripts/family_conformance.py
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,12 @@ def parse_skill(skill: Path) -> dict:
tight: dict[int, list[str]] = {}

def record(code: int, ctx: str, is_tight: bool = False) -> None:
if not 0 <= code <= 3:
# 0/1/2 are the universal verdict contract; codes >=3 are each tool's own
# "no verdict was produced" taxonomy. charactercheck already spends 3 on
# could-not-retrieve, so its usage-error code has to go higher. Capping at
# 3 silently dropped anything above it, which then surfaced as a bogus
# UNDOCUMENTED_EXIT_CODE against a tool that documented itself correctly.
if not 0 <= code <= 9:
return
ctx = " ".join(ctx.split())[:220]
documented.setdefault(code, []).append(ctx)
Expand All @@ -233,7 +238,7 @@ def record(code: int, ctx: str, is_tight: bool = False) -> None:
if "exit" not in head and "exit" not in para.lower():
continue
# "`0` = clean" / "0 passes the named scope" / "· 2 cannot-adjudicate"
for m in re.finditer(r"(?:^|[·(\s])`?([0-3])`?\s*(?:=|\s)\s*([A-Za-z][^·|]{2,90})", para):
for m in re.finditer(r"(?:^|[·(\s])`?([0-9])`?\s*(?:=|\s)\s*([A-Za-z][^·|]{2,90})", para):
record(int(m.group(1)), f"{m.group(1)} {m.group(2)}", is_tight=True)

honest, usage = set(), set()
Expand Down
Loading