The one-command test: `python -m cw.testing parity` over self-contained fixtures

[thorwhalen]

This issue IS the definition of v1. v1 is done when this prints identical and exits 0.

Why it is re-cut from the spec's version

As specified — "for each of the seven hard-case repos … rebuilds that repo's command set
through cw" — it cannot run in cw's CI, for five independent reasons:

1. It contradicts D4: cw/testing.py is specified to import no cw, yet parity must
   "rebuild the command set through cw".
2. It contradicts its own headline ("argh is NOT a runtime or test dependency of cw"): to
   rebuild a repo's command set you must import that repo, and theremin/script_utils.py:12,
   coact/__main__.py:26 and epythet/cli.py:15 are module-scope import argh.
3. It needs seven fleet packages installed in the CI of a package whose selling point is zero
   dependencies — including t/theremin, which depends on cw. Circular.
4. "214 cases" was an invented number (see the corpus issue).
5. It cannot pass on the Windows runner pyproject.toml:130 turns on.

The fix

Parity runs over the seven hard-case shapes in cw/tests/fixtures.py (from the corpus
issue), against the committed argh-recorded goldens (from the goldens issue). It then
needs stdlib + cw only, pulls no argh, installs no fleet package, and runs on Windows.

The seven real repos become the migration gate — cw.testing.replay per repo, run in each
repo's own CI — which is what D4 actually asks for.

Structure: parity() lives in cw/testing.py but imports cw lazily inside the function,
so the standalone-copy guarantee (D4) survives — assert this with the AST test.

What is asserted

For each case, with every seam on its default (convention=cw.ARGH,
decode=cw.argh_decode, egress=cw.argh_egress): exit code, stdout, stderr, and the
normalised usage: line, byte-identical to the golden. The full --help body is diffed
advisorily into a human review queue (diff_help), never asserted — --help wraps to
COLUMNS and priv --help is 489 lines at default width, 365 at COLUMNS=100.

Acceptance criteria

• python -m cw.testing parity exits 0 and prints N shapes / M cases: identical with
  real, derived N and M.
• It runs in cw's CI on Linux, macOS and Windows (per the Windows policy issue).
• The CI environment contains no argh — asserted by the job, not by convention.
• It installs no fleet package.
• cw/testing.py's module-level imports are still stdlib-only (AST test still green).
• A deliberately introduced grammar bug (e.g. disable short-flag collision suppression)
  makes it fail with a readable diff — demonstrated in the PR.
• All 20 argh-contract rows are exercised, per the corpus coverage test.

Depends on

• cw.cli: mk_parser, run, dispatch, and the ingress stash #21 — cw.cli: mk_parser, run, dispatch, and the ingress stash
• Record the argh goldens once, at a pinned argh version, and commit them #7 — Record the argh goldens once, at a pinned argh version, and commit them
• Windows: decide what python -m cw.testing parity means on the Windows runner #14 — Windows: decide what python -m cw.testing parity means on the Windows runner

---

Research & rationale: the canonical spec and the full fleet audit live in the private research ledger (https://github.com/thorwhalen/priv/discussions/65). Effort: medium.

[thorwhalen]

Closed with #33. One caveat worth recording rather than leaving implied, because this issue's acceptance criterion says "all 20 argh-contract rows are exercised" and the gate alone discharges 17 of 20.

python -m cw.testing parity is 8 shapes / 137 cases and asserts, per case, the exit code and both streams in full (tier 1) or the return code and the normalised usage: line (tier 3, anything containing --help). Rows 14, 15 and 20 are help-rendering rows — visible only in a --help body, which the tier rule keeps as a snapshot, deliberately: --help wraps to COLUMNS, and CPython rewrites it between versions (3.13 renders -i, --ignore VALUE where 3.12 rendered -i VALUE, --ignore VALUE), so a committed golden replayed across the CI matrix would fail for reasons nobody caused.

Those three are asserted by tests/argh_parity/, which diffs rendered help against live argh in one process at one interpreter — a comparison a committed golden structurally cannot make. tests/test_corpus_coverage.py::HELP_ONLY_ROWS records the split mechanically and a test fails if a named file stops existing.

The gate is falsifiable, which is the part that matters: nine plausible reimplementation mistakes each turn it red (36, 17, 7, 2, 3, 2, 2, 3, 37 cases), re-verified on 3.10 and 3.12 with matching counts.

Two things the adversarial reviews changed here, both in #33:

• The corpus had a shape-shaped hole, not a case-count-shaped one. Every choices case in both corpora was on an option, and every Literal positional had a one-word name — so a real D2 violation (a hyphenated positional with choices printing project-dir where argh printed {a,b}) was invisible to a green gate. lacing convert-tree closes it, and the gate now goes 3 DIFFER on a reintroduction.
• replay no longer reports identical about a --help body that moved. A formatter change touches neither the usage: line nor any exit code, so the tool cw ships to prove a migration changed nothing said 6/6 identical on a migration whose help visibly changed. It now reports help-differs (non-fatal), with --strict-help to make it fatal.