cw v1: an MIT replacement for argh, bit-for-bit compatible by default [bump minor] - #33
Merged
Merged
Conversation
Closes #2, #3, #4. Lands #15. #4 — delete cw/scrap.py. 85 dead lines, imported by nothing (verified fleet-wide: t/theremin is cw's only importer and it takes only resolve_to_function). The `"scrap"` entries in pyproject's ruff exclude / per-file-ignores are directory globs from the wads template, not references to the module, and are left in place. #3 — make `import cw` stdlib-only. `from i2.wrapper import Ingress, wrap` was at module scope in cw/resolution.py and used by one function; it now lives in `_i2_wrapper()`, called from the body of `resource_inputs`, and raises an ImportError naming `pip install 'cw[resource]'` when i2 is absent. Measured on p12, best-of-11 subprocess wall clock over a `pass` baseline of 55.8 ms: argparse +0.7 ms cw BEFORE +38.2 ms -X importtime cumulative 37.6 ms cw AFTER +12.3 ms -X importtime cumulative 11.8 ms argh +21.5 ms i2 +34.6 ms cw now imports at ~57% of the argh it replaces, across 66 fleet console scripts. tests/test_import_is_cheap.py guards it two ways: a fresh subprocess asserting no third-party module is present after `import cw`, and a source scan rejecting any module-scope third-party import under cw/. #2 — drop `argh` from dependencies (declared, never imported; LGPL-3.0-or-later, so cw would otherwise be `wads licence-check`'s first finding). `i2` moves to an optional `resource` extra, `argcomplete>=3` gets a `completion` extra, and `dependencies` is now empty. NOTE: thorwhalen/theremin#9 (PR #10) has NOT landed — theremin still imports argh without declaring it on its default branch, and receives it transitively through cw. That PR must merge before cw is released. #15 — cw/base.py: MISSING, HIDE, CommandError, Codec, ArghHelpFormatter, and the Decode/Egress seam contracts. ArghHelpFormatter was written from recorded help *output*, not transcribed from argh's implementation (argh is LGPL-3.0-or-later, cw is MIT): the three behaviours — repr()-ed defaults, None as '-', joined choices, __name__ for anything carrying one — were read off real --help text and then reproduced. A 13-argument differential against live argh 0.31.3's own formatter is byte-identical. The rendering rules live in three class attributes (NONE_IN_HELP, CHOICES_SEPARATOR, render_default) plus a `_help_params` method, so the look is reparametrizable by subclassing rather than by editing cw. Also: tests/ created; pytest now runs doctests (`addopts = --doctest-modules`, `testpaths = ["tests", "cw"]`). 54 tests pass, cw/base.py at 100% coverage. refs #16 #17 #18 #19 #20 #21 Claude-Session: https://claude.ai/code/session_01K6LB3AwUmKDxaFNZ2NqPGr
refs #16, partially #17 argh reaches its grammar through two uncoordinated inference paths -- the annotation guesser (assembling.py:741-801) and the default-value guesser (:311-364) -- which collide on `bool` and are reconciled by two copy-pasted hand-patches. cw states them once, with one precedence order, in `specs_for_function`. Written from behaviour, not from argh's source: argh is LGPL-3.0-or-later, cw is MIT. `cw/` imports argh nowhere; `argh==0.31.3` is pinned in the dev extra only, for the differential. - cw/grammar.py: ArgSpec (argh's field-specific merge per ADR-0003, NOT dict.update), argh_decode / modern_decode (seam 1), cli_name / command_name, the short-flag collision rule, the flag append-merge rule, the four-tier ladder. Never imports argparse; ArgSpec.add_argument_args() is the only place that knows what an add_argument call looks like. - cw/convention.py: Convention / ARGH / MODERN, minus the `egress` field, which lands with cw.egress (#20). - tests/argh_parity/: builds the same parser with argh and with cw for a 32-function corpus x 2 naming policies, and asserts the argparse action tables match field-by-field AND `--help` is byte-identical. - tests/test_grammar.py: the errors, the MODERN improvements (all defaulting OFF), and the negative control proving no Convention switch is dead. Verified: 326 passed, 2 skipped; cw/grammar.py and cw/convention.py at 100% statement coverage; eight deliberate mutations of the grammar each break the differential.
The rest of the core. `cw.dispatch` now builds and runs a real CLI, and 57 CLI-level differential cases say it is argh's CLI: exit code, stdout and stderr byte-identical over subcommands, groups, every result shape and every error shape. refs #17 #18 #19 #20 #21 #24 - convention (#17): `egress` joins `decode` as a field, so flipping to `cw.MODERN` stays one act; `MODERN` keeps `default_in_help=True` per ADR-0004 rule 4. The merge ladder is documented here and implemented once, in `grammar.specs_for_function`. No second copy, no argparse. - commands (#18): the six forms, discriminated by value. A mapping key or `__all__` entry beats `__name__` and is then hyphenated -- the one rule under which xa's `gen-secret`, `list` and priv's `parse_pth_paths` all come out right, and which gives priv's `functools.partial` the command name argh gets wrong. A list of strings is refused with both working spellings; a factory is not called for you. - ingress (#19): every parameter kind, `i2.wrapper.Ingress`'s call contract, and no i2. `*args` extends rather than re-packs, `**kwargs` collects leftovers, and a `cw.HIDE`d parameter falls back to the function's own default -- so priv loses `--config-type` from the CLI and still receives `'setup.cfg'`. - egress (#20): argh's type whitelist as the default, the Iterable protocol as MODERN's, JSON, `write_lines` and `confirm`. Streams resolve at CALL time, which is what makes a cw CLI testable with capsys and an argh CLI not. A coroutine is refused at the call site, so no choice of egress can switch the check off. - cli (#21): `mk_parser` returns a plain `ArgumentParser` and does no I/O; everything `run` needs travels in one reserved `set_defaults` key, and a parameter of that name is an error naming it. `run` gains `convention=` and `config=` (ADR-0002) and resolves them from the subcommand's own stash. `group_kwargs` reaches `add_subparsers` whole and `title` reaches the parent row, `help=` passed even as None (ADR-0004 rule 1). A config key naming no command is a startup error, which is what closes the MODERN group-rename trap. Both `group_name=` and argh's dead `namespace=` are accepted. - __main__ + completion (#24): `python -m cw specs|help|parity`, built with cw. Completion fires at dispatch time, never in `mk_parser`, and argcomplete is imported inside the function. Also deletes the empty `cw/util.py` (docstring only, zero importers, flagged by both previous phases). Verified, not asserted: the COMP_LINE protocol round-trips against a built parser at both levels; 18 deliberate mutations each produce failures; 100% statement coverage on every module in this phase.
`python -m cw.testing parity` -> "8 shapes / 133 cases: identical", exit 0. Refs #5 #6 #7 #14 #22 #23. cw/testing.py (#5) -- characterize / replay / assert_replay / diff_help / read_cases / normalise_usage / parity. Module-scope imports are exactly argparse, difflib, json, os, re, shlex, subprocess, sys; a test parses the file's own AST to assert it, and another copies the file into a directory with no cw on the path and uses it. `parity` is the one function that imports cw, lazily, inside the body (D4). The capture is fd-level dup2 AND a sys.stdout/sys.stderr re-point at those same descriptors. Both halves are needed: argh binds output_file=sys.stdout at import so redirect_stdout captures nothing from it, and pytest rebinds sys.stdout to something that is not descriptor 1. A first version flushed only the streams it installed, not the ones it displaced -- so recording through a pipe drained argh's buffer after the descriptor was restored and wrote goldens whose every command produced no output. tests/test_corpus_coverage.py:: test_every_shape_records_real_command_output is the guard for exactly that; it is the only check that would have caught it. cw/tests/ (#6, #7) -- eight self-contained shapes reproducing the seven hard repos' grammar, plus a `contract` shape for the egress and error rows no repo shape reaches. 133 argv vectors, each with a comment naming the argh rule it pins. Goldens recorded once from real argh 0.31.3 by misc/record_goldens.py (dev-only, under misc/ so pytest cannot reach it) and committed; re-recording is byte-identical. Recorded text is newline-normalised and heap addresses are scrubbed, so a golden asserts behaviour and not the machine that made it. Parity runs over shapes rather than the seven repos because the spec's version cannot run in cw's CI: three of those repos import argh at module scope, t/ theremin depends on cw, and installing seven fleet packages to test a zero-dependency package defeats the exercise. Verified from a clean venv: `pip install cw-0.0.15-py3-none-any.whl` pulls nothing, and `python -m cw.testing parity` passes from site-packages outside the source tree. 17 of the 20 contract rows are asserted by the gate. Rows 14, 15 and 20 are observable only in a --help body, which the golden format keeps at tier 3 -- recorded, diffed advisorily, never asserted, because --help wraps to COLUMNS and 3.13 reformatted argparse's option column. They are covered by tests/ argh_parity instead, and the coverage test refuses to let a shape claim one. Nine mutations of cw's grammar, egress, ingress and cli each turn the gate red (scratchpad/mutate_gate.py). Two corpus gaps were found that way and closed: no parameter began with `h`, so nothing proved `-h` is stripped, and no declaration carried a falsy nargs. cw/compat.py (#23) -- the eleven measured argh names, each three statements or fewer, each warning once (CW_COMPAT_QUIET=1 silences). The four repairs are tested: a usage error still exits 2 rather than 0; no stream is bound in a signature default; argh's dispatch keywords are split out so output_file= does not reach ArgumentParser; and both group_name= and namespace= are accepted. named/aliases/add_subcommands raise an informative module __getattr__ instead of shipping -- `named` as specified wrote func._cw['name'], which nothing reads. `func_kwargs` is accepted and refused out loud: it is per-command ArgumentParser keywords, not a cw config, and no fleet call site passes it. Windows (#14) -- decision A, normalise, recorded in cw/testing.py's docstring and cw/tests/README.md. Goldens store LF and both sides normalise; parity spawns no subprocess so console shims and code pages never arise; read_cases parses a JSON list as well as a shlex line; every golden is asserted pure ASCII. pyproject: a `test` extra (no argh) for CI, `dev` (argh 0.31.3) for the differential and the recorder; tests/argh_parity skips itself when argh is absent, so the suite is green either way. 790 passed, 2 skipped. cw/testing.py, cw/compat.py, cw/__main__.py at 100% statement coverage; 96% overall.
Six ADRs in docs/adr/ (the fleet's convention -- openloops, paces, comparanda, rubricator all use it), a README that teaches cw instead of argh, and a pyproject that is actually publishable. refs #8 #9 #10 #11 #12 #13 docs/adr/0001..0006 -- the seam table, the ingress stash, the merge ladder, the grammar errata, the release/rollback policy, and the v1 cut list. Audited rather than trusted: every claim was re-checked against the code and the fleet, and three were wrong. * The argcomplete census. The spec says 7 repos, an earlier draft said 8. Counted over $PP it is **10 marker files across 10 repos**, every one an argh consumer. ADR-0001 now lists them. * Three transcripts in ADR-0003/0004 had never been run -- one showed its outputs as trailing comments and referenced an undefined `h2`. The *facts* were right; the blocks were not runnable. Rewritten as real transcripts. * ADR-0005's rollback drill claims "the transcript below is real". It is: re-executed end to end (two wheels, a throwaway consumer pinned `cw>=0.1,<0.2`, a fresh venv) and it reproduces byte for byte, including the GrammarError text and `36 DIFFER`. Every example in the docs now runs, and tests/test_docs_examples.py keeps it that way -- 65 examples over 8 files, mutation-checked to prove it fails when an expected output drifts. Three things that would have broken the release, found by running the environments CI actually uses rather than only the dev one: * **`docs/*` is gitignored** (wads template, for Sphinx output), so every ADR here would have been silently dropped on commit. Negated with `!docs/adr/`. * **`pip install -e '.[test]' && pytest` failed at collection**: tests/test_fleet_shapes.py imports argh at module scope, and argh is a *dev* extra. The argh-free capture helper moves to tests/capture.py, the one genuine differential skips itself, and the ten cw-only assertions in that file now run in CI. tests/test_grammar.py had the same problem via the shared corpus; test_corpus_coverage.py used `tomllib`, which is 3.11+, on a matrix that includes 3.10. * **The parity gate was red on 3.10, 3.11 and 3.13 with a correct cw.** argparse quoted its `invalid choice` items until 3.11 and stopped in 3.12; 3.13 rewrapped the usage block's trailing `...`. Those are CPython's renderings, identical for argh and cw on any one interpreter, so a golden recorded on 3.12 was asserting the recording machine's version. `cw.testing.canonical_argparse_text` neutralises exactly those two, on both sides. The nine-mutation battery still goes 9/9 red with the *same* counts on 3.10 and on 3.12, so nothing real was forgiven. pyproject: PEP 639 `license = "MIT"` + `license-files` (was the deprecated `[project.license] text` table), classifiers (there were none), keywords, author, Issues URL. LICENSE still said `Copyright (c) [year] [fullname]` -- for a package whose whole pitch is replacing an LGPL one. `cw[test]` gains `cw[resource]`: i2 is the house's own package, not the one cw exists to replace, and without it CI silently skipped cw.resolution's tests and doctests. Verified (real output, /Users/thorwhalen/.pyenv/versions/p12/bin/python3): dev (p12) 805 passed, 2 skipped CI 3.12 537 passed, 10 skipped (cw[test], no argh) CI 3.10 536 passed, 11 skipped (cw[test], no argh) parity identical on 3.10 / 3.11 / 3.12 / 3.13 clean venv cw + pip only, gate passes from an installed cw wheel License-Expression: MIT, no unconditional Requires-Dist ruff check + format clean Claude-Session: https://claude.ai/code/session_01K6LB3AwUmKDxaFNZ2NqPGr
…hat hid it
Two blockers, five majors and the minors worth fixing, plus the tests that should
have caught each one. ADR-0007 records the four that were defects in a *decision*
rather than in its implementation.
Blockers
- cw.compat.ArghParser never set formatter_class, so the advertised one-line
migration changed --help for the 25 fleet files that hold a parser object: `-`
became `None`, `'0.0.0.0'` lost its quotes, multi-paragraph docstrings reflowed.
Fixed as argh does it, which is NOT what it looks like: ArghParser.__init__
defaults it, subparsers get it when the parent still carries argparse's stock
one, and a plain parser's own formatter is left alone (promoting it there would
have added a divergence while removing one -- the new differential caught that).
- CI installed no extras, so three shipped cw.resolution items failed on a missing
i2. [tool.wads.ci.install] extras = "test"; two false comments corrected.
Majors
- D2 violation: a hyphenated positional carrying `choices` printed `project-dir`
where argh printed `{a,b}`, in usage: and in --help. argparse reads a
positional's registered name twice and a synthesised metavar wins only the
error-message reading. cw now registers the hyphenated name, as argh does, and
renames the dest back in _call_args. Spec 9.3's "one permitted divergence" is
retired and the harness forgives nothing.
- Two commands deriving one name silently kept the last and ran the wrong
function with exit 0. Now CommandTreeError naming both callables, as argh
refuses it too.
- @arg(..., completer=...) crashed with a raw argparse TypeError -- in the shim
through which the ten # PYTHON_ARGCOMPLETE_OK repos migrate, for the sake of
which cw is argparse-based at all.
- add_argument failures caught only ArgumentError, so ValueError/TypeError escaped
naming neither cw nor the function nor the parameter -- strictly worse than the
argh being replaced, on `config=`, which is the newest surface.
- `replay` said "identical" about a migration whose --help visibly changed: a
formatter change moves only the help column and the description. It now compares
the body through normalise_help (width-independent) and reports `help-differs`;
--strict-help makes it fatal.
Minors
- @arg(..., dest=...) named a parameter that does not exist.
- MODERN's Enum help advertised member reprs the converter rejects.
- The bound-keyword UserWarning printed a hex id() and a '<command>' placeholder,
and its only remedy removed a flag argh exposed. Now BoundKeywordWarning, named
per command, silenceable with CW_QUIET=1.
- egress= on mk_parser blamed argparse; ADR-0001 gains the row saying where each
seam lives.
- CommandTreeError / IngressError / BoundKeywordWarning exported; resolve_object
demoted from the facade (no call sites anywhere, uncovered, TODO'd).
- README: the add_commands/ArghParser shape, the mapping-value-must-be-a-list
trap, and two more import forms on the migration grep list.
The corpus gains the shape both corpora were blind to (lacing convert-tree): a
hyphenated positional with Literal choices. 8 shapes / 137 cases, and the gate
goes red (3 DIFFER) on a reintroduction. tests/argh_parity/test_compat_parity.py
is the missing differential -- it renders help through every compat entry point
and diffs it against live argh.
868 passed, 2 skipped (dev, argh present); 574 passed on 3.12 and 573 on 3.10
under cw[test]; parity identical on 3.10/3.11/3.12/3.13; mutation battery 9/9 red.
refs #25, #26
Claude-Session: https://claude.ai/code/session_01K6LB3AwUmKDxaFNZ2NqPGr
Both were latent before this branch; the first CI run on it is what surfaced them.
- wads' reusable workflow runs pytest with
`-o doctest_optionflags='ELLIPSIS IGNORE_EXCEPTION_DETAIL'`, unconditionally,
which DROPS the NORMALIZE_WHITESPACE that pyproject sets. Two grammar doctests
wrapped their expected output and therefore only ever passed locally. One now
carries an inline `# doctest: +NORMALIZE_WHITESPACE`; the other prints a line at
a time, which reads better anyway.
- Windows, three failures:
* `cw.testing._as_command` shlex-split the command string in POSIX mode, which
eats the backslashes out of `C:\py\python.exe` and leaves a command nothing can
run -- surfacing as `FileNotFoundError: [WinError 2]` with no clue where it came
from. Non-POSIX mode on `os.name == 'nt'`, with the quotes it keeps stripped
back off, plus four tests that pin both platforms.
* `cw.resolution.resolve_func_from_dot_path`'s doctest asserted `os.path.join`
returns `'a/b'`.
Verified with CI's exact flags: 872 passed, 2 skipped. Gate still identical.
Claude-Session: https://claude.ai/code/session_01K6LB3AwUmKDxaFNZ2NqPGr
This was referenced Aug 30, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cw v1: an MIT replacement for
argh(LGPL-3.0-or-later), reproducing its grammarbit-for-bit by default, with every improvement one keyword away.
Spec discussion: https://github.com/thorwhalen/priv/discussions/65
The gate
That is the definition of v1 and it is falsifiable: nine plausible reimplementation
mistakes were each injected and each turns it red (36, 17, 7, 2, 3, 2, 2, 3, 37 cases).
The goldens are what live argh 0.31.3 printed, recorded once and committed;
cwitself never imports argh.The design
Three seams, each exactly one keyword argument —
decode=(how a parameter's type isinferred),
egress=(result → lines → exit code),convention=(a re-binding of what thedefaults ARE). No registry, no base class, no plugin loader. Each default is a real
implementation; each has a shipped replacement (
modern_decode,iterable_egress,json_egress,MODERN). The list of things that are deliberately not seams is asbinding as the list that are — ADR-0001.
mk_parserreturns a plainargparse.ArgumentParser, never a subclass. This isload-bearing, not tasteful:
argcomplete.autocomplete()is argparse-typed at itssignature, so the ten fleet files marked
# PYTHON_ARGCOMPLETE_OKkeep working. Becausethe parser is plain, cw keeps no attributes on it — everything travels in one reserved
set_defaultskey (ADR-0002).The core does not use
i2.import cwis stdlib-only and costs 14.7 ms againstargh's 19.4 ms (
-X importtime, cumulative).cw.mk_ingresskeepsi2.wrapper.Ingress'scall contract —
{name: value} -> (args, kwargs)— so an i2Ingressstays a drop-in.i2remains an optional extra forcw.resolution.resource_inputsalone, imported lazily.cw.compatis a one-line migration (from cw import compat as argh) covering argh'seleven measured fleet names, deprecated from day one.
cw.testingis a single filewhose
characterizehalf imports no cw at all, so it serves the repos that will neverdepend on cw.
What the adversarial reviews changed
Three independent reviewers were asked to refute the build. The grammar survived — 88
signature/annotation/collision cases and 20 end-to-end invocations byte-identical to argh,
argcomplete driven through its real
COMP_LINEprotocol, the live installedprivCLIdiffed command by command. Four of their findings were defects in a decision, and all
four hid in the same place: a suite that asserted behaviour everywhere and rendering
nowhere. ADR-0007 records them.
The sharpest: spec §9.3 permitted one divergence and asserted it was invisible — argh
registers a hyphenated positional as
add_argument('project-dir'), cw registeredadd_argument('project_dir', metavar='project-dir'). argparse reads that one stringtwice, as the displayed name and as the name in
error: argument ..., and ametavarwins only the second. So a hyphenated positional carryingchoicesprintedproject-dirwhere argh printed{a,b}— with an identical error message, which is whyevery error-level test agreed. cw now registers the hyphenated name and renames the
destback on the way into the call. There is no permitted divergence left, and the parity
harness forgives nothing.
Also fixed:
cw.compat.ArghParsernever setformatter_class(25 fleet files); twocommands deriving one name silently ran the wrong one;
@arg(completer=)crashed in theshim the argcomplete repos migrate through;
add_argumentfailures escaped naming neithercw nor the function nor the parameter; and
cw.testing replayreported "identical" abouta migration whose
--helpvisibly changed.CI itself was red on the previous commit and is fixed here: wads' reusable workflow reads
[tool.wads.ci.install].extras, which was missing, so it installed no extras and threeshipped
cw.resolutionitems failed on a missingi2.Version
[bump minor]in this PR's title is deliberate. cw is wads-managed and the mergepublishes to PyPI; the latest version across every source is
0.0.15, so a minor bumplands 0.1.0, which is what
dependencies = ["cw>=0.1,<0.2"]in the migration notespins against (ADR-0005, whose rollback
drill was re-executed and still reproduces byte for byte).
Closes #2
Closes #3
Closes #4
Closes #5
Closes #6
Closes #7
Closes #8
Closes #9
Closes #10
Closes #11
Closes #12
Closes #13
Closes #14
Closes #15
Closes #16
Closes #17
Closes #18
Closes #19
Closes #20
Closes #21
Closes #23
Closes #24
Closes #25
Closes #26
Not closed: #22 (17 of the 20 argh-contract rows are asserted by the gate; rows 14, 15 and
20 are help-rendering rows covered by the dev-extra differential — its wording needs
amending or the split accepting), #27, #28, #29, #30 (fleet migration, outside cw), #31 and
#32 (follow-ups filed from the reviews).
https://claude.ai/code/session_01K6LB3AwUmKDxaFNZ2NqPGr