Backend-neutral builtin argument contract + three-engine benchmark data

[lu-zero]

Draft for design/data inspection. Contains and supersedes the stack in #1302 (bpaf port); this PR's diff = bpaf port + the contract work on top.

What this adds

Builtins declare their arguments as compile-time, backend-neutral data:

static ECHO_SPEC: CommandSpec = CommandSpec {
    args: &[
        ArgSpec::flag("no_newline", &['n'], &[], "Do not output a trailing newline"),
        ArgSpec::value("delimiter", &['d'], &[], "DELIM", "Output delimiter"),
    ],
    positionals: &[PositionalSpec::many("operands", "OPERANDS")],
};

A build selects one engine via features (parser-usage / parser-clap / parser-bpaf; priority in that order). ArgParserBackend translates the spec into engine structures, parses, maps results into slot-indexed ParsedValues. Shell-specific semantics (declaration routing, +x/-x toggles, verbatim trailing words) live once in core instead of per engine. All ~49 builtins converted.

⚠️ Authoring experience is not final — this needs your input

The manual static specs above are interim. Hand-writing stringly-typed ids is worse DX than today's clap derive structs, and unacceptable for third-party builtin authors. The intended endgame is a small proc-macro crate:

#[derive(SpecCommand)]          // emits the static CommandSpec + from_matches
struct Echo {
    #[spec(short = 'n')]
    no_newline: bool,
    #[spec(short = 'd', metavar = "DELIM")]
    delimiter: String,
    operands: Vec<String>,
}

i.e. the ergonomic shape of #[derive(Args)]/usage's #[derive(Cli)], but emitting backend-neutral data so engines stay swappable. Nothing in the contract layer changes for that to slot in — specs are already plain static data. Opening this draft now because the contract shape and the benchmark data below are what need review first; if the direction is wrong, the derive never gets written.

Why this shape at all

usage-rs's #[derive(Cli)] proved static spec → parser graph is the fast path. This keeps that shape but makes two things differ from a single-engine derive: spec source (hand-written today, derived later) and target engine (swappable). The runtime translation layer reconstructs what derive gets free (compile-time statics); an intern cache bounds it to one leaked graph per (builtin, name) — pointer-stability tested.

Benchmark data

benchmarks/three-way.py (committed): pinned CPU, interleaved sampling across shells (machine drift hits every build equally), byte-for-byte output parity before timing, median of 12 samples. Workloads in benchmarks/real-world/.

7 shells, one machine, one run:

workload	bash	main¹	contract+clap	bpaf-single²	contract+bpaf	usage-single³	contract+usage
startup (ms)	1.6	1.7	1.7	1.7	1.7	1.7	1.7
wordops-10k (ms)	151.8	54.3	53.0	52.7	53.1	52.8	52.6
config-lint-500 (ms)	23.8	95.5	79.5	74.7	71.6	50.6	50.0
deploy-sim (ms)	15.0	20.8	20.6	20.3	20.2	19.4	19.5
interp-loop (ms)	139.0	275.0	284.1	287.4	280.7	227.3	275.1

¹ upstream main (clap derive builtins) · ² #1302 head (native bpaf impls) · ³ usage fork with #[derive(Cli)] builtins

Reading:

• Engine ranking, parse-dense work: usage ≫ bpaf > clap (config-lint dominated by builtin invocation).
• Contract overhead vs its own backend: ≤2%, often negative — contract+clap beats main's own clap code by ~17% on config-lint (leaner spec-built Command vs derive path).
• interp-loop caveat: zero argument parsing in that loop; its spread reflects whole-binary layout variance between independently compiled trees, not arg-model cost. bash wins it outright regardless.
• Startup/RSS identical across all six brush builds (±0.1 MiB).

Correctness

• Compat suite: 1854 passed / 0 failed / 456 known-fail — identical to baselines.
• Backend-parity unit tests: every engine must interpret a spec identically. These caught two latent bugs in the clap backend here (stale positional storage; allow_hyphen_values accepting echo -x where bash errors) — fixed.
• usage graphs interned per (spec, name); total leakage equals what derive would emit as statics.

Open questions for review

1. Derive macro as sketched above — wanted? (then hand-written specs become the escape hatch, not the interface)
2. Keep all three engines behind the contract, or use the data to pick one and delete the rest?
3. Per-event id→slot scans are linear-in-args (tiny n), measured invisible; resolve slot tables at intern time only if profiles ever disagree.

Assisted-by: ox-alpha (opencode)

[lu-zero]

Closing per author — approach needs rework before review.