testing: pin stdin closed while recording

[thorwhalen]

Found while migrating grub as part of the fleet rollout. grub's headline behaviour is that omitting the query drops you into an interactive prompt — and that behaviour could not be characterized at all from a terminal.

The defect

cw.testing._run_subprocess never set stdin, so the recorded child inherited the recorder's. For any CLI with an interactive path — grub's REPL, a cw.confirm prompt — the recording itself depended on where it was run:

recorder's stdin	recorded case
a terminal	returncode: None, "cw.testing: timed out after 30s"
/dev/null (CI, pytest)	returncode: 0, "prompt> read: <EOF>"

Neither is a fact about the CLI. The practical damage is a golden recorded in CI that fails when a developer replays it locally — precisely the class of failure RECORDING_ENV exists to remove ("to remove a source of variation between two machines"). stdin belonged in that list and was missing.

The fix

stdin=subprocess.DEVNULL in _run_subprocess, and the same treatment on descriptor 0 in capture(), saved and restored beside 1 and 2. The module argues that using one mechanism for both sides of a diff is what makes the diff mean something; that only holds if the two recording mechanisms agree about stdin too.

The test

Records the same case twice under two different stdins — an open pipe nobody writes to (what a terminal looks like to the child), then /dev/null — and asserts the two recordings are equal. Verified it fails without the fix: {'returncode': None, 'stderr': 'timed out'} vs {'returncode': 0, 'stdout': 'prompt> read: <EOF>'}.

Worth noting for whoever reads the test: the obvious test — "a stdin-reading command records EOF" — passes either way under pytest, because pytest already hands fd 0 a devnull. Only the two-stdin comparison catches this, which is presumably why it was not caught before.

Checks

• 874 passed, 2 skipped
• python -m cw.testing parity → 8 shapes / 137 cases: identical
• No behaviour change for any CLI that does not read stdin, so no existing golden is invalidated.

---

Second defect, same theme: Windows' .exe (added in 364c6a3 → 1e43fc2)

Found on the very next CI run of the same migration. The module promises "a golden recorded on a Mac asserts cleanly on a Windows runner". It did not.

argparse takes its prog from basename(sys.argv[0]), and a console script is installed as opsward.exe on Windows:

usage: opsward     [-h] {diagnose,...}     on a Mac
usage: opsward.EXE [-h] {diagnose,...}     on the runner

22 of 31 cases failed on that alone — a wall of red saying nothing about the CLI. The docstring's Windows section actually names the .exe console-script shim as a reason parity avoids subprocesses; characterize and replay do spawn subprocesses and had no defence against the thing that paragraph named.

scrub_exe_suffix rewrites only the program's own stem (a CLI that mentions some other .exe is untouched), case-insensitively (the extension surfaces as .EXE on some Windows configurations and .exe on others — otherwise two Windows runners disagree with each other), applied in _run_subprocess, the single choke point all three subprocess paths share, at record time as well as replay time — the same treatment and the same stated rationale as scrub_addresses.

Two hazards worth keeping in review: the replacement is a function, not a template string, and the stem is split on both separators. A Windows path reaches this on a POSIX host through a golden's recorded prog, where os.path.basename does not split on a backslash and re.sub reads the leftover \p as a bad escape. The doctest caught that on the first run.

The test installs one toy CLI under two names, toy and toy.EXE, records from the first and replays against the second — an exact simulation, not a mock. Verified it fails without the fix.

No POSIX-recorded golden changes, because there is no .exe in one.

Updated checks: 877 passed, 2 skipped; parity 8 shapes / 137 cases: identical.