[jehrr]

Problem

The 2026-10-14 audit left this as an owner question (L6). finish_run treated --allow-empty as "write whatever happened":

• Blocked or never-obtained run with --allow-empty. It wrote an empty JSON and CSV over the previous good output, plus a sidecar saying failed. The exit code stayed 3 or 5, but the data the flag's own docstring says it protects was gone.
• Genuinely empty, complete run. Its sidecar said status: failed, so diff_runs.py refused to compare it.

Contract (owner decision 2026-10-15)

--allow-empty only permits writing an empty output for a genuine empty result. It never turns a blocked, failed or parser-failure run into success.

Resulting behaviour

Run	Exit	Row files	Sidecar
Finished, answer was nothing, with --allow-empty	4	[] and a header-only CSV	status: empty plus its stop_reason
Finished, answer was nothing, without the flag	4	none	none (unchanged)
Blocked, with --allow-empty, earlier output exists	3	earlier output kept byte for byte	earlier sidecar kept
Never obtained or parser failure (failed page), with --allow-empty, earlier output exists	5	kept	kept
Blocked or failed, with --allow-empty, nothing earlier at --out	3 or 5	none	status: blocked or failed
Rows, then a failure	6	rows	partial (unchanged)

diff_runs.py compares an empty run like a complete one. The README and CHANGELOG [Unreleased] describe the contract.

Evidence

• New offline check. allow_empty_writes_only_a_genuine_empty_result calls finish_run for every row in the table and compares file bytes before and after.

• Core-only venv. The suite passes (540 checks) with PYTHONDONTWRITEBYTECODE=1 and caches cleared, and ci_checks.py --help-check --sample-check --secret-check is OK.

• Planted faults. Each one turned the new check red on the named lines:

  1. genuine_empty = not rows (blocked or failed treated as empty). Result: "kept byte for byte" failed, and "blocked, nothing earlier" got (3, True, True, 'empty').
  2. Status empty changed back to failed. Result: "the sidecar says empty" failed, and so did "diff_runs compares…".
  3. The preservation branch disabled. Result: "kept byte for byte" failed three times.
  4. diff_runs back to != "complete". Result: "diff_runs compares a complete run with an empty one" failed.
  5. output_writer.py and diff_runs.py from main. Result: red.

  After restore the suite was green.

Not changed

• No engine file changed. Every engine reaches this through finish_run.
• No version bump, tag or release.

🤖 Generated with Claude Code