chore(hygiene): make the repo's self-checks able to see what they claim

[Wynelson94]

Third and last pass of the 2026-08-11 audit. Nothing here changes what the tool extracts — it changes what the repo can prove about itself.

One definition for the bot-wall phrases

content._BLOCK_PHRASES said "keep phrases in sync with fetch._BLOCK_MARKERS". They had drifted anyway: content carried "checking your browser" and the looser "attention required", fetch carried neither. So a sub-30 KB "Checking your browser before accessing…" interstitial was invisible to the raw-body check and rested entirely on the post-extraction backstop — which only fires when the extraction lands under 600 chars.

fetch._BLOCK_MARKERS is now derived from the single list, with a test asserting they cannot diverge. A comment cannot enforce a shared list; one definition can.

Three findings in the trust instrument itself

This is the part that concerned me most: evals/run_eval.py is what certifies the tool, and it had no tests at all.

• The guard-calibration gate stopped testing the guard it names. expect_warning marks a known-bad pricing page, but was satisfied by ANY warning: line — so once the stub guard shipped in v0.1.5, a stub warning could satisfy a gate written for the price guard. One of the three RED-triggering gates got weaker with nobody touching it. Now requires a price warning.
  The silent-omission gate still accepts any warning on purpose: a page that stub-warns while dropping figures was not silent, so scoring it as a silent omission would be false. Documented in place so it doesn't read as an oversight.
• The oracle was blind exactly where the tool was. run_eval.MONEY carried the same four gaps the tool's matcher did, so it under-counted the figures actually on the page and a genuine omission could not be scored. Re-derived from the same requirements but still written separately — importing the tool's matcher would recreate the shared-fate flaw the harness exists to avoid. A test asserts the two independent implementations agree.
• A second run the same day overwrote the first report in place. Not hypothetical — two runs were needed today and the first had to be rescued by hand. It now archives to <date>-runN first, matching the convention already on disk.

tests/test_eval_harness.py covers the oracle matcher, the report preservation (a function that renames directories containing evidence), and wall detection. The network run stays manual by design.

check_consistency.py sees four things it couldn't

Each falsified in both directions before commit, per this repo's own practice:

blind spot	consequence
\d{2,4}	README's 6 tests claim was invisible to the script entirely
no notion of which suite	that claim can now be checked against pytest -m playwright instead of being wrongly compared to the default suite — or skipped
heading-only CHANGELOG match	## [0.1.5] — TBD would have passed as released
installed dist metadata	stale at 0.1.0 for three releases, silently mislabelling every eval report, since run_eval.py stamps reports with the metadata version

Two smaller ones

• README's Development command is now the actual gate. It omitted scripts from the lint target and left out check_consistency.py, so following the README could push a failure CI would then catch.
• CI tests Python 3.14 — the interpreter the local venv actually runs. Every local gate run and every eval report to date used a version CI never exercised. 3.12 stays the floor.

Verification

ruff · mypy · 292 tests (+14) · check_consistency.py — all green. The consistency changes were each falsified in both directions; the new suite-aware check correctly fails on a corrupted playwright count and passes when restored.

Not addressed, deliberately: the repo still has zero git tags, which CHANGELOG.md documents as intentional (its links point at shipping commits). The live marker remains wired-up-but-empty; the README now says so rather than implying a suite exists.

🤖 Generated with Claude Code