[jehrr]

Fixes from the 2026-10-14 audit of main @ 8b8b839 (v0.2.0).

Behaviour changes: --pages 0, --concurrency 0 and negative --pages/--concurrency/--delay/--retry-delay/--proxy-block-retries are now exit 2. pyppeteer with no --proxy and a page that never loads now ends as a load failure (exit 5/6) instead of exit 1. With a pool, pyppeteer no longer rotates on a repeated timeout (only on an unusable exit), matching Playwright and Selenium. The canary's Scraping Browser job is red until KOSE_CDP_ENDPOINT is set.

Problems and resulting behaviour

1. pyppeteer crash, no proxy, load failure (high). puppeteer_scraper.py called pool.advance(...) on a None pool after --retries failed navigations, so the run ended with AttributeError and exit 1. It now rotates only on an unusable exit (ERR_PROXY_*) and only when has_pool, the same as the other two engines.
2. The solve allowance was spent by a no-op detection (med). _SolveBudget.spend counted every call to handle_captcha_if_present. The pre-classification call, which detects nothing on a normal page, used up SOLVES_PER_PAGE = 1, so a challenge first seen after classification was never solved. Now handle_captcha_if_present(..., budget) calls budget.claim() immediately before solve_recaptcha. Detecting nothing, content already rendered, or no key claim nothing. A failed purchase still counts. There is no reset on retry or rotation.
3. Credential scan read 0 files under build//dist/ (med). Exclusions are now tested on path.relative_to(REPO).parts, and a scan with zero files fails. .env* backups, .json, .csv, .toml, .cfg and .ini are read. The pyvenv.cfg venv detection is unchanged.
4. CLI validation. New page_flow.at_least_one, non_negative_int and non_negative_seconds types, used in all four engines and in scraper_api_client.py --retry-delay.
5. .gitignore gains *.page[0-9]*, /*.html and the working-dir kinds. No tracked file becomes ignored (git ls-files | git check-ignore --stdin --no-index is empty).
6. .env.example ships KOSE_URL commented out. The env-example checks were adapted and pin this.
7. Comment and CI-output cleanup (F5). Fixes: the parser_found_nothing exit comment (it is 5), the donor column names, the "discounted" summary (now "priced"), the Cloudflare wording, the undated 403-vs-residential claim, "304 is all of" now dated, and the old banned product word in captcha_solver.py.
8. Private citations (F4). Every CLAUDE.md / §N / "playbook" reference was removed from tracked files, including README, --help, .gitignore, workflows and the released 0.1.1 CHANGELOG line. Each rule is restated or the citation dropped. New guard: check_no_citations_of_private_notes (tokens assembled from pieces). The v0.1.1 release title is now "v0.1.1 — audit pass", and one sentence of its body changed; the rest is byte-identical.
9. Canary (owner request). The listing, HTTP-vs-browser and product jobs already run ungated daily and require real rows; the latest scheduled run returned 72 products. The Scraping Browser job used to pass by skipping. It now fails with ::error:: naming KOSE_CDP_ENDPOINT when the secret is absent, and exit 5 over CDP is red, not a warning. The retracted "credentials live about a day" text is removed.
10. smoke_test.py -v crashed on a non-string detail. Fixed.

Evidence

• Offline suite results:

  venv	passed	failed	skipped groups
  core-only	580	0	19
  playwright	624	0	10
  selenium	612	0	14
  puppeteer	612	0	14

  No browser download in any of them; run with PYTHONDONTWRITEBYTECODE=1 and __pycache__ removed.

• New checks: check_a_page_that_never_loads_without_a_proxy_does_not_crash (timeout and dead-proxy errors, pool=None, all three engines), check_the_solve_allowance_is_spent_only_on_a_purchase (late challenge gives 1 purchase, always-challenge gives 1, failing solver gives 1; stub solver, no network), check_the_credential_scan_reads_a_clone_under_any_parent, check_counts_and_pauses_refuse_negative_values, check_no_citations_of_private_notes.

• Planted-fault controls, each red on the named check and green again after restore:

  Planted fault	Named check that went red
  old pyppeteer if load_failed and ...: pool.advance	puppeteer_scraper: a timeout with no --proxy does not raise (AttributeError, as in the audit)
  old budget semantics (claim at function entry, not at purchase)	only playwright_scraper: a challenge seen only after classification is solved (got 0, want 1)
  no counting at all	the late-challenge, every-call and failed-purchase checks (got 3–4, want 1)
  absolute path.parts	a key planted in a clone under build/ is caught (and dist/)
  zero-scan guard removed	a scan that read no files fails
  .env* reading removed	a key in an untracked .env.bak is caught
  selenium --pages type=int	selenium_scraper: --pages goes through page_flow.at_least_one
  non_negative_int accepting -1	non_negative_int('-1') is refused
  CLAUDE.md planted in README	README.md:342 cites private notes
  § planted in product_parser	product_parser.py:361 cites private notes

• CLI: http_scraper.py --pages 0 gives error: argument --pages: must be at least 1. --delay -1, --proxy-block-retries -1 and --retry-delay -2 each give cannot be negative, as do the three browser engines and scraper_api_client.py --retry-delay -1.

Not changed, and why

• F8 (HTTP 429 as blocked, no throttle state) and F9 (x-debug JSON key fields): marked as owner decisions in the audit, and both unmeasured on this site.
• Shared fetch loop (T10) and schema/sinks (T11): refactor or feature, out of scope.
• Version bump, tag, release: not done. The entries are under [Unreleased].
• Atomic writes, CSV formula escape, exit codes, --retries 0, diff_runs: already compliant, verified by the audit.
• Repo description (33 words): informational only in the org audit; the wording is already correct.

🤖 Generated with Claude Code