Read the newer EMI line, test the parser in the canary, validate numeric flags

[jehrr]

Fixes from the 2026-10-14 audit of af36221, plus the canary rule that a run which did not test the site must not end green.

Problems and resulting behaviour

1. EMI missed, EMI line read as the hub (parser defect behind the red canary). The audit could not tell whether the red canary on 2026-10-08/09/10 came from the site or the parser. The artefacts of 2026-10-08, -10 and -15 settle it. The cards there print EMI 15,916 (no /m) next to a new Save extra ₹19.5K on promotion. The 2026-10-14 artefact still has EMI 18,833/m *. _EMI_RE required /m, which had two effects:

• 0 EMIs were read on those days.
• EMI 15,916 contains a comma, so it became the hub on 82–102 rows a run.

Now both wordings are read, the promotion is recognised and not left over as the tag, and a fixture cut from the 2026-10-15 canary dump pins the values. The trimmed fixture was verified to parse identically to the untrimmed card.

Re-parsing the four downloaded dumps (2026-10-08, 10-10, 10-14, 10-15) with this branch reads an EMI on 60/60, 102/102, 82/82 and 94/95 rows (by artefact date), with 0 EMI lines in the hub.

2. Canary (owner rule, section 12).

• Before: red 3 of 8 days on an emi_monthly ≥ 90% floor and on "strike price always travels with the discount badge". On 2026-10-10 the site printed 20 strike prices with no badge, which is a merchandising choice. Exit 5 ended green with a ::warning::, even though a runner-side navigation timeout also exits 5.

• After: the assertions moved to .github/canary_check.py (runnable against an artefact and exercised by the offline suite). They test the parser:

  • row floor and core-field floors;
  • EMI-box text never lands in hub/tag/variant;
  • where both a strike and the site's discount are printed, they agree within ₹1,000;
  • the existing price-basis, sku, page+position, scroll and advertised-total invariants.

  The EMI share and the strike share are printed, not asserted. Exit 3/4/5/6/2/1 all fail with ::error::.

• Run against the old rows of the three red days, the new check fails each one by name: "N row(s) carry EMI-box text in another column". Against the same dumps re-parsed by this branch, it passes all four.

• No secret is needed: the canary runs from a bare runner. SPINNY_CDP_ENDPOINT stays optional and env-only.

3. --retry-delay -1 crash and other numeric flags.

• In all three engines, these now go through page_flow validators and fail with exit 2 naming the flag: --pages and --concurrency (≥1), --proxy-block-retries, --delay and --retry-delay (≥0). --retries was already validated and is kept.
• scraper_api_client.py now validates --timeout (≥1), --retries (≥0, extra attempts) and --retry-delay (≥0).

4. Credential scan.

• Excluded directory names are tested against path.relative_to(REPO).parts. Before, a clone under any venv/ parent scanned 0 files and printed ok.
• A scan of 0 files now fails.
• .env* backups, .json, .csv, .html and NAME.pageN dumps are read too.
• pyvenv.cfg directories are still skipped.

5. .gitignore. Now covers root /*.html, *.page[0-9]*, and live/ out/ output/ runs/ results/ tmp/ scratch/ dumps/. check-ignore confirms that workflows, tests, samples, .env.example and smoke_test.py stay tracked.

6. HTTP status to the classifier. Playwright and pyppeteer pass the goto() response status. The status is reset after a solve navigates the page, and a non-int status is treated as unknown. Selenium has no status and keeps the page's own 404 copy.

7. Private citations.

• Removed every §N and the CLAUDE.md / "family notes" mentions from tracked files; the rule is stated in place.
• A suite guard (tokens assembled from pieces) fails if any reappear.
• Release notes v0.1.0–v0.1.2 contain none, so they were not edited.

8. Wording.

• The README tagline names the 2Captcha Scraping Browser API.
• The README describes exit 5 as "the content was never obtained".
• The run_meta docstring no longer describes a product mode and a sold column.

Evidence

• Offline suite: core-only venv exit 0 (635 PASS). Each engine was also run in its own venv with only its requirements installed (no browser download): playwright, pyppeteer and selenium each exit 0 with 0 FAIL, that engine's group ran (only the other engines skipped), and pip check is clean. pytest -q passes.
• Planted-fault controls: 13 of 13 went red on the named check, with PYTHONDONTWRITEBYTECODE=1 and __pycache__ cleared between runs; the restored tree is green. The faults planted:
  • scanner absolute parts;
  • zero-scan → ok;
  • .env backups unread;
  • a § reinserted;
  • --delay type=float;
  • --retry-delay minimum −9;
  • status dropped from the Playwright classify;
  • status coercion removed;
  • /m required again;
  • promo regex removed;
  • EMI floor re-added to the canary;
  • EMI-leak check disabled;
  • canary exit 5 made a warning again.
• Real scan from a clone under a venv/ parent: 37 files scanned. A planted .env.bak key was found.
• scraper_api_client.py --retries -1 gives error: argument --retries: --retries must be at least 0, not -1.

Not changed

• No version bump, tag or release: [Unreleased] carries the changes, led by behaviour-change blockquotes.
• The shared fetch loop refactor and the schema/sinks work are decisions, not fixes.
• The donor names in provenance comments (captcha_solver.py, product_parser.py) describe where a lesson came from and have no code effect.
• Repo metadata: description, homepage and 15 topics were already correct, so they were left alone.

🤖 Generated with Claude Code