[jehrr]

Problem

An audit of main at 85553e8 (2026-10-14) found these, each reproduced by running code rather than by reading it:

1. One refused page could buy several CAPTCHA solves. page_flow.land() ran its own for _ in range(SOLVES_PER_PAGE) on every landing, and fetch_one_page lands again after every block-retry and rotation. With the solver stubbed: 2 solves with no pool, 4 with a 3-exit pool, against SOLVES_PER_PAGE = 1.
2. Output writes truncated first. A failure inside json.dump turned a good out.json into 13 bytes of invalid JSON. The same applied to the CSV and the .meta.json sidecar.
3. CSV cells were not formula-escaped. =HYPERLINK(...) and -cmd were written as they came. Advertiser, portfolio and announcement names are third-party text.
4. A copied .env.example broke every flag-built query. It shipped BINANCE_URL set to a P2P page. --mode announcements, --mode copytrading and --asset/--fiat then exited 2 with a conflict against a URL the user never typed.
5. HTTP 451 advice said a US exit is refused. Nobody has measured that, and the daily canary is served from a GitHub runner with no proxy. A CDP 401 was explained as credentials that "last about a day", which was never measured either.
6. Comments cited private notes ("CLAUDE.md §N", bare §N) across 14 files.
7. Minor: a root-level *.html capture was not ignored, .dockerignore listed .env but not .env*, there were two donor-repo mentions in comments, and the README said "Five things" above six.

What happens now

• Each page gets one page_flow.SolveBudget, created once in fetch_one_page and passed to every land() for that page. The budget is checked before each purchase. Retries and rotations do not reset it. The --pay-type pre-check lands without solving, so page 1 pays from its own allowance.
• _atomic() in output_writer writes to a temporary file in the target's directory, then flushes, fsyncs and calls os.replace, and unlinks the temporary file on any error. CSV keeps newline="". A new file gets 0o666 & ~umask and an existing file keeps its mode. write_json, write_csv and write_run_meta all go through it. Each file is replaced atomically on its own; the three are not one transaction.
• In CSV only, string cells starting with = + - @ \t \r \n get a leading '. This runs after the list cells are joined. Numbers and JSON are unchanged. The sidecar records csv_cells_escaped, merged under the caller's extra.
• Each of the four engines records args.typed_url before env_config.apply. build_query drops an environment URL when the user typed --mode or a query flag. A typed --url together with a flag is still refused. .env.example now ships BINANCE_URL commented out.
• The 451 advice says the status has not been observed and asks for the exit's country. The README, TROUBLESHOOTING, CONTRIBUTING, .env.example and the bug template no longer claim a US refusal. A 401 is reported as "credentials refused (expired, revoked or mistyped)".
• Every private citation is replaced by the reason in plain words, or removed. A new suite check fails on such citations in any shipped file. Its tokens are assembled from pieces so the check does not match itself.
• /*.html is now in .gitignore, .env* is in .dockerignore, and the README heading says "Six things".

Evidence

• Offline suite, core-only venv (Python 3.13): 426 passed, 0 failed. Before this change it was 390. The five skipped groups are the absent engines.

• Per-engine venvs (requirements.txt + requirements-<engine>.txt, no browser download): playwright 431/0, selenium 428/0, puppeteer 428/0. Each run skipped only the other engines, and pip check was clean in each.

• ci_checks.py --secret-check: 48 files, nothing credential-shaped.

• No-network probe of the env URL fix: with BINANCE_URL=<p2p url>, http_scraper.py --mode announcements --pages 0 reaches the --pages validation instead of failing on the mode conflict. A typed --url ... --fiat TRY is still refused.

• Controls. Each fault below was planted with PYTHONDONTWRITEBYTECODE=1 and __pycache__ removed. Each one turned the suite RED on the named check, and the restored tree was green again:

  • allowance reset per landing (land(..., SolveBudget())): got 2 solves, want 1
  • no allowance at all
  • non-atomic JSON, CSV and sidecar: each previous file was clobbered
  • temp-file cleanup removed
  • flat 0o644 & ~umask: umask 002 gave 644
  • no chmod: umask 022 gave 600
  • CSV escape removed; CSV escape applied before list joining
  • housekeeping merged over the caller's extra
  • env URL treated as typed
  • typed_url recorded after env_config.apply
  • US claim restored in TROUBLESHOOTING
  • a private citation planted in page_flow.py, CHANGELOG.md and canary.yml

  One planted fault (env URL treated as typed) first crashed the suite with an uncaught SystemExit instead of failing the named check. The check now catches SystemExit and reports by name.

• Release notes for v0.1.0, v0.1.1 and v0.2.0 contain no private citations, so none were edited. No released CHANGELOG section contained one either.

Not changed

• The solve allowance is per page, not per run. That is the family rule. Whether binance should allow one solve per run, since one landing serves every page, is an open owner question.
• .env.example and the env-URL fix: both options from the audit were taken (comment it out, and record the typed URL). Either can be reverted on its own.
• US ToS wording: the "Binance's terms exclude … the United States" sentence was removed rather than kept with a source. Restoring it as a terms-of-service statement with a link is an owner choice.
• The full challenge-settle wait before solving a visible CAPTCHA (low, read in source only). Changing it would alter the shared loop's timing, and it costs time, not money.
• Version bump, tag and release: [Unreleased] now holds these fixes and the earlier --retries 0 change.
• Repo metadata: the description already names "2Captcha Scraping Browser API" and the topics are correct. The org-profile audit lists this description at 29 words, which it calls informational only.
• History: the first commit's subject names its donor. History is not rewritten.

🤖 Generated with Claude Code