Repository navigation
Paid solves stay within one per page, atomic outputs, CSV formula guard, canary red unless it read Medium - #12
Merged
Merged
Conversation
… and the canary is red unless it read Medium - One SolveBudget per page covers both solver call sites in all three engines, checked before each purchase and never reset by a retry. - JSON, CSV and the sidecar are written atomically; new files follow the umask, existing files keep their mode. - CSV cells starting with a formula character are neutralised (CSV only), counted as csv_cells_escaped in the sidecar. - A page the state policy does not parse yields no rows. - Playwright and pyppeteer pass the HTTP status to the classifier. - Numeric flags reject impossible values (--pages 0, negative delays...). - Credential scan: exclusions relative to the repo, an empty scan fails, data files and .env.* backups are read. - .gitignore covers multi-page and top-level dumps and capture directories. - --category is documented as accepted and unused; README exit 5 row, captcha-charge sentences and product naming corrected; an unmeasured credential-lifetime claim removed; other-site comments fixed. - Private-note citations removed from public files, with a guard. - Canary: exit 3/4/5 fail the job after one rested retry per half. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jehrr
marked this pull request as ready for review
October 15, 2026 13:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes from the 2026-10-14 audit of
main@23bcabb(no open PRs at the time).Problems and resulting behaviour
handle_captcha_if_presentcall had no budget, so with detection forced and the solver stubbed, one page with no pool bought 4 solves againstSOLVES_PER_PAGE = 1in every engine. Now onepage_flow.SolveBudgetper page is shared by both call sites, checked right before the purchase, and not reset by retries or relaunches. README/TROUBLESHOOTING no longer say nothing can ever be charged: the managed challenge is never sent to a solver; a rendered reCAPTCHA (not seen on Medium) is the one paid path, at most once per page._atomic(temp file in the same dir, fsync,os.replace, cleanup on error,newline=""for CSV). New file0o666 & ~umask; existing file keeps its mode.= + - @ TAB CR LFget a leading'in CSV only, after list joining; numbers untouched;csv_cells_escapedin the sidecar, under the caller'sextra.should_parsenow gates the parse (F4): anemptypage gives 0 rows (it gave 6 with the classifier forced).relative_to(REPO).parts; 0 files scanned fails;.json/.html/.csv/.toml/.cfg/.ini,Dockerfileand.env.*are read (hits in git-ignored files warn).EXIT_BLOCKEDcomment describes Medium's two Cloudflare refusals;shop_ratingandFINDINGS.mdreferences gone; sample check countsis_paywalledinstead oforiginal_price.--category(F9): help says it is accepted and unused; a warning when passed. (Removing it is left to the owner.).env.exampleand the 401 advice, dump shapes ignored (F15), "2Captcha Scraping Browser API" naming (F17).--pages 0,--concurrency 0, negative--delay/--retry-delay/--proxy-block-retries(and negative--retrieson scraper_api_client) are argparse errors naming the flag.tests.yml: inline--help/sample heredocs removed (ci_checks.py already ran both); suite fails if a workflow imports a local module inline.::error::saying whether it was access or the parser. Exit 0/6 must pass the data floors. No secret required (bare runner is served most days);MEDIUM_PROXYstays optional.Evidence
smoke_test: OK, 0 FAIL. Per-engine venvs (playwright, selenium, pyppeteer; no browser download): OK, 0 FAIL, only the other engines skipped,pip checkclean.emptypage = 0 rows; status 403 seen by classifier.PYTHONDONTWRITEBYTECODE=1, all RED on the named check, no crash: budget bypass (playwright),spend()removed (selenium), parse gate removed (pyppeteer), status dropped (pyppeteer), non-atomic JSON, CSV escape off, flat 0644, scan by absolute parts, planted citation,/*.htmlun-ignored,--pages type=int, canary exit 3 as warning, inline workflow import.Not changed
--allow-emptysidecar status, F16 version/tag/release, T15 commit-subject labels: owner decisions.--concurrency/--fingerprintwith--cdp-endpointinstead of warning: behaviour change left for the owner.not_foundstate for 404 (no real Medium 404 capture to build it on).🤖 Generated with Claude Code