Repository navigation
One solve per page, CSV formula guard, parser failures and 404s reported, canary that must test the site - #13
Merged
Merged
Conversation
…ted correctly - Every paid captcha solve in all three engines goes through one per-page allowance (page_flow.SolveBudget), checked before the purchase; a challenged page bought 3 tokens against a limit of 1 before. - pyppeteer no longer crashes (exit 1) when a page fails to load without a proxy. - The HTTP status reaches the classifier; dubizzle's 404 is recognised from the body where Selenium has no status. - A served page with ads that parses to zero rows ends as parser_found_nothing (exit 5, or 6 with rows), not exit 4 or a complete run. - CSV cells that start with = + - @ or a control character are prefixed with an apostrophe (CSV only), counted in the sidecar as csv_cells_escaped. - Count and delay flags reject 0/negative values with an argparse error. - Capture directories ignored and flagged by kind; wording scan covers the suite; private-note citations removed and guarded. - Canary: fails without its secret, red on exit 3/5/6, weekly schedule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jehrr
marked this pull request as ready for review
October 15, 2026 21:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes from the 2026-10-14 audit of
8c653e3, plus the owner's canary requirement.Behaviour changes
stop_reason: parser_found_nothing→ exit 5 (or 6 with rows from earlier pages). Before: exit 4 on page 1, a complete run (exit 0) on page 2+.--pages 0,--concurrency 0, negative--delay/--retry-delay/--proxy-block-retries(and inscraper_api_client.pynegative--retries/--retry-delay,--timeout< 1) are usage errors (exit 2) naming the flag.= + - @ \t \r \nget a leading apostrophe (CSV only; JSON untouched;csv_cells_escapedin the sidecar, merged under the caller'sextra).DUBIZZLE_CDP_ENDPOINTis missing (named in::error::), red on exit 3/5/6, red on a loggedparser_found_nothing, weekly schedule added, no longer requires a merchandised "car of the week" row.Fixes
page_flow.SolveBudget, passed to bothhandle_captcha_if_presentcall sites in all three engines,allows()/spend()immediately beforesolve_recaptchapool.advanceon Nonehas_poolas Playwright doesgoto,pool=None→load_failed, no exception_csv_escapeafter list/dict serialisation, strings only=HYPERLINK(...)→'=HYPERLINK(...),-5.0unchangedgoto()'s status (reset to None after a solve reload); Selenium relies onproduct_parser.is_not_found_page(empty<title>+ empty Redux action list: matches only the 404 among the 13 captures)not_foundfixture →emptyin all three enginesPageOutcome.parse_failed,page_flow.failure_reasonused by every stop-reason sitemotors_p1→parser_found_nothing;finish_run→ 5 / 6§N/ private-file references reworded or removed; new suite guard (tokens assembled from pieces) over every tracked file. Release v0.1.0 notes checked: nonekeyword=kopi/buy-box/scroll-count/2026-09-09 comments, umlaut URL docstring,mugsDocker example rewritten for this site.gitignore+CAPTURE_SHAPESby kind (live/ out/ output/ run(s)/ result(s)/ tmp/ scratch/ dump(s)/, root*.html)git check-ignore; intent-to-addlive/dump.html→ci_checks.py --secret-checkFAILEDgh repo edit)Verification
PYTHONDONTWRITEBYTECODE=1,__pycache__cleared), each RED on the named check, then restored green: CSV escape removed;budget.spend()removed (AST check in core, behavioural in selenium venv); pyppeteer guard removed; one_classifywithout status;is_not_found_page→ False;parse_failednot set (playwright venv);--delayback totype=float; a§in README; a banned phrase in the suite; canary exit 3 not failing; canary parser grep disabled; missing-secret step not failing.ci_checks.py --allgreen.Not changed
--allow-emptysidecar status word, F12 refusing CDP combinations, F11 tagging 0.1.1 / release: owner decisions.🤖 Generated with Claude Code