Repository navigation
Remove donor-repo leftovers (Tokopedia/Catawiki/VRBO prose) - #11
Merged
Merged
Conversation
- scraper_api_client --help: description and --url no longer describe a JS-hydrated GraphQL grid and a lodging-card selector; --category help no longer claims a URL-segment default the client does not implement. - Issue templates rewritten for foodpanda (were an auction site's). - pyppeteer/Selenium docstrings: one listing mode, real --url example; unreachable --mode product sidecar branch and shop_facts removed. - Engine, captcha_solver and output_writer comments: donor stories (hub pages, search grid, kopi tracking tail, Akamai 394-byte refusal, lots, Orlando search, stream reset, shop_rating) replaced with this repo's measurements or attributed to the sibling; dangling "No DataDome solver" reference removed. - SECURITY.md: no longer claims the project has no releases. - Access badge: no account needed, from a residential IP. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jehrr
force-pushed
the
fix/donor-leftovers
branch
from
September 29, 2026 08:13
0406aae to
7621731
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixed
scraper_api_client.py --help: the description said these pages need JavaScript and a single fetch returns "about 5 products where a browser engine returns 60";--urlsaid to pass--wait-element '[data-stid="lodging-card-responsive"]'because "the grid arrives over client-side GraphQL" (VRBO/Tokopedia). This file's own docstring measures 48 of 48 vendors from one request on a server-rendered grid. Both rewritten;--categoryhelp no longer claims a URL-segment default (the client never readsargs.category).seller_score, page-100 cap,/en/locales,__NEXT_DATA__,c-lot-card,--out grills, "96 products"). Rewritten from README / CONTRIBUTING / TROUBLESHOOTING.--mode product(brand, EAN) and usedfoodpanda.pk/p/makanan-minuman/minuman/kopi-bubuk(a Tokopedia path) as the example. Now the onelistingmode and/city/lahore/area/gulberg.if args.mode == "product"sidecar branch (shop_facts, "Shop: %s") was unreachable (choices=["listing"]). Removed with theshop_factsfield; the listing branch runs unchanged. The twins'stop_reason = "single_page_mode" if args.mode == "product"is now"completed", as in playwright_scraper (same value at runtime)./p/<slug>hub +akamaihd.net(now attributed to sibling tokopedia-scraper), "hub category" / "taxonomy hub", CATEGORY vs SEARCH grid over GraphQL,keyword=kopi+ UTM tracking tail, detail-page buy box (engines); Akamai 394-byte "Access Denied" and "lots" (captcha_solver.py, replaced with the README's headless 0/4 vs headful 4/4 and PerimeterX's widget-less stub); "Oops, produk nggak ditemukan", HTTP/2 stream reset, "Orlando search",shop_rating(output_writer.py). Dangling "See 'No DataDome solver' below" removed.SECURITY.mdsaid "no releases or version tags"; v0.2.0/v0.2.1 exist.runs without an account->runs without an account · residential IP, matching README's "Do I need to buy anything?" (measured from a residential address; the canary notes a datacentre runner is worse).Repo metadata
price-monitoringremoved (a vendor tile carries no price: README measures 0 price nodes over 1,952 tiles),restaurantsadded.Kept, deliberately
scraper_api_client.pydocstring "a sibling repo where this path reaches 3 of 50 cards": names the sibling.captcha_solver.py"The sibling repo ... image captcha" paragraph, farfetch-scraper notes;product_parser.py/smoke_test.pyamazon-scraper review-count notes;.gitignorecraigslist note: sibling provenance.product_parser.py"datadome"in the vendor marker list: a generic vendor name, not donor prose.CHANGELOG.mdreleased sections (incl. mediamarkt mention): history.smoke_test.pydeliveroo URL: a negative-host fixture.Not changed, found in passing
selenium_scraper.pyandpuppeteer_scraper.pyaccept--categorybut never record it asrun_label, while their help says they do (playwright does). That is twin drift in code, not donor text, so it is reported here rather than fixed.total_availablethat is always None on this site.Suite: 535 checks, OK before and after (engine groups skipped, engines absent).
ci_checks.py --secret-check --help-check --sample-checkgreen.🤖 Generated with Claude Code