Skip to content

Remove etsy-scraper leftovers from issue templates, SECURITY and CONTRIBUTING - #8

Merged
jehrr merged 5 commits into
mainfrom
fix/donor-leftovers
Sep 28, 2026
Merged

jehrr merged 5 commits into
mainfrom
fix/donor-leftovers

Conversation

@jehrr

@jehrr jehrr commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Leftovers from etsy-scraper (the donor this repo was bootstrapped from) that describe Etsy as if it were this site.

Fixed

  • .github/ISSUE_TEMPLATE/bug_report.yml — the pre-filing tips described Etsy's DataDome (403 to datacentre IPs, t=bv), shop_rating, and Etsy's /fr/ → /de/ redirects; the example command used an etsy.com search URL and --out grills; the "expected" placeholder was Farfetch's "96 products". Replaced with what this README/TROUBLESHOOTING measure: a scored address gets no response at all (no challenge, so a key does not help; any residential exit clears it), category listings print no rating/sold/was-price, image_url/slug_id partial by design; example uses the README's kopi-bubuk category URL and the README's 119-row healthy run.
  • site_changed.yml — "Etsy changed something", etsy.com, "JSON-LD first", a lowest_price_30d tile-overlay option (no such column here), a "below 90%" DOM confirmation option and an etsy.com placeholder. Now describes the DOM-only anchors (grid container, tile marker, reading order; window.__cache on product pages) and the per-page-kind confirmation floor from CONTRIBUTING; evidence asks for --dump-html.
  • SECURITY.md — "Etsy changing its markup is expected" → Tokopedia.
  • CONTRIBUTING.md — removed the "rating is the LISTING's and shop_rating is the SELLER's" property: this repo has no shop_rating column and the 825/375/16,679 figures are Etsy's. Dropped the stale "Five properties … all five" count (the list has eight). "the registration form" → "a registration or login form".
  • CONTRIBUTING.md (second commit) — the site-change section described a structured-price confirmation share with Etsy's per-page-kind floors (search 8%, category 70%, shop 80%). Tokopedia listings carry no structured price and the engines deliberately log plain price coverage against PRICE_FLOOR (95%); the paragraph and the template option now say that.
  • captcha_solver.py — docstring pointed at a "No DataDome solver" section that went with the inherited etsy-scraper solver in 0.1.0; now says so and points at the CHANGELOG.
  • smoke_test.py — privacy-pattern comment said "MediaMarkt's pages embed…" as if this site's; now names mediamarkt-scraper as a sibling.
  • output_writer.py — a comment used shop_rating as its example sidecar extra; it now names scroll, which this repo's engines actually write.

Kept, deliberately

  • README.md:34 — "www.etsy.com 403, www.mediamarkt.de 403 … from the same machine in the same minute" is a measured contrast naming other sites as other sites.
  • Sibling provenance comments (MediaMarkt's instalment line (§4), mediamarkt-scraper's first live run, amazon-scraper's situation, the farfetch note in captcha_solver.py, the craigslist .gitignore note) — they name the sibling and state the lesson.
  • Generic "a DataDome shell, a branded 403" lists in page_flow.py/product_parser.py/playwright_scraper.py — they enumerate kinds of refusal, not Etsy's behaviour.
  • README badges were already complete and in family order, with an honest needs a residential exit; repo description/topics have no donor words. Released CHANGELOG sections untouched.

Verification

python3 smoke_test.py exit 0 before and after (same two engine groups skipped locally); ci_checks.py --secret-check and --help-check --sample-check exit 0; issue templates parse as YAML.

🤖 Generated with Claude Code

jehrr and others added 5 commits September 28, 2026 19:49
- bug_report.yml: Etsy/DataDome/t=bv/shop_rating/storefront-redirect tips
  replaced with Tokopedia's measured behaviour (scored address gets no
  response; category listings print no rating/sold/was-price); etsy.com
  example command replaced with the README's kopi-bubuk category URL.
- site_changed.yml: "Etsy changed something", etsy.com, JSON-LD-first and
  lowest_price_30d overlay replaced with the DOM-only anchors this repo
  really reads; evidence asks for --dump-html instead of JSON-LD.
- SECURITY.md: "Etsy changing its markup" -> Tokopedia.
- CONTRIBUTING.md: drop the rating/shop_rating property (no such column
  here, Etsy's figures) and the stale "five properties" count; "the
  registration form" generalised.
- output_writer.py: sidecar-extra example names `scroll`, not shop_rating.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…epo's PRICE_FLOOR

Tokopedia listings carry no structured price, so the engines log plain
price coverage against PRICE_FLOOR (95%), not a confirmation share with
Etsy's search/category/shop floors. site_changed.yml option updated to match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The section went with the inherited etsy-scraper DataDome solver in 0.1.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tern comment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SECURITY.md said the project has no releases or version tags; it has both
  (gh release list). Supported versions now names the latest release and main,
  in the wording bbb-scraper and foodpanda-scraper already use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jehrr
jehrr force-pushed the fix/donor-leftovers branch from 69be687 to 277b2da Compare September 28, 2026 19:50
@jehrr
jehrr merged commit 6e0a1b5 into main Sep 28, 2026
7 checks passed
@jehrr
jehrr deleted the fix/donor-leftovers branch September 28, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant