Remove donor leftovers: twin engines' --mode help, access badge, issue templates

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -78,7 +78,7 @@ body:
       placeholder: |
         python3 playwright_scraper.py \
           --url "https://www.etsy.com/search?q=handmade+mug" \
-          --pages 3 --out grills
+          --pages 3 --out mugs
     validations:
       required: true
 
@@ -98,7 +98,7 @@ body:
     attributes:
       label: What you expected instead
       placeholder: >-
-        96 products, as the README says a category page yields.
+        184 rows from three search pages, as the README's healthy run shows.
     validations:
       required: true
 

.github/ISSUE_TEMPLATE/site_changed.yml

@@ -9,8 +9,11 @@ body:
         own report because the fix is different from a code bug: something on
         etsy.com moved.
 
-        Useful to know before filing: the parser tries **JSON-LD first**, then a
-        CSS + URL-pattern fallback. Which one broke narrows the fix a lot.
+        Useful to know before filing: the primary anchor is Etsy's own
+        `data-listing-id` attribute, the JSON-LD `ItemList` only enriches and
+        cross-checks it (it is partial — 8 entries for 64 tiles on a search
+        page), and the `/listing/<id>/<slug>` URL pattern is the fallback.
+        Which one broke narrows the fix a lot.
 
   - type: dropdown
     id: what_broke
@@ -19,8 +22,8 @@ body:
       options:
         - Products (0 rows on a page that visibly has products)
         - A specific field is null or wrong (price, sku, image, URL)
-        - original_price or lowest_price_30d empty out (the tile overlay)
-        - DOM price confirmation dropped below 90% in the run log
+        - original_price or discount_pct empty out, or come out wrong
+        - The structured-price confirmation share warned below its per-page-kind floor in the run log
         - Pagination stops early or repeats a page
         - The captcha detector finds nothing on a page that has one
         - Something else

CHANGELOG.md

@@ -25,6 +25,30 @@ with it, so nobody discovers it from a bill or from a diff.
   client handed `status` onward, so a target 403/503 never reached the page
   classifier. It now reads `http_code`, falling back to `status` only if that
   is an integer. After the fix, one live call (`--wait-text mug` on the canary's search URL, no `--cdp-url`) was accepted — no 422 — but ran into the API's own sync-wait limit (HTTP 408, task still pending at the default 60 s), so no target status came back from that call to show.
+- **`--help` for `--mode` in `selenium_scraper.py` and `puppeteer_scraper.py`
+  described mediamarkt-scraper's product mode** — "one /product/ page, with
+  brand, EAN, description" and "listing or product", while all three engines
+  take `listing`, `product` and `shop` and Etsy's detail page is
+  `/listing/<id>`. Both twins now carry the Playwright engine's help text and
+  module docstring for the three modes.
+- **The access badge said "needs a residential exit".** This README measures
+  a residential exit as NOT sufficient on its own (Playwright Chromium via a
+  residential US exit: 403, `t=bv`) and the Scraping Browser API as the path
+  that works, so the badge now says "needs Scraping Browser API".
+- **Leftovers from the donor repos in the issue templates and docs:** the
+  bug-report example wrote `--out grills` and expected Farfetch's "96
+  products"; it now uses `--out mugs` and the README's 184-row healthy run.
+  The site-change template described a JSON-LD-first parser, a
+  `lowest_price_30d` overlay (no such column here) and a "below 90%" DOM
+  confirmation; it now names `data-listing-id` as the primary anchor and the
+  per-page-kind confirmation floors. `SECURITY.md` named Akamai rather than
+  DataDome, and a comment in `output_writer.py` used MediaMarkt's
+  `/de/category/tv-audio-202.html` as its example of a page with no listings.
+  A smoke-test comment described "MediaMarkt's pages" without saying it was
+  a sibling site; it now does.
+
+- `SECURITY.md` said this project has no releases or version tags; it has
+  both. "Supported versions" now names the latest release and `main`.
 
 ## [0.2.3] — 2026-09-16
 

CONTRIBUTING.md

@@ -182,7 +182,7 @@ datacentre address will be refused outright, so "it returned nothing" from a
 VPS is not a finding. Product counts differ by category and by URL, so a bare
 "worked for me" is not reproducible.
 
-Do not add anything that submits the registration form. This project
+Do not add anything that submits a registration or login form. This project
 deliberately never does, and a captcha token proved valid by creating a real
 account is not a result worth having.
 

README.md

@@ -10,7 +10,7 @@ blocking, JSON/CSV output and a run-metadata sidecar.
 ![Python](https://img.shields.io/badge/python-3.9%2B-blue?logo=python&logoColor=white)
 ![licence](https://img.shields.io/badge/licence-MIT-green)
 ![engines](https://img.shields.io/badge/engines-Playwright%20%7C%20Selenium%20%7C%20Puppeteer-blueviolet)
-![access](https://img.shields.io/badge/needs-a%20residential%20exit-orange)
+![access](https://img.shields.io/badge/needs-Scraping%20Browser%20API-orange)
 
 ---
 

SECURITY.md

@@ -55,7 +55,7 @@ Not because these do not matter, but because they belong somewhere else:
 
 - **Bypassing Etsy's bot protection.** This scraper drives an ordinary
   browser and passes challenges the way a browser does. Anything about how
-  Akamai or reCAPTCHA behave is not a vulnerability in this repository.
+  DataDome behaves is not a vulnerability in this repository.
 - **The scraper stopped working.** Etsy changing its markup is expected —
   file it as a normal issue, there is a template for exactly that.
 - **Anything about 2Captcha's services** — the solver API, the Scraping Browser
@@ -79,9 +79,9 @@ Not because these do not matter, but because they belong somewhere else:
 
 ## Supported versions
 
-`main` only. This project has no releases or version tags; fixes land on `main`
-and you update by pulling. If you are running an old clone, update before
-reporting.
+The latest release and `main`. Fixes land on `main` first and ship in the
+next tagged release (see the Releases page). If you are running an old clone,
+update before reporting.
 
 ## If you have leaked a key
 

output_writer.py

@@ -286,9 +286,8 @@ def write_csv(rows: Sequence[Any], path: str, row_cls: Type = Product) -> None:
 # content". See product_parser.detect_bot_challenge.
 #
 # On Etsy this code specifically does NOT cover a page with no listings — a URL
-# like /de/category/tv-audio-202.html that answers 200 with a real page and
-# no product grid, because it is a landing page of sub-categories rather than
-# a listing. That is EXIT_NO_PRODUCTS: the request was served exactly as
+# like a taxonomy hub that answers 200 with a real page and no listing grid,
+# because it is a landing page of sub-categories rather than a listing. That is EXIT_NO_PRODUCTS: the request was served exactly as
 # asked and simply has no products on it. Reporting it as blocked would send
 # a user hunting for a proxy problem that does not exist.
 EXIT_BLOCKED = 3

puppeteer_scraper.py

@@ -9,9 +9,15 @@
 and output_writer.finish_run(), so this file is browser plumbing and nothing
 else.
 
-    --mode listing   (default)  category grids and search results
-    --mode product              one /product/ page, with brand, EAN,
-                                description and the full image list
+    --mode listing   (default)  search results (/search?q=...), category
+                                listings (/c/...) and /market/ pages,
+                                paginated
+    --mode product              one /listing/<id> page, with the shop, the
+                                material, shipping origin, description and
+                                the full image list
+    --mode shop                 one /shop/<name> front: that seller's
+                                catalogue as listing rows, plus the shop's
+                                own rating and location in the sidecar
 
 Two things to know before choosing this engine:
 
@@ -1021,10 +1027,17 @@ def parse_args():
                         "environment or in .env.")
     p.add_argument("--mode", choices=["listing", "product", "shop"],
                    default="listing",
-                   help="listing (default) or product. product reads one "
-                        "/product/ page and adds brand, EAN, description and "
-                        "the full image list — the columns a listing row "
-                        "cannot carry. No --pages in product mode.")
+                   help="listing (default): paginated search results "
+                        "(/search?q=...), a category listing (/c/...) or a "
+                        "/market/ page — around 64 listings per page. "
+                        "product: one /listing/<id> page, which adds the "
+                        "material, shipping origin, description and the full "
+                        "image list, and whose rating is the LISTING's rather "
+                        "than the shop's. shop: one /shop/<name> front — that "
+                        "seller's catalogue as listing rows, with the shop's "
+                        "own rating and location in the sidecar. --pages "
+                        "applies to listing and shop; there is one page to "
+                        "read in product mode.")
     p.add_argument("--category", default=None, help="Label to tag output rows with.")
     p.add_argument("--pages", type=int, default=1, help="Listing pages to crawl")
     p.add_argument("--delay", type=float, default=2.0, help="Delay between pages, seconds")

selenium_scraper.py

@@ -9,9 +9,15 @@
 and output_writer.finish_run(), so this file is browser plumbing and nothing
 else.
 
-    --mode listing   (default)  category grids and search results
-    --mode product              one /product/ page, with brand, EAN,
-                                description and the full image list
+    --mode listing   (default)  search results (/search?q=...), category
+                                listings (/c/...) and /market/ pages,
+                                paginated
+    --mode product              one /listing/<id> page, with the shop, the
+                                material, shipping origin, description and
+                                the full image list
+    --mode shop                 one /shop/<name> front: that seller's
+                                catalogue as listing rows, plus the shop's
+                                own rating and location in the sidecar
 
 Two limits of this engine, stated here rather than left to be discovered.
 Neither is a bug in this code and neither can be fixed from here:
@@ -1018,10 +1024,17 @@ def parse_args():
                         "environment or in .env.")
     p.add_argument("--mode", choices=["listing", "product", "shop"],
                    default="listing",
-                   help="listing (default) or product. product reads one "
-                        "/product/ page and adds brand, EAN, description and "
-                        "the full image list — the columns a listing row "
-                        "cannot carry. No --pages in product mode.")
+                   help="listing (default): paginated search results "
+                        "(/search?q=...), a category listing (/c/...) or a "
+                        "/market/ page — around 64 listings per page. "
+                        "product: one /listing/<id> page, which adds the "
+                        "material, shipping origin, description and the full "
+                        "image list, and whose rating is the LISTING's rather "
+                        "than the shop's. shop: one /shop/<name> front — that "
+                        "seller's catalogue as listing rows, with the shop's "
+                        "own rating and location in the sidecar. --pages "
+                        "applies to listing and shop; there is one page to "
+                        "read in product mode.")
     p.add_argument("--category", default=None, help="Label to tag output rows with.")
     p.add_argument("--pages", type=int, default=1, help="Listing pages to crawl")
     p.add_argument("--delay", type=float, default=2.0, help="Delay between pages, seconds")

smoke_test.py

@@ -3583,8 +3583,8 @@ def test_no_capture_leaks():
                 "(%d fixtures, %d chars)" % (len(names), len(fixtures)),
                 len(names) >= 8 and len(fixtures) > 50000)
     # Guarded with PATTERNS rather than with the literals a previous capture
-    # happened to contain, so the NEXT capture is checked too. MediaMarkt's
-    # pages embed a front-end configuration blob — a Sentry DSN, a Woosmap
+    # happened to contain, so the NEXT capture is checked too. On a sibling
+    # site (mediamarkt-scraper) the pages embed a front-end configuration blob — a Sentry DSN, a Woosmap
     # public key, a store-code JWT — none of which is needed to test a
     # parser, and none of which belongs in a public repository.
     patterns = {