From c67f439413595d1866b33f8d73905956e204e41b Mon Sep 17 00:00:00 2001
From: jehrr <79864894+jehrr@users.noreply.github.com>
Date: Wed, 23 Sep 2026 13:57:03 +0000
Subject: [PATCH 1/2] Remove donor-site leftovers (Akamai/headless,
 shop/product modes, euro/IDR, auctions)

- --help: scraper_api_client "auction scraper" description and /p/<cat>
  URL help; diff_runs --price-tolerance-pct "quotes IDR"; playwright
  --pages "infinitely scrolling search".
- playwright --concurrency warning: "refuses a headless browser ... by
  answering NOTHING at all" -> refuses exits outside the UAE.
- Engines, captcha_solver, output_writer: Akamai 394-byte headless refusal,
  --mode product/shop, unaddressable search, /p/<slug> hub prose replaced
  with this repo's measured Imperva/UAE facts or deleted; contradictory
  "HEADFUL by default" comments removed; unused shop_facts field removed.
- diff_runs: euro/18 locales/auction prose -> AED, two languages, Car of
  the Week lifecycle; farfetch-style example filenames -> cars.
- CONTRIBUTING: site-change section rewritten for __NEXT_DATA__ / JSON-LD
  join / DOM fallback; datacentre note corrected; pyppeteer crash
  attributed to the sibling it happened in.
- Issue templates: donor options (reserve price, favorite_count, suggested
  lots, 100 cap, headless refused) and orphan lines removed; bug report
  example prefix/count fixed.
- .gitignore/.dockerignore: dubizzle_listings.* -> dubizzle_products.*.
- smoke_test: CATAWIKI_* env names -> DUBIZZLE_*; EUR comment -> AED.
- README: last badge "runs without an account" -> "needs a UAE exit".
- CHANGELOG: new [Unreleased] section.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 .dockerignore                           |  3 +-
 .github/ISSUE_TEMPLATE/bug_report.yml   |  4 +-
 .github/ISSUE_TEMPLATE/site_changed.yml | 25 ++++----
 .gitignore                              |  4 +-
 CHANGELOG.md                            | 38 ++++++++++++
 CONTRIBUTING.md                         | 72 +++++++++-------------
 README.md                               |  2 +-
 captcha_solver.py                       | 23 ++++---
 diff_runs.py                            | 46 ++++++--------
 output_writer.py                        | 80 +++++++++----------------
 playwright_scraper.py                   | 73 ++++++++--------------
 puppeteer_scraper.py                    | 39 ++++--------
 scraper_api_client.py                   | 20 +++----
 selenium_scraper.py                     | 37 ++++--------
 smoke_test.py                           | 15 ++---
 15 files changed, 209 insertions(+), 272 deletions(-)

diff --git a/.dockerignore b/.dockerignore
index 9b99008..cee7160 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -11,7 +11,8 @@ __pycache__/
 .venv/
 venv/
 env/
-dubizzle_listings.*
+dubizzle_products.*
+dubizzle_products_*.*
 *_debug.html
 *_debug.png
 live_results/
diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml
index 13b6a04..e50286d 100644
--- a/.github/ISSUE_TEMPLATE/bug_report.yml
+++ b/.github/ISSUE_TEMPLATE/bug_report.yml
@@ -92,7 +92,7 @@ body:
       placeholder: |
         python3 playwright_scraper.py \
           --url "https://uae.dubizzle.com/motors/used-cars/" \
-          --pages 3 --out grills
+          --pages 3 --out cars
     validations:
       required: true
 
@@ -112,7 +112,7 @@ body:
     attributes:
       label: What you expected instead
       placeholder: >-
-        96 products, as the README says a category page yields.
+        25 ads a page, as the README says a motors listing yields.
     validations:
       required: true
 
diff --git a/.github/ISSUE_TEMPLATE/site_changed.yml b/.github/ISSUE_TEMPLATE/site_changed.yml
index edd796e..d48cb8b 100644
--- a/.github/ISSUE_TEMPLATE/site_changed.yml
+++ b/.github/ISSUE_TEMPLATE/site_changed.yml
@@ -21,27 +21,24 @@ body:
           instead `brand`, `seller_name` or `in_stock` are empty while
           everything else is fine, it is the page's JSON-LD ItemList or the
           path-keyed join to it, not the payload.
-          not carry it: bids arrive at hydration.
 
-        So "no rows at all" and "rows with no prices" are two different
+        So "no rows at all" and "no brand / seller_name" are two different
         breakages with two different fixes, and `price_source` on the rows
-        tells you which happened: `next_data+dom` is healthy, `next_data`
-        means nothing hydrated.
+        tells you which source each row's price came from: `next_data+jsonld`
+        where the page publishes both, `next_data` on the verticals with no
+        ItemList, `dom` on the fallback path.
 
   - type: dropdown
     id: what_broke
     attributes:
       label: What stopped matching
       options:
-        - No rows at all on a page that visibly has lots (the payload moved)
-        - Rows come out with no prices (the card, or its price class, moved)
+        - No rows at all on a page that visibly has ads (the payload moved)
+        - Rows come out with no prices on motors, property or classified
         - brand / seller_name / in_stock empty (the JSON-LD join moved)
-        - A null price with no reserve_price_set flag (the invariant broke)
-        - favorite_count empty on every row (the card's counter moved)
-        - Pagination stops early, repeats a page, or runs past the 100 cap
+        - Pagination stops early, repeats a page, or runs past the site's totalPages
         - One vertical returns nothing while the others are fine
         - The Arabic pages lost brand / seller_name / in_stock
-        - A no-results search returns 24 suggested lots as if they were real
         - Something else
     validations:
       required: true
@@ -52,8 +49,8 @@ body:
       label: URL
       description: >-
         The exact URL, its locale prefix, and how the browser was reached.
-        Please say whether the run was headful or over --cdp-endpoint: a
-        headless run is refused by the site and is not a parser problem.
+        Please say which country the exit was in: an exit outside the UAE is
+        refused by the site and is not a parser problem.
       placeholder: "https://uae.dubizzle.com/motors/used-cars/ (English, headless Chromium over --cdp-endpoint, exit: UAE residential)"
     validations:
       required: true
@@ -74,8 +71,8 @@ body:
           here (`AED` in one, the digits in
           `[data-testid="listing-price"]`), so both are needed.
         - For a missing-column problem: which vertical, and whether the
-          page has a JSON-LD ItemList at all
-          ("Current bid", "Aktuelles Gebot", ...) and the locale.
+          page has a JSON-LD ItemList at all, and the locale (`/ar/` or
+          none).
       render: text
     validations:
       required: true
diff --git a/.gitignore b/.gitignore
index 3474a3a..4011e13 100644
--- a/.gitignore
+++ b/.gitignore
@@ -15,8 +15,8 @@
 # real captures, verified to parse identically to them and scrubbed;
 # it has to be IN the repo or the suite cannot run on a clean clone.
 !fixtures_generated.json
-dubizzle_listings.*
-dubizzle_listings_*.*
+dubizzle_products.*
+dubizzle_products_*.*
 
 # --- the live-test harness's working directory ---------------------------
 # `--dump-html PATH` writes PATH.page1, PATH.page2, ... — FILES, not a
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c9494fb..f4bb145 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,44 @@ user would notice, the release notes say so first.
   classifier. It now reads `http_code`, falling back to `status` only if that
   is an integer. After the fix, one live call (`--wait-text AED` on the canary's used-cars listing, no `--cdp-url`) answered HTTP 200 (no 422) with an integer upstream status, 200; the body was a 1,156-byte Incapsula interstitial, correctly reported as blocked (exit 3), as the module docstring already says for the API's own exit.
 
+- **Prose and strings inherited from the sites this repo was copied from.**
+  Several comments, docstrings and user-visible strings still described
+  another site's behaviour as if it were dubizzle's. Rewritten to what this
+  repo's own README measures, or deleted where it has no equivalent:
+  - `--help` text: `scraper_api_client.py` called itself an "auction
+    scraper" and described a scrolling grid and a `/p/<cat>/…` URL shape;
+    `diff_runs.py --price-tolerance-pct` said the site quotes IDR; the
+    Playwright engine's `--pages` help described an infinitely scrolling
+    search page.
+  - The Playwright engine's `--concurrency` warning said dubizzle refuses a
+    headless browser by answering nothing at all. It refuses exits outside
+    the UAE.
+  - The engines, `captcha_solver.py` and `output_writer.py` described an
+    Akamai refusal of HEADLESS browsers (a 394-byte "Access Denied"), a
+    `--mode product` / `--mode shop`, a search that cannot be paginated and
+    a `/p/<slug>` discovery hub. None of those exist here: the refusal is
+    Imperva's, triggered by the exit's country, and the only mode is
+    `listing`. A contradictory "HEADFUL by default" comment is removed from
+    all three engines and the suite (the default is, and was, headless).
+  - `diff_runs.py` said the site quotes euro on 18 locales and described
+    auctions and bids. It quotes AED in two languages; what moves without a
+    price change is the Car of the Week slot.
+  - An unused `shop_facts` field is removed from all three engines.
+- `CONTRIBUTING.md`'s "Reporting a site change" described another site's
+  selectors and page kinds; rewritten for the SSR payload, the JSON-LD join
+  and the DOM fallback this repo actually reads. Its note that a datacentre
+  run gets no response at all is corrected to the measured Imperva refusal.
+- The site-change issue template carried another site's options (reserve
+  prices, favourite counts, suggested lots, a 100-page cap) and said a
+  headless run is refused. The bug-report template's example used another
+  repo's output prefix and product count.
+- `.gitignore` / `.dockerignore` ignored `dubizzle_listings.*`, which no
+  engine writes; the default `--out` prefix is `dubizzle_products`.
+- The suite's env-placeholder and proxy-line checks used another repo's
+  variable names; they now use `DUBIZZLE_CDP_ENDPOINT` and `DUBIZZLE_PROXY`.
+- README: the last badge read "runs without an account", which is not true
+  here — a plain run needs a UAE exit. It now says so.
+
 ## [0.1.1] — 2026-09-16
 
 ### Fixed
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 0f0fa6c..7ae8ec8 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -42,38 +42,25 @@ not the check caught it.
 
 dubizzle changing its markup is the normal way this stops working, and it
 has its own issue template. The detail that saves the most time is WHICH
-anchor broke, because on this site there is no structured data on a listing
-page to fall back on — measured zero `application/ld+json`, zero
-`__NEXT_DATA__` and zero Apollo state across six captures — so the DOM is
-not the primary path by preference, it is the only one.
-
-1. **The grid container.** `[data-testid="divSRPContentProducts"]` on a
-   search page, `[data-ssr="productsCategoryL2/L3SSR"]` on a category
-   listing. If one of these moves the run reports 0 rows and exit 4, which
-   is loud.
-2. **The tile marker.** `[data-testid="imgLeg-c"]` on a search page (one per
-   tile), `[data-testid="divProductWrapper"]` inside
-   `a[data-testid="lnkProductContainer"]` on a category listing.
-3. **The reading ORDER inside the tile** — badge, title, price, was-price,
-   rating, sold, shop, location. The field reads rest on it, deliberately,
-   because the classes around each field are build hashes:
-   `<span class="+tnoqZhn89+NHUA43BpiJg==">` is the title today. If dubizzle
-   reorders a tile, `title` and the prices are what break.
-4. **`span.flip`**, the shop name and the shop's city in that order, exactly
-   two per search tile.
-
-The one place structured data does exist is a DETAIL page's
-`window.__cache` Apollo blob, which is where `--mode product` reads the real
-product id, the exact sold count and the review count.
+source broke, because a row here is read from more than one:
+
+1. **The SSR payload** — `<script id="__NEXT_DATA__">`, the Redux action
+   `listings/fetchListingDataForQuery/fulfilled`. Every ad on every vertical,
+   with its ids, url, price, location, attributes and the pagination
+   contract. If this moves the run reports 0 rows and exit 4, which is loud.
+2. **The JSON-LD ItemList**, on motors and property only — the one source
+   for `brand`, `seller_name` and `in_stock`, joined to the payload on the
+   ad's locale-stripped path.
+3. **The DOM fallback**, anchored on the ad-URL pattern (every ad path
+   carries `/{yyyy}/{m}/{d}/`), with the tile's price read from the wrapper
+   around `[data-testid="listing-price"]`. Every class on a tile but
+   `lpv-cards` is a build hash, so nothing is anchored on one.
 
 A third thing can break without any path failing: the **join** between the
-tiles and the structured data. When it breaks, the row count and the prices
-stay healthy while `in_stock` and part of `brand` quietly empty out — so
-every run logs its structured-price confirmation share per page and warns
-below a floor set PER PAGE KIND (search 8%, category 70%, shop 80%; the
-achievable share differs by a factor of eight between them). If you are
-reporting a change, that percentage and the page kind are the numbers to
-include.
+payload and the JSON-LD. When it breaks, the row count and the prices stay
+healthy while `brand`, `seller_name` and `in_stock` quietly empty out. If
+you are reporting a change, say which vertical and which language (`/ar/`
+or none) — the Arabic join is the one that broke before.
 
 `--dump-html PATH` writes the exact bytes the parser was given, on success as
 well as failure, and a run that finds nothing writes a dump and a screenshot
@@ -223,25 +210,24 @@ Most do not — the suite covers the parser, the writers, the captcha classifier
 and the CLI contract against inline fixtures. If yours genuinely needs
 dubizzle.com, say in the PR what you ran, which URL and page kind, from
 which exit, and what you got — including the price and image coverage
-percentages the run prints, and the scroll trace from the sidecar. Note that
-a run from a datacentre address gets NO RESPONSE AT ALL, so "it returned
-nothing" from a VPS is not a finding. Product counts differ by category, by
-URL and by how far the scroll got, so a bare "worked for me" is not
-reproducible.
+percentages the run prints. Note that a run from an exit outside the UAE
+is refused by Imperva — a 403, or a "Pardon Our Interruption" page under
+HTTP 200 — so "it returned nothing" from a non-UAE address is not a
+finding. Ad counts differ by vertical and by URL, so a bare "worked for me"
+is not reproducible.
 
 **Run more than the primary engine.** "Mirror them exactly" is a design rule,
-not a verification: the first live run of the pyppeteer engine crashed on its
-FIRST fetch on a signature mismatch that four separate offline checks and 400
-green assertions had not caught.
+not a verification: in a sibling repo (tokopedia-scraper) the first live run
+of the pyppeteer engine crashed on its FIRST fetch on a signature mismatch
+that four separate offline checks and 400 green assertions had not caught.
 
-Do not add anything that submits the registration form. This project
-deliberately never does, and a captcha token proved valid by creating a real
-account is not a result worth having.
+Do not add anything that submits a form on the site. This project
+deliberately never does.
 
 ## Scope
 
-This repo scrapes **public pages** on dubizzle: category listings, search
-listings and product pages, exactly as an anonymous visitor is served them.
+This repo scrapes **public pages** on dubizzle: category listings on
+`uae.dubizzle.com`, exactly as an anonymous visitor is served them.
 Out of scope: anything behind a login, anything that submits a form, and
 anything that defeats a protection rather than passing it the way an ordinary
 browser does.
diff --git a/README.md b/README.md
index 9a4e704..e3047fc 100644
--- a/README.md
+++ b/README.md
@@ -6,7 +6,7 @@
 [![python](https://img.shields.io/badge/python-3.9%20%7C%203.13-blue)](pyproject.toml)
 [![licence](https://img.shields.io/badge/licence-MIT-green)](LICENSE)
 [![engines](https://img.shields.io/badge/engines-Playwright%20%7C%20Selenium%20%7C%20pyppeteer%20%7C%20CDP-informational)](#engines)
-[![runs without an account](https://img.shields.io/badge/runs%20without-an%20account-brightgreen)](#do-you-need-any-of-the-paid-products)
+[![needs a UAE exit](https://img.shields.io/badge/needs-a%20UAE%20exit-orange)](#do-you-need-any-of-the-paid-products)
 
 Scrapes [dubizzle](https://uae.dubizzle.com) UAE classifieds — cars, property,
 electronics, jobs and community services — to JSON or CSV, with four
diff --git a/captcha_solver.py b/captcha_solver.py
index e38fec4..69edf4e 100644
--- a/captcha_solver.py
+++ b/captcha_solver.py
@@ -710,16 +710,15 @@ class CaptchaUnsolvable(RuntimeError):
 # a JPEG of distorted text and a GET form). Roughly 190 lines of it, and none
 # of it is ported here, because this site has no such page.
 #
-# What this site does instead is refuse a HEADLESS browser. Measured
-# 2026-09-10 from five different addresses, four of them residential: Akamai
-# answers with HTTP 403 and a 394-byte "Access Denied" page carrying a
-# reference id — no form, no image, no widget, nothing for a solver to
-# answer. And the trigger is the CLIENT rather than the address: the very
-# same addresses were served HTTP 200 and the full catalogue by a browser
-# with a real window. So the response to a block here is `--headful` or
-# `--cdp-endpoint`, not a solve and not a better proxy, and
-# product_parser.detect_page_state reports it as "blocked" rather than
-# "challenge" precisely so no solve is attempted and nothing is charged.
+# What this site does instead is refuse an exit outside the UAE. Imperva
+# answers with an HTTP 403 or a "Pardon Our Interruption" page (see the
+# README's table, measured 2026-09-14), and the trigger is the ADDRESS's
+# country rather than the client: a UAE residential exit was served the full
+# catalogue by headless Chromium. So the response to a block here is a UAE
+# exit (`--proxy` with region-ae, or `--cdp-endpoint` with country-ae), not
+# a solve, and product_parser.detect_page_state reports it as "blocked"
+# rather than "challenge" precisely so no solve is attempted and nothing is
+# charged.
 #
 # The reCAPTCHA / hCaptcha / Turnstile machinery above IS kept, and that is a
 # deliberate asymmetry rather than an inconsistency. Detection stays broad
@@ -727,6 +726,6 @@ class CaptchaUnsolvable(RuntimeError):
 # what the address has been doing — a narrow list is how a challenge gets
 # reported as an empty page months later. A solver for a challenge this site
 # has never been observed to serve is dead code; a DETECTOR for one is cheap
-# insurance: a detection that fires on a page whose lots have
+# insurance: a detection that fires on a page whose ads have
 # already rendered guards nothing, which is why the default is
-# `when-blocked` and why it counts lot links before it spends.
+# `when-blocked` and why it counts ad links before it spends.
diff --git a/diff_runs.py b/diff_runs.py
index 8d6d3f4..858a4a3 100644
--- a/diff_runs.py
+++ b/diff_runs.py
@@ -8,15 +8,15 @@
 monitoring and assortment tracking, but that nothing in this repo actually
 computed.
 
-    python3 diff_runs.py --old watches.2026-09-01.json \\
-                          --new watches.2026-09-07.json
+    python3 diff_runs.py --old cars.2026-09-01.json \\
+                          --new cars.2026-09-07.json
 
 Typical use is a scheduled re-run of one of the four scraper engines, kept
 under a dated filename, diffed against the previous one:
 
-    python3 playwright_scraper.py --url "$URL" --out "girls_$(date +%F)"
-    python3 diff_runs.py --old "girls_$(ls -t girls_*.json | sed -n 2p)" \\
-                          --new "girls_$(date +%F).json" --out diff.json
+    python3 playwright_scraper.py --url "$URL" --out "cars_$(date +%F)"
+    python3 diff_runs.py --old "$(ls -t cars_*.json | sed -n 2p)" \\
+                          --new "cars_$(date +%F).json" --out diff.json
 
 Four buckets, each keyed on sku:
 
@@ -46,13 +46,6 @@
 
 from output_writer import UNIQUE_BY_SKU_MODES
 
-# `sold` is tracked alongside the price, and `sold_is_floor` with it, because
-# without the flag a `sold` change is unreadable: a tile's figure is a floor
-# the site rounded down ("100rb+ terjual" = 100_000) while a product page's
-# is exact (207785 for that same product). A monitor watching `sold` alone
-# would report a jump of 107,785 the moment someone diffed a listing run
-# against a product run, and none of it would be a sale.
-#
 # No `price_is_from` / `price_max` here: a dubizzle card prints one amount,
 # not a range. If variant pricing ever appears on a listing page, this is
 # where it goes.
@@ -97,14 +90,13 @@ def _within_tolerance(before: dict, after: dict, changes: dict,
     because the alternative is a comment inventing a reason. A sibling repo
     needs it: that site converts prices for a cross-border visitor and the
     exchange rate ticks between two runs of the same command. NO EQUIVALENT
-    CATAWIKI BEHAVIOUR WAS MEASURED — this site quotes euro to every
-    visitor, on all 18 locales including Japanese and Chinese and from every
-    exit tried, so a run holds no conversion and every cent of a difference
-    is a real move.
-
-    What DOES move here is the auction itself, and a tolerance is the wrong
-    instrument for that: a bid is not a small drift but a jump to whatever
-    the next bidder offered. The `lifecycle` bucket in `diff_products` is
+    DUBIZZLE BEHAVIOUR WAS MEASURED — this site quotes AED to every visitor
+    (see the README's "Languages, hosts and currency"), so a run holds no
+    conversion and every dirham of a difference is a real move.
+
+    What does move here without the price changing is an ad's placement —
+    in and out of the promoted Car of the Week slot — and a tolerance is the
+    wrong instrument for that. The `lifecycle` bucket in `diff_products` is
     the right one, and it is exact rather than approximate.
 
     So the flag stays available and DEFAULTS TO ZERO, which makes it inert
@@ -318,9 +310,9 @@ def _check_comparable(args) -> bool:
     #
     # The sibling repos guard cross-storefront diffs with `source`: eleven
     # country hostnames, so a run of one against another is refused on the
-    # hostname alone. dubizzle is ONE host with ONE currency — 18 locales as
-    # a path prefix, euro on every one of them, measured 2026-09-10 — so
-    # `source` is "dubizzle.com" on both sides and there is no storefront
+    # hostname alone. dubizzle is browsed on ONE host with ONE currency —
+    # two languages as a path prefix (`/ar`), AED on both — so
+    # `source` is the same on both sides and there is no storefront
     # split for it to catch.
     #
     # This check is therefore expected never to fire, and it is kept for one
@@ -345,11 +337,11 @@ def _check_comparable(args) -> bool:
     if len(set(currencies.values())) > 1:
         problems.append(
             f"the two runs quote different currencies ({currencies}). "
-            f"dubizzle quotes euro to every visitor — verified on 18 locales "
-            f"and every exit tried — so this should be impossible: either the "
+            f"dubizzle quotes AED to every visitor, in both languages, "
+            f"so this should be impossible: either the "
             f"site has grown a second currency or one of these runs invented "
             f"one, and either way every row's price is incomparable. "
-            f"`source` cannot catch it: it is 'dubizzle.com' on both sides.")
+            f"`source` cannot catch it: it is the same on both sides.")
 
     if not problems:
         return True
@@ -380,7 +372,7 @@ def parse_args():
                         "rate tick rather than a price change: reported "
                         "separately and ignored by --fail-on-change. Default 0 "
                         "(report every cent), which is what a dubizzle "
-                        "run wants: the site quotes IDR to every visitor, so "
+                        "run wants: the site quotes AED to every visitor, so "
                         "there is no conversion drift to absorb. The flag is "
                         "inherited from this scraper family; set it non-zero "
                         "only with a reason you can state.")
diff --git a/output_writer.py b/output_writer.py
index 1cd98a1..8d442d7 100644
--- a/output_writer.py
+++ b/output_writer.py
@@ -3,24 +3,14 @@
 -----------------
 Shared row models + JSON/CSV writers used by all three scrapers.
 
-Two modes, one row shape
-------------------------
-    --mode listing   a search grid or a category listing -> Product
-    --mode product   one /{shop}/{slug} detail page -> Product, with the
-                     trailing detail-only fields populated
-
-The listing and lot modes yield the SAME class, because here a lot page is not a
-different kind of object from a tile — it is the same product described more
-fully. So there is no second dataclass here (a sibling repo needs one for
-reviews; this one does not), and `diff_runs.py` can compare a listing run
-against a product run on the columns both populate.
-
-There is deliberately no `--mode seller`. A seller's own page is its own
-application shell with its own markup, none of which has been captured or
-measured, and a mode that ships untested would be worse than a mode that is
-absent. `product_parser.shop_metadata` reads the seller's facts off a detail
-page for the sidecar, which is what a run covering one product can honestly
-say about its seller.
+One mode, one row shape
+-----------------------
+    --mode listing   a category listing -> Product
+
+There is deliberately no detail mode. An individual ad's page has not been
+captured or measured, and a mode that ships untested would be worse than a
+mode that is absent. So there is one dataclass here (a sibling repo needs a
+second one for reviews; this one does not).
 
 `Product` keeps the family's first sixteen columns in the family's order,
 with this site's own ones appended after `position`, so a consumer
@@ -359,19 +349,17 @@ def write_csv(rows: Sequence[Any], path: str, row_cls: Type = Product) -> None:
 # search genuinely matched nothing" from "something stood between us and the
 # content". See product_parser.detect_bot_challenge.
 #
-# On this site this code specifically does NOT cover the three ways to get a
-# real page with no products on it: a `/p/<slug>` discovery hub, which
-# answers 200 with banners and carousels and no grid; a search whose query
-# matches nothing ("Oops, produk nggak ditemukan"); and one page past the
-# end of a category listing. All three are EXIT_NO_PRODUCTS — the request
-# was served exactly as asked and simply has no products on it. Reporting
-# any of them as blocked would send a user hunting for a proxy problem that
-# does not exist.
+# On this site this code specifically does NOT cover the two ways to get a
+# real page with no ads on it: a hub page (`/`, `/motors/`), which is
+# category tiles and promo rails with no result grid; and one page past the
+# end of a listing, which answers HTTP 200 with `totalHits: 0`. Both are
+# EXIT_NO_PRODUCTS — the request was served exactly as asked and simply has
+# no ads on it. Reporting either as blocked would send a user hunting for a
+# proxy problem that does not exist.
 #
-# What EXIT_BLOCKED means here is unusually literal: this site refuses a
-# address it has scored NOTHING at all. No status code, no interstitial, no
-# vendor marker — the HTTP/2 stream is reset and the run sees a connection
-# error rather than a page.
+# What EXIT_BLOCKED means here: Imperva refused the exit — an HTTP 403, or
+# the "Pardon Our Interruption" page, which arrives as HTTP 200. Both are
+# recognised by carrying none of the site's own asset hosts (see the README).
 EXIT_BLOCKED = 3
 
 # Exit code for a run that gathered SOME rows and then stopped early — a
@@ -451,22 +439,15 @@ def run_meta(status: str, stop_reason: str, pages_requested: int,
       partial  — rows were gathered, then the run stopped early
       failed   — nothing was gathered at all
 
-    `mode` and `source` are recorded because `mode` is not implied by the
-    repo: the same output prefix can hold a listing run or a product run,
-    and those populate different columns — `sold` is a FLOOR on a listing
-    row and exact on a product row, so diffing one against the other would
-    report every row as changed. diff_runs.py refuses a pair whose modes or
-    sources differ. `source` is `dubizzle.com` on every row of every run
+    `mode` and `source` are recorded because the family's sidecar carries
+    them: in a sibling repo with several modes the output prefix alone does
+    not say which one produced a file, and diff_runs.py refuses a pair whose
+    modes or sources differ. Here `mode` is always "listing". `source` is `dubizzle.com` on every row of every run
     here, since the site has one storefront and one currency; it is kept
     because consumers read these columns by name across the family.
 
-    `extra` carries facts about the run that are not about any single row.
-    `--mode shop` uses it for the SELLER's own name, location, rating and
-    review count: a run covers exactly one shop, so those belong to the run
-    rather than repeated down a column, and the shop's review count (16679
-    on the captured seller) is a different number from its listings' own
-    (827 on one of them) — putting them in one column would make the schema
-    lie.
+    `extra` carries facts about the run that are not about any single row,
+    so they are not repeated down a column.
 
     `pages_failed` lists the pages that did not yield data, by number.
     `pages_completed` alone was enough only while pages were fetched strictly
@@ -537,15 +518,12 @@ def save(rows: Sequence[Any], out_prefix: str, fmt: str,
 # the DATA (a page contributed nothing not already seen, so the listing is
 # over), while the second is a property of a CSS SELECTOR and is therefore
 # the weaker signal — a renamed attribute looks identical to a short
-# catalogue. On this site that ordering is not a preference, it is the only
-# thing that works: the site publishes NO `link[rel=next]` and no numbered
-# anchors anywhere, a CATEGORY listing is addressable by `?page=N`, and a
-# SEARCH is not addressable at all — `?page=2` there returns an empty result
-# set rather than page 2. So "no new products" is the one termination
-# condition available on a search. See page_flow.pagination_is_addressable.
+# catalogue. On this site the listing states its own end: `?page=N` past
+# `totalPages` answers with `totalHits: 0` (see page_flow's pagination notes).
 #
-# "single_page_mode" is complete by construction: --mode product reads one
-# page because one page is all there is.
+# "single_page_mode" is the family's name for a mode that reads one page by
+# construction. This repo has no such mode; the reason is kept in the set so
+# the family's status mapping stays identical.
 COMPLETE_STOP_REASONS = ("completed", "pagination_exhausted", "no_new_products",
                          "single_page_mode")
 
diff --git a/playwright_scraper.py b/playwright_scraper.py
index a9a3bf2..c25c40a 100644
--- a/playwright_scraper.py
+++ b/playwright_scraper.py
@@ -173,11 +173,6 @@ class PageOutcome:
     # still growing when the round budget ran out is partial, and a run that
     # reported it as complete would read as a shrinking catalogue.
     scroll: Optional[dict] = None
-    # In --mode product, the seller's own id/name/slug read off the page's
-    # Apollo cache. Stored as the small dict rather than by keeping the
-    # page's HTML around: a detail page is 285 KB and a scrolled listing
-    # nearly 1 MB.
-    shop_facts: Optional[dict] = None
 
     @property
     def ok(self) -> bool:
@@ -550,11 +545,10 @@ def _connect_remote(pw, args):
     # Tried first when --cdp-endpoint is set; this script's own detect+solve
     # logic still runs as a fallback if the endpoint does not support it.
     # Note it does NOT cover this site's refusal, which is not a challenge:
-    # Akamai answers a headless browser with a 394-byte "Access Denied" that
-    # has nothing to solve on it, and a real window rather than a better
-    # address is the answer. No challenge has ever been observed here; this is
-    # wired
-    # up because one can appear between deploys.
+    # Imperva refuses an exit outside the UAE with nothing to solve on the
+    # page, and a UAE exit rather than a solve is the answer. No challenge
+    # has ever been observed here; this is wired up because one can appear
+    # between deploys.
     try:
         cdp_session = context.new_cdp_session(page)
         cdp_session.send("Captcha.setAutoSolve", {"autoSolve": True, "options": [{"type": "*"}]})
@@ -643,8 +637,9 @@ def handle_captcha_if_present(page, args) -> bool:
     about the variant and the parameters for one are rejected for the other.
 
     NOTE what this cannot help with. This site's refusal carries nothing to
-    solve — a 394-byte "Access Denied" — and it is triggered by the browser
-    being headless rather than by the address, so a 2Captcha key does nothing
+    solve — Imperva's 403 or its "Pardon Our Interruption" page — and it is
+    triggered by the exit's country rather than by the browser, so a
+    2Captcha key does nothing
     about the state a blocked run is most likely to meet, and
     `detect_page_state` reports that as "blocked" rather than "challenge"
     precisely so no solve is attempted or billed. No challenge has ever been
@@ -839,9 +834,9 @@ def _fetch_one_page(session, args, pool, page_num: int, url: str) -> PageOutcome
 
         # No interstitial-settling step here, and its absence is measured
         # rather than an omission. This site has no interstitial to settle: a
-        # refused request gets a 394-byte "Access Denied" with no markup at
-        # all, so there is nothing to wait out and nothing to reclassify. See
-        # page_flow's "There is no block page".
+        # refused request gets Imperva's refusal, which does not clear itself
+        # on the page, so there is nothing to wait out and nothing to
+        # reclassify. See page_flow's notes above RETRY_ON_BLOCKED.
         #
         # The paid path is reached only for state "challenge", which NO
         # capture of this site has ever produced. It is wired up because a
@@ -913,15 +908,15 @@ def _fetch_one_page(session, args, pool, page_num: int, url: str) -> PageOutcome
     if state == "blocked":
         # What a caller needs to know here is that there is nothing to solve.
         # This site does not send a challenge or a captcha to a request it
-        # refuses: Akamai answers with HTTP 403 and a 394-byte "Access
-        # Denied" page carrying a reference id and nothing to solve. So a
+        # refuses: Imperva answers with an HTTP 403 or a "Pardon Our
+        # Interruption" page, and nothing on either to solve. So a
         # 2Captcha key does nothing about the state a blocked run is most
         # likely to meet, and saying what DOES clear it is more use than a
         # captcha hint that would cost money.
         #
-        # The dump is written even when it is empty, because "0 bytes" is
-        # itself the diagnosis on this site and a reader who finds no file at
-        # all cannot tell that from a run that never got here.
+        # The dump is written even when it is empty, because a reader who
+        # finds no file at all cannot tell an empty response from a run that
+        # never got here.
         debug_html = f"{args.out}_page{page_num}_debug.html"
         with open(debug_html, "w", encoding="utf-8") as f:
             f.write(html or "")
@@ -1228,14 +1223,8 @@ def scrape(args) -> int:
     dedupe_key = "sku"
     # Why the loop ended. "completed" means every requested page was fetched;
     # "no_new_products" means the listing itself ran out (also a complete
-    # result). "single_page_mode" is complete by construction — a detail page
-    # has no page 2. Anything else is an early stop, and the run is only a
-    # partial view.
-    # Only --mode product is single-page. A SHOP FRONT paginates exactly like
-    # a category listing — ?page=N, the same tiles — and treating it as
-    # single-page made `--mode shop --pages 2` fetch one page and report
-    # "complete", which is the silent-success failure this family exists to
-    # avoid. Found on the first live shop run.
+    # result). Anything else is an early stop, and the run is only a partial
+    # view. This repo has only --mode listing, which is never single-page.
     stop_reason = "completed"
 
     pool = proxy_pool_from_args(args)
@@ -1258,9 +1247,8 @@ def scrape(args) -> int:
             logger.warning("--concurrency %d with no proxy pool: every worker "
                            "leaves from the SAME address, which is a faster way "
                            "to get that address scored than to gather data. "
-                           "dubizzle already refuses a headless browser whatever the address it has "
-                           "scored by answering NOTHING at all, so an address "
-                           "that works is one worth not burning. Pass "
+                           "dubizzle refuses any exit outside the UAE, so "
+                           "an address that works is one worth not burning. Pass "
                            "--proxy-file to spread the load.", concurrency)
         if pool and pool.rotates_per_page():
             logger.info("--proxy-rotate per-page is redundant under "
@@ -1458,11 +1446,9 @@ def scrape(args) -> int:
 
     # One-per-run context, in the sidecar rather than repeated down a column.
     #
-    # In --mode product that is the seller's own id and name, off the page's
-    # Apollo cache. In --mode listing it is the scroll trace and the page's
-    # own result header: both say how much of the listing this run actually
-    # saw, which is the question a consumer of an infinitely-scrolling site
-    # most needs answered and which no column can carry.
+    # It is the scroll trace and the page's own result header, where the
+    # engine recorded them: both say how much of the listing this run
+    # actually saw, which no column can carry.
     extra = None
     scrolls = {o.page_num: o.scroll for o in outcomes if o.scroll}
     headers = {o.page_num: o.header for o in outcomes if o.header}
@@ -1518,12 +1504,9 @@ def parse_args():
                         "to override it with a label of your own.")
     p.add_argument("--pages", type=int, default=1,
                    help="Number of listing pages to crawl. Applies to "
-                        "--mode listing, which is the only mode. "
-                        "has one page. Note that only a CATEGORY URL has "
-                        "per-page addresses: a search is one infinitely "
-                        "scrolling page, so --pages above 1 there is honoured "
-                        "by scrolling further rather than by fetching more "
-                        "URLs.")
+                        "--mode listing, which is the only mode. Every "
+                        "listing page has its own ?page=N address, and the "
+                        "site's own totalPages caps how many there are.")
     p.add_argument("--delay", type=float, default=2.0, help="Delay between pages, seconds")
     p.add_argument("--concurrency", type=int, default=1, metavar="N",
                    help="Fetch pages through N parallel workers (default 1 — "
@@ -1635,12 +1618,6 @@ def parse_args():
                    help="Save the exact HTML the parser is given, on success as "
                         "well as failure. Useful when the row count is right but "
                         "a column comes back empty — see TROUBLESHOOTING.md.")
-    # HEADFUL by default, a departure from every sibling repo and a measured
-    # one: Akamai refuses a headless browser here whatever the exit -- HTTP
-    # 403 and a 394-byte "Access Denied" from four residential exits and one
-    # datacentre address, against 200 and the full catalogue from those very
-    # same addresses with a real window. A --headless default would be a
-    # scraper whose default cannot fetch the site.
     # HEADLESS is the default, as in the rest of the family, and the
     # measurement behind that is about the EXIT rather than the window: every
     # live run of this repo was headless and was served the full catalogue —
diff --git a/puppeteer_scraper.py b/puppeteer_scraper.py
index 0ff1ebe..6dd413c 100644
--- a/puppeteer_scraper.py
+++ b/puppeteer_scraper.py
@@ -231,10 +231,6 @@ class PageOutcome:
     # whose grid was still growing when the budget ran out is partial, and a
     # run that reported it as complete would read as a shrinking catalogue.
     scroll: Optional[dict] = None
-    # In --mode product, the seller's own id/name/slug from the page's Apollo
-    # cache. Stored as the small dict rather than by keeping the page's HTML
-    # around: a detail page is 285 KB and a scrolled listing nearly 1 MB.
-    shop_facts: Optional[dict] = None
 
     @property
     def ok(self) -> bool:
@@ -615,8 +611,9 @@ def _fetch_one_page(session, args, pool, page_num: int, url: str) -> PageOutcome
 
         # No interstitial-settling step, and its absence is measured rather
         # than an omission: this site has no interstitial. A refused request
-        # gets a 394-byte "Access Denied" with nothing on it to settle, and
-        # what triggers it is the browser being headless, not the address. See page_flow's "There is no block page".
+        # gets Imperva's refusal, with nothing on it to settle, and what
+        # triggers it is the exit's country, not the browser. See page_flow's
+        # notes above RETRY_ON_BLOCKED.
         #
         # The paid path is reached only for state "challenge", which no
         # capture of this site has ever produced. Wired up because a bot
@@ -680,12 +677,12 @@ def _fetch_one_page(session, args, pool, page_num: int, url: str) -> PageOutcome
     outcome.state = state
 
     if state == "blocked":
-        # There is no challenge on this site to solve. Akamai answers a
-        # refused request with HTTP 403 and a 394-byte "Access Denied" that
-        # carries a reference id and nothing else, so a 2Captcha key does not
-        # help and a REAL WINDOW does — not a better address. The dump is written even when empty: "0 bytes" is itself
-        # the diagnosis here, and a reader who finds no file cannot tell that
-        # from a run that never got this far.
+        # There is no challenge on this site to solve. Imperva answers a
+        # refused request with an HTTP 403 or a "Pardon Our Interruption"
+        # page and nothing to solve, so a 2Captcha key does not help and a
+        # UAE exit does. The dump is written even when empty: a reader who
+        # finds no file cannot tell an empty response from a run that never
+        # got this far.
         debug_html = f"{args.out}_page{page_num}_debug.html"
         with open(debug_html, "w", encoding="utf-8") as f:
             f.write(html or "")
@@ -866,11 +863,7 @@ def scrape(args) -> int:
     blocked = False
     # Both modes are one row per product, so `sku` is the key for both.
     dedupe_key = "sku"
-    # Only --mode product is single-page. A SHOP FRONT paginates exactly like
-    # a category listing — ?page=N, the same tiles — and treating it as
-    # single-page made `--mode shop --pages 2` fetch one page and report
-    # "complete", which is the silent-success failure this family exists to
-    # avoid. Found on the first live shop run.
+    # This repo has only --mode listing, which is never single-page.
     stop_reason = "completed"
 
     pool = proxy_pool_from_args(args)
@@ -1037,10 +1030,9 @@ def scrape(args) -> int:
                  if ok_pages else args.url)
 
     # One-per-run context, in the sidecar rather than repeated down a column.
-    # Mirrors the Playwright engine exactly: the seller's own facts in
-    # --mode product, and the scroll trace plus the page's own result header
-    # in --mode listing, because on an infinitely-scrolling site those are
-    # what say how much of the listing the run actually saw.
+    # Mirrors the Playwright engine exactly: the scroll trace and the page's
+    # own result header, where recorded, because those are what say how much
+    # of the listing the run actually saw.
     extra = None
     scrolls = {o.page_num: o.scroll for o in outcomes if o.scroll}
     headers = {o.page_num: o.header for o in outcomes if o.header}
@@ -1152,11 +1144,6 @@ def parse_args():
                         "code at all, so it is the build, not this engine). "
                         "Point it at a Chrome or Chromium of your own — "
                         "Playwright's, if you have it installed.")
-    # HEADFUL by default, a departure from every sibling repo and a measured
-    # one: Akamai refuses a headless browser here whatever the exit -- HTTP
-    # 403 and a 394-byte "Access Denied" from four residential exits and one
-    # datacentre address, against 200 and the full catalogue from those same
-    # addresses with a real window.
     # HEADLESS is the default, as in the rest of the family, and the
     # measurement behind that is about the EXIT rather than the window: every
     # live run of this repo was headless and was served the full catalogue —
diff --git a/scraper_api_client.py b/scraper_api_client.py
index c4ba0b7..62478aa 100644
--- a/scraper_api_client.py
+++ b/scraper_api_client.py
@@ -294,13 +294,13 @@ def _run_once(args, attempt: int = 1, attempts: int = 1) -> int:
 
 def parse_args():
     p = argparse.ArgumentParser(
-        description="dubizzle auction scraper — 2captcha Scraper API edition (no "
-                    "local browser). NOTE: these pages DO need JavaScript, "
-                    "and their grid hydrates only as the page is scrolled, "
-                    "so a single fetch returns about 5 products where a "
-                    "browser engine returns 60. Pass --cdp-url to reach the "
-                    "site at all; see this file's docstring for the measured "
-                    "numbers, and prefer playwright_scraper.py.")
+        description="dubizzle classifieds scraper — 2captcha Scraper API "
+                    "edition (no local browser). NOTE: measured 2026-09-14, "
+                    "the API's own fetcher is outside the UAE and was served "
+                    "Imperva's \"Pardon Our Interruption\" page, with and "
+                    "without --cdp-url; see this file's docstring for the "
+                    "measured numbers, and prefer playwright_scraper.py with "
+                    "a UAE exit.")
     # NOT required: prefer the TWOCAPTCHA_KEY env var. A key passed on the
     # command line is visible to anyone who can run `ps`, and it lands in
     # shell history and in any log that echoes the command line.
@@ -308,10 +308,8 @@ def parse_args():
                    help="2captcha.com API key (sent as a Bearer token). "
                         "Defaults to $TWOCAPTCHA_KEY, which is the safer way to pass it.")
     p.add_argument("--url", default=None,
-                   help="A dubizzle listing URL — a category listing "
-                        "(/p/<cat>/<sub>/<subsub>) is the only kind this path "
-                        "can read at all, since a search grid has no "
-                        "server-rendered container. Required, unless "
+                   help="A dubizzle category listing URL on uae.dubizzle.com, "
+                        "e.g. /motors/used-cars/. Required, unless "
                         "DUBIZZLE_URL is set in the environment or in .env.")
     p.add_argument("--category", default=None, help="Label to tag output rows with. Defaults to the category segment of the URL, so the column is never empty just because the flag was omitted.")
     p.add_argument("--format", choices=["json", "csv", "both"], default="both")
diff --git a/selenium_scraper.py b/selenium_scraper.py
index 44edc85..23b346b 100644
--- a/selenium_scraper.py
+++ b/selenium_scraper.py
@@ -135,9 +135,6 @@ class PageOutcome:
     # What the lazy-load scroll did, and crucially whether it SETTLED. A page
     # whose grid was still growing when the budget ran out is partial.
     scroll: Optional[dict] = None
-    # In --mode product, the seller's own id/name/slug from the page's Apollo
-    # cache.
-    shop_facts: Optional[dict] = None
 
     @property
     def ok(self) -> bool:
@@ -563,8 +560,9 @@ def _fetch_one_page(session, args, pool, page_num: int, url: str) -> PageOutcome
 
         # No interstitial-settling step, and its absence is measured rather
         # than an omission: this site has no interstitial. A refused request
-        # gets a 394-byte "Access Denied" with nothing on it to settle, and
-        # what triggers it is the browser being headless, not the address. See page_flow's "There is no block page".
+        # gets Imperva's refusal, with nothing on it to settle, and what
+        # triggers it is the exit's country, not the browser. See page_flow's
+        # notes above RETRY_ON_BLOCKED.
         #
         # The paid path is reached only for state "challenge", which no
         # capture of this site has ever produced. Wired up because a bot
@@ -627,11 +625,12 @@ def _fetch_one_page(session, args, pool, page_num: int, url: str) -> PageOutcome
     outcome.state = state
 
     if state == "blocked":
-        # There is no challenge on this site to solve. Akamai answers a
-        # refused request with HTTP 403 and a 394-byte "Access Denied" that
-        # carries a reference id and nothing else, so a 2Captcha key does not
-        # help and a REAL WINDOW does — not a better address. The dump is written even when empty: "0 bytes" is itself
-        # the diagnosis here.
+        # There is no challenge on this site to solve. Imperva answers a
+        # refused request with an HTTP 403 or a "Pardon Our Interruption"
+        # page and nothing to solve, so a 2Captcha key does not help and a
+        # UAE exit does. The dump is written even when empty: a reader who
+        # finds no file cannot tell an empty response from a run that never
+        # got this far.
         debug_html = f"{args.out}_page{page_num}_debug.html"
         with open(debug_html, "w", encoding="utf-8") as f:
             f.write(html or "")
@@ -805,11 +804,7 @@ def scrape(args) -> int:
     blocked = False
     # Both modes are one row per product, so `sku` is the key for both.
     dedupe_key = "sku"
-    # Only --mode product is single-page. A SHOP FRONT paginates exactly like
-    # a category listing — ?page=N, the same tiles — and treating it as
-    # single-page made `--mode shop --pages 2` fetch one page and report
-    # "complete", which is the silent-success failure this family exists to
-    # avoid. Found on the first live shop run.
+    # This repo has only --mode listing, which is never single-page.
     stop_reason = "completed"
 
     pool = proxy_pool_from_args(args)
@@ -971,10 +966,9 @@ def scrape(args) -> int:
                  if ok_pages else args.url)
 
     # One-per-run context, in the sidecar rather than repeated down a column.
-    # Mirrors the other two engines exactly: the seller's own facts in
-    # --mode product, and the scroll trace plus the page's own result header
-    # in --mode listing, because on an infinitely-scrolling site those are
-    # what say how much of the listing the run actually saw.
+    # Mirrors the other two engines exactly: the scroll trace and the page's
+    # own result header, where recorded, because those are what say how much
+    # of the listing the run actually saw.
     extra = None
     scrolls = {o.page_num: o.scroll for o in outcomes if o.scroll}
     headers = {o.page_num: o.header for o in outcomes if o.header}
@@ -1096,11 +1090,6 @@ def parse_args():
     p.add_argument("--dump-html", default=None, metavar="PATH",
                    help="Save the exact HTML the parser is given, on success "
                         "as well as failure.")
-    # HEADFUL by default, a departure from every sibling repo and a measured
-    # one: Akamai refuses a headless browser here whatever the exit -- HTTP
-    # 403 and a 394-byte "Access Denied" from four residential exits and one
-    # datacentre address, against 200 and the full catalogue from those same
-    # addresses with a real window.
     # HEADLESS is the default, as in the rest of the family, and the
     # measurement behind that is about the EXIT rather than the window: every
     # live run of this repo was headless and was served the full catalogue —
diff --git a/smoke_test.py b/smoke_test.py
index 7918614..227fda4 100644
--- a/smoke_test.py
+++ b/smoke_test.py
@@ -902,8 +902,8 @@ def test_output_contract():
                 "single_page_mode" in COMPLETE_STOP_REASONS)
 
     # No defaulted currency anywhere: a row that could not establish one says
-    # None rather than claiming EUR, which would be wrong for the four
-    # non-euro country sites.
+    # None rather than claiming AED, even though AED is the only currency
+    # this site has been measured to quote: a default is a guess, not a fact.
     ok &= check("Product defaults currency to None, not a guess",
                 Product().currency is None)
     ok &= check("Product defaults price_source to None",
@@ -1055,7 +1055,7 @@ def test_diff():
     ok &= check("a new product is reported as added",
                 [r["sku"] for r in d["added"]] == ["4"])
     # A price difference that comes with a price_source difference says
-    # something about OUR two snapshots, not about the shop.
+    # something about OUR two snapshots, not about the site.
     ok &= check("a price move with a source change is not 'changed'",
                 not any(c["sku"] == "3" for c in d["changed"]))
     ok &= check("...it is reported separately as source_changed",
@@ -2106,7 +2106,7 @@ def _placeholder_reads_unset(raw):
     are not already set, so writing a temporary .env would be shadowed by
     whatever the suite has already loaded.
     """
-    name = "CATAWIKI_CDP_ENDPOINT"
+    name = "DUBIZZLE_CDP_ENDPOINT"
     saved = os.environ.get(name)
     try:
         os.environ[name] = raw
@@ -2323,7 +2323,7 @@ def test_proxy_pool():
               "SOMELOGIN-zone-custom-region-de:SOMEPASSWORD")
     raised = None
     try:
-        parse_proxy_line(pasted, source="CATAWIKI_PROXY")
+        parse_proxy_line(pasted, source="DUBIZZLE_PROXY")
     except ProxyError as exc:
         raised = str(exc)
     ok &= check("a proxy-list line pasted as a URL is refused, not crashed on",
@@ -2489,11 +2489,6 @@ def test_engines(skips):
                     'choices=["listing"]' in src)
         ok &= check("%s has no unmeasured detail mode" % name,
                     '"product"]' not in src and '"lot"]' not in src)
-        # HEADFUL is the default here, against headless in every sibling: a
-        # headless browser is refused with HTTP 403 from every address tried,
-        # residential included, while a real window is served from the same
-        # ones. A --headless default would be a scraper whose default cannot
-        # fetch the site.
         # HEADLESS is the default, as in the rest of the family: every live
         # run of this repo was headless and was served. What was refused was
         # a non-UAE address, headful and headless alike — the discriminator

From 75cd2b118779530c3ed967c4fa1e5ac0226c1018 Mon Sep 17 00:00:00 2001
From: Petr <petr@2captcha.com>
Date: Wed, 23 Sep 2026 15:04:29 +0000
Subject: [PATCH 2/2] Fix stale family-core lines (SECURITY releases / DataDome
 pointer)

- SECURITY.md said the project has no releases or version tags; it has both
  (gh release list). Supported versions now names the latest release and main,
  in the wording bbb-scraper and foodpanda-scraper already use.
- captcha_solver.py's docstring pointed at a "No DataDome solver" section this
  repo does not have; the line came with the copied core. Removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---
 CHANGELOG.md      | 5 +++++
 SECURITY.md       | 6 +++---
 captcha_solver.py | 2 --
 3 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index f4bb145..3d687ba 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -61,6 +61,11 @@ user would notice, the release notes say so first.
 - README: the last badge read "runs without an account", which is not true
   here — a plain run needs a UAE exit. It now says so.
 
+- `SECURITY.md` said this project has no releases or version tags; it has
+  both. "Supported versions" now names the latest release and `main`.
+- `captcha_solver.py`'s docstring pointed at a "No DataDome solver" section
+  that does not exist in this repo (it came with the copied core). Removed.
+
 ## [0.1.1] — 2026-09-16
 
 ### Fixed
diff --git a/SECURITY.md b/SECURITY.md
index 8775d97..3a8dc95 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -79,9 +79,9 @@ Not because these do not matter, but because they belong somewhere else:
 
 ## Supported versions
 
-`main` only. This project has no releases or version tags; fixes land on `main`
-and you update by pulling. If you are running an old clone, update before
-reporting.
+The latest release and `main`. Fixes land on `main` first and ship in the
+next tagged release (see the Releases page). If you are running an old clone,
+update before reporting.
 
 ## If you have leaked a key
 
diff --git a/captcha_solver.py b/captcha_solver.py
index 69edf4e..0a4fab4 100644
--- a/captcha_solver.py
+++ b/captcha_solver.py
@@ -23,8 +23,6 @@
 purchase, and `page_flow.STATE_POLICY` — not this file — decides which state
 is worth money at all.
 
-There is deliberately no DataDome path here. See "No DataDome solver" below.
-
 Flow:
   1. Both detectors run and are reconciled (see reconcile_detections) to decide
      the variant: v3, v2-invisible or v2-checkbox. The parameters differ per