docs: sync README/DESIGN.md against actual repo state

README.md

@@ -40,12 +40,13 @@ python webapp/app.py
 # open http://127.0.0.1:8000
 ```
 
-Pick a researched movie (Sixth Sense, Fight Club, Get Out, Parasite, The
-Prestige, Se7en, Arrival, or Gone Girl), read the spoiler-free entry, and
-when you're ready, open the curtain. The other 12 titles in the catalogue
-show a live TMDB poster/synopsis instead (see [Browse tier](#browse-tier-tmdb)) — no
-API key needed for any of this, `TMDB_READ_ACCESS_TOKEN` just upgrades
-those placeholders to real posters.
+Pick a researched movie — 23 now (Sixth Sense, Fight Club, Get Out,
+Parasite, The Prestige, Se7en, Arrival, Gone Girl, and 15 more), read the
+spoiler-free entry, and when you're ready, open the curtain. The two
+remaining measurement titles show a live TMDB poster/synopsis instead
+(see [Browse tier](#browse-tier-tmdb)) — no API key needed for any of
+this, `TMDB_READ_ACCESS_TOKEN` just upgrades those placeholders to real
+posters.
 
 ## What makes this different from "another movie CRUD"
 
@@ -72,8 +73,9 @@ those placeholders to real posters.
   [Research-assist tool](#research-assist-tool-scaling-the-researched-catalogue))
   drafts a new researched entry from real Wikipedia + TMDB retrieval, but
   every claim still needs a real source URL — a code-level sanitizer strips
-  any citation the model invents, and a human review gate still sits
-  between a draft and `content/researched/`.
+  any citation the model invents, and a human review gate sits between a
+  draft and `content/researched/` for the CLI workflow (one deliberate
+  exception exists — see [Limitations](#limitations)).
 
 ## Stack
 
@@ -98,10 +100,15 @@ retrieval) · MyMemory (free — on-the-fly Spanish translation, cached) ·
 Upstash Redis free REST API (optional — durable comments/movie-requests
 on a redeploy; falls back to local files without it).
 
-**CI:** GitHub Actions runs 18 of the 26 tests on every push (see badge
-above) — the other 8 (`SimilarityJudge`, `TrainedClassifierJudge`) need
-optional `sentence-transformers`/`scikit-learn` and are skipped by design
-(`pytest.importorskip`) when those aren't installed, which CI doesn't.
+**CI:** currently broken on every push (see badge above, and
+[Limitations](#limitations)) — `tests.yml` only installs
+`pydantic`/`pytest`/`pyyaml`, but two newer test files import
+`fastapi`/`groq`/`httpx` directly instead of through
+`pytest.importorskip` the way `test_similarity_judge.py`/
+`test_trained_classifier_judge.py` correctly do, so collection is
+interrupted before any test runs. Locally, with every dependency
+installed except `sentence-transformers`, 31 of the suite's 35 tests
+pass (the rest need that optional package too).
 
 ## How it's built
 
@@ -111,23 +118,23 @@ optional `sentence-transformers`/`scikit-learn` and are skipped by design
 | `webapp/research_assist.py` | Drafts a new researched entry from real Wikipedia + TMDB retrieval (never LLM memory) — see [Research-assist tool](#research-assist-tool-scaling-the-researched-catalogue). |
 | `webapp/prewarm_translations.py` | One-time build step: caches Spanish translations of researched + browse-tier content so the live deploy never calls the translation API on a visitor's request. |
 | `webapp/resolve_tmdb_ids.py` | One-time helper: resolves a `tmdb_id` for every title in `evals/dataset/titles.yaml` so the browse tier can show a poster for all 20. |
-| `content/researched/*.json` | 8 hand-researched entries (with cited sources: Wikipedia, Hollywood Reporter, No Film School…), not generated by an unverified LLM. |
+| `content/researched/*.json` | 23 cited entries: 8 hand-researched (Wikipedia, Hollywood Reporter, No Film School…) plus 15 drafted by `research_assist.py` — most human-reviewed before publishing, but one path isn't (see [Limitations](#limitations)). |
 | `src/preshow/` | Data contracts (Pydantic) for both the researched content and the measurement harness, plus the TMDB/Wikipedia/translation/KV-store clients. |
 | `evals/` | The real experiment: leakage/grounding/richness metrics, calibrated judge, 20-title stratified dataset, external calibration scripts. |
-| `docs/DESIGN.md` | Every non-trivial design decision (D1–D16) with its trade-off, written as it was made. |
+| `docs/DESIGN.md` | Every non-trivial design decision (D1–D17) with its trade-off, written as it was made. |
 
 ## Status
 
 | What | Status |
 |---|---|
-| Twistify app (catalogue, spoiler gate, filters, comments) | ✅ 8/20 entries researched |
-| Browse catalogue (TMDB posters, live search, ES/EN) | ✅ 20/20 have posters, search reaches all of TMDB |
-| Offline evals harness | ✅ 18/18 passing in CI, 26/26 with optional `scikit-learn`/`sentence-transformers` installed |
+| Twistify app (catalogue, spoiler gate, filters, comments) | ✅ 23 entries researched — 18/20 measurement titles plus 5 beyond that set |
+| Browse catalogue (TMDB posters, live search, ES/EN) | ✅ posters for every title in the catalogue, search reaches all of TMDB |
+| Offline evals harness | ⛔ CI currently failing (collection error — see [Limitations](#limitations)); 31/35 tests pass locally, 35/35 with optional `sentence-transformers` too |
 | Spoiler ground truth (20 titles) | ✅ 20/20, LLM-researched with cited sources (never hand-labeled — see [Ground truth, precisely](#ground-truth-precisely) below) |
 | Baseline generator (no retrieval) | ✅ two providers — Anthropic (paid) and Groq (free tier, no card) |
 | Judge calibration (offline + real spoiler reviews) | ⛔ **closed, unsolved** — six judges built and tested against real generator output; none clears the bar to trust a `leakage_rate`. `SubstringJudge` (recall=0.0) stays the default because its failure mode is bounded and known (see [Limitations](#limitations)) |
 | Measure the baseline over the 20 titles | ✅ done — see numbers and caveats below |
-| Research-assist tool (D14) | ✅ drafts a researched entry from Wikipedia + TMDB; tested end-to-end on one title (Citizen Kane) |
+| Research-assist tool (D14) | ✅ drafted 15 of the 23 researched entries so far (batch-promoted, human-reviewed); a separate, un-reviewed auto-publish path also exists — see [Limitations](#limitations) |
 | Retrieval (Wikipedia, GREEN-only corpus) + `--generator retrieval-groq` | ✅ `grounded_fact_rate` 0.0→1.0 confirmed live, all 20/20 titles hand-read (D16) |
 
 ## Ground truth, precisely
@@ -155,7 +162,7 @@ a system that **measures**, instead of promising, three things per entry:
    this one)
 
 ```bash
-python -m pytest tests/ -q                            # 18/18 (26/26 with scikit-learn + sentence-transformers), no network, no API key
+python -m pytest tests/ -q                            # 31/35 (35/35 with sentence-transformers too), no network, no API key -- CI itself is currently broken, see Limitations
 python evals/run_eval.py --generator baseline-groq    # free tier, no card
 python evals/run_eval.py --generator baseline         # or the paid Anthropic version
 ```
@@ -269,11 +276,14 @@ Recall is essentially flat — a model roughly 9x larger doesn't catch
 meaningfully more real spoiler reveals. Precision looks better but the
 CIs still overlap substantially. **Combined with the truncation result
 above, this is a clean answer**: giving the judge more text mattered a
-lot; paying for a bigger model didn't. `llama-3.1-8b-instant` is the
-better default going forward — no documented daily token cap, and it's
-what survived four attempts of testing without a single free-tier
-failure. `LLMJudge`'s ceiling at reasonable free-tier settings looks to
-be around recall≈0.35–0.4.
+lot; paying for a bigger model didn't. `LLMJudge`'s ceiling at
+reasonable free-tier settings looks to be around recall≈0.35–0.4 — a
+conclusion about truncation vs. model size, not about a specific model
+name: Groq decommissioned both `llama-3.3-70b-versatile` and
+`llama-3.1-8b-instant` on 2026-08-18, so these two are cited here as the
+models that actually produced the numbers above, not as what to run
+today (current default: `openai/gpt-oss-120b`, `openai/gpt-oss-20b` for
+the judge).
 
 ### A trained classifier beats every judge above
 
@@ -470,17 +480,32 @@ python webapp/research_assist.py "Citizen Kane" 1941
    `content/researched/` — a human still has to read and promote a draft
    before it's published, same bar as the existing 8 entries.
 
-**Status:** tested end-to-end on one title (Citizen Kane, 3/3 candidates
-succeeded: 4, 4, and 5 grounded claims, correctly picked the 5, zero
-fabricated citations reached the output). Needs `GROQ_API_KEY` (free).
-See [Roadmap](#roadmap) for what's next.
+**Status:** used to draft 15 of the app's 23 researched entries so far,
+batch-promoted from `content/_drafts/` after human review — the same
+gate the original 8 hand-researched entries went through. That review
+caught a real problem, not just typos: 4 of 13 candidates in the first
+batch stated a core/major spoiler directly in the spoiler-free `story`
+field, traced to the script giving one LLM call both the plot text and
+an instruction not to leak it — a genuine partition (a separate call
+that's only ever given non-plot text) fixed 2 of the 4; the other 2
+(Los cronocrímenes, Tetsuo) are still held back, and Tetsuo's own
+Wikipedia overview states its premise-is-the-spoiler directly, which no
+retrieval partition can fix. Needs `GROQ_API_KEY` (free).
+
+**A second, separate path skips that gate entirely.** The live app's
+"+ Suggest a movie" flow auto-publishes a `research_assist.py` draft
+straight to `content/researched/` with no human or AI review step at
+all — a deliberate, disclosed, knowingly-risky decision (see
+[Limitations](#limitations)), not an oversight. See
+[Roadmap](#roadmap) for what's next.
 
 ## Browse tier: TMDB
 
-The 12 titles in the 20-title measurement set that aren't hand-researched
-yet still show a real poster and synopsis instead of an empty
-placeholder, and `/api/search` reaches effectively all of TMDB — this is
-a deliberately separate, lower tier: it never claims to be spoiler-safe
+The 2 titles in the 20-title measurement set that aren't researched yet
+(Los cronocrímenes, Tetsuo: The Iron Man) still show a real poster and
+synopsis instead of an empty placeholder, and `/api/search` reaches
+effectively all of TMDB — this is a deliberately separate, lower tier:
+it never claims to be spoiler-safe
 or cited the way `content/researched/*.json` is (see D10 in
 `docs/DESIGN.md`). Needs `TMDB_READ_ACCESS_TOKEN`; without it, the app
 still runs, it just shows the "not researched yet" placeholder instead of
@@ -497,16 +522,16 @@ root (`src/preshow/env.py`, falls back to the real environment too — no
 
 | Variable | Unlocks | Cost |
 |---|---|---|
-| `TMDB_READ_ACCESS_TOKEN` | Browse-tier posters/search for the 12 not-yet-researched titles (D10) | Free, [themoviedb.org](https://www.themoviedb.org/settings/api) |
+| `TMDB_READ_ACCESS_TOKEN` | Browse-tier posters/search for the 2 not-yet-researched measurement titles (D10) | Free, [themoviedb.org](https://www.themoviedb.org/settings/api) |
 | `GROQ_API_KEY` | `--generator baseline-groq`, `LLMJudge` external calibration, `research_assist.py` | Free tier, no card, [console.groq.com/keys](https://console.groq.com/keys) |
 | `ANTHROPIC_API_KEY` | `--generator baseline` (Claude instead of Groq for the same baseline generator) | Paid — the *only* piece of this project that costs money, and it's opt-in |
 | `UPSTASH_REDIS_REST_URL` + `UPSTASH_REDIS_REST_TOKEN` | Comments/movie-requests survive a redeploy on a free host with an ephemeral filesystem (D11) | Free, no card, [console.upstash.com](https://console.upstash.com) |
 
 Full install, including the optional pieces:
 
 ```bash
-pip install fastapi "uvicorn[standard]" pydantic pyyaml pytest   # core app + harness + tests
-pip install groq                                          # optional: baseline-groq, LLMJudge calibration, research_assist.py
+pip install fastapi "uvicorn[standard]" pydantic pyyaml pytest   # core app + harness
+pip install groq httpx                                    # needed for the full test suite too (see Limitations) — also unlocks baseline-groq, LLMJudge calibration, research_assist.py
 pip install anthropic                                     # optional: paid baseline generator only
 ```
 
@@ -548,14 +573,14 @@ evals/                   the real experiment (measurement track)
   dataset/titles.yaml                      20-title stratified spoiler ground truth
 
 content/
-  researched/*.json        8 hand-researched, cited entries
+  researched/*.json        23 cited entries (8 hand-researched + 15 via research_assist.py)
   _translations/            cached ES translations (committed — see D9)
   _drafts/                    research_assist.py output (gitignored, pre-review)
   _tmdb_cache/                 (gitignored)
   _wikipedia_cache/            research_assist.py's fetched articles (gitignored)
 
-tests/                   offline pytest suite (26 tests; 18 run in CI, 8 need optional scikit-learn/sentence-transformers), no network, no API key
-docs/DESIGN.md            every design decision (D1–D16) with its trade-off
+tests/                   offline pytest suite (35 tests, no network, no API key); needs fastapi/groq/httpx installed too, or collection fails outright — CI doesn't install those, so it currently collects zero tests, see Limitations
+docs/DESIGN.md            every design decision (D1–D17) with its trade-off
 docs/screenshots/          the two screenshots at the top of this README
 .github/workflows/tests.yml   CI: installs core deps, runs pytest on every push/PR
 ```
@@ -578,14 +603,21 @@ and `CLAUDE.md`'s "Next task":
   iteration was closed: it would carry exactly the false-negative risk
   every judge attempt already demonstrated. What comes after this
   milestone is an open choice, not a queued fix.
-- **Research the remaining 12 measurement titles** the same way Gone Girl
-  was (cited sources, no invented facts) — `research_assist.py` is the
-  start of automating this, once its output quality is more consistent
-  than a single Citizen Kane run has proven so far.
-- **Automate the "+ Suggest a movie" pipeline.** It already resolves a
-  `tmdb_id` per suggestion via TMDB autocomplete, but doesn't research or
-  add anything automatically yet — that's exactly what
-  `research_assist.py` is the first step toward.
+- **Research the remaining 2 measurement titles.** Los cronocrímenes is
+  a normal case — `research_assist.py`'s output improved once its plot
+  text was fully partitioned out of the pre-viewing call, just held back
+  pending another review pass. Tetsuo: The Iron Man is not: its own
+  Wikipedia overview states the film's premise-is-the-spoiler directly,
+  so no retrieval partition can produce a synopsis that's both honest
+  and safe — it needs a hand-written entry, the same way the original 8
+  were done, or it stays out of the demo.
+- **"+ Suggest a movie" is now automated, not pending.** It resolves a
+  `tmdb_id` via TMDB autocomplete and auto-publishes a
+  `research_assist.py` draft with no review step — a deliberate,
+  disclosed risk rather than the missing piece it used to be (see
+  [Limitations](#limitations)). Whether to add a review gate back for
+  this specific path is the open question now, not whether to automate
+  it.
 - **Upstash on the live Render deploy** — deliberately deferred (the
   project owner's call, not a blocker): comments/movie-requests on the
   live demo reset on every idle spin-down until an Upstash account is
@@ -630,13 +662,24 @@ own stated goal applied to itself:
 - **The mainstream vs. long-tail hypothesis is unconfirmed.** Both strata
   scored identically in the Milestone 0 run — a judge with ~0 real recall
   can't reveal a gap that might genuinely exist.
-- **`research_assist.py` is tested on one title.** Best-of-3 generation
-  fixed the run-to-run inconsistency seen on that one title (Citizen
-  Kane); it hasn't been run across a larger, varied sample yet, so
-  "consistent output quality" is a hypothesis, not yet a measured claim.
-  It also can't cite Rotten Tomatoes/Metacritic directly (no simple free
-  API for either) — a real, disclosed gap against the 8 hand-researched
-  entries, which do cite those sites directly.
+- **`research_assist.py` has been run across many titles now, not just
+  one** (15 of the 23 researched entries), but it still can't cite Rotten
+  Tomatoes/Metacritic directly (no simple free API for either) — a real,
+  disclosed gap against the 8 hand-researched entries, which do cite
+  those sites directly.
+- **"+ Suggest a movie" auto-publishes with no review step, unlike every
+  other path into `content/researched/`** (see
+  [Research-assist tool](#research-assist-tool-scaling-the-researched-catalogue)
+  above for the leak that review step actually caught). An explicit,
+  disclosed, knowingly-risky decision, not an oversight — see
+  `webapp/app.py`'s module docstring.
+- **CI is currently failing on every push (since 2026-08-19).**
+  `tests.yml` only installs `pydantic`/`pytest`/`pyyaml`, but two newer
+  test files (`test_auto_publish.py`, `test_groq_retry.py`) import
+  `fastapi`/`groq`/`httpx` directly instead of through
+  `pytest.importorskip` — collection is interrupted before a single
+  test runs. Locally, with those installed (but not
+  `sentence-transformers`), 31/35 tests pass.
 - **The live demo's comments/movie-requests reset on idle spin-down**
   (Render free tier wipes the filesystem; Upstash isn't wired in yet —
   see [Roadmap](#roadmap)). Known and accepted, not a bug to chase.