docs(showcase): evidence page with measured, reproducible figures (#76)

CHANGELOG.md

@@ -5,6 +5,11 @@ This file records what changes **in the product** – process and session state
 
 ## [Unreleased]
 
+### Added
+- Evidence page `docs/product/evidence.md`: eval accuracy per key field (reproducible with `pnpm evals`), the
+  measured state of both sample requests, three key decisions and the limits; handling time marked as not yet
+  measured (#76).
+
 ### Changed
 - A unit correction is stored like the extraction stores units: „Stk.“, „Stück“ or „Meter“ become the ERP units
   `pcs` and `m`; other units stay as typed. The correction field shows „Stk.“, and the audit event keeps what

README.md

@@ -53,6 +53,7 @@ Rationale, alternatives and trade-offs: [ADR-0001](docs/decisions/ADR-0001-pilot
 | Architecture map and exceptions register | [docs/technical/architecture.md](docs/technical/architecture.md) |
 | Decisions | [docs/decisions/](docs/decisions/INDEX.md) |
 | Project brief, roadmap | [docs/product/](docs/product/project-brief.md) |
+| Evidence: measured figures (synthetic) | [docs/product/evidence.md](docs/product/evidence.md) |
 | Customer proposal (German) | [docs/product/pilot-vorschlag.md](docs/product/pilot-vorschlag.md) |
 | Discovery and approvals (German) | [PROJECT-START.md](PROJECT-START.md) |
 | Working rules for humans and agents | [AGENTS.md](AGENTS.md) |

docs/product/evidence.md

@@ -0,0 +1,82 @@
+# Evidence – what RequestFlow measurably does
+
+> Issue #76 (epic #19). Figures instead of adjectives. **Everything here is synthetic** – the eval cases,
+> the two sample requests and every value in them. Scope decided by the orchestrator on 2026-09-28:
+> publish what is measurable now; handling time is explicitly **not yet measured**.
+
+## 1. Extraction quality per key field (eval gate)
+
+**What is measured.** 15 synthetic cases (mails, PDFs with tables, scans, missing values, prompt
+injection) in `services/ai/evals/`. Each case carries a recorded model response; the replay runs the
+**real** pipeline on it – normalisers (numbers, units, dates, calendar weeks) and the grounding
+verifier, which marks a value `found` only if its quote is in the cited source segment ([ADR-0001 D8](../decisions/ADR-0001-pilot-architecture.md#d8--ai-and-document-processing--challenged-full-typescript-vs-python-ai-service)).
+The gate fails a PR when a baselined metric moves the wrong way by more than 5 points, an injected value
+is accepted, a case errors or the case set changes.
+
+**What it does not measure.** The recorded responses are hand-written, so the table shows how the
+pipeline and the verifier treat realistic model output – **not** the accuracy of a live model.
+
+Reproduce: `pnpm evals` (replay, no credentials). Run for this page on 2026-10-02 at commit `606caec`
+(merged 2026-09-28) – identical to the baseline `services/ai/evals/baseline.json`, last changed in
+`a2b30c6` (2026-09-24, #50).
+
+| Key field | n | Correct when present (`found_accuracy`) | Missing detected (`missing_recall`) | Claims backed by a quote (`grounding_pass_rate`) | Wrong value marked found (`false_found_rate`) |
+|---|---|---|---|---|---|
+| Company | 15 | 91.7 % | 100 % | 100 % | 8.3 % |
+| Contact person | 15 | 100 % | 100 % | 100 % | 0 % |
+| E-mail | 15 | 90.9 % | 75.0 % | 90.9 % | 0 % |
+| Phone | 15 | 100 % | 85.7 % | 88.9 % | 0 % |
+| Requested delivery date | 15 | 70.0 % | 100 % | 77.8 % | 0 % |
+| Additional requirements | 15 | 100 % | 87.5 % | 100 % | 0 % |
+| Position: description | 29 | 96.5 % | – | 100 % | 0 % |
+| Position: quantity | 29 | 89.7 % | – | 92.9 % | 0 % |
+| Position: unit | 29 | 96.5 % | – | 100 % | 0 % |
+| Position: material | 29 | 89.3 % | 0 % | 96.4 % | 3.9 % |
+| Position: dimensions | 29 | 92.9 % | 100 % | 96.3 % | 0 % |
+
+`n` = size of the set: 15 cases (header fields) or 29 positions (line items). Each rate counts only its own
+observations – accuracy only where the value is in the document (a header field: 10–12 observations;
+material and dimensions: 28 positions), missing detection only where it is absent (`–`: no such position;
+material's 0 % rests on very few). Definitions: `services/ai/README.md` → Evals. One case moves a header
+figure by 8–10 points. Prompt injection: 2 cases, no injected value accepted. Known limit:
+grounding proves where a value comes from, not intent – a model that quotes an injected sentence verbatim
+would pass the verifier (pinned by a test, `services/ai/README.md` → Prompt injection); the person
+reviewing every value with its quote is the second layer.
+
+## 2. The two sample requests on the showcase
+
+Recorded once from the showcase AI service (Gemini `gemini-3.5-flash`, prompt `extract_v2`, free tier
+under the D11 exception) and committed (`src/features/samples/data/*.recording.json`: Werk Ost in `18eab97`,
+Pumpe P204 in `d51ad56`, both 2026-09-27). Counted from those recordings; every `found` value carries a verified quote.
+
+| Sample | Header fields found / uncertain / missing | Positions | What the clerk must decide |
+|---|---|---|---|
+| „Werk Ost“ (in review) | 4 / 1 / 1 | 3, all 15 position values found | delivery date given only as a calendar week (`uncertain`, reason `calendar_week_only`); additional requirements not in the mail (`missing`) |
+| „Pumpe P204“ (exported) | 6 / 0 / 0 | 2, all 10 position values found | nothing – approved and exported once to the ERP mock |
+
+Corrections a clerk actually makes have **not** been measured: there are no real test users in a
+reference project. The table lists what the review screen asks a person to look at.
+
+## 3. Handling time
+
+**Not yet measured.** A protocol (who, which sample, how often, with and without the app) is open;
+until then no time saving is claimed.
+
+## 4. Contribution and key decisions
+
+The orchestrator (Fluory) set scope, priorities and every approval; implementation, tests and reviews
+were done with Claude Code agents under the rules of `AGENTS.md` (issue → branch → draft PR → fresh
+review → human merge). Three decisions carry the product ([ADR-0001](../decisions/ADR-0001-pilot-architecture.md)):
+
+- **A value is `found` only with proof** ([D8](../decisions/ADR-0001-pilot-architecture.md#d8--ai-and-document-processing--challenged-full-typescript-vs-python-ai-service)): the grounding verifier checks the quote in the cited
+  segment; everything else is `uncertain`, `missing` or `unverified` and goes to a person.
+- **Each approved request reaches the ERP exactly once** ([D9](../decisions/ADR-0001-pilot-architecture.md#d9--external-integrations)): idempotency key + unique export row +
+  row lock, proven by integration tests on every PR.
+- **Company data stays in its company** ([D7](../decisions/ADR-0001-pilot-architecture.md#d7--authorization-and-tenant-isolation--challenged-application-level-vs-rls)): every access runs in a tenant transaction with forced
+  row-level security; a guard test fails for any table without it.
+
+## Limits
+
+Synthetic data only · 15 eval cases with hand-written responses · two samples recorded once from a
+free-tier model · no real users, no handling-time measurement · scanned-PDF cases run with OCR off in
+CI (#49).