v3 Slice 7 (HITL): Freeze canonical Benchpress-Causal v1 + contamination audit

[mark-allwyn]

Parent

#13

What to build

Freeze the first canonical benchmark and stand up the contamination integrity check. HITL: a human signs off on the shipped item set.

• Freeze Benchpress-Causal v1: an immutable manifest recording generator seeds, the kept item IDs, and the benchmark version. Regenerating from the manifest reproduces the kept items exactly.
• Transparency posture enforced: generator code is public; the seed and the full instantiated item set are held out (gitignored); only example items and aggregate results are publishable.
• Contamination audit: a audit command samples a fresh held-out set from the same distribution and reports each model's public-set vs fresh-set score gap; a large gap flags contamination/overfitting.
• Human sign-off on the frozen set before it becomes canonical.

Acceptance criteria

• Manifest committed; regeneration from manifest seeds reproduces the kept items exactly
• Seed and full item set are gitignored; only examples + aggregates are tracked
• audit produces a public-vs-fresh gap report per model
• Human has reviewed and signed off on the frozen set
• pytest green

Blocked by

• v3 Slice 4: Full causal domain - all 20 bundles x 5 variants + remaining scorers #17
• v3 Slice 6: Pareto frontier + over-time tracking + saturation + classical item stats #19