Phase R — Statistical robustness (Wilson CIs + SPRT)

[aberson]

Umbrella: #211 (Track 10 — Statistical robustness)
Track: 10 — Statistical robustness
Stack: Utility
Status: Investigation-blocked
Blocked by: #163 (Phase 9 — gates exist); Phase 8 (throughput unlock — already merged); #204 (soak-volume statistical robustness investigation)
Plan: documentation/plans/alpha4gate-master-plan.md §"## Phase R — Statistical robustness (Wilson CIs + SPRT)"

Goal

Replace raw-count gate thresholds with Wilson 95% lower bound. Add SPRT early-stop in run_fitness_eval and run_regression_eval.

Scope summary

Wilson interval helper + SPRT helper + gate-threshold contract update + dashboard schema bump (useEvolveRun.ts cacheKey + new fields).

Tests

tests/test_stats_primitives.py, tests/test_evolve_gates_with_wilson.py.

Validation

Retrofit on past 5 evolve runs shows decision-flip rate ≤20% AND wall-clock saved ≥15% per generation. (Numbers come from investigation #204 §5.)

Gate

Both retrofit criteria.

Effort

~1-2 days.

Kill criterion

Retrofit shows decision-flip rate >30% OR wall-clock saved <5%; investigation under-promised; revisit primitive choice.

Rollback

Revert gate-threshold contract; primitives stay as helpers, unused.

Phase S+ backlog

Build-order entropy metric, time-to-win distribution, per-difficulty WR breakdowns at high n, Bayesian gates with beta priors. Each is its own future phase, scoped by investigation §6 backlog.

---

Synced from master plan by /repo-sync