auto/queue → main: autopilot window 2 (slices 7-10, Phase 3 code)

AUTOPILOT.md

@@ -8,13 +8,13 @@
 
 | Day (ET) | Slice |
 |---|---|
-| Tue Oct 20 (4pm, one-off) | 7: Phase 2 carry-forwards (overtaking call-site test, per-circuit fit, Miami quali fallback) |
+| Tue Oct 20 (afternoon, one-off) | 7: Phase 2 carry-forwards (overtaking call-site test, per-circuit fit, Miami quali fallback) |
 | Wed Oct 21 | 8: season chaining and `gp season` |
 | Fri Oct 23 | 9: season-level backtest harness (built and tested on the fixture, **not run**) |
 | **Sat Oct 24, 8pm** | **US GP (2026 R18) post-quali forecast.** One-off, runs from the `forecast-data` branch and posts to #19 |
 | Sun Oct 25 | 10: post-quali calibration |
 
-Builds run around 9am ET, review around 2pm ET, and the digest around 7pm ET. Tuesday's one-off build gets a one-off 9pm review so Wednesday doesn't halt.
+Builds run around 9am ET, review around 2pm ET, and the digest around 7pm ET. Tuesday's one-off build gets an extra 6:30pm review so Wednesday doesn't halt.
 
 ## Review in this order when you're back
 

docs/autopilot/decisions.md

@@ -148,3 +148,57 @@ Question:    `RaceTrialInputs.dnf_prob` is the fold of both channels, so the res
 Chosen:      Carry the channels as two *optional* fields beside `dnf_prob`, which stays what `simulate_positions` draws against. A caller holding only a combined probability leaves them None and gets the pre-slice-6 flat scaling rather than an invented split. `cli.forecast_weekend` fills them; `backtest.forecast_race_challenger` does not, because it resolves the Grand Prix alone through `simulate_positions` and has no sprint to price.
 Rejected:    Making the channels the primary fields and deriving `dnf_prob` as a computed property. Tidier, and it removes the chance of the fold disagreeing with its parts — but a dataclass property cannot be passed to the constructor, so every existing `RaceTrialInputs(dnf_prob=...)` call site and four test modules would have to be rewritten, for no change in behaviour. Also rejected: a single `incident_share` scalar, which assumes one split for the whole field when the mechanical channel is per team and the incident channel per driver.
 Blast radius: Two fields on `RaceTrialInputs` and two keyword arguments in `cli.forecast_weekend`. The risk worth naming: `dnf_prob` and the channels are now two statements of the same quantity and nothing enforces that they agree — a caller could set channels inconsistent with the fold and the Grand Prix would use one while the sprint used the other. `tests/test_cli.py::test_both_channels_reach_the_resolver_and_fold_to_dnf_prob` asserts the production path keeps them consistent; the dataclass itself does not.
+
+## 2026-10-20 — slice-7
+Question:    The criterion allows either fitting only the forecast round's circuit or caching the full fit per pre-round window. Which?
+Chosen:      Fit only the forecast circuit: `build_overtaking_data(..., event_name=...)`, passed by both call sites through a new `cli._round_event_name`. One fit per forecast, whatever the window holds. Omitting `event_name` still fits the whole window, so the slice-4 tests and any diagnostic caller are unchanged.
+Rejected:    A cache keyed on the pre-round window. Each backtest round has a different window, so a cache only helps when the same round is forecast twice in one process, and it adds invalidation questions (a mutated `driver_rounds` frame) for no gain on the backtest path the carry-forward was about.
+Blast radius: One keyword on `build_overtaking_data`, two call sites. The fitted value for the forecast circuit is identical either way (`tests/test_overtaking_wiring.py::TestSingleCircuitFit` asserts equality with the full fit), so no forecast number moves.
+
+## 2026-10-20 — slice-7
+Question:    How should `collect_v2` flag a quali row whose time came from the session laps (2025 Miami), and which laps count?
+Chosen:      `q_segment = "laps"` on the row plus a WARNING log naming the drivers. Per row: only a row with a `Position` and no Q1/Q2/Q3 time falls back, to that driver's fastest lap with `Deleted` laps excluded. Pole is still the official Q3→Q2→Q1 minimum wherever any official time exists; lap times set pole only when the session has none at all.
+Rejected:    A new `time_source` column. Clearer, but it changes the `driver_rounds.csv` schema (COLUMNS, the validator, every existing file) for one round in nine seasons; `q_segment` has no readers in `src/` or `tests/`, so overloading it is free. Also rejected: triggering only when the whole session lacks times. Simpler, but the criterion is per row, and a partial upstream gap would then still drop drivers silently.
+Blast radius: `_fastest_session_laps` and the fallback block in `collect_v2._build_quali_rows`. Fully-timed sessions are byte-for-byte unchanged (the laps are never read; a test asserts it). Lap-derived times can order near-ties differently from the official classification (NOR/PIA/ANT in Miami sit within 0.002s); `finish_position` still comes from the results, so the grid is unaffected.
+
+## 2026-10-21 — slice-8
+Question:    §3.4 says carry the team random walk forward. It does not say whether the other posterior terms (driver skill, noise scales, both DNF hazards) are redrawn per round or held for the season.
+Chosen:      One posterior draw per trial, held for every remaining round. Only `car` steps between rounds, by the pace model's own `SIGMA_DEV` (doubled for a reset year's first rounds, as in `_step_sigma_scale`). The incident hazard's grid slot is the season-start posterior-mean quali ranking (`cli._hazard_grid`, pre-weekend rule).
+Rejected:    Redrawing the parameters each round. That treats parameter uncertainty as round-to-round noise, so it averages out over a season and makes title odds too confident.
+Blast radius: `sim.season.simulate_season`, `cli.forecast_season`. Nothing in `gp race` or the backtest changes.
+
+## 2026-10-21 — slice-8
+Question:    Which line-up races the remaining rounds, and how are ties on points broken?
+Chosen:      The last completed round's entrants. Drivers who scored earlier but are no longer entered keep their points and score nothing more. Team current points come from the recorded points on R/S rows, so they follow whoever drove the car. Ties go to more Grand Prix wins, then column order.
+Rejected:    An explicit entry list (nothing in the repo maintains one) and the full FIA countback (second places onward). The countback adds a per-position tally for ties that fractional expected points almost never hit.
+Blast radius: `cli.forecast_season`, `sim.season.final_positions`.
+
+## 2026-10-23 — slice-9
+Question:    §7 asks for constructor-share Spearman "comparable to the legacy 0.88 ... match the cutoff". The 0.88 is a leave-one-season-out CV at `EARLY_ROUNDS=2`, so it trains on seasons *after* the one it scores. What legacy number goes next to v2's at each cutoff?
+Chosen:      A rolling legacy number per (season, cutoff): `features.build_features(early_rounds=cutoff)`, then `train.py`'s Ridge pipeline (`SimpleImputer` + `Ridge(alpha=1.0)`) on all `FEATURE_COLS`, trained on seasons strictly before the held-out one. It sees what the v2 fit sees and nothing later. The 0.88 is reported beside it as `legacy_reference_spearman`, never scored. If the legacy inputs (`data/race_results.csv`, `data/constructor_standings.csv`) can't be read, the cutoff has no legacy number and the verdict is null.
+Rejected:    Reproducing the 0.88 method (LOSO). It would let the legacy model learn from 2024-2025 when scoring 2022, which the rolling v2 fit cannot do, so the comparison would favour legacy for a reason that isn't skill. Also rejected: replaying `train.py`'s LassoCV selection, rule-change weight tuning and Ridge-vs-XGBoost pick per fold. More faithful, but it is a nested CV per (season, cutoff), and the shipped pkl picks among those settings with all seasons in view, so the tuned choice leaks too.
+Blast radius: `backtest.legacy_constructor_spearman`, `backtest._legacy_features`. Only the legacy column of the season-level block. Swapping in another legacy method is one function.
+
+## 2026-10-23 — slice-9
+Question:    How are the title Brier and the 80% coverage defined, and how is the held-out season's calendar supplied?
+Chosen:      Title Brier is the multi-class Brier, summed over the field (0 = certain and right, 2 = certain and wrong), not the per-driver mean the race-level `_brier` uses: a per-driver mean shrinks with field size and makes every title forecast look near-perfect. Coverage counts an entity when its real final points lie in `[p10_final_points, p90_final_points]` inclusive, two columns now added to both championship tables. Driver coverage is over the cutoff round's line-up only; a driver who had already left has a fixed total that any interval covers. The calendar is read off the held-out season's own R/S rows through `remaining_calendar`'s loader seam, and the chain is handed only rows up to the cutoff.
+Rejected:    The mean Brier, for the reason above. Also rejected: FastF1's schedule for past seasons. Network-bound, and it would list cancelled or uncollected rounds that have no actual result to score against.
+Blast radius: `backtest.score_season_cutoff`, `backtest.backtest_schedule`, and two added columns on `drivers_championship.csv` / `constructors_championship.csv` (§6 columns unchanged).
+
+## 2026-10-23 — slice-9
+Question:    Does `--season-level` run alongside the race-level backtest or instead of it, and where does its block go?
+Chosen:      Instead of. `gp backtest --season-level` runs only the season harness and writes `season_level` into `backtest_summary.json`. If that file already exists in `--out-dir` (a race-level run pointed at the same dir), the block is merged in and the race-level keys are kept. A cutoff past a season's last round, a failed fit, or a failed chain is a named failed cutoff, and any failed or unattempted cutoff, or a scored cutoff with no legacy number, makes `v2_beats_legacy_spearman` null.
+Rejected:    Running both in one invocation. The race-level grid is the slow part (~2.7 h with `--post-quali`) and the season harness is 12 fits for 2022-2025; coupling them makes the cheap one wait on the expensive one.
+Blast radius: `cli.cmd_backtest` (a new branch, the old path untouched) and `backtest.run_season_backtest`.
+
+## 2026-10-25 — slice-10
+Question:    The queue asks for "an expected calibration error (or similar single number, defined in the PR)" for both arms, but not which bins, which rounds, or what file shape.
+Chosen:      Ten equal-width bins on [0, 1] (p = 1.0 in the top bin), the same 0.1 bins `docs/phase2-status.md` already reads the pre-weekend file with. ECE = n-weighted mean of |mean predicted - observed podium rate| over the non-empty bins, in probability units. The summary gets `calibration_podium.pre` (every scored round), and with `--post-quali` also `postquali` and `pre_paired` (pre restricted to the rounds post-quali scored), so the two ECEs compare the same races. Both CSVs carry `season, round` after the original two columns, which is what makes `pre_paired` computable.
+Rejected:    Equal-count (quantile) bins. They would put different bin edges on each arm and make the two tables impossible to read side by side. Also rejected: comparing post-quali against the all-rounds pre table only, which would let a failed post-quali round change one side and not the other.
+Blast radius: `backtest.calibration_summary`, `_podium_calibration`, `run_backtest`'s summary. `calibration_podium.csv` gains two trailing columns; the existing two keep their names and order. No forecast moves.
+
+## 2026-10-25 — slice-10
+Question:    The queue asks for rounds where post-quali log loss is worse than pre-weekend "by more than a stated threshold", and leaves the threshold to the PR.
+Chosen:      1.0 nats (`POSTQUALI_WORSE_THRESHOLD_NATS`), strictly greater. That is post-quali putting about 2.7x less probability on the actual winner than pre-weekend did. On the Oct 4 run it would include the three rounds `phase2-status.md` names (deltas 2.39, 2.24, 1.57). How many of the other 10 worse rounds clear it is unknown from the sandbox, which has no `backtest_metrics.csv`. The grid-pole driver and the actual winner are read off the race rows (`grid_position == 1`, `finish_position == 1`) and added to every row of `backtest_metrics.csv`.
+Rejected:    Any positive delta. It would list all 13 worse rounds, most of them noise. Also rejected: a threshold relative to the round's pre log loss, which is harder to read and not what the queue asked for.
+Blast radius: One constant and `backtest.postquali_worse_rounds`; listing only, feeds no gate.

docs/autopilot/queue.md

@@ -337,7 +337,26 @@ includes `--grid-from-quali` and the post-quali backtest arm.
 
 ## slice-7: Phase 2 carry-forwards
 
-Status: todo
+Status: merged — PR #20 merged by the review routine 2026-10-20 (`b14a307`).
+
+All four criteria were verified against the diff rather than against the PR
+body. CI (`test`) was green on `3bd3b22`; the diff is 403 lines, inside the
+600-line review cap. The review re-ran the full suite on the PR head
+independently (Python 3.13, pymc 6.3.2, arviz 1.3.0): 297 passed. Five of the
+PR's seven mutations were re-applied independently and each reproduced its
+stated failure count: cli call site 3, backtest call site 3, `event_name`
+ignored 6, no laps fallback 5, deleted laps counted 3. Nothing in the body was
+found overstated.
+
+The backtest has not been run against this slice. `TestSingleCircuitFit`
+asserts that the one-circuit fit equals that circuit's entry in the full fit, so
+no forecast number should move. The Miami fallback is tested only against a
+mocked session. Whether `repair_round.py 2025 6 Q` now matches the hand-repaired
+rows is the owner's check (review pack step 4).
+
+Carried forward, not blocking: `q_segment="laps"` overloads a column that
+currently has no readers. A future reader that treats `q_segment` as Q1/Q2/Q3
+has to handle the flag (decision 2, 2026-10-20).
 
 Spec: `docs/phase2-status.md` "What is still open"; slice-4 notes above.
 
@@ -360,7 +379,24 @@ Acceptance:
 
 ## slice-8: season chaining and `gp season`
 
-Status: todo
+Status: merged — PR #21 merged by the review routine 2026-10-21 (`e69f8c3`).
+
+All six criteria were verified against the diff rather than against the PR
+body. CI (`test`) was green on `503aded`; the diff is 594 lines, inside the
+600-line review cap, with the CLI wiring included. The review re-ran the full
+suite on the PR head independently (Python 3.11, fresh venv): 309 passed.
+Three mutations were applied independently: stepping from the fitted car
+instead of the running one (1 failed), dropping one driver per team from the
+constructor sum (2 failed), and a loader that returns nothing (5 failed).
+
+The backtest has not been run against this slice, and the real
+`fastf1.get_event_schedule` path is untested (every test stubs the loader).
+Review pack step 4 (`gp season` on a real fit, check `current_points` against
+the real 2026 standings) is the owner's check.
+
+Carried forward, not blocking: `final_positions` orders on
+`points * 100 + wins`, which assumes points come in steps of at least 0.5 and
+wins stay under 50. True for every current points table.
 
 Spec: `docs/grid-prophet-v2-spec.md` §3.4, §6 (`gp season`, the two
 championship CSVs), §8 Phase 3.
@@ -387,7 +423,25 @@ Acceptance:
 
 ## slice-9: season-level backtest harness
 
-Status: todo
+Status: merged — PR #22 merged by the review routine 2026-10-23 (`9bef025`).
+
+All four criteria were verified against the diff rather than against the PR
+body. CI (`test`) was green on `28615d9`; the diff is 556 lines, inside the
+600-line review cap. The review re-ran the full suite on the PR head
+independently (Python 3.11, fresh venv): 326 passed. Five mutations were
+applied independently, and each one failed a test: the chain is handed the full
+`driver_rounds` (a leak), failed cutoffs no longer void the gate, the title
+Brier is a mean instead of a sum, the legacy model trains on every other season
+(LOSO), and the CLI ignores `--cutoffs`.
+
+**The real season-level backtest has not been run.** That is the Phase 3 gate,
+and only the owner runs it (PR #22, review pack step 3). The legacy column is a
+rolling Ridge on earlier seasons, not the 0.88 LOSO method (decision
+2026-10-23), so expect it to come in below 0.88.
+
+Carried forward, not blocking: the fixture only has 2024 R1-R2, so the
+end-to-end test runs at cutoff 1. The default 2/6/12 cutoffs have only been
+exercised as `no_rounds_after_cutoff` failures.
 
 Spec: §7 season-level metrics, §8 Phase 3, §4 (legacy benchmark).
 
@@ -410,7 +464,7 @@ Acceptance:
 
 ## slice-10: post-quali calibration
 
-Status: todo
+Status: merged — PR #23, squash-merged into `auto/queue` as c9ae98b by the review routine 2026-10-25.
 
 Spec: `docs/phase2-status.md` caveat 1 ("Calibration was not re-checked for
 post-quali").