Skip to content

slice-7: Phase 2 carry-forwards (overtaking call-site tests, per-circuit fit, Miami quali fallback) - #20

Merged
Meerav29 merged 2 commits into
auto/queuefrom
auto/grid-prophet-slice-7-carry-forwards
Oct 20, 2026
Merged

Meerav29 merged 2 commits into
auto/queuefrom
auto/grid-prophet-slice-7-carry-forwards

Conversation

@Meerav29

Copy link
Copy Markdown
Owner

Tracking: #19 · Slice 7 of docs/autopilot/queue.md · base auto/queue (2df87bb)

What this slice does

Three carry-forwards from docs/phase2-status.md ("What is still open") and the slice-4 review notes:

  1. Both overtaking-fit call sites are now test-protected. tests/test_overtaking_wiring.py builds a fixture where the fitted Bahrain value (2.116) is well away from the 4.5 hand rating. It then runs cli.forecast_weekend and backtest.forecast_race_challenger end to end and checks that the resolver gets the fitted value. If either call site stops building the fit, the forecast falls back to 4.5 and the test fails.
  2. Each forecast fits only its own circuit. build_overtaking_data gains an event_name= keyword, and both call sites pass the round's event, via a new cli._round_event_name. A forecast now runs one fit_circuit call whether the window holds 1 circuit or 5, where before it ran one per circuit. With no keyword it still fits the whole window, which is what the slice-4 tests use.
  3. collect_v2 handles quali results that have no times (the 2025 Miami case). A quali row with a Position and empty Q1/Q2/Q3 now takes the driver's fastest session lap, with deleted laps excluded. The row is flagged q_segment="laps" and a WARNING log names the drivers. Sessions that do have times work exactly as before: the laps are never read and the pole calculation is unchanged.

Size: 8 files, +388 / −15 (403 lines including this PR's queue and decisions edits).

Review pack

About 15 minutes.

  1. Read src/model/overtaking.py (+8 lines) and the two call-site hunks in src/cli.py / src/backtest.py. Each is one keyword argument.
  2. Read the fallback block in src/collect_v2.py::_build_quali_rows and _fastest_session_laps. The one thing to judge is whether a per-row fallback with deleted laps excluded is the behaviour you want; see decision 2.
  3. Run:
    pip install -r requirements.txt pytest
    PYTHONPATH=src pytest tests/test_overtaking_wiring.py tests/test_collect_quali_fallback.py -q   # 18 passed
    PYTHONPATH=src pytest tests -q                                                                  # 297 passed
  4. Optional, when you're back at the real data (I could not check this here because driver_rounds.csv isn't in the sandbox): PYTHONPATH=src python src/repair_round.py 2025 6 Q should now return Miami rows with q_segment == "laps" and times close to your hand repair (NOR/PIA/ANT within 0.002s). Compare them against your hand-repaired rows before replacing those.

Acceptance criteria

All of these ran in this sandbox (Python 3.11, arviz 0.23.4, pymc 5.28.5). Baseline on auto/queue: 279 passed. With this branch: 297 passed (18 new tests).

  • A test fails if build_overtaking_data is removed from either production call site. TestBothCallSitesFitTheirOwnCircuit is parametrized over both call sites. Mutation results on the full suite:
    • M1: cli.forecast_weekend never builds the fit → 3 failed, 294 passed
    • M2: backtest.forecast_race_challenger passes None → 3 failed, 294 passed
  • The number of circuits fitted per forecast call no longer scales with the window. test_one_circuit_is_fitted_however_many_the_window_holds checks for exactly ["Bahrain Grand Prix"] with 5 circuits in the window. test_the_fit_count_does_not_grow_with_the_window checks for [1, 1] with 1 vs 5 circuits. Both run at both call sites.
    • M3: drop event_name= in cli → 2 failed
    • M4: drop event_name= in backtest → 2 failed
    • M5: build_overtaking_data ignores event_name → 6 failed
    • The single-circuit fit equals that circuit's entry in the full fit (TestSingleCircuitFit), so no forecast number moves.
  • collect_v2 falls back to session laps for a classified row with empty Q1/Q2/Q3, flags it, and is tested with a mocked FastF1 session and no network. tests/test_collect_quali_fallback.py monkeypatches collect_v2.fastf1.get_session with a stub session. It checks: fastest lap with deleted laps excluded, q_segment="laps" and classified=True, gap to the lap-derived pole, positions kept from the results, a WARNING naming the drivers, a driver with only a deleted lap staying untimed, fully-timed sessions never reading laps, a partial gap filling only the missing row while keeping the official pole, and unclassified rows left alone.
    • M6: no fallback → 5 failed
    • M7: deleted laps counted → 3 failed
  • Existing tests still pass, and new behaviour has tests that fail without it. 297 passed. All 7 mutations above were reverted after each run.

A sandbox note so nobody is surprised: the first baseline run here showed 9 failures. All of them were ImportError: cannot import name '_imaging' from 'PIL' from the container's Debian Pillow. After pip install --ignore-installed pillow the baseline was 279 passed. No repo change was involved, and CI is the authoritative gate.

Decisions made without you

Both are also recorded in docs/autopilot/decisions.md (2026-10-20, slice-7).

  1. Fit only the forecast circuit instead of caching the full fit. Every backtest round has a different pre-round window, so a cache would only help when the same round is forecast twice in one process. It would also add invalidation questions. Rejected: a cache keyed on the window.
  2. Flag fallback rows with q_segment="laps" and decide per row. q_segment has no readers in src/ or tests/, so reusing it avoids a schema change to driver_rounds.csv. The fallback is per row (the criterion's wording), with deleted laps excluded. Official pole is kept whenever any official time exists.
    • Rejected: a new time_source column, which changes the schema and the validator for one round in nine seasons.
    • Rejected: triggering only when the whole session has no times, which would still silently drop drivers in a partial gap.

Deliberately not included

  • The backtest was not run. No forecast numbers should move, because the single-circuit fit equals the full fit for that circuit, but only the backtest can confirm that.
  • repair_round.py and the hand-repaired Miami rows in your local driver_rounds.csv are untouched. Re-collecting Miami is your call.
  • The other open items in phase2-status (ablation, post-quali calibration, which is slice 10, and a re-run with slice 6).
  • confidence.py and the pkl are untouched.

How to revert

git revert <merge commit> on auto/queue. To drop part of it: removing the event_name= keyword from the two call sites restores full-window fitting. Deleting the if needs_laps: block in collect_v2._build_quali_rows restores the pre-slice collector.


Generated by Claude Code

…uit fit, Miami quali fallback)

- build_overtaking_data(event_name=...) fits only the forecast circuit;
  cli.forecast_weekend and backtest.forecast_race_challenger pass it.
- tests/test_overtaking_wiring.py fails if either call site stops building
  the fit, or fits more than one circuit per forecast.
- collect_v2 falls back to the fastest non-deleted session lap when a
  classified quali row has no Q1/Q2/Q3 time; flagged q_segment='laps' and logged.

Refs #19

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EiJgo5GgDN14Q7LndX14Qv
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EiJgo5GgDN14Q7LndX14Qv
@Meerav29
Meerav29 merged commit b14a307 into auto/queue Oct 20, 2026
1 check passed
Meerav29 pushed a commit that referenced this pull request Oct 20, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LnhCkGvJgZEMRYLWEWTFA1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants