Skip to content

Integrate CarMax PR #57 with reviewed fixes and harder tasks - #129

Merged
QianhuiWu merged 10 commits into
mainfrom
fix/pr57-carmax
Sep 17, 2026
Merged

QianhuiWu merged 10 commits into
mainfrom
fix/pr57-carmax

Conversation

@QianhuiWu

Copy link
Copy Markdown
Collaborator

Summary

Follow-up integration PR for #57, retaining the CarMax contribution from #24 and
the review commit from #57, followed by the reviewed fixes. Please use a regular
merge, not squash or rebase, to preserve the original authors and dependency history.

  • Restore make → model → year value browsing; fix pagination and mobile layouts.
  • Replace LLM-dependent graders with offline navigation, answer-field/unit/date,
    and exact saved-database delta checks. Missing snapshots fail closed.
  • Coordinate task wording and answer-free judge rubrics with the verifiers.
    Expand tasks 12, 16 and 19 into account-specific multi-page research and
    comparison workflows: 5 / 5 / 4 → 12 / 12 / 12 recorded steps.
  • Append CarMax at index 37 / port 40037 without moving existing sites; reconcile
    registries, Docker EXPOSE and documentation with main.
  • Pin CarMax's original immutable HF bundle independently; use explicit missing
    image fallback and disclose representative stock imagery. Remove accidentally
    tracked runtime databases, retaining the original seed.

History and attribution

Both original PR heads are actual ancestors of this branch, not copied or
squashed replacements. The separate closed #56 had already been superseded by
#57; no new history-only reconciliation was added.

Validation

  • 20/20 selected browser trajectories pass the official offline grading entrypoint.
  • 44/44 harder-task grading controls match expectations: 41 negatives rejected,
    three equivalent-format positives accepted. Previous functional-fix controls:
    37 targeted outcomes plus 60 no-op/wrong-answer/navigation controls matched.
  • 16 unit/route tests pass; Python syntax, diff whitespace and 38-site registry
    checks pass.
  • Organized dashboard: 20 GIFs, 232 recorded steps = 192 task actions + 40
    viewport checks. Revised GIFs, desktop/mobile layouts, filters and deep links
    verified; original evidence retained separately.
  • Full build at commit 2a40f17 using the freshly
    downloaded CarMax pin: PASS. Image webharbor:pr57-integration-20260917,
    sha256:4a75378e6d247d03d139fb3191975b6b11e37fc16ca4f77ffb5b6bdcc6cbda71.
  • All 38 site homepages return 200; all 38 services are healthy/ready.
    CarMax reset after a deliberate runtime mutation restores byte-identical seed
    contents; restart and reset-all pass. The disposable container was stopped.
  • Tasks 12, 16 and 19 replayed again against that clean-pin Docker image:
    all complete in 12 steps and pass official grading, with no browser errors
    or 390px document overflow. All 78 tracked CarMax runtime files match the
    candidate checkout.

The recordings are scripted UI regressions, not independent LLM-agent discovery.
No secondary LLM judge was run because no API key was configured.

Asset and release limitations

The merge candidate uses the freshly downloaded original CarMax asset pin
189b9cdcfd41a21b99ce4797f3bec152d8564a7f. Its seed is unchanged. The 11 newly
recovered photos are a local supplement, documented in recovered_assets.json;
they are not published by this code PR. A fresh checkout uses explicit
placeholders for those photos until a separate HF bundle publication and pin
update. Seven other model-year photos remain unavailable/unsuitable. The fuller
local review preview therefore has more photos than the pinned-bundle build.

This PR merges source only. It does not publish a Docker image or deploy a service.

Violet24K and others added 10 commits May 14, 2026 22:28
…com. - 13 SQLAlchemy models (User / Store / Vehicle / SavedVehicle / Comparison + ComparisonItem / Reservation / TestDrive / Appraisal / FinancePreQual / Order / Review / Article) - 59 routes covering search / browse / detail / research / compare / saved / sell-my-car / pre-qual / reserve / test-drive / checkout / account / articles / FAQ / MaxCare / stores / auth - Token-overlap scored search with multi-field weighting - 141 deterministically-seeded vehicles across 31 templates - 12 real CarMax store locations - 5 benchmark users with pre-populated saved/reservation/test-drive/ appraisal/order data - 20 WebVoyager tasks in tasks.jsonl (6 Easy / 9 Medium / 5 Hard, including 2 disambiguation tasks) - Idempotent seed at function level; byte-identical reset verified
Per-task deterministic verifiers (verify_<N>.py) + shared verify_lib.py under
sites/carmax/verify/, plus verifier_path + judge_rubric on each task in tasks.jsonl.
Each verifier checks navigation (anti knowledge-shortcut), DB after-state for stateful
tasks, and the answer against frozen ground truth. Validated against the react agent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@QianhuiWu
QianhuiWu merged commit 2206530 into main Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants