Integrate CarMax PR #57 with reviewed fixes and harder tasks - #129
Merged
Merged
Conversation
…com. - 13 SQLAlchemy models (User / Store / Vehicle / SavedVehicle / Comparison + ComparisonItem / Reservation / TestDrive / Appraisal / FinancePreQual / Order / Review / Article) - 59 routes covering search / browse / detail / research / compare / saved / sell-my-car / pre-qual / reserve / test-drive / checkout / account / articles / FAQ / MaxCare / stores / auth - Token-overlap scored search with multi-field weighting - 141 deterministically-seeded vehicles across 31 templates - 12 real CarMax store locations - 5 benchmark users with pre-populated saved/reservation/test-drive/ appraisal/order data - 20 WebVoyager tasks in tasks.jsonl (6 Easy / 9 Medium / 5 Hard, including 2 disambiguation tasks) - Idempotent seed at function level; byte-identical reset verified
Per-task deterministic verifiers (verify_<N>.py) + shared verify_lib.py under sites/carmax/verify/, plus verifier_path + judge_rubric on each task in tasks.jsonl. Each verifier checks navigation (anti knowledge-shortcut), DB after-state for stateful tasks, and the answer against frozen ground truth. Validated against the react agent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up integration PR for #57, retaining the CarMax contribution from #24 and
the review commit from #57, followed by the reviewed fixes. Please use a regular
merge, not squash or rebase, to preserve the original authors and dependency history.
and exact saved-database delta checks. Missing snapshots fail closed.
Expand tasks 12, 16 and 19 into account-specific multi-page research and
comparison workflows: 5 / 5 / 4 → 12 / 12 / 12 recorded steps.
registries, Docker EXPOSE and documentation with main.
image fallback and disclose representative stock imagery. Remove accidentally
tracked runtime databases, retaining the original seed.
History and attribution
4220681.
701247f.
and harder-task contracts 2a40f17.
Both original PR heads are actual ancestors of this branch, not copied or
squashed replacements. The separate closed #56 had already been superseded by
#57; no new history-only reconciliation was added.
Validation
three equivalent-format positives accepted. Previous functional-fix controls:
37 targeted outcomes plus 60 no-op/wrong-answer/navigation controls matched.
checks pass.
viewport checks. Revised GIFs, desktop/mobile layouts, filters and deep links
verified; original evidence retained separately.
downloaded CarMax pin: PASS. Image webharbor:pr57-integration-20260917,
sha256:4a75378e6d247d03d139fb3191975b6b11e37fc16ca4f77ffb5b6bdcc6cbda71.
CarMax reset after a deliberate runtime mutation restores byte-identical seed
contents; restart and reset-all pass. The disposable container was stopped.
all complete in 12 steps and pass official grading, with no browser errors
or 390px document overflow. All 78 tracked CarMax runtime files match the
candidate checkout.
The recordings are scripted UI regressions, not independent LLM-agent discovery.
No secondary LLM judge was run because no API key was configured.
Asset and release limitations
The merge candidate uses the freshly downloaded original CarMax asset pin
189b9cdcfd41a21b99ce4797f3bec152d8564a7f. Its seed is unchanged. The 11 newly
recovered photos are a local supplement, documented in recovered_assets.json;
they are not published by this code PR. A fresh checkout uses explicit
placeholders for those photos until a separate HF bundle publication and pin
update. Seven other model-year photos remain unavailable/unsuitable. The fuller
local review preview therefore has more photos than the pinned-bundle build.
This PR merges source only. It does not publish a Docker image or deploy a service.