Skip to content

Review: validate IMDb mirror, sourced seed and task grading - #89

Open
jackjin1997 wants to merge 20 commits into
aiming-lab:mainfrom
jackjin1997:review/imdb-pr33
Open

Review: validate IMDb mirror, sourced seed and task grading#89
jackjin1997 wants to merge 20 commits into
aiming-lab:mainfrom
jackjin1997:review/imdb-pr33

Conversation

@jackjin1997

@jackjin1997 jackjin1997 commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

This reviewer branch preserves @hqhq1025's IMDb contribution and commit history from #33, corrects sourced profile/title metadata and interactive flows, adds deterministic grading for 20 tasks, and expands the homepage and featured editorial path from sourced captures. It integrates main at 36004932bdf82afbe36dc14e00f66841eccf9946; IMDb is site 25 at port 40024.

Ready for maintainer review — nothing is merged. Implementation fixed point: code 1c8a1eba19c084e202ecbab5977280701b9068c2. The current head fba88dd6385ee4029cd49851a86f642da6b3d774 is a docs-only commit on top of it (only review-reports/ changes; application, task, verifier, registry, Dockerfile and .assets-revision trees are identical). Assets: open HF PR #57 revision f9ddfd2596229f2610418d57fc88c3051e1056bb; imdb.tar.gz is 39,111,545 bytes, SHA256 0663d30fe90ed2a0659dfe78ecf7ab1718970f565a64c6855a2c6aaadc606513; seed SHA256 9d843c5388ecbc0d5265ac316cdaee915f8b203121b5a5330d207af5fb6206b0. Only imdb.tar.gz changes in that PR; it reports no conflicting files.

What was verified on the fixed candidate

  • Owner accepted the expanded homepage/feature result after comparing it with live IMDb. Chromium at 1440×900 and 390×844: homepage and /feature/featured-today-1 HTTP 200, no horizontal overflow, 17/17 sourced entries with local media, 17/17 detail routes 200.
  • Immutable candidate image with matching code/HF labels and no host mounts: 25/25 sites HTTP 200, 25/25 control-plane ready, byte-identical IMDb reset, reset-all 5.35 s, IMDb suite 307/307. The image was assembled from the verified shared dependency image plus the exact code/HF bytes (host lacked headroom for an unknown-peak full rebuild); the standard source build remains the reproduction path.
  • All 20 task paths replayed from the homepage on that image: 20/20 UI, 20/20 deterministic verifier, 311 steps, 13/13 read-only databases unchanged, 7/7 expected write deltas, 20/20 resets restoring the seed. Guided change-impact regression only — not new independent exploration or a blind review.
  • Formal independent blind review (Claude Code, canonical model claude-opus-5, fresh restricted session, no network) over a hash-verified 826-file packet covering all 20 sealed canonical runs: 20 PASS / 0 FAIL (comment). Reconciliation against the deterministic results: 0 differences, 0 substantive findings; common-PASS spot checks of tasks 9/15/16/17 against frozen DOM and full-table DB diffs confirmed the exact single-row changes.
  • The sealed canonical runs were recorded on earlier reviewed versions (task 17 on r1 ebe92f0/HF 4d5709e; the rest on r2 50bcce5/HF e70f49d) and are disclosed as such, not relabeled. Reuse basis: tasks.jsonl and sites/imdb/verify are byte-identical since the expansion freeze, all ten business tables are logically identical between the execution seed and the current seed (only home_features was added), and the later code changes are confined to homepage/feature/news/session routes.

Retained rulings: Task 19 is graded from displayed amounts (30.1M / 52.0M → 57.9%; raw values would give 57.8% with the same winner). Task 26 completes its registration/queue/sign-out/re-login flow with exactly one new user and two Watchlist rows, but its run-start and seal-time provenance metadata disagree, so it is not claimed as a reliable independent-exploration attestation.

Self-contained report: review-reports/PR-33-IMDB.md at fba88dd with public summaries (formal-blind-review-1c8a1eb-summary.json, reconciliation-1c8a1eb-summary.json, candidate-regression-1c8a1eb-summary.json) and current-candidate screenshots under review-reports/assets/pr33-imdb/.

Maintainer handoff — directly mergeable now: no. The code is MERGEABLE/CLEAN against main, but .assets-revision pins the open HF PR revision rather than a merged asset-main commit. Suggested order: (1) review and merge HF #57 (adds only imdb.tar.gz; asset main has since gained two unrelated archives and the PR reports no conflicts); (2) confirm the merged commit still serves imdb.tar.gz with the SHA256 above; (3) update .assets-revision on this branch to that commit, or confirm the immutable PR revision is acceptable — a pin-only change with identical bytes needs only fetch_assets.sh plus the hash check, not new task runs or blind review; (4) merge this PR (supersedes #33) and close #33 and HF #23 with references so the old seed is not reintroduced. The reviewer will not merge either PR.

Builder and others added 8 commits May 26, 2026 14:48
Adds sites/imdb/ — a Flask mirror of imdb.com built per the
.claude/skills/ six-phase contributor pipeline (clone-website,
design-tasks, evolve-env, harden-env, seed-database).

Catalog (scraped from imdb.com via Playwright, gitignored in
sites/imdb/scraped_data/; shipped as instance_seed/imdb.db +
static/images/ via HF dataset):
  - 392 titles  (Top 250 + Top TV 250 + Most Popular + Box Office)
  - 3796 persons (cast / crew with real bios + headshots)
  - 4280 real posters + headshots
  - 19 canonical IMDb genres

Coverage:
  - 22 routes: homepage / title detail / fullcredits / reviews /
    rate / watchlist toggle / person detail / scored search /
    advanced search (genre / year / rating / sort) /
    4 charts (Top 250 / Top TV / Most Popular / Box Office) /
    genre browse / news / auth (login / register / account)
  - scored token-overlap search (not strict-AND), 18 stop words
  - 4 benchmark users alice.j / bob.c / carol.d / david.k
    @ test.com (password TestPass123!) — pre-seeded with
    4-item watchlists, 3-5 ratings, one written review each
  - 12 seeded news items + 12 featured reviews (high
    helpful-count) across major titles
  - 18 benchmark tasks in tasks.jsonl across 6 functional areas:
    chart browse, title detail, person filmography, advanced
    search, genre browse, user state — including 4 hard tasks
    (>= 5 steps) and 2 disambiguation tasks (bob's SF/fantasy
    watchlist subset; carol's crime film subset)

Data integrity:
  - canonical-URL guard skips 65 IMDb-redirected tt_ids
    (some unknown tt_ids returned the wrong page)
  - garbage filter skips 1854 names that hit IMDb's 403/
    error fallback during concurrent scraping
  - html.unescape on all string fields; hero h1 preferred
    over ld.name (which is sometimes original-language)
  - lowercase substring match for box-office data-testid keys

HF asset PR:
  https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/23
@jackjin1997 jackjin1997 mentioned this pull request Sep 8, 2026
9 tasks
@jackjin1997

Copy link
Copy Markdown
Contributor Author

Independent blind execution review

  • Reviewer model: Claude Opus 5, fresh non-persistent safe/restricted session
  • Frozen candidate: code 1c8a1eba19c084e202ecbab5977280701b9068c2; HF asset revision f9ddfd2596229f2610418d57fc88c3051e1056bb
  • Input integrity: 826/826 files verified, no missing, size, or SHA256 mismatches
  • Coverage: 20/20 task runs reviewed once in manifest order
  • Result: 20 PASS, 0 FAIL

This was an execution-completion judgment only. The blind reviewer did not inspect or certify source fidelity, deterministic verifier correctness, runner independence, repository history, PR discussion, or prior reviews, and did not replay the runs live. The frozen run records predate the latest presentation-only candidate changes; their reuse basis and the current-candidate guided/deterministic validation are tracked separately and were intentionally outside this blind pass.

The structured first-pass result was frozen and hashed before comparison. The Review PR remains Draft; reconciliation and final readiness have not started.

Refresh review-reports/PR-33-IMDB.md to the fixed candidate (code 1c8a1eb,
HF PR aiming-lab#57 revision f9ddfd25): drop the superseded HF pin and Owner-pending
text, record the formal 20-task blind review and reconciliation, the
fixed-candidate engineering and guided impact regression, Owner visual
acceptance, known limitations and the maintainer handoff order.

Add public summaries and current-candidate screenshots under
review-reports/assets/pr33-imdb/ and mark the historical probe summaries
as superseded. Docs-only: no application, task, verifier, seed or asset
bytes change.
@jackjin1997
jackjin1997 marked this pull request as ready for review September 12, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant