Review: validate IMDb mirror, sourced seed and task grading - #89
Open
jackjin1997 wants to merge 20 commits into
Open
Review: validate IMDb mirror, sourced seed and task grading#89jackjin1997 wants to merge 20 commits into
jackjin1997 wants to merge 20 commits into
Conversation
Adds sites/imdb/ — a Flask mirror of imdb.com built per the
.claude/skills/ six-phase contributor pipeline (clone-website,
design-tasks, evolve-env, harden-env, seed-database).
Catalog (scraped from imdb.com via Playwright, gitignored in
sites/imdb/scraped_data/; shipped as instance_seed/imdb.db +
static/images/ via HF dataset):
- 392 titles (Top 250 + Top TV 250 + Most Popular + Box Office)
- 3796 persons (cast / crew with real bios + headshots)
- 4280 real posters + headshots
- 19 canonical IMDb genres
Coverage:
- 22 routes: homepage / title detail / fullcredits / reviews /
rate / watchlist toggle / person detail / scored search /
advanced search (genre / year / rating / sort) /
4 charts (Top 250 / Top TV / Most Popular / Box Office) /
genre browse / news / auth (login / register / account)
- scored token-overlap search (not strict-AND), 18 stop words
- 4 benchmark users alice.j / bob.c / carol.d / david.k
@ test.com (password TestPass123!) — pre-seeded with
4-item watchlists, 3-5 ratings, one written review each
- 12 seeded news items + 12 featured reviews (high
helpful-count) across major titles
- 18 benchmark tasks in tasks.jsonl across 6 functional areas:
chart browse, title detail, person filmography, advanced
search, genre browse, user state — including 4 hard tasks
(>= 5 steps) and 2 disambiguation tasks (bob's SF/fantasy
watchlist subset; carol's crime film subset)
Data integrity:
- canonical-URL guard skips 65 IMDb-redirected tt_ids
(some unknown tt_ids returned the wrong page)
- garbage filter skips 1854 names that hit IMDb's 403/
error fallback during concurrent scraping
- html.unescape on all string fields; hero h1 preferred
over ld.name (which is sometimes original-language)
- lowercase substring match for box-office data-testid keys
HF asset PR:
https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/23
Contributor
Author
Independent blind execution review
This was an execution-completion judgment only. The blind reviewer did not inspect or certify source fidelity, deterministic verifier correctness, runner independence, repository history, PR discussion, or prior reviews, and did not replay the runs live. The frozen run records predate the latest presentation-only candidate changes; their reuse basis and the current-candidate guided/deterministic validation are tracked separately and were intentionally outside this blind pass. The structured first-pass result was frozen and hashed before comparison. The Review PR remains Draft; reconciliation and final readiness have not started. |
Refresh review-reports/PR-33-IMDB.md to the fixed candidate (code 1c8a1eb, HF PR aiming-lab#57 revision f9ddfd25): drop the superseded HF pin and Owner-pending text, record the formal 20-task blind review and reconciliation, the fixed-candidate engineering and guided impact regression, Owner visual acceptance, known limitations and the maintainer handoff order. Add public summaries and current-candidate screenshots under review-reports/assets/pr33-imdb/ and mark the historical probe summaries as superseded. Docs-only: no application, task, verifier, seed or asset bytes change.
jackjin1997
marked this pull request as ready for review
September 12, 2026 15:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This reviewer branch preserves @hqhq1025's IMDb contribution and commit history from #33, corrects sourced profile/title metadata and interactive flows, adds deterministic grading for 20 tasks, and expands the homepage and featured editorial path from sourced captures. It integrates
mainat36004932bdf82afbe36dc14e00f66841eccf9946; IMDb is site 25 at port 40024.Ready for maintainer review — nothing is merged. Implementation fixed point: code
1c8a1eba19c084e202ecbab5977280701b9068c2. The current headfba88dd6385ee4029cd49851a86f642da6b3d774is a docs-only commit on top of it (onlyreview-reports/changes; application, task, verifier, registry, Dockerfile and.assets-revisiontrees are identical). Assets: open HF PR #57 revisionf9ddfd2596229f2610418d57fc88c3051e1056bb;imdb.tar.gzis 39,111,545 bytes, SHA2560663d30fe90ed2a0659dfe78ecf7ab1718970f565a64c6855a2c6aaadc606513; seed SHA2569d843c5388ecbc0d5265ac316cdaee915f8b203121b5a5330d207af5fb6206b0. Onlyimdb.tar.gzchanges in that PR; it reports no conflicting files.What was verified on the fixed candidate
/feature/featured-today-1HTTP 200, no horizontal overflow, 17/17 sourced entries with local media, 17/17 detail routes 200.claude-opus-5, fresh restricted session, no network) over a hash-verified 826-file packet covering all 20 sealed canonical runs: 20 PASS / 0 FAIL (comment). Reconciliation against the deterministic results: 0 differences, 0 substantive findings; common-PASS spot checks of tasks 9/15/16/17 against frozen DOM and full-table DB diffs confirmed the exact single-row changes.ebe92f0/HF4d5709e; the rest on r250bcce5/HFe70f49d) and are disclosed as such, not relabeled. Reuse basis:tasks.jsonlandsites/imdb/verifyare byte-identical since the expansion freeze, all ten business tables are logically identical between the execution seed and the current seed (onlyhome_featureswas added), and the later code changes are confined to homepage/feature/news/session routes.Retained rulings: Task 19 is graded from displayed amounts (
30.1M / 52.0M→ 57.9%; raw values would give 57.8% with the same winner). Task 26 completes its registration/queue/sign-out/re-login flow with exactly one new user and two Watchlist rows, but its run-start and seal-time provenance metadata disagree, so it is not claimed as a reliable independent-exploration attestation.Self-contained report: review-reports/PR-33-IMDB.md at
fba88ddwith public summaries (formal-blind-review-1c8a1eb-summary.json,reconciliation-1c8a1eb-summary.json,candidate-regression-1c8a1eb-summary.json) and current-candidate screenshots underreview-reports/assets/pr33-imdb/.Maintainer handoff — directly mergeable now: no. The code is MERGEABLE/CLEAN against
main, but.assets-revisionpins the open HF PR revision rather than a merged asset-main commit. Suggested order: (1) review and merge HF #57 (adds onlyimdb.tar.gz; asset main has since gained two unrelated archives and the PR reports no conflicts); (2) confirm the merged commit still servesimdb.tar.gzwith the SHA256 above; (3) update.assets-revisionon this branch to that commit, or confirm the immutable PR revision is acceptable — a pin-only change with identical bytes needs onlyfetch_assets.shplus the hash check, not new task runs or blind review; (4) merge this PR (supersedes #33) and close #33 and HF #23 with references so the old seed is not reintroduced. The reviewer will not merge either PR.