Review: Ohio State (osu) mirror + task verifiers (site by @richard-peng-xia, verifiers by reviewer) - #3
Merged
Conversation
Keep coach, records, titles, research directors, program credits/deadlines, department chairs, and news bylines on detail pages so listing/search shortcuts cannot complete the benchmark tasks. Drop the redundant "Dean" prefix from college names so Academics renders "Dean: <name>". Co-authored-by: Lxr-max <Lxr-max@users.noreply.github.com>
Independent reviewer grading contract for PR aiming-lab#12: one verify_N.py per accepted task, shared verify_lib, verifier_path + rule-only judge_rubric on all 20 rows, and a verify/tests harness (no-op, shortcut, wrong-answer, listing-without-detail, swapped comparison, missing filter, DB mutation). Re-anchor task 4 so the expenditure amount is not named in the question. Co-authored-by: Lxr-max <Lxr-max@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is the review vehicle for aiming-lab#12 (Ohio State University mirror,
sites/osu, original site by @richard-peng-xia). It does not merge aiming-lab#12 itself.Lineage
richard-peng-xia:add-osu-mirror, commit635808d(richard-peng-xia <richard.peng.xia@gmail.com>), port 40015 / 16-site eramain(site index 20 / port 40020). That merge rewrote the original commit author to StephenQSstarThomas / later hardening by raibows.main: keep the hardened site, upgrade the grading contract to current Review-PR practice, de-leak listing cards, re-anchor task 4.Cherry-picking
635808donto currentmainwould collide with the already-landedsites/osu/tree and regress aiming-lab#63 security/image/reset work, so this branch does not rewrite those commits. Original authorship is preserved by leaving aiming-lab#12's commit intact on the contributor branch and crediting @richard-peng-xia here; reviewer commits areLxr-max <xuanrui.li@se24.qmul.ac.uk>.Slot after rebase onto current main
osuted)http://localhost:40020/)websyn_start.sh,control_server.py,Dockerfile EXPOSE 8101 40000-40027(28 sites)Tasks and grading contract
Ohio State University--0…--19)sites/osu/verify/verify_0.py…verify_19.pyplusverify_lib.pytasks.jsonlhasverifier_path+ Englishjudge_rubric(rules/checkpoints only — noanswerkey, no ground-truth names in the rubric)sites/osu/verify/tests/(excluded from the image by.dockerignore)Original aiming-lab#12 questions were almost all knowledge-shortcuts / listing-only ("What conference does Ohio State athletics compete in?", "In what year was The Ohio State University founded?", …). They were previously re-anchored onto explicit on-site workflows. This review keeps that re-anchoring and additionally:
$1.3 billion; the agent must read amount + date from the article after searchingresearch expenditures.Reviewer fixes on this branch
"Dean "prefix in seed college names so Academics rendersDean: Anil Makhijarather thanDean: Dean Anil Makhija.Validation (what was actually run)
From this branch, not a full 28-site Docker image (no Docker daemon on the review host):
Offline verifier matrix (scripted trajectories, no LLM):
Live standalone Flask on
:41020(HFosu.tar.gzimages from pin64264d0…+ build-generated seed):alice@osu.edu/test1234→ account 200; GET/logout405; POST logout without CSRF 400cancer research→ article byline, login + Save Program persists on account, 390px home has no page-level overflowAssets
osu.tar.gzis on the current HF pin (ChilleD/WebHarbor@64264d065cdb0b7755ee99dab356be5733d9ddef);./scripts/fetch_assets.sh osuinstalled 19 official WebP files. No.assets-revisionbump.sites/osu/.build-generated-seed); not shipped in the tarball. Byte-deterministic rebuild tested twice in the unittest.Blockers
docker build/ 28-site/health//reset/osumd5 identity not run here (no Docker). Pre-PR image checks still needed on a Docker host before merge.Commits on this branch
3d22182858de51verify/testsharness, task 4 re-anchor