Review: IGN mirror — self-contained verifiers, 20th task, orphaned links (supersedes #61/#69) - #102
Open
TabsPhasers wants to merge 6 commits into
Open
Review: IGN mirror — self-contained verifiers, 20th task, orphaned links (supersedes #61/#69)#102TabsPhasers wants to merge 6 commits into
TabsPhasers wants to merge 6 commits into
Conversation
…ers by reviewer)
verify_lib.py imports simpleArgParser, which was only installed in the author's local venv and is not declared in requirements.txt or shipped in the repo. In a clean environment every verify_N.py crashed with ModuleNotFoundError before emitting any JSON (infrastructure failure, not a grade). Vendor the exact module the author's environment uses (simpleArgParser 0.2.4) under sites/ign/verify/ so the verifiers run anywhere the Flask app runs. Found and fixed during PR aiming-lab#61 review (review-and-fix round A; first-run infra evidence frozen separately).
…esolve screenshot paths Review-and-fix round B for PR aiming-lab#61 (findings frozen after the first full native round on the untouched PR aiming-lab#69 baseline): - account.html: /saved (Saved Stories) and /logout had routes and views but no inbound link anywhere in the UI. IGN--14 ('go to Saved Stories and remove the Deals item') was therefore infeasible through real navigation; the first-round actor failed it after 40 steps of exhaustive search and the verifier correctly FAILed. Add both links to the profile actions row. - tasks.jsonl: this Env shipped 19 tasks/19 verifiers against the batch's 20+20 hard gate. Add IGN--19 (register flow, deterministic after-state: new users row with email/username/display_name) and its dedicated verify_19.py. - verify_lib._shot: only screenshots under screenshots/step_*.png were resolved; producers that store absolute paths (e.g. browser/after-NNNN.png) silently degraded the screenshot layer to SKIP. Resolve recorded paths directly, confined to the run_dir for relative names. New task IGN--19 runs as its own assignment; existing task rows are byte-identical to the PR aiming-lab#69 baseline.
db_query returns fetchall() rows; indexing [1]/[2] on the list crashed with IndexError whenever the registered user existed, so IGN--19's first actor run graded as an infrastructure failure instead of a verdict (frozen in runs/ign-rerun-b-001). Select row 0 explicitly. Regrade of the frozen IGN--19 before/after/trajectory is recorded separately as verifier-only.
Open ign.tar.gz as HF dataset PR aiming-lab#73 under the review-side account (oratory3n), byte-identical to the author's aiming-lab#41 tarball (sha256 7bf24ca9...). Keeps the asset lifecycle under reviewer control; either PR can be merged, after which this pin must move to the merged SHA.
Contributor
Author
证据索引（review-and-fix 全记录）本 PR 的完整审查在隔离环境完成（宿主机端口段 48500–48599，单站 harness 租约 site 48500-48504 / control 48505-48509 / broker 48510；全站门禁映射 48520+i → 容器 40000+i）。 判分环境 pin：候选代码 = 运行批次（每题 fresh baseline 容器 + no-op FAIL 门禁 → fresh actor 容器/profile）：
质量探测：38 项 near-miss/归一化用例全部符合预期（错误账号、错误文件夹/note/status/频率、未暂停、未完成、错误答案、无导航捷径 → FAIL；大小写/空白归一 → PASS）。 机械门禁：17/17 站点 HTTP 200；IGN 视觉矩阵：27 页 × 1440/390/320 = 87 组实测，1440 全站 PASS；6 页窄屏横向溢出（已知限制节）。 Actor 返回凭证：每题真实 native session id 与主 Agent 保存的工具返回记录逐题存档（attach-evidence），与 channel/step 配对可追溯。 完整审计包（冻结轨迹、逐题 verify 输入输出、探测 fixture、视觉矩阵截图）由 review 侧留存，维护者需要时随时提供。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述与归属
本 PR 是对 #61（IGN 站点，作者 @Edlison） 与 #69（verifier 与 task-quality 修复，作者 @Django-Jiang） 的 review-and-fix replacement：两条 PR 属同一
sites/ignEnv（61→69 延续链），按批次口径合并为一个审查单元后，以 #69 的 headf67b604为基线完成完整 review，并叠加 4 个 review 修复 commit（2f742a6、69ece8e、c1cdf0a、cc68503）。原 19 条任务（IGN--0..18）的
ques行与 #69 逐字节一致（IGN--16 的 seed 预满足修复沿用 #69：原题目标视频已在 alice 的 seed playlist 中，无法判分；改题后已实证可判分）。任务 ID 集合除新增 IGN--19 外不变。修复内容（相对 #69 的 4 个 commit）
2f742a6verifier 自含化（阻塞级修复）：Add IGN mirror + task verifiers (site by @Edlison, verifiers by reviewer) #69 的 19 个 verifier 顶层import simpleArgParser——该模块仅在作者本地环境安装，不在requirements.txt/仓库/镜像内。干净环境首个 baseline no-op 即ModuleNotFoundError，全部任务判分变成 infra 失败（exit 2、无 JSON）。本 PR 将作者环境所用的 simpleArgParser 0.2.4 vendor 到sites/ign/verify/，verifier 随站点镜像自包含。69ece8e孤儿链接恢复 + 第 20 题 + 截图路径解析：/saved（Saved Stories）与/logout路由和页面存在但全站 UI 无任何入链：IGN--14（去 Saved Stories 移除 Deals 项）经真实 UI 不可完成——首轮纯视觉 actor 40 步穷举失败，verifier 正确 FAIL。在account.htmlprofile 操作区补两条链接后，IGN--14 fresh rollout 12 步 PASS。verify_19.py。verify_lib._shot原只解析screenshots/step_NNN.png布局，绝对路径截图静默退化为 SKIP；已按记录路径直接解析（相对路径限制在 run_dir 内）。c1cdf0averify_19 修复：db_query返回行列表被当成单行索引，已注册场景 IndexError → infra；改为取首行。首轮 infra 记录保留，对冻结材料 verifier-only regrade。cc68503资产 pin 换至 review 侧自控的 HF PR：.assets-revision由refs/pr/41（@Edlison 的 open 资产 PR）换为refs/pr/73（review 侧用 HF 账号 oratory3n 开启的 discussion #73）。两者字节一致（tarball sha2567bf24ca9…；seed DB sha256d662dd28…）；注意 upstream main 的.assets-revision已在近期被维护者 bump 过，本 PR 的 pin 变更是有意为之（该 ref 可直接拉取、无需合并，见"已知限制"）。验证证据（review 环境全记录，隔离端口段 48500–48599）
机械门禁：干净源码+固定素材构建；17/17 站点 HTTP 200；IGN
_health={items:109, users:4, ok:true}；POST /reset-all17/17 ready；IGN reset 字节一致（md5da9379739f89373db60e2c32e801bc60，与 #69 提交说明独立吻合）；preflight 的 baseline no-op 全部正常 FAIL（exit 1 + 合法 JSON）。原生纯视觉 actor E2E（每题独立容器/profile/session，坐标+按键协议，1280×900）：20/20 任务 PASS。
21/21 baseline no-op 全部正常 FAIL，无假阳性。状态题 DB before/after 全表对账与题意逐一相符（含 IGN--12 预存 Weekend 行的文件夹迁移、IGN--13 active 翻转、IGN--5 users 行内更新）；信息题无 DB diff 且答案与站内证据吻合。
判分精度探测（38 项，全符合预期）：每题最小 near-miss——错误账号/错误文件夹/错误 note/错误 status/错误频率/未暂停/未完成/错误答案/无导航捷径 → 全部正确 FAIL；大小写/空白归一正例 → 正确 PASS。
持久化与 reset（独立诊断副本）：IGN--5 修改经页面重入、登出重登均保持（UI+DB 双证）；
POST /reset/ign后精确回到 seed 值。全站视觉矩阵（27 页 × 1440/390/320 = 87 组）：1440 全站 PASS。唯一遗留 finding F10（低-中，仅窄屏）：news、news/television、reviews、reviews/tech、search、account 六页在 390/320 宽横向溢出（hub 分类导航未收起，news 类 sw=1426 最重）。不影响桌面判分与 1280 actor 视口，如实披露。
已知限制（披露）
ign.tar.gz已通过 review 侧自控的 HF PR #73 提交，.assets-revision钉在refs/pr/73，fetch_assets.sh可直接拉取（本 PR 的全部构建即基于该 ref，tarball sha2567bf24ca9…，seed DB sha256d662dd28…）。合并 feat(google_finance): add Google Finance mirror (site 17, port 40016) #73（或 @Edlison 的 Add Versus mirror site #41，两者字节一致）并非必需，仅为将资产转正进 main；维护者合并后可选地将 pin bump 到 merged SHA。关联
Supersedes #61, #69（站点归属与历史讨论见原 PR；本 PR 合并后两条均可用 replacement 关闭）。