Review: harden WebHarbor task validation (#45)

[jackjin1997]

Summary

Reviewer-owned continuation of #45, preserving XuanRui LI / @Lxr-max's original commit
6b2a41a600bd0e1b260c3c80494f2d50f2b1d2fa and attribution before current-main
integration and reviewer fixes.

The review hardens the task validator for stable invalid-input diagnostics, registry
consistency, answer-key exclusion, complete site-local grading contracts, exact task
identity, verifier uniqueness, and lower-noise heuristics.

Detailed review report and reproduction commands

Verified candidate

• Current head: fdc58de0f179eec31da85f6094d78f57b69b5ac1
• Behavior commit: 142bae2c32c4f3fc8b1ceae51b1b63511b401f7d
• Base: 36004932bdf82afbe36dc14e00f66841eccf9946
• 31/31 tests passed on Python 3.11.3 and Python 3.12
• Strict corpus scan: 24 task files / 805 rows / 0 errors / 0 warnings
• 18/18 guided contract executions matched their predeclared outcomes with unchanged inputs
• Independent oracle-free blind review: 18 PASS / 0 FAIL
• ruff check, ruff format --check, pyright, and git diff --check passed

The final commit after the blind-reviewed head changes only the public report; validator,
tests, registries, task corpus, and verifier files are unchanged.

Scope

This is repository tooling, not a mirror contribution. It changes no site UI, route, seed,
database, Docker behavior, asset archive, or Hugging Face revision; those checks are not
applicable and are not reported as passing. No HF action is required.

Ready for maintainer review. This review does not merge GitHub or Hugging Face PRs.

[jackjin1997]

Independent blind first-pass review (PR #45 validator scenarios)

• Model: claude-fable-5-1
• Packet: handoff/pr45-142bae2-blind-v1, manifest SHA-256 bce1317c4b1985a248a0e0d3e1d9c55634cd19e8631d4f8014cd7cf2c173cae9
• Behavior commit: 142bae2c32c4f3fc8b1ceae51b1b63511b401f7d; reviewed head: 1a87f18f16ff83b6549a6a4e75cdb8e8ffca2cfe; run 142bae2c32c4-guided
• Packet validity: manifest checksum OK, 171/171 file hashes OK, no missing or unlisted files; before/after state byte-identical in all 18 scenarios
• Result: 18 PASS / 0 FAIL (TV-C01 … TV-C18); no failures
• Verdict artifact: independent-review/claude-pr45-v1.json, SHA-256 f9a125c1f8b9e52eca09fb58ce6e406357dc50b96ca5eb437f0e6656ac22b239 (hashed before any excluded material was opened)

Unexecuted checks:

• tree_sha256 construction is undocumented in the packet and could not be reproduced; non-mutation was judged from byte-identical before/after state files and per-file SHA-256 entries instead.
• TV-C18: the 162 verify/*.py files are recorded by path/size/hash only and are not in the packet, so their contents were not inspected. Task counts (805 across 24 files), registry/port consistency, and verifier references were independently re-checked from packet inputs.
• No re-execution of the validator; exit_code values are as reported by the thin adapter over run_validation, so CLI exit-code wiring was not exercised.

Non-blocking notes: TV-C11 message says "site order differs" where the site sets differ (wording only); TV-C07 counts an unparseable line as task_count=1; TV-C17 lists a non-existent tasks.jsonl under files[] as checked.

Scope: verdict covers only the frozen executions; it does not certify validator source or broader site behavior.