Skip to content

Review: MEGA mirror + task verifiers (site by @plzdoo, verifiers by reviewer) - #121

Open
Lxr-max wants to merge 4 commits into
aiming-lab:mainfrom
Lxr-max:cursor/review-mega-55ad
Open

Lxr-max wants to merge 4 commits into
aiming-lab:mainfrom
Lxr-max:cursor/review-mega-55ad

Conversation

@Lxr-max

@Lxr-max Lxr-max commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

This PR is the review vehicle for #2 (feat(mega): add new site, site folder sites/mega, original site by @plzdoo). It does not merge #2 itself.

Fork work branch: https://github.com/Lxr-max/WebHarbor/tree/cursor/review-mega-55ad

Site commits keep original authorship (yimingl9 <yimingl9@illinois.edu>). Reviewer commits are Lxr-max <xuanrui.li@se24.qmul.ac.uk>.

Lineage

Item Detail
Contributor PR #2plzdoo:main, tip d1abd2c (yimingl9 <yimingl9@illinois.edu>), port 40015 / 16-site era
Cherry-pick d1abd2c onto current main as 2b30aa6only sites/mega/** (contributor main was init + one site commit, 262 behind current main; registry files were not taken)
This PR Re-slot + mechanical fixes + grading contract on current aiming-lab/WebHarbor main

Slot after rebase onto current main

  • Site slug: mega
  • Index: 28 (after kaggle)
  • Container port: 40028 (http://localhost:40028/)
  • Registry: websyn_start.sh, control_server.py, Dockerfile EXPOSE 8101 40000-40028 (29 sites)

Tasks and grading contract

  • 18 / 18 tasks accepted (MEGA--0--17)
  • 18 deterministic verifiers: sites/mega/verify/verify_0.pyverify_17.py plus verify_lib.py
  • tasks.jsonl has verifier_path + English judge_rubric (rules/checkpoints only — no answer key)
  • web URLs updated from outdated 40015http://localhost:40028/
  • Harness: sites/mega/verify/test_verifiers.py

Reviewer fixes on this branch

  • Safe form parsing (form_int / form_float), password length on register, Cloud Drive folder/upload path normalization.
  • Plan detail now shows Users included; Cloud Drive listing no longer leaks N items counts.
  • S4 Fixed Storage tagline no longer says “S3-compatible” (that wording stays on the plan-detail features only).
  • Unused harvested CSS buttons.css.css / theme.css.css are not linked (base.html loads mega.css only) — left in place; they do not break loading.

Task re-anchors (knowledge-shortcut / ill-posed)

  • MEGA--0/1/10/11/13/15: require a specific on-page fact, not “open a page”.
  • MEGA--2: uniquely Pro II (8 TB + MEGA VPN in features), yearly, no purchase (Pro III also has ≥8 TB).
  • MEGA--5: share Atlas b-roll (seed launch footage was already shared with Bob/Carol).
  • MEGA--12: object-storage filter + S3-compatible API feature on the detail page.
  • Login tasks now include password: TestPass123!.

Validation (what was actually run)

From this branch, not a full 29-site Docker image (docker is not available on this Cloud Agent VM):

python3 scripts/check_site_registry.py
# [registry] 29 sites consistent … EXPOSE (8101 40000-40028)
python3 sites/mega/verify/test_verifiers.py

Offline verifier matrix (--no_llm true):

  • no-op FAIL: 18/18
  • knowledge-shortcut FAIL: 18/18
  • wrong-answer FAIL (read-only / comparison tasks)
  • genuine pass PASS
  • stateful self-report-without-write FAIL

Local Flask test_client: homepage, product pages, pricing filters, downloads, help search/articles, login, Cloud drive search, vault search, Pro I monthly checkout, Pro II yearly cart-without-purchase, folder+upload, share link, favorite, vault create, account company/2FA update. Real MEGA images (128 files) loaded from the unpaired HF discussion pack for rendering checks.

Idempotent seed import: md5(instance/mega.db) == md5(instance_seed/mega.db) after import app (generated DB; bcrypt salts are not stable across regenerations).

Assets / blockers (not merge-ready until these land)

  1. HF pin does not contain mega.tar.gz. Current .assets-revision is 64264d065cdb0b7755ee99dab356be5733d9ddef (28-site main). Contributor assets live on HF discussion Add TED mirror site #1 / commit 8997072c and are not on that pin. Do not bump the pin to 8997072c (it would drop the other 28 site archives). scripts/fetch_assets.sh / scripts/check_assets.sh will fail for sites/mega until a new HF dataset PR is merged onto current ChilleD/WebHarbor main and .assets-revision is bumped.
  2. No Docker in this review environment, so byte-identical /reset/mega inside the full image was not repeated here. Contributor feat(mega): add new site #2 reported matching md5s on their older 16-site image; that needs a re-run on the 29-site image after assets are pinned.
  3. Seed DB is generated from seed_data.py. The frozen instance_seed/mega.db must ship via HF, not git.

Commits on this branch

| Commit | Author | Content |
|---|---|
| 2b30aa6 | yimingl9 | feat(mywebsite): add new site (cherry-pick of d1abd2c, sites/mega/** only) |
| cc5df46 | Lxr-max | re-slot onto current main (index 28 / port 40028) |
| 4834593 | Lxr-max | harden forms, folders, and plan-detail leaks |
| 8ce83b9 | Lxr-max | add deterministic task verifiers |

plzdoo and others added 4 commits September 14, 2026 02:30
Co-authored-by: Lxr-max <Lxr-max@users.noreply.github.com>
Append mega after kaggle in websyn_start.sh and control_server.py, raise
Dockerfile EXPOSE to 40000-40028, and align docs with the 29-site inventory.

Co-authored-by: Lxr-max <Lxr-max@users.noreply.github.com>
Safe int/float parsing on checkout and uploads, normalize Cloud drive folder
paths, show included users on plan detail, drop the listing count label, and
strip S3-compatible wording from the S4 Fixed Storage card tagline so that
fact lives on the detail page.

Co-authored-by: Lxr-max <Lxr-max@users.noreply.github.com>
Re-anchor ill-posed / leaky tasks onto page-specific facts, put demo
credentials in login tasks, point web URLs at localhost:40028, and add
verify_lib.py plus verify_0.py–verify_17.py with judge_rubric checkpoints.

Co-authored-by: Lxr-max <Lxr-max@users.noreply.github.com>
@Lxr-max

Lxr-max commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

HF assets PR opened

Opened a new Hugging Face dataset PR with mega.tar.gz rebased onto current ChilleD/WebHarbor main (existing archives preserved):

https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/87

  • Same bytes as contributor discussion Add TED mirror site #1 / commit 8997072c
  • sha256 cb989a9a2026fafd8c0def8055c08f7fb866822882d764439de7fe56d85e08f3 (24,209,535 bytes)
  • validate_asset_archive.py: 135 managed members

Once #87 is merged, we can bump .assets-revision on this PR to the new HF merge sha. Do not pin to 8997072c (would drop other site archives).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants