Skip to content

Add phet_simulations mirror site (port 40015) - #29

Open
ZhongyangLi0124 wants to merge 3 commits into
aiming-lab:mainfrom
ZhongyangLi0124:add-phet-simulations
Open

Add phet_simulations mirror site (port 40015)#29
ZhongyangLi0124 wants to merge 3 commits into
aiming-lab:mainfrom
ZhongyangLi0124:add-phet-simulations

Conversation

@ZhongyangLi0124

@ZhongyangLi0124 ZhongyangLi0124 commented May 18, 2026

Copy link
Copy Markdown

Summary

Adds a 16th mirror site to WebHarbor: PhET Interactive Simulations
(https://phet.colorado.edu/). Bound to port 40015.

Real site: University of Colorado Boulder's CC-BY-licensed catalog of
free interactive math and science simulations. Selected for its clean
taxonomy (subjects × grade bands × languages), stable academic content,
and no commercial anti-bot defenses.

Seeded catalog

Dimension Count
Simulations 98
Subjects 5 (physics, chemistry, math, biology, earth-science)
Grade levels 4 (elementary, middle, high, university)
Languages 28 (including Arabic, Hebrew, Persian RTL)
Teacher activities 14
Benchmark users 4
Benchmark tasks 43

Visual fidelity

Mirror is reskinned to closely match the real PhET site:

  • Real PhET trademark logo + CU Boulder logo, right-aligned nav
    (Simulations / Studio / Teaching / Research / Initiatives)
  • Photo-backed homepage hero, pastel subject squares in canonical order,
    photo-backed Teaching Resources callout, Interact/Discover/Learn stats,
    second STEM hero, sponsor footer (Moore / NSF / Hewlett / Mastercard / Yidan)
  • /simulations Browse / Filter / Customize tabs with underwater banner;
    Filter view has left-sidebar with collapsible subject tree, grade level,
    compatibility, locale, active-filter chips, sort
  • Simulation detail page with tabs (About / Teaching / Translations /
    Credits), large screenshot, supported-by sidebar, Donate CTA
  • Dismissible educator banner

Assets ship via the paired HF dataset PR (see below). 98 sim screenshots
(49 from PhET CDN, 49 styled placeholders for slugs not in HTML5 lib),
plus logos, sponsor logos, hero photos, subject icons, app-store badges,
underwater banner. ~5 MB tarball total.

Paired HF dataset PR

https://huggingface.co/datasets/ChilleD/WebHarbor/discussions/19

This PR is in Draft status until the HF PR merges. Once merged,
.assets-revision will be bumped to the merge commit SHA and the
draft will be marked ready for review.

Test plan

  • python3 -m py_compile sites/phet_simulations/app.py
  • ./scripts/build.sh succeeds (3.07 GB image)
  • docker run -d --rm -p 8101:8101 -p 40000-40015:40000-40015 webharbor:dev
  • Control plane reports phet_simulations alive on port 40015
  • All 10 key routes return 200 (/, /simulations, /simulations?view=browse,
    /simulations?view=filter&subject=physics, /simulation/build-an-atom,
    /simulation/algebra-tiles, /teachers, /translations, /about,
    /accessibility)
  • POST /reset/phet_simulations returns "ready": true
  • Byte-identical reset verified: md5 of instance/phet_simulations.db
    and instance_seed/phet_simulations.db both equal
    e094a2ee23369d3b60232f49f4ac691c post-reset and after docker restart

Reset evidence

POST /reset/phet_simulations  ->  {"pid":151,"ready":true,"site":"phet_simulations"}

e094a2ee23369d3b60232f49f4ac691c  /opt/WebSyn/phet_simulations/instance/phet_simulations.db
e094a2ee23369d3b60232f49f4ac691c  /opt/WebSyn/phet_simulations/instance_seed/phet_simulations.db

ZhongyangLi0124 and others added 2 commits May 18, 2026 07:54
Adds Flask app, templates, and seed DB for PhET Interactive Simulations
(https://phet.colorado.edu/), claiming port slot 40015.

Catalog: 98 simulations across 5 subjects (physics, chemistry, math,
biology, earth-science), 4 grade levels (elementary, middle, high,
university), and 28 languages (incl. 3 RTL scripts). Every primary
filter bucket clears the >=20-record threshold.

Models: User, Subject, GradeLevel, Language, Simulation, Activity,
SavedSimulation. Routes: 17 public + 2 JSON APIs + /_health, all
reachable from /. Auth via Flask-Login + bcrypt; saves via CSRF-protected
JSON endpoints.

Idempotency: every seed_* helper early-returns when its table is
populated; verified byte-identical (md5 e094a2ee...) across the
control_server reset cycle (rm -rf instance; cp -a instance_seed instance;
re-import app). Tested under the exact Dockerfile pin set (Flask 3.1.0,
SQLAlchemy 2.0.36, Werkzeug 3.1.3).

Per the port-slot convention, also:
- websyn_start.sh: append to SITES, bump 15 -> 16 in startup messages
- control_server.py: append 'phet_simulations' to SITES list
- Dockerfile: EXPOSE 40000-40014 -> 40000-40015

Seed DB (instance_seed/phet_simulations.db, 143KB) packs into a 17KB
phet_simulations.tar.gz via scripts/extract_assets.sh and ships
separately via the Hugging Face dataset. .assets-revision will need a
bump after the HF PR merges.

tasks.jsonl: 43 benchmark prompts covering catalog browse, subject
filters, simulation detail extraction, search, translations, teacher
activities, and the account save flow.
Replace placeholder UI with a high-fidelity mirror of the real
PhET Interactive Simulations site.

Visual:
- Real PhET yellow trademark logo + CU Boulder logo in white header bar
- Right-aligned nav (Simulations / Studio / Teaching / Research / Initiatives)
  with hover dropdowns; collapsing search icon + profile icon
- Dismissible pink educator banner
- Photo-backed top hero ("Interactive Simulations for Science and Math")
- 5 pastel subject squares in PhET's canonical order (Physics, Math &
  Statistics, Chemistry, Earth & Space, Biology) with their SVG icons
- Photo-backed Teaching Resources callout
- "Interact. Discover. Learn." stats section with real iconography
- Secondary STEM hero photo + centered "Supported in part by" + Donate CTA
- Real footer: social row, 4-column links, language selector, app-store
  badges, sponsor logos strip (Moore, Hewlett, NSF, Mastercard, Yidan)

Simulations browse page:
- Underwater illustration banner + Browse / Filter / Customize sub-tabs
- Filter view is the default; left sidebar with collapsible subject tree,
  grade level, compatibility, release type, inclusive features, locale,
  apply / clear, active-filter chips, sort dropdown, results count
- Browse view groups sims by subject with "View all »" links

Simulation detail page:
- Large screenshot with hover "Play" overlay
- Tabbed sections (About / Teaching Resources / Activities /
  Translations / Credits) plus topics, learning goals, related sims
- "Supported by" sidebar, share row, pink Explore More + Donate CTA

Backend wiring:
- Index route adds total_activities for the stats section
- Simulations route exposes view tab and per-subject buckets
- App.py exposes available_thumbnails so cards fall back gracefully
  when a slug has no screenshot

Assets ship via the paired HF dataset tarball (not committed):
- 98 sim screenshots (49 from PhET CDN, 49 generated placeholders)
- PhET + CU Boulder logos, subject icons, sponsor logos, hero photos,
  app-store badges, underwater simulations banner

Seed DB unchanged, /reset/phet_simulations remains byte-identical
(md5 e094a2ee23369d3b60232f49f4ac691c).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@ZhongyangLi0124
ZhongyangLi0124 marked this pull request as ready for review May 19, 2026 19:26
hqhq1025 pushed a commit to hqhq1025/WebHarbor that referenced this pull request May 26, 2026
hqhq1025 pushed a commit to hqhq1025/WebHarbor that referenced this pull request May 26, 2026
Conflicts resolved:
- websyn_start.sh / control_server.py: append berkeley after phet_simulations
  (PR base predated aiming-lab#29).
- Dockerfile EXPOSE 40000-40015 → 40000-40016.

Fixes on top of PR:
- sites/berkeley/seed_data.py: pin bcrypt hash for benchmark users.
  set_password() uses bcrypt.generate_password_hash which mixes a random
  salt on every call, breaking the byte-identical reset invariant. We
  store a pinned $2b$12$ hash for 'test1234' directly; check_password_hash
  still accepts it so login works unchanged.
- sites/berkeley/app.py: __main__ port 40015 → 40016 (cosmetic — site_runner
  passes port via -c string, not via __main__).
hqhq1025 pushed a commit to hqhq1025/WebHarbor that referenced this pull request May 26, 2026
Conflicts resolved:
- websyn_start.sh: keep PR's dynamic count (${#SITES[@]}) — removes the
  hardcoded "16" from aiming-lab#11/aiming-lab#29 base; append drugs_com after berkeley.
- control_server.py: append drugs_com after berkeley.
- Dockerfile EXPOSE 40000-40016 → 40000-40017; keep berkeley build-time
  RUN; pick up requests==2.32.3 dep from PR.
- .gitignore: merge both sides (keep main's dashboard ignores + PR's
  local artifact ignores).

Note: drugs_com seeds via pre-built HF db (refs/pr/13), so the dynamic
bcrypt salt in app.py's set_password does NOT cycle on reset.
@MufanQiu

MufanQiu commented Jul 1, 2026

Copy link
Copy Markdown

Review — phet_simulations (PR #29)

Verdict: REQUEST CHANGES.

We deployed the environment locally and ran the full review checklist; every finding below was independently reproduced, and fidelity was checked against the real upstream site.

Issues (summary)

  • BLOCKER: .assets-revision pinned to 'main' but this NEW site's tarball is only on the HF PR ref -> CI fail-closed
  • MAJOR: Task --41 is INFEASIBLE: homepage renders zero simulations (template ignores most_played/featured/new_sims)
  • MAJOR: Task --21 not solvable via UI: the 'New' catalog filter is decorative (backend ignores release=)
  • MAJOR: play_count/download_count mutate on every GET -> non-deterministic answers for tasks --6, --31, --41
  • MINOR: Ambiguous / degenerate tasks: --12 (download tie), --6 & --24 (two valid sims), --28 (all versions 1.0.0)
  • MINOR: Naive substring search + decorative filter controls (no tokenization; topic/compat/release/inclusive ignored)

Mechanical checks: FAIL

  • Asset pin is fail-closed (BLOCKER, F1). origin/pr/29:.assets-revision pins revision: main, but phet_simulations is a brand-new site whose phet_simulations.tar.gz lives only on the Hugging Face PR ref (hf_pr=19), not on the dataset's main.

Visual fidelity: PASS

Strong PhET reproduction. Homepage shows the PhET wordmark + CU Boulder logo, underwater/laptop hero, "Over 1.8 billion simulations delivered", five pastel subject cards, teaching-resources band, 98/28/14 interact stats, and the sponsor strip.

Functional depth: PASS (with nits)

Login (teacher@phet.test / phet-teacher-pass) → "Welcome back, Ada Lovelace!"; /account shows 4 saved sims. Registration validates email regex, name ≥ 2, password ≥ 8, and duplicate email.

Task quality: FAIL

43 tasks with broad coverage (search, browse, category, filter, sort, translations, teacher activities, account/CRUD, detail-page fact extraction). Distractor density is genuinely strong: a 98-sim catalog, 21–32 sims/subject, and minimal listing cards (title + language dots only) so grades/subjects/topics/release-date are not leaked on listing pages (verified across 11 searches).

Required fixes before approval

  1. (BLOCKER) Repin .assets-revision from main to the merged HF commit SHA containing sites/phet_simulations/phet_simulations.tar.gz (verify via hf download ChilleD/WebHarbor --revision <sha> ...phet_simulations.tar.gz).
  2. (BLOCKER) Render simulations on the homepage: make index.html display the route's most_played (and featured/new_sims) grids.
  3. (MAJOR) Implement the "New" catalog filter (read release/is_new in simulations() and filter Simulation.is_new) so T21 is UI-solvable; also wire up or remove the other decorative checkboxes (release=updated/topic/compat/inclusive) and fix the blank Customize subtab.
  4. (MAJOR) Stop mutating the DB on GET.
  5. (MINOR) Disambiguate tasks: break the 8220 download tie (T12); reword T6 and T24 to name the target sim or add a distinguishing constraint; vary sim versions or drop T28 (all v1.0.0).

Addresses all points from MufanQiu's review:

1. Homepage now renders simulation grids (BLOCKER). index.html displays
   the route's featured / new_sims / most_played context in three
   subject-section rows (20 sim cards), so task --41 is solvable from /.

2. 'New' release filter is functional (MAJOR). simulations() now reads
   release=new (is_new) and release=updated (released >= 2024-01-01);
   the sidebar checkboxes persist state and the active-filter chip,
   sort, and pagination links carry the param. Task --21 is solvable
   via UI (answer: 7 New sims, all released 2025).

3. No DB mutation on GET (MAJOR). Removed play_count increment in
   simulation_detail and download_count increment in activity_detail;
   counts are now fixed seed data, so tasks --6/--31/--41 have
   deterministic answers.

4. Seed/task disambiguation (MINOR):
   - Activities carry explicit pairwise-distinct download counts;
     unique most-downloaded = Net Force Investigation (8742), paired
     with Forces and Motion: Basics (task --12).
   - Sim versions derived deterministically from seed constants
     (47 distinct versions; Wave Interference = 1.5.2, task --28).
   - Tasks --6, --24, --28, --31 reworded with tighter constraints so
     each has exactly one valid answer (verified programmatically).

Seed DB rebuilt; reset cycle remains byte-idempotent
(md5 48ca438b8ac6dab37a6503a4e5574503). 66/66 routes return 200.
New phet_simulations.tar.gz must be re-uploaded to the HF dataset and
.assets-revision repinned to the HF merge SHA (separate commit).
@ZhongyangLi0124

Copy link
Copy Markdown
Author

Thanks for the thorough review — all five points addressed in b15feb3:

  1. Homepage now renders the route's featured / new_sims / most_played as three sim grids (20 cards), so task --41 is solvable from /.
  2. 'New' filter is functional: release=new filters is_new, release=updated filters release date ≥ 2024-01-01; checkbox state, active chip, sort, and pagination all persist the param. Task --21 is solvable via UI (7 New sims, all released 2025).
  3. No more GET mutations — the play_count / download_count increments are gone; counts are fixed seed data, so --6/--31/--41 are deterministic.
  4. Task ambiguities: activities now carry explicit pairwise-distinct download counts (unique max: Net Force Investigation, 8742 → Forces and Motion: Basics, task --12); sim versions are now varied and deterministic (47 distinct, Wave Interference = 1.5.2, task --28); tasks --6/--24/--28/--31 reworded so each has exactly one valid answer — verified programmatically against the seed.
  5. .assets-revision: the regenerated seed tarball (md5 48ca438b8ac6dab37a6503a4e5574503) is uploaded to the HF PR (ChilleD/WebHarbor #19, commit b3cf18f9). The repin commit will land here immediately once the HF PR merges and the merge SHA exists — happy to follow whichever merge order you prefer.

Re-verified locally: 66/66 routes 200, reset cycle byte-idempotent (seed md5 48ca438b8ac6dab37a6503a4e5574503), and repeated GETs leave the DB untouched.

@jackjin1997

Copy link
Copy Markdown
Contributor

I'm taking over the review and remediation of this PhET Interactive Simulations contribution. I'll build a separate Review: PR on top of @ZhongyangLi0124's original commits, preserve the original authorship, and link the reviewer PR here.

The review will cover current-main integration (site registry/port slot and the .assets-revision pin against the paired HF asset PR #19), re-verification of the b15feb3 fixes against the findings already raised by @MufanQiu, browser/UI fidelity against phet.colorado.edu, task solvability and quality across the 43 tasks (the accepted set will be decided by quality rather than count), deterministic verifiers backed by recorded UI executions, positive/negative and valid-alternative grading tests, and an independent review of the frozen runs. Maintainers retain final approval and merge responsibility. Please avoid duplicating this review while it is in progress.

@jackjin1997

Copy link
Copy Markdown
Contributor

Review PR is up as a Draft: #114 (branch review/pr-29-phet-simulations). Your three commits
are preserved with authorship intact, rebased onto current main as site 25 / port 40024.

Two findings drove most of the work, and I want to be direct about the larger one.

43 of the 98 seeded simulations do not exist on phet.colorado.edu. I checked every one of
the 49 placeholder-thumbnail slugs against upstream: 2 exist as written, 4 exist under a
de-mangled slug (beer-s-law-labbeers-law-lab, and the same pattern for Coulomb's,
Faraday's and Hooke's Law), and the remaining 43 return 404. Cellular Respiration,
Photosynthesis, Water Cycle, Predator-Prey Dynamics, Mendelian Genetics, Punnett Squares,
Mitosis and Meiosis, Simple Machines and 35 others are invented, along with their titles,
descriptions, subjects, grade bands, version numbers, release dates, play counts and language
lists. So this could not be fixed by swapping in better images — there is no screenshot to
fetch for a simulation that does not exist.

The catalogue is now the real HTML5 library from PhET's official metadata service: 120
simulations with upstream titles, descriptions and learning goals, real subject and topic
membership (I confirmed the numeric facet ids by intersecting each against the live filter —
Physics 65, Math 51, Chemistry 35, Earth & Space 19, Biology 8, exact on all four), real grade
bands, version strings, release and updated dates, per-simulation language lists, upstream's
relatedSimulations, the 131-language translation table, and the official 600px screenshot for
all 120. Facts PhET does not publish — play counts, download counts, per-sim runtime — are
removed rather than invented, so the homepage's third rail is Recently Updated and the catalogue
sorts by Recently updated / Most translated.

Every page overflowed horizontally below about 840px. The desktop nav, search form and user
menu never collapsed and by themselves forced the document to 837px at both 768 and 390, leaving
the content in the left 390px and the rest of the page blank. That is fixed: the nav collapses
behind a toggle below 900px with sub-links inline so they stay reachable without hover. 63 of 63
page/width captures now fit their viewport, and all 20 desktop pages keep byte-identical layout
heights.

Also fixed from @MufanQiu's earlier list, which I re-verified at b15feb3: the homepage grids,
the release= filter, and the GET-time mutation are genuinely fixed — thank you. Still open
were the decorative filter controls (11 topic, 3 compatibility and 3 inclusive checkboxes the
route never read), the blank Customize tab, and the non-tokenised search where build atom
returned nothing. All three are addressed.

The task set is rebuilt from 43 to 18. Nine targeted fabricated simulations, four read the
removed play counter, one asked which scripts are right-to-left (answerable without opening the
site), and one only asked the agent to click through to a page with nothing to report. The new
tasks are anchored on facts that live on this site and are not visible on a listing card. They
ship with 18 deterministic verifiers and rules-only rubrics; an 88-cell adversarial matrix
matches every expectation and 16 legitimate rephrasings still pass.

Assets: ChilleD/WebHarbor discussions/79 supersedes #19, since #19's seed is the fabricated
catalogue.

Still outstanding on the review PR: the independent review of the frozen runs, and a full
25-site image build. It stays in Draft until those land. Maintainers keep final approval and
merge.

jackjin1997 pushed a commit to jackjin1997/WebHarbor that referenced this pull request Sep 13, 2026
Addresses all points from MufanQiu's review:

1. Homepage now renders simulation grids (BLOCKER). index.html displays
   the route's featured / new_sims / most_played context in three
   subject-section rows (20 sim cards), so task --41 is solvable from /.

2. 'New' release filter is functional (MAJOR). simulations() now reads
   release=new (is_new) and release=updated (released >= 2024-01-01);
   the sidebar checkboxes persist state and the active-filter chip,
   sort, and pagination links carry the param. Task --21 is solvable
   via UI (answer: 7 New sims, all released 2025).

3. No DB mutation on GET (MAJOR). Removed play_count increment in
   simulation_detail and download_count increment in activity_detail;
   counts are now fixed seed data, so tasks --6/--31/--41 have
   deterministic answers.

4. Seed/task disambiguation (MINOR):
   - Activities carry explicit pairwise-distinct download counts;
     unique most-downloaded = Net Force Investigation (8742), paired
     with Forces and Motion: Basics (task --12).
   - Sim versions derived deterministically from seed constants
     (47 distinct versions; Wave Interference = 1.5.2, task --28).
   - Tasks --6, --24, --28, --31 reworded with tighter constraints so
     each has exactly one valid answer (verified programmatically).

Seed DB rebuilt; reset cycle remains byte-idempotent
(md5 48ca438b8ac6dab37a6503a4e5574503). 66/66 routes return 200.
New phet_simulations.tar.gz must be re-uploaded to the HF dataset and
.assets-revision repinned to the HF merge SHA (separate commit).
@jackjin1997

Copy link
Copy Markdown
Contributor

Review complete. #114 is out of Draft and ready for maintainer review.

Since my last comment the branch was rebased again: upstream merged fedex, webmd_doctor,
healthline and kaggle during the review, which took port 40024. The site is now registered as
site 29 on port 40028, based on current main, mergeable and clean. Your three commits are
still intact with authorship preserved.

Two independent review rounds were run on the frozen executions in sessions with no access to the
verifiers or to any of my conclusions. Both returned 18 PASS / 0 FAIL, and the second agrees with
the deterministic run 18/18.

The first round flagged evidence-quality gaps rather than wrong verdicts, and chasing them turned
up three real defects worth naming, since two were mine:

  • The saved-simulation task was not completable through the UI. The task asks for a note, the
    API stores one, but the form had no note field. My own run harness had masked this by issuing a
    raw API call and recording it as a button click. The form now has the field, the page sends it,
    and the harness types into it and clicks the real button.
  • Right-to-left language cards reversed their English text — Arabic rendered
    simulations 119 instead of 119 simulations, because direction: rtl was applied to the
    whole card. Now scoped to the native name.
  • State snapshots were batch-level, copied into all 18 task directories, with no reset between
    runs. Every task now resets and snapshots its own before/after.

A 90-cell adversarial matrix then found three verifiers a wrong answer could satisfy: the two
save tasks graded purely on the database delta, so an answer about an unrelated simulation
passed; and a bare number check accepted version 9.9.9 as the count 9. Counts are now bound to
the thing they count, and six legitimate rephrasings still pass.

One thing for the maintainers rather than for you:
sites/rotten_tomatoes/tests/test_environment_quality.py::test_task_manifest_and_registry fails
on pristine main at line 101 — it asserts a literal site list that upstream reflowed across
lines. I reproduced it in a clean worktree of origin/main and left it alone, since it belongs to
another site.

Merge order: HF discussion #79 first, then .assets-revision pinned to its merge commit, then
#114. Maintainers keep final approval and merge on both.

Thanks for the mirror — the Flask app, auth, CSRF and reset behaviour were solid throughout, and
the reskin work carried over unchanged.

jackjin1997 pushed a commit to jackjin1997/WebHarbor that referenced this pull request Sep 14, 2026
Addresses all points from MufanQiu's review:

1. Homepage now renders simulation grids (BLOCKER). index.html displays
   the route's featured / new_sims / most_played context in three
   subject-section rows (20 sim cards), so task --41 is solvable from /.

2. 'New' release filter is functional (MAJOR). simulations() now reads
   release=new (is_new) and release=updated (released >= 2024-01-01);
   the sidebar checkboxes persist state and the active-filter chip,
   sort, and pagination links carry the param. Task --21 is solvable
   via UI (answer: 7 New sims, all released 2025).

3. No DB mutation on GET (MAJOR). Removed play_count increment in
   simulation_detail and download_count increment in activity_detail;
   counts are now fixed seed data, so tasks --6/--31/--41 have
   deterministic answers.

4. Seed/task disambiguation (MINOR):
   - Activities carry explicit pairwise-distinct download counts;
     unique most-downloaded = Net Force Investigation (8742), paired
     with Forces and Motion: Basics (task --12).
   - Sim versions derived deterministically from seed constants
     (47 distinct versions; Wave Interference = 1.5.2, task --28).
   - Tasks --6, --24, --28, --31 reworded with tighter constraints so
     each has exactly one valid answer (verified programmatically).

Seed DB rebuilt; reset cycle remains byte-idempotent
(md5 48ca438b8ac6dab37a6503a4e5574503). 66/66 routes return 200.
New phet_simulations.tar.gz must be re-uploaded to the HF dataset and
.assets-revision repinned to the HF merge SHA (separate commit).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants