Review: Add Healthline mirror + task verifiers (site by @JeremyJC67, verifiers by reviewer) (#59)

.assets-revision

@@ -5,4 +5,4 @@
 # is a git revision (branch name like `main`, a tag, or a specific commit
 # sha). Override at runtime with the ASSETS_REVISION env var.
 repo: ChilleD/WebHarbor
-revision: ad6f424f72cada9e6f5c09a58093d0ceeab9c52b
+revision: 64264d065cdb0b7755ee99dab356be5733d9ddef

.claude/skills/clone-website/SKILL.md

@@ -204,7 +204,7 @@ Then run the site once locally to produce `instance/<site>.db`, copy it to
 ```bash
 ./scripts/build.sh webharbor:dev
 docker run -d --rm --name wh-test \
-  -p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev
+  -p 8201:8101 -p 41000-41026:40000-40026 webharbor:dev
 
 # your new site is on port 41000 + its index
 curl -so /dev/null -w "%{http_code}\n" http://localhost:41000NN/
@@ -228,7 +228,7 @@ After Phase 1, you should have:
 - `sites/<your_site>/static/` with real CSS/JS/icons (and images under HF assets)
 - `sites/<your_site>/instance_seed/<site>.db` with seeded data
 - Site registered in `websyn_start.sh`, `control_server.py`, `Dockerfile`
-- All 26 sites still return 200 on the alt-port container
+- All 27 sites still return 200 on the alt-port container
 - Byte-identical reset passes
 
 ## Next step

.claude/skills/evolve-env/SKILL.md

@@ -65,15 +65,15 @@ through `BASE_DIR = os.path.dirname(os.path.abspath(__file__))`.
 After any DB-affecting change:
 
 ```bash
-# 1. Stop test container (don't touch user's working container on :40000-40025)
+# 1. Stop test container (don't touch user's working container on :40000-40026)
 docker stop wh-test 2>/dev/null || true
 
 # 2. Rebuild
 ./scripts/build.sh webharbor:dev
 
 # 3. Run on alt ports
 docker run -d --rm --name wh-test \
-  -p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev
+  -p 8201:8101 -p 41000-41026:40000-40026 webharbor:dev
 
 # 4. Reset your site and confirm byte-identity
 curl -X POST http://localhost:8201/reset/<your_site>
@@ -149,7 +149,7 @@ After Phase 3:
 - Every task in `tasks.jsonl` is hand-verified to work end-to-end
 - The mirror has no obvious leaks, broken forms, or empty pages
 - Byte-identical reset passes
-- All 26 sites still return 200
+- All 27 sites still return 200
 
 ## Next step
 

.claude/skills/harden-env/SKILL.md

@@ -130,7 +130,7 @@ Hardening changes seed data. After any DB change:
 ```bash
 ./scripts/build.sh webharbor:dev
 docker run -d --rm --name wh-test \
-  -p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev
+  -p 8201:8101 -p 41000-41026:40000-40026 webharbor:dev
 curl -X POST http://localhost:8201/reset/<your_site>
 docker exec wh-test md5sum \
   /opt/WebSyn/<your_site>/instance/<your_site>.db \

.claude/skills/review-env/SKILL.md

@@ -33,18 +33,18 @@ gh pr checkout <pr-number>
 ./scripts/fetch_assets.sh             # pull the pinned HF revision
 ./scripts/build.sh webharbor:dev
 docker run -d --rm --name wh-review \
-  -p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev
+  -p 8201:8101 -p 41000-41026:40000-40026 webharbor:dev
 ```
 
-Confirm the new/changed site is on the expected port (40000 + index). Note: the image now runs 26 sites (40000-40025).
+Confirm the new/changed site is on the expected port (40000 + index). Note: the image now runs 27 sites (40000-40026).
 
 ### Step 2: The mechanical checks (5 minutes)
 
 Run the same Pre-PR checks the contributor was supposed to run.
 
 ```bash
-# 1. all 26 sites return 200
-for p in $(seq 41000 41025); do
+# 1. all 27 sites return 200
+for p in $(seq 41000 41026); do
   curl -so /dev/null -w "$p:%{http_code}\n" http://localhost:$p/
 done
 
@@ -231,7 +231,7 @@ Leave a structured comment on the PR:
 ## Review: <site_name>
 
 ### Mechanical checks: PASS / FAIL
-- [x] All 26 sites return 200
+- [x] All 27 sites return 200
 - [x] Control plane healthy
 - [x] Byte-identical reset (md5 match)
 - [x] Parallel reset <10s

.claude/skills/seed-database/SKILL.md

@@ -169,10 +169,10 @@ gh pr create
 ./scripts/check_assets.sh                       # every site has instance_seed/
 ./scripts/build.sh webharbor:dev                # docker build succeeds
 docker run -d --rm --name wh-test \
-  -p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev
+  -p 8201:8101 -p 41000-41026:40000-40026 webharbor:dev
 
-# all 26 sites return 200
-for p in $(seq 41000 41025); do
+# all 27 sites return 200
+for p in $(seq 41000 41026); do
   curl -so /dev/null -w "$p:%{http_code}\n" http://localhost:$p/
 done
 

AGENTS.md

@@ -48,17 +48,17 @@ Inside the image, sites live at `/opt/WebSyn/<site>/`. The path predates the ren
 # fresh clone
 ./scripts/fetch_assets.sh                     # pulls assets from HF
 ./scripts/build.sh                            # docker build -t webharbor:dev .
-docker run -d -p 8101:8101 -p 40000-40025:40000-40025 webharbor:dev
+docker run -d -p 8101:8101 -p 40000-40026:40000-40026 webharbor:dev
 ```
 
 Or use the published image directly:
 
 ```bash
-docker run -d -p 8101:8101 -p 40000-40025:40000-40025 \
+docker run -d -p 8101:8101 -p 40000-40026:40000-40026 \
   battalion7244/webharbor:latest
 ```
 
-Sites are on `40000`-`40025` in the order declared by `SITES=( ... )` in `websyn_start.sh`. Control plane:
+Sites are on `40000`-`40026` in the order declared by `SITES=( ... )` in `websyn_start.sh`. Control plane:
 
 | Method | Path                | Purpose                                   |
 |--------|---------------------|-------------------------------------------|
@@ -136,13 +136,13 @@ python3 -m py_compile sites/<site>/app.py
 
 # 3. run on alt ports (don't collide with anything you already have running)
 docker run -d --rm --name wh-test \
-  -p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev
+  -p 8201:8101 -p 41000-41026:40000-40026 webharbor:dev
 
 # 4. control plane healthy, all sites alive
 curl -s http://localhost:8201/health | python3 -m json.tool | head
 
 # 5. every site renders 200
-for p in $(seq 41000 41025); do
+for p in $(seq 41000 41026); do
   curl -so /dev/null -w "$p:%{http_code}\n" http://localhost:$p/
 done
 

CLAUDE.md

@@ -16,4 +16,4 @@ The full agent guide is loaded above via `@AGENTS.md`. The notes below apply onl
 
 ## Existing containers
 
-If a container is already running on `:8101` / `:40000-40025`, treat it as the user's working environment — don't `docker stop` or `docker rm` it without explicit confirmation. Spin up your test container under a different name on alt ports (`:8201`, `:41000-41025`).
+If a container is already running on `:8101` / `:40000-40026`, treat it as the user's working environment — don't `docker stop` or `docker rm` it without explicit confirmation. Spin up your test container under a different name on alt ports (`:8201`, `:41000-41026`).

CONTRIBUTING.md

@@ -24,7 +24,7 @@ git clone https://github.com/<you>/webharbor && cd webharbor
 ./scripts/fetch_assets.sh                       # pull current assets
 ./scripts/new_site.py mywebsite                 # OR edit an existing site
 ./scripts/build.sh && docker run -d --rm \
-  -p 8101:8101 -p 40000-40025:40000-40025 webharbor:dev
+  -p 8101:8101 -p 40000-40026:40000-40026 webharbor:dev
 # iterate locally...
 
 ./scripts/extract_assets.sh ../webharbor-static-pr/   # split assets out

Dockerfile

@@ -1,5 +1,5 @@
 # WebHarbor — slim, self-contained image.
-# 26 Flask mirror sites + control plane on :8101.
+# 27 Flask mirror sites + control plane on :8101.
 
 FROM python:3.12-slim-bookworm
 
@@ -63,6 +63,12 @@ RUN cd /opt/WebSyn/fedex && rm -rf instance instance_seed && \
 RUN python3 /opt/WebSyn/webmd_doctor/check_generated_assets.py
 RUN cd /opt/WebSyn/webmd_doctor && rm -rf instance instance_seed && \
     PYTHONHASHSEED=0 python seed_data.py && rm -rf instance __pycache__
+# Healthline's downloaded seed carries tracked corrections (image reassignment) and the
+# pinned archive bundles unreferenced images; apply the deterministic migration and prune
+# the unreferenced files before they are shipped.
+RUN cd /opt/WebSyn/healthline && test -f instance_seed/healthline.db && \
+    PYTHONHASHSEED=0 python3 migrate_seed.py && \
+    python3 prune_unreferenced_images.py --apply && rm -rf instance
 
 COPY websyn_start.sh    /opt/websyn_start.sh
 COPY control_server.py  /opt/control_server.py
@@ -86,6 +92,6 @@ os.makedirs('instance_seed', exist_ok=True); \
 shutil.copy2('instance/rotten_tomatoes.db', 'instance_seed/rotten_tomatoes.db'); \
 print('Rotten Tomatoes seed DB generated at build time.')" && rm -rf /opt/WebSyn/rotten_tomatoes/instance
 
-EXPOSE 8101 40000-40025
+EXPOSE 8101 40000-40026
 
 CMD ["/opt/websyn_start.sh"]

README.md

@@ -36,17 +36,17 @@ WebHarbor takes a different approach. We leverage coding agent (e.g., Claude Cod
 - **Deep features unlocked** — carts, checkouts, accounts, all fully testable
 - **Evolving** — harder tasks drive richer mirrors; the environment grows with agents
 - **RL-ready** — sub-second database resets between rollouts
-- **Community-driven** — 26 sites today, scaling to 100+ together
+- **Community-driven** — 27 sites today, scaling to 100+ together
 
 ## 🚀 Quickstart
 
 One command to run all web environments:
 
 ```bash
-docker run -p 8101:8101 -p 40000-40025:40000-40025 battalion7244/webharbor:latest
+docker run -p 8101:8101 -p 40000-40026:40000-40026 battalion7244/webharbor:latest
 ```
 
-Then point your agent at `http://localhost:40000` through `http://localhost:40025` to explore 26 local mirrors of WebVoyager sites: `Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, GitHub, Google Flights, Google Maps, Google Search, Hugging Face, Wolfram Alpha, Cambridge Dictionary, Coursera, ESPN, Merriam-Webster, IKEA, Phys.org, Target, TED, Ohio State University, Rotten Tomatoes, Compass, Walmart Careers, FedEx, and WebMD Doctor`.
+Then point your agent at `http://localhost:40000` through `http://localhost:40026` to explore 27 local mirrors of WebVoyager sites: `Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, GitHub, Google Flights, Google Maps, Google Search, Hugging Face, Wolfram Alpha, Cambridge Dictionary, Coursera, ESPN, Merriam-Webster, IKEA, Phys.org, Target, TED, Ohio State University, Rotten Tomatoes, Compass, Walmart Careers, FedEx, WebMD Doctor, and Healthline`.
 
 For sub-second reset between rollouts, expose the control plane and call `/reset/<site>`:
 

agent_demo/README.md

@@ -19,7 +19,7 @@ export OPENAI_BASE_URL=https://api.openai.com/v1   # or your Azure / vLLM endpoi
 
 ## Run a task
 
-WebHarbor must already be running locally (`docker run -p 8101:8101 -p 40000-40025:40000-40025 battalion7244/webharbor:latest`).
+WebHarbor must already be running locally (`docker run -p 8101:8101 -p 40000-40026:40000-40026 battalion7244/webharbor:latest`).
 
 Run a single task from a site's `tasks.jsonl`:
 

control_server.py

@@ -23,12 +23,13 @@
 from flask import Flask, jsonify
 
 SITES = [
-    'allrecipes', 'amazon', 'apple', 'arxiv', 'bbc_news', 'booking',
-    'github', 'google_flights', 'google_map', 'google_search',
-    'huggingface', 'wolfram_alpha', 'cambridge_dictionary',
-    'coursera', 'espn', 'merriam_webster', 'ikea', 'phys_org', 'target',
-    'ted', 'osu', 'rotten_tomatoes', 'compass', 'walmart_careers', 'fedex',
-    'webmd_doctor',
+    'allrecipes', 'amazon', 'apple', 'arxiv',
+    'bbc_news', 'booking', 'github', 'google_flights',
+    'google_map', 'google_search', 'huggingface', 'wolfram_alpha',
+    'cambridge_dictionary', 'coursera', 'espn', 'merriam_webster',
+    'ikea', 'phys_org', 'target', 'ted',
+    'osu', 'rotten_tomatoes', 'compass', 'walmart_careers',
+    'fedex', 'webmd_doctor', 'healthline',
 ]
 BASE_PORT = 40000
 WEBSYN_DIR = '/opt/WebSyn'