Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
4fe15e8
feat(webmd_doctor): add WebMD Doctor mirror (site 24, port 40024)
evanz37 Sep 10, 2026
c88d2ea
feat(webmd_doctor): add 20 benchmark tasks
evanz37 Sep 10, 2026
0362f46
fix(webmd_doctor): evolve mirror to support all tasks
evanz37 Sep 10, 2026
a3167ad
fix(webmd_doctor): visual parity with reference captures
evanz37 Sep 10, 2026
0bbd841
fix(webmd_doctor): finalize deterministic build-generated seed
evanz37 Sep 10, 2026
00365d1
docs: bump site count to 25 (ports 40000-40024)
evanz37 Sep 10, 2026
a2b4c7f
fix(webmd_doctor): ship images via HF tarball (build-generated DB unc…
evanz37 Sep 10, 2026
a6ece6e
chore(webmd_doctor): pin assets to HF dataset PR (ChilleD/WebHarbor d…
evanz37 Sep 10, 2026
434b3cc
fix(webmd_doctor): audit B hardening — payer-row consistency, unmatch…
evanz37 Sep 10, 2026
b26a1d4
fix(webmd_doctor): faithful empty state, per-office phones, perspecti…
evanz37 Sep 10, 2026
3859db9
fix(webmd_doctor): audit-C fixes (task wording, booking reference, sp…
evanz37 Sep 11, 2026
273ca14
fix(webmd_doctor): audit-D realism fixes (curated specialty pools, tr…
evanz37 Sep 11, 2026
3c22d4a
fix(webmd_doctor): audit-D nits (logout clears the session, stricter …
evanz37 Sep 11, 2026
5657fdd
test(webmd_doctor): add deterministic task verifiers and verify_lib
evanz37 Sep 11, 2026
82af7dd
chore(webmd_doctor): backfill verifier_path + judge_rubric
evanz37 Sep 11, 2026
df19ae7
test(webmd_doctor): harden verifiers from the validation matrix
evanz37 Sep 11, 2026
e7862ed
chore(webmd_doctor): polish judge rubrics from real agent runs
evanz37 Sep 11, 2026
b87db8f
Harden the WebMD Doctor mirror after full-stack review
Chilled Sep 11, 2026
317bb13
Remove booking-widget patient-type preselection; fix enhanced-profile…
Chilled Sep 11, 2026
a03a8af
Bind the T5 entity leak test to answer facts and restore the tarball …
Chilled Sep 11, 2026
74ddb0b
Darken the bio meet accent to meet WCAG AA for normal text
Chilled Sep 11, 2026
f47b6a8
Merge upstream main (FedEx mirror, WebHarbor #82) into the WebMD Doct…
Chilled Sep 13, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .assets-revision
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
# is a git revision (branch name like `main`, a tag, or a specific commit
# sha). Override at runtime with the ASSETS_REVISION env var.
repo: ChilleD/WebHarbor
revision: 68dcbf2cbd0cbc2fbf97dcb71da13ace67c11bf6
revision: ad6f424f72cada9e6f5c09a58093d0ceeab9c52b
4 changes: 2 additions & 2 deletions .claude/skills/clone-website/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ Then run the site once locally to produce `instance/<site>.db`, copy it to
```bash
./scripts/build.sh webharbor:dev
docker run -d --rm --name wh-test \
-p 8201:8101 -p 41000-41024:40000-40024 webharbor:dev
-p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev

# your new site is on port 41000 + its index
curl -so /dev/null -w "%{http_code}\n" http://localhost:41000NN/
Expand All @@ -228,7 +228,7 @@ After Phase 1, you should have:
- `sites/<your_site>/static/` with real CSS/JS/icons (and images under HF assets)
- `sites/<your_site>/instance_seed/<site>.db` with seeded data
- Site registered in `websyn_start.sh`, `control_server.py`, `Dockerfile`
- All 25 sites still return 200 on the alt-port container
- All 26 sites still return 200 on the alt-port container
- Byte-identical reset passes

## Next step
Expand Down
6 changes: 3 additions & 3 deletions .claude/skills/evolve-env/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,15 +65,15 @@ through `BASE_DIR = os.path.dirname(os.path.abspath(__file__))`.
After any DB-affecting change:

```bash
# 1. Stop test container (don't touch user's working container on :40000-40024)
# 1. Stop test container (don't touch user's working container on :40000-40025)
docker stop wh-test 2>/dev/null || true

# 2. Rebuild
./scripts/build.sh webharbor:dev

# 3. Run on alt ports
docker run -d --rm --name wh-test \
-p 8201:8101 -p 41000-41024:40000-40024 webharbor:dev
-p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev

# 4. Reset your site and confirm byte-identity
curl -X POST http://localhost:8201/reset/<your_site>
Expand Down Expand Up @@ -149,7 +149,7 @@ After Phase 3:
- Every task in `tasks.jsonl` is hand-verified to work end-to-end
- The mirror has no obvious leaks, broken forms, or empty pages
- Byte-identical reset passes
- All 25 sites still return 200
- All 26 sites still return 200

## Next step

Expand Down
2 changes: 1 addition & 1 deletion .claude/skills/harden-env/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Hardening changes seed data. After any DB change:
```bash
./scripts/build.sh webharbor:dev
docker run -d --rm --name wh-test \
-p 8201:8101 -p 41000-41024:40000-40024 webharbor:dev
-p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev
curl -X POST http://localhost:8201/reset/<your_site>
docker exec wh-test md5sum \
/opt/WebSyn/<your_site>/instance/<your_site>.db \
Expand Down
10 changes: 5 additions & 5 deletions .claude/skills/review-env/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,18 @@ gh pr checkout <pr-number>
./scripts/fetch_assets.sh # pull the pinned HF revision
./scripts/build.sh webharbor:dev
docker run -d --rm --name wh-review \
-p 8201:8101 -p 41000-41024:40000-40024 webharbor:dev
-p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev
```

Confirm the new/changed site is on the expected port (40000 + index). Note: the image now runs 25 sites (40000-40024).
Confirm the new/changed site is on the expected port (40000 + index). Note: the image now runs 26 sites (40000-40025).

### Step 2: The mechanical checks (5 minutes)

Run the same Pre-PR checks the contributor was supposed to run.

```bash
# 1. all 25 sites return 200
for p in $(seq 41000 41024); do
# 1. all 26 sites return 200
for p in $(seq 41000 41025); do
curl -so /dev/null -w "$p:%{http_code}\n" http://localhost:$p/
done

Expand Down Expand Up @@ -231,7 +231,7 @@ Leave a structured comment on the PR:
## Review: <site_name>

### Mechanical checks: PASS / FAIL
- [x] All 25 sites return 200
- [x] All 26 sites return 200
- [x] Control plane healthy
- [x] Byte-identical reset (md5 match)
- [x] Parallel reset <10s
Expand Down
6 changes: 3 additions & 3 deletions .claude/skills/seed-database/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,10 +169,10 @@ gh pr create
./scripts/check_assets.sh # every site has instance_seed/
./scripts/build.sh webharbor:dev # docker build succeeds
docker run -d --rm --name wh-test \
-p 8201:8101 -p 41000-41024:40000-40024 webharbor:dev
-p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev

# all 25 sites return 200
for p in $(seq 41000 41024); do
# all 26 sites return 200
for p in $(seq 41000 41025); do
curl -so /dev/null -w "$p:%{http_code}\n" http://localhost:$p/
done

Expand Down
10 changes: 5 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,17 +48,17 @@ Inside the image, sites live at `/opt/WebSyn/<site>/`. The path predates the ren
# fresh clone
./scripts/fetch_assets.sh # pulls assets from HF
./scripts/build.sh # docker build -t webharbor:dev .
docker run -d -p 8101:8101 -p 40000-40024:40000-40024 webharbor:dev
docker run -d -p 8101:8101 -p 40000-40025:40000-40025 webharbor:dev
```

Or use the published image directly:

```bash
docker run -d -p 8101:8101 -p 40000-40024:40000-40024 \
docker run -d -p 8101:8101 -p 40000-40025:40000-40025 \
battalion7244/webharbor:latest
```

Sites are on `40000`-`40024` in the order declared by `SITES=( ... )` in `websyn_start.sh`. Control plane:
Sites are on `40000`-`40025` in the order declared by `SITES=( ... )` in `websyn_start.sh`. Control plane:

| Method | Path | Purpose |
|--------|---------------------|-------------------------------------------|
Expand Down Expand Up @@ -136,13 +136,13 @@ python3 -m py_compile sites/<site>/app.py

# 3. run on alt ports (don't collide with anything you already have running)
docker run -d --rm --name wh-test \
-p 8201:8101 -p 41000-41024:40000-40024 webharbor:dev
-p 8201:8101 -p 41000-41025:40000-40025 webharbor:dev

# 4. control plane healthy, all sites alive
curl -s http://localhost:8201/health | python3 -m json.tool | head

# 5. every site renders 200
for p in $(seq 41000 41024); do
for p in $(seq 41000 41025); do
curl -so /dev/null -w "$p:%{http_code}\n" http://localhost:$p/
done

Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ The full agent guide is loaded above via `@AGENTS.md`. The notes below apply onl

## Existing containers

If a container is already running on `:8101` / `:40000-40024`, treat it as the user's working environment — don't `docker stop` or `docker rm` it without explicit confirmation. Spin up your test container under a different name on alt ports (`:8201`, `:41000-41024`).
If a container is already running on `:8101` / `:40000-40025`, treat it as the user's working environment — don't `docker stop` or `docker rm` it without explicit confirmation. Spin up your test container under a different name on alt ports (`:8201`, `:41000-41025`).
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ git clone https://github.com/<you>/webharbor && cd webharbor
./scripts/fetch_assets.sh # pull current assets
./scripts/new_site.py mywebsite # OR edit an existing site
./scripts/build.sh && docker run -d --rm \
-p 8101:8101 -p 40000-40024:40000-40024 webharbor:dev
-p 8101:8101 -p 40000-40025:40000-40025 webharbor:dev
# iterate locally...

./scripts/extract_assets.sh ../webharbor-static-pr/ # split assets out
Expand Down
12 changes: 10 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# WebHarbor — slim, self-contained image.
# 25 Flask mirror sites + control plane on :8101.
# 26 Flask mirror sites + control plane on :8101.

FROM python:3.12-slim-bookworm

Expand Down Expand Up @@ -56,6 +56,14 @@ RUN python3 /opt/check_asset_inventory.py /opt/WebSyn/fedex
RUN cd /opt/WebSyn/fedex && rm -rf instance instance_seed && \
PYTHONHASHSEED=0 python seed_data.py && rm -rf instance

# WebMD Doctor's generated avatars / posters come from the pinned asset bundle,
# while its SQLite seed is rebuilt deterministically from tracked source code.
# The inventory gate enforces exact coverage + per-file SHA-256 + PNG decode of
# all 317 generated images (same contract as the compass / walmart inventories).
RUN python3 /opt/WebSyn/webmd_doctor/check_generated_assets.py
RUN cd /opt/WebSyn/webmd_doctor && rm -rf instance instance_seed && \
PYTHONHASHSEED=0 python seed_data.py && rm -rf instance __pycache__

COPY websyn_start.sh /opt/websyn_start.sh
COPY control_server.py /opt/control_server.py
COPY site_runner.py /opt/site_runner.py
Expand All @@ -78,6 +86,6 @@ os.makedirs('instance_seed', exist_ok=True); \
shutil.copy2('instance/rotten_tomatoes.db', 'instance_seed/rotten_tomatoes.db'); \
print('Rotten Tomatoes seed DB generated at build time.')" && rm -rf /opt/WebSyn/rotten_tomatoes/instance

EXPOSE 8101 40000-40024
EXPOSE 8101 40000-40025

CMD ["/opt/websyn_start.sh"]
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,17 @@ WebHarbor takes a different approach. We leverage coding agent (e.g., Claude Cod
- **Deep features unlocked** — carts, checkouts, accounts, all fully testable
- **Evolving** — harder tasks drive richer mirrors; the environment grows with agents
- **RL-ready** — sub-second database resets between rollouts
- **Community-driven** — 25 sites today, scaling to 100+ together
- **Community-driven** — 26 sites today, scaling to 100+ together

## 🚀 Quickstart

One command to run all web environments:

```bash
docker run -p 8101:8101 -p 40000-40024:40000-40024 battalion7244/webharbor:latest
docker run -p 8101:8101 -p 40000-40025:40000-40025 battalion7244/webharbor:latest
```

Then point your agent at `http://localhost:40000` through `http://localhost:40024` to explore 25 local mirrors of WebVoyager sites: `Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, GitHub, Google Flights, Google Maps, Google Search, Hugging Face, Wolfram Alpha, Cambridge Dictionary, Coursera, ESPN, Merriam-Webster, IKEA, Phys.org, Target, TED, Ohio State University, Rotten Tomatoes, Compass, Walmart Careers, and FedEx`.
Then point your agent at `http://localhost:40000` through `http://localhost:40025` to explore 26 local mirrors of WebVoyager sites: `Allrecipes, Amazon, Apple, ArXiv, BBC News, Booking, GitHub, Google Flights, Google Maps, Google Search, Hugging Face, Wolfram Alpha, Cambridge Dictionary, Coursera, ESPN, Merriam-Webster, IKEA, Phys.org, Target, TED, Ohio State University, Rotten Tomatoes, Compass, Walmart Careers, FedEx, and WebMD Doctor`.

For sub-second reset between rollouts, expose the control plane and call `/reset/<site>`:

Expand Down
2 changes: 1 addition & 1 deletion agent_demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ export OPENAI_BASE_URL=https://api.openai.com/v1 # or your Azure / vLLM endpoi

## Run a task

WebHarbor must already be running locally (`docker run -p 8101:8101 -p 40000-40024:40000-40024 battalion7244/webharbor:latest`).
WebHarbor must already be running locally (`docker run -p 8101:8101 -p 40000-40025:40000-40025 battalion7244/webharbor:latest`).

Run a single task from a site's `tasks.jsonl`:

Expand Down
1 change: 1 addition & 0 deletions control_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
'huggingface', 'wolfram_alpha', 'cambridge_dictionary',
'coursera', 'espn', 'merriam_webster', 'ikea', 'phys_org', 'target',
'ted', 'osu', 'rotten_tomatoes', 'compass', 'walmart_careers', 'fedex',
'webmd_doctor',
]
BASE_PORT = 40000
WEBSYN_DIR = '/opt/WebSyn'
Expand Down
3 changes: 3 additions & 0 deletions scripts/check_assets.sh
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ for site in sites/*/; do
if [[ -f "sites/$s/asset_inventory.json" ]]; then
python3 scripts/check_asset_inventory.py "sites/$s"
fi
if [[ -f "sites/$s/generated_asset_inventory.json" && -f "sites/$s/check_generated_assets.py" ]]; then
python3 "sites/$s/check_generated_assets.py"
fi
done

if (( missing > 0 )); then
Expand Down
2 changes: 1 addition & 1 deletion sites/fedex/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# FedEx

An offline FedEx mirror contributed in [WebHarbor #50](https://github.com/aiming-lab/WebHarbor/pull/50) and revised in [WebHarbor #82](https://github.com/aiming-lab/WebHarbor/pull/82), with package tracking, rate estimates, a shipping flow, pickup scheduling, a location directory, local support articles, accounts, shipment history, invoices and claims. FedEx uses port **40024** in the current 25-site registry.
An offline FedEx mirror contributed in [WebHarbor #50](https://github.com/aiming-lab/WebHarbor/pull/50) and revised in [WebHarbor #82](https://github.com/aiming-lab/WebHarbor/pull/82), with package tracking, rate estimates, a shipping flow, pickup scheduling, a location directory, local support articles, accounts, shipment history, invoices and claims. FedEx uses port **40024** in the current 26-site registry.

## Data and assets

Expand Down
4 changes: 4 additions & 0 deletions sites/rotten_tomatoes/tests/test_environment_quality.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ def test_task_manifest_and_registry(self):
sites = re.search(r'SITES=\((.*?)\)', startup, re.S).group(1).split()
self.assertIn('rotten_tomatoes', sites)
self.assertEqual(40000 + sites.index('rotten_tomatoes'), 40021)
self.assertIn('fedex', sites)
self.assertEqual(40000 + sites.index('fedex'), 40024)
self.assertIn('webmd_doctor', sites)
self.assertEqual(40000 + sites.index('webmd_doctor'), 40025)
self.assertIn("'ted', 'osu', 'rotten_tomatoes'", (ROOT / 'control_server.py').read_text())
self.assertIn(f'40000-{40000 + len(sites) - 1}', (ROOT / 'Dockerfile').read_text())
self.assertTrue((SITE / '.build-generated-seed').is_file())
Expand Down
2 changes: 1 addition & 1 deletion sites/walmart_careers/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Walmart Careers mirror

This directory contains an offline Flask mirror modeled on `https://careers.walmart.com`. In the 24-site registry it runs on container port `40023`. All jobs, stores, requisition identifiers, user accounts, saved roles and applications are deterministic synthetic benchmark data.
This directory contains an offline Flask mirror modeled on `https://careers.walmart.com`. In the 26-site registry it runs on container port `40023`. All jobs, stores, requisition identifiers, user accounts, saved roles and applications are deterministic synthetic benchmark data.

## Runtime

Expand Down
11 changes: 7 additions & 4 deletions sites/walmart_careers/tests/test_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"cambridge_dictionary", "coursera", "espn", "merriam_webster", "ikea", "phys_org",
"target", "ted", "osu", "rotten_tomatoes", "compass",
]
FIXED_PORTS = {"rotten_tomatoes": 40021, "compass": 40022, "walmart_careers": 40023, "fedex": 40024}
FIXED_PORTS = {"rotten_tomatoes": 40021, "compass": 40022, "walmart_careers": 40023, "fedex": 40024, "webmd_doctor": 40025}


def registered_sites() -> list[str]:
Expand Down Expand Up @@ -54,7 +54,7 @@ def control_sites():
def test_registry_is_consistent_and_preserves_assigned_ports():
sites = registered_sites()
assert sites[:len(PREFIX_SITES)] == PREFIX_SITES
assert "walmart_careers" in sites and "fedex" in sites
assert "walmart_careers" in sites and "fedex" in sites and "webmd_doctor" in sites
assert len(sites) == len(set(sites)), "duplicate site in the registry"
for site, port in FIXED_PORTS.items():
assert site_port(site) == port, f"{site} moved to {site_port(site)}, expected {port}"
Expand All @@ -68,6 +68,9 @@ def test_docker_preserves_build_gates_for_every_inventoried_site():
for site in ("compass", "walmart_careers", "fedex"):
assert f"check_asset_inventory.py /opt/WebSyn/{site}" in text, site
assert f"cd /opt/WebSyn/{site}" in text, site
# WebMD Doctor ships its own generated-asset gate plus a source-built seed.
assert "/opt/WebSyn/webmd_doctor/check_generated_assets.py" in text
assert "cd /opt/WebSyn/webmd_doctor" in text
assert "walmart_careers/check_tracked_assets.py" in text
for site in ("osu", "rotten_tomatoes"):
assert f"cd /opt/WebSyn/{site}" in text, site
Expand All @@ -85,7 +88,7 @@ def test_tasks_and_verifiers_are_complete_and_use_site_24():
def test_assets_pin_is_immutable_merged_revision():
text = (ROOT / ".assets-revision").read_text()
revision = re.search(r"^revision:\s*([0-9a-f]+)$", text, re.M).group(1)
assert revision == "68dcbf2cbd0cbc2fbf97dcb71da13ace67c11bf6"
assert revision == "ad6f424f72cada9e6f5c09a58093d0ceeab9c52b"
assert (SITE / ".build-generated-seed").is_file()
assert (SITE / ".requires-images").is_file()
assert (SITE / "asset_inventory.json").is_file()
Expand All @@ -94,7 +97,7 @@ def test_assets_pin_is_immutable_merged_revision():

def test_shared_documentation_uses_the_current_site_range():
current = port_range()
stale = {f"40000-400{end}" for end in range(20, 24)} - {current}
stale = {f"40000-400{end}" for end in range(20, 25)} - {current}
for relative in ["README.md", "AGENTS.md", "CONTRIBUTING.md", "CLAUDE.md", "agent_demo/README.md"]:
text = (ROOT / relative).read_text()
for old in stale:
Expand Down
1 change: 1 addition & 0 deletions sites/webmd_doctor/.build-generated-seed
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The Dockerfile generates instance_seed/webmd_doctor.db deterministically from tracked source code (seed_data.py); static/images/ ships from the pinned Hugging Face archive.
2 changes: 2 additions & 0 deletions sites/webmd_doctor/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Local build/inspection helpers (ground truth, assertions, audits): never shipped with the site.
scripts_dev/
1 change: 1 addition & 0 deletions sites/webmd_doctor/.requires-images
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
WebMD Doctor requires its generated initials avatars and video poster frames (static/images/{avatars,posters}/) from the pinned Hugging Face archive.
Loading