Automated CLI + VS Code screenshots, per-diagnostic /errors/ pages, and Zed README fix

.github/workflows/ci.yml

@@ -66,7 +66,10 @@ jobs:
             # `website` gates the site build: only files the published site is
             # actually built from (templates/assets + the benchmark CSVs it reads).
             case "$f" in
-              website/*|benchmarks/status/*) website=true ;;
+              # The site build, plus the checker rule sources the diagnostic
+              # data (website/src/_data/rules.json) and /errors/ pages are
+              # generated from — so a new/renamed rule re-runs the drift guard.
+              website/*|benchmarks/status/*|crates/basilisk-checker/src/rules/*) website=true ;;
             esac
             # `code` gates the heavy Rust/extension/mutation matrix. Everything
             # that never reaches the Rust toolchain is excluded: the whole site,
@@ -113,6 +116,15 @@ jobs:
         working-directory: website
         run: npm ci
 
+      # The /errors/ pages and the rules reference are generated from the checker
+      # source ([WEBSITE-ERROR-PAGES]); fail if the committed data is stale so the
+      # pages the CLI deep-links to can never drift from the diagnostics it emits.
+      - name: Check generated diagnostic data is in sync with the checker
+        run: |
+          python3 scripts/gen_rules_reference.py --data /tmp/rules.json
+          diff -u website/src/_data/rules.json /tmp/rules.json \
+            || { echo "::error::rules.json is stale — run: python3 scripts/gen_rules_reference.py --data"; exit 1; }
+
       - name: Build site
         working-directory: website
         # GITHUB_TOKEN raises the GitHub API rate limit for _data/releases.js
@@ -122,13 +134,16 @@ jobs:
           GITHUB_TOKEN: ${{ github.token }}
         run: npm run build
 
-      # Navigation smoke tests ([WEBSITE-E2E-SMOKE]). Both presets (Desktop
-      # Chrome + Pixel 5) run on Chromium, so only chromium is installed.
+      # Navigation smoke tests ([WEBSITE-E2E-SMOKE]) and CLI-screenshot render
+      # checks ([WEBSITE-SCREENSHOTS-VERIFY]). Both presets (Desktop Chrome +
+      # Pixel 5) run on Chromium, so only chromium is installed. The screenshots
+      # are committed, regenerated locally with `npm run screenshots` against the
+      # real binary — CI only verifies they render, it never captures them.
       - name: Install Playwright browser
         working-directory: website
         run: npx playwright install --with-deps chromium
 
-      - name: Run navigation smoke tests (desktop + mobile)
+      - name: Run navigation + screenshot smoke tests (desktop + mobile)
         working-directory: website
         # CI uses the stdout `list` reporter only — no HTML report, trace,
         # video or screenshot is produced or uploaded ([GITHUB-NO-ARTIFACTS]).

CLAUDE.md

@@ -196,17 +196,39 @@ Please register before starting work.
 
 ## Generating CLI Screenshots (real `basilisk check` output)
 
-Marketing/doc screenshots of CLI output must be **real screen captures of the actual binary**, never hand-typed code fences or synthetic renders (those drift and are usually inaccurate). Canonical location: `website/src/assets/images/` (referenced as `/assets/images/<name>.png`). Rule screenshots are named after the code (`e0001.png` … `e0025.png`); the homepage demo pair is `cli-demo.png` (errors) + `cli-clean.png` (pass).
+Marketing/doc screenshots of CLI output must be **real output of the actual binary**, never hand-typed code fences or synthetic renders (those drift and are usually inaccurate). Canonical location: `website/src/assets/images/` (referenced as `/assets/images/<name>.png`). Rule screenshots are named after the code (`e0001.png` … `e0025.png`); the homepage demo pair is `cli-demo.png` (errors) + `cli-clean.png` (pass).
 
-Process (macOS, Terminal.app + `screencapture` + ImageMagick):
+These are now generated **automatically** — no manual Terminal.app / `screencapture` / ImageMagick. From `website/`:
 
-1. **Verify the example first.** Many rule examples do NOT trigger the rule they claim — always run `basilisk check` on the snippet and confirm the *exact* target code appears before screenshotting. E.g. E0003/E0005 fire on empty collections (`data = []`), not plain literals; E0014 needs a single annotated assignment (`count: int = "zero"`); E0016 needs `@override` (else it's E0025); E0018 fires on an undefined name in a `return`. Craft minimal snippets that isolate the target code.
-2. **No PII.** Run from a neutrally-named dir (`/tmp/basilisk-demo`, so the title bar reads "basilisk-demo", not the home-dir/username) and set a clean prompt (`export PS1='$ '`) so no username/host appears. Reference relative filenames so diagnostic paths stay clean (`e0001.py:1:13`).
-3. **Drive Terminal deterministically.** Open an empty window, size it (`set number of columns/rows`), then run `cd /tmp/basilisk-demo; export PS1='$ '; clear` followed by `basilisk check <file>.py` *in that window*. Resize BEFORE typing (resizing mid-type corrupts the line).
-4. **Capture the exact window by CGWindowID** (not a screen region — overlapping windows contaminate region captures). Get the frontmost Terminal window id via JXA/CoreGraphics (`CGWindowListCopyWindowInfo`, first owner=="Terminal" layer 0), then `screencapture -x -o -l<id>`. Close stale Terminal windows first so the wrong one isn't picked.
-5. **Crop** with ImageMagick — flatten the rounded-corner alpha onto the terminal background, then trim: `magick raw.png -background 'srgb(30,30,30)' -alpha remove -alpha off -fuzz 6% -trim +repage out.png`.
+```bash
+npm run screenshots                       # regenerate every image
+node screenshots/generate.mjs e0001 e0012 # a subset by name
+BASILISK_BIN=../target/release/basilisk npm run screenshots  # pin the binary
+```
+
+`screenshots/generate.mjs` runs the real `basilisk check --color always` on each snippet in `screenshots/shots.mjs` (in a throwaway, neutrally-named temp dir so paths read `e0001.py:1:13` with no PII), **asserts the documented diagnostic actually fires** (the snippet→code pairing lives in the manifest, so a checker change can't silently ship a misleading image), then renders the genuine coloured output in a faithful macOS Terminal window via Playwright. See `[WEBSITE-SCREENSHOTS]` (`docs/specs/WEBSITE-SCREENSHOTS-SPEC.md`).
+
+To add/change a screenshot, edit `screenshots/shots.mjs` (snippet + expected code) and rerun — never craft images by hand. The committed PNGs are regenerated locally when CLI output changes; CI only verifies they render (`website/tests/e2e/screenshots.spec.ts`, `[WEBSITE-SCREENSHOTS-VERIFY]`), never captures, per `[GITHUB-NO-ARTIFACTS]`. After regenerating, rebuild (`npm run build`) and confirm the images copy to `_site/assets/images/`. VSIX integration tests capture their own editor screenshots to the gitignored `vscode-extension/.screenshots/` (never committed, never a CI artifact).
+
+## Per-diagnostic error pages (`/errors/BSK-XXXX/`)
+
+Every diagnostic the CLI prints ends with `see: https://www.basilisk-python.dev/errors/BSK-XXXX` (the `docs_url` on each rule's `ErrorCode`). Those pages are **generated for all codes** from the checker source — see `[WEBSITE-ERROR-PAGES]` (`docs/specs/WEBSITE-ERROR-PAGES-SPEC.md`). The single source is `website/src/_data/rules.json`, produced by:
+
+```bash
+python3 scripts/gen_rules_reference.py --data   # writes website/src/_data/rules.json
+```
+
+It extracts the `//! BSK-XXXX:` summary + doc-comment body (prose and ```python examples) from each `crates/basilisk-checker/src/rules/*.rs`. **After adding or renaming a rule, rerun it** — CI fails otherwise: the website job regenerates and `diff`s `rules.json` (`[WEBSITE-ERROR-PAGES-DRIFT]`), and rule-source edits are classified as website changes so the guard runs. The same data drives the `/docs/rules/` table and counts (no hand-maintained code lists). Pages render via `website/src/errors/error.njk`; a worked-example screenshot appears automatically for any code present in `screenshots/shots.mjs`.
+
+## VS Code editor screenshots (`vscode-*.png`)
+
+Real screenshots of the extension running in VS Code (diagnostics, hover, quick-fix, activity panel) are captured automatically — see `[VSIX-EDITOR-SCREENSHOTS]` (`docs/specs/VSIX-EDITOR-SCREENSHOTS-SPEC.md`). From `vscode-extension/`, after `cargo build -p basilisk-cli -p basilisk-profiler-helper`:
+
+```bash
+npm run screenshots:editor
+```
 
-After generating, rebuild (`cd website && npm run build`) and confirm the images copy to `_site/assets/images/` and render. VSIX integration tests capture their own editor screenshots to the gitignored `vscode-extension/.screenshots/` (never committed, never a CI artifact — see [GITHUB-NO-ARTIFACTS]).
+This stages the binary into the dev extension, copies `shipwright.json` (gitignored dev artifacts), launches the **headed** "Editor screenshots" suite with `BASILISK_SCREENSHOTS=1`, and a dependency-free CDP sidecar (`screenshot-watcher.mjs`, Node's built-in WebSocket — no Playwright) captures the window to `website/src/assets/images/vscode-*.png`. The suite is a no-op without the env flag, so normal `npm test` never opens these windows. To add one, add a `test(...)` that makes the feature visible and calls `takeWindowScreenshot(...)`. As with the CLI shots, PNGs are committed and regenerated locally; CI only verifies they render (`website/tests/e2e/screenshots.spec.ts`), per `[GITHUB-NO-ARTIFACTS]`.
 
 ## Architecture
 

basilisk-zed/README.md

@@ -5,7 +5,7 @@
 Zed editor extension for Basilisk — WASM-based Python type checking and language server integration.
 
 <p align="center">
-  <img src="images/screenshot.png" alt="Basilisk in action — type checking, diagnostics, and refactoring in the editor" width="900">
+  <img src="images/zed-screenshot.png" alt="Basilisk in the Zed editor — Python type checking and diagnostics inline" width="900">
 </p>
 
 ## Role in Basilisk

basilisk-zed/README.zh.md

@@ -7,7 +7,7 @@
 Basilisk 的 Zed 编辑器扩展 —— 基于 WASM 的 Python 类型检查与语言服务器集成。
 
 <p align="center">
-  <img src="images/screenshot.png" alt="Basilisk in action — type checking, diagnostics, and refactoring in the editor" width="900">
+  <img src="images/zed-screenshot.png" alt="Zed 编辑器中的 Basilisk —— 行内 Python 类型检查与诊断" width="900">
 </p>
 
 ## 在 Basilisk 中的角色

basilisk-zed/images/zed-screenshot.png

@@ -0,0 +1 @@
+../../website/src/assets/images/zed-screenshot.png

docs/INDEX.md

@@ -34,6 +34,9 @@ Specifications define the target behavior and architecture. They are the source
 | [LSP-TEST-INTEGRATION-SPEC.md](specs/LSP-TEST-INTEGRATION-SPEC.md) | Test discovery, execution, and editor integration — pytest/unittest, TestItem model, coverage overlay. |
 | [EXTENSION-ACTIVITY-PANEL-SPEC.md](specs/EXTENSION-ACTIVITY-PANEL-SPEC.md) | Cross-editor activity panel — module explorer, type health, feature dashboard (VS Code, Zed, Neovim). |
 | [WEBSITE-E2E-SPEC.md](specs/WEBSITE-E2E-SPEC.md) | Website navigation/e2e smoke tests (Playwright, desktop + mobile) — top-nav resolution, docs sidebar, and the mobile docs-submenu reachability guard. |
+| [WEBSITE-SCREENSHOTS-SPEC.md](specs/WEBSITE-SCREENSHOTS-SPEC.md) | Automated CLI screenshots — `npm run screenshots` runs the real binary on each documented snippet, self-verifies the diagnostic fires, and renders it in a faithful Terminal window (no manual screencapture). |
+| [WEBSITE-ERROR-PAGES-SPEC.md](specs/WEBSITE-ERROR-PAGES-SPEC.md) | Per-diagnostic `/errors/BSK-XXXX/` pages generated from the checker source so every CLI `see:` link resolves — with severity, explanation, worked example, drift guard, and render verification. |
+| [VSIX-EDITOR-SCREENSHOTS-SPEC.md](specs/VSIX-EDITOR-SCREENSHOTS-SPEC.md) | Automated real VS Code editor screenshots (`npm run screenshots:editor`) — drives the extension headed, captures the window over CDP (no Playwright dep), embeds diagnostics/hover/quick-fix/activity-panel on the docs. |
 
 ## Plans
 

docs/specs/VSIX-EDITOR-SCREENSHOTS-SPEC.md

@@ -0,0 +1,80 @@
+# VS Code Editor Screenshots {#VSIX-EDITOR-SCREENSHOTS}
+
+**Version**: 0.1.0
+**Status**: Active
+**License**: MIT
+
+---
+
+## Purpose {#VSIX-EDITOR-SCREENSHOTS-PURPOSE}
+
+The website needs **real screenshots of the Basilisk extension running in VS Code**
+— diagnostics with squiggles, the hover type popup, the Quick Fix menu, the
+activity panel — not mockups. This spec defines an automated pipeline that drives
+the actual extension in a headed VS Code instance and captures the real window,
+so the marketing/docs editor screenshots are genuine and reproducible.
+
+It is the editor-side companion to the CLI pipeline ([WEBSITE-SCREENSHOTS]): both
+produce committed PNGs from the real product and are verified (not captured) in
+CI, honouring [GITHUB-NO-ARTIFACTS].
+
+## Capture pipeline {#VSIX-EDITOR-SCREENSHOTS-PIPELINE}
+
+Prerequisite — build the binaries the dev extension resolves:
+
+```bash
+cargo build -p basilisk-cli -p basilisk-profiler-helper
+```
+
+Then, from `vscode-extension/`:
+
+```bash
+npm run screenshots:editor
+```
+
+`scripts/capture-screenshots.mjs` orchestrates one run:
+
+1. Stages the built binaries into `vscode-extension/bin/<platform>/` via
+   `stage-runtime.mjs` (the same path the packaged VSIX and the extension's
+   shipwright resolver use) and mirrors the repo-root `shipwright.json` into the
+   extension (both are gitignored dev artifacts).
+2. Compiles the extension and launches the **"Editor screenshots"** suite
+   (`src/test/suite/screenshots-capture.test.ts`) headed, with
+   `BASILISK_SCREENSHOTS=1`. The harness (`.vscode-test.mjs`) then adds
+   `--remote-debugging-port` so the window is reachable over CDP.
+3. Runs the sidecar `scripts/screenshot-watcher.mjs`, which speaks the
+   Chrome DevTools Protocol over Node's **built-in WebSocket** (no Playwright
+   dependency, no browser download), forces a uniform Retina viewport
+   (1440×900 @2×), and captures the workbench page on demand.
+
+Each test drives a feature until it is visible, strips transient chrome
+(`notifications.clearAll`, close the Chat auxiliary bar), then calls
+`takeWindowScreenshot(name)` (`screenshot.ts`), which hands the sidecar a
+`.signal` file and waits for the PNG. Output lands in
+`website/src/assets/images/vscode-*.png`.
+
+The suite is a **no-op unless `BASILISK_SCREENSHOTS=1`** (it `skip()`s in
+suiteSetup), so a normal `npm test` run never opens these windows or writes into
+the repo.
+
+## Captured set {#VSIX-EDITOR-SCREENSHOTS-SET}
+
+| Image | Feature | Embedded on |
+|---|---|---|
+| `vscode-diagnostics.png` | Inline squiggles + Problems panel | `/docs/install-vscode/` |
+| `vscode-hover.png` | Hover type popup | `/docs/quick-start/` |
+| `vscode-quickfix.png` | Quick Fix / code-action menu | `/docs/refactoring/` |
+| `vscode-module-explorer.png` | Basilisk activity panel | `/docs/` |
+
+Add a capture by adding a `test(...)` to the suite that makes the feature visible
+and calls `takeWindowScreenshot('vscode-<name>.png')`.
+
+## Verification {#VSIX-EDITOR-SCREENSHOTS-VERIFY}
+
+The committed PNGs are regenerated locally (the capture needs a headed VS Code and
+the built binary — it does not run in CI). CI only verifies they render:
+`website/tests/e2e/screenshots.spec.ts` visits each embedding page and asserts the
+`vscode-*.png` is present and decodes to non-zero pixels — so a missing or
+zero-byte capture fails the build. No screenshot is ever produced or uploaded by
+CI ([GITHUB-NO-ARTIFACTS]); the legacy gitignored full-screen capture in
+`screenshot.ts` (`captureScreenshot`) remains for local debugging only.

docs/specs/WEBSITE-ERROR-PAGES-SPEC.md

@@ -0,0 +1,73 @@
+# Website: Per-Diagnostic Error Pages {#WEBSITE-ERROR-PAGES}
+
+**Version**: 0.1.0
+**Status**: Active
+**License**: MIT
+
+---
+
+## Purpose {#WEBSITE-ERROR-PAGES-PURPOSE}
+
+Every diagnostic Basilisk reports ends with a deep link:
+
+```
+   = see: https://www.basilisk-python.dev/errors/BSK-E0001
+```
+
+That URL is baked into each rule (`docs_url` on the `ErrorCode` in
+`crates/basilisk-checker/src/rules/*.rs`). Before this spec **none of those pages
+existed** — every "learn more" link the CLI printed (155 codes) was a 404.
+
+This spec defines a generated landing page for **every** diagnostic code at
+`/errors/BSK-XXXX/`, built from the checker source so the pages can never drift
+from the diagnostics the binary emits. It turns the tool's own output into a
+navigable, explained reference.
+
+## Data generation {#WEBSITE-ERROR-PAGES-DATA}
+
+`scripts/gen_rules_reference.py --data` extracts one record per code from the
+`//! BSK-XXXX: …` doc-comment header — and the prose/`​```python` examples beneath
+it — on each rule module, and writes `website/src/_data/rules.json`:
+
+```json
+{ "code", "severity", "summary", "summaryHtml", "body": [{type:"text"|"code"}], "group", "docsUrl" }
+```
+
+`docsUrl` is read from the rule's own `docs_url` literal, so the page URL and the
+CLI link are guaranteed identical. The same data drives the complete reference
+table on `/docs/rules/` and the headline counts (`_data/ruleStats.js`,
+`_data/ruleGroups.js`), so the prose, the table, and the pages share one source.
+
+### Drift guard {#WEBSITE-ERROR-PAGES-DRIFT}
+
+The CI website job regenerates the data and `diff`s it against the committed
+`rules.json`, failing if they differ; the change-classifier treats edits under
+`crates/basilisk-checker/src/rules/` as website changes so adding or renaming a
+rule re-runs the guard. A new rule therefore cannot ship without its page.
+
+## Pages {#WEBSITE-ERROR-PAGES-PAGES}
+
+`website/src/errors/error.njk` paginates `rules` (size 1) to emit
+`/errors/{{ code }}/` for every record. Each page shows the code, a severity
+badge, the summary, the doc-comment body (text + code blocks), a worked
+`basilisk check` screenshot when one exists, how-to-handle guidance, and the
+canonical `docsUrl`. `website/src/errors/index.njk` is a grouped, browsable
+directory of all codes. Pages deliberately omit `eleventyNavigation` so the 160
+entries never flood the docs sidebar.
+
+### Worked examples {#WEBSITE-ERROR-PAGES-EXAMPLES}
+
+`_data/examples.js` maps a code to the screenshot that demonstrates it by reading
+the screenshot manifest's `expect` field ([WEBSITE-SCREENSHOTS-MANIFEST]) — so the
+mapping is correct even where the image stem and code differ (e.g. `e0011.png`
+demonstrates `BSK-W0014`). Adding a verified shot to `screenshots/shots.mjs` and
+regenerating is all it takes for a code's page to gain a real-output example.
+
+## Verification {#WEBSITE-ERROR-PAGES-VERIFY}
+
+`website/tests/e2e/errors.spec.ts` (same Playwright config as [WEBSITE-E2E-SMOKE])
+asserts: every code in `rules.json` has a built `/errors/<code>/index.html`
+(≥ 155 — every CLI-linked code); a sampled page renders its code, title and
+severity badge; the `/errors/` index links every code; and every worked-example
+screenshot decodes on its page. With the drift guard above, this closes the loop
+from checker source → data → page → render.