Automated CLI + VS Code screenshots, per-diagnostic /errors/ pages, and Zed README fix#196
Merged
Merged
Conversation
Replace the manual macOS Terminal.app + screencapture + ImageMagick process
with a reproducible generator. `npm run screenshots` runs the real
`basilisk check --color always` on each documented snippet in a throwaway,
PII-free temp dir, asserts the documented diagnostic actually fires, and
renders the genuine coloured output inside a faithful macOS Terminal window
via Playwright.
- screenshots/{shots,ansi,terminal,generate}.mjs — manifest + ANSI→HTML +
Terminal chrome + generator. The snippet→expected-code pairing is the
automated form of "verify the example triggers the rule".
- tests/e2e/screenshots.spec.ts — verifies every image renders on the docs
pages and homepage (imports the manifest, so it can't drift).
- All e0001–e0025 + cli-demo/cli-clean regenerated from basilisk 0.20.0.
- Spec: docs/specs/WEBSITE-SCREENSHOTS-SPEC.md [WEBSITE-SCREENSHOTS];
INDEX, CLAUDE.md and the CI website job updated.
CI only verifies the committed images render; it never captures or uploads
them ([GITHUB-NO-ARTIFACTS]).
…orked examples The CLI prints `see: https://www.basilisk-python.dev/errors/BSK-XXXX` for every diagnostic, but none of those ~155 pages existed — each was a 404. Generate a landing page per code, plus richer worked examples, all from the checker source. Error pages ([WEBSITE-ERROR-PAGES]): - scripts/gen_rules_reference.py --data extracts code, severity, summary, the doc-comment body (prose + ```python examples) and docs_url for all 160 codes into website/src/_data/rules.json (single source of truth). - src/errors/error.njk paginates it to /errors/BSK-XXXX/ (summary, severity badge, explanation, worked-example screenshot when present, fix guidance, canonical URL); src/errors/index.njk is a grouped directory. All 160 pages land in sitemap.xml. - /docs/rules/ table + counts now render from the same data (removes the hand-maintained 155-row table; fixes a stale 155 vs real 160 count). - CI drift guard: the website job regenerates rules.json and diffs it; rule source edits now classify as website changes so a new rule can't ship without its page. More worked examples ([WEBSITE-SCREENSHOTS-MANIFEST]): - Added 12 verified, isolated snippets (E0017, E0020, E0023, E0026, E0027, E0029, E0031, E0033, E0040, E0041, E0099, E0115) — worked-example coverage 15 → 27 codes. _data/examples.js maps each code to its screenshot via the manifest's `expect` field, so error pages embed real CLI output automatically. Verification: tests/e2e/errors.spec.ts asserts every code has a rendered page, the index links them all, and every worked example decodes on its page. Spec: docs/specs/WEBSITE-ERROR-PAGES-SPEC.md; INDEX + CLAUDE.md updated.
The site described editor features (diagnostics, hover, refactoring, the activity panel) in prose with no visuals — and the only VS Code capture in the repo was a gitignored, full-screen local-debug grab. Add an automated pipeline that drives the real extension headed and captures the actual window, then embed the results on the docs. Pipeline ([VSIX-EDITOR-SCREENSHOTS], vscode-extension/): - screenshot-watcher.mjs — dependency-free CDP sidecar over Node's built-in WebSocket (no Playwright, no browser download); forces a uniform 1440×900@2× viewport and captures the workbench page on a signal file. - screenshot.ts takeWindowScreenshot() — signals the sidecar, waits for the PNG; no-op unless BASILISK_SCREENSHOTS=1. - .vscode-test.mjs — adds --remote-debugging-port when capturing. - screenshots-capture.test.ts — "Editor screenshots" suite (skipped unless the env flag is set) that drives each feature, strips transient chrome, and shoots. - scripts/capture-screenshots.mjs + `npm run screenshots:editor` — stage binary, copy shipwright.json, launch sidecar, run the suite. (Prereq: cargo build -p basilisk-cli -p basilisk-profiler-helper.) Captured + embedded (real, 2880×1800): - vscode-diagnostics.png → /docs/install-vscode/ - vscode-hover.png → /docs/quick-start/ - vscode-quickfix.png → /docs/refactoring/ - vscode-module-explorer.png → /docs/ Verification: tests/e2e/screenshots.spec.ts asserts each editor screenshot is embedded and decodes. PNGs are committed and regenerated locally; CI only verifies they render, never captures ([GITHUB-NO-ARTIFACTS]). Spec: docs/specs/VSIX-EDITOR-SCREENSHOTS-SPEC.md; INDEX + CLAUDE.md updated.
The basilisk-zed README (and its published mirror) showed website/src/assets/images/screenshot.png — a VS Code window — as its hero, misrepresenting the Zed extension. Add a real Zed screenshot (website/src/assets/images/zed-screenshot.png), repoint the extension's image symlink to it, and update both READMEs (en + zh) to reference images/zed-screenshot.png with Zed-specific alt text. The render-zed-mirror.sh `-L` deref already copies the real file into the standalone mirror, so the published Nimblesite/basilisk-zed repo now ships the Zed image (verified by rendering the mirror to a temp tree). The shared screenshot.png stays for the VS Code / root / nvim READMEs that still reference it.
CI-prep fixes (no behaviour change): - ruff-format scripts/gen_rules_reference.py (matches the pinned ruff 0.15.17 `ruff format --check` gate). - Move the editor-screenshot CDP sidecar from vscode-extension/src/test/suite/screenshot-watcher.mjs to vscode-extension/scripts/screenshot-watcher.mjs: the extension's `eslint src/` project-service config rejects a stray .mjs under src/, and the scripts/ dir (where the orchestrator already lives) is not linted. Path math and references updated accordingly; the spec doc and capture-suite comment follow. Verified locally against every CI job: lint (fmt/clippy/eslint/deslop 0.5.1), website (drift+build+e2e), zed (wasm+clippy+97 tests), Rust coverage + conformance + FP gates, shipwright binary gate, VS Code e2e (the lone local failure is the macOS-only cooperative-sampler test, skipped on CI's ubuntu runner and green in isolation), and Neovim (45% ≥ 39%).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Replace hand-made/incorrect screenshots with automated pipelines that capture real CLI and VS Code output, generate a per-diagnostic
/errors/reference, and fix the Zed README's editor screenshot — all verified to render in CI.What Was Added?
website/screenshots/):npm run screenshotsruns the realbasilisk check --color alwayson each documented snippet, asserts the expected diagnostic actually fires, and renders the genuine colored output in a faithful macOS-Terminal window via Playwright (ansi.mjs,terminal.mjs,shots.mjs,generate.mjs).website/src/errors/): an Eleventy-paginated template emits/errors/BSK-XXXX/for all 160 codes fromwebsite/src/_data/rules.json, so everysee: …/errors/BSK-XXXXlink the CLI prints now resolves. Data is generated from the checker source by an extendedscripts/gen_rules_reference.py --data;_data/{examples,ruleGroups,ruleStats}.jsdrive the worked-example image, grouping, and counts.vscode-extension/):npm run screenshots:editordrives the extension headed and captures the window over a dependency-free Chrome DevTools Protocol sidecar (Node built-in WebSocket — no Playwright/browser download):scripts/screenshot-watcher.mjs,scripts/capture-screenshots.mjs,src/test/suite/screenshots-capture.test.ts,takeWindowScreenshot()inscreenshot.ts, and a CDP--remote-debugging-portflag in.vscode-test.mjs. Producesvscode-{diagnostics,hover,quickfix,module-explorer}.png, embedded on the docs.zed-screenshot.png.WEBSITE-SCREENSHOTS,WEBSITE-ERROR-PAGES,VSIX-EDITOR-SCREENSHOTS.What Was Changed or Deleted?
website/src/docs/rules/index.md: the hand-maintained ~155-row code table is replaced by a data-driven table (links each code to its/errors/page); the headline count is computed (corrects a stale 155 → real 160).install-vscode,quick-start,refactoring,index) now embed the real editor screenshots.basilisk-zed/README.md+README.zh.md: hero image was a VS Code screenshot — repointed to a real Zed screenshot (images/zed-screenshot.png);render-zed-mirror.shcopies the real file into the published mirror (verified)..github/workflows/ci.yml: the website job now regeneratesrules.jsonand fails on drift; rule-source edits classify as website changes; e2e step also covers screenshot render checks.e0*.png,cli-demo.png,cli-clean.pngregenerated from the real binary.How Do The Automated Tests Prove It Works?
website/tests/e2e/errors.spec.ts: asserts every code inrules.jsonhas a built/errors/<code>/index.html(≥155), the/errors/index links all of them, a sampled page renders its code/title/severity, and every worked-example screenshot decodes.website/tests/e2e/screenshots.spec.ts: asserts the rule-docs CLI screenshots, the homepage before/after demo, and all fourvscode-*.pngeditor screenshots are embedded and decode to non-zero pixels.gen_rules_reference.py --datais regenerated anddiffed against the committedrules.json.this.skip()-ed on CI's ubuntu runner and passes in isolation); Neovim (45% ≥ 39%). Mutation testing is unaffected (no.rschanged); CI runs the full sharded suite regardless.Spec / Doc Changes
New specs
docs/specs/{WEBSITE-SCREENSHOTS,WEBSITE-ERROR-PAGES,VSIX-EDITOR-SCREENSHOTS}-SPEC.md, indexed indocs/INDEX.md.CLAUDE.mddocuments the three regeneration workflows (npm run screenshots,gen_rules_reference.py --data,npm run screenshots:editor).Breaking Changes
BASILISK_SCREENSHOTS=1); committed images are regenerated locally and only render-verified in CI.