diff --git a/.claude/skills/playwright-cli/SKILL.md b/.claude/skills/playwright-cli/SKILL.md new file mode 100644 index 0000000..6f34a18 --- /dev/null +++ b/.claude/skills/playwright-cli/SKILL.md @@ -0,0 +1,420 @@ +--- +name: playwright-cli +description: Automate browser interactions, test web pages and work with Playwright tests. +allowed-tools: Bash(playwright-cli:*) Bash(npx:*) Bash(npm:*) +--- + +# Browser Automation with playwright-cli + +## Quick start + +```bash +# open new browser +playwright-cli open +# navigate to a page +playwright-cli goto https://playwright.dev +# interact with the page using refs from the snapshot +playwright-cli click e15 +playwright-cli type "page.click" +playwright-cli press Enter +# take a screenshot (rarely used, as snapshot is more common) +playwright-cli screenshot +# close the browser +playwright-cli close +``` + +## Commands + +### Core + +```bash +playwright-cli open +# open and navigate right away +playwright-cli open https://example.com/ +playwright-cli goto https://playwright.dev +playwright-cli type "search query" +playwright-cli click e3 +playwright-cli dblclick e7 +# --submit presses Enter after filling the element +playwright-cli fill e5 "user@example.com" --submit +playwright-cli drag e2 e8 +# drop files or data onto an element (from outside the page) +playwright-cli drop e4 --path=./image.png +playwright-cli drop e4 --data="text/plain=hello world" +playwright-cli hover e4 +playwright-cli select e9 "option-value" +playwright-cli upload ./document.pdf +playwright-cli check e12 +playwright-cli uncheck e12 +playwright-cli snapshot +# search the snapshot for text or a regexp, returns matching nodes with surrounding context +playwright-cli find "Sign in" +playwright-cli find --regex "Sign (in|up)" +# wrap the regexp in slashes to add flags, e.g. /i for case-insensitive +playwright-cli find --regex "/sign (in|up)/i" +playwright-cli eval "document.title" +playwright-cli eval "el => el.textContent" e5 +# get element id, class, or any attribute not visible in the snapshot +playwright-cli eval "el => el.id" e5 +playwright-cli eval "el => el.getAttribute('data-testid')" e5 +playwright-cli dialog-accept +playwright-cli dialog-accept "confirmation text" +playwright-cli dialog-dismiss +playwright-cli resize 1920 1080 +playwright-cli close +``` + +### Navigation + +```bash +playwright-cli go-back +playwright-cli go-forward +playwright-cli reload +``` + +### Keyboard + +```bash +playwright-cli press Enter +playwright-cli press ArrowDown +playwright-cli keydown Shift +playwright-cli keyup Shift +``` + +### Mouse + +```bash +playwright-cli mousemove 150 300 +playwright-cli mousedown +playwright-cli mousedown right +playwright-cli mouseup +playwright-cli mouseup right +playwright-cli mousewheel 0 100 +``` + +### Save as + +```bash +playwright-cli screenshot +playwright-cli screenshot e5 +playwright-cli screenshot --filename=page.png +playwright-cli screenshot --hires +playwright-cli pdf --filename=page.pdf +``` + +### Tabs + +```bash +playwright-cli tab-list +playwright-cli tab-new +playwright-cli tab-new https://example.com/page +playwright-cli tab-close +playwright-cli tab-close 2 +playwright-cli tab-select 0 +``` + +### Storage + +```bash +playwright-cli state-save +playwright-cli state-save auth.json +playwright-cli state-load auth.json + +# Cookies +playwright-cli cookie-list +playwright-cli cookie-list --domain=example.com +playwright-cli cookie-get session_id +playwright-cli cookie-set session_id abc123 +playwright-cli cookie-set session_id abc123 --domain=example.com --httpOnly --secure +playwright-cli cookie-delete session_id +playwright-cli cookie-clear + +# LocalStorage +playwright-cli localstorage-list +playwright-cli localstorage-get theme +playwright-cli localstorage-set theme dark +playwright-cli localstorage-delete theme +playwright-cli localstorage-clear + +# SessionStorage +playwright-cli sessionstorage-list +playwright-cli sessionstorage-get step +playwright-cli sessionstorage-set step 3 +playwright-cli sessionstorage-delete step +playwright-cli sessionstorage-clear +``` + +### Network + +```bash +playwright-cli route "**/*.jpg" --status=404 +playwright-cli route "https://api.example.com/**" --body='{"mock": true}' +playwright-cli route-list +playwright-cli unroute "**/*.jpg" +playwright-cli unroute +``` + +### DevTools + +```bash +playwright-cli console +playwright-cli console warning +playwright-cli requests +playwright-cli request 5 +playwright-cli run-code "async page => await page.context().grantPermissions(['geolocation'])" +playwright-cli run-code --filename=script.js +playwright-cli tracing-start +playwright-cli tracing-stop +playwright-cli video-start video.webm +playwright-cli video-chapter "Chapter Title" --description="Details" --duration=2000 +playwright-cli video-stop + +# annotate each subsequent action (click, type, ...) with a callout naming the action and highlighting the target +playwright-cli video-show-actions --duration=600 --position=top-right +playwright-cli video-hide-actions + +# launch the dashboard for UI review / design feedback — user annotates the page, you receive the annotated screenshot, snapshot, and notes +playwright-cli show --annotate + +# generate a Playwright locator for an element from its ref or selector +playwright-cli generate-locator e5 --raw + +# show a persistent highlight overlay for an element, optionally with a custom style +playwright-cli highlight e5 +playwright-cli highlight e5 --style="outline: 3px dashed red" +# hide a single element highlight, or all page highlights when no target is given +playwright-cli highlight e5 --hide +playwright-cli highlight --hide +``` + +## Raw output + +The global `--raw` option strips page status, generated code, and snapshot sections from the output, returning only the result value. Use it to pipe command output into other tools. Commands that don't produce output return nothing. + +```bash +playwright-cli --raw eval "JSON.stringify(performance.timing)" | jq '.loadEventEnd - .navigationStart' +playwright-cli --raw eval "JSON.stringify([...document.querySelectorAll('a')].map(a => a.href))" > links.json +playwright-cli --raw snapshot > before.yml +playwright-cli click e5 +playwright-cli --raw snapshot > after.yml +diff before.yml after.yml +TOKEN=$(playwright-cli --raw cookie-get session_id) +playwright-cli --raw localstorage-get theme +``` + +For structured output wrapping every reply as JSON, pass --json +```bash +playwright-cli list --json +``` + +## Open parameters +```bash +# Use specific browser when creating session +playwright-cli open --browser=chrome +playwright-cli open --browser=firefox +playwright-cli open --browser=webkit +playwright-cli open --browser=msedge + +# Emulate a generic mobile device (Pixel 10 for Chromium, iPhone 17 for WebKit). +# Prefer this when a mobile layout is acceptable: mobile pages are usually +# lighter, so snapshots are smaller and cheaper. +playwright-cli open --mobile +playwright-cli open --device="iPhone 15" + +# Use persistent profile (by default profile is in-memory) +playwright-cli open --persistent +# Use persistent profile with custom directory +playwright-cli open --profile=/path/to/profile + +# Connect to browser via Playwright Extension +playwright-cli attach --extension=chrome + +# Connect to a running Chrome or Edge by channel name +playwright-cli attach --cdp=chrome +playwright-cli attach --cdp=msedge + +# Connect to a running browser via CDP endpoint +playwright-cli attach --cdp=http://localhost:9222 + +# Start with config file +playwright-cli open --config=my-config.json + +# Close the browser +playwright-cli close +# Detach from an attached browser (leaves the external browser running) +playwright-cli -s=msedge detach +# Delete user data for the default session +playwright-cli delete-data +``` + +## URLs with `&` on Windows + +On Windows, `cmd.exe` and PowerShell treat `&` as a command separator, so URLs with multiple query parameters get truncated before `playwright-cli` runs. Escape `&` with `^&` in `cmd.exe`, or use `--%` in PowerShell: + +```batch +playwright-cli goto "https://example.com/?a=1^&b=2" +``` + +```powershell +playwright-cli --% goto "https://example.com/?a=1&b=2" +``` + +## Snapshots + +After each command, playwright-cli provides a snapshot of the current browser state. + +```bash +> playwright-cli goto https://example.com +### Page +- Page URL: https://example.com/ +- Page Title: Example Domain +### Snapshot +[Snapshot](.playwright-cli/page-2026-02-14T19-22-42-679Z.yml) +``` + +You can also take a snapshot on demand using `playwright-cli snapshot` command. All the options below can be combined as needed. + +```bash +# default - save to a file with timestamp-based name +playwright-cli snapshot + +# save to file, use when snapshot is a part of the workflow result +playwright-cli snapshot --filename=after-click.yaml + +# snapshot an element instead of the whole page +playwright-cli snapshot "#main" + +# limit snapshot depth for efficiency, take a partial snapshot afterwards +playwright-cli snapshot --depth=4 +playwright-cli snapshot e34 + +# include each element's bounding box as [box=x,y,width,height] +playwright-cli snapshot --boxes + +# search a large snapshot instead of capturing it all — returns matching nodes +# with 3 lines of context around each match (like grep -C) +playwright-cli find "Add to cart" +playwright-cli find --regex "\\$[0-9]+\\.[0-9]{2}" +``` + +## Targeting elements + +By default, use refs from the snapshot to interact with page elements. + +```bash +# get snapshot with refs +playwright-cli snapshot + +# interact using a ref +playwright-cli click e15 +``` + +You can also use css selectors or Playwright locators. + +```bash +# css selector +playwright-cli click "#main > button.submit" + +# role locator +playwright-cli click "getByRole('button', { name: 'Submit' })" + +# test id +playwright-cli click "getByTestId('submit-button')" +``` + +## Browser Sessions + +```bash +# create new browser session named "mysession" with persistent profile +playwright-cli -s=mysession open example.com --persistent +# same with manually specified profile directory (use when requested explicitly) +playwright-cli -s=mysession open example.com --profile=/path/to/profile +playwright-cli -s=mysession click e6 +playwright-cli -s=mysession close # stop a named browser +playwright-cli -s=mysession delete-data # delete user data for persistent session + +playwright-cli list +# Close all browsers +playwright-cli close-all +# Forcefully kill all browser processes +playwright-cli kill-all +``` + +## Installation + +If global `playwright-cli` command is not available, try a local version via `npx playwright cli`: + +```bash +npx --no-install playwright --version +``` + +When local version is available, use `npx playwright cli` in all commands. Otherwise, install `playwright-cli` as a global command: + +```bash +npm install -g @playwright/cli@latest +``` + +## Example: Form submission + +```bash +playwright-cli open https://example.com/form +playwright-cli snapshot + +playwright-cli fill e1 "user@example.com" +playwright-cli fill e2 "password123" +playwright-cli click e3 +playwright-cli snapshot +playwright-cli close +``` + +## Example: Multi-tab workflow + +```bash +playwright-cli open https://example.com +playwright-cli tab-new https://example.com/other +playwright-cli tab-list +playwright-cli tab-select 0 +playwright-cli snapshot +playwright-cli close +``` + +## Example: Debugging with DevTools + +```bash +playwright-cli open https://example.com +playwright-cli click e4 +playwright-cli fill e7 "test" +playwright-cli console +playwright-cli requests +playwright-cli close +``` + +```bash +playwright-cli open https://example.com +playwright-cli tracing-start +playwright-cli click e4 +playwright-cli fill e7 "test" +playwright-cli tracing-stop +playwright-cli close +``` + +## Example: Interactive session + +Ask the user for UI review or design feedback. The user draws boxes on the live page and types comments; you receive the annotated screenshot, the snapshot of the marked region, and the user's notes. Use this whenever the user asks for "UI review", "design feedback", or to "ask the user what they think / want / mean": + +```bash +playwright-cli open https://example.com +playwright-cli show --annotate +``` + +## Specific tasks + +* **Running and Debugging Playwright tests** [references/playwright-tests.md](references/playwright-tests.md) +* **Request mocking** [references/request-mocking.md](references/request-mocking.md) +* **Running Playwright code** [references/running-code.md](references/running-code.md) +* **Browser session management** [references/session-management.md](references/session-management.md) +* **Storage state (cookies, localStorage)** [references/storage-state.md](references/storage-state.md) +* **Test generation (plan / generate / heal)** [references/test-generation.md](references/test-generation.md) +* **Tracing** [references/tracing.md](references/tracing.md) +* **Video recording** [references/video-recording.md](references/video-recording.md) +* **Inspecting element attributes** [references/element-attributes.md](references/element-attributes.md) diff --git a/.claude/skills/playwright-cli/references/element-attributes.md b/.claude/skills/playwright-cli/references/element-attributes.md new file mode 100644 index 0000000..4e9fa6b --- /dev/null +++ b/.claude/skills/playwright-cli/references/element-attributes.md @@ -0,0 +1,23 @@ +# Inspecting Element Attributes + +When the snapshot doesn't show an element's `id`, `class`, `data-*` attributes, or other DOM properties, use `eval` to inspect them. + +## Examples + +```bash +playwright-cli snapshot +# snapshot shows a button as e7 but doesn't reveal its id or data attributes + +# get the element's id +playwright-cli eval "el => el.id" e7 + +# get all CSS classes +playwright-cli eval "el => el.className" e7 + +# get a specific attribute +playwright-cli eval "el => el.getAttribute('data-testid')" e7 +playwright-cli eval "el => el.getAttribute('aria-label')" e7 + +# get a computed style property +playwright-cli eval "el => getComputedStyle(el).display" e7 +``` diff --git a/.claude/skills/playwright-cli/references/playwright-tests.md b/.claude/skills/playwright-cli/references/playwright-tests.md new file mode 100644 index 0000000..bec2ec9 --- /dev/null +++ b/.claude/skills/playwright-cli/references/playwright-tests.md @@ -0,0 +1,39 @@ +# Running Playwright Tests + +To run Playwright tests, use the `npx playwright test` command, or a package manager script. To avoid opening the interactive html report, use `PLAYWRIGHT_HTML_OPEN=never` environment variable. + +```bash +# Run all tests +PLAYWRIGHT_HTML_OPEN=never npx playwright test + +# Run all tests through a custom npm script +PLAYWRIGHT_HTML_OPEN=never npm run special-test-command +``` + +# Debugging Playwright Tests + +To debug a failing Playwright test, run it with `--debug=cli` option. This command will pause the test at the start and print the debugging instructions. + +**IMPORTANT**: run the command in the background and check the output until "Debugging Instructions" is printed. Make sure to stop the command after you have finished. + +Once instructions containing a session name are printed, use `playwright-cli` to attach the session and explore the page. + +```bash +# Run the test +PLAYWRIGHT_HTML_OPEN=never npx playwright test --debug=cli +# ... +# ... debugging instructions for "tw-abcdef" session ... +# ... + +# Attach to the test +playwright-cli attach tw-abcdef +``` + +Keep the test running in the background while you explore and look for a fix. +The test is paused at the start, so you should step over or pause at a particular location +where the problem is most likely to be. + +Every action you perform with `playwright-cli` generates corresponding Playwright TypeScript code. +This code appears in the output and can be copied directly into the test. Most of the time, a specific locator or an expectation should be updated, but it could also be a bug in the app. Use your judgement. + +After fixing the test, stop the background test run. Rerun to check that test passes. diff --git a/.claude/skills/playwright-cli/references/request-mocking.md b/.claude/skills/playwright-cli/references/request-mocking.md new file mode 100644 index 0000000..9005fda --- /dev/null +++ b/.claude/skills/playwright-cli/references/request-mocking.md @@ -0,0 +1,87 @@ +# Request Mocking + +Intercept, mock, modify, and block network requests. + +## CLI Route Commands + +```bash +# Mock with custom status +playwright-cli route "**/*.jpg" --status=404 + +# Mock with JSON body +playwright-cli route "**/api/users" --body='[{"id":1,"name":"Alice"}]' --content-type=application/json + +# Mock with custom headers +playwright-cli route "**/api/data" --body='{"ok":true}' --header="X-Custom: value" + +# Remove headers from requests +playwright-cli route "**/*" --remove-header=cookie,authorization + +# List active routes +playwright-cli route-list + +# Remove a route or all routes +playwright-cli unroute "**/*.jpg" +playwright-cli unroute +``` + +## URL Patterns + +``` +**/api/users - Exact path match +**/api/*/details - Wildcard in path +**/*.{png,jpg,jpeg} - Match file extensions +**/search?q=* - Match query parameters +``` + +## Advanced Mocking with run-code + +For conditional responses, request body inspection, response modification, or delays: + +### Conditional Response Based on Request + +```bash +playwright-cli run-code "async page => { + await page.route('**/api/login', route => { + const body = route.request().postDataJSON(); + if (body.username === 'admin') { + route.fulfill({ body: JSON.stringify({ token: 'mock-token' }) }); + } else { + route.fulfill({ status: 401, body: JSON.stringify({ error: 'Invalid' }) }); + } + }); +}" +``` + +### Modify Real Response + +```bash +playwright-cli run-code "async page => { + await page.route('**/api/user', async route => { + const response = await route.fetch(); + const json = await response.json(); + json.isPremium = true; + await route.fulfill({ response, json }); + }); +}" +``` + +### Simulate Network Failures + +```bash +playwright-cli run-code "async page => { + await page.route('**/api/offline', route => route.abort('internetdisconnected')); +}" +# Options: connectionrefused, timedout, connectionreset, internetdisconnected +``` + +### Delayed Response + +```bash +playwright-cli run-code "async page => { + await page.route('**/api/slow', async route => { + await new Promise(r => setTimeout(r, 3000)); + route.fulfill({ body: JSON.stringify({ data: 'loaded' }) }); + }); +}" +``` diff --git a/.claude/skills/playwright-cli/references/running-code.md b/.claude/skills/playwright-cli/references/running-code.md new file mode 100644 index 0000000..98b541f --- /dev/null +++ b/.claude/skills/playwright-cli/references/running-code.md @@ -0,0 +1,241 @@ +# Running Custom Playwright Code + +Use `run-code` to execute arbitrary Playwright code for advanced scenarios not covered by CLI commands. + +## Syntax + +```bash +playwright-cli run-code "async page => { + // Your Playwright code here + // Access page.context() for browser context operations +}" +``` + +You can also load the function from a file: + +```bash +playwright-cli run-code --filename=./my-script.js +``` + + +The code must be a single function expression, it is wrapped in `(...)` and evaluated. +import/export/require syntax is not supported. + +## Geolocation + +```bash +# Grant geolocation permission and set location +playwright-cli run-code "async page => { + await page.context().grantPermissions(['geolocation']); + await page.context().setGeolocation({ latitude: 37.7749, longitude: -122.4194 }); +}" + +# Set location to London +playwright-cli run-code "async page => { + await page.context().grantPermissions(['geolocation']); + await page.context().setGeolocation({ latitude: 51.5074, longitude: -0.1278 }); +}" + +# Clear geolocation override +playwright-cli run-code "async page => { + await page.context().clearPermissions(); +}" +``` + +## Permissions + +```bash +# Grant multiple permissions +playwright-cli run-code "async page => { + await page.context().grantPermissions([ + 'geolocation', + 'notifications', + 'camera', + 'microphone' + ]); +}" + +# Grant permissions for specific origin +playwright-cli run-code "async page => { + await page.context().grantPermissions(['clipboard-read'], { + origin: 'https://example.com' + }); +}" +``` + +## Media Emulation + +```bash +# Emulate dark color scheme +playwright-cli run-code "async page => { + await page.emulateMedia({ colorScheme: 'dark' }); +}" + +# Emulate light color scheme +playwright-cli run-code "async page => { + await page.emulateMedia({ colorScheme: 'light' }); +}" + +# Emulate reduced motion +playwright-cli run-code "async page => { + await page.emulateMedia({ reducedMotion: 'reduce' }); +}" + +# Emulate print media +playwright-cli run-code "async page => { + await page.emulateMedia({ media: 'print' }); +}" +``` + +## Wait Strategies + +```bash +# Wait for network idle +playwright-cli run-code "async page => { + await page.waitForLoadState('networkidle'); +}" + +# Wait for specific element +playwright-cli run-code "async page => { + await page.locator('.loading').waitFor({ state: 'hidden' }); +}" + +# Wait for function to return true +playwright-cli run-code "async page => { + await page.waitForFunction(() => window.appReady === true); +}" + +# Wait with timeout +playwright-cli run-code "async page => { + await page.locator('.result').waitFor({ timeout: 10000 }); +}" +``` + +## Frames and Iframes + +```bash +# Work with iframe +playwright-cli run-code "async page => { + const frame = page.locator('iframe#my-iframe').contentFrame(); + await frame.locator('button').click(); +}" + +# Get all frames +playwright-cli run-code "async page => { + const frames = page.frames(); + return frames.map(f => f.url()); +}" +``` + +## File Downloads + +```bash +# Handle file download +playwright-cli run-code "async page => { + const downloadPromise = page.waitForEvent('download'); + await page.getByRole('link', { name: 'Download' }).click(); + const download = await downloadPromise; + await download.saveAs('./downloaded-file.pdf'); + return download.suggestedFilename(); +}" +``` + +## Clipboard + +```bash +# Read clipboard (requires permission) +playwright-cli run-code "async page => { + await page.context().grantPermissions(['clipboard-read']); + return await page.evaluate(() => navigator.clipboard.readText()); +}" + +# Write to clipboard +playwright-cli run-code "async page => { + await page.evaluate(text => navigator.clipboard.writeText(text), 'Hello clipboard!'); +}" +``` + +## Page Information + +```bash +# Get page title +playwright-cli run-code "async page => { + return await page.title(); +}" + +# Get current URL +playwright-cli run-code "async page => { + return page.url(); +}" + +# Get page content +playwright-cli run-code "async page => { + return await page.content(); +}" + +# Get viewport size +playwright-cli run-code "async page => { + return page.viewportSize(); +}" +``` + +## JavaScript Execution + +```bash +# Execute JavaScript and return result +playwright-cli run-code "async page => { + return await page.evaluate(() => { + return { + userAgent: navigator.userAgent, + language: navigator.language, + cookiesEnabled: navigator.cookieEnabled + }; + }); +}" + +# Pass arguments to evaluate +playwright-cli run-code "async page => { + const multiplier = 5; + return await page.evaluate(m => document.querySelectorAll('li').length * m, multiplier); +}" +``` + +## Error Handling + +```bash +# Try-catch in run-code +playwright-cli run-code "async page => { + try { + await page.getByRole('button', { name: 'Submit' }).click({ timeout: 1000 }); + return 'clicked'; + } catch (e) { + return 'element not found'; + } +}" +``` + +## Complex Workflows + +```bash +# Login and save state +playwright-cli run-code "async page => { + await page.goto('https://example.com/login'); + await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com'); + await page.getByRole('textbox', { name: 'Password' }).fill('secret'); + await page.getByRole('button', { name: 'Sign in' }).click(); + await page.waitForURL('**/dashboard'); + await page.context().storageState({ path: 'auth.json' }); + return 'Login successful'; +}" + +# Scrape data from multiple pages +playwright-cli run-code "async page => { + const results = []; + for (let i = 1; i <= 3; i++) { + await page.goto(\`https://example.com/page/\${i}\`); + const items = await page.locator('.item').allTextContents(); + results.push(...items); + } + return results; +}" +``` diff --git a/.claude/skills/playwright-cli/references/session-management.md b/.claude/skills/playwright-cli/references/session-management.md new file mode 100644 index 0000000..bf39acd --- /dev/null +++ b/.claude/skills/playwright-cli/references/session-management.md @@ -0,0 +1,225 @@ +# Browser Session Management + +Run multiple isolated browser sessions concurrently with state persistence. + +## Named Browser Sessions + +Use `-s` flag to isolate browser contexts: + +```bash +# Browser 1: Authentication flow +playwright-cli -s=auth open https://app.example.com/login + +# Browser 2: Public browsing (separate cookies, storage) +playwright-cli -s=public open https://example.com + +# Commands are isolated by browser session +playwright-cli -s=auth fill e1 "user@example.com" +playwright-cli -s=public snapshot +``` + +## Browser Session Isolation Properties + +Each browser session has independent: +- Cookies +- LocalStorage / SessionStorage +- IndexedDB +- Cache +- Browsing history +- Open tabs + +## Browser Session Commands + +```bash +# List all browser sessions +playwright-cli list + +# Stop a browser session (close the browser) +playwright-cli close # stop the default browser +playwright-cli -s=mysession close # stop a named browser + +# Stop all browser sessions +playwright-cli close-all + +# Forcefully kill all daemon processes (for stale/zombie processes) +playwright-cli kill-all + +# Delete browser session user data (profile directory) +playwright-cli delete-data # delete default browser data +playwright-cli -s=mysession delete-data # delete named browser data +``` + +## Environment Variable + +Set a default browser session name via environment variable: + +```bash +export PLAYWRIGHT_CLI_SESSION="mysession" +playwright-cli open example.com # Uses "mysession" automatically +``` + +## Common Patterns + +### Concurrent Scraping + +```bash +#!/bin/bash +# Scrape multiple sites concurrently + +# Start all browsers +playwright-cli -s=site1 open https://site1.com & +playwright-cli -s=site2 open https://site2.com & +playwright-cli -s=site3 open https://site3.com & +wait + +# Take snapshots from each +playwright-cli -s=site1 snapshot +playwright-cli -s=site2 snapshot +playwright-cli -s=site3 snapshot + +# Cleanup +playwright-cli close-all +``` + +### A/B Testing Sessions + +```bash +# Test different user experiences +playwright-cli -s=variant-a open "https://app.com?variant=a" +playwright-cli -s=variant-b open "https://app.com?variant=b" + +# Compare +playwright-cli -s=variant-a screenshot +playwright-cli -s=variant-b screenshot +``` + +### Persistent Profile + +By default, browser profile is kept in memory only. Use `--persistent` flag on `open` to persist the browser profile to disk: + +```bash +# Use persistent profile (auto-generated location) +playwright-cli open https://example.com --persistent + +# Use persistent profile with custom directory +playwright-cli open https://example.com --profile=/path/to/profile +``` + +## Attaching to a Running Browser + +Use `attach` to connect to a browser that is already running, instead of launching a new one. + +### Attach by channel name + +Connect to a running Chrome or Edge instance by its channel name. The browser must have remote debugging enabled — navigate to `chrome://inspect/#remote-debugging` in the target browser and check "Allow remote debugging for this browser instance". + +```bash +# Attach to Chrome +playwright-cli attach --cdp=chrome + +# Attach to Chrome Canary +playwright-cli attach --cdp=chrome-canary + +# Attach to Microsoft Edge +playwright-cli attach --cdp=msedge + +# Attach to Edge Dev +playwright-cli attach --cdp=msedge-dev +``` + +Supported channels: `chrome`, `chrome-beta`, `chrome-dev`, `chrome-canary`, `msedge`, `msedge-beta`, `msedge-dev`, `msedge-canary`. + +When `--session` is not provided, the session is named after the channel (e.g. `--cdp=msedge` creates a session called `msedge`), so parallel attaches to Chrome and Edge don't collide on `default`. Pass `--session=` to override. + +### Attach via CDP endpoint + +Connect to a browser that exposes a Chrome DevTools Protocol endpoint: + +```bash +playwright-cli attach --cdp=http://localhost:9222 +``` + +### Attach via browser extension + +Connect to a browser with the Playwright extension installed: + +```bash +playwright-cli attach --extension +``` + +### Detach + +Tear down an attached session without affecting the external browser: + +```bash +# Detach the default attached session +playwright-cli detach + +# Detach a specific attached session +playwright-cli -s=msedge detach +``` + +`detach` only works on sessions created via `attach`. For sessions created via `open`, use `close`. + +## Default Browser Session + +When `-s` is omitted, commands use the default browser session: + +```bash +# These use the same default browser session +playwright-cli open https://example.com +playwright-cli snapshot +playwright-cli close # Stops default browser +``` + +## Browser Session Configuration + +Configure a browser session with specific settings when opening: + +```bash +# Open with config file +playwright-cli open https://example.com --config=.playwright/my-cli.json + +# Open with specific browser +playwright-cli open https://example.com --browser=firefox + +# Open in headed mode +playwright-cli open https://example.com --headed + +# Open with persistent profile +playwright-cli open https://example.com --persistent +``` + +## Best Practices + +### 1. Name Browser Sessions Semantically + +```bash +# GOOD: Clear purpose +playwright-cli -s=github-auth open https://github.com +playwright-cli -s=docs-scrape open https://docs.example.com + +# AVOID: Generic names +playwright-cli -s=s1 open https://github.com +``` + +### 2. Always Clean Up + +```bash +# Stop browsers when done +playwright-cli -s=auth close +playwright-cli -s=scrape close + +# Or stop all at once +playwright-cli close-all + +# If browsers become unresponsive or zombie processes remain +playwright-cli kill-all +``` + +### 3. Delete Stale Browser Data + +```bash +# Remove old browser data to free disk space +playwright-cli -s=oldsession delete-data +``` diff --git a/.claude/skills/playwright-cli/references/storage-state.md b/.claude/skills/playwright-cli/references/storage-state.md new file mode 100644 index 0000000..bb5021a --- /dev/null +++ b/.claude/skills/playwright-cli/references/storage-state.md @@ -0,0 +1,275 @@ +# Storage Management + +Manage cookies, localStorage, sessionStorage, and browser storage state. + +## Storage State + +Save and restore complete browser state including cookies and storage. + +### Save Storage State + +```bash +# Save to auto-generated filename (storage-state-{timestamp}.json) +playwright-cli state-save + +# Save to specific filename +playwright-cli state-save my-auth-state.json +``` + +### Restore Storage State + +```bash +# Load storage state from file +playwright-cli state-load my-auth-state.json + +# Reload page to apply cookies +playwright-cli open https://example.com +``` + +### Storage State File Format + +The saved file contains: + +```json +{ + "cookies": [ + { + "name": "session_id", + "value": "abc123", + "domain": "example.com", + "path": "/", + "expires": 1893456000, + "httpOnly": true, + "secure": true, + "sameSite": "Lax" + } + ], + "origins": [ + { + "origin": "https://example.com", + "localStorage": [ + { "name": "theme", "value": "dark" }, + { "name": "user_id", "value": "12345" } + ] + } + ] +} +``` + +## Cookies + +### List All Cookies + +```bash +playwright-cli cookie-list +``` + +### Filter Cookies by Domain + +```bash +playwright-cli cookie-list --domain=example.com +``` + +### Filter Cookies by Path + +```bash +playwright-cli cookie-list --path=/api +``` + +### Get Specific Cookie + +```bash +playwright-cli cookie-get session_id +``` + +### Set a Cookie + +```bash +# Basic cookie +playwright-cli cookie-set session abc123 + +# Cookie with options +playwright-cli cookie-set session abc123 --domain=example.com --path=/ --httpOnly --secure --sameSite=Lax + +# Cookie with expiration (Unix timestamp) +playwright-cli cookie-set remember_me token123 --expires=1893456000 +``` + +### Delete a Cookie + +```bash +playwright-cli cookie-delete session_id +``` + +### Clear All Cookies + +```bash +playwright-cli cookie-clear +``` + +### Advanced: Multiple Cookies or Custom Options + +For complex scenarios like adding multiple cookies at once, use `run-code`: + +```bash +playwright-cli run-code "async page => { + await page.context().addCookies([ + { name: 'session_id', value: 'sess_abc123', domain: 'example.com', path: '/', httpOnly: true }, + { name: 'preferences', value: JSON.stringify({ theme: 'dark' }), domain: 'example.com', path: '/' } + ]); +}" +``` + +## Local Storage + +### List All localStorage Items + +```bash +playwright-cli localstorage-list +``` + +### Get Single Value + +```bash +playwright-cli localstorage-get token +``` + +### Set Value + +```bash +playwright-cli localstorage-set theme dark +``` + +### Set JSON Value + +```bash +playwright-cli localstorage-set user_settings '{"theme":"dark","language":"en"}' +``` + +### Delete Single Item + +```bash +playwright-cli localstorage-delete token +``` + +### Clear All localStorage + +```bash +playwright-cli localstorage-clear +``` + +### Advanced: Multiple Operations + +For complex scenarios like setting multiple values at once, use `run-code`: + +```bash +playwright-cli run-code "async page => { + await page.evaluate(() => { + localStorage.setItem('token', 'jwt_abc123'); + localStorage.setItem('user_id', '12345'); + localStorage.setItem('expires_at', Date.now() + 3600000); + }); +}" +``` + +## Session Storage + +### List All sessionStorage Items + +```bash +playwright-cli sessionstorage-list +``` + +### Get Single Value + +```bash +playwright-cli sessionstorage-get form_data +``` + +### Set Value + +```bash +playwright-cli sessionstorage-set step 3 +``` + +### Delete Single Item + +```bash +playwright-cli sessionstorage-delete step +``` + +### Clear sessionStorage + +```bash +playwright-cli sessionstorage-clear +``` + +## IndexedDB + +### List Databases + +```bash +playwright-cli run-code "async page => { + return await page.evaluate(async () => { + const databases = await indexedDB.databases(); + return databases; + }); +}" +``` + +### Delete Database + +```bash +playwright-cli run-code "async page => { + await page.evaluate(() => { + indexedDB.deleteDatabase('myDatabase'); + }); +}" +``` + +## Common Patterns + +### Authentication State Reuse + +```bash +# Step 1: Login and save state +playwright-cli open https://app.example.com/login +playwright-cli snapshot +playwright-cli fill e1 "user@example.com" +playwright-cli fill e2 "password123" +playwright-cli click e3 + +# Save the authenticated state +playwright-cli state-save auth.json + +# Step 2: Later, restore state and skip login +playwright-cli state-load auth.json +playwright-cli open https://app.example.com/dashboard +# Already logged in! +``` + +### Save and Restore Roundtrip + +```bash +# Set up authentication state +playwright-cli open https://example.com +playwright-cli eval "() => { document.cookie = 'session=abc123'; localStorage.setItem('user', 'john'); }" + +# Save state to file +playwright-cli state-save my-session.json + +# ... later, in a new session ... + +# Restore state +playwright-cli state-load my-session.json +playwright-cli open https://example.com +# Cookies and localStorage are restored! +``` + +## Security Notes + +- Never commit storage state files containing auth tokens +- Add `*.auth-state.json` to `.gitignore` +- Delete state files after automation completes +- Use environment variables for sensitive data +- By default, sessions run in-memory mode which is safer for sensitive operations diff --git a/.claude/skills/playwright-cli/references/test-generation.md b/.claude/skills/playwright-cli/references/test-generation.md new file mode 100644 index 0000000..35a8d57 --- /dev/null +++ b/.claude/skills/playwright-cli/references/test-generation.md @@ -0,0 +1,433 @@ +# Test generation (plan → generate → heal) + +End-to-end workflow for authoring and maintaining Playwright tests with `playwright-cli`. Every `playwright-cli` action emits the equivalent Playwright TypeScript, and that generated code is the raw material for every test. The sections below can be used independently: + +- **How generation works** — the core mechanic everything else relies on: actions become TypeScript, plus how to add assertions. +- **Plan** — explore the app, produce a spec file describing what to test. +- **Generate** — turn a spec into Playwright test files. Update the spec if it's vague or stale. +- **Heal** — diagnose failing tests, fix the code, reconcile the spec with reality. + +Plan / generate / heal lean on the same mechanic: run `npx playwright test --debug=cli` in the background, then `playwright-cli attach tw-XXXX` to drive the paused page interactively. See [playwright-tests.md](playwright-tests.md) for the debug/attach mechanics. + +--- + +## 0. How generation works + +Every action you perform with `playwright-cli` generates corresponding Playwright TypeScript code. This code appears in the output and can be copied directly into your test files. + +```bash +# Start a session +playwright-cli open https://example.com/login + +# Take a snapshot to see elements +playwright-cli snapshot +# Output shows: e1 [textbox "Email"], e2 [textbox "Password"], e3 [button "Sign In"] + +# Fill form fields - generates code automatically +playwright-cli fill e1 "user@example.com" +# Ran Playwright code: +# await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com'); + +playwright-cli fill e2 "password123" +# Ran Playwright code: +# await page.getByRole('textbox', { name: 'Password' }).fill('password123'); + +playwright-cli click e3 +# Ran Playwright code: +# await page.getByRole('button', { name: 'Sign In' }).click(); +``` + +### Building a test file + +Collect the generated code into a Playwright test: + +```typescript +import { test, expect } from '@playwright/test'; + +test('login flow', async ({ page }) => { + // Generated code from playwright-cli session: + await page.goto('https://example.com/login'); + await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com'); + await page.getByRole('textbox', { name: 'Password' }).fill('password123'); + await page.getByRole('button', { name: 'Sign In' }).click(); + + // Add assertions + await expect(page).toHaveURL(/.*dashboard/); +}); +``` + +### Use semantic locators + +The generated code uses role-based locators when possible, which are more resilient: + +```typescript +// Generated (good - semantic) +await page.getByRole('button', { name: 'Submit' }).click(); + +// Avoid (fragile - CSS selectors) +await page.locator('#submit-btn').click(); +``` + +### Explore before recording + +Take snapshots to understand the page structure before recording actions: + +```bash +playwright-cli open https://example.com +playwright-cli snapshot +# Review the element structure +playwright-cli click e5 +``` + +### Add assertions manually + +Generated code captures actions but not assertions. Add expectations in your test using one of the recommended matchers: + +- `toBeVisible()` — element is rendered and visible +- `toHaveText(text)` — element text content matches +- `toHaveValue(value) / toBeEmpty()` — input/select value matches +- `toBeChecked() / toBeUnchecked()` — checkbox state matches +- `toMatchAriaSnapshot(snapshot)` — page (or locator) matches a partial accessibility snapshot + +Use `playwright-cli generate-locator ` to produce the locator expression for the assertion, and the snapshot/eval commands to capture the expected value. + +When asserting text content, make sure that generated locator does not contain text from the element itself. `getByTestId()` or `getByLabel()` usually work well with asserting text. When locator is text-based, prefer `toBeVisible()` instead. + +Snapshot to be matched does not have to contain all the information - only capture what's necessary for the assertion. You can use regular expressions for unstable values. + +```bash +# Get a stable locator for an element ref to use in the assertion +playwright-cli --raw generate-locator e5 +# getByRole('button', { name: 'Submit' }) + +# Capture expected text content for toHaveText +playwright-cli --raw eval "el => el.textContent" e5 + +# Capture expected input value for toHaveValue/toBeEmpty +playwright-cli --raw eval "el => el.value" e5 + +# Capture expected aria snapshot for toMatchAriaSnapshot/toBeChecked +# (whole page, or use a ref to scope to a region) +playwright-cli --raw snapshot +playwright-cli --raw snapshot e5 +``` + +```typescript +// Generated action +await page.getByRole('button', { name: 'Submit' }).click(); + +// Manual assertions using the outputs above: +await expect(page.getByRole('alert', { name: 'Success' })).toBeVisible(); +await expect(page.getByTestId('main-header')).toHaveText('Welcome, user'); +await expect(page.getByRole('textbox', { name: 'Email' })).toHaveValue('user@example.com'); +await expect(page.getByRole('checkbox', { name: 'Enable notifications' })).toBeChecked(); + +// toMatchAriaSnapshot on the whole page, finds a matching region +await expect(page).toMatchAriaSnapshot(` + - heading "Welcome, user" + - link /\\d+ new messages?/ + - button "Sign out" +`); + +// toMatchAriaSnapshot scoped to a region +await expect(page.getByRole('navigation')).toMatchAriaSnapshot(` + - link "Home" + - link /\\d+ new messages?/ + - link "Profile" +`); +``` + +--- + +## 1. Planning + +Goal: produce a spec file (e.g. `specs/.plan.md`) that enumerates the scenarios to test. **Always** write the spec to a file. + +### 1.1 Prerequisite: workspace + +Check the workspace has Playwright installed before anything else: + +```bash +# Either of these confirms a workspace: +test -f playwright.config.ts || test -f playwright.config.js +npx --no-install playwright --version +``` + +If there is no Playwright install, bootstrap one and let the user pick the defaults: + +```bash +npm init playwright@latest +``` + +### 1.2 Prerequisite: seed test + +A **seed test** is a minimal test that lands the page in the state every scenario starts from: navigation to the app, any required login, feature flags, etc. Scenarios assume a fresh start *after* the seed. `--debug=cli` pauses *inside* this test, so the seed is where every planning and generation session begins. + +Minimum viable seed: + +```ts +// tests/seed.spec.ts +import { test } from '@playwright/test'; + +test('seed', async ({ page }) => { + await page.goto('https://example.com/'); +}); +``` + +Preferred — push navigation into a fixture so scenario tests reuse it: + +```ts +// tests/fixtures.ts +import { test as baseTest } from '@playwright/test'; +export { expect } from '@playwright/test'; + +export const test = baseTest.extend({ + page: async ({ page }, use) => { + await page.goto('https://example.com/'); + await use(page); + }, +}); +``` + +```ts +// tests/seed.spec.ts +import { test } from './fixtures'; + +test('seed', async ({ page }) => { + // Fixture already navigates. This empty body tells agents where to start. +}); +``` + +If no seed exists, create one that at least navigates to the app. + +### 1.3 Explore the app + +Launch the app via the seed in the background and attach: + +```bash +PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/seed.spec.ts --debug=cli +# wait for "Debugging Instructions" and the session name tw-XXXX +playwright-cli attach tw-XXXX +``` + +Resume so the seed runs, then probe the app: + +```bash +playwright-cli resume # resume so that seed test runs fully +playwright-cli snapshot # inventory of interactive elements +playwright-cli click e5 # follow a flow +playwright-cli eval "location.href" # read URL / state +playwright-cli show --annotate # ask the user to point at something +``` + +Map out: + +- Interactive surfaces (forms, buttons, lists, filters, modals). +- Primary user journeys end-to-end. +- Edge cases: empty states, validation errors, very long input, boundary values. +- Persistence: reload, local/session storage, URL fragments. +- Navigation: which controls change the URL, back/forward behaviour. + +**Important**: Do not just open the app url with playwright-cli, always go through the test to capture any custom setup done there. +**Important**: Stop the background test when done exploring. + +### 1.4 Write the spec file + +Save under `specs/.plan.md`. Use this structure: + +```markdown +# Test Plan + +## Application Overview + + + +## Test Scenarios + +### 1. + +**Seed:** `tests/seed.spec.ts` + +#### 1.1. + +**File:** `tests//.spec.ts` + +**Steps:** + 1. + - expect: + - expect: + 2. + - expect: + +#### 1.2. +... + +### 2. + +**Seed:** `tests/seed.spec.ts` +... +``` + +Guidelines: + +- Each scenario is independent and starts from the seed's fresh state — never chain scenarios. +- Scenario names are kebab-case and match the test file name (`should-add-single-todo` → `should-add-single-todo.spec.ts`). +- Cover happy path, edge cases, validation, negative flows, persistence. +- Write steps at the user level ("Type 'Buy milk' into the input"), not the API level ("call `fill`"). +- Put observable outcomes in `- expect:` bullets; each becomes an assertion during generation. + +--- + +## 2. Generate + +Goal: take a spec file and produce Playwright test files. Optionally update the spec if it has drifted. + +### 2.1 Inputs + +- **Spec file**, e.g. `specs/basic-operations.plan.md`. +- **Target**: either a single scenario (e.g. `1.2`), a whole group (`1`), or all. +- **Seed file**, read from the `**Seed:**` line of the scenario's group. + +### 2.2 Generate one scenario + +For each target scenario, in sequence (never in parallel — scenarios share the seed session): + +```bash +PLAYWRIGHT_HTML_OPEN=never npx playwright test --debug=cli # background +playwright-cli attach tw-XXXX +# resume +``` + +**Do not** just open the app url with playwright-cli, always go through the test to capture any custom setup done there. + +Walk the scenario's `Steps:` one by one with `playwright-cli`, treating the spec as the plan and the live app as the source of truth. If a step is vague ("click the button" — which button?), references an element that no longer exists, or contradicts the app's actual behaviour, use your judgement: update the spec to match what the app really does, then keep going. Editing the spec mid-generation is expected. + +Every action prints the equivalent Playwright TypeScript (see [How generation works](#0-how-generation-works)): + +```bash +playwright-cli snapshot # find refs +playwright-cli fill e3 "John Doe" # -> page.getByRole('textbox', {...}).fill(...) +playwright-cli press Enter +playwright-cli click e7 +``` + +For each `- expect:` bullet, add an explicit assertion. See [How generation works](#0-how-generation-works) for details. + +Collect the generated code and write the test file at the path given in the spec: + +```ts +// spec: specs/basic-operations.plan.md +// seed: tests/seed.spec.ts +import { test, expect } from './fixtures'; // or '@playwright/test' if no fixtures file + +test.describe('Signing in and out', () => { + test('should sign in', async ({ page }) => { + // 1. Navigate to the application + // (handled by the seed fixture) + + // 2. Type 'John Doe' into the username field + await page.getByRole('textbox', { name: 'username' }).fill('John Doe'); + + // 3. Type password + await page.getByRole('textbox', { name: 'password' }).fill('TestPassword'); + + // 4. Press Enter to submit + await page.getByRole('textbox', { name: 'password' }).press('Enter'); + + await expect(page.getByRole('heading')).toContainText('Welcome, John Doe!'); + }); +}); +``` + +Rules: + +- **One test per file.** File path, describe name, and test name come verbatim from the spec (minus the ordinal). +- Prefix each numbered step with a `// N. ` comment before its actions. +- Use the describe group name verbatim from the spec (no `1.` ordinal). +- Import from `./fixtures` if the project has one; otherwise `@playwright/test`. +- **Important**: close the CLI session and stop the background test before moving to the next scenario. + +### 2.3 Generate multiple scenarios + +Loop 2.2 over the targeted scenarios one at a time, restarting the seed between each so every test starts from a clean page. This is safe to parallelise due to unique generated session names - just make sure each test run is stopped. + +### 2.4 Run generated tests + +After generation, run the new tests once: + +```bash +PLAYWRIGHT_HTML_OPEN=never npx playwright test tests//.spec.ts +``` + +Any failure goes to Section 3. + +--- + +## 3. Heal + +Goal: fix failing tests, and update the spec if the app's intended behaviour changed. + +### 3.1 Find failing tests + +```bash +PLAYWRIGHT_HTML_OPEN=never npx playwright test +``` + +Record the list of failing `:` entries and process them one at a time. Do not attempt parallel fixes — shared state and the single CLI session make that fragile. + +### 3.2 Debug one failure + +Run the single failing test in debug mode in the background, then attach: + +```bash +PLAYWRIGHT_HTML_OPEN=never npx playwright test tests//.spec.ts: --debug=cli +# wait for "Debugging Instructions" and the tw-XXXX session name +playwright-cli attach tw-XXXX +``` + +The test is paused at the start. Step forward or run to until just before the failing action or assertion, then diagnose: + +```bash +playwright-cli snapshot # did the element change / move / rename? +playwright-cli console # app-side errors? +playwright-cli requests # failed request? wrong payload? +playwright-cli show --annotate # ask the user to point somewhere +``` + +Common causes: selector drift, new wrapper element, label/ARIA rename, timing (transition, async load), assertion text updated in the app, test data leaking between runs. + +Rehearse the corrected interaction with `playwright-cli` — the generated code in the output is what you paste back into the test. + +### 3.3 Apply the fix + +Edit the test file: update the locator, assertion, step order, or inputs to match the corrected behaviour. Stop the background debug run. Rerun the single test to confirm green. + +Never skip hooks or add sleeps as a fix. Never use `networkidle`. + +### 3.4 Reconcile with the spec + +Open the spec referenced by the `// spec:` header in the test file and locate the scenario that matches the test. + +- **Fix was purely technical** (locator drift, better assertion shape) and the spec's user-level behaviour still matches the app → leave the spec alone. +- **Fix changed user-visible steps, inputs, order, or expected outcomes** that the spec describes → update the spec to match reality. Keep the scenario id and file path stable; only the step / expect lines change. +- **Unclear whether the app change is intentional** (spec is stale) **or a regression** (test was right, app is wrong) → **stop and ask the user**. Provide: + - the scenario id (e.g. `2.3`), + - the spec lines that no longer match, + - the observed app behaviour (quote a snapshot excerpt or a concrete outcome). + +Only after the user answers, either update the spec (intentional change) or file/flag the test as covering a bug (regression). + +### 3.5 Iteration and giving up + +- Fix failures one at a time; rerun after each. +- If after thorough investigation you are confident the test is correct but the app is wrong *and* the user has confirmed it's a bug: mark the test `test.fixme(...)` with a comment pointing at the user's decision or issue link. Never silently skip. + +--- + +## Cross-references + +| For... | See | +|---|---| +| `--debug=cli` / attach mechanics | [playwright-tests.md](playwright-tests.md) | +| Mocking requests during exploration/generation | [request-mocking.md](request-mocking.md) | +| Managing the CLI browser session | [session-management.md](session-management.md) | diff --git a/.claude/skills/playwright-cli/references/tracing.md b/.claude/skills/playwright-cli/references/tracing.md new file mode 100644 index 0000000..7ce7bab --- /dev/null +++ b/.claude/skills/playwright-cli/references/tracing.md @@ -0,0 +1,139 @@ +# Tracing + +Capture detailed execution traces for debugging and analysis. Traces include DOM snapshots, screenshots, network activity, and console logs. + +## Basic Usage + +```bash +# Start trace recording +playwright-cli tracing-start + +# Perform actions +playwright-cli open https://example.com +playwright-cli click e1 +playwright-cli fill e2 "test" + +# Stop trace recording +playwright-cli tracing-stop +``` + +## Trace Output Files + +When you start tracing, Playwright creates a `traces/` directory with several files: + +### `trace-{timestamp}.trace` + +**Action log** - The main trace file containing: +- Every action performed (clicks, fills, navigations) +- DOM snapshots before and after each action +- Screenshots at each step +- Timing information +- Console messages +- Source locations + +### `trace-{timestamp}.network` + +**Network log** - Complete network activity: +- All HTTP requests and responses +- Request headers and bodies +- Response headers and bodies +- Timing (DNS, connect, TLS, TTFB, download) +- Resource sizes +- Failed requests and errors + +### `resources/` + +**Resources directory** - Cached resources: +- Images, fonts, stylesheets, scripts +- Response bodies for replay +- Assets needed to reconstruct page state + +## What Traces Capture + +| Category | Details | +|----------|---------| +| **Actions** | Clicks, fills, hovers, keyboard input, navigations | +| **DOM** | Full DOM snapshot before/after each action | +| **Screenshots** | Visual state at each step | +| **Network** | All requests, responses, headers, bodies, timing | +| **Console** | All console.log, warn, error messages | +| **Timing** | Precise timing for each operation | + +## Use Cases + +### Debugging Failed Actions + +```bash +playwright-cli tracing-start +playwright-cli open https://app.example.com + +# This click fails - why? +playwright-cli click e5 + +playwright-cli tracing-stop +# Open trace to see DOM state when click was attempted +``` + +### Analyzing Performance + +```bash +playwright-cli tracing-start +playwright-cli open https://slow-site.com +playwright-cli tracing-stop + +# View network waterfall to identify slow resources +``` + +### Capturing Evidence + +```bash +# Record a complete user flow for documentation +playwright-cli tracing-start + +playwright-cli open https://app.example.com/checkout +playwright-cli fill e1 "4111111111111111" +playwright-cli fill e2 "12/25" +playwright-cli fill e3 "123" +playwright-cli click e4 + +playwright-cli tracing-stop +# Trace shows exact sequence of events +``` + +## Trace vs Video vs Screenshot + +| Feature | Trace | Video | Screenshot | +|---------|-------|-------|------------| +| **Format** | .trace file | .webm video | .png/.jpeg image | +| **DOM inspection** | Yes | No | No | +| **Network details** | Yes | No | No | +| **Step-by-step replay** | Yes | Continuous | Single frame | +| **File size** | Medium | Large | Small | +| **Best for** | Debugging | Demos | Quick capture | + +## Best Practices + +### 1. Start Tracing Before the Problem + +```bash +# Trace the entire flow, not just the failing step +playwright-cli tracing-start +playwright-cli open https://example.com +# ... all steps leading to the issue ... +playwright-cli tracing-stop +``` + +### 2. Clean Up Old Traces + +Traces can consume significant disk space: + +```bash +# Remove traces older than 7 days +find .playwright-cli/traces -mtime +7 -delete +``` + +## Limitations + +- Traces add overhead to automation +- Large traces can consume significant disk space +- Some dynamic content may not replay perfectly diff --git a/.claude/skills/playwright-cli/references/video-recording.md b/.claude/skills/playwright-cli/references/video-recording.md new file mode 100644 index 0000000..5209d21 --- /dev/null +++ b/.claude/skills/playwright-cli/references/video-recording.md @@ -0,0 +1,143 @@ +# Video Recording + +Capture browser automation sessions as video for debugging, documentation, or verification. Produces WebM (VP8/VP9 codec). + +## Basic Recording + +```bash +# Open browser first +playwright-cli open + +# Start recording +playwright-cli video-start demo.webm + +# Add a chapter marker for section transitions +playwright-cli video-chapter "Getting Started" --description="Opening the homepage" --duration=2000 + +# Navigate and perform actions +playwright-cli goto https://example.com +playwright-cli snapshot +playwright-cli click e1 + +# Add another chapter +playwright-cli video-chapter "Filling Form" --description="Entering test data" --duration=2000 +playwright-cli fill e2 "test input" + +# Stop and save +playwright-cli video-stop +``` + +## Best Practices + +### 1. Use Descriptive Filenames + +```bash +# Include context in filename +playwright-cli video-start recordings/login-flow-2024-01-15.webm +playwright-cli video-start recordings/checkout-test-run-42.webm +``` + +### 2. Record entire hero scripts. + +When recording a video for the user or as a proof of work, it is best to create a code snippet and execute it with run-code. +It allows inserting appropriate pauses between the actions and annotating the video. There are new Playwright APIs for that. + +1) Perform scenario using CLI and take note of all locators and actions. You'll need those locators to request their bounding boxes for highlight. +2) Create a file with the intended script for video (below). Use pressSequentially w/ delay for nice typing, make reasonable pauses. +3) Use playwright-cli run-code --filename your-script.js + +**Important**: Overlays are `pointer-events: none` — they do not interfere with page interactions. You can safely keep sticky overlays visible while clicking, filling, or performing any actions on the page. + +```js +async page => { + await page.screencast.start({ path: 'video.webm', size: { width: 1280, height: 800 } }); + await page.goto('https://demo.playwright.dev/todomvc'); + + // Show a chapter card — blurs the page and shows a dialog. + // Blocks until duration expires, then auto-removes. + // Use this for simple use cases, but always feel free to hand-craft your own beautiful + // overlay via await page.screencast.showOverlay(). + await page.screencast.showChapter('Adding Todo Items', { + description: 'We will add several items to the todo list.', + duration: 2000, + }); + + // Perform action + await page.getByRole('textbox', { name: 'What needs to be done?' }).pressSequentially('Walk the dog', { delay: 60 }); + await page.getByRole('textbox', { name: 'What needs to be done?' }).press('Enter'); + await page.waitForTimeout(1000); + + // Show next chapter + await page.screencast.showChapter('Verifying Results', { + description: 'Checking the item appeared in the list.', + duration: 2000, + }); + + // Add a sticky annotation that stays while you perform actions. + // Overlays are pointer-events: none, so they won't block clicks. + const annotation = await page.screencast.showOverlay(` +
+ ✓ Item added successfully +
+ `); + + // Perform more actions while the annotation is visible + await page.getByRole('textbox', { name: 'What needs to be done?' }).pressSequentially('Buy groceries', { delay: 60 }); + await page.getByRole('textbox', { name: 'What needs to be done?' }).press('Enter'); + await page.waitForTimeout(1500); + + // Remove the annotation when done + await annotation.dispose(); + + // You can also highlight relevant locators and provide contextual annotations. + const bounds = await page.getByText('Walk the dog').boundingBox(); + await page.screencast.showOverlay(` +
+
+
Check it out, it is right above this text +
+ `, { duration: 2000 }); + + await page.screencast.stop(); +} +``` + +Embrace creativity, overlays are powerful. + +### Overlay API Summary + +| Method | Use Case | +|--------|----------| +| `page.screencast.showChapter(title, { description?, duration?, styleSheet? })` | Full-screen chapter card with blurred backdrop — ideal for section transitions | +| `page.screencast.showOverlay(html, { duration? })` | Custom HTML overlay — use for callouts, labels, highlights | +| `disposable.dispose()` | Remove a sticky overlay added without duration | +| `page.screencast.hideOverlays()` / `page.screencast.showOverlays()` | Temporarily hide/show all overlays | + +## Tracing vs Video + +| Feature | Video | Tracing | +|---------|-------|---------| +| Output | WebM file | Trace file (viewable in Trace Viewer) | +| Shows | Visual recording | DOM snapshots, network, console, actions | +| Use case | Demos, documentation | Debugging, analysis | +| Size | Larger | Smaller | + +## Limitations + +- Recording adds slight overhead to automation +- Large recordings can consume significant disk space diff --git a/.env.example b/.env.example index 0b07de2..05b0f95 100644 --- a/.env.example +++ b/.env.example @@ -6,3 +6,20 @@ ADMIN_PASSWORD= ADMIN_OTP_SECRET= GPP_APP_BASE_URL= GPP_PUBLICATIEBANK_BASE_URL= +GPP_BURGERPORTAAL_BASE_URL= + +# OpenRouter API key for the Stagehand-driven @beheer scenarios (Testscript 1). +# Get one at https://openrouter.ai/keys. Without it the @beheer scenarios skip. +OPENROUTER_API_KEY= + +# GPP-publicatiebank (ODRC) REST API — used to own publicaties/documenten/organisaties +# test data (TS6-9). Defaults to GPP_PUBLICATIEBANK_BASE_URL when ODRC_BASE_URL is unset. +ODRC_BASE_URL= +ODRC_API_KEY= + +# DiWoo sitemap change-propagation waits (ms) before refetching a monthly +# sitemap. Only set when the burgerportaal has a short cache override +# (SITEMAP_CACHE_DURATION_HOURS=0 or ~0.016≈1min). Unset → membership scenarios +# that call waitForSitemapCacheExpiry fail fast instead of sleeping ~23h. +# Example for cache=0: SITEMAP_CACHE_WAIT_MS=0 +SITEMAP_CACHE_WAIT_MS= diff --git a/.github/workflows/e2e-k8s.yml b/.github/workflows/e2e-k8s.yml new file mode 100644 index 0000000..468c296 --- /dev/null +++ b/.github/workflows/e2e-k8s.yml @@ -0,0 +1,275 @@ +name: E2E on local k8s +# Spins up the whole GPP stack from the Helm charts in a kind cluster (same hack/up.sh +# used locally) and runs the Playwright suite against it. Validates the real charts on every PR. +on: + pull_request: + # Nightly full-matrix run. PRs run chromium only (see BROWSERS below): firefox + # and webkit are 138 of the 310 tests and exist to catch engine differences, + # which change on the browsers' release cadence, not on a PR. Once a night is + # the right frequency for that; twice per PR is not. + schedule: + - cron: '0 2 * * *' + workflow_dispatch: + inputs: + browsers: + description: Browsers to run + type: choice + options: [all, chromium] + default: all + +# `pages: write` + `id-token: write` are needed by the deploy job below. Fork PRs +# get a read-only token regardless, but the deploy is gated off for PRs anyway. +permissions: + contents: read + pages: write + id-token: write + +# A repo has exactly one Pages site, so two runs deploying at once clobber each +# other. Serialise (don't cancel — a half-finished deploy is worse than a queue). +concurrency: + group: ${{ github.workflow }}-pages + cancel-in-progress: false + +jobs: + e2e: + runs-on: ubuntu-latest + # ~10 min to build the images and bring the stack up, then the suite itself. + # 60 was not enough once the @todo gap scenarios were implemented and the run + # got cancelled mid-suite, which uploads no report and tells you nothing. + timeout-minutes: 90 + env: + # Single source of truth: keep in sync with the Helm used locally for + # hack/up.sh (needs Helm 4 for `--server-side --force-conflicts`). + HELM_VERSION: v4.2.3 + # `all` -> every project in playwright.config.ts; `chromium` -> just that + # one (Playwright pulls in its `setup` dependency by itself). + BROWSERS: ${{ inputs.browsers || (github.event_name == 'pull_request' && 'chromium' || 'all') }} + steps: + - uses: actions/checkout@v4 + with: {path: e2e-tests} + # Sibling checkouts: hack/up.sh builds every app image from source (no emulation, and + # the published Django/openzaak images are amd64-only), so all app repos must be here. + # open-zaak is cloned by up.sh itself. + - uses: actions/checkout@v4 + # hack/ (up.sh, kind-config.yaml) lives on this branch, not charts' main yet. + with: {repository: GPP-Woo/charts, ref: explore/local-via-k3d, path: charts} + - uses: actions/checkout@v4 + with: {repository: GPP-Woo/GPP-app, ref: feat/local-dev-stack, path: GPP-app} + - uses: actions/checkout@v4 + with: {repository: GPP-Woo/GPP-burgerportaal, ref: feat/local-dev-stack, path: GPP-burgerportaal} + - uses: actions/checkout@v4 + # Pinned, not default: main still returns `(None, token)` from + # TokenAuthentication, and SessionProfileMiddleware then dereferences + # request.user -> every token API call 500s while an admin session is + # active. Drop the ref once fix/token-auth-anonymoususer is merged. + with: {repository: GPP-Woo/GPP-publicatiebank, ref: fix/token-auth-anonymoususer, path: GPP-publicatiebank} + - uses: actions/checkout@v4 + # submodules: shared/dotgithub provides fileTypes.json, required by the image build + with: {repository: GPP-Woo/GPP-zoeken, path: GPP-zoeken, submodules: true} + + # Building every app image from source + loading them into the kind node + # fills the runner's ~14 GB disk, so `playwright install --with-deps` then + # fails with "No space left on device". Reclaim the big preinstalled + # toolchains we don't use (Android SDK ~9 GB, .NET, GHC, Swift). + - name: Free up runner disk space + run: | + sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc \ + /usr/local/share/boost /usr/share/swift /opt/hostedtoolcache/CodeQL || true + df -h / + + # Installs kind + kubectl + helm and creates the cluster with our port mappings. + - uses: helm/kind-action@v1 + with: + cluster_name: gpp-e2e + config: charts/hack/kind-config.yaml + + # hack/up.sh uses `helm upgrade --server-side --force-conflicts`, which is + # a Helm 4 feature. Pin the exact version we develop against (HELM_VERSION) + # so the runner, up.sh and this pipeline all run the same Helm. get-helm-3 + # only fetches Helm 3, so install the v4 tarball straight over PATH. + - name: Install Helm ${{ env.HELM_VERSION }} + run: | + curl -fsSL "https://get.helm.sh/helm-${HELM_VERSION}-linux-amd64.tar.gz" | tar xz + sudo install -m 0755 linux-amd64/helm /usr/local/bin/helm + rm -rf linux-amd64 + helm version + + - name: Raise vm.max_map_count for Elasticsearch (ECK) + run: sudo sysctl -w vm.max_map_count=262144 + + # Building the five app images from source is ~8 of the ~10 min up.sh takes, + # and the app repos are pinned refs that rarely move. Key the cache on their + # resolved HEADs (which cover their Dockerfiles too) plus up.sh itself — + # that script holds the build commands and the OPENZAAK_VERSION default, so + # hashing it is what keeps an openzaak bump from restoring a stale image. + - name: Compute image cache key + id: imgkey + run: | + key=$({ for d in GPP-app GPP-burgerportaal GPP-publicatiebank GPP-zoeken; do + git -C "$d" rev-parse HEAD + done + sha256sum charts/hack/up.sh; } | sha256sum | cut -c1-16) + echo "key=gpp-images-v1-$key" >> "$GITHUB_OUTPUT" + - uses: actions/cache@v4 + id: image-cache + with: + path: /tmp/gpp-images.tar + key: ${{ steps.imgkey.outputs.key }} + - name: Load cached images + if: steps.image-cache.outputs.cache-hit == 'true' + run: docker load -i /tmp/gpp-images.tar + + - name: Bring up the GPP stack from charts + id: up + working-directory: charts + env: + # up.sh reuses whatever *:local images are already in the daemon. + SKIP_BUILD: ${{ steps.image-cache.outputs.cache-hit == 'true' && '1' || '' }} + run: USE_EXISTING_CLUSTER=1 bash hack/up.sh + + # Only on a clean bring-up: a tar written from a half-built stack would be + # restored next run under SKIP_BUILD=1 and fail with no build to fix it. + # actions/cache's post-job step uploads the file, so writing it here is enough. + - name: Stage images for the cache + if: ${{ steps.image-cache.outputs.cache-hit != 'true' && steps.up.outcome == 'success' }} + # Write to a temp name and rename: actions/cache uploads whatever is at + # the path in the post-job step, so a save killed halfway would otherwise + # cache a truncated tar under a key that looks valid forever. + run: | + docker save -o /tmp/gpp-images.tar.part \ + odpc:local odbp:local gpp-publicatiebank:local gpp-zoeken:local openzaak:local + mv /tmp/gpp-images.tar.part /tmp/gpp-images.tar + ls -lh /tmp/gpp-images.tar; df -h / + + # The charts do not wire the Documenten API (no DRC Service, no + # GlobalConfiguration.documents_api_service, and OpenZaak has no ORC service + # for the publicatiebank catalogi), so POST /api/v2/documenten answers 500 and + # every scenario that seeds a document fails. This is the same once-per-fresh- + # stack step the README documents for a local bring-up. It also sets the + # publication URL templates, which ship empty (see the script's header). + - name: Provision the Documenten API on the stack + working-directory: e2e-tests + run: ./setup/provision-documenten-api.sh + + - uses: actions/setup-node@v4 + with: {node-version: lts/*} + - name: Install e2e deps + working-directory: e2e-tests + run: | + npm ci + npx playwright install --with-deps + + - name: Run Playwright suite + working-directory: e2e-tests + env: + # Local realm test users (from charts/dev/infra/files/realm.json). + DEFAULT_EMAIL: user@example.com + DEFAULT_PASSWORD: user + DEFAULT_OTP_SECRET: gpp-user-otp-seed-0001 + ADMIN_EMAIL: admin@example.com + ADMIN_PASSWORD: admin + ADMIN_OTP_SECRET: gpp-admin-otp-seed-0001 + # NodePort URLs (see charts/hack/kind-config.yaml). + GPP_APP_BASE_URL: http://localhost:8130 + GPP_PUBLICATIEBANK_BASE_URL: http://localhost:8000 + GPP_BURGERPORTAAL_BASE_URL: http://localhost:8140 + ODRC_BASE_URL: http://localhost:8000 + ODRC_API_KEY: insecure-ea1a8d297e3b2d3313b8a30b18959c3 + GPP_ZOEKEN_BASE_URL: http://localhost:8110 + # The sitemap refetch steps refuse to guess this (the prod output cache is + # ~23h — see waitForSitemapCacheExpiry). charts dev/values/odbp.yaml sets + # sitemapCacheDurationHours: 0, so no wait is needed here. + SITEMAP_CACHE_WAIT_MS: 0 + # OpenZaak Documenten API (dev/openzaak chart); client creds come from the + # auto-loaded configuration.json fixture. + OPENZAAK_BASE_URL: http://localhost:8001 + DRC_ROOT: http://localhost:8001/documenten/api/v1/ + OPENZAAK_CLIENT_ID: woo-publications-dev + OPENZAAK_SECRET: insecure-yQL9Rzh4eHGVmYx5w3J2gu + # `setup` is named explicitly rather than left to chromium's `dependencies`: + # Playwright does run dependencies of a --project selection, but if that + # ever changes the failure mode is every scenario failing on a missing + # .auth/*.json, which is a miserable thing to debug for one saved flag. + run: | + if [ "$BROWSERS" = all ]; then + npm test + else + npm test -- --project=setup --project=chromium + fi + + - name: Dump cluster state on failure + # cancelled() too: a timeout-cancel is the case where a wedged pod is the + # likeliest cause, so that is when this dump is worth the most. + if: ${{ failure() || cancelled() }} + run: | + kubectl -n gpp-e2e get pods -o wide || true + kubectl -n gpp-e2e get elasticsearch || true + helm list -n gpp-e2e || true + kubectl -n gpp-e2e describe pods || true + # By deployment name, not by app.kubernetes.io/instance: the publicatiebank + # pods carry no instance label, and `instance=odrc`/`zoeken` match only the + # *redis* pods — so this used to dump redis logs and never the app or worker + # logs anyone actually needs. openzaak has no `app` label either. + for d in gpp-publicatiebank gpp-publicatiebank-worker gpp-burgerportaal \ + gpp-app gpp-zoeken gpp-zoeken-worker openzaak-web openzaak-celery; do + echo "::group::logs deploy/$d" + kubectl -n gpp-e2e logs "deploy/$d" --tail=120 || true + echo "::endgroup::" + done + # Why a document never reaches the sitemap (gepubliceerd + upload_complete + + # publisher not zelf_toegevoegd). An empty sitemap is almost always one of + # those three, and none of them show up in a pod log. + echo "::group::ODRC document/publisher state" + ./e2e-tests/setup/dump-odrc-state.sh || true + echo "::endgroup::" + + # Zip for PR runs and for anyone who wants the raw report. `always()`, not + # `!cancelled()`: a job that hits timeout-minutes counts as cancelled, and + # that is exactly when you most want the partial report. + - uses: actions/upload-artifact@v4 + if: ${{ always() }} + with: + name: playwright-report + path: e2e-tests/playwright-report + + # The HTML report is a self-contained static site: playwright-report/trace/ + # is the bundled trace viewer and data/*.zip are the traces, so serving the + # directory over HTTP gives clickable, fully interactive traces — no upload + # to trace.playwright.dev, no `npx playwright show-trace`. (Only over HTTP — + # opening index.html from the unzipped artifact via file:// cannot load them.) + - uses: actions/upload-pages-artifact@v3 + if: ${{ !cancelled() }} + with: + path: e2e-tests/playwright-report/ + + # Mirrors the run summary onto a fixed Confluence page. REPORT_URL is the + # Pages site (stable URL, always the latest run); the CI-run link in the + # summary points at this specific run's zip artifact. + # Per-environment page: set CONFLUENCE_PAGE_ID as a GitHub *Environment* + # variable once this job takes an `environment:`. + - name: Publish summary to Confluence + if: ${{ !cancelled() && vars.CONFLUENCE_PAGE_ID != '' }} + working-directory: e2e-tests + env: + CONFLUENCE_BASE: ${{ vars.CONFLUENCE_BASE }} + CONFLUENCE_USER: ${{ vars.CONFLUENCE_USER }} + CONFLUENCE_TOKEN: ${{ secrets.CONFLUENCE_TOKEN }} + CONFLUENCE_PAGE_ID: ${{ vars.CONFLUENCE_PAGE_ID }} + REPORT_URL: ${{ vars.REPORT_URL }} + run: node scripts/publish-confluence.mjs + + # Separate job: deploy-pages must run in the `github-pages` environment. + # PR runs deploy too. A repo has exactly ONE Pages site and every deploy + # replaces it, so the URL always shows the most recent run — PR or not. + # Per-PR routes would need a gh-pages branch with pr-/ directories (and a + # Pages source switch); at ~112 MB a report that hits the 1 GB site cap fast. + deploy-report: + if: ${{ !cancelled() }} + needs: e2e + runs-on: ubuntu-latest + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + steps: + - id: deployment + uses: actions/deploy-pages@v4 diff --git a/.github/workflows/playwright.yml b/.github/workflows/playwright.yml index 0bf2e54..cbc7862 100644 --- a/.github/workflows/playwright.yml +++ b/.github/workflows/playwright.yml @@ -1,9 +1,11 @@ name: Playwright Tests +# Manual only. This runs the suite against the shared maykin/icatt accept +# environments, which have no burgerportaal deployed and no ODRC API token — so +# `bdd/_core/types.ts` throws a ZodError on GPP_BURGERPORTAAL_BASE_URL and +# ODRC_API_KEY before a single test starts. Set both on the environment to make +# it runnable again (and add them to the `env:` block below); until then it is +# superseded by e2e-k8s.yml, which runs the same suite against a real stack. on: - push: - branches: [main] - pull_request: - branches: [main] workflow_dispatch: concurrency: group: ${{ github.workflow }} diff --git a/.gitignore b/.gitignore index 2ea7adc..e331df1 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,9 @@ yarn-error.log* lerna-debug.log* .pnpm-debug.log* +# playwright cli output +.playwright-cli + # Diagnostic reports (https://nodejs.org/api/report.html) report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json @@ -135,9 +138,15 @@ dist .yarn/install-state.gz .pnp.* +# Playwright BDD generated specs +.features-gen/ + # Playwright /test-results/ /playwright-report/ /blob-report/ /playwright/.cache/ .auth + +# Throwaway run artifacts (ad-hoc reports/logs kept out of the tree) +/scratch/ diff --git a/.npmrc b/.npmrc new file mode 100644 index 0000000..e52967d --- /dev/null +++ b/.npmrc @@ -0,0 +1,4 @@ +# openai@4 still declares an optional peer of zod@^3, but this suite runs zod@4 +# (Stagehand 3.6 supports it) and never uses openai's zod helpers. Skip the +# unsatisfiable optional peer so clean/CI installs don't need a manual flag. +legacy-peer-deps=true diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..93fb0a4 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,72 @@ +# context-mode — MANDATORY routing rules + +You have context-mode MCP tools available. These rules are NOT optional — they protect your context window from flooding. A single unrouted command can dump 56 KB into context and waste the entire session. + +## BLOCKED commands — do NOT attempt these + +### curl / wget — BLOCKED + +Any Bash command containing `curl` or `wget` is intercepted and replaced with an error message. Do NOT retry. +Instead use: + +- `ctx_fetch_and_index(url, source)` to fetch and index web pages +- `ctx_execute(language: "javascript", code: "const r = await fetch(...)")` to run HTTP calls in sandbox + +### Inline HTTP — BLOCKED + +Any Bash command containing `fetch('http`, `requests.get(`, `requests.post(`, `http.get(`, or `http.request(` is intercepted and replaced with an error message. Do NOT retry with Bash. +Instead use: + +- `ctx_execute(language, code)` to run HTTP calls in sandbox — only stdout enters context + +### WebFetch — BLOCKED + +WebFetch calls are denied entirely. The URL is extracted and you are told to use `ctx_fetch_and_index` instead. +Instead use: + +- `ctx_fetch_and_index(url, source)` then `ctx_search(queries)` to query the indexed content + +## REDIRECTED tools — use sandbox equivalents + +### Bash (>20 lines output) + +Bash is ONLY for: `git`, `mkdir`, `rm`, `mv`, `cd`, `ls`, `npm install`, `pip install`, and other short-output commands. +For everything else, use: + +- `ctx_batch_execute(commands, queries)` — run multiple commands + search in ONE call +- `ctx_execute(language: "shell", code: "...")` — run in sandbox, only stdout enters context + +### Read (for analysis) + +If you are reading a file to **Edit** it → Read is correct (Edit needs content in context). +If you are reading to **analyze, explore, or summarize** → use `ctx_execute_file(path, language, code)` instead. Only your printed summary enters context. The raw file content stays in the sandbox. + +### Grep (large results) + +Grep results can flood context. Use `ctx_execute(language: "shell", code: "grep ...")` to run searches in sandbox. Only your printed summary enters context. + +## Tool selection hierarchy + +1. **GATHER**: `ctx_batch_execute(commands, queries)` — Primary tool. Runs all commands, auto-indexes output, returns search results. ONE call replaces 30+ individual calls. +2. **FOLLOW-UP**: `ctx_search(queries: ["q1", "q2", ...])` — Query indexed content. Pass ALL questions as array in ONE call. +3. **PROCESSING**: `ctx_execute(language, code)` | `ctx_execute_file(path, language, code)` — Sandbox execution. Only stdout enters context. +4. **WEB**: `ctx_fetch_and_index(url, source)` then `ctx_search(queries)` — Fetch, chunk, index, query. Raw HTML never enters context. +5. **INDEX**: `ctx_index(content, source)` — Store content in FTS5 knowledge base for later search. + +## Subagent routing + +When spawning subagents (Agent/Task tool), the routing block is automatically injected into their prompt. Bash-type subagents are upgraded to general-purpose so they have access to MCP tools. You do NOT need to manually instruct subagents about context-mode. + +## Output constraints + +- Keep responses under 500 words. +- Write artifacts (code, configs, PRDs) to FILES — never return them as inline text. Return only: file path + 1-line description. +- When indexing content, use descriptive source labels so others can `ctx_search(source: "label")` later. + +## ctx commands + +| Command | Action | +| ------------- | ------------------------------------------------------------------------------------- | +| `ctx stats` | Call the `ctx_stats` MCP tool and display the full output verbatim | +| `ctx doctor` | Call the `ctx_doctor` MCP tool, run the returned shell command, display as checklist | +| `ctx upgrade` | Call the `ctx_upgrade` MCP tool, run the returned shell command, display as checklist | diff --git a/E2E-FIXES-2026-08-10.md b/E2E-FIXES-2026-08-10.md new file mode 100644 index 0000000..0bc72a5 --- /dev/null +++ b/E2E-FIXES-2026-08-10.md @@ -0,0 +1,182 @@ +# E2E failures — 2026-08-10 + +`npx playwright test` (304 tests, 3 browsers, retries:1) + +| run | passed | failed | flaky | did not run | time | +| --------------- | ------- | ------ | ----- | ----------- | ---- | +| 1 — baseline | 235 | **22** | 12 | 35 | 7.7m | +| 7 — after fixes | 303 | 0 | 1 | 0 | 6.7m | +| 8 — final | **304** | **0** | **0** | 0 | 4.5m | + +Logs: `scratchpad/e2e-run{1,8}.log` · parser: `scratchpad/parse.mjs` + +`npm run lint`, `npm run typecheck` and `bddgen` are all clean. + +## Still open + +- **Not deployed**: the `SitemapController.cs` null-guard (item 10) is committed to + source but needs a burgerportaal image rebuild + rollout. Until then the webkit + sitemap 500 can still flake (it did in run 4; absent in runs 7–8). +- The one run-7 flake (`firefox @burgerportaal/zoeken.feature:116` — "A document + result offers a download", `page.waitForURL` 15s) did not recur in run 8. Left + alone; the single retry absorbs it if it returns. +- **Host**: something prunes `~/Library/Caches/ms-playwright` mid-session — Firefox, + WebKit and `chromium_headless_shell-1169` all vanished between runs and had to be + reinstalled with `./node_modules/.bin/playwright install` (use the _project's_ + binary; the cache holds two layouts, `chromium-1169` and `chromium-1234`, plus a + `daemon/` from the playwright-cli skill). + +--- + +## T1 — sitemap.feature: 30s test timeout on the sitemap walk (14 non-green) ⬅ biggest + +`Test timeout of 30000ms exceeded.` +chromium 102/108/114/120/126/132 · firefox 54/60/66/72/78/84/90/96 + +`collectAllUrlEntries` GETs the index + every monthly sitemap. Measured: the +monthly sitemap takes **6.7s** to serve. Each scenario walks it at least once, +change-propagation scenarios walk it twice, and 3 browsers × N workers hit the +same endpoint concurrently. 30s (Playwright default) does not fit. + +## T2 — sitemap.feature: freshly seeded document absent from the sitemap (7 non-green) + +`Error: document E2E doc -0- present in a sitemap → expect(received).toBeTruthy()` +chromium 54/60/66/72/78/96 · webkit 54 (flaky), webkit 114 (`Received: undefined`) + +Seed-then-read with no tolerance for propagation lag. Cluster has +`SITEMAP_CACHE_DURATION_HOURS=0`, so verify whether this is an output cache, an +ODRC read lag, or genuinely missing data before touching the steps. + +## T3 — burgerportaal beheer: `Beheermenu` navigation never visible (2) + +`Timed out 5000ms waiting for expect(locator).toBeVisible()` +`getByRole('navigation', { name: 'Beheermenu' })` +chromium @beheer/configuratie.feature 11 (flaky) + 41 (failed) + +## T4 — publicatiebank: `POST documenten -> 502 Bad Gateway` (2 flaky) + +chromium @burgerportaal/zoeken 82 · chromium @admin/documenten 49 +nginx/uwsgi 502 under parallel load during document seeding. + +## T5 — zoeken: second search result article missing (2) + +`getByRole('article').nth(1).getByText('Document', { exact: true })` 5s timeout +chromium zoeken 89 (failed) · firefox zoeken 89 (flaky) + +## T6 — zoeken: `page.waitForURL` 15s timeout (1) + +firefox @burgerportaal/zoeken 123 — "A document result links to its publicatie" + +## T7 — gpp-app "Bekijk online" points at the wrong origin (1) + +`expect(received).toContain("http://localhost:8140")` — got `:8130` +chromium @gpp-app/publicaties 113. Known GPP-app deploy-config bug, not a test bug. + +## T8 — informatiecategorieen: 30s timeout in `beforeEach` (1 flaky) + +firefox @admin/informatiecategorieen 75 — "Fully edit a self-added information category" + +## T9 — organisaties: `expect(received).toBeGreaterThan(0)` (1 flaky) + +chromium @admin/organisaties 39 — "Sort the organisaties alphabetically by name" + +## T10 — organisaties: `expect(...).resolves.toBeTruthy()` rejected (1 flaky) + +webkit @admin/organisaties 50 — "Active organisaties match the GPP-app waardelijst" + +## T11 — publicaties: select2 option never visible (1 flaky) + +`locator('.select2-container--open .select2-results__option').filter({hasText:'E2E 24-0-…'})` 8s +chromium @admin/publicaties 39 — "Edit the informatiecategorieën of a publicatie" + +--- + +## ROOT CAUSE for T3, T4, T8–T11: the Docker VM was memory-starved + +`docker exec gpp-e2e-control-plane free -m` → **7935 MB total, 340 MB available, +swapping**. `gpp-publicatiebank` last-terminated with **exit 137 (SIGKILL)**, no +pod resource limits set — the kernel was reaping the biggest process under node +memory pressure. + +Every one of T3, T4, T8–T11 passes in isolation on a quiet stack: + +| item | isolated result | +| --------------------------------------- | ------------------ | +| T3 beheer/configuratie (chromium) | 16/16 pass | +| T8 informatiecategorieen (firefox) | 18/18 pass | +| T9+T10 organisaties (chromium / webkit) | 12/12 / 12/12 pass | +| T11 publicaties (chromium) | 25/25 pass | + +So they were never test bugs — a starved backend surfacing as unrelated UI +timeouts. Addressed by capping local `workers` to 4 in `playwright.config.ts`. + +The VM was later resized to **32 GB (24 GB available)**, which restarted every +pod mid-run-2 and made run 2's 502-heavy output meaningless as a measurement. + +## Fixes applied + +1. `bdd/@burgerportaal/support/sitemap.ts` — `pollUntil` + `findDocumentEntry` + polls; new `currentMonthEntriesUntil` / `currentMonthSitemapPath`. + The sitemap is eventually consistent with the token API. +2. `bdd/@burgerportaal/sitemap.steps.ts` — 6 change-propagation Thens poll for + the expected presence instead of reading one possibly-stale body. +3. `bdd/@burgerportaal/sitemap.feature` — `@timeout:120000` (the walk is O(n) in + seeded documents; 6.7s per fetch mid-suite). +4. `bdd/@burgerportaal/zoeken.steps.ts` — `searchResults()` scoped to the result + list + `resultTypeLabels()`; the filter assertion polls the rendered labels + instead of snapshotting a pre-filter count. +5. `setup/provision-documenten-api.sh` — provisions the two GlobalConfiguration + publication URL templates (both shipped **empty**, so `urlPublicatieExtern` + was `""` and "Bekijk online" reloaded the GPP-app → T7); and keys every + `Service` on `api_root` (the UNIQUE column) so re-running actually works. +6. `playwright.config.ts` — `workers: process.env.CI ? 1 : 4`. + +### Round 2 (failures revealed once the stack was healthy) + +7. `bdd/@gpp-app/support/publicatie-ui.ts` — `intrekkenDialog` located by heading. + `PromptModal.vue` renders a bare `` with **no** aria-label/labelledby, + and a `` does not take its name from an inner `

`, so + `getByRole('dialog', { name: /intrekken/i })` matched nothing, ever. + _(The missing accessible name is a real a11y defect in the GPP-app — screen + readers announce every PromptModal unnamed. Worth fixing there.)_ +8. `bdd/@gpp-app/publicaties.steps.ts` — confirm the intrekken dialog via the + existing `confirmWrite` retry helper instead of a bare click. The confirm + button is clickable a beat before the app wires its handler (already + documented in that file), so the click was dropped and nothing was withdrawn. +9. `bdd/@gpp-app/support/publicatie-ui.ts` + steps — new `publicatieListItem`; + the ingetrokken assertion now reads the row's `role="status"`. The old + page-wide `getByText(/ingetrokken/i).first()` resolved to the **hidden** + `