Skip to content

Latest commit

 

History

History
263 lines (214 loc) · 9.63 KB

File metadata and controls

263 lines (214 loc) · 9.63 KB
name agent-browser-testing
description Run structured, automated test suites against web applications using agent-browser CLI. Use this skill whenever the user wants to test a web app, run E2E tests, verify user flows, regression test after deploys, validate forms, check responsive behavior, compare environments, or run any kind of browser-based test suite. Triggers include: "test my app", "run E2E tests", "verify the login flow", "check if signup works", "regression test", "smoke test", "test this URL", "validate this form", "compare staging vs prod", "run test suite", "browser tests", "test these user flows", "acceptance tests", "QA this page". Also triggers when user provides a test plan, test cases, or asks to verify specific functionality in a web application. Prefer this skill over writing raw Playwright/Puppeteer scripts — agent-browser is faster and more token-efficient.

Agent-Browser Testing

Run structured test suites against web applications using agent-browser CLI. Produces pass/fail results with screenshots as evidence for every assertion.

Prerequisites

agent-browser must be installed and available on PATH. If not:

npm install -g agent-browser
agent-browser install   # downloads Chrome (first time only)

Always use agent-browser directly — never npx agent-browser. The direct binary is the fast Rust client. npx routes through Node.js and is significantly slower.

Setup

Parameter Default Override example
Target URL (required) https://myapp.com, http://localhost:3000
Session name test-{domain} --session my-tests
Output directory ./test-output/ user specifies path
Test scope Infer from user request Test the checkout flow
Auth None Login as user@example.com
Viewport Default (1280×720) Test at mobile size

If the user says something like "test myapp.com", start immediately. Ask clarifying questions only when the target URL is missing or authentication is mentioned without credentials.

Core Workflow

1. Plan         Define test cases from user request
2. Initialize   Session, output dirs, results file
3. Authenticate Sign in if needed
4. Execute      Run each test case: actions → assertions → evidence
5. Report       Write structured results with pass/fail per test

1. Plan Test Cases

Translate the user's request into concrete test cases. Each test case has:

  • ID: TC-001, TC-002, etc.
  • Name: Short description (e.g., "Login with valid credentials")
  • Steps: Ordered list of actions
  • Assertions: What to verify after steps complete
  • Severity: critical | high | medium | low

If the user provides a test plan or specific flows, follow those. Otherwise, infer sensible tests from the target URL and user intent. Read references/test-patterns.md for common patterns.

Present the test plan to the user before execution:

Test Plan for {URL}:
  TC-001: [critical] Login with valid credentials
  TC-002: [critical] Login with invalid credentials shows error
  TC-003: [high] Navigation links work
  ...
Proceed? (or suggest changes)

If the user says "just test it" or similar, skip confirmation and run.

2. Initialize

mkdir -p {OUTPUT_DIR}/screenshots {OUTPUT_DIR}/evidence

Copy the results template:

cp {SKILL_DIR}/templates/test-results-template.md {OUTPUT_DIR}/results.md

Start session:

agent-browser --session {SESSION} open {TARGET_URL}
agent-browser --session {SESSION} wait --load networkidle

3. Authenticate (if needed)

agent-browser --session {SESSION} snapshot -i
# Identify login form, fill credentials
agent-browser --session {SESSION} fill @eX "{USERNAME}"
agent-browser --session {SESSION} fill @eY "{PASSWORD}"
agent-browser --session {SESSION} click @eZ
agent-browser --session {SESSION} wait --load networkidle
agent-browser --session {SESSION} state save {OUTPUT_DIR}/auth-state.json

For OTP flows, ask the user and wait for their response.

4. Execute Test Cases

Run each test case sequentially. For every test case:

a) Setup

Navigate to the starting state. Take a "before" snapshot:

agent-browser --session {SESSION} open {START_URL}
agent-browser --session {SESSION} wait --load networkidle
agent-browser --session {SESSION} snapshot -i

b) Execute steps

Perform each action in order. Between actions, take snapshots to verify the page state is as expected:

# Example: fill a form and submit
agent-browser --session {SESSION} fill @e3 "test@example.com"
agent-browser --session {SESSION} fill @e4 "password123"
agent-browser --session {SESSION} click @e5
agent-browser --session {SESSION} wait --load networkidle

c) Assert

After steps complete, verify assertions. Use these agent-browser commands for assertions:

Assertion type Command Example
Element exists snapshot -i then check refs Button "Submit" present
Element has text get text @eN Heading says "Welcome"
Element visible is visible @eN or is visible "selector" Error message shows
Element enabled/disabled is enabled @eN Submit button enabled
Element checked is checked @eN Checkbox is checked
URL changed get url Redirected to /dashboard
Title changed get title Title is "Dashboard"
Input has value get value @eN Field contains entered text
No console errors errors No JS errors
Page content snapshot (no -i) Page contains expected text
Visual state screenshot --annotate Screenshot for manual review
DOM change diff snapshot Snapshot changed after action

Assertion evaluation rules:

  • PASS: Assertion met exactly as specified
  • FAIL: Assertion not met — capture evidence immediately
  • SKIP: Could not evaluate (element not found, page didn't load)
  • WARN: Partially met or ambiguous — note why

d) Capture evidence

For EVERY test case (pass or fail), take an annotated screenshot:

agent-browser --session {SESSION} screenshot --annotate {OUTPUT_DIR}/screenshots/TC-{NNN}-result.png

For FAIL cases, also capture:

agent-browser --session {SESSION} errors > {OUTPUT_DIR}/evidence/TC-{NNN}-errors.txt
agent-browser --session {SESSION} console > {OUTPUT_DIR}/evidence/TC-{NNN}-console.txt
agent-browser --session {SESSION} screenshot --full {OUTPUT_DIR}/evidence/TC-{NNN}-full.png

e) Record result

Append to results.md immediately after each test. Do not batch.

5. Report

After all tests complete:

  1. Update summary counts in results.md (total, passed, failed, skipped)
  2. Calculate pass rate
  3. Close session:
agent-browser --session {SESSION} close
  1. Present results to user: total tests, pass rate, list of failures with severity

Advanced Patterns

Responsive Testing

Test at multiple viewports:

agent-browser --session {SESSION} set viewport 375 667    # iPhone SE
agent-browser --session {SESSION} set viewport 768 1024   # iPad
agent-browser --session {SESSION} set viewport 1920 1080  # Desktop

Environment Comparison

Compare staging vs production using diff:

agent-browser diff url {STAGING_URL} {PROD_URL} --screenshot

Visual Regression

Screenshot baseline, then compare after deploy:

agent-browser --session {SESSION} screenshot baseline.png
# ... deploy ...
agent-browser --session {SESSION} diff screenshot --baseline baseline.png

Network Monitoring

Track failed requests during test execution:

agent-browser --session {SESSION} network requests --status 4xx,5xx

Form Edge Cases

Test boundary inputs systematically:

  • Empty submission
  • Maximum length strings
  • Special characters (<script>alert(1)</script>, '; DROP TABLE users;--)
  • Unicode and emoji
  • Whitespace-only input

State Isolation

Use separate sessions for test isolation:

agent-browser --session test-login open {URL}
agent-browser --session test-signup open {URL}

Guidance

  • Snapshot before interacting. Always take snapshot -i before clicking or filling. Refs are invalidated by page changes — re-snapshot after navigation.
  • Wait for page loads. Use wait --load networkidle after navigation and form submissions. Use wait --text "Expected text" when waiting for async content.
  • Be specific with assertions. "Page looks right" is not an assertion. "H1 contains 'Welcome back, User'" is.
  • Test the unhappy path. Invalid inputs, empty forms, double-clicks, back-button behavior. The happy path usually works — bugs live in edge cases.
  • Check console errors. Run errors periodically. JS errors on page load are always bugs.
  • One thing per test case. Each TC should test one behavior. If login and dashboard load are separate concerns, they're separate test cases.
  • Write results incrementally. Append each test result immediately. If session dies, prior results survive.
  • Use diff for before/after. diff snapshot after an action is the most reliable way to verify something changed.
  • Don't assume element positions. Always use refs from the most recent snapshot, never hardcoded selectors. If a ref doesn't work, re-snapshot.
  • Scroll before snapshotting. Content below the fold won't have refs unless you scroll to it: agent-browser --session {SESSION} scroll down 300

References

Reference When to Read
references/test-patterns.md Start of session — common test patterns for forms, auth, nav, CRUD

Templates

Template Purpose
templates/test-results-template.md Copy into output dir as the results file