diff --git a/.changeset/android-unreadable-snapshot-poll.md b/.changeset/android-unreadable-snapshot-poll.md new file mode 100644 index 000000000..a671d67de --- /dev/null +++ b/.changeset/android-unreadable-snapshot-poll.md @@ -0,0 +1,5 @@ +--- +'@e2e-dev/mobile': patch +--- + +Locators and `expect` now keep waiting while Android reports insufficient foreground app content (an app still loading after a cold start) instead of failing at once. A wait that runs out on an unreadable screen fails with that reason. diff --git a/.changeset/bump-agent-device-0-21-28.md b/.changeset/bump-agent-device-0-21-28.md new file mode 100644 index 000000000..0847ef1e8 --- /dev/null +++ b/.changeset/bump-agent-device-0-21-28.md @@ -0,0 +1,6 @@ +--- +"@e2e-dev/mobile": patch +"@e2e-dev/limrun": patch +--- + +Bump agent-device to 0.21.28. diff --git a/.changeset/connect-cdp-endpoint-required.md b/.changeset/connect-cdp-endpoint-required.md new file mode 100644 index 000000000..8adaa230b --- /dev/null +++ b/.changeset/connect-cdp-endpoint-required.md @@ -0,0 +1,5 @@ +--- +"@e2e-dev/web": patch +--- + +`web({ connect })` is `INVALID_CONFIG` at config load when `cdpEndpoint` is missing or not a function, instead of failing every attempt with `ENGINE_FAILURE: connection failed: resolve is not a function` (with `reconnectEndpoint`) or worker init with `browser connect failed: source.resolve is not a function` (without it). diff --git a/.changeset/lightpanda-provider.md b/.changeset/lightpanda-provider.md new file mode 100644 index 000000000..460fa88a8 --- /dev/null +++ b/.changeset/lightpanda-provider.md @@ -0,0 +1,5 @@ +--- +"@e2e-dev/lightpanda": patch +--- + +New package: `lightpanda()`, a browser provider that starts a `lightpanda serve` per worker slot on a free port and stops it when the run ends. diff --git a/.changeset/scroll-strides-back-to-back.md b/.changeset/scroll-strides-back-to-back.md new file mode 100644 index 000000000..915841094 --- /dev/null +++ b/.changeset/scroll-strides-back-to-back.md @@ -0,0 +1,6 @@ +--- +"e2e": patch +"@e2e-dev/web": patch +--- + +`scrollUntilVisible` no longer pauses 100 ms between swipes. On web, a swipe resolves once the scroll it caused is on screen, so the next read sees the scrolled page. diff --git a/AGENTS.md b/AGENTS.md index e2d60cfb1..ad1a79cc4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -89,6 +89,9 @@ suites that consume the built packages the way a user would. engine's provider seam (`BrowserProvider` for web, `DeviceProvider` for mobile) and imports the engine's types only; the engines never know it exists. +- `packages/lightpanda` - the published `@e2e-dev/lightpanda` package: a + `BrowserProvider` that starts a local `lightpanda serve` per worker slot + and stops it with the run; no vendor SDK, the binary is the dependency. - `packages/eas` - the published `@e2e-dev/eas` package: EAS Simulators hosted iOS simulators and Android emulators for the mobile engine (`DeviceProvider`). Expo publishes no SDK for the sessions API, so it calls diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3dbb4f0da..f7f3083b0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -288,5 +288,5 @@ build against, and what a third-party engine builds against too. A change to that contract bumps all three packages together in one release, with a changeset for each, so an engine and a runner from the same release always match. Engines declare a peer range on `e2e` that points one way only -(engine to runner, `>=x <1`), and each integration (`@e2e-dev/kernel`, `@e2e-dev/eas`, `@e2e-dev/smol`, `@e2e-dev/testmu`) does +(engine to runner, `>=x <1`), and each integration (`@e2e-dev/kernel`, `@e2e-dev/lightpanda`, `@e2e-dev/eas`, `@e2e-dev/smol`, `@e2e-dev/testmu`) does the same on the engines it plugs into; do not make it mutual or narrow it. diff --git a/README.md b/README.md index 0309e0470..1a0671c4b 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,7 @@ and Flutter, each a standalone project with a passing suite. | [`@e2e-dev/mobile`](https://www.npmjs.com/package/@e2e-dev/mobile) | iOS and Android engine: simulators and emulators through agent-device. | | [`@e2e-dev/github`](https://www.npmjs.com/package/@e2e-dev/github) | Reporter that posts results as a pull request comment. | | [`@e2e-dev/kernel`](https://www.npmjs.com/package/@e2e-dev/kernel) | Kernel hosted browsers for the web engine. | +| [`@e2e-dev/lightpanda`](https://www.npmjs.com/package/@e2e-dev/lightpanda) | Lightpanda, the open-source headless browser, started per run for the web engine. | | [`@e2e-dev/limrun`](https://www.npmjs.com/package/@e2e-dev/limrun) | Limrun hosted iOS simulators and Android emulators for mobile tests. | | [`@e2e-dev/eas`](https://www.npmjs.com/package/@e2e-dev/eas) | EAS Simulators hosted iOS simulators and Android emulators for the mobile engine. | | [`@e2e-dev/smol`](https://www.npmjs.com/package/@e2e-dev/smol) | smol machines browsers for the web engine: by default, each attempt branches a warm Chromium microVM. | diff --git a/apps/testbed/README.md b/apps/testbed/README.md index 17df95223..ff3dbbfd2 100644 --- a/apps/testbed/README.md +++ b/apps/testbed/README.md @@ -28,6 +28,11 @@ here. swipes (`pointer`), viewport and node scrolling (`scroll`), the `browser` fixture's own verbs (`browser`), and the per-action speed floor (`speed`). `tests/helpers.ts` holds the shared `failure`, `boxOf`, and `centerOf`. +- `e2e.lightpanda.config.ts` and `e2e.lightpanda-agent.config.ts`: the + local suite and the agent suite on `@e2e-dev/lightpanda`, which starts + [Lightpanda](https://lightpanda.io) from `LIGHTPANDA_PATH` (or `PATH`), + run by hand to see which features its nightly covers; the docs page + `integrations/lightpanda` lists the result. - `e2e.agent.config.ts` + `tests-agent/` — opt-in agentic suite against the playground: `agent.act` flows, assisted polling, judgments, schema-validated extraction with zod, and the pixel tools on the canvas diff --git a/apps/testbed/e2e.lightpanda-agent.config.ts b/apps/testbed/e2e.lightpanda-agent.config.ts new file mode 100644 index 000000000..52a154e86 --- /dev/null +++ b/apps/testbed/e2e.lightpanda-agent.config.ts @@ -0,0 +1,23 @@ +import type { E2EConfig } from 'e2e'; +import { web } from '@e2e-dev/web'; +import { lightpanda } from '@e2e-dev/lightpanda'; +import agentConfig from './e2e.agent.config.ts'; + +/** + * The agent suite on Lightpanda, run manually with a model key: + * + * AI_GATEWAY_API_KEY=... pnpm --filter @e2e-dev/testbed test:lightpanda:agent + * + * The failures to expect are the drawn, dragged, and below-the-fold flows + * the Lightpanda integration page lists. + */ +export default { + ...agentConfig, + projectId: 'dev.e2e.testbed-agent-lightpanda', + targets: agentConfig.targets.map((target) => ({ + ...target, + name: 'lightpanda', + engine: web({ browser: lightpanda({ loadResources: ['iframe'] }) }), + app: { ...target.app, url: 'http://127.0.0.1:4273', command: { ...target.app.command, env: { PORT: '4273' } } }, + })), +} satisfies E2EConfig; diff --git a/apps/testbed/e2e.lightpanda.config.ts b/apps/testbed/e2e.lightpanda.config.ts new file mode 100644 index 000000000..6a800927d --- /dev/null +++ b/apps/testbed/e2e.lightpanda.config.ts @@ -0,0 +1,15 @@ +import type { E2EConfig } from 'e2e'; +import { web } from '@e2e-dev/web'; +import { lightpanda } from '@e2e-dev/lightpanda'; +import base from './e2e.config.ts'; + +/** + * The local suite on Lightpanda, run by hand to see which features its + * nightly covers: + * + * pnpm --filter @e2e-dev/testbed test:lightpanda + */ +export default { + ...base, + targets: base.targets.map((target) => ({ ...target, name: 'lightpanda', engine: web({ browser: lightpanda({ loadResources: ['iframe'] }) }) })), +} satisfies E2EConfig; diff --git a/apps/testbed/package.json b/apps/testbed/package.json index 715e90198..b8558a21f 100644 --- a/apps/testbed/package.json +++ b/apps/testbed/package.json @@ -8,6 +8,8 @@ "typecheck": "tsc --noEmit", "test": "tsc --noEmit && node node_modules/e2e/dist/cli/bin.js run", "test:headed": "node node_modules/e2e/dist/cli/bin.js run --headed", + "test:lightpanda": "node node_modules/e2e/dist/cli/bin.js run --config e2e.lightpanda.config.ts", + "test:lightpanda:agent": "node node_modules/e2e/dist/cli/bin.js run --config e2e.lightpanda-agent.config.ts", "app": "node app/server.mjs", "test:agent": "node node_modules/e2e/dist/cli/bin.js run --config e2e.agent.config.ts", "test:dogfood": "node node_modules/e2e/dist/cli/bin.js run --config e2e.dogfood.config.ts", @@ -17,6 +19,7 @@ "explore:bench": "node scripts/explore-bench.mjs" }, "dependencies": { + "@e2e-dev/lightpanda": "workspace:*", "@e2e-dev/web": "workspace:*", "e2e": "workspace:*" }, diff --git a/docs/browser.mdx b/docs/browser.mdx index dd5d28490..916bad48e 100644 --- a/docs/browser.mdx +++ b/docs/browser.mdx @@ -201,7 +201,9 @@ bunx @e2e-dev/web install webkit --with-deps ``` -`connect` attaches to a remote Chromium over CDP instead of launching one. +`connect` attaches to a remote Chromium over CDP instead of launching one, +or to another browser that speaks CDP, such as a running +[Lightpanda](/integrations/lightpanda). To set sizes, see [Viewports](/web#viewports). ## Hosted browsers @@ -215,14 +217,15 @@ when the run no longer uses it. | --- | --- | | [Kernel](/integrations/kernel) | `kernel()` from `@e2e-dev/kernel` | | [smol machines](/integrations/smol) microVMs on your own computer | `smol()` from `@e2e-dev/smol` | +| [Lightpanda](/integrations/lightpanda), the open-source headless browser, on your own machine | `lightpanda()` from `@e2e-dev/lightpanda` | | Any other service | Your own provider, a few dozen lines in your project | When the service shows the window in a live view, add `viewport: null` so the page [fills the window](/web#fill-the-window). -A provider implies Chromium, the only browser CDP attach works on, and -can't be combined with `connect`. A target with nothing to run leases -nothing. +A provider implies a browser the engine drives as Chromium over CDP, +Chromium itself or Lightpanda, and can't be combined with `connect`. A +target with nothing to run leases nothing. ### Write a provider diff --git a/docs/bug-bash.mdx b/docs/bug-bash.mdx index 5c305263e..80a09bf8f 100644 --- a/docs/bug-bash.mdx +++ b/docs/bug-bash.mdx @@ -78,7 +78,7 @@ Bug bash the checkout and account settings on this branch. environment, the intended design, or the seed data. 6. Writes a repro test that asserts the expected behavior for each remaining finding. It can start one subagent per area, each with its own - [`e2e mcp` session](/reference/mcp#several-sessions) to check locators. + [`e2e mcp` session](/mcp) to check locators. 7. Runs each repro test. The finding is a bug only when the test fails with `ASSERTION_FAILED`. 8. Reports, in this order: confirmed bugs, production risks that it cannot @@ -124,7 +124,7 @@ regression test. What one `e2e explore` run does, its budgets, and its report. - + Live sessions for coding agents, several at once. diff --git a/docs/coding-agents.mdx b/docs/coding-agents.mdx index 87670b16a..eb8ef3545 100644 --- a/docs/coding-agents.mdx +++ b/docs/coding-agents.mdx @@ -87,39 +87,10 @@ codex plugin marketplace upgrade e2e ## Let the agent use the app -`e2e mcp` lets the coding agent open the app and use it. Each agent, -subagents included, opens its own session, so several can use the app at -the same time. The `locate` tool checks a `screen.*` locator against the -current app and returns the test code when exactly one element matches. - -`init` registers the server in `.mcp.json` for Claude Code and -`.cursor/mcp.json` for Cursor. To register it with Claude Code by hand: - - -```bash npm -claude mcp add --scope project e2e -- npx e2e mcp -``` - -```bash pnpm -claude mcp add --scope project e2e -- pnpm exec e2e mcp -``` - -```bash bun -claude mcp add --scope project e2e -- bunx e2e mcp -``` - - -For Codex, add the server to the project's `.codex/config.toml`, trust the -project, and start a new session: - -```toml -[mcp_servers.e2e] -command = "npx" -args = ["e2e", "mcp"] -``` - -The server does not run tests. See the [MCP server reference](/reference/mcp) -for its tools, session rules, and flags. +`e2e mcp` gives the agent a live session on your app. It checks each +locator against the screen before it writes the test, and checks its own UI +changes. `init` registers the server. See [MCP server](/mcp) for setup and +prompts. ## How an agent writes and runs a test diff --git a/docs/docs.json b/docs/docs.json index 606eb2b47..10f900b03 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -134,7 +134,8 @@ "index", "quickstart", "core-concepts", - "coding-agents" + "coding-agents", + "mcp" ] }, { @@ -180,6 +181,7 @@ "pages": [ "integrations/index", "integrations/kernel", + "integrations/lightpanda", "integrations/eas", "integrations/smol", "integrations/limrun", diff --git a/docs/images/icons/lightpanda.svg b/docs/images/icons/lightpanda.svg new file mode 100644 index 000000000..147095508 --- /dev/null +++ b/docs/images/icons/lightpanda.svg @@ -0,0 +1 @@ + diff --git a/docs/images/integrations/lightpanda.svg b/docs/images/integrations/lightpanda.svg new file mode 100644 index 000000000..147095508 --- /dev/null +++ b/docs/images/integrations/lightpanda.svg @@ -0,0 +1 @@ + diff --git a/docs/images/og/integrations/lightpanda.png b/docs/images/og/integrations/lightpanda.png new file mode 100644 index 000000000..e3bde4a3f Binary files /dev/null and b/docs/images/og/integrations/lightpanda.png differ diff --git a/docs/images/og/mcp.png b/docs/images/og/mcp.png new file mode 100644 index 000000000..1a7523ae8 Binary files /dev/null and b/docs/images/og/mcp.png differ diff --git a/docs/integrations/index.mdx b/docs/integrations/index.mdx index 283bcca14..ac3cbe3f8 100644 --- a/docs/integrations/index.mdx +++ b/docs/integrations/index.mdx @@ -15,6 +15,9 @@ Only the target's `browser` or `device` in the config changes. `@e2e-dev/kernel`: hosted Chromium for web targets, one browser per worker or per test. + + `@e2e-dev/lightpanda`: the open-source headless browser, started per worker slot on your own machine, no Chromium. + `@e2e-dev/eas`: hosted iOS simulators and Android emulators for mobile targets. diff --git a/docs/integrations/lightpanda.mdx b/docs/integrations/lightpanda.mdx new file mode 100644 index 000000000..9e003d7ee --- /dev/null +++ b/docs/integrations/lightpanda.mdx @@ -0,0 +1,192 @@ +--- +title: Lightpanda +sidebarTitle: Lightpanda +description: Run web targets in Lightpanda, the open-source headless browser, with the lightpanda() browser provider from @e2e-dev/lightpanda. +icon: "/images/icons/lightpanda.svg" +"og:image": "https://e2e.tester.army/docs/images/og/integrations/lightpanda.png?v=b97a3ebae4" +"twitter:image": "https://e2e.tester.army/docs/images/og/integrations/lightpanda.png?v=b97a3ebae4" +--- + +[Lightpanda](https://lightpanda.io) is an open-source headless browser +written from scratch, without Chromium. It starts in milliseconds, holds a +fraction of Chrome's memory, and speaks the Chrome DevTools Protocol. +`lightpanda()` from `@e2e-dev/lightpanda` is a +[browser provider](/browser#hosted-browsers) for it: each worker slot gets +its own `lightpanda serve`, started on a free port when the run starts and +stopped when it ends. Tests and CI stay the same; the config changes only +in the target's `browser`. + +Lightpanda has no layout engine. It builds the DOM, runs JavaScript, and +exposes the accessibility tree the engine reads, which is what locators, +assertions, and [agent steps](/agent-steps) use. Anything that depends on +where a node is painted behaves differently; see +[What does not work](#what-does-not-work) before moving a suite over. + +## Setup + +Install the package: + + +```bash npm +npm install --save-dev @e2e-dev/lightpanda +``` + +```bash pnpm +pnpm add -D @e2e-dev/lightpanda +``` + +```bash bun +bun add -d @e2e-dev/lightpanda +``` + + +And the Lightpanda nightly binary, which the provider starts: + + +```bash macOS +brew install lightpanda-io/browser/lightpanda +``` + +```bash Linux +curl -L -o lightpanda https://github.com/lightpanda-io/browser/releases/download/nightly/lightpanda-x86_64-linux +install -D -m 755 lightpanda ~/.local/bin/lightpanda +``` + + +```ts title="e2e.config.ts" +import type { E2EConfig } from 'e2e'; +import { web } from '@e2e-dev/web'; +import { lightpanda } from '@e2e-dev/lightpanda'; + +export default { + targets: [ + { + name: 'lightpanda', + engine: web({ browser: lightpanda() }), + app: { url: 'http://localhost:3000' }, + }, + ], +} satisfies E2EConfig; +``` + +The provider runs `lightpanda` from `PATH`, `~/.lightpanda/lightpanda`, or +`~/.local/bin/lightpanda`, the places +[agent-browser](https://agent-browser.dev/engines/lightpanda) looks too, or +the binary `LIGHTPANDA_PATH` names in the environment `e2e run` starts in, +or `binary` in the options. None found fails the run before any test, +naming the install page. +Each worker gets its own server, so `workers` sets how many run at once, +and the run's log names each one: + +```text +ℹ lightpanda (1 of 2): lightpanda 1.1.0-nightly at ws://127.0.0.1:51165/ +``` + +## Attach to a running server + +A Lightpanda already running, such as the Docker image in CI, is a +[`connect`](/reference/web#options) target rather than a provider: the +engine attaches to its CDP endpoint and leaves it running. + +```ts title="e2e.config.ts" +web({ connect: { cdpEndpoint: () => process.env.LIGHTPANDA_URL ?? 'ws://127.0.0.1:9222/' } }) +``` + +```bash +docker run -d --name lightpanda -p 127.0.0.1:9222:9222 lightpanda/browser:nightly +``` + +Every worker then shares that server. One `lightpanda serve` takes 16 +connections by default; raise its `--cdp-max-connections` above `workers` +when you run more. + +## Options + +| Option | Default | Meaning | +| --------------- | ------------------------------- | ------- | +| `binary` | `LIGHTPANDA_PATH`, else found | The binary to start: a path, or a name on `PATH`. | +| `loadResources` | `[]` | Sub-resources the server fetches: `iframe`, `image`, `stylesheet`. See [Fetch more of the page](#fetch-more-of-the-page). | +| `args` | `[]` | Further `lightpanda serve` flags, after the ones the provider sets. | + +The provider leases one server per worker slot for the run; it has no +`scope: 'attempt'`, since Lightpanda does not expose the default context +identity a per-attempt lease reattaches by. A server that does not answer +on its port within ten seconds fails the lease, and so does a binary that +exits first, with its stderr in the message. + +## Fetch more of the page + +`lightpanda serve` requests no sub-resources by default: no images, no +external stylesheets, no iframes. `loadResources` opts into each, as +`--load-resources` on the server: + +```ts +web({ browser: lightpanda({ loadResources: ['stylesheet', 'iframe'] }) }) +``` + +Inline styles, `