Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/testmuai-browsers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@e2e-dev/testmuai": minor
---

`@e2e-dev/testmuai`: `web({ browser: testmuai() })` runs a web target in TestMu AI's hosted Chrome or Edge on Windows and macOS, one session per worker slot, or per attempt with `scope: 'attempt'`. It builds the session's CDP URL from `browserName`, `browserVersion`, `platform`, `build`, and further `LT:Options` `capabilities`, names each session after its target and its slot or attempt inside one build per run, reads `LT_USERNAME` and `LT_ACCESS_KEY` from the run's environment and keeps them out of every log line, and serves raw CDP from the hub's `/puppeteer` route by default, with `/playwright-cdp` as an option. A TestMu AI session starts when its websocket opens and ends when it closes, so it calls no API and has no SDK; `@e2e-dev/web` is its peer.
5 changes: 5 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ suites that consume the built packages the way a user would.
engine's provider seam (`BrowserProvider` for web, `DeviceProvider` for
mobile) and imports the engine's types only; the engines never know it
exists.
- `packages/testmuai` - the published `@e2e-dev/testmuai` package:
TestMu AI hosted Chrome and Edge for the web engine (`BrowserProvider`).
A TestMu AI session starts when its CDP websocket opens and ends when it
closes, so the provider only builds the URL: no SDK, no API call, and
`@e2e-dev/web` is its only peer.
- `packages/eas` - the published `@e2e-dev/eas` package: EAS Simulators
hosted iOS simulators and Android emulators for the mobile engine
(`DeviceProvider`). Expo publishes no SDK for the sessions API, so it calls
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,5 +254,5 @@ build against, and what a third-party engine builds against too. A change to
that contract bumps all three packages together in one release, with a
changeset for each, so an engine and a runner from the same release always
match. Engines declare a peer range on `e2e` that points one way only
(engine to runner, `>=x <1`), and each integration (`@e2e-dev/kernel`, `@e2e-dev/eas`) does
(engine to runner, `>=x <1`), and each integration (`@e2e-dev/kernel`, `@e2e-dev/testmuai`, `@e2e-dev/eas`) does
the same on the engines it plugs into; do not make it mutual or narrow it.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ config and an example test. The
| [`@e2e-dev/mobile`](https://www.npmjs.com/package/@e2e-dev/mobile) | iOS and Android engine: simulators and emulators through agent-device. |
| [`@e2e-dev/github`](https://www.npmjs.com/package/@e2e-dev/github) | Reporter that posts results as a pull request comment. |
| [`@e2e-dev/kernel`](https://www.npmjs.com/package/@e2e-dev/kernel) | Kernel hosted browsers for the web engine. |
| [`@e2e-dev/testmuai`](https://www.npmjs.com/package/@e2e-dev/testmuai) | TestMu AI hosted Chrome and Edge for the web engine. |
| [`@e2e-dev/eas`](https://www.npmjs.com/package/@e2e-dev/eas) | EAS Simulators hosted iOS simulators and Android emulators for the mobile engine. |

## Documentation
Expand Down
5 changes: 3 additions & 2 deletions docs/browser.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -158,8 +158,9 @@ lacks: it asks for a browser when one is needed and gives it back when its
scope ends, on every exit path, so a session billed by the minute stops
when the run no longer uses it. The engine knows no vendor: for
[Kernel](/integrations/kernel), `kernel()` from `@e2e-dev/kernel` is the
provider, and for any other service a provider is a few dozen lines in your
project. Add `viewport: null` when
provider, for [TestMu AI](/integrations/testmuai) it is `testmuai()` from
`@e2e-dev/testmuai`, and for any other service a provider is a few dozen lines
in your project. Add `viewport: null` when
the service shows the window in a live view: the page then
[fills the window](/viewports#fill-the-window) instead of a fixed 1280 by
720 area of it.
Expand Down
1 change: 1 addition & 0 deletions docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,7 @@
"pages": [
"integrations/index",
"integrations/kernel",
"integrations/testmuai",
"integrations/eas"
]
},
Expand Down
3 changes: 3 additions & 0 deletions docs/integrations/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ integration decides where it comes from.
<Card title="Kernel" icon="/images/integrations/kernel.svg" href="/integrations/kernel">
`@e2e-dev/kernel`: hosted Chromium for web targets, one browser per worker or per test.
</Card>
<Card title="TestMu AI" icon="cloud" href="/integrations/testmuai">
`@e2e-dev/testmuai`: hosted Chrome and Edge on Windows and macOS for web targets, one session per worker or per test.
</Card>
<Card title="EAS Simulators" icon="/images/integrations/expo.svg" href="/integrations/eas">
`@e2e-dev/eas`: hosted iOS simulators and Android emulators for mobile targets.
</Card>
Expand Down
159 changes: 159 additions & 0 deletions docs/integrations/testmuai.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
---
title: TestMu AI browsers
sidebarTitle: TestMu AI
description: Run web targets in TestMu AI's hosted Chrome and Edge with the testmuai() browser provider from @e2e-dev/testmuai.
---

`testmuai()` from `@e2e-dev/testmuai` is a
[browser provider](/browser#hosted-browsers) for
[TestMu AI](https://www.lambdatest.com). By default each worker slot gets its
own TestMu AI session for the run; `scope: 'attempt'` gives every test
attempt its own session instead. Tests and CI stay the same; the config
changes only in the target's `browser`.

## Setup

<CodeGroup>
```bash npm
npm install --save-dev @e2e-dev/testmuai
```

```bash pnpm
pnpm add -D @e2e-dev/testmuai
```

```bash bun
bun add -d @e2e-dev/testmuai
```
</CodeGroup>

```ts title="e2e.config.ts"
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testmuai } from '@e2e-dev/testmuai';

export default {
targets: [
{
name: 'testmuai',
engine: web({ browser: testmuai(), viewport: null }),
app: { url: 'https://staging.example.com' },
},
],
} satisfies E2EConfig;
```

Set `LT_USERNAME` and `LT_ACCESS_KEY` in the environment `e2e run` starts in,
such as CI secrets. To get them, [sign up for TestMu AI](https://accounts.lambdatest.com/register)
or log in to your account, then copy your username and access key from
**Account Settings → Password & Security → Username and Access Key**
([accounts.lambdatest.com/security/username-accesskey](https://accounts.lambdatest.com/security/username-accesskey)).

Each worker gets its own session, so `workers` sets how
many run at once. Keep it within your TestMu AI plan's parallel sessions:
a session beyond it waits in TestMu AI's queue, and the web engine gives a
session 60 seconds to accept the CDP connection, so one that waits longer
fails its attempt. See [Limits](#limits).

## Options

| Option | Default | Meaning |
| ------ | ------- | ------- |
| `scope` | `'worker'` | `'worker'`: one session per worker slot for the run. `'attempt'`: a fresh session per test attempt, retries included. See [Scopes](#scopes). |
| `browserName` | `'Chrome'` | `'Chrome'` or `'MicrosoftEdge'`. The web engine attaches over CDP, so the browser must be Chromium-based; any other value fails the config. |
| `browserVersion` | `'latest'` | Any version TestMu AI offers for the platform, such as `'140'`. |
| `platform` | `'Windows 11'` | A TestMu AI desktop platform, such as `'Windows 10'` or `'macOS Sequoia'`. |
| `build` | `'e2e <run id>'` | The TestMu AI build the sessions are grouped under. |
| `route` | `'/puppeteer'` | The hub route that serves the session's CDP endpoint. See [Routes](#routes). |
| `hub` | `'cdp.lambdatest.com'` | The hub host, without a scheme or path. |
| `capabilities` | none | Further `LT:Options` capabilities, such as `video`, `network`, `console`, `idleTimeout`, `resolution`, or `tunnel` and `tunnelName` for [TestMu AI Tunnel](https://www.lambdatest.com/support/docs/testing-locally-hosted-pages/). `idleTimeout` is 600 seconds when absent. `user`, `accessKey`, `build`, `name`, and `platform` fail the config here: the provider sets them from the environment and the options above. |

```ts
testmuai({
scope: 'attempt',
platform: 'Windows 10',
build: `checkout ${process.env.GITHUB_SHA}`,
capabilities: { video: true, network: true, idleTimeout: 300 },
})
```

A misspelled option, a `route` other than the two below, or a `hub` with a
scheme or path fails with `INVALID_CONFIG` when the config loads.

## Scopes

`scope: 'worker'` keeps one session per worker slot for the whole run and
gives every attempt a new browser context in it, as a local run does: a test
never sees another test's cookies, storage, or permissions. TestMu AI shows
it as one session, named after the target and the slot, that covers every
test the worker ran.

`scope: 'attempt'` starts a session per attempt, so each test is its own
TestMu AI session, named after the target and the attempt. It costs a
session start per test and carries the limits of a
[per-attempt lease](/browser#hosted-browsers): no `headers`, `basicAuth`, or
`userAgent`, no `app.clearState()`, and no [sessions](/authentication). A
dropped connection cannot be resumed. The engine reconnects to the same CDP
URL, which starts a new TestMu AI session under the same name; the engine
sees a different browser, closes it, and fails the attempt with
`CDP recovery failed`. That second session shows in the dashboard and counts
against your plan while it lasts.

## Routes

A TestMu AI session starts when its CDP websocket opens, on one of the
hub's routes:

- `'/puppeteer'` (default) serves a raw CDP endpoint and works with every
account today. TestMu AI's dashboard labels these sessions Puppeteer.
- `'/playwright-cdp'` is TestMu AI's route for Playwright's
`connectOverCDP`. Its sessions are labelled Playwright. Use it once
TestMu AI serves raw CDP on it for your account.

## Limits

- **60-second connect.** The web engine waits 60 seconds for the browser to
accept the CDP connection, and `launchTimeout` does not change that. A
session still in TestMu AI's queue, or a platform slow to provision (Edge on
macOS can take close to a minute), fails its attempt with
`connectOverCDP: Timeout 60000ms exceeded`. Windows with Chrome or Edge
connects well inside it.
- **About 25 workers per target in `worker` scope.** The engine hands every
worker's CDP URL to the workers in one environment variable capped at 16 KB,
and each URL carries its capabilities. Past about 25 workers (fewer with
many `capabilities`) the run fails at start with `ENGINE_FAILURE: the
browser leases take ... bytes`. `scope: 'attempt'` leases in each worker and
has no such ceiling.

## Recordings

TestMu AI records the session's screen itself when you pass
`capabilities: { video: true }`; watch it in the TestMu AI dashboard. The
provider does not hand that recording to e2e, so an attempt that records
[video](/reference/config#video) gets the web engine's screencast of the
page, as a local run does.

## Downloads

TestMu AI's browsers save downloads on their own machines, and the provider
has no way to read them back, so `browser.waitForDownload` fails with
`ENGINE_FAILURE` naming the provider. Assert on the download link or the
request instead.

## What the provider does

- Builds the session's CDP URL with the capabilities above and returns it as
the lease. It calls no TestMu AI API: opening the websocket starts the
session and the engine closing it ends the session, so `release` has
nothing to call.
- Names every session `e2e <target> <slot or attempt>` inside the build, and
logs `TestMu AI session "<name>" in build "<build>"` to the reporter as
each lease is made.
- Reads `LT_USERNAME` and `LT_ACCESS_KEY` from the run's environment and puts
them only in the CDP URL: they never appear in a log line. A missing or
blank one fails the lease with `LT_USERNAME and LT_ACCESS_KEY must be set`;
in `worker` scope that ends the run before any test.
- Sets `idleTimeout` to 600 seconds unless `capabilities` sets one: the hub
otherwise ends a session after 300 seconds without browser traffic, which a
slow step between two browser calls can exceed. The same timeout ends a
session a dead worker held.
1 change: 1 addition & 0 deletions docs/reference/environment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ What is sent and how to read it is on the [Telemetry](/telemetry) page.
| --- | --- | --- |
| `PLAYWRIGHT_BROWSERS_PATH` | value | Playwright's browser cache, honored by the engine's `prepare` hook |
| `KERNEL_API_KEY` | value | The API key `kernel()` from `@e2e-dev/kernel` creates browsers with; unset, the browser lease fails. See [Kernel](/integrations/kernel) |
| `LT_USERNAME`, `LT_ACCESS_KEY` | value | The TestMu AI username and access key `testmuai()` from `@e2e-dev/testmuai` starts sessions with, from Account Settings → Password & Security → Username and Access Key on your TestMu AI account; either unset, the browser lease fails. See [TestMu AI](/integrations/testmuai) |
| `EXPO_TOKEN` | value | The Expo access token `easSimulators()` from `@e2e-dev/eas` starts and stops simulator sessions with; unset, it uses the eas-cli login in `.expo/state.json` under `HOME` (`USERPROFILE` on Windows), and with neither the device lease fails. See [EAS Simulators](/integrations/eas) |
| `HOME`, `USERPROFILE` | value | The home `easSimulators()` finds the eas-cli login under when `EXPO_TOKEN` is unset: `USERPROFILE` on Windows, `HOME` elsewhere, the OS user's home when unset |
| `E2E_AGENT_DEVICE_POOL_<TARGET>_<digest>` | value | Written by `@e2e-dev/mobile` in `prepare` and read by each worker to find its booted device. Not meant to be set by hand |
Expand Down
3 changes: 3 additions & 0 deletions docs/security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,9 @@ Built-in features may connect to:
- the GitHub API, only from the opt-in `@e2e-dev/github` reporter
- the Kernel API and its hosted browsers, only from
[`kernel()`](/integrations/kernel)
- TestMu AI's CDP hub and its hosted browsers, only from
[`testmuai()`](/integrations/testmuai). The username and access key
travel in the session's CDP URL and are never logged
- the Expo API and the simulators' agent-device daemons, only from
[`easSimulators()`](/integrations/eas). It authenticates with `EXPO_TOKEN`,
else with the eas-cli login in `~/.expo/state.json`: on a CI runner, set
Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
],
"type": "module",
"scripts": {
"build": "pnpm --filter e2e run build && pnpm --filter @e2e-dev/web run build && pnpm --filter @e2e-dev/mobile run build && pnpm --filter @e2e-dev/github run build && pnpm --filter @e2e-dev/kernel run build && pnpm --filter @e2e-dev/eas run build",
"build": "pnpm --filter e2e run build && pnpm --filter @e2e-dev/web run build && pnpm --filter @e2e-dev/mobile run build && pnpm --filter @e2e-dev/github run build && pnpm --filter @e2e-dev/kernel run build && pnpm --filter @e2e-dev/testmuai run build && pnpm --filter @e2e-dev/eas run build",
"check": "pnpm run lint && pnpm run check:dead-code && pnpm run typecheck && pnpm run docs:check-errors && pnpm run check:peer-ranges && pnpm run docs:check",
"check:dead-code": "fallow dead-code",
"check:peer-ranges": "node scripts/check-peer-ranges.ts",
Expand All @@ -24,7 +24,7 @@
"canary": "node scripts/canary-changeset.ts && changeset version --snapshot canary && pnpm run build",
"canary:publish": "changeset publish --tag canary --no-git-tag && node scripts/restore-peer-ranges.ts",
"typecheck": "pnpm run build && pnpm -r run typecheck && tsc -p scripts",
"test": "pnpm run build && pnpm --filter e2e run test && pnpm --filter @e2e-dev/web run test && pnpm --filter @e2e-dev/mobile run test && pnpm --filter @e2e-dev/github run test && pnpm --filter @e2e-dev/kernel run test && pnpm --filter @e2e-dev/eas run test && pnpm run test:scripts",
"test": "pnpm run build && pnpm --filter e2e run test && pnpm --filter @e2e-dev/web run test && pnpm --filter @e2e-dev/mobile run test && pnpm --filter @e2e-dev/github run test && pnpm --filter @e2e-dev/kernel run test && pnpm --filter @e2e-dev/testmuai run test && pnpm --filter @e2e-dev/eas run test && pnpm run test:scripts",
"test:testbed": "pnpm run build && pnpm --filter @e2e-dev/testbed test",
"test:web-benchmark": "pnpm run build && pnpm --filter @e2e-dev/web-benchmark test",
"test:mobile-benchmark": "pnpm run build && pnpm --filter @e2e-dev/mobile-benchmark test",
Expand Down
Loading