Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/testmu-browsers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@e2e-dev/testmu": minor
---

`@e2e-dev/testmu/web`: `web({ browser: testmuBrowsers() })` runs a web target in TestMu AI (formerly LambdaTest) hosted Chrome or Edge on Windows and macOS, one session per worker slot, or one per attempt with `scope: 'attempt'`. A session starts when its CDP websocket opens and ends when the engine closes it, so the provider only builds the CDP URL (`hub` `cdp.lambdatest.com`) from `browserName`, `browserVersion`, `platform`, `geoLocation`, `timezone`, and further `LT:Options` `capabilities`, with `LT_USERNAME` and `LT_ACCESS_KEY` from the run's environment, which never appear in a log line. `route` is `/puppeteer` by default, which serves raw CDP on every machine; `/playwright-cdp` labels sessions as Playwright but serves raw CDP only on machines with TestMu AI's newest backend, which is still rolling out. Sessions are named as `testmu()`'s are: `build` defaults to the run id, so a run's device and browser sessions share one build, `sessionName` to `e2e-<run id>-<target>-<slot or attempt id>`, and `project` (default `e2e`) is sent as `LT:Options.project`. `idleTimeout` defaults to 600 seconds. Unknown options, a `scope` or `route` outside their values, a `hub` with a scheme or path, a non-Chromium `browserName`, an empty or non-string string option, `null` for any option, and `capabilities` that set what the provider sets (`user`, `accessKey`, `build`, `name`, `project`, `platform`, `geoLocation`, `timezone`, `browserName`, `browserVersion`, or a nested `LT:Options`) fail with `INVALID_CONFIG` at config load. The package root stays device-only, and `@e2e-dev/mobile`, `@e2e-dev/web`, and `agent-device` are optional peers, so a project installs only the side it uses.
7 changes: 7 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,13 @@ suites that consume the built packages the way a user would.
cloud provider is the vendor client: the package allocates agent-device
leases from a daemon it starts per run, so `agent-device` (which must be
the same copy `@e2e-dev/mobile` uses) and `@e2e-dev/mobile` are its peers.
`@e2e-dev/testmu/web` exports `testmuBrowsers()`, TestMu AI hosted Chrome
and Edge for the web engine (`BrowserProvider`): a session starts when its
CDP websocket opens and ends when it closes, so it only builds the URL and
calls no API. The root stays device-only, so neither side's types or
imports reach the other. Every
peer (`@e2e-dev/mobile`, `@e2e-dev/web`, `agent-device`) is optional, so a
project installs only the side it uses.
- `apps/testbed` (`@e2e-dev/testbed`, private) — dogfood project that
consumes the **built** packages like a real user would: the playground app
where every runner feature (sessions, routes, downloads, frames, uploads,
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ config and an example test. The
| [`@e2e-dev/kernel`](https://www.npmjs.com/package/@e2e-dev/kernel) | Kernel hosted browsers for the web engine. |
| [`@e2e-dev/eas`](https://www.npmjs.com/package/@e2e-dev/eas) | EAS Simulators hosted iOS simulators and Android emulators for the mobile engine. |
| [`@e2e-dev/decision`](https://e2e.tester.army/docs/decision-models) | Decision-model executors for bounded semantic actions and assertions. |
| [`@e2e-dev/testmu`](https://www.npmjs.com/package/@e2e-dev/testmu) | TestMu AI (formerly LambdaTest) hosted Android emulators, iOS simulators, and real devices for the mobile engine. |
| [`@e2e-dev/testmu`](https://www.npmjs.com/package/@e2e-dev/testmu) | TestMu AI (formerly LambdaTest) hosted Android emulators, iOS simulators, and real devices for the mobile engine, and hosted Chrome and Edge for the web engine (`@e2e-dev/testmu/web`). |

## Documentation

Expand Down
5 changes: 3 additions & 2 deletions docs/browser.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,9 @@ lacks: it asks for a browser when one is needed and gives it back when its
scope ends, on every exit path, so a session billed by the minute stops
when the run no longer uses it. The engine knows no vendor: for
[Kernel](/integrations/kernel), `kernel()` from `@e2e-dev/kernel` is the
provider, and for any other service a provider is a few dozen lines in your
project. Add `viewport: null` when
provider, for [TestMu AI](/integrations/testmu-browsers),
`testmuBrowsers()` from `@e2e-dev/testmu/web` is the provider, and for any other
service a provider is a few dozen lines in your project. Add `viewport: null` when
the service shows the window in a live view: the page then
[fills the window](/viewports#fill-the-window) instead of a fixed 1280 by
720 area of it.
Expand Down
3 changes: 2 additions & 1 deletion docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,8 @@
"integrations/index",
"integrations/kernel",
"integrations/eas",
"integrations/testmu"
"integrations/testmu",
"integrations/testmu-browsers"
]
},
{
Expand Down
13 changes: 13 additions & 0 deletions docs/examples/web/testmu-browsers.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testmuBrowsers } from '@e2e-dev/testmu/web';

export default {
targets: [
{
name: 'testmu-web',
engine: web({ browser: testmuBrowsers(), viewport: null }),
app: { url: 'https://staging.example.com' },
},
],
} satisfies E2EConfig;
3 changes: 3 additions & 0 deletions docs/integrations/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ integration decides where it comes from.
<Card title="TestMu AI" icon="mobile-screen" href="/integrations/testmu">
`@e2e-dev/testmu`: hosted Android emulators, iOS simulators, and real devices for mobile targets.
</Card>
<Card title="TestMu AI browsers" icon="cloud" href="/integrations/testmu-browsers">
`@e2e-dev/testmu`: hosted Chrome and Edge on Windows and macOS for web targets, one session per worker or per test.
</Card>
</CardGroup>

A service without an integration is one small provider in your project:
Expand Down
171 changes: 171 additions & 0 deletions docs/integrations/testmu-browsers.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
---
title: TestMu AI browsers
sidebarTitle: TestMu AI browsers
description: Run web targets in TestMu AI's hosted Chrome and Edge with the testmuBrowsers() browser provider from @e2e-dev/testmu/web.
---

`testmuBrowsers()` from `@e2e-dev/testmu/web` is a
[browser provider](/browser#hosted-browsers) for
[TestMu AI](https://www.lambdatest.com) (formerly LambdaTest). By default each worker slot gets its
own TestMu AI session for the run; `scope: 'attempt'` gives every test
attempt its own session instead. Tests and CI stay the same; the config
changes only in the target's `browser`.

## Setup

<CodeGroup>
```bash npm
npm install --save-dev @e2e-dev/testmu
```

```bash pnpm
pnpm add -D @e2e-dev/testmu
```

```bash bun
bun add -d @e2e-dev/testmu
```
</CodeGroup>

```ts title="e2e.config.ts"
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testmuBrowsers } from '@e2e-dev/testmu/web';

export default {
targets: [
{
name: 'testmu-web',
engine: web({ browser: testmuBrowsers(), viewport: null }),
app: { url: 'https://staging.example.com' },
},
],
} satisfies E2EConfig;
```

The browser provider lives at `@e2e-dev/testmu/web`, so a web-only project
needs neither `@e2e-dev/mobile` nor agent-device. The package root exports
the [device provider](/integrations/testmu), which uses both.

Set `LT_USERNAME` and `LT_ACCESS_KEY` in the environment `e2e run` starts in,
such as CI secrets. To get them, [sign up for TestMu AI](https://accounts.lambdatest.com/register)
or log in to your account, then copy your username and access key from
**Account Settings → Password & Security → Username and Access Key**
([accounts.lambdatest.com/security/username-accesskey](https://accounts.lambdatest.com/security/username-accesskey)).

Each worker gets its own session, so `workers` sets how
many run at once. Keep it within your TestMu AI plan's parallel sessions:
a session beyond it waits in TestMu AI's queue, and the web engine gives a
session 60 seconds to accept the CDP connection, so one that waits longer
fails its attempt. See [Limits](#limits).

## Options

| Option | Default | Meaning |
| ------ | ------- | ------- |
| `scope` | `'worker'` | `'worker'`: one session per worker slot for the run. `'attempt'`: a fresh session per test attempt, retries included. See [Scopes](#scopes). |
| `browserName` | `'Chrome'` | `'Chrome'` or `'MicrosoftEdge'`. The web engine attaches over CDP, so the browser must be Chromium-based; any other value fails the config. |
| `browserVersion` | `'latest'` | Any version TestMu AI offers for the platform, such as `'140'`. |
| `platform` | `'Windows 11'` | A TestMu AI desktop platform, such as `'Windows 10'` or `'macOS Sequoia'`. |
| `project` | `'e2e'` | Sent as `LT:Options.project`, as [`testmu()`](/integrations/testmu) sends its own. |
| `build` | the run id | The dashboard build the sessions are grouped under. The same default as `testmu()`'s, so a run's device and browser sessions share one build. |
| `sessionName` | `e2e-<run id>-<target>-<slot or attempt id>` | Every session's dashboard name. A given name gets `-<slot>` appended when the target has more than one worker slot, and `-<attempt id>` in `attempt` scope, so each session has its own. |
| `geoLocation` | none | Country the browser's IP geolocates to, as a code TestMu AI takes: `'US'`, `'FR'`. |
| `timezone` | none | The machine's time zone, as TestMu AI takes it: `'UTC+05:30'`. |
| `route` | `'/puppeteer'` | The hub route that serves the session's CDP endpoint. See [Routes](#routes). |
| `hub` | `'cdp.lambdatest.com'` | The hub host, without a scheme or path. |
| `capabilities` | none | Further `LT:Options` capabilities, such as `video`, `network`, `console`, `idleTimeout`, `resolution`, or `tunnel` and `tunnelName` for [TestMu AI Tunnel](https://www.lambdatest.com/support/docs/testing-locally-hosted-pages/). `idleTimeout` is 600 seconds when absent. `user`, `accessKey`, `build`, `name`, `project`, `platform`, `geoLocation`, `timezone`, `browserName`, `browserVersion`, and a nested `LT:Options` fail the config here: the provider sets them from the environment and the options above. |

```ts
testmuBrowsers({
scope: 'attempt',
platform: 'Windows 10',
build: `checkout ${process.env.GITHUB_SHA}`,
capabilities: { video: true, network: true, idleTimeout: 300 },
})
```

A misspelled option, a `route` other than the two below, or a `hub` with a
scheme or path fails with `INVALID_CONFIG` when the config loads.

## Scopes

`scope: 'worker'` keeps one session per worker slot for the whole run and
gives every attempt a new browser context in it, as a local run does: a test
never sees another test's cookies, storage, or permissions. TestMu AI shows
it as one session, named after the run, the target, and the slot, that covers every
test the worker ran.

`scope: 'attempt'` starts a session per attempt, so each test is its own
TestMu AI session, named after the run, the target, and the attempt. It costs a
session start per test and carries the limits of a
[per-attempt lease](/browser#hosted-browsers): no `headers`, `basicAuth`, or
`userAgent`, no `app.clearState()`, and no [sessions](/authentication). A
dropped connection cannot be resumed. The engine reconnects to the same CDP
URL, which starts a new TestMu AI session under the same name; the engine
sees a different browser, closes it, and fails the attempt with
`CDP recovery failed`. That second session shows in the dashboard and counts
against your plan while it lasts.

## Routes

A TestMu AI session starts when its CDP websocket opens, on one of the
hub's routes:

- `'/puppeteer'` (default) serves a raw CDP endpoint on every TestMu AI
machine. The dashboard labels these sessions Puppeteer.
- `'/playwright-cdp'` labels sessions as Playwright, but serves raw CDP only on
machines that run TestMu AI's newest backend, which is still rolling out. On
a machine without it the session fails to connect, so keep the default
unless you need the Playwright label.

## Limits

- **60-second connect.** The web engine waits 60 seconds for the browser to
accept the CDP connection, and `launchTimeout` does not change that. A
session still in TestMu AI's queue, or one that is slow to provision, fails
its attempt with `connectOverCDP: Timeout 60000ms exceeded`
(`LAUNCH_TIMEOUT`). Chrome on Windows typically connects in about 10 to 30
seconds; Edge sessions can take longer than the limit.
- **About 24 to 30 workers per target in `worker` scope.** The engine hands
every worker's CDP URL to the workers in one environment variable capped at
16 KB, and each URL carries its capabilities. Past about 30 workers with the
defaults, or 24 with several `capabilities`, the run fails at start with
`ENGINE_FAILURE: the browser leases take ... bytes`. `scope: 'attempt'` leases in each worker and
has no such ceiling.

## Recordings

TestMu AI records the session's screen itself when you pass
`capabilities: { video: true }`; watch it in the TestMu AI dashboard. The
provider does not hand that recording to e2e, so an attempt that records
[video](/reference/config#video) gets the web engine's screencast of the
page, as a local run does.

## Downloads

TestMu AI's browsers save downloads on their own machines, and the provider
has no way to read them back, so `browser.waitForDownload` fails with
`ENGINE_FAILURE` naming the provider. Assert on the download link or the
request instead.

## What the provider does

- Builds the session's CDP URL with the capabilities above and returns it as
the lease. It calls no TestMu AI API: opening the websocket starts the
session and the engine closing it ends the session, so `release` has
nothing to call.
- Names every session as `testmu()` does, `e2e-<run id>-<target>-<slot or
attempt id>` unless `sessionName` is given, in the `build`, sends `project`
as `LT:Options.project`, and
logs `TestMu AI session "<name>" in build "<build>"` to the reporter as
each lease is made.
- Reads `LT_USERNAME` and `LT_ACCESS_KEY` from the run's environment and puts
them only in the CDP URL: they never appear in a log line. A missing or
blank one fails the lease with an error naming it, such as
`LT_ACCESS_KEY is not set`; in `worker` scope that ends the run before any
test.
- Sets `idleTimeout` to 600 seconds unless `capabilities` sets one: the hub
otherwise ends a session after 300 seconds without browser traffic, which a
slow step between two browser calls can exceed. The same timeout ends a
session a dead worker held.
4 changes: 4 additions & 0 deletions docs/integrations/testmu.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,10 @@ package, not the upload name.
Set `LT_USERNAME` and `LT_ACCESS_KEY` to your TestMu AI username and access
key in the environment `e2e run` starts in, such as CI secrets. Without
either, the lease fails before anything starts, naming the missing one.
To get them, [sign up for TestMu AI](https://accounts.lambdatest.com/register)
or log in to your account, then copy your username and access key from
**Account Settings → Password & Security → Username and Access Key**
([accounts.lambdatest.com/security/username-accesskey](https://accounts.lambdatest.com/security/username-accesskey)).

agent-device's `testmu` runtime reads the credentials from the environment
of the daemon that drives the sessions. The provider starts that daemon from
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/environment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ What is sent and how to read it is on the [Telemetry](/telemetry) page.
| `KERNEL_API_KEY` | value | The API key `kernel()` from `@e2e-dev/kernel` creates browsers with; unset, the browser lease fails. See [Kernel](/integrations/kernel) |
| `EXPO_TOKEN` | value | The Expo access token `easSimulators()` from `@e2e-dev/eas` starts and stops simulator sessions with; unset, it uses the eas-cli login in `.expo/state.json` under `HOME` (`USERPROFILE` on Windows), and with neither the device lease fails. See [EAS Simulators](/integrations/eas) |
| `HOME`, `USERPROFILE` | value | The home `easSimulators()` finds the eas-cli login under when `EXPO_TOKEN` is unset: `USERPROFILE` on Windows, `HOME` elsewhere, the OS user's home when unset |
| `LT_USERNAME`, `LT_ACCESS_KEY` | value | The TestMu AI username and access key `testmu()` from `@e2e-dev/testmu` leases devices with; it sets them in the runner process's environment only while it calls the agent-device daemon it starts, which reads them from there, and with either unset the device lease fails. See [TestMu AI](/integrations/testmu) |
| `LT_USERNAME`, `LT_ACCESS_KEY` | value | The TestMu AI username and access key `testmu()` from `@e2e-dev/testmu` leases devices with; it sets them in the runner process's environment only while it calls the agent-device daemon it starts, which reads them from there, and with either unset the device lease fails. `testmuBrowsers()` puts them only in the CDP URL of each browser session, and with either unset the browser lease fails. Both come from Account Settings → Password & Security → Username and Access Key on your TestMu AI account. See [TestMu AI](/integrations/testmu) and [TestMu AI browsers](/integrations/testmu-browsers) |
| `E2E_AGENT_DEVICE_POOL_<TARGET>_<digest>` | value | Written by `@e2e-dev/mobile` in `prepare` and read by each worker to find its booted device. Not meant to be set by hand |

## What app processes inherit
Expand Down
3 changes: 3 additions & 0 deletions docs/security.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,9 @@ Built-in features may connect to:
- the GitHub API, only from the opt-in `@e2e-dev/github` reporter
- the Kernel API and its hosted browsers, only from
[`kernel()`](/integrations/kernel)
- TestMu AI's CDP hub and its hosted browsers, only from
[`testmuBrowsers()`](/integrations/testmu-browsers). The username and
access key travel only in the CDP URL, never in a log line
- the Expo API and the simulators' agent-device daemons, only from
[`easSimulators()`](/integrations/eas). It authenticates with `EXPO_TOKEN`,
else with the eas-cli login in `~/.expo/state.json`: on a CI runner, set
Expand Down
40 changes: 39 additions & 1 deletion packages/testmu/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,45 @@ whole session instead, found by the session's build and name and starting
at the session's start time. Runs that overlap and share a fixed `build`
and `sessionName` can link each other's videos; the defaults never do.

Full documentation lives at [e2e.tester.army/docs/integrations/testmu](https://e2e.tester.army/docs/integrations/testmu).
## Browsers

`web({ browser: testmuBrowsers() })` runs a web target in TestMu AI's hosted
Chrome or Edge on Windows and macOS, one session per worker slot, or one per
attempt with `scope: 'attempt'`. It is exported from `@e2e-dev/testmu/web`,
which needs only `@e2e-dev/web`; the package root exports the device
provider.

```ts title="e2e.config.ts"
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testmuBrowsers } from '@e2e-dev/testmu/web';

export default {
targets: [
{
name: 'testmu-web',
engine: web({ browser: testmuBrowsers({ platform: 'Windows 11' }), viewport: null }),
app: { url: 'https://staging.example.com' },
},
],
} satisfies E2EConfig;
```

A session starts when its CDP websocket opens and ends when the engine
closes it, so the provider only builds the CDP URL from the options and
`LT_USERNAME`/`LT_ACCESS_KEY`, and calls no API.

## Credentials

Both providers read `LT_USERNAME` and `LT_ACCESS_KEY` from the run's
environment. To get them, [sign up for TestMu AI](https://accounts.lambdatest.com/register)
or log in to your account, then copy your username and access key from
**Account Settings → Password & Security → Username and Access Key**
([accounts.lambdatest.com/security/username-accesskey](https://accounts.lambdatest.com/security/username-accesskey)).

Full documentation lives at [e2e.tester.army/docs/integrations/testmu](https://e2e.tester.army/docs/integrations/testmu)
(devices) and [e2e.tester.army/docs/integrations/testmu-browsers](https://e2e.tester.army/docs/integrations/testmu-browsers)
(browsers).

## License

Expand Down
Loading