Skip to content

feat(testmu): hosted Chrome and Edge for the web engine with testmuBrowsers() - #4

Merged
amankansal-lt merged 1 commit into
LambdaTest:mainfrom
SahilSawLT:feat/testmu-browsers
Oct 6, 2026
Merged

amankansal-lt merged 1 commit into
LambdaTest:mainfrom
SahilSawLT:feat/testmu-browsers

Conversation

@SahilSawLT

@SahilSawLT SahilSawLT commented Oct 5, 2026 •

Copy link
Copy Markdown

What and why

The upstream maintainer closed tester-army#765 (TestMu AI desktop browsers) and asked to consolidate the TestMu AI integration in tester-army#772, which covers devices only. This PR adds the browser side to @e2e-dev/testmu so tester-army#772 carries both.

  • testmuBrowsers(), exported from @e2e-dev/testmu/web, is a BrowserProvider for @e2e-dev/web: TestMu AI hosted Chrome and Edge on Windows and macOS, one session per worker slot, or one per attempt with scope: 'attempt'.
  • A session starts when its CDP websocket opens and ends when the engine closes it, so the provider only builds the CDP URL. There's no API call and no SDK. Credentials come from the package's existing testmuCredentials() (LT_USERNAME/LT_ACCESS_KEY) and never appear in a log line.
  • Follows testmu()'s conventions:
    • build (default the run id) and sessionName (default e2e-<run id>-<target>-<slot or attempt id>) name sessions as the device provider does, so a run's device and browser sessions land in one build. project (default e2e) is sent as LT:Options.project, as testmu() sends its own.
    • geoLocation and timezone are top-level options, as on testmu().
    • Optional strings must be non-empty, defaults apply only to undefined (so null fails), and unknown keys are refused. These are the same rules cubic asked for on feat(testmu): add @e2e-dev/testmu device provider tester-army/e2e#772.
  • Other options: browserName (Chrome/MicrosoftEdge), browserVersion, platform, route, hub, scope, capabilities (further LT:Options). idleTimeout defaults to 600 s. capabilities cannot set what the provider sets (user, accessKey, build, name, project, platform, geoLocation, timezone, browserName, browserVersion, or a nested LT:Options).
  • route: /puppeteer (default) serves raw CDP on every machine. /playwright-cdp labels sessions as Playwright but serves raw CDP only on machines with TestMu AI's newest backend, which is still rolling out, and fails to connect elsewhere.
  • The package root stays device-only (unchanged index.ts). The browser provider lives only at @e2e-dev/testmu/web, so a web-only project never loads agent-device or @e2e-dev/mobile, and the root's types never reference @e2e-dev/web.
  • Optional peers: @e2e-dev/mobile, @e2e-dev/web and agent-device are now marked optional. Device users already install agent-device explicitly (no published release satisfies the range yet), so nothing changes for them. Without this, a web-only npm install @e2e-dev/testmu would try to resolve the unpublished agent-device range.
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testmuBrowsers } from '@e2e-dev/testmu/web';

export default {
  targets: [
    {
      name: 'testmu-web',
      engine: web({ browser: testmuBrowsers({ platform: 'Windows 11', capabilities: { video: true } }), viewport: null }),
      app: { url: 'https://staging.example.com' },
    },
  ],
} satisfies E2EConfig;
sequenceDiagram
  participant W as web engine
  participant P as testmuBrowsers()
  participant H as cdp.lambdatest.com
  W->>P: acquire(request)
  P-->>W: { id, cdpEndpoint } (caps + LT_* from request.env)
  W->>H: connectOverCDP(cdpEndpoint)
  Note over H: socket open = session starts
  W->>H: close
  Note over H: socket closed = session ends
  W->>P: release() (no-op)
Loading
  • Docs: new docs/integrations/testmu-browsers.mdx (options, scopes, routes, recordings, downloads, Limits), with its config sample checked as docs/examples/web/testmu-browsers.config.ts (registered in scripts/check-docs-examples.ts, like the mobile example), plus the nav, integrations card, environment table, security page, browser.mdx, package README, AGENTS.md and the skills/e2e setup row. Both TestMu AI pages now say where to get the credentials (Account Settings → Password & Security → Username and Access Key).

Verified

Ran it locally: yes

  • pnpm check passes (lint, dead code, typecheck, error codes, peer ranges, install scripts, docs). pnpm --filter @e2e-dev/testmu run test: 108/108, of which 35 are new for the browser provider. They cover the URL and capabilities, credentials from request.env only, every INVALID_CONFIG case (including null values and the reserved capabilities keys), the 600 s idleTimeout default, testmu()-style project/build/session naming, log lines without the key, the /web export with a device-only root, and 24 worker leases fitting the engine's 16 KB hand-off.
  • Packed @e2e-dev/testmu (pnpm pack) and installed it browser-only into a scratch project on e2e 0.15.1: npm pulled neither agent-device nor @e2e-dev/mobile, and the import came from @e2e-dev/testmu/web.
  • Real runs on TestMu AI production:
Route Platform Result
/puppeteer (default) Windows 11 + Chrome ✅ 1/1, recorded with build <run id> and session e2e-<run id>-<target>-1
/puppeteer, scope: 'attempt' Windows 11 + Chrome ✅ 2/2, one session per test
/playwright-cdp macOS Sequoia + Chrome, pinned to a machine with the backend fix ✅ 2/2, session labelled Playwright
timezone: 'UTC+05:30' Windows 11 + Chrome ✅ browser reports Asia/Calcutta (+330 min)
  • No report, log or artifact file from any run contains the access key.
  • project is sent in LT:Options the way testmu() sends it. The desktop sessions API doesn't return a project field, so the docs only say it is sent, not that the dashboard groups by it.
  • Platform issues seen in the sanity run. Both reproduce with plain Puppeteer/Playwright, without this provider, and are being reported to TestMu AI:
    • Edge over CDP: on Windows 11 the hub answers 500 after about 4 minutes (Chrome connects in about 11 s), so Edge sessions hit e2e's 60 s connect limit.
    • Geo exits: geoLocation: 'DE' exits in Portugal and 'FR' in Spain. The option is applied (the default exit is Mumbai), but those exits sit in neighbouring countries.
  • Not included: linking TestMu AI's recording through record(), as testmu() does for devices. The web engine supports it, but it needs the desktop sessions API, so it's left for a follow-up.
  • In worker scope about 24–30 workers per target fit the engine's 16 KB lease hand-off (documented under Limits). scope: 'attempt' has no such ceiling.
  • A session that waits longer than 60 s for a machine (queued, or pinned to a busy one) fails with LAUNCH_TIMEOUT. The web engine's CDP connect is fixed at 60 s, as the Limits section documents.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KoNxaN1YAAP9PwmHtMm1WX

@SahilSawLT
SahilSawLT force-pushed the feat/testmu-browsers branch 2 times, most recently from b5d120e to 8ca6a64 Compare October 5, 2026 15:10

@amankansal-lt amankansal-lt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Sahil. This is in good shape, and it's what the upstream maintainer asked for when closing tester-army#765. Two wording fixes are needed before it goes into tester-army#772. Neither needs a code change.

  1. project grouping isn't verified. docs/integrations/testmu-browsers.mdx (options table and FAQ) and the JSDoc in packages/testmu/src/browsers.ts say project is the "Dashboard project the sessions are grouped under". Your PR description says the desktop sessions API returns no project field, so that couldn't be confirmed. Please change it to what we know, e.g. "sent as LT:Options.project", or confirm the grouping on the dashboard and say so. A tester-army reviewer would quote this line back at us on tester-army#772.
  2. /playwright-cdp is still rolling out. The docs say to use it "once TestMu AI serves raw CDP on it for your account", but a user can't find that out. Please say plainly that it's rolling out and may fail on machines without the new backend, keeping /puppeteer as the default. Or drop the route option until the rollout finishes, which also shrinks tester-army#772's API surface. Either is fine.

Optional, take or leave:

  • capabilities containing browserName, browserVersion or a nested LT:Options is quietly nested inside LT:Options rather than refused. Refusing them like the reserved keys would match the rest of the validation.
  • With skipLibCheck: false, a device-only user importing the package root gets a type error, because dist/browsers.d.ts imports types from the now-optional @e2e-dev/web. The repo uses skipLibCheck: true, so this is minor.

What I checked:

  • Key exposure. The only log line holds just the name and build. Playwright 1.63 prints only the path of a failing wss:// URL, never the query. The http:// case that would leak is unreachable, because hub can't carry a scheme.
  • Imports. dist/web.js imports no agent-device and no @e2e-dev/mobile.
  • Contract and peers. The BrowserProvider contract matches, and @e2e-dev/web >=0.11.0 already has 'attempt' scope.
  • Gates. pnpm check exits 0, pnpm --filter @e2e-dev/testmu run test passes 106/106, and test:scripts passes.

After you update, I'll merge it. Separately from this PR, tester-army#772 needs another sync with tester-army main, because their tester-army#882 touched the same AGENTS.md list, and I'll do that right after.

…owsers()

@e2e-dev/testmu/web exports testmuBrowsers(), a BrowserProvider for TestMu AI
(formerly LambdaTest) hosted Chrome and Edge on Windows and macOS. A session
starts when its CDP websocket opens and ends when the engine closes it, so the
provider only builds the CDP URL from the options and LT_USERNAME /
LT_ACCESS_KEY (via the package's testmuCredentials()); it calls no API.

- One session per worker slot, or per attempt with scope: 'attempt'.
- route /puppeteer (default, raw CDP on every machine) or /playwright-cdp
  (Playwright label; raw CDP only where TestMu AI's newest backend has rolled
  out). hub cdp.lambdatest.com.
- Sessions are named as testmu()'s: build (default the run id, so a run's
  device and browser sessions share one build) and sessionName (default
  e2e-<run id>-<target>-<slot or attempt>); project (default e2e) is sent as
  LT:Options.project. geoLocation and timezone are options, as on testmu().
- idleTimeout defaults to 600 s. Unknown options, out-of-range values, empty
  or null strings, and capabilities that set what the provider sets
  (including browserName, browserVersion and a nested LT:Options) fail with
  INVALID_CONFIG at config load.
- The package root stays device-only; the browser provider lives at
  @e2e-dev/testmu/web, and @e2e-dev/mobile, @e2e-dev/web and agent-device
  are optional peers, so neither side's imports or types reach the other.
- Docs: integrations/testmu-browsers page with a checked example
  (docs/examples/web/testmu-browsers.config.ts), nav, card, env, security,
  browser.mdx, README, AGENTS, skill row; credentials sourcing on both pages.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KoNxaN1YAAP9PwmHtMm1WX
@SahilSawLT
SahilSawLT force-pushed the feat/testmu-browsers branch from 8ca6a64 to fe91e5f Compare October 6, 2026 08:28
@SahilSawLT

Copy link
Copy Markdown
Author

Thanks, Aman. All four points are in fe91e5fe (still one commit):

  1. project: now described as "sent as LT:Options.project, as testmu() sends its own" in the JSDoc, the docs options table, "What the provider does" and the changeset. There's no grouping claim anywhere.
  2. /playwright-cdp: Routes and the TestmuBrowsersRoute JSDoc now say it serves raw CDP only on machines with TestMu AI's newest backend, which is still rolling out, and fails to connect elsewhere. /puppeteer stays the default.
  3. capabilities: now refuses browserName, browserVersion and a nested LT:Options like the other reserved keys, with tests.
  4. Types for device-only users: the package root is device-only again (your original index.ts), and testmuBrowsers() lives only at @e2e-dev/testmu/web. dist/index.d.ts no longer references @e2e-dev/web, and a web-only project never loads agent-device.

pnpm check exits 0, and @e2e-dev/testmu tests pass 108/108. I re-ran a browser-only fresh install (published e2e 0.17.0 + @e2e-dev/web 0.12.0) on prod: 2/2.

Two platform issues from my sanity run, not provider bugs, since both reproduce with plain Puppeteer/Playwright: Edge over CDP gets a 500 from the hub on prod (Windows 11), and geoLocation DE/FR exit in Portugal/Spain. I noted both in the PR description and will report them to TestMu AI separately.

@amankansal-lt amankansal-lt left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at fe91e5fe1:

  • No browser-provider text claims project groups sessions any more.
  • /playwright-cdp is documented as rolling out, and /puppeteer stays the default.
  • capabilities refuses browserName, browserVersion and nested LT:Options.
  • The package root is device-only again.
  • pnpm check exits 0, and @e2e-dev/testmu passes 108/108.

The Edge-over-CDP hub 500 is a platform issue (it reproduces without the provider), and it's covered in Limits and the description. Thanks.

@amankansal-lt
amankansal-lt merged commit 7bfdd6e into LambdaTest:main Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants