Skip to content

feat(testmuai): @e2e-dev/testmuai, TestMu AI hosted browsers for the web engine - #765

Closed
SahilSawLT wants to merge 4 commits into
tester-army:mainfrom
SahilSawLT:feat/testmuai-provider
Closed

SahilSawLT wants to merge 4 commits into
tester-army:mainfrom
SahilSawLT:feat/testmuai-provider

Conversation

@SahilSawLT

@SahilSawLT SahilSawLT commented Oct 2, 2026 •

Copy link
Copy Markdown

What and why

  • New package @e2e-dev/testmuai: web({ browser: testmuai() }) runs a web target in TestMu AI (formerly LambdaTest) hosted Chrome or Edge on Windows and macOS. One session per worker slot, or one per attempt with scope: 'attempt', like kernel().
  • A TestMu AI session starts when its CDP websocket opens and ends when it closes, so the provider only builds the URL (wss://cdp.lambdatest.com/<route>?capabilities=…). There's no SDK and no API call, release is a no-op, and @e2e-dev/web is the only peer, like @e2e-dev/eas.
  • LT_USERNAME/LT_ACCESS_KEY are read from request.env and travel only in the CDP URL. The reporter line is TestMu AI session "<name>" in build "<build>", never the URL.
  • Options: scope, browserName, browserVersion, platform, build, route, hub, and capabilities for further LT:Options (video, network, tunnel, idle timeout). Unknown keys, a bad route/hub/browserName, or capabilities that would override what the provider sets fail with INVALID_CONFIG at config load. idleTimeout defaults to 600 s.
  • route defaults to /puppeteer, which serves raw CDP on every account today. /playwright-cdp labels sessions as Playwright on the TestMu AI dashboard; the backend change that makes it serve raw CDP is rolling out, and the default can flip once it reaches production.
  • No record, downloads or sweep: TestMu AI records its own video, downloads land on its machines, and a session a dead worker held ends at TestMu AI's idle timeout. The integration page documents all three.
  • The hub host, LT:Options and the LT_* variable names keep the old brand because they are the service's live contract.
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testmuai } from '@e2e-dev/testmuai';

export default {
  targets: [
    {
      name: 'testmuai',
      engine: web({ browser: testmuai({ platform: 'Windows 11', capabilities: { video: true } }), viewport: null }),
      app: { url: 'https://staging.example.com' },
    },
  ],
} satisfies E2EConfig;
sequenceDiagram
  participant W as web engine
  participant P as testmuai()
  participant H as cdp.lambdatest.com
  W->>P: acquire(request)
  P-->>W: { id, cdpEndpoint } (caps + LT_* from request.env)
  W->>H: connectOverCDP(cdpEndpoint)
  Note over H: socket open = session starts
  W->>H: close
  Note over H: socket closed = session ends
  W->>P: release() (no-op)
Loading

Verified

Ran it locally: yes

  • Packed the package (pnpm pack) and installed the tarball into a scratch project on e2e 0.15.1, so exports/files were exercised: the tarball holds only dist/, LICENSE, NOTICE, README.md and package.json.
  • Real runs on TestMu AI production, /puppeteer, Windows 11 + Chrome, video on, 3 tests on the-internet.herokuapp.com (taps, assertions, evaluate):
Scope Result Sessions
worker 2/3 passed 1 session for the worker
attempt 2/3 passed 1 session per test
  • The failing test in both is add and remove an element: the-internet.herokuapp.com stalls its jQuery for about 30 s on a fresh browser's first load, so the tap lands before the page's script binds. Session, connect and the other assertions are fine; the same test passes when the page is warm.
  • Missing LT_ACCESS_KEY fails before any test with browser provider "testmuai" could not lease a browser: LT_USERNAME and LT_ACCESS_KEY must be set, and no session is created.
  • No report, log or artifact file from any run contains the access key.
  • Known limits, documented under Limits on the integration page: in worker scope about 25 workers per target fit the engine's 16 KB lease hand-off (packages/web/src/provider.ts, MAX_HANDOFF_ENV_BYTES; a unit test pins it), and hosts that take more than 60 s to provision (Edge on macOS, for one) fail with connectOverCDP: Timeout 60000ms exceeded. BROWSER_LAUNCH_TIMEOUT_MS is fixed at 60 s in packages/web/src/surface.ts and launchTimeout doesn't reach the CDP connect. Happy to follow up with a separate PR that lets it, if you want that.

Checklist

  • Changeset added (pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only.
  • Breaking change: n/a, new package.
  • Docs updated in the same PR: docs/integrations/testmuai.mdx, nav, integrations card, environment variables, security (outbound connections), README packages table, AGENTS.md, CONTRIBUTING.md, and the skills/e2e setup row.
  • Contract change: n/a, no change to e2e or @e2e-dev/web contracts.
  • Engine contract change (e2e/engine): n/a.

Important

Before merge (maintainers): @e2e-dev/testmuai is a new public package. Per CONTRIBUTING.md, publish its first version by hand and connect it to release.yml as a trusted publisher; otherwise the next release's publish step fails. If you'd rather land the code first, I can drop the changeset from this PR and add it in a follow-up once the package exists on npm.

The changeset takes the package from 0.0.0 to a minor, like EAS. The integrations card uses the generic cloud icon until there's a brand SVG to add.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KoNxaN1YAAP9PwmHtMm1WX

…web engine

testmuai() is a BrowserProvider for @e2e-dev/web that leases TestMu AI
(formerly LambdaTest) hosted Chrome and Edge on Windows and macOS. A
session starts when its CDP websocket opens, so the provider only builds
the CDP URL from the target's options and LT_USERNAME/LT_ACCESS_KEY in
the run's environment; it calls no API and release is a no-op. One
session per worker slot by default, or per attempt with scope: 'attempt'.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KoNxaN1YAAP9PwmHtMm1WX

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 23 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/testmuai/src/provider.ts
- testmuai() rejects unknown keys, a route other than the two routes, a hub
  with a scheme or path, a non-Chromium browserName, and capabilities that
  would override user, accessKey, build, name or platform, with
  INVALID_CONFIG at config load instead of a failed attempt.
- idleTimeout defaults to 600 s; the hub otherwise ends a session after
  300 s without browser traffic.
- Lease ids drop the run id: worker-scope leases share the engine's 16 KB
  hand-off. Docs state the ~25-worker ceiling, the 60 s connect cap, and
  what a dropped attempt-scope connection does; browser.mdx lists the
  provider; the lockfile importer is regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KoNxaN1YAAP9PwmHtMm1WX

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Tip: cubic used a learning from your PR history. Let your coding agent read cubic learnings directly with the cubic MCP.

Re-trigger cubic

Comment thread packages/testmuai/src/provider.ts
Comment thread packages/testmuai/src/provider.ts
SahilSawLT and others added 2 commits October 2, 2026 18:45
…CONFIG

A JavaScript config can pass null, an array, or a primitive where the types
say object: testmuai(null) and capabilities: null threw a raw TypeError, and
arrays were accepted. Both now fail with INVALID_CONFIG.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KoNxaN1YAAP9PwmHtMm1WX
Sign up or log in to TestMu AI and copy the username and access key from
Account Settings > Password & Security. Added to the integration page, the
package README, and the environment reference.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KoNxaN1YAAP9PwmHtMm1WX
@okwasniewski

Copy link
Copy Markdown
Member

Thanks for the PR! This will be superseded by #772 from the TestMu AI team, so we'll consolidate the TestMu AI integration there. Closing for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants