Repository navigation
feat(testmu): add @e2e-dev/testmu device provider - #771
Closed
amankansal-lt wants to merge 4 commits into
Closed
amankansal-lt wants to merge 4 commits into
amankansal-lt wants to merge 4 commits into
Conversation
Add @e2e-dev/testmu, a DeviceProvider for the mobile engine that runs
targets on TestMu AI (formerly LambdaTest) Android emulators, iOS
simulators, and real devices through agent-device's testmu cloud
provider:
mobile({ device: testmu({ device, osVersion, app }) })
Each worker slot allocates an agent-device lease from a daemon the
provider starts for the run under stateDir (default .e2e/testmu). The
lease's client configuration carries the lease scope and the device
selectors, so the worker's first command starts the TestMu AI session
and installs the app; releasing the lease ends it.
- Credentials come from LT_USERNAME and LT_ACCESS_KEY in the run's
environment; a missing one fails the lease before anything starts.
agent-device's client spawns its daemon with process.env and takes no
environment of its own, so the provider copies the run's values there
before the first allocation.
- A lease granted after an interrupt, or one that cannot be handed to
the run, is released before acquire throws. Release is idempotent and
works from the lease alone, after a round trip through JSON.
- A relative local app path resolves against the project root, not the
daemon's working directory. The target's app.appPath is refused, since
TestMu AI installs the app itself.
- Unknown options, missing device/osVersion/app, and a deviceType other
than virtual or real fail with INVALID_CONFIG.
agent-device is a peer (>0.21.18 <1): the provider and the engine's
workers must share one agent-device, and no published release includes
the testmu provider yet, so the range excludes only the releases known
to lack it. No record hook: TestMu AI records whole sessions, not
attempts, and its video is only available once the session ends.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a TestMu AI (formerly LambdaTest) integration page for the testmu() device provider from @e2e-dev/testmu, in the shape of the EAS Simulators page: setup, authentication, installing the app, real devices, an options table, sessions, what the provider does, and limits. The page covers the requirement for an agent-device that includes the testmu provider and the override that makes @e2e-dev/mobile use the same copy, the catalog-exact device names and OS versions (18.0 for a virtual iOS device, 18 for a real one, 14 on Android), the app formats per device, and the limits of agent-device's WebDriver runtime. The config and test examples are shown verbatim and typecheck against the built package. The integrations overview, the hosted devices section of the mobile page, the environment reference (LT_USERNAME, LT_ACCESS_KEY), and the agent skill's setup reference name the package. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agent-device ends a lease after 60 seconds without activity by default, and a command still running does not count as activity. The worker's first command uploads the app, allocates the hosted session, and boots it; on an iOS simulator that takes about 70 seconds, so the lease lapsed mid-start and the engine's prepare failed with "boot failed: Lease is not active" (LEASE_NOT_FOUND). Each lease now asks for agent-device's longest inactivity window (10 minutes), and the provider heartbeats every lease it holds every 2 minutes from the runner process until the lease is released, including the release acquire does itself when it cannot hand a lease to the run. The timer is unref'd so it never keeps the runner alive. A failed heartbeat is logged once per lease and the next one is still tried; it never fails the run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agent-device's native iOS runner reports element types as `Button`,
`StaticText`, `NavigationBar`. Its WebDriver runtime, which hosted Appium
clouds (TestMu AI, BrowserStack, AWS Device Farm) go through, reports the
XCUITest class names instead: `XCUIElementTypeButton`,
`XCUIElementTypeStaticText`. normalizeKind kebab-cased those to
`xcuielement-type-button`, which maps to no role, so on a real iPhone
`getByRole('button', 'CPU Load')` matched nothing and observe listed the
node by its raw class name.
normalizeKind now drops a leading `XCUIElementType` before kebab-casing,
so both spellings land on one vocabulary: buttons, tab bar items as tabs,
the navigation bar as the screen title. Only that exact prefix followed by
a type name is stripped; Android class names are untouched, and the golden
device trees project unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
amankansal-lt
requested review from
okwasniewski and
szymonrybczak
as code owners
October 2, 2026 16:25
Author
|
Superseded by #772, opened from the LambdaTest fork with the same commits. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Adds
@e2e-dev/testmu, aDeviceProviderthat runs mobile targets on TestMu AI (formerly LambdaTest) Android emulators, iOS simulators and real devices, through agent-device'stestmucloud provider (feat(provider-webdriver): add TestMu AI device cloud provider callstack/agent-device#3117).Leases: each worker slot allocates an agent-device lease. The worker's first command starts the session and installs the app; release ends it. Release is idempotent and also runs when a lease arrives after an interrupt.
Credentials:
LT_USERNAMEandLT_ACCESS_KEYcome from the run's environment. If one is missing, the lease fails before anything starts.Options:
deviceType: 'real'selects real devices.project,build,sessionNameandstateDirare optional. Unknown options fail at config load.Lease keep-alive: agent-device leases expire after 60 s idle, and an iOS simulator's first command (upload, allocation, boot) takes about 70 s. The provider allocates with the daemon's maximum window (10 min) and heartbeats every 2 minutes until release.
fix(mobile):XCUIElementType-prefixed iOS element types. agent-device's WebDriver runtime, used by every hosted Appium cloud including BrowserStack and AWS Device Farm, reportsXCUIElementTypeButtonwhere the native runner reportsButton. Before this,getByRole('button', …)matched nothing on those iOS devices. Patch changeset for@e2e-dev/mobile.Requirement: an agent-device release with the
testmuprovider (feat(provider-webdriver): add TestMu AI device cloud provider callstack/agent-device#3117). agent-device is a peer (>0.21.18 <1) so the provider and the engine share one copy. Until@e2e-dev/mobilepins such a release, the docs page shows a package-manager override, with the version left as a placeholder to fill on release.Verified
Ran it locally: yes
@e2e-dev/testmuunit tests: 28 passed;@e2e-dev/mobileunit tests: 237 passedpnpm lint,pnpm -r run typecheck,pnpm docs:check,check-dist,check:peer-ranges: passgetByRole/getByText; every device was released afterwards..apkby URL.app.apkby URL.ipaChecklist
pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only.!, the changeset body names what breaks and what replaces it, and the deprecation warning landed a window earlier (stability policy in CONTRIBUTING.md). n/a, not breaking.docs/**/*.mdxpage for the behavior, andskills/e2e/if the skill describes it..d.tsreviewed,tests/types/sdk-types.tsupdated; wire change edits the schema and both fixtures. n/ae2e/engine): changesets fore2e,@e2e-dev/web, and@e2e-dev/mobile. n/a🤖 Generated with Claude Code