feat(provider-webdriver): add TestMu AI device cloud provider - #3117
Closed
amankansal-lt wants to merge 6 commits into
Closed
amankansal-lt wants to merge 6 commits into
amankansal-lt wants to merge 6 commits into
Conversation
BrowserStack's app upload, install adapter, --provider-app resolution, session-details URL artifacts, and orientation check were written for one vendor. A second hosted Appium hub needs the same mechanics with a different form field, reference scheme, and response shape, so move them into shared helpers and have BrowserStack use them: - webdriver-utils.ts: postHubAppUpload, createHubUploadApp, resolveHubAppReference, appFileUploadForm, asRecord, readProviderJsonBody, and requireProviderDeviceOrientation. - artifact-results.ts: urlArtifactFromDetails. - browserstack.ts: resolveBrowserStackAppReference, moved out of provider-definitions.ts. Two small BrowserStack behaviour changes come with the shared code: - An upload response that is not JSON (a gateway error page, an empty body) now fails with a typed COMMAND_FAILED that carries the HTTP status, instead of a raw JSON SyntaxError. - The http(s) scheme of a --provider-app URL is matched case-insensitively, so HTTPS://... is passed through to the hub rather than being treated as a local path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The BrowserStack session-details lookup behind `artifacts` had no deadline, so a stalled API call could hang the command indefinitely. A transport failure or a body that was not JSON surfaced as an untyped fetch or SyntaxError, and a JSON array passed the object check and was read as session details. Add fetchProviderSessionDetails to webdriver-utils.ts and use it for BrowserStack. It sends basic auth with a 15 second deadline and reports every failure as COMMAND_FAILED: a timeout or network error with a retry hint and the original error as its cause, and a non-2xx answer or a body that is not a JSON object with the HTTP status and the parsed response. Connection verification gets the same treatment through fetchProviderVerificationJson, which BrowserStack now uses in place of its private fetch. Behaviour is unchanged: 401/403 is UNAUTHORIZED with a credential hint, any other HTTP failure points at the provider's service status, and a transport failure points at network access. A new test pins the two non-credential hints. sameOsVersion moves alongside it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add `testmu`, a hosted WebDriver provider for TestMu AI (formerly LambdaTest) virtual devices: Android emulators and iOS simulators behind the TestMu AI Appium hub. It follows the BrowserStack shape: `connect testmu` verifies and saves a local profile, and `open` creates the hosted session. The service hostnames still carry the lambdatest.com domain. connect - Reads LT_USERNAME and LT_ACCESS_KEY, the variables TestMu AI SDKs use. - Checks the device and OS version against the public virtual-device catalog. The match is exact because the hub rejects `18` for a device listed as `18.0`; the error lists the versions the device offers. - Uses the authenticated app listing as the credential check, and looks an lt:// id up in the list for the session's runtime (`emulator` or `simulator`). An id that is not listed is reported as configured, since TestMu AI validates it at session creation. - Verifies against TESTMU_API_ENDPOINT when it is set, as the runtime does, and composes catalog and listing URLs with URL so a base or override that carries a query keeps it. - Never creates a session. Sessions - Standard Appium keys stay `appium:`-prefixed; everything vendor- specific goes in `lt:options`, merged per key with any configured `lt:options`. `isRealMobile: false` and `w3c: true` are applied last, so configuration cannot move the session to another device pool or off the W3C dialect agent-device speaks. - `appiumVersion` is sent only when --provider-appium-version pins one; otherwise TestMu AI starts its default server for the device. - Orientation, geo-location, timezone, Appium version, language, and locale map onto `lt:options` through a table. The BrowserStack-only network-profile, custom-network, and no-resign flags are refused by flag name at connect and at session preparation, instead of being silently dropped. Apps - --provider-app takes an lt:// id, an http(s) URL, or a local path. The hub only accepts lt:// references, so a URL is handed to the upload API to fetch (`storage=url`) and a local file is uploaded. - An unzipped iOS `.app` directory is rejected before any request, with a hint to zip it. - Only a well-formed lt:// reference or app id in the upload response counts as success. Artifacts - Read from the session-details API through the shared bounded lookup. A 404 reads as pending until TestMu AI publishes the details. `console_logs_url` is the device log on virtual devices. - Session video, Appium, network, and command logs, screenshots, and the dashboard link are returned once at least one artifact URL exists. TESTMU_WEBDRIVER_ENDPOINT, TESTMU_APP_UPLOAD_ENDPOINT, and TESTMU_API_ENDPOINT redirect the endpoints. The session, upload, verification, and device-feature modules are loaded with dynamic import, so the package entry's eager module graph does not grow. The CLI reaches the device-feature checks through a new `./testmu-device-features` subpath export. A .fallowrc entry covers the exports that are read only through the dynamic import. `connect testmu` is listed in the connect usage, the remote help topic, and the artifacts provider description. Co-authored-by: gautam-jain-dev <gautamj@lambdatest.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TestMu AI serves real devices and virtual devices from the same Appium
hub; `lt:options.isRealMobile` selects the pool. Add
`--provider-device-type real|virtual` to choose it. The default is
`virtual`, so existing `testmu` profiles and sessions keep their current
behaviour.
The value is a cloud provider profile field like the others. It is
defined in contracts as PROVIDER_DEVICE_TYPES and carried through the
lease_allocate projection, request overrides, the remote-config schema,
`connect`, the CLI request flags, and the session doctor's provider
keys. It reaches session preparation from a `connect testmu` profile and
from `client.leases.allocate({ providerDeviceType })`.
For `real`, TestMu AI sessions:
- Set `isRealMobile: true`. It is applied after any configured
`lt:options`, so configuration cannot switch pools.
- Upload through the real-device upload API. It has its own override,
TESTMU_REAL_DEVICE_APP_UPLOAD_ENDPOINT; TESTMU_APP_UPLOAD_ENDPOINT
keeps redirecting virtual-device uploads only. An `.app` directory is
refused with a hint to pass a signed .ipa.
- Verify against the real-device catalog
(`capability/generator?isVirtualDevice=false`). It lists devices
directly under the platform key rather than under `app.devices`. The
OS version must still match exactly, and real iOS devices are listed
by major version, such as `18`. A catalog response without the
expected pool shape fails with a typed error.
- Look an lt:// id up in the real-device app list (`type=android` or
`type=ios`), the same way virtual uploads are looked up under
`emulator` or `simulator`.
BrowserStack, AWS Device Farm, and Limrun refuse the flag at connect.
BrowserStack and AWS Device Farm also refuse it at session preparation,
which the typed client and hand-written profiles reach without
`connect`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Install from a remote source materializes an iOS build by extracting the `.app` bundle from a zipped simulator build or an .ipa. The WebDriver deployment runtime handed that extracted `.app` directory to the provider's uploader. Hosted upload APIs take a file, not a directory, so the upload could not succeed. When the materialized artifact is an iOS `.app` extracted from a `.zip` or `.ipa` archive, upload the archive it came from. Every other case uploads the installable path as before: no archive, an archive of another type, or an Android build. A provider without an uploader still installs the extracted bundle path. The bundle id and launch target hints are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a TestMu AI guide next to the BrowserStack and AWS Device Farm ones and link it from the sidebar. It covers credentials and `connect`, the exact device and OS version match, app references and uploads, device features, real devices with `--provider-device-type real`, the CLI and Node.js client workflows, artifacts, and endpoint overrides. List TestMu AI among the device clouds in the README, the device clouds overview, the client API page, and the command reference. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds TestMu AI (formerly LambdaTest) as a hosted device cloud alongside BrowserStack and AWS Device Farm. agent-device can drive TestMu AI Android emulators, iOS simulators and real devices through its existing cloud WebDriver runtime:
From Node,
client.leases.allocate({ leaseProvider: 'testmu', ... })gives the same session to frameworks built on agent-device.Commits:
--provider-appresolution and URL artifacts, extracted from the BrowserStack code so a second hub reuses them. Two small BrowserStack changes: typed errors for non-JSON upload bodies, and a case-insensitive URL scheme.COMMAND_FAILEDerrors, and an object check for every provider.testmuprovider for emulators and simulators:connectchecks credentials, device and exact OS version against the public catalogue, and thelt://app against the right app list.lt:options, withw3cand the device pool enforced.lt:options; unsupported flags are refused atconnect.--provider-device-type real|virtual, defaultvirtual, carried through contracts, lease scope, remote config andleases.allocate. Real devices use their own upload API and catalogue. Other providers refuse the flag.installfrom a remote source uploads the original.zip/.ipainstead of the extracted.appdirectory, which hosted upload APIs cannot accept.Endpoints can be overridden with
TESTMU_WEBDRIVER_ENDPOINT,TESTMU_APP_UPLOAD_ENDPOINT,TESTMU_REAL_DEVICE_APP_UPLOAD_ENDPOINTandTESTMU_API_ENDPOINT.Validation
Every commit passes
format:check,check:quick, provider-webdriver tests,test:integration:provider,test:unit,check:layeringandcheck:fallowon its own. On the head:test:unit: 12082 passedtest:integration:provider: 228 passedbuildandcheck:package: passLive against TestMu AI production. The local
.apkand local zipped.appruns used this exact code; the other three used earlier revisions of the same work. Each run coveredconnect→open→snapshot -i→screenshot→close→artifacts --json, and every run returned its artifacts ready (video, Appium/device/network/command logs, screenshots, dashboard link):open.apkupload.apkby URL.app.ipa.apkby URLThe hub rejects
platformVersion: "18"for a simulator the catalogue lists as18.0, which is whyconnectmatches versions exactly.Known limits are those of the cloud WebDriver runtime:
settings,alert,record,logsandportReverseare unsupported. For local services, use TestMu AI Tunnel.🤖 Generated with Claude Code