Skip to content

fix(browserstack-service): don't mark external-grid runs as Automate#8

Open
AakashHotchandani wants to merge 2 commits into
mainfrom
fix/browserstack-service-external-grid-origin
Open

fix(browserstack-service): don't mark external-grid runs as Automate#8
AakashHotchandani wants to merge 2 commits into
mainfrom
fix/browserstack-service-external-grid-origin

Conversation

@AakashHotchandani

Copy link
Copy Markdown
Owner

Summary

@wdio/browserstack-service classifies every non-App-Automate run as Automate:

// config.ts
this.appAutomate = !isUndefined(options.app) || detectAppAutomate(capabilities)
this.automate    = !this.appAutomate

This never checks whether the session actually runs on BrowserStack. When a user runs on an external (non-BrowserStack) grid and uses the service only for Test Observability — the documented "tests running locally or elsewhere" flow, with user/key placed inside testObservabilityOptionsautomate is still set to true. That flows into getProductMap (buildProductMap), and the build is attributed with origin Automate, even though browserstackAutomation is correctly false in the very same build-start event.

Reproducible build-start payload from such a run (web Chrome on an external CI grid):

"browserstackAutomation": false,
"product_map": { "observability": true, "automate": true, "app_automate": false }

This is a different axis from webdriverio#15267 (which fixed App-Automate ↔ Automate detection from capabilities). That change left this.automate = !this.appAutomate untouched, so external-grid web runs are still mislabeled.

Fix

  • config.ts: the BrowserStackConfig constructor / getInstance take an optional isBrowserStackInfra (default true, preserving behaviour for the existing getInstance() retrieval callers). appAutomate/automate are gated on it:
    this.appAutomate = isBrowserStackInfra && (!isUndefined(options.app) || detectAppAutomate(capabilities))
    this.automate    = isBrowserStackInfra && !this.appAutomate
  • launcher.ts: passes isBrowserstackInfra(_config) — the same helper that already drives the browserstackAutomation field via shouldAddServiceVersion — so the product map stays consistent with it.

Result for an external-grid Test-Observability run: { observability: true, automate: false, app_automate: false } → no longer attributed to Automate. On-BrowserStack Automate / App-Automate / TurboScale runs keep isBrowserStackInfra = true and are unchanged. The credentials-in-testObservabilityOptions case is covered because isBStackSession is false when no root user/key is present.

Test plan

  • Added tests/config.test.ts cases: external-grid web run → automate:false/app_automate:false; app caps off-infra → both false; web run on-infra → automate:true (regression guard). Existing cases are unchanged (they rely on the default isBrowserStackInfra = true).
  • ⚠️ This PR was authored via the GitHub API; the local vitest suite / tsc / eslint were not run in this environment (no disk space to install deps). Please rely on CI to validate.

🤖 Generated with Claude Code

When the service runs on an external (non-BrowserStack) grid - e.g. Test
Observability only, with credentials inside testObservabilityOptions - config.automate
was still set to !appAutomate (true), so the build was reported with origin Automate
even though it never ran on BrowserStack.

Gate automate/app_automate on isBrowserstackInfra() (the same signal that already
drives the browserstackAutomation field). External-grid runs now report
automate:false / app_automate:false; on-BrowserStack runs are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@pkg-pr-new

pkg-pr-new Bot commented Jun 4, 2026

Copy link
Copy Markdown

Open in StackBlitz

create-wdio

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/create-wdio@8

eslint-plugin-wdio

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/eslint-plugin-wdio@8

@wdio/allure-reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/allure-reporter@8

@wdio/appium-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/appium-service@8

@wdio/browser-runner

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/browser-runner@8

@wdio/browserstack-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/browserstack-service@8

@wdio/cli

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/cli@8

@wdio/concise-reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/concise-reporter@8

@wdio/config

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/config@8

@wdio/cucumber-framework

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/cucumber-framework@8

@wdio/dot-reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/dot-reporter@8

@wdio/firefox-profile-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/firefox-profile-service@8

@wdio/globals

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/globals@8

@wdio/jasmine-framework

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/jasmine-framework@8

@wdio/json-reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/json-reporter@8

@wdio/junit-reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/junit-reporter@8

@wdio/lighthouse-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/lighthouse-service@8

@wdio/local-runner

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/local-runner@8

@wdio/logger

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/logger@8

@wdio/mocha-framework

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/mocha-framework@8

@wdio/protocols

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/protocols@8

@wdio/repl

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/repl@8

@wdio/reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/reporter@8

@wdio/runner

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/runner@8

@wdio/sauce-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/sauce-service@8

@wdio/shared-store-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/shared-store-service@8

@wdio/smoke-test-cjs-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/smoke-test-cjs-service@8

@wdio/smoke-test-reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/smoke-test-reporter@8

@wdio/smoke-test-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/smoke-test-service@8

@wdio/spec-reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/spec-reporter@8

@wdio/static-server-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/static-server-service@8

@wdio/sumologic-reporter

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/sumologic-reporter@8

@wdio/testingbot-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/testingbot-service@8

@wdio/types

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/types@8

@wdio/utils

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/utils@8

@wdio/webdriver-mock-service

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/webdriver-mock-service@8

@wdio/xvfb

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/@wdio/xvfb@8

webdriver

npm i https://pkg.pr.new/AakashHotchandani/webdriverio/webdriver@8

webdriverio

npm i https://pkg.pr.new/AakashHotchandani/webdriverio@8

commit: 1f514f6

…or origin check

Address review feedback: isBrowserstackInfra also inspects per-capability `hostname`
overrides (multi-remote configs where only some capabilities target an external
grid). The origin classification call omitted `caps`, so those overrides were
ignored. Thread `capabilities` through, consistent with shouldAddServiceVersion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants