diff --git a/.changeset/abandoned-fixture-setup.md b/.changeset/abandoned-fixture-setup.md deleted file mode 100644 index d6fce36f9..000000000 --- a/.changeset/abandoned-fixture-setup.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'e2e': patch ---- - -Stop a fixture setup that finishes after its attempt was abandoned from starting later fixtures, hooks, or the test body. The late fixture still releases its own teardown. diff --git a/.changeset/aria-boolean-casing.md b/.changeset/aria-boolean-casing.md deleted file mode 100644 index e3c1beb6c..000000000 --- a/.changeset/aria-boolean-casing.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/web": patch ---- - -Read ARIA selected states and hidden-content flags without letter-case sensitivity. Native option selection still takes precedence. diff --git a/.changeset/auto-only-tool-choice.md b/.changeset/auto-only-tool-choice.md deleted file mode 100644 index 1ce5ccfad..000000000 --- a/.changeset/auto-only-tool-choice.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Agent steps on Meta's Muse Spark models, through AI Gateway or OpenCode Console, no longer fail with `MODEL_PROVIDER_FAILED: only auto is supported for tool_choice`. The refusal now reads as one, so the step retries with `auto` and the tool-calls-only rule, as it does for Anthropic and DeepSeek. diff --git a/.changeset/cache-key-engine-version.md b/.changeset/cache-key-engine-version.md deleted file mode 100644 index 748f1b4fa..000000000 --- a/.changeset/cache-key-engine-version.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -The replay cache key no longer carries the engine's version, so upgrading `@e2e-dev/web` or `@e2e-dev/mobile` keeps every committed recording. A recording re-finds its nodes at replay, which is where a node an engine release resolves differently is caught. Entries recorded before this release sit under the old keys: a `read-write` run records them again, and `--strict-cache` names them as `REPLAY_STALE` until it does. diff --git a/.changeset/cancel-code-exchange.md b/.changeset/cancel-code-exchange.md deleted file mode 100644 index 40cd22b98..000000000 --- a/.changeset/cancel-code-exchange.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Cancel the ChatGPT authorization-code token exchange when sign-in is interrupted. diff --git a/.changeset/cancel-queued-mcp-calls.md b/.changeset/cancel-queued-mcp-calls.md deleted file mode 100644 index 3ef5ba7a9..000000000 --- a/.changeset/cancel-queued-mcp-calls.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Skip MCP tool execution when the client cancels a queued call or cancels during argument validation. diff --git a/.changeset/cancel-readiness-probes.md b/.changeset/cancel-readiness-probes.md deleted file mode 100644 index c11d3688b..000000000 --- a/.changeset/cancel-readiness-probes.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Cancel in-flight app and service readiness HTTP probes when the run is interrupted. diff --git a/.changeset/cancellable-device-code-request.md b/.changeset/cancellable-device-code-request.md deleted file mode 100644 index 5bf45dacb..000000000 --- a/.changeset/cancellable-device-code-request.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Cancel the initial OAuth device-code request promptly when login is interrupted, without showing or saving a cancelled login. diff --git a/.changeset/credential-username-string.md b/.changeset/credential-username-string.md deleted file mode 100644 index 9c11d245b..000000000 --- a/.changeset/credential-username-string.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -A credential whose `username` is not a string, or with a key other than `username` and `password`, fails the config load with `INVALID_CONFIG` naming the field, instead of a raw "value of type function is not JSON-safe" error. Only `password` may be a function. A credential with no username in the config or `E2E_USER__USERNAME` is `INVALID_CONFIG` too. diff --git a/.changeset/dblclick-alias.md b/.changeset/dblclick-alias.md deleted file mode 100644 index 7c927aed5..000000000 --- a/.changeset/dblclick-alias.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": minor ---- - -`locator.dblclick()` is an alias of `doubleTap()`, as `click()` is of `tap()`, so a test ported from Playwright or Cypress keeps its double clicks. diff --git a/.changeset/decision-openai.md b/.changeset/decision-openai.md deleted file mode 100644 index 41e2b2417..000000000 --- a/.changeset/decision-openai.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/decision": minor ---- - -Ask the operation first and its target second, with the goal and the recent actions in every question. Every target question carries a `none` option. Offer the whole action grammar: hover, right-click, double-tap, long-press, drag, scroll-to, and upload. Add `vision: true`, which sends the screenshot as a file part of the decision state (`ai` 7.0.134, `openai.decisionModel('gpt-6-luna')` with `@ai-sdk/openai` 4.0.90) for screenshot verdicts and a `tap_at` that the text model names and score questions locate. Name a provider refusal. diff --git a/.changeset/decision-provider-options.md b/.changeset/decision-provider-options.md deleted file mode 100644 index fc26e6db5..000000000 --- a/.changeset/decision-provider-options.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'@e2e-dev/decision': minor ---- - -`decisionExecutor({ providerOptions })` sends provider options with every decide call, the completion checks included, such as `{ gateway: { zeroDataRetention: true } }` for Vercel AI Gateway. They reach the decision model only: the text model keeps the agents entry's `providerOptions`. A value that does not map provider names to option objects fails config load with `INVALID_CONFIG`. diff --git a/.changeset/direct-read-retry-deadline.md b/.changeset/direct-read-retry-deadline.md deleted file mode 100644 index cab3918da..000000000 --- a/.changeset/direct-read-retry-deadline.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Keep direct locator reads within their original action timeout when a stale reference requires resolution again. diff --git a/.changeset/e2e-traces.md b/.changeset/e2e-traces.md deleted file mode 100644 index 8af695de8..000000000 --- a/.changeset/e2e-traces.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -'e2e': minor -'@e2e-dev/web': minor -'@e2e-dev/mobile': minor ---- - -`trace` keeps its modes but now writes e2e's own trace instead of a Playwright zip: a markdown page per test, on every engine, and the terminal names it under each failed test (`❯ trace .e2e/results//trace.md`). The default is `retain-on-failure`, locally and in CI. Each step on the page shows what it did: the node an action landed on, the values an assertion or `waitFor` read while it waited, the replay cache's decision (where a replay stopped and why, what became of the recording, the entry file), what the app logged meanwhile (console output, uncaught exceptions, failed requests, and 4xx and 5xx responses, also gathered in one App log section with each line's step), where the page went (a navigation, a new tab, a frame), and how the screen changed since the step before. The page also names the browser and user agent, or the device, the test ran on. The `markdown` reporter's `summary.md` names each failure's trace page when one was kept. `report.json` is still written for scripts, but the `list` summary no longer prints its path (`--debug` does), and prints `Report (not written)` only when it is missing. Everything a test left now sits in one directory, `/results//` (`.e2e` by default): its `trace.md` (which shows the failure screenshot inline and links each file) and an `attempt-/` per attempt, numbered from 1, with the screen at failure as `screen-at-failure.txt`. It replaces `.e2e/artifacts////attempt-/`; a `--last-failed` rerun writes under each test's `rerun-/`. Point CI uploads at `.e2e/results`, and add `.e2e/results/` to `.gitignore` (or rerun `e2e init`). A stale recording made from this version on names what changed in its key under `--strict-cache`, such as the agent's context. For engine authors: `startTrace` and `stopTrace` are no longer called, `trace` leaves `artifactCapabilities`, `EngineAttemptContext.appLog` takes what the app logs, `navigation` where it went on its own, `screen` takes a screen for the trace (one a `locate` matched against, or one read right after an action, bounded), and `environment` takes what the attempt runs on. report-1 drops the `trace` artifact kind and the `not-required` redaction (`StoredArtifact.kind` and `redaction` narrow the same way), drops `TRACE_WITHHELD`, and adds `attempt.appLog` (each line with its step), `navigation` events, `failure.viewport` and `failure.nodes`, `step.phase`, `step.screen`, `attempt.environment`, and `step.cache.entry`, `detail`, and `write` (report only; `result.cache` from `agent.act()` gains `entry` and `detail`). `FinishedRun.traces` maps results to their pages. diff --git a/.changeset/eas-cancel-before-create.md b/.changeset/eas-cancel-before-create.md deleted file mode 100644 index 3df3d7c48..000000000 --- a/.changeset/eas-cancel-before-create.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/eas": patch ---- - -Avoid creating billed simulator sessions for acquisition requests cancelled before creation starts. diff --git a/.changeset/executor-vision.md b/.changeset/executor-vision.md deleted file mode 100644 index 20794791d..000000000 --- a/.changeset/executor-vision.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": minor ---- - -A custom `StepExecutor` that declares `vision: true` receives `agent.assert(..., { vision })` as `ctx.step.vision` instead of `UNSUPPORTED_CAPABILITY`. diff --git a/.changeset/focused-keyboard-leaves.md b/.changeset/focused-keyboard-leaves.md deleted file mode 100644 index 7424e769e..000000000 --- a/.changeset/focused-keyboard-leaves.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/web": patch ---- - -Check the focused element inside shadow roots and frames, so focus on a button there is refused. diff --git a/.changeset/init-skips-symlinked-mcp-config.md b/.changeset/init-skips-symlinked-mcp-config.md deleted file mode 100644 index e5374ca95..000000000 --- a/.changeset/init-skips-symlinked-mcp-config.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -`e2e init` no longer writes an MCP config through a symlink: a `.mcp.json` or `.cursor/mcp.json` reached through a link, or under a linked directory, is skipped with a warning naming the link and its target, the way a linked skill directory already was. Before, the merge read and overwrote the file the link pointed at, which could be a dotfiles repo or a shared config outside the project. diff --git a/.changeset/inline-line-text.md b/.changeset/inline-line-text.md deleted file mode 100644 index 99162bcdf..000000000 --- a/.changeset/inline-line-text.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/web": patch ---- - -The agent now reads a sentence with an inline link, emphasis, or code as one line of text, so `

Read our privacy policy for details.

` no longer reads "Read our for details." The link is still listed as its own control. diff --git a/.changeset/kernel-undefined-idle.md b/.changeset/kernel-undefined-idle.md deleted file mode 100644 index cee09bbad..000000000 --- a/.changeset/kernel-undefined-idle.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/kernel": patch ---- - -Keep the documented 600-second idle timeout when a JavaScript config passes an undefined timeout_seconds option. diff --git a/.changeset/located-frame-boxes.md b/.changeset/located-frame-boxes.md deleted file mode 100644 index f615c548f..000000000 --- a/.changeset/located-frame-boxes.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/web": patch ---- - -A node inside a frame reports its box in the page's viewport, past the frame's border and padding, whether it was located or observed, so `tap({ position })`, a point the agent taps, and `toHaveScreenshot` on it land on the node. diff --git a/.changeset/openai-compatible-provider-hints.md b/.changeset/openai-compatible-provider-hints.md deleted file mode 100644 index bc60a17c8..000000000 --- a/.changeset/openai-compatible-provider-hints.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -`store: false` and a prompt cache key now go only to OpenAI and Azure OpenAI Responses models and to `openai/*` models on the gateway. An OpenAI-compatible provider named `openai` no longer fails with `Unknown parameter: promptCacheKey`. diff --git a/.changeset/page-function-methods.md b/.changeset/page-function-methods.md deleted file mode 100644 index 07b85dedc..000000000 --- a/.changeset/page-function-methods.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/web": patch ---- - -Serialize method shorthand passed to browser.evaluate and function init scripts as runnable function expressions. diff --git a/.changeset/poll-last-read-at-deadline.md b/.changeset/poll-last-read-at-deadline.md deleted file mode 100644 index 2c4f6ca5d..000000000 --- a/.changeset/poll-last-read-at-deadline.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -A polling assertion takes its last read at its timeout instead of up to 100 ms after it, so it no longer passes on a state that arrived after the timeout. diff --git a/.changeset/provider-error-body.md b/.changeset/provider-error-body.md deleted file mode 100644 index 12bb740c8..000000000 --- a/.changeset/provider-error-body.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -When the provider package can't read a provider's error body, `MODEL_PROVIDER_FAILED` no longer shows an empty message. One example is the ChatGPT backend's HTTP 400 `{"detail": "Unsupported service_tier: fast"}`. The message and the `--ai-trace` entry now show the HTTP status and the body, with secrets redacted and the body cut at 1 KB. An `agent.act` step that used up its transport retries now names the last attempt's failure, as judgments already did. diff --git a/.changeset/public-runner-list.md b/.changeset/public-runner-list.md deleted file mode 100644 index 0ca526810..000000000 --- a/.changeset/public-runner-list.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'e2e': minor ---- - -`list` from `e2e/runner` reports the tests a config, file selection, and title grep would select, in process, without starting an app, an engine, or a worker. The result includes pairs a filter removed, positional arguments that matched no file, and the selected target names. `e2e list --reporter json` renames `skipReason` to `reason` and adds each pair's identity: `id`, `source`, `session`, `sessions`, `serialId`, and `agent`, beside the fields it already printed. diff --git a/.changeset/quiet-dialog-errors.md b/.changeset/quiet-dialog-errors.md deleted file mode 100644 index 955a60305..000000000 --- a/.changeset/quiet-dialog-errors.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/web": patch ---- - -Dismiss an unanswered native dialog when its handler throws, so the page operation completes and the handler error reaches the next step. diff --git a/.changeset/readonly-login-preflight.md b/.changeset/readonly-login-preflight.md deleted file mode 100644 index 755f91fc9..000000000 --- a/.changeset/readonly-login-preflight.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Reject read-only environment credentials before starting a provider login that cannot be saved. diff --git a/.changeset/shadow-section-landmarks.md b/.changeset/shadow-section-landmarks.md deleted file mode 100644 index 3945c6bc9..000000000 --- a/.changeset/shadow-section-landmarks.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/web": patch ---- - -Keep shadow-root headers and footers scoped to their surrounding article or section instead of reporting them as page landmarks. diff --git a/.changeset/smol-browsers.md b/.changeset/smol-browsers.md deleted file mode 100644 index e00120aeb..000000000 --- a/.changeset/smol-browsers.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/smol": minor ---- - -`@e2e-dev/smol`: `web({ browser: smol() })` runs a web target in Chromium microVMs on the runner's own computer through the `smolmachines` SDK. Each worker slot boots one warm browser machine, and with the default `scope: 'attempt'` every attempt runs in a copy-on-write branch of that running browser, made in about a second and deleted when the attempt ends. `prepare(cdpEndpoint)` drives the warm browser once before it is branched, so every attempt starts from its cookies, storage, and pages. `hostPorts` relays ports on the runner's loopback to the browser's `localhost`, `setup` runs a shell script in the machine before Chromium starts, `cpus` and `memoryMb` size each machine, and `scope: 'worker'` keeps one browser machine per slot instead. Machines are named after the run and target, so `sweep` deletes what a dead worker left, and downloads are read off the machine's disk. `app: { source, setup, start, port, env }` runs the app under test inside each browser machine instead of on the runner's computer: `source` is copied in, `setup` and `start` run there, and the run waits for `port` to answer, so in `attempt` scope every attempt's branch holds its own copy of the running app and its data. diff --git a/.changeset/soft-failure-outranks-skip.md b/.changeset/soft-failure-outranks-skip.md deleted file mode 100644 index a10f9d505..000000000 --- a/.changeset/soft-failure-outranks-skip.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'e2e': minor ---- - -Breaking: a failed `expect.soft` before `test.skip(...)` now fails the test, as in Playwright. Before, the test was reported `skipped` and the run exited 0 unless `failOnSkippedFailure` was set. The attempt is `failed` with the soft failures as its error, keeps the skip reason in its `skip` field, and retries like any failure. In `report.json`, an attempt or serial member may now carry `skip` with a failed status. The CLI prints the reason under the failure, and the trace page (`trace.md`) names it. `failOnSkippedFailure` now covers only a retry that skips after an earlier failed attempt. diff --git a/.changeset/strict-cache-never-writes.md b/.changeset/strict-cache-never-writes.md deleted file mode 100644 index 876e1a1c3..000000000 --- a/.changeset/strict-cache-never-writes.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -`--strict-cache` (and `cache.strict`) never writes or deletes a cache entry, even in `read-write` mode, and retries replay like the first attempt. Before, a retry skipped replay, called the model, and overwrote the committed entry, and a failed attempt could evict it. diff --git a/.changeset/tidy-cache-columns.md b/.changeset/tidy-cache-columns.md deleted file mode 100644 index f219a0a2a..000000000 --- a/.changeset/tidy-cache-columns.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Align `e2e cache ls` columns by displayed width for test file names containing wide characters, combining marks, or emoji. diff --git a/.changeset/trace-end-wait-after-handoff.md b/.changeset/trace-end-wait-after-handoff.md deleted file mode 100644 index bfb84ee87..000000000 --- a/.changeset/trace-end-wait-after-handoff.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -A recorded step the cache hands to the agent no longer accumulates the replay wait or the agent's turn into the next replay's end wait. The wait is measured from the agent's verdict if it settled without acting, or from its last action if it acted. diff --git a/.changeset/unicode-glob-character.md b/.changeset/unicode-glob-character.md deleted file mode 100644 index 3adf1ad16..000000000 --- a/.changeset/unicode-glob-character.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"e2e": patch ---- - -Match a Unicode code point with the `?` test glob wildcard, so a test file named with one emoji is selected by a single `?`. diff --git a/.changeset/visual-comparison.md b/.changeset/visual-comparison.md deleted file mode 100644 index 2a46b334a..000000000 --- a/.changeset/visual-comparison.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -"e2e": minor -"@e2e-dev/mobile": patch ---- - -Add `toHaveScreenshot` for visual comparison on every engine that captures pixels: `await expect(screen).toHaveScreenshot('home.png')` compares the screen and `expect(locator).toHaveScreenshot()` one element against a PNG stored beside the test file, one per target and operating system. A missing one fails the test and is written beside it, or in CI attached to the results instead; `--update-snapshots` (`-u`) writes missing and different ones and passes. Options: `threshold`, `maxDiffPixels`, `maxDiffPixelRatio`, `mask`, `maskColor`, `timeout`. A mismatch attaches the expected, actual, and diff images to the step. On iOS and Android the status bar is held in a fixed state for these screenshots. diff --git a/.changeset/web-check-async-state.md b/.changeset/web-check-async-state.md deleted file mode 100644 index fab33ac56..000000000 --- a/.changeset/web-check-async-state.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@e2e-dev/web": patch ---- - -`check()`, `uncheck()`, and the agent's `check` verb wait until the action timeout for the control's checked state to change after the click, so a controlled checkbox or switch that updates asynchronously passes instead of failing with `ACTION_FAILED`. diff --git a/.changeset/windowed-list-row-key.md b/.changeset/windowed-list-row-key.md deleted file mode 100644 index ebd6e20c7..000000000 --- a/.changeset/windowed-list-row-key.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -'e2e': patch ---- - -A recorded tap on a row of a windowed list no longer depends on which row the list rendered first. The container key of a row was its list's first text, a scroll position on Android (`Row 0512 in Row 0500`), so the replay found the row and still handed off whenever the window started one row later. diff --git a/packages/decision/CHANGELOG.md b/packages/decision/CHANGELOG.md index 7a9a29927..f01ff5eea 100644 --- a/packages/decision/CHANGELOG.md +++ b/packages/decision/CHANGELOG.md @@ -1,5 +1,13 @@ # @e2e-dev/decision +## 0.2.0 + +### Minor Changes + +- [#965](https://github.com/tester-army/e2e/pull/965) [`b92658b`](https://github.com/tester-army/e2e/commit/b92658be6eb61a086c30b11fc36399872b863910) Thanks [@okwasniewski](https://github.com/okwasniewski)! - Ask the operation first and its target second, with the goal and the recent actions in every question. Every target question carries a `none` option. Offer the whole action grammar: hover, right-click, double-tap, long-press, drag, scroll-to, and upload. Add `vision: true`, which sends the screenshot as a file part of the decision state (`ai` 7.0.134, `openai.decisionModel('gpt-6-luna')` with `@ai-sdk/openai` 4.0.90) for screenshot verdicts and a `tap_at` that the text model names and score questions locate. Name a provider refusal. + +- [#937](https://github.com/tester-army/e2e/pull/937) [`f1a1ac2`](https://github.com/tester-army/e2e/commit/f1a1ac241042c089ae773213c4e4f0da34f2f3d8) Thanks [@outof-place](https://github.com/outof-place)! - `decisionExecutor({ providerOptions })` sends provider options with every decide call, the completion checks included, such as `{ gateway: { zeroDataRetention: true } }` for Vercel AI Gateway. They reach the decision model only: the text model keeps the agents entry's `providerOptions`. A value that does not map provider names to option objects fails config load with `INVALID_CONFIG`. + ## 0.1.0 ### Minor Changes diff --git a/packages/decision/package.json b/packages/decision/package.json index 38f871074..3476c1262 100644 --- a/packages/decision/package.json +++ b/packages/decision/package.json @@ -1,6 +1,6 @@ { "name": "@e2e-dev/decision", - "version": "0.1.0", + "version": "0.2.0", "description": "Decision-model executors for e2e", "license": "Apache-2.0", "repository": { diff --git a/packages/e2e/CHANGELOG.md b/packages/e2e/CHANGELOG.md index 1fe33de10..a2df4b99c 100644 --- a/packages/e2e/CHANGELOG.md +++ b/packages/e2e/CHANGELOG.md @@ -1,5 +1,61 @@ # e2e +## 0.19.0 + +### Minor Changes + +- [#947](https://github.com/tester-army/e2e/pull/947) [`f3aa4c8`](https://github.com/tester-army/e2e/commit/f3aa4c8c9ec6c04db0f76cc243573e3480370ef5) Thanks [@kblok](https://github.com/kblok)! - `locator.dblclick()` is an alias of `doubleTap()`, as `click()` is of `tap()`, so a test ported from Playwright or Cypress keeps its double clicks. + +- [#869](https://github.com/tester-army/e2e/pull/869) [`1b92a6d`](https://github.com/tester-army/e2e/commit/1b92a6ddfa8768c303953548e1b1e6cc5afbac8d) Thanks [@okwasniewski](https://github.com/okwasniewski)! - `trace` keeps its modes but now writes e2e's own trace instead of a Playwright zip: a markdown page per test, on every engine, and the terminal names it under each failed test (`❯ trace .e2e/results//trace.md`). The default is `retain-on-failure`, locally and in CI. Each step on the page shows what it did: the node an action landed on, the values an assertion or `waitFor` read while it waited, the replay cache's decision (where a replay stopped and why, what became of the recording, the entry file), what the app logged meanwhile (console output, uncaught exceptions, failed requests, and 4xx and 5xx responses, also gathered in one App log section with each line's step), where the page went (a navigation, a new tab, a frame), and how the screen changed since the step before. The page also names the browser and user agent, or the device, the test ran on. The `markdown` reporter's `summary.md` names each failure's trace page when one was kept. `report.json` is still written for scripts, but the `list` summary no longer prints its path (`--debug` does), and prints `Report (not written)` only when it is missing. Everything a test left now sits in one directory, `/results//` (`.e2e` by default): its `trace.md` (which shows the failure screenshot inline and links each file) and an `attempt-/` per attempt, numbered from 1, with the screen at failure as `screen-at-failure.txt`. It replaces `.e2e/artifacts////attempt-/`; a `--last-failed` rerun writes under each test's `rerun-/`. Point CI uploads at `.e2e/results`, and add `.e2e/results/` to `.gitignore` (or rerun `e2e init`). A stale recording made from this version on names what changed in its key under `--strict-cache`, such as the agent's context. For engine authors: `startTrace` and `stopTrace` are no longer called, `trace` leaves `artifactCapabilities`, `EngineAttemptContext.appLog` takes what the app logs, `navigation` where it went on its own, `screen` takes a screen for the trace (one a `locate` matched against, or one read right after an action, bounded), and `environment` takes what the attempt runs on. report-1 drops the `trace` artifact kind and the `not-required` redaction (`StoredArtifact.kind` and `redaction` narrow the same way), drops `TRACE_WITHHELD`, and adds `attempt.appLog` (each line with its step), `navigation` events, `failure.viewport` and `failure.nodes`, `step.phase`, `step.screen`, `attempt.environment`, and `step.cache.entry`, `detail`, and `write` (report only; `result.cache` from `agent.act()` gains `entry` and `detail`). `FinishedRun.traces` maps results to their pages. + +- [#965](https://github.com/tester-army/e2e/pull/965) [`b92658b`](https://github.com/tester-army/e2e/commit/b92658be6eb61a086c30b11fc36399872b863910) Thanks [@okwasniewski](https://github.com/okwasniewski)! - A custom `StepExecutor` that declares `vision: true` receives `agent.assert(..., { vision })` as `ctx.step.vision` instead of `UNSUPPORTED_CAPABILITY`. + +- [#978](https://github.com/tester-army/e2e/pull/978) [`2f60b54`](https://github.com/tester-army/e2e/commit/2f60b547f4f7f0f2a9931884a46d44ccbd2e0a84) Thanks [@mvanhorn](https://github.com/mvanhorn)! - `list` from `e2e/runner` reports the tests a config, file selection, and title grep would select, in process, without starting an app, an engine, or a worker. The result includes pairs a filter removed, positional arguments that matched no file, and the selected target names. `e2e list --reporter json` renames `skipReason` to `reason` and adds each pair's identity: `id`, `source`, `session`, `sessions`, `serialId`, and `agent`, beside the fields it already printed. + +- [#924](https://github.com/tester-army/e2e/pull/924) [`d3d9081`](https://github.com/tester-army/e2e/commit/d3d9081eedd9f09d317c6443940dc9304a754e78) Thanks [@okwasniewski](https://github.com/okwasniewski)! - Breaking: a failed `expect.soft` before `test.skip(...)` now fails the test, as in Playwright. Before, the test was reported `skipped` and the run exited 0 unless `failOnSkippedFailure` was set. The attempt is `failed` with the soft failures as its error, keeps the skip reason in its `skip` field, and retries like any failure. In `report.json`, an attempt or serial member may now carry `skip` with a failed status. The CLI prints the reason under the failure, and the trace page (`trace.md`) names it. `failOnSkippedFailure` now covers only a retry that skips after an earlier failed attempt. + +- [#977](https://github.com/tester-army/e2e/pull/977) [`5137778`](https://github.com/tester-army/e2e/commit/5137778251928c42a736510fda002a984e924d17) Thanks [@okwasniewski](https://github.com/okwasniewski)! - Add `toHaveScreenshot` for visual comparison on every engine that captures pixels: `await expect(screen).toHaveScreenshot('home.png')` compares the screen and `expect(locator).toHaveScreenshot()` one element against a PNG stored beside the test file, one per target and operating system. A missing one fails the test and is written beside it, or in CI attached to the results instead; `--update-snapshots` (`-u`) writes missing and different ones and passes. Options: `threshold`, `maxDiffPixels`, `maxDiffPixelRatio`, `mask`, `maskColor`, `timeout`. A mismatch attaches the expected, actual, and diff images to the step. On iOS and Android the status bar is held in a fixed state for these screenshots. + +### Patch Changes + +- [#893](https://github.com/tester-army/e2e/pull/893) [`7eb03ca`](https://github.com/tester-army/e2e/commit/7eb03cafdb869c23fadf07ff15facc3f2ccc0790) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Stop a fixture setup that finishes after its attempt was abandoned from starting later fixtures, hooks, or the test body. The late fixture still releases its own teardown. + +- [#921](https://github.com/tester-army/e2e/pull/921) [`f2cf2ad`](https://github.com/tester-army/e2e/commit/f2cf2ad12aa1b6e5eeed4737d2d5b61e6be43531) Thanks [@okwasniewski](https://github.com/okwasniewski)! - Agent steps on Meta's Muse Spark models, through AI Gateway or OpenCode Console, no longer fail with `MODEL_PROVIDER_FAILED: only auto is supported for tool_choice`. The refusal now reads as one, so the step retries with `auto` and the tool-calls-only rule, as it does for Anthropic and DeepSeek. + +- [#929](https://github.com/tester-army/e2e/pull/929) [`e77c17a`](https://github.com/tester-army/e2e/commit/e77c17ae0c1e0ac9a1abadd69b88e31e1cd54baf) Thanks [@okwasniewski](https://github.com/okwasniewski)! - The replay cache key no longer carries the engine's version, so upgrading `@e2e-dev/web` or `@e2e-dev/mobile` keeps every committed recording. A recording re-finds its nodes at replay, which is where a node an engine release resolves differently is caught. Entries recorded before this release sit under the old keys: a `read-write` run records them again, and `--strict-cache` names them as `REPLAY_STALE` until it does. + +- [#935](https://github.com/tester-army/e2e/pull/935) [`3c37d98`](https://github.com/tester-army/e2e/commit/3c37d981d98eb729edfa1b2d61f994fdfaf0db49) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Cancel the ChatGPT authorization-code token exchange when sign-in is interrupted. + +- [#901](https://github.com/tester-army/e2e/pull/901) [`67b11b5`](https://github.com/tester-army/e2e/commit/67b11b5af5f42e647b313b0634714344fc340c45) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Skip MCP tool execution when the client cancels a queued call or cancels during argument validation. + +- [#936](https://github.com/tester-army/e2e/pull/936) [`5cdd3c6`](https://github.com/tester-army/e2e/commit/5cdd3c60be848f709af1424a890c26755bbbf86a) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Cancel in-flight app and service readiness HTTP probes when the run is interrupted. + +- [#899](https://github.com/tester-army/e2e/pull/899) [`d5a4eab`](https://github.com/tester-army/e2e/commit/d5a4eab34ea7b278cd61ef03f04f657c9b6405fb) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Cancel the initial OAuth device-code request promptly when login is interrupted, without showing or saving a cancelled login. + +- [#918](https://github.com/tester-army/e2e/pull/918) [`9fb70b9`](https://github.com/tester-army/e2e/commit/9fb70b9e99a379bda65b404d994a4f7aac9403c6) Thanks [@okwasniewski](https://github.com/okwasniewski)! - A credential whose `username` is not a string, or with a key other than `username` and `password`, fails the config load with `INVALID_CONFIG` naming the field, instead of a raw "value of type function is not JSON-safe" error. Only `password` may be a function. A credential with no username in the config or `E2E_USER__USERNAME` is `INVALID_CONFIG` too. + +- [#907](https://github.com/tester-army/e2e/pull/907) [`fef1246`](https://github.com/tester-army/e2e/commit/fef1246fb07fb35fb088d4489a2d7b16fc7a1187) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Keep direct locator reads within their original action timeout when a stale reference requires resolution again. + +- [#887](https://github.com/tester-army/e2e/pull/887) [`4132715`](https://github.com/tester-army/e2e/commit/4132715cd55f840018402faafd07376af76a68d7) Thanks [@DeryFerd](https://github.com/DeryFerd)! - `e2e init` no longer writes an MCP config through a symlink: a `.mcp.json` or `.cursor/mcp.json` reached through a link, or under a linked directory, is skipped with a warning naming the link and its target, the way a linked skill directory already was. Before, the merge read and overwrote the file the link pointed at, which could be a dotfiles repo or a shared config outside the project. + +- [#919](https://github.com/tester-army/e2e/pull/919) [`b6efcf7`](https://github.com/tester-army/e2e/commit/b6efcf71492192d1dc03b4b9ee9ccfe84b489e8e) Thanks [@okwasniewski](https://github.com/okwasniewski)! - `store: false` and a prompt cache key now go only to OpenAI and Azure OpenAI Responses models and to `openai/*` models on the gateway. An OpenAI-compatible provider named `openai` no longer fails with `Unknown parameter: promptCacheKey`. + +- [#920](https://github.com/tester-army/e2e/pull/920) [`745b353`](https://github.com/tester-army/e2e/commit/745b353b7a9e4686e7e996369725bd0f48a1b5d6) Thanks [@okwasniewski](https://github.com/okwasniewski)! - A polling assertion takes its last read at its timeout instead of up to 100 ms after it, so it no longer passes on a state that arrived after the timeout. + +- [#922](https://github.com/tester-army/e2e/pull/922) [`c10dc00`](https://github.com/tester-army/e2e/commit/c10dc00992fb0207ef09e3d983822c53d076a39e) Thanks [@okwasniewski](https://github.com/okwasniewski)! - When the provider package can't read a provider's error body, `MODEL_PROVIDER_FAILED` no longer shows an empty message. One example is the ChatGPT backend's HTTP 400 `{"detail": "Unsupported service_tier: fast"}`. The message and the `--ai-trace` entry now show the HTTP status and the body, with secrets redacted and the body cut at 1 KB. An `agent.act` step that used up its transport retries now names the last attempt's failure, as judgments already did. + +- [#905](https://github.com/tester-army/e2e/pull/905) [`ea0681a`](https://github.com/tester-army/e2e/commit/ea0681ae8bded230f857ef25dac951d4917c5efb) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Reject read-only environment credentials before starting a provider login that cannot be saved. + +- [#923](https://github.com/tester-army/e2e/pull/923) [`99d5ab5`](https://github.com/tester-army/e2e/commit/99d5ab53367c5dc8df5a1c9959a08f2b37d3db47) Thanks [@okwasniewski](https://github.com/okwasniewski)! - `--strict-cache` (and `cache.strict`) never writes or deletes a cache entry, even in `read-write` mode, and retries replay like the first attempt. Before, a retry skipped replay, called the model, and overwrote the committed entry, and a failed attempt could evict it. + +- [#984](https://github.com/tester-army/e2e/pull/984) [`bc453bd`](https://github.com/tester-army/e2e/commit/bc453bd52d5fe1b9a9fd59405db0c04676bfa188) Thanks [@ASVLCII](https://github.com/ASVLCII)! - Align `e2e cache ls` columns by displayed width for test file names containing wide characters, combining marks, or emoji. + +- [#881](https://github.com/tester-army/e2e/pull/881) [`cae1c65`](https://github.com/tester-army/e2e/commit/cae1c656380fd6b3b517d8c1f452472e5b326eb9) Thanks [@fecolinhares](https://github.com/fecolinhares)! - A recorded step the cache hands to the agent no longer accumulates the replay wait or the agent's turn into the next replay's end wait. The wait is measured from the agent's verdict if it settled without acting, or from its last action if it acted. + +- [#927](https://github.com/tester-army/e2e/pull/927) [`0ddea76`](https://github.com/tester-army/e2e/commit/0ddea76a7091ac8768e0ced2a9926b37e72e9497) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Match a Unicode code point with the `?` test glob wildcard, so a test file named with one emoji is selected by a single `?`. + +- [#998](https://github.com/tester-army/e2e/pull/998) [`841ea3f`](https://github.com/tester-army/e2e/commit/841ea3fa1810a570f4f46bd48b6418ee1a1dbc8d) Thanks [@okwasniewski](https://github.com/okwasniewski)! - A recorded tap on a row of a windowed list no longer depends on which row the list rendered first. The container key of a row was its list's first text, a scroll position on Android (`Row 0512 in Row 0500`), so the replay found the row and still handed off whenever the window started one row later. + ## 0.18.0 ### Minor Changes diff --git a/packages/e2e/package.json b/packages/e2e/package.json index 096ede61f..a359b4f96 100644 --- a/packages/e2e/package.json +++ b/packages/e2e/package.json @@ -1,6 +1,6 @@ { "name": "e2e", - "version": "0.18.0", + "version": "0.19.0", "description": "SDK, runner, and CLI for agentic end-to-end testing", "keywords": [ "e2e", diff --git a/packages/eas/CHANGELOG.md b/packages/eas/CHANGELOG.md index 03b1784fc..7c17e737f 100644 --- a/packages/eas/CHANGELOG.md +++ b/packages/eas/CHANGELOG.md @@ -1,5 +1,11 @@ # @e2e-dev/eas +## 0.3.1 + +### Patch Changes + +- [#906](https://github.com/tester-army/e2e/pull/906) [`9236bea`](https://github.com/tester-army/e2e/commit/9236bea370579ee149981ec6f938f151df35a434) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Avoid creating billed simulator sessions for acquisition requests cancelled before creation starts. + ## 0.3.0 ### Minor Changes diff --git a/packages/eas/package.json b/packages/eas/package.json index 65795e911..8f16ecbb2 100644 --- a/packages/eas/package.json +++ b/packages/eas/package.json @@ -1,6 +1,6 @@ { "name": "@e2e-dev/eas", - "version": "0.3.0", + "version": "0.3.1", "description": "EAS Simulators for e2e: run mobile targets on Expo's hosted iOS simulators and Android emulators", "keywords": [ "e2e", diff --git a/packages/kernel/CHANGELOG.md b/packages/kernel/CHANGELOG.md index 2cb8b6d9c..ceb88253d 100644 --- a/packages/kernel/CHANGELOG.md +++ b/packages/kernel/CHANGELOG.md @@ -1,5 +1,11 @@ # @e2e-dev/integrations +## 0.2.1 + +### Patch Changes + +- [#904](https://github.com/tester-army/e2e/pull/904) [`34b90ae`](https://github.com/tester-army/e2e/commit/34b90ae647cf7989793c858e37e22ffb63594dc0) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Keep the documented 600-second idle timeout when a JavaScript config passes an undefined timeout_seconds option. + ## 0.2.0 ### Minor Changes diff --git a/packages/kernel/package.json b/packages/kernel/package.json index 4546973a9..000344162 100644 --- a/packages/kernel/package.json +++ b/packages/kernel/package.json @@ -1,6 +1,6 @@ { "name": "@e2e-dev/kernel", - "version": "0.2.0", + "version": "0.2.1", "description": "Kernel hosted browsers for e2e: run web targets in Kernel's hosted Chromium", "keywords": [ "e2e", diff --git a/packages/mobile/CHANGELOG.md b/packages/mobile/CHANGELOG.md index 7f04a9f63..6b83d8cba 100644 --- a/packages/mobile/CHANGELOG.md +++ b/packages/mobile/CHANGELOG.md @@ -1,5 +1,15 @@ # @e2e-dev/mobile +## 0.11.0 + +### Minor Changes + +- [#869](https://github.com/tester-army/e2e/pull/869) [`1b92a6d`](https://github.com/tester-army/e2e/commit/1b92a6ddfa8768c303953548e1b1e6cc5afbac8d) Thanks [@okwasniewski](https://github.com/okwasniewski)! - `trace` keeps its modes but now writes e2e's own trace instead of a Playwright zip: a markdown page per test, on every engine, and the terminal names it under each failed test (`❯ trace .e2e/results//trace.md`). The default is `retain-on-failure`, locally and in CI. Each step on the page shows what it did: the node an action landed on, the values an assertion or `waitFor` read while it waited, the replay cache's decision (where a replay stopped and why, what became of the recording, the entry file), what the app logged meanwhile (console output, uncaught exceptions, failed requests, and 4xx and 5xx responses, also gathered in one App log section with each line's step), where the page went (a navigation, a new tab, a frame), and how the screen changed since the step before. The page also names the browser and user agent, or the device, the test ran on. The `markdown` reporter's `summary.md` names each failure's trace page when one was kept. `report.json` is still written for scripts, but the `list` summary no longer prints its path (`--debug` does), and prints `Report (not written)` only when it is missing. Everything a test left now sits in one directory, `/results//` (`.e2e` by default): its `trace.md` (which shows the failure screenshot inline and links each file) and an `attempt-/` per attempt, numbered from 1, with the screen at failure as `screen-at-failure.txt`. It replaces `.e2e/artifacts////attempt-/`; a `--last-failed` rerun writes under each test's `rerun-/`. Point CI uploads at `.e2e/results`, and add `.e2e/results/` to `.gitignore` (or rerun `e2e init`). A stale recording made from this version on names what changed in its key under `--strict-cache`, such as the agent's context. For engine authors: `startTrace` and `stopTrace` are no longer called, `trace` leaves `artifactCapabilities`, `EngineAttemptContext.appLog` takes what the app logs, `navigation` where it went on its own, `screen` takes a screen for the trace (one a `locate` matched against, or one read right after an action, bounded), and `environment` takes what the attempt runs on. report-1 drops the `trace` artifact kind and the `not-required` redaction (`StoredArtifact.kind` and `redaction` narrow the same way), drops `TRACE_WITHHELD`, and adds `attempt.appLog` (each line with its step), `navigation` events, `failure.viewport` and `failure.nodes`, `step.phase`, `step.screen`, `attempt.environment`, and `step.cache.entry`, `detail`, and `write` (report only; `result.cache` from `agent.act()` gains `entry` and `detail`). `FinishedRun.traces` maps results to their pages. + +### Patch Changes + +- [#977](https://github.com/tester-army/e2e/pull/977) [`5137778`](https://github.com/tester-army/e2e/commit/5137778251928c42a736510fda002a984e924d17) Thanks [@okwasniewski](https://github.com/okwasniewski)! - Add `toHaveScreenshot` for visual comparison on every engine that captures pixels: `await expect(screen).toHaveScreenshot('home.png')` compares the screen and `expect(locator).toHaveScreenshot()` one element against a PNG stored beside the test file, one per target and operating system. A missing one fails the test and is written beside it, or in CI attached to the results instead; `--update-snapshots` (`-u`) writes missing and different ones and passes. Options: `threshold`, `maxDiffPixels`, `maxDiffPixelRatio`, `mask`, `maskColor`, `timeout`. A mismatch attaches the expected, actual, and diff images to the step. On iOS and Android the status bar is held in a fixed state for these screenshots. + ## 0.10.0 ### Minor Changes diff --git a/packages/mobile/package.json b/packages/mobile/package.json index c8c5e20aa..e382647ae 100644 --- a/packages/mobile/package.json +++ b/packages/mobile/package.json @@ -1,6 +1,6 @@ { "name": "@e2e-dev/mobile", - "version": "0.10.0", + "version": "0.11.0", "description": "Mobile engine (iOS and Android) for e2e agentic end-to-end testing, built on agent-device", "keywords": [ "e2e", diff --git a/packages/smol/CHANGELOG.md b/packages/smol/CHANGELOG.md new file mode 100644 index 000000000..328ed396e --- /dev/null +++ b/packages/smol/CHANGELOG.md @@ -0,0 +1,7 @@ +# @e2e-dev/smol + +## 0.1.0 + +### Minor Changes + +- [#956](https://github.com/tester-army/e2e/pull/956) [`06bb726`](https://github.com/tester-army/e2e/commit/06bb72653f9c8ff3377db9124b06e0b2af746c30) Thanks [@BinSquare](https://github.com/BinSquare)! - `@e2e-dev/smol`: `web({ browser: smol() })` runs a web target in Chromium microVMs on the runner's own computer through the `smolmachines` SDK. Each worker slot boots one warm browser machine, and with the default `scope: 'attempt'` every attempt runs in a copy-on-write branch of that running browser, made in about a second and deleted when the attempt ends. `prepare(cdpEndpoint)` drives the warm browser once before it is branched, so every attempt starts from its cookies, storage, and pages. `hostPorts` relays ports on the runner's loopback to the browser's `localhost`, `setup` runs a shell script in the machine before Chromium starts, `cpus` and `memoryMb` size each machine, and `scope: 'worker'` keeps one browser machine per slot instead. Machines are named after the run and target, so `sweep` deletes what a dead worker left, and downloads are read off the machine's disk. `app: { source, setup, start, port, env }` runs the app under test inside each browser machine instead of on the runner's computer: `source` is copied in, `setup` and `start` run there, and the run waits for `port` to answer, so in `attempt` scope every attempt's branch holds its own copy of the running app and its data. diff --git a/packages/smol/package.json b/packages/smol/package.json index 967442aec..cb4c23b2b 100644 --- a/packages/smol/package.json +++ b/packages/smol/package.json @@ -1,6 +1,6 @@ { "name": "@e2e-dev/smol", - "version": "0.0.0", + "version": "0.1.0", "description": "smol machines browsers for e2e: every test attempt branches a warm Chromium microVM", "keywords": [ "e2e", diff --git a/packages/web/CHANGELOG.md b/packages/web/CHANGELOG.md index a1c7e7d1e..bf43a54bd 100644 --- a/packages/web/CHANGELOG.md +++ b/packages/web/CHANGELOG.md @@ -1,5 +1,29 @@ # @e2e-dev/web +## 0.14.0 + +### Minor Changes + +- [#869](https://github.com/tester-army/e2e/pull/869) [`1b92a6d`](https://github.com/tester-army/e2e/commit/1b92a6ddfa8768c303953548e1b1e6cc5afbac8d) Thanks [@okwasniewski](https://github.com/okwasniewski)! - `trace` keeps its modes but now writes e2e's own trace instead of a Playwright zip: a markdown page per test, on every engine, and the terminal names it under each failed test (`❯ trace .e2e/results//trace.md`). The default is `retain-on-failure`, locally and in CI. Each step on the page shows what it did: the node an action landed on, the values an assertion or `waitFor` read while it waited, the replay cache's decision (where a replay stopped and why, what became of the recording, the entry file), what the app logged meanwhile (console output, uncaught exceptions, failed requests, and 4xx and 5xx responses, also gathered in one App log section with each line's step), where the page went (a navigation, a new tab, a frame), and how the screen changed since the step before. The page also names the browser and user agent, or the device, the test ran on. The `markdown` reporter's `summary.md` names each failure's trace page when one was kept. `report.json` is still written for scripts, but the `list` summary no longer prints its path (`--debug` does), and prints `Report (not written)` only when it is missing. Everything a test left now sits in one directory, `/results//` (`.e2e` by default): its `trace.md` (which shows the failure screenshot inline and links each file) and an `attempt-/` per attempt, numbered from 1, with the screen at failure as `screen-at-failure.txt`. It replaces `.e2e/artifacts////attempt-/`; a `--last-failed` rerun writes under each test's `rerun-/`. Point CI uploads at `.e2e/results`, and add `.e2e/results/` to `.gitignore` (or rerun `e2e init`). A stale recording made from this version on names what changed in its key under `--strict-cache`, such as the agent's context. For engine authors: `startTrace` and `stopTrace` are no longer called, `trace` leaves `artifactCapabilities`, `EngineAttemptContext.appLog` takes what the app logs, `navigation` where it went on its own, `screen` takes a screen for the trace (one a `locate` matched against, or one read right after an action, bounded), and `environment` takes what the attempt runs on. report-1 drops the `trace` artifact kind and the `not-required` redaction (`StoredArtifact.kind` and `redaction` narrow the same way), drops `TRACE_WITHHELD`, and adds `attempt.appLog` (each line with its step), `navigation` events, `failure.viewport` and `failure.nodes`, `step.phase`, `step.screen`, `attempt.environment`, and `step.cache.entry`, `detail`, and `write` (report only; `result.cache` from `agent.act()` gains `entry` and `detail`). `FinishedRun.traces` maps results to their pages. + +### Patch Changes + +- [#898](https://github.com/tester-army/e2e/pull/898) [`b36ea5c`](https://github.com/tester-army/e2e/commit/b36ea5c4410d99df96d12ccc4d1a60d044846c1a) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Read ARIA selected states and hidden-content flags without letter-case sensitivity. Native option selection still takes precedence. + +- [#903](https://github.com/tester-army/e2e/pull/903) [`7949c25`](https://github.com/tester-army/e2e/commit/7949c25c8b47d1cadd3c2f6f91a6907f64dd0a6f) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Check the focused element inside shadow roots and frames, so focus on a button there is refused. + +- [#926](https://github.com/tester-army/e2e/pull/926) [`5fb69de`](https://github.com/tester-army/e2e/commit/5fb69de2ce829a7b3a84dbd965d3d2a99c313138) Thanks [@okwasniewski](https://github.com/okwasniewski)! - The agent now reads a sentence with an inline link, emphasis, or code as one line of text, so `

Read our privacy policy for details.

` no longer reads "Read our for details." The link is still listed as its own control. + +- [#977](https://github.com/tester-army/e2e/pull/977) [`5137778`](https://github.com/tester-army/e2e/commit/5137778251928c42a736510fda002a984e924d17) Thanks [@okwasniewski](https://github.com/okwasniewski)! - A node inside a frame reports its box in the page's viewport, past the frame's border and padding, whether it was located or observed, so `tap({ position })`, a point the agent taps, and `toHaveScreenshot` on it land on the node. + +- [#941](https://github.com/tester-army/e2e/pull/941) [`56c25bd`](https://github.com/tester-army/e2e/commit/56c25bd37a2210a94f01fdd2ec6b6e4e16aedfe3) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Serialize method shorthand passed to browser.evaluate and function init scripts as runnable function expressions. + +- [#889](https://github.com/tester-army/e2e/pull/889) [`0dffb3b`](https://github.com/tester-army/e2e/commit/0dffb3baad1378f6bd95f75cc5f972f66ade45af) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Dismiss an unanswered native dialog when its handler throws, so the page operation completes and the handler error reaches the next step. + +- [#943](https://github.com/tester-army/e2e/pull/943) [`9e33593`](https://github.com/tester-army/e2e/commit/9e33593d211ec704b1ccdea99a046e70e6cb3b90) Thanks [@rudycelekli](https://github.com/rudycelekli)! - Keep shadow-root headers and footers scoped to their surrounding article or section instead of reporting them as page landmarks. + +- [#920](https://github.com/tester-army/e2e/pull/920) [`745b353`](https://github.com/tester-army/e2e/commit/745b353b7a9e4686e7e996369725bd0f48a1b5d6) Thanks [@okwasniewski](https://github.com/okwasniewski)! - `check()`, `uncheck()`, and the agent's `check` verb wait until the action timeout for the control's checked state to change after the click, so a controlled checkbox or switch that updates asynchronously passes instead of failing with `ACTION_FAILED`. + ## 0.13.0 ### Minor Changes diff --git a/packages/web/package.json b/packages/web/package.json index fe04e5af2..7dc3bb186 100644 --- a/packages/web/package.json +++ b/packages/web/package.json @@ -1,6 +1,6 @@ { "name": "@e2e-dev/web", - "version": "0.13.0", + "version": "0.14.0", "description": "Web engine for e2e agentic end-to-end testing, built on Playwright", "keywords": [ "e2e", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index bb96f177b..3167d917b 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -6,9 +6,6 @@ importers: .: configDependencies: {} packageManagerDependencies: - '@pnpm/exe': - specifier: 12.3.4 - version: 12.3.4 pnpm: specifier: 12.3.4 version: 12.3.4 @@ -59,11 +56,6 @@ packages: cpu: [x64] os: [win32] - '@pnpm/exe@12.3.4': - resolution: {integrity: sha512-Rq7JokWAyYF9IFfn8zofB/q2AgnZlmoMuH5fMHB2+Ta61I9aD7sNm+woZpx6iH+5vb38chdSoYRvOtSqlH72YQ==} - engines: {node: '>=18.*'} - hasBin: true - pnpm@12.3.4: resolution: {integrity: sha512-lhqkH7B32joEpEHZ+OFevAyW2o73ELLrZ7+e58sGEOq9SPH9hfUc/+c4RnhfoPh8VqOocqHYk/hEZ0G1zORUVw==} engines: {node: '>=18.*'} @@ -95,17 +87,6 @@ snapshots: '@pnpm/exe.win32-x64@12.3.4': optional: true - '@pnpm/exe@12.3.4': - optionalDependencies: - '@pnpm/exe.darwin-arm64': 12.3.4 - '@pnpm/exe.darwin-x64': 12.3.4 - '@pnpm/exe.linux-arm64': 12.3.4 - '@pnpm/exe.linux-arm64-musl': 12.3.4 - '@pnpm/exe.linux-x64': 12.3.4 - '@pnpm/exe.linux-x64-musl': 12.3.4 - '@pnpm/exe.win32-arm64': 12.3.4 - '@pnpm/exe.win32-x64': 12.3.4 - pnpm@12.3.4: optionalDependencies: '@pnpm/exe.darwin-arm64': 12.3.4