Repository navigation
feat: e2e trace pages replace the Playwright trace - #869
Merged
Merged
Conversation
This was referenced Oct 5, 2026
Contributor
🟢 e2e web: 91 passedAll 91 tests in 41 files
e2e 0.18.0 · 2m 3s · web · run artifacts |
Contributor
🟢 e2e web agent: 33 passed34 agent steps · 26 replayed from cache · 20 model calls · 76.7k tokens (84% cached) · $0.0050 All 33 tests in 5 files
e2e 0.18.0 · 2m 57s · web · run artifacts |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 166 files
Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.
Re-trigger cubic
Contributor
🟢 e2e android: 54 passed, 1 skippedAll 55 tests in 7 files
e2e 0.18.0 · 8m 41s · android-emulator · run artifacts |
Contributor
🟢 e2e android agent: 1 flaky, 19 passed24 agent steps · 115 model calls · 767.1k tokens (83% cached) · $0.0502
|
| Test | Agent | Time | |
|---|---|---|---|
| tests-agent/scenarios.e2e.ts · 1 flaky, 19 passed | 24 steps · 115 calls | 9m 20s | |
| 🟢 | act completes Login Form | 1 step · 3 calls | 32.7s |
| 🟢 | act completes Infinite Scroll List | 1 step · 3 calls | 26.0s |
| 🟢 | act completes Modal Flow | 1 step · 4 calls | 12.6s |
| 🟢 | act completes Bottom Tabs | 1 step · 3 calls | 8.9s |
| 🟢 | act completes Text Input Variations | 1 step · 3 calls | 10.2s |
| 🟢 | act completes Broken Accessibility | 2 steps · 6 calls | 37.6s |
| 🟢 | act completes Vision Only | 2 steps · 6 calls | 31.0s |
| 🟢 | act completes Gestures | 1 step · 10 calls | 34.9s |
| 🟢 | act completes Async States | 1 step · 15 calls | 56.6s |
| 🟢 | act completes Debounced Search | 1 step · 4 calls | 11.2s |
| 🟢 | act completes Huge Virtualized List | 1 step · 3 calls | 1m 11s |
| 🟢 | act completes Flattened Registration Form | 2 steps · 8 calls | 43.7s |
| 🟢 | act completes Flattened Login | 2 steps · 6 calls | 27.5s |
| 🟢 | act completes Sticky Chrome Target | 1 step · 4 calls | 15.9s |
| 🟢 | act completes WebView Accessibility | 1 step · 6 calls | 21.6s |
| 🟢 | act completes Permission Prompt | 1 step · 3 calls | 8.9s |
| 🟢 | act completes Product Catalog | 1 step · 5 calls | 13.0s |
| 🟢 | act completes Error Recovery | 1 step · 7 calls | 29.4s |
| 🟢 | act completes Choice Controls | 1 step · 6 calls | 32.0s |
| act completes Sequential Onboarding (1 failed attempt first) | 1 step · 10 calls | 34.6s |
e2e 0.18.0 · 10m 42s · android-emulator · run artifacts
Contributor
🟢 e2e ios: 52 passed, 1 skippedAll 53 tests in 7 files
e2e 0.18.0 · 6m 7s · ios-simulator · run artifacts |
Contributor
🟢 e2e ios agent: 21 passed, 1 skipped25 agent steps · 129 model calls · 790.8k tokens (82% cached) · $0.0538 All 22 tests in 1 file
e2e 0.18.0 · 12m 34s · ios-simulator · run artifacts |
okwasniewski
force-pushed
the
oskar/e2e-traces
branch
2 times, most recently
from
October 6, 2026 09:41
d752c7a to
0967590
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 25 files (changes from recent commits).
Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
okwasniewski
force-pushed
the
oskar/e2e-traces
branch
from
October 6, 2026 10:48
98ac87d to
4a28879
Compare
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 5 files (changes from recent commits).
Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 21 files (changes from recent commits).
Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 18 files (changes from recent commits).
Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Contributor
There was a problem hiding this comment.
All reported issues were addressed across 62 files (changes from recent commits).
Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.
Tip: Review your code locally with the cubic CLI to iterate faster.
Turn on auto-fix | Re-trigger cubic
okwasniewski
force-pushed
the
oskar/e2e-traces
branch
from
October 6, 2026 13:52
d36688e to
0cadb88
Compare
trace (config, target, test/describe) and --trace are refused, pointing at the failure pages and video. Removes startTrace/stopTrace from the engine contract, the trace artifact kind and not-required redaction from report-1, TRACE_WITHHELD, trace redaction, and the web engine's tracing. Recording modes stay, for video.
…replacement for this commit An @e2e-dev/web published before the removal still declares artifacts.startTrace/stopTrace and sits inside its e2e peer range; the manifest now accepts and never calls them. The refusal and docs name the markdown reporter's failure pages, the replacement at this commit, and stop calling on-all-retries a Playwright video mode.
…e, and tighten docs
…-blocking script in the late handler tests A screencast on Linux delivers the frame of a page that just painted after the paint, so a segment stopped right after a navigation had no frame and Playwright wrote it white (seen on main in the mid-attempt video test). A segment now waits up to 1s for its first frame. The late route handler tests routed a fetch the page issues after load, which raced the body's end; they route a script the document waits for instead.
…he download fragment pass
…polls, cache decision, and app log Every run writes .e2e/failures/<test>.md and the terminal names it. Steps record the node an action hit, assertion and waitFor readings, the cache's decision and write outcome, and what the app logged (new engine appLog hook, implemented by the web engine). REPLAY_STALE for a rekeyed recording names the key parts that changed.
…nd share one node describer A failure page that cannot be written is a stderr line, as junit.xml and summary.md are. cache.write moves to the report's step cache, since it is decided after act() returns. Kept entries are no longer rewritten only to gain keyedBy, so committed caches stay clean. Node and clip helpers reuse targetLabel and bound.
…ps in their video segment, and record point actions
… entry they now report
…aces/ instead of a Playwright zip
…e info, and what the test ran on
…empts from 1, and state the screen's facts once
… page beside its attempts
… cancelled requests, and word bounded logs and trace layout exactly
…nd where output goes
…r a store that did not write
…ce, the app log as its own record, and screens compared as data
…oll events, environment clip, and the web trace feed
…d, count screen bytes in UTF-8, and never cut a character in two
okwasniewski
force-pushed
the
oskar/e2e-traces
branch
from
October 7, 2026 09:11
0cadb88 to
641039a
Compare
Merged
This was referenced Oct 8, 2026
Open
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tracekeeps its modes but writes e2e's own trace instead of a Playwright zip: one markdown page per test,.e2e/results/<test>/trace.md, on every engine. Defaultretain-on-failure, locally and in CI. Not breaking for test authors. Replaces #822 + #823.Each page tells every step:
tap button "Add" (39ms))expect gave up after 15 reads in 1.6s: text "0 remaining" x15)EngineAttemptContext.appLog)↪ navigated to /login, a new tab the app opened, a frame that loaded (newnavigationsource)EngineAttemptContext.screen; web captures after each action, mobile hands over the snapshot its locate already took, so no extra device snapshots; not asked for when the attempt keeps no trace)EngineAttemptContext.environment)Same data lands in
report.json: step events (engine,poll,navigation),attempt.appLog(typed lines, each with its step index),step.phase,step.screen,attempt.environment,failure.viewport/nodes,step.cache.entry/detail/write(writegainsnot-written).Trace and video share one keep rule (
keeps): anything that ran and did not pass keeps them, interrupted included. An attempt that keeps no trace takes no screens at all (engine hook and observations gated by one session flag). The web engine reads a screen after each action when traced; measured on the web benchmark (91 tests, 804 steps, 1 worker): 108.0s with--trace offvs 119.6s default, about 14ms per step.Output layout: everything a test left is in one directory,
.e2e/results/<test>/, replacing.e2e/artifacts/+.e2e/traces/. Each run overwrites it;report.json,junit.xml,summary.mdstay at the.e2e/root.Old path for comparison:
artifacts/web/tests_todos.e2e.ts__todos__adds_20todos_20with…-92fc0c69/default/attempt-0/failure/screen.txt. Serial groups share the group's directory; MCP sessions write underresults/mcp/. CI uploads move to.e2e/results.Terminal per failure:
❯ app 2 errors, 2 warnings logged,❯ trace .e2e/results/<test>/trace.md; summary rowTraces .e2e/results/ (N pages, trace.md in each test's directory).summary.mdlinks each failure to its trace.report.jsonis now framed as data for scripts: the terminal drops itsReportrow (--debugprints the path;Report (not written)still shows when missing), and docs/skill lead withtrace.mdandsummary.md.Engine authors
startTrace/stopTraceno longer called (older engines declaring them still load);traceleavesartifactCapabilities.traceartifact kind,not-requiredredaction,TRACE_WITHHELD.Other fixes in here
--strict-cache(new entries storekeyedBy)Test plan
pnpm typecheck,pnpm lint,pnpm docs:check,pnpm docs:check-errors--retries 1,--last-failedrerun, serial group), App log section fromconsole.log/warn/error,--trace offwrites no page and no step screens, stress suite's hostile titles give safe folder names--strict-cacheafter the rebase: no REPLAY_STALE, 26/33 replayed (the rest need a model, covered by CI){ trace: 'on' }test gets a ✓ page,--trace offhonors the test option, no zip written; step screen diffs,ℹ console, popup + redirect lines,Ran on:browser/UARan on: platform ios · device <udid>