[okwasniewski]

What and why

• Replay matched a recorded control only when every recorded field still held. A relabeled button with a stable test id, a placeholder edit, a link restyled as a button, or Like (0 likes) becoming Like (1 like) handed the step to the model on every run, and the entry was refreshed only after a relocation failed.
• relocateWithFallbacks (packages/e2e/src/cache/locate.ts) now walks a ladder once the exact match finds nothing. The first rung that settles on exactly one control wins.
• Every rung still holds the recorded row (within) and the recorded place among twins. An ambiguous exact match never falls back, an anonymous control has no fallback, and a test id that points at one control while the name points at another hands off as a conflict.
• A fallback match is acted on only when the next look finds the same control by the same rung (sightingKey in agent/replay.ts), so a page still leaving is never clicked.
• A replay that needed a fallback re-records the step in read-write mode, unless only a tally or time in a label moved (that moves again next run and would rewrite the entry every time).
• step.cache.relocated (report schema, both fixtures), · N relocated in the run summary, and agent_steps_relocated in telemetry count those steps, so read-only CI shows what to re-record.
• Fixes from review: the tally fold reads only a leading count that agrees with its noun; nodes still carrying the recorded test id veto a name pick even when there are several; a test id and name conflict is retried while the screen settles and never resolved by a recorded point; a fallback must hold its place across both looks; an entry counts as consumed only once an action is dispatched.
• REPLAY_POLICY_VERSION stays conservative/6: the exact match runs first and is unchanged, so every existing recording still finds the same node. Committed benchmark recordings replay as before.
• packages/e2e/src/cache/README.md is the design doc: lifecycle, key, ladder, anchors, routes, templates, write side, testing map, and the audit below. AGENTS.md points at it.

flowchart LR
  E{exact match} -- one --> F[act]
  E -- several --> H[hand off]
  E -- none --> T1[test id + role] --> T2[test id] --> A[role + name] --> R[name in role family] --> N[hand off]
  T1 & T2 & A & R -- one, same on next look, no test id/name conflict --> F

Loading

Rung	Absorbs	Report
test id + role, test id	copy change, role change	test-id
role + name	test id or placeholder change	accessible
name in role family	link/button, checkbox/switch, textbox/combobox	role-family

One locator for every lookup

• Action targets, live stale-node re-finds, a target's place among twins at record time, and end-state anchors all go through one lookup (locate in packages/e2e/src/cache/locate.ts): one projection per observation, the ladder, the container filter, and the test id veto.
• An anchor is split into identity and content. Content is its value, states, text, and its name when that is all the node says (a text node, a button that now reads "Following"). Identity walks the same ladder as a target; content is never loosened.
• A status found by its test id now holds after the region around it was relabeled or an iOS backend reports its role another way. The recorded reading on another node no longer counts while the node with the recorded test id says something else.
• An anchor with no identity left (an unnamed text node), or whose test id was re-minted, is matched on every field at once, as before. REPLAY_POLICY_VERSION stays: target matching is unchanged and anchors never act, and the committed caches replay as before (below).

flowchart LR
  R[recorded descriptor] --> P[candidates: in container, carrying content]
  P --> X{exact tiers}
  X -- match --> V[test id veto]
  X -- none --> L[fallback rungs] --> V
  V --> T[target: exactly one or recorded place]
  V --> A[anchor: any match]

Loading

Replay pacing

• A replay waited the full 2 s change timeout after every action whose effect the tree never shows (a right-click opening a native menu, caret keys, a tap that arms the next control), exactly as the recording did.
• The recorder now notes how each action settled: ObservationFeed reports whether any capture of the settled look left the action's starting shape, the dispatcher tells the recorder which action armed the wait, and an action that changed nothing is stored quiet. A note is dropped when it cannot answer for one action alone (another wait still pending, another action before the look).
• A replay gives a quiet action a 300 ms change wait (QUIET_CHANGE_WAIT_MS, ActionDispatcher.withChangeWait); held-still checks, relocation polling, and the end-state wait are unchanged. The pace is cleared once the call settles, so it never reaches the executor after a hand-off.
• quiet is a timing and never counts as a new flow. Entries recorded before learn it once, from the next recording or from a whole replay. A step whose entry did not serve it (anything but a gap) replaces the entry, so a stale mark cannot outlive it. The committed web benchmark entries gain their pacing in this PR.

Found on a simulator and an emulator

• agent-device switches between two iOS snapshot backends mid-session, and they report one screen differently. A recording now keeps one end-state anchor per label (a wrapper other "Flow completed" around the status exists in one backend only), and an iOS screen is named by its navigation bar's identifier: its label is the back button's text on every pushed native-stack screen, so every scenario read as "Benchmark Examples".
• A snapshot the iOS runner rejects mid-animation (regular-node-outside-cumulative-clip and two other reasons the engine did not recognize) is retried after the transition lands instead of failing the step with APP_UNREACHABLE.
• A replayed action gets one more try, after the screen holds still, only when the engine proves no input was sent (NODE_STALE, FRAME_NOT_FOUND, NOT_ACTIONABLE). An engine fault or timeout may have landed and hands off. agent-device's untouchable-target refusals (covered_by_interactive_descendants, target_covered, target_bounds_invalid) are now NOT_ACTIONABLE on mobile (refusedOr), which is what the debounced-search result tap hits; a web check whose click changed nothing, and a web press that timed out on the navigation its key started (log ends at elementHandle.press(...)), are ACTION_MAY_HAVE_COMMITTED.
• A step that starts before the app reaches the screen its recording began on waits for it on the settling backoff (awaitStartRoute), then settles on it, instead of missing as wrong-context, keeping the step's hand-off reserve. Stripe's demo presents its sheet once a request returns.
• A tapped checkbox, switch, or radio records the state it flipped and relocates only onto the control in that state: a recording that undid an accidental check would otherwise check it.
• A list row was keyed by the list's first visible row (Row 0499), unreachable after any scroll; a container key that is a neighbour of the target's own role is dropped.
• The committed iOS and Android recordings were keyed under older engines and replayed nothing on main (about 130 model calls per CI run). Re-recorded: iOS on iPhone 17 Pro iOS 26.5, Android on Pixel 7 API 36. The web entries are main's (test(web-benchmark): re-record agent recordings for @e2e-dev/web 0.12 #833).

Bugs from the audit, fixed here

• A step whose only effect read a time or id (Saved at 10:42) recorded it verbatim and failed its own replay every run. Anchors now compare by shape (anchorShape).
• The re-minted test id tier matched by name even when another control still carried the recorded test id. Now a conflict.
• A control recorded among twins was found when only one twin was left. Now ambiguous.
• A unique() param whose key contains | or }} wrote a placeholder that never parsed back (invalid-entry forever, REPLAY_STALE under strict). Pointers are escaped; old placeholders still read.
• Device screen titles with a record id (Order 48213) never matched across runs; hash-bang routes (#!/a, #!/b) all compared equal.
• A check passing after an expect.soft failure confirmed the entries staged before it (SoftFailures.verifiedBeforeFirstFailure).
• A replay cut off by a step timeout never marked its entry consumed, so it survived the failure.

Builds on #637 (closed): keeps its twins rule and its drift probe. Its label fold was tried as a fifth rung and dropped (word rules, a false match found in review); its element id rung is left out (framework counter ids typed into the wrong field in its own probe).

Verified

Ran it locally: yes

• Drift probe (the scratch server from fix(e2e): identify recorded controls by test id, labels only by shape #637, ported to the current config): 7 steps recorded once on main (b5dc0f87), the same entries copied into each build, then replayed against an app that drifted. e2e run --config e2e.probe.config.ts, real model (openai/gpt-6-luna-fast):

Step, drift	main	this branch
no drift, Saved at <clock> status	end-mismatch, model call	replayed
Save becomes Save changes, same test id	target-not-found, model	replayed, relocated: 1 (test-id)
Like (0 likes) becomes Like (3 likes), no test id	target-not-found, model	target-not-found, model (no label-shape rung)
link Settings becomes a button	target-not-found, model	replayed, relocated: 1 (role-family)
Increment click does nothing (Count: 0)	end-mismatch, hand off	end-mismatch, hand off
Delete item 3 gone, only Delete item 4	target-not-found	target-not-found, item 4 untouched
Email field id moves to Phone	replayed by label	replayed by label
model calls, drifted run	9	4

• Second drifted run on the branch with the cache the first one wrote: testid and settings match exactly (healed entry now reads Save changes).
• Web benchmark agentic suite, committed recordings, CI=1 --strict-cache, on main and on this branch: 33/33 passed on both, 26 replayed · 2 handed off · 4 missed on both, every step's cache mode identical, no relocated.
• Testbed suite: 71 passed.
• Final head, committed recordings, CI=1 --strict-cache, iPhone 17 Pro iOS 26.5 and Pixel 7 API 36:

Suite	before the locator commits (27bfcc7a)	this head
web agentic	33/33, 26 replayed · 2 handed off · 4 missed	33/33, same counts
iOS agentic	19/22, 14 replayed; Debounced Search action-failed, Choice Controls target-not-found, Stripe wrong-context	21/22, 16 replayed; Stripe target-not-found
Android agentic	20/20, 15 replayed	20/20, 15 replayed

• Debounced Search: the result tap was refused by agent-device (covered_by_interactive_descendants) and surfaced as ENGINE_FAILURE, so it was never retried. Now NOT_ACTIONABLE, retried, replays.
• Choice Controls: the entry carried an accidental Mushrooms check and its undo; the toggle-state rule refused it as designed. Re-recorded, replays.
• Stripe: the replay now waits for the sheet and starts on it, then the first tap is swallowed. Reproduced with agent-device alone: press on New card right after the sheet appears does nothing, the same press 1 s later opens the card form, and the node reports hittable: true at its final rect throughout, so no snapshot signal tells a presenting sheet from a ready one. Follow-up in agent-device (wait for app idle before a press, as XCUITest's tap() does).
• iOS before the mobile fixes, Modal Flow replay: end check passed 1 in 3 (backend switching, mid-animation snapshot). After: 12/12 replays of Login, Modal Flow, Bottom Tabs with no model call, Login 18.2-18.4 s, Modal 6.1-6.2 s, Bottom Tabs 3.4 s.
• iOS drift, recorded targets edited to an older app: relabel and placeholder change replay through test id + role, a role change through test id alone, an alert button recorded as a link through the role family; a renamed system sheet button with no test id hands off.
• Pacing, web benchmark agentic suite, three read-only runs before and after (median per replayed step): right-click menu 2420 to 774 ms, upload 2306 to 626 ms, context-menu trap 2497 to 834 ms, rich-text editor 2632 to 985 ms, gift card 2434 to 735 ms; all replayed steps except the 431-page scrollUntil 28.9 s to 20.3 s (-30%). Run to run spread under 2%. The scrollUntil step is unchanged in an interleaved A/B (107.3 vs 107.6 s).
• Pacing integration test (/arm fixture, real Chromium): recording 2.44 to 2.46 s, replay 0.61 to 0.62 s across three runs.
• Dynamic-content drift probe, 13 pages, recorded on main and replayed on both builds against the drifted app:

Page, drift	main	this branch
async control renders 1.2 s late	replayed	replayed
rows shuffled per load	replayed	replayed
new confirm dialog after Delete	end-mismatch, model	end-mismatch, model, evicted, re-recorded next run
control removed	target-not-found, test fails	same
A/B label under a stable test id	target-not-found when it flips	replayed, test-id
model calls, drifted run	16	6

[github-actions]

🟢 e2e web: 91 passed

All 91 tests in 41 files

	Test	Time
🟢	tests/aria-hidden-flow.e2e.ts · 2 passed	609ms
🟢	aria hidden flow › picks the Pro plan by its visible text and confirms it	444ms
🟢	aria hidden flow › another plan shows the error and keeps the picker open	165ms
🟢	tests/async-states.e2e.ts · 2 passed	7.1s
🟢	async states › reads the code off the transient toast and verifies it once the button enables	2.2s
🟢	async states › a wrong code is rejected after the toast has gone	4.9s
🟢	tests/canvas-only.e2e.ts · 2 passed	320ms
🟢	canvas only › taps square, circle, then diamond at their pixel coordinates	175ms
🟢	canvas only › a shape out of order paints the error and resets progress	145ms
🟢	tests/canvas-whiteboard.e2e.ts · 2 passed	496ms
🟢	canvas whiteboard › drags both shapes into their slots and saves	279ms
🟢	canvas whiteboard › bounces a shape off the wrong slot and refuses to save early	217ms
🟢	tests/cart-totals.e2e.ts · 2 passed	459ms
🟢	cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug)	246ms
🟢	cart totals › a quantity never drops below one	213ms
🟢	tests/checkout-review.e2e.ts · 1 passed	234ms
🟢	shows a raw greeting token and an undefined delivery date (planted bug), yet places the order	234ms
🟢	tests/context-menu-trap.e2e.ts · 2 passed	762ms
🟢	context menu trap › right-click opens nothing; the row menu renames the file	389ms
🟢	context menu trap › the decoy menu items refuse and keep the file	373ms
🟢	tests/control-inventory.e2e.ts · 8 passed	4.1s
🟢	control inventory › check sets the checkbox and the radio without flipping a set one	320ms
🟢	control inventory › check picks a radio the pick replaces with its summary	232ms
🟢	control inventory › a long press and a double click are told apart from a tap	1.0s
🟢	control inventory › the file menu opens on right-click only and Rename renames the file	315ms
🟢	control inventory › setInputFiles attaches files named from the project root	201ms
🟢	control inventory › the details view is left through the browser history	241ms
🟢	control inventory › scrollIntoView brings the footnote into view	341ms
🟢	control inventory › every exercise done shows the success message	1.4s
🟢	tests/css-content-ui.e2e.ts · 2 passed	519ms
🟢	css content ui › the tile labelled Ship it advances and confirming ships	293ms
🟢	css content ui › a wrong tile shows the CSS-rendered error and keeps the picker	226ms
🟢	tests/date-picker.e2e.ts · 2 passed	856ms
🟢	date picker › navigates three months ahead and books the required date	495ms
🟢	date picker › confirming without the exact date shows the error	361ms
🟢	tests/debounced-search.e2e.ts · 2 passed	3.1s
🟢	debounced search › a decoy one character off is refused	1.6s
🟢	debounced search › picking the exact item adds it to the cart	1.5s
🟢	tests/deferred-dialog.e2e.ts · 1 passed	1.2s
🟢	one submit opens one confirm, even when double-clicked	1.2s
🟢	tests/div-soup.e2e.ts · 2 passed	1.0s
🟢	div soup › a wrong code is rejected	439ms
🟢	div soup › the right code grants access after the delete tile fixes a typo	563ms
🟢	tests/drag-and-drop.e2e.ts · 2 passed	826ms
🟢	drag and drop › an out-of-order item resets the dropzone	411ms
🟢	drag and drop › dropping Banana then Cherry passes	415ms
🟢	tests/filter-deep-link.e2e.ts · 2 passed	418ms
🟢	filter deep link › filtering writes the query to the URL	224ms
🟢	filter deep link › opening a shared link restores the filter	194ms
🟢	tests/gift-card-purchase.e2e.ts · 3 passed	898ms
🟢	gift card purchase › refuses to buy without an amount	264ms
🟢	gift card purchase › refuses a recipient without an at sign	313ms
🟢	gift card purchase › sends the gift card	321ms
🟢	tests/hover-menu.e2e.ts · 2 passed	623ms
🟢	hover menu › a decoy leaf is refused	298ms
🟢	hover menu › hovering Account, then Billing, reaches Redeem voucher	325ms
🟢	tests/iframe-form.e2e.ts · 2 passed	981ms
🟢	iframe form › a wrong code is refused by the host page	467ms
🟢	iframe form › the code read from one frame applies through the other	514ms
🟢	tests/image-only-ui.e2e.ts · 3 passed	678ms
🟢	image only ui › the icons have no names, so the bare svg locator is ambiguous	148ms
🟢	image only ui › a wrong icon shows the error image and keeps the icons	252ms
🟢	image only ui › tapping the star, then the heart, verifies the icons	278ms
🟢	tests/infinite-scroll.e2e.ts · 1 passed	4.1s
🟢	scrolls the feed until the Golden Ticket loads, then claims it	4.1s
🟢	tests/kanban-board.e2e.ts · 3 passed	997ms
🟢	kanban board › pointer drags move both cards into the goal layout	461ms
🟢	kanban board › a swipe along a path drags a card as well	251ms
🟢	kanban board › a plain tap moves nothing and the untouched board is rejected	285ms
🟢	tests/login-form.e2e.ts · 4 passed	1.1s
🟢	login form › rejects a malformed email	264ms
🟢	login form › rejects a short password	285ms
🟢	login form › rejects an unknown account	248ms
🟢	login form › signs in and logs out	353ms
🟢	tests/lying-labels.e2e.ts · 4 passed	972ms
🟢	lying labels › the button whose accessible name says Pay now is the wrong one	257ms
🟢	lying labels › a text match for Pay now catches the decoy phrase too	145ms
🟢	lying labels › the field whose accessible name says Amount is the wrong one	291ms
🟢	lying labels › visible text drives the payment through	279ms
🟢	tests/native-dialogs.e2e.ts · 3 passed	667ms
🟢	native dialogs › answers the confirm, prompt, and alert chain	233ms
🟢	native dialogs › dismissing the confirm cancels the order	217ms
🟢	native dialogs › a wrong gift code is rejected	217ms
🟢	tests/newsletter-signup.e2e.ts · 1 passed	2.4s
🟢	subscribe renders no feedback and logs a TypeError (planted bug)	2.4s
🟢	tests/onboarding-wizard.e2e.ts · 5 passed	5.5s
🟢	onboarding wizard › back keeps what was typed and the brief comes back with it	376ms
🟢	onboarding wizard › the address step takes the brief through its bare-div dropdown	438ms
🟢	onboarding wizard › the profile step steers the calendar three decades back	1.4s
🟢	onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off	1.5s
🟢	onboarding wizard › the review step repeats the brief and finishes with the PIN	1.8s
🟢	tests/order-history.e2e.ts · 2 passed	1.6s
🟢	order history › overview and settings render their panels	275ms
🟢	order history › the orders tab never leaves its loading state (planted bug)	1.3s
🟢	tests/otp-auto-advance.e2e.ts · 3 passed	1.7s
🟢	otp auto-advance › focus advances with each digit and the full code verifies	575ms
🟢	otp auto-advance › backspace on an empty box clears and refocuses the previous one	213ms
🟢	otp auto-advance › a wrong code clears every box and starts over	921ms
🟢	tests/overlay-trap.e2e.ts · 1 passed	732ms
🟢	continues only after the banner and the overlay are gone	732ms
🟢	tests/playbook-cleanup.e2e.ts · 2 passed	688ms
🟢	playbook cleanup › deleting the draft removes it from an exact-name search	343ms
🟢	playbook cleanup › cancelling the dialog keeps the draft searchable	345ms
🟢	tests/price-sorting.e2e.ts · 3 passed	650ms
🟢	price sorting › opens in the featured order with sale prices struck through	193ms
🟢	price sorting › low to high sorts by the original price, not the paid one (planted bug)	225ms
🟢	price sorting › high to low is the same wrong order reversed (planted bug)	232ms
🟢	tests/product-reviews.e2e.ts · 1 passed	1.2s
🟢	the running-shoe page lists coffee-grinder reviews (planted bug)	1.2s
🟢	tests/promo-storefront.e2e.ts · 1 passed	1.4s
🟢	buys a deal once the promo chrome is out of the way	1.4s
🟢	tests/rich-text-editor.e2e.ts · 2 passed	773ms
🟢	rich text editor › bolding exactly the required word passes the check	374ms
🟢	rich text editor › bold that spills into the previous word fails the check	399ms
🟢	tests/saved-addresses.e2e.ts · 3 passed	913ms
🟢	saved addresses › adds an address and clears the form	338ms
🟢	saved addresses › ignores an add with an empty field	310ms
🟢	saved addresses › delete shows its toast but keeps the address, on retry too (planted bug)	265ms
🟢	tests/shadow-dom-form.e2e.ts · 2 passed	748ms
🟢	shadow DOM form › rejects a wrong access code	370ms
🟢	shadow DOM form › grants access with the code from the hint inside the root	378ms
🟢	tests/smoke.e2e.ts · 1 passed	527ms
🟢	home lists the scenarios and opens one	527ms
🟢	tests/stale-dom.e2e.ts · 1 passed	2.2s
🟢	taps the moving target three times across rebuilds of the list	2.2s
🟢	tests/sticky-chrome.e2e.ts · 1 passed	288ms
🟢	accepts the terms once the button clears the sticky footer	288ms
🟢	tests/virtualized-table.e2e.ts · 1 passed	1m 5s
🟢	scrolls the windowed table to the Golden Row and claims it	1m 5s
🟢	tests/wishlist.e2e.ts · 2 passed	1.7s
🟢	wishlist › saving toggles the button and the tab badge	320ms
🟢	wishlist › the wishlist tab renders the empty state despite the badge count (planted bug)	1.4s

_{e2e 0.17.0 · 2m 2s · web · run artifacts}

[github-actions]

🟢 e2e web agent: 33 passed

34 agent steps · 26 replayed from cache · 20 model calls · 76.3k tokens (92% cached) · $0.0035

All 33 tests in 5 files

	Test	Agent	Time
🟢	tests-agent/control-inventory.e2e.ts · 7 passed	7 steps	7.7s
🟢	control inventory › check sets the checkbox and the radio	1 step	674ms
🟢	control inventory › check picks a radio the pick replaces with its summary	1 step	298ms
🟢	control inventory › a long press and a double click are chosen over a plain tap	1 step	953ms
🟢	control inventory › a right-click opens the file menu and Rename renames the file	1 step	2.6s
🟢	control inventory › upload attaches the named project files	1 step	2.4s
🟢	control inventory › back returns from the details view to the inventory	1 step	375ms
🟢	control inventory › scroll_to brings the footnote into view	1 step	382ms
🟢	tests-agent/login-form.e2e.ts · 1 passed	1 step	353ms
🟢	act signs in with the benchmark account	1 step	353ms
🟢	tests-agent/planted-bugs.e2e.ts · 3 passed	4 steps · 8 calls	11.8s
🟢	cart totals: a judgment catches the stale order summary	2 steps · 4 calls	6.0s
🟢	checkout review: a judgment catches the broken copy	1 step · 1 call	1.5s
🟢	wishlist: a judgment catches the lost saved items	1 step · 3 calls	4.3s
🟢	tests-agent/scenarios.e2e.ts · 21 passed	21 steps · 12 calls	3m 5s
🟢	act completes shadow-dom-form	1 step · 3 calls	4.2s
🟢	act completes lying-labels	1 step · 3 calls	4.6s
🟢	act completes iframe-form	1 step · 3 calls	4.4s
🟢	act completes infinite-scroll	1 step	9.1s
🟢	act completes virtualized-table	1 step	2m 10s
🟢	act completes overlay-trap	1 step	935ms
🟢	act completes sticky-chrome	1 step	982ms
🟢	act completes async-states	1 step · 3 calls	8.2s
🟢	act completes debounced-search	1 step	2.4s
🟢	act completes hover-menu	1 step	652ms
🟢	act completes drag-and-drop	1 step	562ms
🟢	act completes kanban-board	1 step	918ms
🟢	act completes context-menu-trap	1 step	2.8s
🟢	act completes stale-dom	1 step	632ms
🟢	act completes otp-auto-advance	1 step	786ms
🟢	act completes date-picker	1 step	1.1s
🟢	act completes rich-text-editor	1 step	8.2s
🟢	act completes playbook-cleanup	1 step	667ms
🟢	act completes promo-storefront	1 step	1.3s
🟢	act completes gift-card-purchase	1 step	2.4s
🟢	act filters the list and the URL carries the filter	1 step	273ms
🟢	tests-agent/unique-cache.e2e.ts · 1 passed	1 step	326ms
🟢	act adds an address with a run-unique label	1 step	326ms

_{e2e 0.17.0 · 3m 26s · web · run artifacts}

[cubic-dev-ai]

All reported issues were addressed across 32 files

<sub>Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.

Re-trigger cubic</sub>

[github-actions]

🟢 e2e android: 54 passed, 1 skipped

All 55 tests in 7 files

	Test	Time
⏭️	tests/controls.e2e.ts · 24 passed, 1 skipped	3m 8s
🟢	control inventory › a label query resolves a labeled text field	23.9s
🟢	control inventory › a labeled field answers name, display value, value reads, fill, and clear	12.4s
🟢	control inventory › a placeholder query and attribute reads find the field	5.5s
🟢	control inventory › a secure field takes a fill and denies value reads	7.1s
🟢	control inventory › press sends a character and Enter through the soft keyboard	8.3s
🟢	control inventory › a field autofocuses on mount and focus() moves the focus	12.1s
🟢	control inventory › toBeFocused follows the focus	8.9s
🟢	control inventory › a switch checks, unchecks, and reports its state	10.4s
🟢	control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap	7.2s
🟢	control inventory › checkbox and radio roles check, uncheck, and report their state	12.4s
🟢	control inventory › a tab role answers getByRole	3.6s
🟢	control inventory › a radio group reports the checked option	9.1s
🟢	control inventory › buttons report disabled, enabled, and a state that arrives later	7.8s
🟢	control inventory › a display: none element is absent rather than hidden, its visible twin attached	6.4s
🟢	control inventory › a header answers its accessible name	3.9s
🟢	control inventory › a tab row answers count, all, texts, filters, positions, and switching	7.1s
🟢	control inventory › a list answers counts, list-form text, reads, and boxes in order	6.8s
⏭️	control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢	control inventory › a long press lands as one	7.5s
🟢	control inventory › a double tap lands inside the 300 ms window	8.4s
🟢	control inventory › an ambiguous locator fails at once	5.0s
🟢	control inventory › screenshot, restart, and clearState reopen the app on its home list	12.5s
🟢	refusals › verbs a device never declares are refused before the engine	4ms
🟢	refusals › keys the soft keyboard cannot send are refused by the engine	134ms
🟢	refusals › a missing locator fails as not found once its timeout passes	1.1s
🟢	tests/device-fixture.e2e.ts · 8 passed	1m 12s
🟢	device fixture › setOrientation rotates the window the app measures	5.9s
🟢	device fixture › home leaves the app, openApp brings it back where it was	7.0s
🟢	device fixture › foregroundApp sees the home screen after home()	5.3s
🟢	device fixture › closeApp then openApp relaunches on the home list	6.6s
🟢	device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN	5.0s
🟢	device fixture › device.back pops the scenario	5.2s
🟢	device fixture › setNetwork and setAirplaneMode change what the app sees	16.1s
🟢	device fixture › setLocation and clearLocation change what the app reads	20.6s
🟢	tests/device.e2e.ts · 4 passed	39.2s
🟢	permission prompt › allowing the system dialog enables the microphone	7.3s
🟢	permission prompt › a denied prompt recovers once the permission is granted outside	14.4s
🟢	sequential onboarding echoes the exact values	10.4s
🟢	onboarding rejects a malformed email before advancing	7.1s
🟢	tests/flows-lists.e2e.ts · 7 passed	2m 29s
🟢	infinite scroll reaches the target item	20.0s
🟢	product catalog adds exactly two of the right variant	9.8s
🟢	gestures: long-press, then swipe left	8.7s
🟢	gestures: double-tap completes the flow	10.7s
🟢	sticky chrome: accept below the fold, then continue	13.8s
🟢	async states: load, pull to refresh, claim through the toast	12.4s
🟢	huge virtualized list reaches row 512	1m 14s
🟢	tests/flows.e2e.ts · 7 passed	1m 1s
🟢	text inputs unlock the submit button	7.7s
🟢	filled text inputs report their values	7.4s
🟢	modal flow confirms through the native alert	9.2s
🟢	bottom tabs act inside the last tab	8.4s
🟢	error recovery retries, then confirms the delete	9.7s
🟢	choice controls place the exact order	9.0s
🟢	debounced search selects the target once results arrive	9.4s
🟢	tests/login-form.e2e.ts · 3 passed	32.4s
🟢	login form › rejects a malformed email	9.2s
🟢	login form › rejects an unknown account	9.5s
🟢	login form › signs in and logs out	13.8s
🟢	tests/smoke.e2e.ts · 1 passed	6.5s
🟢	home lists the scenarios and opens one	6.5s

_{e2e 0.17.0 · 9m 8s · android-emulator · run artifacts}

[github-actions]

🟢 e2e android agent: 20 passed

24 agent steps · 14 replayed from cache · 45 model calls · 247.9k tokens (81% cached) · $0.0179

All 20 tests in 1 file

	Test	Agent	Time
🟢	tests-agent/scenarios.e2e.ts · 20 passed	24 steps · 45 calls	8m 2s
🟢	act completes Login Form	1 step	39.1s
🟢	act completes Infinite Scroll List	1 step	32.3s
🟢	act completes Modal Flow	1 step	13.1s
🟢	act completes Bottom Tabs	1 step	9.5s
🟢	act completes Text Input Variations	1 step	11.6s
🟢	act completes Broken Accessibility	2 steps · 5 calls	29.7s
🟢	act completes Vision Only	2 steps · 1 call	12.1s
🟢	act completes Gestures	1 step · 11 calls	47.7s
🟢	act completes Async States	1 step · 14 calls	54.2s
🟢	act completes Debounced Search	1 step	7.5s
🟢	act completes Huge Virtualized List	1 step	1m 14s
🟢	act completes Flattened Registration Form	2 steps · 1 call	12.2s
🟢	act completes Flattened Login	2 steps · 6 calls	29.6s
🟢	act completes Sticky Chrome Target	1 step	23.9s
🟢	act completes WebView Accessibility	1 step · 5 calls	19.2s
🟢	act completes Permission Prompt	1 step · 2 calls	24.9s
🟢	act completes Product Catalog	1 step	11.2s
🟢	act completes Error Recovery	1 step	9.7s
🟢	act completes Choice Controls	1 step	9.8s
🟢	act completes Sequential Onboarding	1 step	11.7s

_{e2e 0.17.0 · 8m 3s · android-emulator · run artifacts}

[github-actions]

🟢 e2e ios: 1 flaky, 51 passed, 1 skipped

⚠️ 1 flaky test passed on a retry

⚠️ control inventory › toBeFocused follows the focus
tests/controls.e2e.ts:101

ASSERTION_FAILED at step 5 of 5: expect.toBeFocused getByRole("textbox", name: "Autofocus field"), after 5.3s

• Expected: focused
• Observed: default states (1 match)
• Failed the same way on 2 attempts: ASSERTION_FAILED at step 5. The retry that passed is the exception.
• Screen: dev.e2e.benchmark / Control Inventory

Evidence: screenshot, log

All 53 tests in 7 files

	Test	Time
⚠️	tests/controls.e2e.ts · 1 flaky, 22 passed, 1 skipped	3m 13s
🟢	control inventory › a label query resolves a labeled text field	9.3s
🟢	control inventory › a labeled field answers name, display value, value reads, fill, and clear	13.7s
🟢	control inventory › a placeholder query and attribute reads find the field	4.9s
🟢	control inventory › a secure field takes a fill and denies value reads	9.5s
🟢	control inventory › press sends a character and Enter through the soft keyboard	7.4s
🟢	control inventory › a field autofocuses on mount and focus() moves the focus	11.1s
⚠️	control inventory › toBeFocused follows the focus (2 failed attempts first)	12.6s
🟢	control inventory › a switch checks, unchecks, and reports its state	11.1s
🟢	control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap	9.1s
🟢	control inventory › checkbox and radio roles check, uncheck, and report their state	14.8s
🟢	control inventory › a radio group reports the checked option	11.9s
🟢	control inventory › buttons report disabled, enabled, and a state that arrives later	10.7s
🟢	control inventory › a display: none element is absent rather than hidden, its visible twin attached	8.2s
🟢	control inventory › a header answers its accessible name	5.0s
🟢	control inventory › a tab row answers count, all, texts, filters, positions, and switching	7.1s
🟢	control inventory › a list answers counts, list-form text, reads, and boxes in order	7.6s
⏭️	control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢	control inventory › a long press lands as one	9.1s
🟢	control inventory › pointer verbs land at the point asked for	10.4s
🟢	control inventory › an ambiguous locator fails at once	6.3s
🟢	control inventory › screenshot, restart, and clearState reopen the app on its home list	11.2s
🟢	refusals › verbs a device never declares are refused before the engine	3ms
🟢	refusals › keys the soft keyboard cannot send are refused by the engine	815ms
🟢	refusals › a missing locator fails as not found once its timeout passes	1.3s
🟢	tests/device-fixture.e2e.ts · 9 passed	1m 17s
🟢	device fixture › setAppearance flips the color scheme the app reads	10.4s
🟢	device fixture › setOrientation rotates the window the app measures	6.4s
🟢	device fixture › the clipboard reads back what was written	4.7s
🟢	device fixture › home leaves the app, openApp brings it back where it was	6.3s
🟢	device fixture › closeApp then openApp relaunches on the home list	7.8s
🟢	device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN	4.6s
🟢	device fixture › device.back pops the scenario	5.9s
🟢	device fixture › setLocation and clearLocation change what the app reads	14.8s
🟢	device fixture › enrollBiometrics and setBiometrics answer a biometric prompt	16.5s
🟢	tests/device.e2e.ts · 2 passed	53.0s
🟢	sequential onboarding echoes the exact values	38.9s
🟢	onboarding rejects a malformed email before advancing	14.1s
🟢	tests/flows-lists.e2e.ts · 7 passed	3m 32s
🟢	infinite scroll reaches the target item	29.6s
🟢	product catalog adds exactly two of the right variant	10.7s
🟢	gestures: long-press, then swipe left	9.8s
🟢	sticky chrome: accept below the fold, then continue	11.0s
🟢	async states: load, pull to refresh, claim through the toast	12.7s
🟢	huge virtualized list reaches row 512	2m 6s
🟢	web view coupon form applies the code shown on the page	12.9s
🟢	tests/flows.e2e.ts · 7 passed	1m 12s
🟢	text inputs unlock the submit button	15.7s
🟢	filled text inputs report their values	9.8s
🟢	modal flow confirms through the native alert	10.1s
🟢	bottom tabs act inside the last tab	7.8s
🟢	error recovery retries, then confirms the delete	9.7s
🟢	choice controls place the exact order	8.1s
🟢	debounced search selects the target once results arrive	10.6s
🟢	tests/login-form.e2e.ts · 3 passed	45.6s
🟢	login form › rejects a malformed email	11.9s
🟢	login form › rejects an unknown account	12.1s
🟢	login form › signs in and logs out	21.6s
🟢	tests/smoke.e2e.ts · 1 passed	9.5s
🟢	home lists the scenarios and opens one	9.5s

_{e2e 0.17.0 · 5m 59s · ios-simulator · run artifacts}

[github-actions]

🟢 e2e ios agent: 21 passed, 1 skipped

25 agent steps · 13 replayed from cache · 57 model calls · 389.2k tokens (80% cached) · $0.0285

All 22 tests in 1 file

	Test	Agent	Time
⏭️	tests-agent/scenarios.e2e.ts · 21 passed, 1 skipped	25 steps · 57 calls	10m 36s
🟢	act completes Login Form	1 step	34.8s
🟢	act completes Infinite Scroll List	1 step	39.1s
🟢	act completes Modal Flow	1 step	12.4s
🟢	act completes Bottom Tabs	1 step	9.6s
🟢	act completes Text Input Variations	1 step	17.8s
🟢	act completes Broken Accessibility	2 steps · 6 calls	31.4s
🟢	act completes Vision Only	2 steps · 6 calls	31.3s
🟢	act completes Gestures	1 step · 7 calls	35.5s
🟢	act completes Async States	1 step · 10 calls	51.3s
🟢	act completes Debounced Search	1 step	13.5s
🟢	act completes Flattened Registration Form	2 steps · 1 call	31.7s
🟢	act completes Flattened Login	2 steps · 14 calls	1m 43s
🟢	act completes Sticky Chrome Target	1 step	33.9s
🟢	act completes WebView Accessibility	1 step	15.1s
🟢	act completes Permission Prompt	1 step · 2 calls	33.8s
🟢	act completes Photo Picker	1 step · 2 calls	28.9s
🟢	act completes Product Catalog	1 step	12.9s
🟢	act completes Error Recovery	1 step	12.6s
🟢	act completes Choice Controls	1 step	16.0s
🟢	act completes Stripe PaymentSheet	1 step · 9 calls	59.5s
🟢	act completes Apple Pay	1 step	11.8s
⏭️	act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

_{e2e 0.17.0 · 10m 36s · ios-simulator · run artifacts}

[cubic-dev-ai]

All reported issues were addressed across 36 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[okwasniewski]

[claude-opus-5-5] responding on behalf of okwasniewski

Ready for review. One thread left for you: volatile-only anchors are now compared by shape (Saved at 10:42 matches Saved at 10:45), gated by the evidence check. A reviewer cited earlier team feedback preferring exact text there; the reasoning is in the thread on packages/e2e/src/cache/README.md.

[okwasniewski]

On 841a384:

Can the label shape rung move a replay onto a different option? AGE_PATTERNS and the today|yesterday|tomorrow branch of RELATIVE_TIME fold a duration or day anywhere in a label, so Snooze 1 hour and Snooze 3 hours, Last 7 days and Last 30 days, or Deliver today and Deliver tomorrow share one shape. If the recorded option is gone at replay and one sibling with the same role is left, the label shape rung finds it as a single match, labelDriftOnly marks it transient so it is never re-recorded, and anchorShape masks the duration in an end anchor like Snoozed for 3 hours, so the step passes while doing something other than what was recorded. Folding these only at an edge or after a separator (as now already is), or only before ago, would keep a missing option handing off instead.

_{reviewed 841a384 by Hermes on behalf of @okwasniewski}

[cubic-dev-ai]

All reported issues were addressed across 10 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 28 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 2 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[okwasniewski]

On 236ebb8:

Can the new replay retry tap a mobile button twice? The retry in packages/e2e/src/agent/replay.ts:454 fires for any failed call that is not ACTION_MAY_HAVE_COMMITTED and not a policy or argument refusal, and dispatched only records that the grammar method was called. On mobile, translateError in packages/mobile/src/errors.ts never emits ACTION_MAY_HAVE_COMMITTED and ignores agent-device's details.dispatched, so a runner failure where the input may already have landed (for example an iOS main thread timeout) arrives as APP_UNREACHABLE and is retried. On a Submit or Send button the app then gets the action twice, and the replay passes or hands off on the doubled state. Toggles are covered by the new state filter, other controls are not. Retrying only on positive proof that nothing was sent (NODE_STALE, NOT_ACTIONABLE, or dispatched: 'no'), or mapping dispatched: 'unknown' and a missing dispatched on action calls to ACTION_MAY_HAVE_COMMITTED in the mobile engine, would keep the promise that a possible commit never repeats.

The duration folding from the last review is gone with the label shape rung.

_{reviewed 236ebb8 by Hermes on behalf of @okwasniewski}

[cubic-dev-ai]

All reported issues were addressed across 4 files (changes from recent commits).

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[okwasniewski]

On 27bfcc7:

The mobile double tap from the last review is closed: mobile only raises NODE_STALE and NOT_ACTIONABLE before the call reaches the device, so a failed press is no longer retried.

Can a replayed web press still go out twice? locator.press sends the key and then waits for any navigation it started, and that wait logs only elementHandle.press("Enter"), never the performing ... action or waiting for scheduled navigations lines POST_DISPATCH_PATTERN looks for. So Enter on a form whose submit is slow times out after the key landed, classifyActionError in packages/web/src/actions.ts returns NOT_ACTIONABLE, and retriesAfter presses Enter again, which can submit the form twice. fill, clear and selectOption have the same gap. Treating a timeout on those non pointer actions as ACTION_MAY_HAVE_COMMITTED, or adding elementHandle\.press\( to POST_DISPATCH_PATTERN, would keep the retry to proven misses.

The branch now has a merge conflict with main.

_{reviewed 27bfcc7 by Hermes on behalf of @okwasniewski}

[cubic-dev-ai]

All reported issues were addressed across 23 files (changes from recent commits).

_{Heads up: you’re close to your included review allowance. Set a flex budget so reviews don’t pause.}

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic</sub>

[okwasniewski]

On dc80df2:

The web press question from the last review is closed for the plain case: POST_DISPATCH_PATTERN now counts the elementHandle.press(...) log line, so a press that timed out waiting on its navigation is reported as possibly committed and not retried.

Can a press that was retried inside Playwright still go out twice? If the input re-renders before focus, Playwright logs element was detached from the DOM, retrying, then on the next pass logs elementHandle.press("Enter"), sends the key, and times out waiting on the navigation it started. classifyActionError in packages/web/src/actions.ts:289 tests DETACHED_PATTERN before the timeout branch, so that log is classified NODE_STALE (retryable), which is in NEVER_REACHED_APP, and replay sends Enter again, e.g. submitting a form twice. Checking POST_DISPATCH_PATTERN before DETACHED_PATTERN, with a test for a log that has the detach line followed by the press line, would keep it uncertain.

_{reviewed dc80df2 by Hermes on behalf of @okwasniewski}