Skip to content

fix(mobile): bound reads by operation budget with last-read grace (#1099) - #1112

Closed
Totopo27 wants to merge 18 commits into
tester-army:mainfrom
Totopo27:fix/1099-mobile-reads-budget
Closed

Totopo27 wants to merge 18 commits into
tester-army:mainfrom
Totopo27:fix/1099-mobile-reads-budget

Conversation

@Totopo27

@Totopo27 Totopo27 commented Oct 13, 2026 •

Copy link
Copy Markdown

What and why

  • On @e2e-dev/mobile a snapshot or screenshot that hung held the step until the test timeout (Bound mobile reads by the operation budget without turning a slow last poll into a timeout #1099). Every device read now runs under the operation's budget (InputBudget.read, packages/mobile/src/support.ts), so a hung read fails the step as OPERATION_TIMEOUT (or ACTION_MAY_HAVE_COMMITTED once an input of that operation went out).
  • Bounding reads alone broke slow polls: on iOS a snapshot takes 1-2 s, so the last read of waitFor/expect/toHaveScreenshot got cut at the deadline and failed as a hang instead of LOCATOR_NOT_FOUND.
  • Core fix: ReadWindow (packages/e2e/src/internal/time.ts). Each read of one wait gets the time left plus the slowest read that wait already finished, capped by the running phase's deadline. A read cut off within that tail ends the wait as a normal timeout; a read that hung with more time left is still the failure.
  • @e2e-dev/mobile now needs e2e >=0.21.4 (this PR's e2e patch changeset ships it): on 0.21.3 core, mobile reads would end at the deadline with no last-read grace.
  • No public type change: ReadWindow is internal; OperationContext.timeoutMs docs note the last read can get more than the deadline leaves.

Issue

Fixes #1099

Verified

Ran it locally: no device run. Unit evidence on the head merged with main (f4e8c8e1a + nit 7f1d05ed8):

  • Reverting the read bound (readWithin -> direct call) fails 6 mobile tests, e.g. "bounds a snapshot by the operation budget" and "takes no second snapshot for a capture the budget abandoned".
  • Reverting the slowest-read grace in ReadWindow.timeoutMs() fails 5 core tests (time, locator-engine, locator-wait-for), e.g. "lets a slow last resolve finish past the deadline: a late match resolves".
  • Reverting the slowest-read term in ReadWindow.cutOff() fails "ends on a read cut off within the longest read time of the deadline, and fails on one that hung with more left".
  • pnpm check passes; e2e (3171), mobile (420), web (290) unit suites pass.
main this branch
A hung snapshot holds the step until the test timeout The step fails at its own timeout
(with bounded reads only) a slow last iOS read in expect fails as a hang The last read finishes; a missing node is LOCATOR_NOT_FOUND / ASSERTION_FAILED

Checklist

  • Changeset added (pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only.
  • Breaking change: title carries !, the changeset body names what breaks and what replaces it, and the deprecation warning landed a window earlier (stability policy in CONTRIBUTING.md).
  • Docs updated in the same PR: the docs/**/*.mdx page for the behavior, and skills/e2e/ if the skill describes it.
  • Contract change: emitted .d.ts reviewed, tests/types/sdk-types.ts updated; wire change edits the schema and both fixtures.
  • Engine contract change (e2e/engine): changesets for e2e, @e2e-dev/web, and @e2e-dev/mobile.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 18 files

Tip: cubic used a learning from your PR history. Let your coding agent read cubic learnings directly with the cubic MCP.

View guided diff | Re-trigger cubic

Comment thread docs/mobile.mdx Outdated
Comment thread packages/mobile/src/surface.ts Outdated
Comment thread docs/reference/engine.mdx Outdated
Comment thread packages/mobile/src/surface.ts Outdated
Comment thread packages/e2e/src/internal/time.ts Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread packages/mobile/src/surface.ts Outdated
Comment thread docs/reference/engine.mdx Outdated
Comment thread docs/reference/engine.mdx Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 5 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread packages/mobile/src/surface.ts

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up, the mobile ReadBudget plumbing and the sparse-retry stop are useful pieces. A few things before this can land:

  1. #1099 asks for the poll to give the last read extra time past the deadline, so a slow read still finishes. This PR keeps the read cut at the deadline and only changes the error it reports. On main a slow last read that finds the element passes (the deleted "lets a locate outlive its budget" test covered that); here it becomes LOCATOR_NOT_FOUND. Could the last read's budget be remaining + latency (capped by the phase budget) instead?

  2. A fixed 2,000 ms constant is a guess that will break on slower CI devices or bigger trees (a 3 s read starting with 2.5 s left is labelled hung again). We'd rather the poll measure it: track how long completed reads take in that poll, and use that for both the extra time and the cut-off check. That also drops the readLatency engine field entirely.

  3. If an engine field stays: newer @e2e-dev/mobile with an older e2e (peer is >=0.15.0) fails with unknown key "readLatency", so the peer floor would need raising. An engine contract change also needs a @e2e-dev/web changeset (see CONTRIBUTING, Engine contract).

  4. Budgets: each snapshot/screenshot creates a fresh ReadBudget, so observe can take up to 3x timeoutMs. Could we create one budget per operation in engine.ts (like InputBudget) and pass it through, instead of optional timeoutMs? params? That also covers the keyboard-dismiss snapshots, which are still unbounded, and lets ReadBudget fold into InputBudget (remainingMs() is unused).

  5. Tests: please add a mobile + LocatorEngine test for slow-but-valid last read -> LOCATOR_NOT_FOUND, hung read with time to spare -> ACTION_FAILED, and a late match that still resolves. The new locator test relies on 100 ms real-timer margins; fake timers would make it deterministic.

  6. Docs/changeset: keep them to what users see, without internal names (pollCondition, "read latency tolerance"). The changeset says the latency is granted past the deadline, which this version doesn't do.

Happy to help shape the measured-latency version if useful.

@Totopo27

Copy link
Copy Markdown
Author

@okwasniewski Thanks for the detailed review! All points have been addressed in the latest update:

  1. Measured read latency & extra time past deadline: Locator and condition polling loops now track how long completed reads take dynamically. When starting a read with less than the measured latency remaining on the deadline, the read is granted
    emaining + latency\ (capped by the attempt budget), allowing slow last reads and late matches to complete instead of being cut prematurely.
  2. Dropped
    eadLatency\ engine field
    : The engine contract property and manifest checks were removed entirely; latency measurement is self-contained within the poll loops, avoiding any engine contract bumps or peer floor changes.
  3. **Folded \ReadBudget\ into \InputBudget**: \ReadBudget\ was merged into \InputBudget\ with a
    ead()\ method for non-committing reads (throwing \OPERATION_TIMEOUT\ rather than \ACTION_MAY_HAVE_COMMITTED). One operation budget is created in \engine.ts\ and passed into \observe, \locate, and \screenshot, ensuring multi-step operations like \observe\ share a single budget rather than multiplying \ imeoutMs. Also bounds keyboard dismiss snapshots.
  4. Deterministic tests with fake timers: Added unit tests in \LocatorEngine\ covering:
    • slow-but-valid last read -> \LOCATOR_NOT_FOUND\
    • hung read with time to spare -> \ACTION_FAILED\
    • late match that resolves
      All using \�i.useFakeTimers()\ for deterministic execution without real-timer margins.
  5. Cleaned changeset & documentation: Updated to user-facing explanations without internal names.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 20 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread packages/e2e/src/locator/engine.ts Outdated
Comment thread packages/e2e/src/internal/time.ts Outdated
Comment thread packages/mobile/src/support.ts Outdated
Comment thread packages/mobile/tests/unit/engine.test.ts Outdated
Comment thread docs/mobile.mdx Outdated
Comment thread packages/mobile/src/surface.ts Outdated
Comment thread packages/e2e/src/locator/engine.ts Outdated
Comment thread packages/e2e/src/locator/screen.ts Outdated
Comment thread packages/mobile/src/support.ts Outdated
Comment thread packages/mobile/src/support.ts Outdated

@okwasniewski okwasniewski left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick and thorough turnaround, @Totopo27! Dropping the engine field, measuring latency, sharing one budget per operation and moving to fake timers are all the right moves. A few things before this can land:

  1. Assertions and waitFor still cut the last read. The grace only applies in resolveCounted, performWithin and scrollUntilVisible. pollCondition (expect(locator), locator.waitFor, toHaveScreenshot) measures latency but still reads with operationWithin(deadline). Mobile now enforces that bound, so expect(screen.getByText('Saved')).toBeVisible() on iOS, where the text shows up during the last snapshot, passes on main and fails here with ASSERTION_FAILED.
  2. One helper instead of four copies. The latency/readTimeout/cut-off logic is in 4 places (engine.ts:231, engine.ts:422, screen.ts:277, time.ts:128), and resolveOnce's retry path still uses the old 100ms rule. Could this be one small helper in internal/time.ts, with pollCondition's evaluate receiving the read timeoutMs? That fixes (1) too. While there: time only the engine call (the sample currently includes retry sleeps), don't hand each retry the full readTimeout again, and keep the Math.max(1, ...) floor on an explicit timeoutMs.
  3. Peer floor. New @e2e-dev/mobile with an older e2e (peer >=0.15.0) bounds reads but gets no grace from the old core, so a slow last read is ACTION_FAILED again, which is the original #1099 failure. The peer floor needs to move to the e2e version that ships this.
  4. InputBudget.read() should honor sent, like check() does. After the keyboard-dismiss swipe goes out, a snapshot that times out should be ACTION_MAY_HAVE_COMMITTED, not OPERATION_TIMEOUT, so a replay doesn't send the input again.
  5. Docs. docs/reference/engine.mdx: please restore the caveat that signal does not abort at timeoutMs. That is still true, and engine authors rely on it. docs/mobile.mdx and the changeset should only claim the grace where it applies. A late match on an action still leaves perform with no budget, so "avoiding timeouts when elements appear late" overstates it today.
  6. Tests. "slow-but-valid last resolve gives LOCATOR_NOT_FOUND" also passes on main, because the fake engine ignores timeoutMs. Could it enforce the timeout it's given? The shared-observe-budget test times out in the viewport probe, so it needs viewport geometry on the snapshot. A mobile-level test for a slow last read that still finishes would close the loop.

Smaller: capture() can still send a second snapshot after the budget has abandoned the read; the InputBudget doc still says "input only"; and the path.posix.join change looks unrelated to this PR.

This was an automated review.

okwasniewski and others added 2 commits October 16, 2026 14:53
…-budget

# Conflicts:
#	packages/mobile/src/surface.ts
Address review: one ReadWindow helper for every wait (pollCondition,
locator waits, scrollUntilVisible, toHaveScreenshot), InputBudget.read
honors sent, no snapshot retry after an abandoned capture, sparse
retries stop when the budget cannot fit one, mobile peer e2e >=0.21.2.

Co-Authored-By: Claude <noreply@anthropic.com>
@okwasniewski

Copy link
Copy Markdown
Member

Automated review (Claude Code on behalf of @okwasniewski)

Since there were no updates after our last review, we finished this ourselves and pushed to your branch. We merged main, no force push.

  • Last read finishes. pollCondition gives its last read the extra time, so expect, waitFor and toHaveScreenshot on iOS behave like main when the read is slow.
  • One ReadWindow in internal/time.ts replaces the 4 latency copies and cutOffAtDeadline. Only the engine call is timed, each retry gets only the time that's left, and resolveOnce follows the same rule.
  • Peer floor: mobile now requires e2e >=0.21.2 <1.
  • InputBudget.read() honors sent: a read that times out after an input went out is ACTION_MAY_HAVE_COMMITTED.
  • Docs: the signal caveat is back in engine.mdx, and mobile.mdx claims the grace only for waitFor, expect and toHaveScreenshot. Split into e2e and mobile changesets.
  • Tests: the fake engine enforces timeoutMs. Added a slow-last-read test for mobile, no second snapshot after an abandoned capture, and read after send. Without the grace, the slow-read tests fail.
  • Also needed for correctness:

pnpm check passes, and so do the full e2e suite (3461), mobile (332) and web unit tests. Not run on a real simulator yet; CI mobile will cover it.

Open: PollConditionOptions (exported from e2e/engine) now has reads?: ReadWindow, but ReadWindow isn't exported, so we'll either export it or keep it off the public type.

Co-Authored-By: Claude <noreply@anthropic.com>
@okwasniewski

Copy link
Copy Markdown
Member

Automated review (Claude Code on behalf of @okwasniewski)

Follow-up in 21aeaa9: ReadWindow is off the public type. pollCondition and PollConditionOptions match main exactly. Internal callers (waitFor, expect, toHaveScreenshot) use an internal pollReads that takes the read window. pnpm check passes, and so do the e2e, mobile and web unit tests.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 21 files (changes from recent commits).

Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

View guided diff | Re-trigger cubic

Comment thread packages/mobile/tests/unit/engine.test.ts
Comment thread packages/e2e/tests/unit/time.test.ts Outdated
Comment thread packages/mobile/src/surface.ts
@okwasniewski
okwasniewski dismissed stale reviews from themself October 16, 2026 13:34

Addressed (maintainer finished the remaining changes or author did). Automated review.

@okwasniewski

Copy link
Copy Markdown
Member

Automated review (Claude Code on behalf of @okwasniewski)

Thanks @Totopo27. I checked aaac086 and 2018295 and pushed a small follow-up on top (no force push):

  • Kept the new ReadWindow cap test. It fails without the cap and without the slowest-read grace. I made the round-2 read 50 ms so it doesn't land exactly on the 100 ms timeout, and dropped the comment block.
  • Reverted the extra abort check in surface.ts (snapshot()). The same window is also unchecked on the other two return last paths, and main has that gap too. Captures and sleep already take the signal, so a real abort is still caught. If it's worth fixing, it belongs in its own PR with a test.
  • Peer floor is now 0.21.3. e2e 0.21.2 shipped in chore: version packages #1176 without this PR's runner changes, so @e2e-dev/mobile now requires >=0.21.3, and the changeset says the same.
  • The merge resolved the conflict in engine.mdx correctly.

pnpm check passes, and so do the e2e time and mobile unit tests. We'll label it once CI is green.

@mintlify

mintlify Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Preview approval required

This pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit 39e3e99.

Approve preview

@okwasniewski

Copy link
Copy Markdown
Member

Thanks @Totopo27. Re-checked after the two main merges (02ad4dd). The merge resolution is right: the launching / app-not-running path from #1281 and the viewport probe from #1291 both carry the read budget, and the PR still merges cleanly into current main. pnpm check, the e2e unit suite (3810 passed) and the mobile unit suite (359 passed) pass on the PR merged into main. Reverting the locateSnapshot cutoff fails fails a locate whose capture spent the budget with the reason, so that test can fail.

I pushed f7f9584 with two small fixes:

  1. Peer floor: e2e 0.21.3 shipped on 2026-10-23 without this PR's runner side (its dist has no ReadWindow or last-read grace), so a user on 0.21.3 would satisfy >=0.21.3 and get mobile reads cut at the deadline. packages/mobile/package.json now requires e2e >=0.21.4 <1 and the changeset says 0.21.4.
  2. locateSnapshot JSDoc: the merge kept main's wording ("leaves no budget for another poll"). Restored a description of the current condition (a capture that leaves less of the budget than it took) and rewrapped it.

Left for you (small)

  • Four cubic threads are still unresolved. The one on engine.test.ts:1058 is fair: that test makes one direct locate inside its budget and never drives a poll's last-read grace (time.test.ts covers that). Reply and resolve it, or drive it through a poll. The other three (time.ts running max, engine.mdx signal caveat, support.ts check() on a refused dismiss) are outdated: reply with your reasoning and resolve them.
  • The title still says "engine readLatency", but the final design is a last-read grace in ReadWindow with no readLatency field. Consider retitling.

This was an automated review.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

View guided diff | Re-trigger cubic

Comment thread packages/mobile/package.json
@Totopo27 Totopo27 changed the title fix(mobile): bound reads by operation budget with engine readLatency tolerance (#1099) fix(mobile): bound reads by operation budget with last-read grace (#1099) Oct 25, 2026
@okwasniewski

Copy link
Copy Markdown
Member

Thanks @Totopo27. Re-checked f9edf91 (a main merge on top of our f7f9584). Our peer floor (e2e >=0.21.4 <1), the changeset text and the locateSnapshot JSDoc all survived. Passing ReadWindow through the new readAllNow / resolveNow from #1354 is also right: toHaveCount and screenshot masks still read a missing frame as zero matches.

The merge dropped main's expect(calls).toMatchObject({ resolve: 1, read: 2 }); from "reads every current match in the order the engine resolved them" (packages/e2e/tests/unit/locator-engine.test.ts:343), which left calls unused and broke oxlint. I pushed the one-line restore as 39e3e99 (lint clean, 70/70 in that file) and approved the fork CI runs.

Nits

  1. packages/e2e/src/locator/engine.ts:262-266 and :300-304: resolveNow and readAllNow now take Deadline | ReadWindow and branch on instanceof. The only callers that pass a Deadline are tests. Take reads: ReadWindow with the default you have now, and use engine.reads(...) in the tests, as you already do in the rest of this file.
  2. Cubic threads: the one on packages/mobile/tests/unit/engine.test.ts:1099 is still open, and it's fair (one direct locate, no poll's last-read grace). Reply and resolve it, or drive it through a poll. The three outdated ones (time.ts, engine.mdx, support.ts) just need a reply and a resolve. The one on packages/mobile/package.json:68 is wrong: this PR's e2e patch changeset is what ships 0.21.4.

This was an automated review.

# Conflicts:
#	packages/e2e/tests/unit/locator-engine.test.ts
#	packages/mobile/src/surface.ts
@github-actions github-actions Bot added size:L Up to 499 changed lines, not counting tests, fixtures, or the lockfile and removed Ready for Human Review labels Oct 28, 2026
Only tests passed a Deadline; drop the instanceof branch.
@Totopo27 Totopo27 closed this by deleting the head repository Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for Human Review size:L Up to 499 changed lines, not counting tests, fixtures, or the lockfile

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bound mobile reads by the operation budget without turning a slow last poll into a timeout

2 participants