Skip to content

iOS selector click taps a point resolved from a pre-tap capture, so a layout shift between capture and tap retargets it silently #3354

Closed

Description

@thymikee

Purpose

An iOS selector click taps the point it resolved from its own pre-tap snapshot. If the layout moves between that capture and the tap, the tap lands on whatever is now under the old point, and the click still reports success. Nothing re-checks where the target is at dispatch time.

Evidence

Main run 37815426076 (2026-10-08, iOS 26 lane), 01-settings.ad. Times are from the session runner.log:

open            postOpenObservation: probe-failed  -> fixed POST_OPEN_SETTLE_MS sleep   (corrected: originally misread as unobservable)
17:41:31.77 COMMAND_ACCEPTED  command=snapshot        (click's pre-tap capture)
17:41:34.57 COMMAND_COMPLETED command=snapshot ok=1   (2.80 s capture)
17:41:35.56 SYNTHESIZED_DISPATCH kind=tap point=(201.0,319.0)   attempt 1 -> following wait failed
17:42:10.96 SYNTHESIZED_DISPATCH kind=tap point=(201.0,406.0)   attempt 2 -> passed

General was at y=319 in a capture taken about 0.5 s after open returned. On the settled root list it is at y=406, 87 pt lower. Attempt 1's tap probably hit the Apple Account band.

This alone does not prove the tap missed. In #2936 a failing first attempt tapped the settled (201,406) and its wait still failed, so a stale point is one cause among others. Diagnostic history: #2948.

Where it happens

  • settleAppleOpen (packages/platform-apple/src/open-policy.ts): when the bridge can't read the app, the open falls back to a fixed sleep. When the bridge does read it, observable only means the tree was readable, not that the layout is stable.
  • The runtime-selector path takes its tap point from the rect in the pre-tap capture. Neither the offscreen/nonHittable checks nor anything in the runner confirm that the node is still at that point when the tap goes out.

Required behavior

  • Choose one owning fix and justify it against the evidence. Candidates:
    • re-resolve or re-check the target's frame in the runner right before tapping;
    • require the target's frame to be stable across captures before tapping;
    • gate the first post-open interaction on the layout settling.
  • A bounded check must have a budget shorter than the click itself and must not add a capture to warm clicks whose layout is already stable.
  • A target that moved and can't be confirmed must fail with a typed reason, not tap the old point.
  • Update the guarantee cell this changes in packages/contracts/src/interaction-guarantees.ts.

Completion

  • A fixture or live run where the target moves between capture and tap either taps the new location or fails with a typed reason.
  • Warm click p50 on 01-settings.ad (about 2.6 s locally) does not regress.

Related: #3335 (click reports dispatch, not landing), #2948, #2491/#3328 (wait observation stall).

Scope after #3356

This issue was rescoped when #3356 merged. It now covers best-effort settling of the next screen read after an iOS Simulator open that returns timing.postOpenObservation: "unobservable". It was the third candidate listed under Required behavior. Correction: the evidence run's open actually reported probe-failed, not unobservable, so #3356 does not cover it. See the correction comment below.

These outcomes are accepted for this scope:

  • If the screen never settles, the click proceeds. Its response carries postGestureOutcome: unsettled and a warning; it does not fail with a typed reason.
  • The first click after such an open costs about 1.8 s more. On an iOS 27.0 simulator it took 3.12 s against 1.30 s, because the two settle reads took 2.55–2.57 s. This breaks the "budget shorter than the click" rule above. Warm clicks gain no capture.
  • Any screen read between the open and the click uses up the settle.

These parts move to #3367:

  • shifts between a click's own capture and its tap
  • shifts after observable opens
  • the typed failure for a target that moved
  • a wall-clock deadline on the wait

Activity

  1. thymikee commented on Oct 10, 2026

    @thymikee
    MemberAuthor

    Completed by #3356 within the rescoped contract in the issue body under "Scope after #3356". The remaining parts of the original contract are tracked in #3367.

  2. thymikee commented on Oct 10, 2026

    @thymikee
    MemberAuthor

    Correction. The evidence above misread the open verdict. In main run 37815426076, both attempts' replay-timing.ndjson report postOpenObservation: "probe-failed", not "unobservable". #3356 marks only unobservable opens, so it would not have changed that run.

    A scan of 30 CI artifacts from 2026-10-03 to 10-10 found:

    • 61 iOS opens: 54 probe-failed, 7 observable, and 0 unobservable.
    • 17 stale-looking taps across 14 runs. Each tapped y=319 on a failed attempt, and a later attempt in the same run tapped y=406 and passed. Every one followed a probe-failed open.

    open already uses a fixed delay for every verdict other than observable on a local iOS Simulator (packages/platform-apple/src/open-policy.ts). The follow-up PR makes the post-open settle use the same rule. The rest of the evidence is on #3367. I'm leaving this issue closed and tracking the fix there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions