Skip to content

Gesture viewport should expose content-safe bounds (app window minus system bars) so scroll planning clamps to real geometry #1821

Open

Description

@thymikee

Follow-up from #1820 (#1781 A1), asked for in its review.

Today

buildScrollGesturePlan (packages/contracts/src/scroll-gesture.ts) keeps scroll gestures out of the outer DEFAULT_EDGE_PADDING_FRACTION = 10 % of each axis. That fraction is a proxy for the real invariant — a scroll must never touch down inside a system bar — tuned to exceed the tallest known bar (Pixel 7 cutout status bar 136/2400 = 5.7 %, iPhone Dynamic Island 59/852 = 6.9 %). It lives in a device-agnostic pure planner because the viewport it receives is the raw application window, which on edge-to-edge apps (API 35+, every system app, iOS) starts at y=0 and includes the bars. #1820 raised it from 5 % after scroll up 3 on a Pixel 7-geometry emulator touched down at y=120, inside the bar, and pulled the notification shade instead of scrolling.

Costs of the proxy: every user pays 10 points of max single-gesture travel (90 % → 80 % for amount ≥ 0.8) whether or not their device has a cutout, and a device with a taller bar than the tuned constant reintroduces the shade pull silently.

Proposed

The layer that knows where the bars are should say so:

  • Android — GestureViewportReader.read() (android/snapshot-helper/.../GestureViewportReader.java) already iterates UiAutomation.getWindows() and keeps only TYPE_APPLICATION; the StatusBar / NavigationBar system windows are right there to subtract (or read the status/navigation-bar insets). Expose content-safe bounds alongside the raw window bounds in the viewport action (helper protocol/API version bump).
  • Apple — the runner has safe-area insets; report the same content-safe rect in the resolved scroll frame.
  • Planner — clamp scroll travel to the content-safe rect and return DEFAULT_EDGE_PADDING_FRACTION to a small generic margin (the original 5 % or less). Keep the raw window rect for absolute swipe/gesture validation so an agent can still deliberately swipe down from the status bar.
  • Parity table contracts/fixtures/scroll-gesture.json (added in fix(scroll): keep saturated scroll gestures out of the status bar; gate Android replays from android/emulator (#1781 A1) #1820) is where the new vectors go — both the TS and Swift suites iterate it.

Also on this issue: suite discovery order

The masking in #1781 A1 came from discoverReplaySourcePaths (src/daemon/replay-source-discovery.ts) enumerating directory inputs in native readdir order (03,05,06,…,04 on the ubuntu runner vs 01…06 locally). The header documents this as deliberate Maestro parity, and it is accurate — Maestro's WorkspaceExecutionPlanner uses Files.walk and names the result unsortedFlowFiles, no sort. Sorting directory inputs would make results reproducible across machines but is a deliberate departure from that parity; decide it here rather than silently. Order-sensitive tests currently pass explicit positionals (see session-test-suite.test.ts).

Activity

  1. thymikee commented on Aug 25, 2026

    @thymikee
    MemberAuthor

    Triage: the content-safe viewport problem is valid, but this is not AFK-ready while it also owns the unrelated replay discovery-order decision. Please split discovery order into its own issue and choose the first evidence-backed platform slice for safe bounds; then the geometry work can be labeled ready-for-agent with cross-language fixture parity kept explicit.

  2. thymikee commented on Oct 9, 2026

    @thymikee
    MemberAuthor

    Decision: accepted as an Android safe-bounds slice, coordinated with #2714.

    GestureViewportReader.java reads the application and keyboard rects but not system-window geometry (status and navigation bars), and scroll-gesture.ts still pads by a flat 10% (DEFAULT_EDGE_PADDING_FRACTION). #3210 published the snapshot's coordinate extent, not safe bounds.

    Shared geometry rule with #2714: swipe endpoints stay inside the intersection of the resolved scroll container and the measured safe bounds. When that shortens the swipe, report the distance actually dispatched instead of the distance requested. Keep the conservative padding wherever safe bounds weren't measured. Split out the replay-ordering part asked for in August as its own issue if it's still wanted.

  3. added and removed
    needs-infoWaiting on reporter or external input
    on Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions