Skip to content

fix(ios): cut modal-contained presentations from bridge snapshots - #2850

Merged
thymikee merged 2 commits into
mainfrom
fix/ios-modal-capture-parity
Sep 24, 2026
Merged

thymikee merged 2 commits into
mainfrom
fix/ios-modal-capture-parity

Conversation

@thymikee

@thymikee thymikee commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Summary

The host AX bridge reports every container hanging off a window, so each UIKit modal presentation left the screens it covers published as live content. Maestro's visibility test is geometric, so tapOn resolved the covered screen's control and flows failed on screens that never arrived; presentation: "formSheet" fields matched intermittently (item 3 of #2638, not closed by this).

The presentation cut now applies modal containment: when the last transition view under a container carries UIKit's dimming view as its direct child and that dimming view takes touches, the earlier transition views that dimmed area spans are cut, with their subtrees, from the regular and interactive projections. Containment is asserted only where the producer states it. An undimmed presentation behaves as before. A sheet resting at an undimmed detent also behaves as before: its dimming view reports userInteractionEnabled: false. snapshot --raw still reports covered screens. Producers that report no UIKit class names, or no dimming-view interaction state, never trigger the rule. Removals are counted in stats.modalContainedNodeCount (ADR 0026). ADR 0004 records the decision and the rejected seams.

The Simulator AX bridge (source version v1.7.0) reads XC_kAXXCAttributeIsUserInteractionEnabled for UIDimmingView nodes only, with one follow-up read for each dimming view. The decoder publishes it as RawSnapshotNode.userInteractionEnabled. Requesting the attribute for every node made captures about 50% slower on a 492-node tree. The targeted read did not change capture time.

The rule is in geometry-policy.ts, a module the fold already evaluates. The eager-closure budget keeps ios-snapshot-engine/index.ts at its merge-base size.

Validation

Head 31defc4c14, rebased on faa69ff784.

  • pnpm check:affected --run on this head: all runnable checks passed (format, lint, typecheck, layering, fallow, build, related Vitest: 860 files, 6739 tests).
  • Live, bridge-served, iPhone 17 Pro Simulator (iOS 26.2), react-navigation example:
    • Undimmed detent (rne://native-stack-form-sheet → Custom Dimming): dimming views report userInteractionEnabled: false. snapshot -i has all 10 presenting buttons plus the sheet's Close. Before this fix it had Close only. press 'label="Height Steps"' reached the presenting screen and opened that sheet.
    • Dimmed form sheet (Height Steps): userInteractionEnabled: true. -i shows the sheet only (5 nodes).
    • Card modal (native-stack-card-modal → Push article → Push albums): userInteractionEnabled: true. -i has 14 nodes, header Albums. The next Push article press reached Article by The Doctor.
  • 13 rule tests, including an undimmed-detent negative and an unread-fact negative. Mutants: dropping the userInteractionEnabled check kills 2 tests. Treating an unread fact as blocking kills 1 test. Decoder test for true, false, absent and malformed values.
  • Earlier evidence (40d72169c5): React Navigation's Maestro suite passed with 39 passed (39) on an iPhone 17 Simulator (iOS 26.2). Five hand-applied mutants were each killed by a named test.
  • No mutation-lane module covers geometry-policy.ts. None was added in this PR.
  • Risk: when UIKit does not dim a presentation, or the bridge cannot read the dimming view's interaction state, the covered screens are still published.

@thymikee
thymikee force-pushed the fix/ios-modal-capture-parity branch from 9b8fc13 to 29d4208 Compare September 23, 2026 18:14
@github-actions

github-actions Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.80 MB 4.80 MB +3.8 kB
Package (unpacked) 4.79 MB 4.80 MB +3.8 kB
Package (download) 1.43 MB 1.44 MB +823 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 25.5 ms 26.3 ms +0.8 ms
CLI --help 71.4 ms 74.2 ms +2.8 ms

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed at 29d4208. The rule looks right for dimmed sheets, but there is no live evidence yet that it leaves a reachable presenting screen alone.

Does findCoveringPresentation cut a screen the user can still reach when a formSheet sits at an undimmed detent? It treats any dimming view with a positive rect as proof of modality. If UIKit keeps a uidimmingview in the tree at that detent, the presenting screen's controls would disappear from the snapshot. Please attach a bridge capture of that case: snapshot --raw (is a uidimmingview a direct child of the last UITransitionView?) and snapshot -i (are the presenting controls still there?). The 39-flow run only covers dimmed card and modal sheets. If the controls are cut, the rule should key on a fact about whether the dimming view blocks input (alpha, hidden, or accessibilityViewIsModal), not on the class name alone.

Not blocking: collectSubtreeIndexes in tree.ts is close to collectSubtreeByParentLinks in noise-structural.ts; could one walker do both? A negative fixture for the undimmed-detent shape would also help.

CI was still running at review time. Mutants (ios-modal-containment), Repo Guards and Typecheck cover the new module and the mutation config, so a failure there would count against this diff.

@thymikee
thymikee force-pushed the fix/ios-modal-capture-parity branch 3 times, most recently from 2074bc4 to 40d7216 Compare September 23, 2026 19:30
@thymikee

Copy link
Copy Markdown
Member Author

Answering the live-evidence finding at 40d72169c5 (rebased onto b3b58caac1). All three probes are snapshot -i --json / snapshot --raw --json against rne://native-stack-card-modal on the iPhone 17 Simulator (iOS 26.2). Every screen in this example renders the same Push article / Push albums / Go back trio, so the header text is what says which screen a tap would reach.

1. An undimmed screen is left alone. Root: raw=85, transitionViews=1, dimmingViews=0. One card pushed over another: raw=86, transitionViews=1, dimmingViews=0. No dimming view means no candidate, so both states publish as they did before this change — interactive=12 and interactive=13, all four buttons (Go back, Push article, Push albums, BackButton) present, header Article by Dalek at depth.

2. A dimmed presentation is cut, measured against the same state with the rule disabled. Sequence Push article → Push albums reaches one state on both builds: raw=495, transitionViews=2, dimmingViews=2, the covered container at y=0 h=874 inside the live dimming view's y=-874 h=2622.

  • rule disabled (dist patched locally, not shipped): interactive=26, headers ['Article by Dalek', 'Albums'] — the covered screen's three buttons publish next to the modal's. The next press text="Push article" resolved that covered copy: the capture afterwards still reports ['Article by Dalek', 'Albums'], so nothing navigated. That is the Maestro failure, reproduced on device.
  • rule enabled: interactive=14, header ['Albums'] only. The same press reached the top screen and the capture afterwards reports Article by The Doctor.

The covering container is never a candidate — the loop cuts siblings.slice(0, -1) — which is why the modal's own 14 nodes survive while the covered 26-node pair does not.

3. Dismissal restores reachability, so the cut is per-capture and not destructive. From the modal: back → interactive=14, header Albums, unchanged after a 15 s wait. Back again → interactive=12, header Article by Dalek, unchanged after a 15 s wait. The screen the user can reach is published the moment UIKit hands it back.

The PR's risk line is now measured rather than assumed: the modals in this app all carry a dimming view, so the "presentation UIKit does not dim" case is the untouched path in item 1, not a live gap I hit.

@thymikee

Copy link
Copy Markdown
Member Author

40d7216 does not add the evidence the earlier review (29d4208) asked for, so this is still evidence-pending. All 18 checks are green at 40d7216, and there is no failing job to attribute.

The open question is whether the cut can fire on a sheet resting at an undimmed detent (largestUndimmedDetentIdentifier, or react-native-screens sheetLargestUndimmedDetentIndex) while the presenting screen stays reachable. findCoveringPresentation (https://github.com/callstack/agent-device/blob/40d7216/packages/capture-kit/src/ios-snapshot-engine/geometry-policy.ts#L276) treats any direct-child uidimmingview with a positive rect as proof the presentation blocks input, but at an undimmed detent that view can sit in the transition view at alpha 0 and let touches through, so isDimmedByPresentation could cut screens the user can still reach. The current evidence covers the dimmed modal and screens with no dimming view, and does not reach this state, so the rule's mechanism is plausible but unconfirmed. Can you capture a sheet at an undimmed detent on the local Simulator bridge (for example a formSheet with sheetLargestUndimmedDetentIndex set), run 40d7216, and show snapshot --raw --json (is a uidimmingview a direct child of the last UITransitionView, and what is its rect?) alongside snapshot -i --json (are the presenting screen's controls still present)? If a press on one of those controls reaches the presenting screen, the cut needs to key off a fact the bridge reports for input blocking, such as alpha, hidden, or accessibilityViewIsModal on the presentation, rather than rect presence alone, with a negative fixture of that shape added to geometry-policy.test.ts.

Separately, the mutation lane that 29d4208 added for this rule is gone from the rebased head now that the rule lives in geometry-policy.ts (https://github.com/callstack/agent-device/blob/40d7216/scripts/mutation/modules.ts#L1). Is this rule covered by an existing mutation module, or does one need to be added so a regression here is caught?

Not blocking: whether an existing mutation module already covers geometry-policy.ts can be checked and folded in later if it's missing.

I could not run the device check myself, so the dimmed-modal cut and the press-navigation result are author-reported. The next thing standing between this and merge is a live bridge capture of a sheet at an undimmed detent showing the presenting screen's controls survive snapshot -i, or the rule gated on an input-blocking fact instead of rect presence.

The host AX bridge reports every sibling container in a window, so each
UIKit modal presentation left the screens it presents over published as
live content. A sheet covers only part of the screen, so the geometric
occlusion pass left the covered controls actionable and selectors resolved
them: a three-screen stack published 567 nodes across three screens where
the same app under the XCTest runner published 76 across the top one.

The presentation cut now applies modal containment: when the last
transition view under a container carries UIKit's dimming view as its
direct child, the earlier transition views that dimmed area spans, and
their subtrees, are cut from the regular and interactive snapshots. Raw
snapshots still report them, and producers that report no UIKit class
names never trigger the rule, so it stays inert on the runner. Every
removed source is counted in the fold's presentation stats (ADR 0026).

The rule joins `geometry-policy.ts`, the module the fold already evaluates
for its per-node decisions, rather than arriving as a new eager module: the
eager-closure budget holds this entry surface at its merge-base size.
A sheet resting at an undimmed detent keeps UIKit's dimming view as a
direct child of its transition view, at the same window-sized frame, while
the presenting screen stays reachable. The modal-containment rule treated
any such dimming view as proof of modality, so the bridge snapshot cut the
presenting screen's controls: on react-navigation's form-sheet example,
`snapshot -i` over the `Custom Dimming` sheet published only the sheet,
although a press on the presenting `Height Steps` button navigates.

The Simulator AX bridge now reads `XC_kAXXCAttributeIsUserInteractionEnabled`
for dimming views (source version v1.7.0) and the decoder publishes it as
`userInteractionEnabled`. It reads false for the undimmed detent and true
for a dimmed form sheet and a card modal; no attribute the bridge already
read tells them apart, and `IsVisible` reads false for both the undimmed
sheet and the card modal. The read targets dimming views alone: requesting
the attribute for every node cost about half again the capture time on a
492-node tree. The rule now asserts containment only when the dimming view
takes touches, and fails closed when the producer did not read it.

The modal rule and the structural identifier rule now share one
parent-link subtree walker in tree.ts.
@thymikee
thymikee force-pushed the fix/ios-modal-capture-parity branch from 40d7216 to 31defc4 Compare September 24, 2026 07:39
@thymikee

Copy link
Copy Markdown
Member Author

Addressed at 31defc4c14. The branch is rebased on faa69ff784. The only conflict was in CHANGELOG.md, and both entries are kept.

1. Undimmed detent (blocking). Confirmed, and fixed. I captured the case on the bridge: iPhone 17 Pro, iOS 26.2, rne://native-stack-form-sheet → Custom Dimming (sheetLargestUndimmedDetentIndex: 0). A UIDimmingView is a direct child of the last UITransitionView at {x:-402, y:-874, w:1206, h:2622}. That is the same rect as in the dimmed case. At 40d72169c5, snapshot -i published only the sheet's Close button. But a tap on the presenting screen's Height Steps button went through the sheet and opened that sheet. So the old rule cut controls the user can reach.

I probed extra AX attributes on the dimming view in three states:

state IsUserInteractionEnabled IsVisible
undimmed detent false false
dimmed form sheet true true
card modal true false

IsVisible does not separate the states. IsUserInteractionEnabled does. The bridge (source v1.7.0) now reads that attribute for UIDimmingView nodes only, with one follow-up read per dimming view. The decoder publishes it as userInteractionEnabled. The rule cuts only when the value is true. When the value is absent, the rule does not cut.

Cost on the 492-node card-modal tree, 8 captures each:

  • baseline: 462–654 ms
  • attribute requested on every node: 537–1055 ms
  • targeted read: 353–477 ms

Live results at the new head:

  • Undimmed detent: userInteractionEnabled: false. -i has all 10 presenting buttons plus Close. press 'label="Height Steps"' reached the presenting screen.
  • Dimmed form sheet: true. -i shows the sheet only (5 nodes).
  • Card modal: true. -i has 14 nodes, header Albums. The next Push article press reached Article by The Doctor.

2. Negative fixture (non-blocking). Added to geometry-policy.test.ts: an undimmed-detent negative and an unread-fact negative. Mutants: dropping the interaction check fails 2 tests. Treating an absent value as blocking fails 1 test. A decoder test in tree.test.ts covers true, false, absent and malformed values.

3. One walker (non-blocking). Done. collectSubtreeByParentLinks moved to tree.ts. The modal rule and noise-structural.ts both use it, and collectSubtreeIndexes is removed.

4. Mutation lane. No scripts/mutation/modules.ts module covers geometry-policy.ts. Membership is for one-file pure kernels, and geometry-policy.ts also holds the fold's traversal policy. A new module for it is not in this PR. I can add one if you want the lane.

Validation. pnpm check:affected --run on 31defc4c14: all runnable checks passed (860 test files, 6739 tests). I made the live run before the walker move, with the same native code and rule. The walker move is covered by the unit tests.

@thymikee

Copy link
Copy Markdown
Member Author

Reviewed at 31defc4. The earlier conflict is resolved, and the code correctly cuts modal-contained presentations from bridge snapshots. All checks pass and there are no conflicts.

The live bridge readings (undimmed detent, dimmed form sheet, card modal) and the Height Steps and Push article results are author-reported, from a run before the walker move. The move reads as behavior-equivalent, so that run still covers the rule. One question: does every UIKit dimming view on iOS 26 report the element type exactly UIDimmingView? A subclass with a different name would skip the cut.

Not blocking: the CHANGELOG's 39-flow Maestro count comes from the 40d7216 run, and the rule was narrowed since. Either re-run the suite at this head or attribute the count to 40d7216.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 24, 2026
@thymikee
thymikee merged commit 7ddaff1 into main Sep 24, 2026
18 checks passed
@thymikee
thymikee deleted the fix/ios-modal-capture-parity branch September 24, 2026 09:07
@github-actions

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-24 09:08 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant