iOS Simulator: Maestro engine and agent-device is/get/alert disagree depending on the bound snapshot backend (Simulator AX vs XCTest) #2638

Activity

1. thymikee commented on Sep 15, 2026

   

   thymikee

   on Sep 15, 2026

   Member

   More actions

   Triage

   Reproduced and root-caused one class here; the out-of-process classes still need evidence.

   Root cause (fixed in #2643). The interactive fold suppresses an Other wrapper that carries only
   an identifier, delegating that identifier to the wrapper's content. A React Native testID wrapper
   has no content in the accessibility tree — the text lands beside it, not inside it, on both producers
   — so suppression left the identifier with no representative at all, and Maestro reads its visible set
   from that presentation while is/get resolve from the capture. What made it backend-dependent is
   that the suppression gate reads hittable === true: the Apple runner answers hittability for every
   node (true on screen, false off screen), and the Simulator AX bridge answers it for none. Same
   screen, same installed app, the two raw trees are node-for-node identical (165 nodes) and differ only
   by the presence of hittable.

   Suppression now needs the node's own declared hittable: false. After that change, snapshot -i over
   both producers publishes the same 18 identifier hooks, and the Maestro flow passes where it previously
   failed with Maestro visible condition did not match.

   Still open, and what would unblock it. Classes you list as PHPickerViewController and MapLibre
   annotation content are probably not this rule: that content may be owned by another process and
   therefore absent from an app-process tree, which no presentation change can recover. To tell those
   apart, please attach from one failing screen each, run twice (once bound to Simulator AX, once served
   by XCTest):

   agent-device snapshot --raw --json
   agent-device snapshot -i --json
   

   Alert.alert was re-checked against the same simulator today and resolves under Simulator AX, so if
   that class still fails for you the captures above will say why. The intermittent formSheet fields
   and the non-interactive leaf Text in a ListHeaderComponent may or may not be the fixed rule — the
   same captures settle it.
2. thymikee

   added

   needs-infoWaiting on reporter or external inputWaiting on reporter or external input

   on Sep 15, 2026
3. thymikee

   added a commit that references this issue on Sep 16, 2026

   fix(ios): keep an identifier hook the fold cannot prove inert (#2643)

   ...

   ca5528d
4. thymikee commented on Sep 16, 2026

   

   thymikee

   on Sep 16, 2026

   Member

   More actions

   Mechanism, confirmed in code (not just symptom)

   The is/get/alert-vs-Maestro split is not a visibility-predicate disagreement. isAssertionVisible (packages/selectors/src/internal/predicates.ts:120) and isMaestroNodeVisible (packages/maestro/src/internal/snapshot-policy.ts:8) are byte-identical, including the shared isUsefulVisibilityAnchor. The split is purely which node set reaches the matcher:

   • Maestro re-applies the interactive compaction to the snapshot it receives and resolves selectors against that compacted tree — daemon-runtime-port-observation.ts:107-113 calls buildIosInteractiveSnapshotPresentation(snapshot.nodes) and matches the result.
   • is/get/click resolve from the regular presentation (selector-capture-policy.ts: interactiveOnly: false), which never runs the semantic compaction fold (engine.ts:121 gates buildIosInteractiveSnapshotPresentation on interactiveOnly).

   So every interactive-fold rule that drops a node is a rule that can make Maestro fail to resolve a selector is/get still resolve from the same capture.

   Verified by experiment: reverting only collectIosStructuralIdentifierSuppression (#2643's rule) flips daemon-runtime-port-identifier-hook.test.ts from ok:true to ok:false, even though that test feeds the hook nodes straight to Maestro and never opens a simulator. The fold is Maestro's sole gate.

   Per class

   1. Childless testID wrapper (list-header Text, members.count) — root-caused and closed in fix(ios): keep an identifier hook the fold cannot prove inert #2643. Its hittable:false requirement removed the Simulator-AX-vs-XCTest split, because the leak only appeared on the producer that reports no hittability. This is the exact cross-backend dependency the report describes.
   2. Alert.alert — re-checked in fix(ios): keep an identifier hook the fold cannot prove inert #2643 triage; resolves under Simulator AX.
   3. PHPicker / MapLibre annotation content — out-of-process. That content is owned by another process and is absent from an app-process tree, so no presentation change recovers it. The only non-presentation remedy is registering the host in IOS_SYSTEM_SURFACE_HOSTS (packages/contracts/src/ios-system-surface.ts), which the snapshot route already consumes to fall back to XCTest (snapshot-route.ts systemSurfacePresent). That registry is deliberately closed and requires live evidence per host. Still need the two snapshot --raw --json / snapshot -i --json captures (once per backend) from one failing screen to add these correctly.

   The remaining fold leak (real, reachable, and NOT yet fixed) — decision needed

   #2643 hardened only the childless branch of collectIosStructuralIdentifierSuppression (noise-structural.ts). Its delegation branch still suppresses an identifier-carrying Other wrapper whenever it has any content, on the assumption that "the identifier goes on living in whatever the wrapper stood for" (ADR 0004). That assumption is unverified: the kept content is chosen by tree position, and ADR 0004's "moves an identifier" is only satisfied if the representative actually carries the identifier.

   Reachable shape — an Other wrapper with an identifier and non-identifier-bearing content, which is exactly the View testID=...-with-Text-children pattern already modelled by noise.test.ts's makeStructuralTree (wrapper identifier: 'wrapper-N', children StaticText with only labels):

   • Runner, on screen: wrapper is hittable:true, skipped at noise-structural.ts:14 → kept → id= resolves.
   • Simulator AX: no hittable reported, so line 14 does not skip it → suppressed, delegating to content that never carried the identifier → id= un-made → Maestro fails while is visible 'id=...' passes.

   That is the same backend-dependent split as the childless class, one branch over. It is currently encoded as intended by two tests (noise.test.ts: "suppresses structural identifier wrappers, keeping their content published" and "delegates an identifier wrapper to its content even without a hittability verdict"), which assert suppressed includes the identifier-bearing wrapper whose content does not carry that identifier.

   Closing it is a design call, not a bug sweep, and it cuts against those two deliberate tests, so it needs a maintainer decision and device verification rather than a guard:

   • (A) Delegation may suppress only when the kept content actually carries the identifier, else keep the wrapper; or
   • (B) literal ADR-0004: transfer the identifier onto the representative when the wrapper is suppressed (keeps the tree compact and the id addressable); or
   • (C) resolve it as part of iOS snapshot backend convergence and fast Simulator observation #2188/Selector queries have no fallback producer on the iOS Simulator, and any fallback needs producer equivalence #2274's single-representation convergence, which is where the "Maestro re-folds, is/get don't" split — the actual reason these two views can diverge at all — goes away.

   Leaning (C) for the split itself and (B) if a narrower slice is wanted first, since (A) only trades a lost identifier for an extra published node and neither (A) nor (B) removes Maestro's dependence on the presentation layer.
5. thymikee commented on Sep 16, 2026

   

   thymikee

   on Sep 16, 2026

   Member

   More actions

   Closing as a symptom tracker, on the model this issue was filed under ("filed as a concrete symptom for the convergence work in #2188 / #2274").

   • The reported symptom (Maestro failing where is/get pass, gated on the bound backend) for the RN testID case was root-caused and fixed in fix(ios): keep an identifier hook the fold cannot prove inert #2643. Alert.alert re-checks clean under Simulator AX.
   • The remaining classes live on in the durable trackers, not here:
     • Out-of-process surfaces (PHPicker, MapLibre annotation content) need host registration in IOS_SYSTEM_SURFACE_HOSTS behind live per-host evidence — the mechanism and gating are Selector queries have no fallback producer on the iOS Simulator, and any fallback needs producer equivalence #2274's producer-equivalence question; the two snapshot --raw --json / -i --json captures (one per backend) from a failing screen are what unblock it.
     • The "Maestro re-folds the interactive presentation, is/get resolve from the regular capture" split — the reason the two views can diverge at all — is iOS snapshot backend convergence and fast Simulator observation #2188's single-representation convergence.
     • The delegation-branch identifier leak and the design choice it implies (keep / transfer / converge) are written up in the preceding comment for whoever picks up the fold slice.

   Nothing here is dropped; it's just tracked where the durable outcome lives. If a failing screen reproduces that is NOT out-of-process and NOT the delegation branch, reopen with the captures above.
6. thymikee

   closed this as completedon Sep 16, 2026
7. thymikee

   added 2 commits that reference this issue on Sep 23, 2026

   fix(ios): derive hittable from geometric actionability on the AX bridge

   ...

   5a24030

   fix(ios): derive hittable from geometric actionability on the AX brid…

   ...

   ca2970b