Repository navigation
spike(ios): can the Simulator AX bridge guest read the app's interface orientation? #2659
Description
Why
PR #2653 (fixes #2612) made the XCTest runner publish every rect under a rotated system surface (the landscape keyboard's UIRemoteKeyboardWindow) in the app's orientation space. The Simulator AX bridge could not do the same, because "the bridge reader carries no orientation fact", so the bridge refuses such a capture (window-coordinate-space-unresolved) and the route pays one XCTest round trip for it. To make that refusal possible the PR added a TypeScript twin of the detection rule (packages/platform-apple/src/snapshot-source/window-coordinate-space.ts), a golden parity table (contracts/fixtures/window-coordinate-space.json) replayed in both languages, a decoder counter (countUnresolvedCoordinateSpaceWindows in snapshot-source/tree.ts), a failure code, a route warning string, and an ADR paragraph.
All of that exists only because nobody has checked whether the bridge guest can read the app's interface orientation. The claim in the ADR is about the attribute set the bridge requests today (apple/snapshot-bridge/SnapshotBridgeRuntime.m lines 28–36: element type, base type, label, value, identifier, frame, automation type, traits, children), not about what the guest process can observe. If the guest can learn the orientation, the bridge normalizes with the same SnapshotGeometrySpace rule the runner uses and the whole detection-only layer above is deleted.
Task
A time-boxed spike (half a day). Answer one question with measurements: can the Simulator AX bridge guest learn the foreground app's UIInterfaceOrientation (1 portrait, 2 upside-down, 3 landscape-right, 4 landscape-left) cheaply and reliably at capture time?
Where the bridge lives and how it is run:
- Guest binary:
apple/snapshot-bridge/*.m(ObjC, ~900 lines). It is spawned inside the Simulator withxcrun simctl spawn <udid> <bin> …bypackages/platform-apple/src/snapshot-source/host.ts/lifecycle.ts, and answers length-prefixed JSON requests. Attribute names are resolved throughXCAXAccessibilityAttributesForStringAttributes(SnapshotBridgeRuntime.m~line 152). - The decoder that consumes its tree:
packages/platform-apple/src/snapshot-source/tree.ts. - The runner reads orientation through a private
XCUIApplicationselector:RunnerSynthesizedGesture.interfaceOrientationForApplication:inapple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerSynthesizedGesture.m(~line 217). That is the value the rotation table switches on.
Candidate routes to try, in order of cheapness (none is verified; the point of the spike is to find out):
- An AX attribute in the same private vocabulary the bridge already uses. Enumerate what
XCAXAccessibilityAttributesForStringAttributesaccepts (dump the string table ofXCTAutomationSupport.framework/XCTestCore.frameworkin the iPhoneSimulator platform and grep forOrientation); if anXC_kAXXCAttribute…Orientation-style name resolves, request it on the application element. - Inference from the tree the bridge already has: with the keyboard up in landscape the keyboard window's native-space x-range sits near 0 for landscape-right and near
appWidth − bandHeightfor landscape-left (a docked keyboard is always at the bottom in the app's space). This is an assumption about docked keyboards, not a fact about arbitrary surfaces; report it as a fallback only if route 1 and 3 fail, and say what it cannot cover. - A SpringBoard/UIKit service call available to a guest process (
SBSGetInterfaceOrientationForDisplayor similar inSpringBoardServices;AXPTranslator/AXRuntimeorientation accessors). Check symbol availability withnm/dlsyminside the guest before writing code against it.
Measure on iPhone 17 Pro (iOS 26.2 Simulator) with the test app (examples/test-app) in portrait, landscape-left and landscape-right, keyboard up and down: does the value match what the runner's selector returns, and what does the read cost per capture (the bridge's warm read is 40–120 ms today, see ADR 0004; anything that adds a Mach round trip per capture must be measured, not guessed).
Deliverable
A comment on this issue (or a short doc under docs/) with: the route that works or the evidence that none does, the measured values per orientation, the per-capture cost, and a one-paragraph follow-up plan. If a route works, the follow-up plan is: bridge sends interfaceOrientation in its envelope, tree.ts rotates through the same table (port CoordinateSpaceRotation.oriented(rect:) to TS, pinned by the existing rotationCases in the fixture), and the following are deleted: isQuarterTurnedWindowFrame as a refusal signal, unresolvedCoordinateSpaceWindows on SnapshotSourceDecodedTree, the window-coordinate-space-unresolved failure and its route warning, and the "A producer that cannot name the app's interface orientation refuses the screen" decision in ADR 0004.
Acceptance criteria
- Each of the three routes has a recorded outcome (works / does not exist / exists but unreliable), with the command or code used to check it.
- For a working route: values recorded for portrait, landscape-left, landscape-right, matching the runner's
interfaceOrientationForApplication:on the same screen; per-capture cost measured over at least 10 warm reads. - Explicit statement of what the spike did not cover (physical devices are out of scope: the bridge is Simulator-only).
- No production code merged from the spike itself; throwaway scripts go under
scripts/only if they are reusable, otherwise attach them to the issue.
Non-goals
Do not change the runner's rotation table, the golden fixture, or the route in this issue. Do not re-open the two-presenter decision (see docs/adr/0004-ios-snapshot-backend-strategy.md, host-side ownership boundary).
Related: #2612, #2653, ADR 0004 amendment "the coordinate space of a captured subtree", ADR 0011 layer 2.
Activity
thymikee commentedon Sep 18, 2026
Verdict: no
Measured 2026-09-18 on iPhone 17 Pro (iOS 26.2 Simulator) with throwaway guests compiled like the bridge and spawned with
xcrun simctl spawn. Full write-up with method, tables and the probe source is the diff of #2667 (closed unmerged on purpose; the diff stays readable there).XC_kAXXCAttributeApplicationOrientationexists inXCTAutomationSupportand resolves to id 1503 through the bridge's own resolver. Read throughuserTestingSnapshotForElement:options:error:(the bridge's call) it returns 0 on the app root and every window in portrait, landscape-left and landscape-right, while the sibling frame attribute in the same snapshot is correct. XCTest's ownappOrientationForElement:error:fails from a guest withkAXErrorServerNotFound. The value is served by XCTest's in-process agent, which only a liveXCUIApplicationsession has.BKHIDServicesGetCurrentDeviceOrientationworks and costs ~0.015 ms, but it is device orientation. Rotation-locked Settings foreground, device turned landscape-left: the read says 3 while the app's root window stays(0,0,402,874). Feeding that into the rotation table would turn a screen that never turned. SpringBoard'sSBGetInterfaceOrientationreturns a fixed0x10000003in every orientation.UIRemoteKeyboardWindowat(0,0,402,874)beside the app window at(0,0,874,402)), but it is a docked-keyboard assumption, blind with no keyboard up, and cannot name an arbitrary surface or tell portrait from upside-down.Consequence. The detect-and-refuse layer from #2653 and the ADR 0004 decision "a producer that cannot name the app's interface orientation refuses the screen" stand, now backed by measurement. The delete-the-layer plan is not unlocked.
The bar for a future attempt. A guest read that (a) survives a rotation-locked foreground app and (b) names the foreground app's interface orientation, not the device's. Attribute 1503 is the handle to watch: if a future bridge channel ever populates it,
tree.tscan portCoordinateSpaceRotation.oriented(rect:in:interfaceOrientation:), pin it on the fixture'srotationCases, and retireisQuarterTurnedWindowFrame,unresolvedCoordinateSpaceWindows,window-coordinate-space-unresolvedand the ADR refusal decision.Not covered: physical devices (the bridge is Simulator-only); the runner's own orientation integer was not read from the host side, interface-space truth came from captured window geometry and the Settings divergence.
ADR 0004 and the
window-coordinate-space.tsheader get a one-line pointer to this comment in #2664.thymikee commentedon Sep 18, 2026
Answered: no cheap, reliable interface-orientation read exists on the bridge guest. See the verdict comment above; write-up in the diff of #2667.
docs(ios): point the bridge refusal decision at the #2659 spike verdict
refactor(ios): keep only the coordinate-space package move; drop scop…
refactor(ios): move the coordinate-space rule into the presentation p…