fix(ios): keep alert activation out of XCTest's interruption handling - #2843
Merged
Merged
Conversation
Size Report
Startup median (7 runs, lower is better):
|
Before every event XCTest looks for SpringBoard banners and alerts over the target and runs its interruption handler on them. For a notification banner the default handler waits up to 15 s for it to leave, so an `alert accept` issued while a banner was visible tapped after the command's deadline and answered ALERT_DEADLINE_EXCEEDED for a button it had in fact pressed (#2546's late tap, from the banner side). Four iOS CI failures of the alert replacement tests carry this trace, including main job 106127169259: banner wait from t=7.98 s, tap at t=15.12 s on a ~15 s deadline. `withBoundedInteractionIdleTimeoutIfSupported(.preEventSkipped)` drops only the idle wait. Alert activation now also sets XCUIApplication's `doesNotHandleUIInterruptions` on the alert's owner app for the one tap. It keeps the XCTest coordinate tap and its post-event settle, which alert verification relies on; the synthesized-event path would have dropped that settle too. The runner host app gains a `--agent-device-alert-banner` fixture that keeps a notification banner over its alert until the alert is answered. The new test fails on the old dispatch with the CI signature (interruption monitors consulted, ALERT_DEADLINE_EXCEEDED after the button was activated) and passes with the opt-out.
…ures assert on Nine iOS CI failures of the alert replacement fixtures answered ALERT_DEADLINE_EXCEEDED with "No alert button was activated": the command deadline also covered resolution, about two dozen accessibility reads before any tap, and a contended hosted simulator spent it all there (job 106268275388 sampled a 0.823 s read, sized a 39.5 s deadline, then spent 39 s resolving). The latency-sampled deadline from #2708 cannot predict a host that slows down mid-command. The fixtures now resolve the alert first under a 60 s allowance and hand it to the command through `alertResolutionOverrideForTesting`, so `timeoutMs` bounds only activation and verification, which is what they prove: the replacement is never activated. That window gets a fixed 30 s. A confirmed answer returns as soon as verification sees the replacement; only the unprovable-replacement fixture pays it in full. The latency sampler and its decomposition test are removed. The busy-app fixture asserted wall-clock success under a fixed 6 s deadline. It now records whether the button was answered while its animation was still in flight, and the test asserts that instead. Reverting the seam to a plain `activateElement` fails it with "Answered after the app went idle".
…t budget The fixture's fixed 20 s backstop could stop the in-flight animation while a contended host was still resolving the alert under the 60 s allowance, so a correct activation read "Answered after the app went idle". The test now passes the backstop as resolution allowance plus activation deadline.
thymikee
force-pushed
the
fix/ios-alert-activation-interruption-monitor
branch
from
September 23, 2026 17:31
61e6976 to
5b1bdd5
Compare
Member
Author
|
This is ready at 5b1bdd5. Not blocking: All 16 checks are still queued or in progress on this commit, and none have reported a failure yet. iOS Smoke Tests is the one to watch here, since it runs the targeted runner XCTest selection this PR changes, plus the |
thymikee
added
the
ready-for-human
|
This was referenced Sep 23, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the alert family in "Run targeted iOS runner XCTest regressions" (13 failures 09-15..23, 2 on main).
ALERT_DEADLINE_EXCEEDEDfor a button it had pressed. Alert activation now setsXCUIApplication.doesNotHandleUIInterruptionson the alert's owner for that one tap. It keeps the XCTest tap and its post-event settle, which verification needs. The synthesized-event path would drop that settle.alertResolutionOverrideForTesting. The deadline then covers only activation and verification, fixed at 30 s. test(ios): size the alert command's deadline to the host's own reads #2708's latency sampler is removed. The busy-app test now asserts a fixture witness ("Answered while busy") instead of wall-clock time.5 files; alert family only.
alertis not aninteraction-guarantees.tspath.Validation
Tested SHA
0bca57c4fe, local iOS 26.2 sim,AGENT_DEVICE_XCUITEST_INCLUDE_UNIT_TESTS=1:New
testAlertActivationDoesNotWaitOutANotificationBannerwith the fixture's live banner. On the old dispatch it fails with the CI signature: the monitor is consulted 10×, thenALERT_DEADLINE_EXCEEDED, "The button was activated once". With the fix it passes 5/5, and the log shows "Application is configured to skip UI interruption handling".All 10 alert tests ×
-test-iterations 5: 50/50 passed.Mutation: reverting the idle seam to a plain
activateElementfails the busy-app test ("Answered after the app went idle").pnpm check:xctest-selectionandpnpm check:affected --runpassed.5b1bdd5(review follow-up, rebased on main): the busy fixture's backstop is now the test's resolution allowance + activation deadline, not a fixed 20 s, so a slow resolution cannot end the busy state first. 3 alert tests × 3 iterations: 9/9 passed;check:affected --runpassed.Risk: private XCTest KVC, guarded by
responds(to:).