Repository navigation
CI overhaul (1/9): route iOS 26 device-contract pointer scenarios through Detox #65
Description
Part of the CI overhaul execution plan (#59, map #53). Order: 1 of 9 — lands first; it fixes the pipeline's dominant failure before anything else builds on candidate green streaks.
Decision: #60 (diagnosis in #54).
Scope
Route every pointer scenario in the ios26.auto-fallback device-contract configuration through Detox instead of Agent Device. Scenario definitions, terminal public-outcome assertions, and the four-engine parity comparison stay unchanged — only the pointer harness route changes. Do not implement the Agent Device input-readiness probe, widen gesture timings, or add scenario retries.
Resulting harness ownership: iOS 27 native and iOS 27 forced-fallback keep Agent Device for pointer scenarios (multi-selection on ios27.native stays Detox); iOS 26 automatic and Android use Detox throughout.
Files
e2e/contracts/pointer-drivers.json— delete the fiveios26.auto-fallbackoverrides (free-form-reorder,virtualized-list-reorder,section-list-reorder,multi-selection-reorder,scoped-drop) so those scenarios fall through to thedefaultDetox driver.scripts/run-device-contract-isolated.mjsalready skips the Agent Device runner preflight lazily (driver === 'agent-device' && !agentDevicePrepared), so no script change is needed.docs/adr/0006-automated-publication-confidence.md— append an amendment recording the per-configuration harness ownership table and the rationale: the first pointer gesture synthesized by the Agent Device XCTest runner into a freshly erased, cold-booted iOS 26.5 simulator on hosted runners arrives seconds late or never while injection reports success (Diagnose the iOS 26 auto-fallback free-form-reorder replay divergence #54); the iOS 26 cell's unique responsibility is proving automatic engine selection picks fallback, which Detox pointer synthesis proves equally.
Ruleset / required-check changes
None.
Acceptance criteria
- All
ios26.auto-fallbackscenarios listdriver: detoxin the resolved plan; no Agent Device daemon/runner is started in that job. - ADR 0006 amendment appended, matching the style of the 2026-09-01/2026-09-02 amendments.
- Five consecutive successful
ios26.auto-fallbackcontract runs on main/nightly, each with the four-engine parity job passing and no pointer-test retry — the acceptance streak decided in Decide the iOS 26 device-contract input-readiness fix #60. The task closes when the streak completes.
Activity
thiagobrez commentedon Sep 4, 2026
AC1's second clause isn't satisfiable through
pointer-drivers.jsonImplemented in b14997c. The first clause holds — resolving the plan the way
scripts/run-device-contract-isolated.mjs:89does now yieldsdriver: detoxfor all fiveios26.auto-fallbackpointer scenarios, and no Agent Device pointer runner or preflight starts for them.The second clause — "no Agent Device daemon/runner is started in that job" — can't be met by the pointer route alone. Two non-pointer scenarios on that job shell out to Agent Device from inside the Detox run, independent of any pointer driver:
e2e/contracts/portable-contract.e2e.cjs:183—prepareIndependentIosSystemSession()runsyarn agent-device openforinterrupt: 'rotate'andinterrupt: 'inactive'.e2e/contracts/portable-contract.e2e.cjs:239—runIndependentIosSystemAction('orientation', 'landscape-left').Both
gesture-system-interruption(rotate) andios-inactivelistiosinplatforms, so both run onios26.auto-fallback. The scope line "only the pointer harness route changes" and this clause are in tension: the job still starts an Agent Device daemon, just not for pointer input.Suggested amendment: re-scope the clause to "no Agent Device pointer harness is started in that job — no runner preflight and no
run-agent-device-pointer.mjsinvocation," which is what the change actually guarantees and what #54 diagnosed.Consequence for the AC3 streak
Worth deciding before anyone starts counting the five runs.
ios26.auto-fallbackpreviously ranscripts/prepare-agent-device-ios-runner.mjsbecausefree-form-reorder— the first applicable scenario — routed to Agent Device. It now never runs, so the first Agent Device use in that job is a coldopeninsidegesture-system-interruption(scenario 7 of 13), under Detox's 180s test timeout.Two reasons I'd rate this a low risk rather than a blocker, but not zero:
AGENT_DEVICE_STATE_DIR(prepare-agent-device-ios-runner.mjs:56) with sessionissue39-ios-runner-preflight, and explicitly callsstopDefaultDaemon()both before (:82) and after (:141). The interruption path uses the default state dir and sessionissue39-detox-interruption. The only shared warm-up was the XCTest runner build.orientationandopenaren't synthesized touches, so the specific failure mode shouldn't transfer.If the streak does break on
gesture-system-interruptionorios-inactiverather than on a pointer scenario, that's the cause to look at first — and the fix would be keeping the runner preflight for the job independent of pointer routing, which is a script change this ticket ruled out.Two smaller notes
routes, not just delete theoverrides, sincerun-device-contract-isolated.mjs:89dispatches onroutes. Deleting the overrides alone would have left the preflight running. b14997c adds a test asserting each route equals the driver its plan resolves to, so the two maps can't drift.CONTRIBUTING.md:26exempts test/CI/doc changes. ButisReleaseSurface(scripts/release-policy.mjs:15) matches anysrc/prefix and the updated test lives atsrc/__tests__/issue39-device-contract.test.ts, so the changeset-requirement check will need theNo changeset neededmaintainer comment. Pre-existing gap between the documented rule and the script; left alone per "Ruleset / required-check changes: None."ci: excise the screenshot-feedback apparatus
thiagobrez commentedon Sep 4, 2026
Update from PR #87's first CI run, correcting my previous comment's risk assessment in both directions:
The Detox route works. All five
ios26.auto-fallbackpointer scenarios passed via Detox, and the two Agent-Device-driven interruption scenarios (gesture-system-interruption,ios-inactive) passed cold, without the runner preflight — the cold-start concern I flagged did not materialize.But the job still failed, in the screenshot-feedback apparatus — a different scenario on each attempt (
free-form-reorder, thenvirtualized-list-reorder), both atdrag released before feedback sampling. That message was misleading: the drags held for 8000ms, past the ~4.9s sampling window, so the assert could only fire when the Detox gesture promise settled early — and its sampling-first ordering swallowed the real error. SinceApprove exact tarball for publication workflowgates oncontract+feedback+parityand is required onmain, this deadlocked the whole plan: the PRs scheduled to delete the apparatus (#68) could not merge either.Resolution: #68 was pulled forward into PR #87 (see the comment there). Consequences for this ticket:
driver: detox; no Agent Device pointer runner or preflight starts).thiagobrez commentedon Sep 4, 2026
Streak accounting — run 1 (push
bc68489, run 33866714821, PR #87 merge): does not count. Streak: 0/5.The
ios26.auto-fallbackcontract job itself passed (first-ever green on main via Detox, no pointer retry), but the four-engine parity job failed, and AC3 requires parity green per run. Parity fell to two failures unrelated to the iOS 26 route:ios27.native—unmountfailed atopenScenario(app launch never showedCallback count: 0within 15s), across both retained attempts, exit 75. A launch flake on the Agent Device configuration, possibly the runner-artifact class tracked in CI overhaul (9/9): purge the agent-device runner-artifact cache before retry attempts #76/File the two agent-device upstream reports (iOS 26 lost touch stream; runner-artifact dyld cache gap) #84.Required quality and performance gates— the geometry microbenchmark blew its 1msmaximumCorrectionMsbudget (observed 3.43ms) on a hosted Ubuntu runner. Pure timing noise; the identical code passed this job on PR ci: route iOS 26 pointer scenarios through Detox and excise the screenshot-feedback apparatus #87. This job doesn't gate the streak, and CI overhaul (7/9): make the exact package candidate the sole push-to-main gate #74 deletes it from the candidate as a duplicate — though CI overhaul (7/9): make the exact package candidate the sole push-to-main gate #74 is itself blocked on this streak, so if the 1ms budget keeps flaking the plan may want it addressed sooner.Next chance to score run 1: tonight's nightly (03:17 UTC) or the next push to main.
thiagobrez commentedon Sep 5, 2026
Streak accounting update — 3/5, with run 4 in flight.
bc68489(Sep 4)ios27.nativeunmount launch flake + geometry budget)e15ad0a(Sep 4)gesture-system-interruption180s timeout, both attempts9a2bcbb(Sep 5)9a2bcbb(Sep 5)9a50e91(Sep 5)6f6f04a(Sep 5)Note on the
e15ad0abreak:gesture-system-interruptionis not a pointer scenario — it's the Agent-Device-driven rotate interruption, running cold now that no pointer scenario triggers the runner preflight on iOS 26 (the residual risk flagged when this ticket landed). One occurrence in six runs so far. If it recurs and keeps resetting the streak, the candidate fixes are a lightweight preflight for the interruption session or moving rotate to a non-Agent-Device input — either needs a scope decision beyond this ticket.Closes when runs 4 and 5 land green consecutively — potentially tonight's nightly.
thiagobrez commentedon Sep 5, 2026
Streak accounting — 4/5. Run 4: 33984737081 (push
6f6f04a, 2026-09-05 18:39 UTC) completed green;ios26.auto-fallbacksucceeded in 24m41s on a single attempt, parity green. One more consecutive qualifying run closes this — tonight's nightly (03:17 UTC) or the next push to main.thiagobrez commentedon Sep 5, 2026
Acceptance streak complete — 5/5. Closing.
9a2bcbb07:029a2bcbb07:439a50e9111:136f6f04a18:39ff7505a20:52All acceptance criteria met:
ios26.auto-fallbackpointer scenario resolves todriver: detox; no Agent Device pointer runner or preflight starts in that job (PR ci: route iOS 26 pointer scenarios through Detox and excise the screenshot-feedback apparatus #87, commit b14997c, plus the route/override consistency test).ios26.auto-fallbackcontract runs on main/nightly, each with four-engine parity passing and no pointer-test retry — the streak decided in Decide the iOS 26 device-contract input-readiness fix #60.This unblocks #74 (7/9). Residual watch item, recorded earlier in this thread: the cold
gesture-system-interruptionAgent Device timeout (one occurrence in eight main runs,e15ad0a). If it recurs, spec a lightweight preflight for the interruption session or a non-Agent-Device rotate input as a follow-up ticket.