You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ios(ci): cold toolchain probes time out below the first-exec signature stall (10 s / 5 s budgets) #2422
The iOS smoke lane fails on cold macOS runners with a toolchain probe timeout, unrelated to the PR under test:
Verify clean-installed Simulator snapshot bridge preparation: AppError: xcrun timed out after 10000ms (from verify-installed-snapshot-bridge.ts → ensureSnapshotBridgeBinary).
Preflight iOS runner through public CLI: Could not read the Xcode toolchain versions the Apple runner cache is keyed on (xcodebuild -version: … timed out after 5000ms).
Occurrences in the last three days, all on the same step and with no code in the failing path changed by the branch: refactor/core-replay-divergence-* (2026-09-07 16:06), claude/2339-contracts-single-owner-moves (09-06 10:23), claude/wait-poll-timeline (09-06 09:07), PR #2418 three consecutive runs on 2026-09-09 (09:50, 10:43 preflight variant, 10:51), PR #2420 (10:49, 45 s variant). Six other branches' iOS runs passed in the same window, so it is per-runner cold state, not a fleet outage.
Cause
Two cold-toolchain probes carry budgets below the first-exec stall of Apple's signature verification on a fresh host (the first xcodebuild/xcrun/large-binary exec after boot blocks roughly 18 to 19 s at 0% CPU while syspolicyd scans; the second call is instant):
packages/platform-apple/src/snapshot-source/cache-identity.ts:87: toolOutput runs xcodebuild -version, sw_vers, uname, xcrun --sdk iphonesimulator --show-sdk-version with timeoutMs: Math.min(10_000, remaining), under a 120 s overall deadline set by the verify script.
packages/platform-apple/src/runner/runner-cache-metadata.ts:27: TOOLCHAIN_PROBE_TIMEOUT_MS = 5_000 for xcodebuild -version in the runner cache key.
Both budgets were sized for a warm toolchain. The overall deadlines (120 s snapshot-source, the runner preflight budget) have room; only the per-call budget trips.
Required behavior
A toolchain identity probe tolerates one cold-start stall: per-call budget of at least 30 s bounded by the remaining deadline, and one retry after a timeout while the deadline allows, because the retry is instant once the signature check has run.
Both sites read the same constant and carry one comment naming the stall, so the two do not drift again.
The timeout error keeps its current text and code so existing tests and the runner-cache hint stay valid.
Unit tests: a fake host whose first probe call exceeds the budget and whose second returns immediately passes; a host that never returns still fails at the deadline with the same error.
Non-goals
Raising the snapshot-bridge native build timeout (BUILD_TIMEOUT_MS) or the overall deadlines.
Third site, seen on PR #2421 (run 34366211699) and earlier on #2420 (10:49): the same CI step fails inside packages/platform-apple/src/snapshot-source/native-runtime.test.ts:13-32, which compiles the bridge with xcrun --sdk iphonesimulator clang … under a hard timeoutMs: 45_000, and the runner reports xcrun timed out after 45000ms. That is the native compile, not the identity probe, so PR #2423 does not cover it. On a cold runner the first xcrun here pays the same signature stall plus the compile itself. Candidate fix, separate from #2423: budget that compile from the test's own deadline (the production path uses BUILD_TIMEOUT_MS bounded by the 120 s snapshot-source deadline) rather than a fixed 45 s, or skip the compile in the unit lane when the CI step already verified it against the clean-installed package moments earlier.
Split: the third site (the fixed 45 s clang compile budget in native-runtime.test.ts) is now tracked as #2439. PR #2423 closes this issue for the two identity-probe sites it names.
Activity
fix(apple): duplicate the cold-toolchain-probe budget instead of a sh…
fix(ios): budget cold toolchain probes for the first-exec signature s…
fix(apple): duplicate the cold-toolchain-probe budget instead of a sh…
fix(apple): bound the cold toolchain probes by the owning request budget
thymikee commentedon Sep 9, 2026
Third site, seen on PR #2421 (run 34366211699) and earlier on #2420 (10:49): the same CI step fails inside
packages/platform-apple/src/snapshot-source/native-runtime.test.ts:13-32, which compiles the bridge withxcrun --sdk iphonesimulator clang …under a hardtimeoutMs: 45_000, and the runner reportsxcrun timed out after 45000ms. That is the native compile, not the identity probe, so PR #2423 does not cover it. On a cold runner the firstxcrunhere pays the same signature stall plus the compile itself. Candidate fix, separate from #2423: budget that compile from the test's own deadline (the production path usesBUILD_TIMEOUT_MSbounded by the 120 s snapshot-source deadline) rather than a fixed 45 s, or skip the compile in the unit lane when the CI step already verified it against the clean-installed package moments earlier.fix(ios): budget cold toolchain probes for the first-exec signature s…
fix(apple): duplicate the cold-toolchain-probe budget instead of a sh…
fix(apple): bound the cold toolchain probes by the owning request budget
thymikee commentedon Sep 10, 2026
Split: the third site (the fixed 45 s clang compile budget in
native-runtime.test.ts) is now tracked as #2439. PR #2423 closes this issue for the two identity-probe sites it names.fix(ios): budget cold toolchain probes for the first-exec signature s…
fix(apple): duplicate the cold-toolchain-probe budget instead of a sh…
fix(apple): bound the cold toolchain probes by the owning request budget
fix(ios): budget cold toolchain probes for the first-exec signature s…
fix(apple): duplicate the cold-toolchain-probe budget instead of a sh…
fix(apple): bound the cold toolchain probes by the owning request budget
fix(ios): budget cold toolchain probes for the first-exec signature s…