test: drop the bounded polls, elapsed bounds and positive-path timeouts

[MarcoDotIO]

Summary

#26 left three kinds of wall-clock bounds in the test waits: for _ in 0..<N sleep polls, elapsed-time asserts, and product timeouts used as positive-path bounds. This PR audits each listed site and converts the ones a stalled cooperative pool (5–8 s on the macOS CI job) could fail, using the pattern from #25/#26.

Several product waits park on a continuation that only their own timer or the awaited event resumes, so cancellation never ends them: EmbeddedAgentRuntime.wait(runID:), the gateway's agent.wait, ApprovalBroker.waitDecision, QuestionBroker.waitAnswer, A2ATaskStore.wait, IMsgRPCClient.request and SpotlightTimeoutRace. If a test drops the timeout on one of these waits, a regression can't show up as a hang, because the time limit can't end the wait. A new awaitCancellable(_ label:) in the Linux TestAsyncHelpers.swift runs the wait in its own task and awaits it through an AsyncStream. When the time limit cancels the test, the stream ends and the helper records and throws a labelled AsyncWaitTimeoutError. Tests call these waits through it with no timeout.

1. Iteration-count polls

Site	Change
ApprovalQuestionBrokerTests :80, :138, :212	waitUntil; suite time limit
AgentRuntimeExtensionsTests :96, :111, :161, :171	waitUntil; suite time limit, replacing two per-test limits
ExecApprovalHardeningTests.firstPending (:11)	waitUntil; suite time limit
GatewayRunLifecycleTests :74	waitUntil; suite time limit
RuntimeIntegrationWiringTests :436	waitUntil; suite time limit
ToolsGatewayMethodsTests :168	waitUntil; suite time limit
GatewayServerChatUICompatTests.FrameRecorder.wait (:47)	Labelled, delegates to waitUntil; suite time limit. It used to return the frames it had after 500 polls.
CoreAIModelRuntimeTests :307	waitUntil; per-test time limit. If the loop gave up early, the cancel could land before the first generation had started.
VoiceNoteRecorderTests :362	waitUntil (the suite already had a time limit)
SystemStateReportingTests :201	waitUntil; per-test time limit. The reporter now uses a frozen TestClock. With the real clock, a pause of 50 ms between the two synchronous reports would forward the second one at once, and the "held" assertion would fail.
MediaPipelineTests.waitForHTTPServer (:424)	Polls without a deadline and fails at once if the Python process has exited; suite time limit. The 20 × 100 ms budget bounded Python's startup on a loaded runner.

Kept, on purpose:

• GatewayNodeSessionRouteTests :239, :624: Task.yield loops before negative checks ("no result is sent"). A slow runner can only make them weaker, never fail them.
• CoreAIModelRuntimeTests :209, :213: sampler repetitions, not waits.
• MediaPipelineTests :407: teardown grace period (SIGTERM, then SIGKILL after about 1 s, then waitUntilExit). It cannot fail a test.

2. Elapsed-time asserts

Site	Change
MCPHardeningTests :377 (< 2, legacy SSE)	Removed. The error already names the 100 ms deadline. Without one, the stream never ends and the time limit fails the test.
MCPStdioTransportTests :94 (< 5, close escalates to KILL)	Removed. The server traps TERM, so kill(pid, 0) != 0 after close() already proves the KILL.
MCPStdioTransportTests :127 (< 3, close must not wait for a blocked write)	The server now runs sleep 3600 instead of sleep 5. A close that waited for the write would hang. A cancellation handler SIGKILLs the server, so the time limit ends that hang.
SpotlightMemoryTests :96 (< 3, bounded writes)	Expects SpotlightTimeoutError(operation:seconds: 0.2), so the 0.2 s deadline is the one that fired. FakeSpotlightStore now keeps the calls it stalls, and stopHanging() answers them on cancellation, so a write with no deadline fails at the time limit instead of hanging the run. Per-test time limit.
SpotlightMemoryTests :242 (< 2, race doesn't join a stalled op)	Removed. The stalled operation now loops until a StallFlag ends it, either at test end or on cancellation. Before, it spun on cancelled Task.sleep calls for the rest of the process. The fast-path check uses a 1 h deadline through awaitCancellable. Per-test time limit.
GatewayRunLifecycleTests :46 (< 5, timed agent.wait)	Removed. The run now sleeps for 1 h and ends only when aborted. A wait that ignored its 50 ms timeout would hang, and awaitCancellable turns that into a failure.
AutomationHardeningTests :156 (< 1, cron search)	Kept. The measured region is synchronous with no suspension, so pool starvation cannot land inside it.
ChannelAutoReplyTests :233 (>= 0.07)	Kept. It's a lower bound, so a stall can only make it pass by a larger margin.
ModelRoutingTests :379 (>= 0.06)	The lower bound is kept. The model.throttle.delay check next to it could fail, though: a stall of more than 70 ms between the two requests lets the window pass, so no delay happens. That check now uses a second router with a 1 h window. The held request emits the event (with delayMs near the window) and is then cancelled.

The drop test in ModelRoutingTests (:326) had the same exposure: a 1 s window, and the second request had to land inside it. The window is now 1 h. A drop never waits, so a longer window costs no test time.

3. Product timeouts on positive paths

Site	Ends on cancellation?	Change
SignInWithChatGPTSessionTests signIn(timeout: 20) ×4	Yes. The task group's callback wait stops the listener on cancel, the timer is a Task.sleep, and the test browsers return once the listener stops.	timeout: 3_600. timeout: 1 in the timed-out case is the behaviour under test and is kept.
ChannelAdaptersLinuxSmokeTests A2A replyTimeoutMs: 5_000	No (A2ATaskStore.wait)	A2AChannelConfig.replyTimeoutRangeMs.upperBound (10 min, the clamp) + awaitCancellable
SubagentRuntimeHardeningTests runtime.wait(timeoutMs: 10_000) ×5	No	awaitCancellable { runtime.wait(runID:) }. runtime.run(timeoutMs: 10_000) ends on cancellation, so it moves to 1 h.

In the same files:

• ApprovalQuestionBrokerTests: waitDecision / waitAnswer(timeoutMs: 2_000) and runtime.wait(timeoutMs: 2_000).
• AgentRuntimeExtensionsTests: run(timeoutMs: 10_000), wait(timeoutMs: 2_000 / 5_000).
• GatewayRunLifecycleTests: three agent.wait(timeoutMs: 5_000).
• ChannelAdaptersLinuxSmokeTests: the imsg request(timeoutMs: 10_000).

All of these go through awaitCancellable with no timeout. The 20 ms negative waitAnswer is kept.

Two tests had a wall-clock window that was not a wait:

• aStaleTimerNeverCancelsANewRunWithTheSameID: the first run has to finish inside its own 200 ms timer. A stall let the timer win, the run reported timeout, and the test failed. The test now starts over with a fresh runtime until the first run finishes first; the time limit bounds a regression that never lets it.
• reusedIdempotencyKeysDoNotStartASecondRunUnderTheSameID: the run stayed in flight for a 3 s sleep while the duplicate sends were checked. It now waits on an AsyncGate that the test opens after those checks.

Not in this PR

The same positive-path pattern (agent.wait / runtime.wait with a 2–5 s timeout) remains in files outside the list: AgentGatewayMethodsTests, SessionBranchGatewayMethodsTests, AgentLoopToolCallingTests, GatewayEventStreamTests, GatewayWireShapeTests, and in OpenClawKitTests EmbeddedAgentStackTests, GatewayServerTests and GatewayServerRegistryTests. With awaitCancellable, converting them is a mechanical follow-up. OpenClawKitTests would need its own copy of the helper.

Stacked on #26 (→ #25 → #24). Merge those first; until then this diff also includes their commits. Only the last commit (ba41908) is new.

Test plan

• swift build --build-tests
• Scripts/lint-swift.sh: 0 violations
• Targeted swift test --filter over the 18 affected suites: everything passes except the known replyCoordinatorRoutesRepliesToOneInvocationAtATime flake, which failed 1 of 6 isolated Spotlight runs.
• Full swift test: OpenClawKitTests 3,150 tests, OpenClawLinuxRuntimeTests 582 and OpenClawKitE2ETests 20, all passing.
• Pool starvation. Uncommitted scratch hooks spawned activeProcessorCount * 2 Task.detached(priority: Task.currentPriority) { usleep(6_000_000) } blockers, which stalls the pool for about 12 s.

  • The hooks fired at each converted wait: at the start of waitUntil, inside awaitCancellable after the operation starts, and inline before connect() / close() / the Spotlight writes / the race, between the two throttled requests, and between the two coalesced reports. Each process ran one test and at most 3 stalls.

  • All 36 converted tests pass, after 12–36 s, so every one of them was stalled.

  • Control, with the same stalls on the old code: all 8 fail.

    Old test	Failure under the stall
    MCP legacy SSE	elapsed < 2
    stubborn stdio close	elapsed < 5
    Spotlight writes	elapsed < 3
    throttle drop	second.providerID == "secondary": the 1 s window had passed
    throttle delay	no model.throttle.delay event
    coalescing reporter	the second update was forwarded at once
    stale timer	runtime.wait(timeoutMs: 10_000) returned timeout
    timed agent.wait	the 10 s run finished inside the stall, so the status was not timeout

  • The stale-timer retry was checked with a scratch 1 ms timer on the first attempt only: the first run reports timeout, the loop starts over, and the test passes.

• Mutation checks: each turns a converted wait into a hang (reverted afterwards). Every one fails at 60.0 s with Time limit was exceeded, plus Timeout waiting for: <label> where a helper is involved, and the run then ends:
  • Approval that never expires: approval expired
  • agent.wait with no timeout on a 1 h run: timed-out agent.wait returned
  • imsg script that never replies: imsg ping answered
  • A2A handler that never returns: A2A task completed
  • Throttle set to .drop: model.throttle.delay emitted
  • Python that never serves: local HTTP server answering
  • Legacy SSE connectionTimeoutMs: 3_600_000: the connect ends with CancellationError.
  • Spotlight writeTimeoutSeconds: .infinity: stopHanging() answers the calls, and the test reports "an error was expected but none was thrown".
  • Spotlight race with no deadline: StallFlag ends the operation, and the test reports value == nil.
  • Stdio close awaiting the blocked write first: the cancellation handler kills the server.
  • SIWC silent browser with a 1 h timeout: sign-in ends with CancellationError.

🤖 Generated with Claude Code

[MarcoDotIO]

Landed on main in dca4976 via #31. #31's branch was main plus #30's head, and its squash-merge went in before this PR, so dca4976 carries this PR's changes and lists its commits. Merging this PR now would only add an empty commit, so I'm closing it as landed. Checked: git merge-tree of this branch into main changes no files.