Slow-but-alive provider streams are undetectable: a ~2 tok/s turn never times out, never retries, and never falls back

[code-yeongyu]

Symptom

A gpt-6-astra turn that keeps streaming at about 2.1 tokens per second never fails, never retries, and never walks a fallback chain. The session looks healthy (working line only shows elapsed time) while remaining effectively unusable.

Why

Every in-progress stream watchdog is a silence detector:

• Stream-start (retry.provider.streamStartTimeoutMs, default 300s) applies only until the first event.
• Inter-event idle (retry.provider.timeoutMs / httpIdleTimeoutMs, default 300s) resets on every event, including thinking deltas.
• There is no total-turn budget on the agent loop.
• WebSocket liveness and HTTP body idle likewise reset on any bytes.
• Fallback runs only after a retryable assistant error. A trickle produces none.

Compaction already documents the missing class ("a stream that keeps trickling events never trips the idle watchdog") and bounds it with a wall-clock budget. That bound was not applied to the main turn, and a wall-clock total-turn cap would be the wrong shape anyway (tool-using turns are long).

The suite currently pins the opposite contract: packages/agent/test/agent-loop-stream-start-timeout.test.ts (lets slow-but-alive streams finish despite gaps above the start bound).

Live tok/s is not shown in the working line or footer. The builtin TPS extension posts TPS N tok/s. … only on agent_end.

Shipped retry.fallbackChains is empty (no implicit/wildcard lane). Category builtins pin ultrabrain/deep to a single Astra model in config; resolve-time chains can add gpt-5.6-sol only when that id is in the live registry, and only after an error. architect is a single-rung pin with no second model.

Expected

1. After the first stream event, the working line shows live tok/s (Working (12s • 42 tok/s • Esc to interrupt)).
2. If streamed text+thinking stays under 8 tok/s for a 20s window (5s grace after first event, tool time excluded), abort with Provider stream throughput degraded: … tok/s over 20s (floor 8 tok/s).
3. Do not spend the same-model retry budget. Fall back immediately when a chain candidate exists; if none, end the turn with that error and the existing "No fallback chain configured — set one with /fallback." copy.
4. A control stream at tens of tok/s is unaffected. Silence is still start/idle, not this path.

Acceptance tests (failing first)

• packages/agent/test/agent-loop-throughput-watchdog.test.ts: 2 tok/s mock for >25s → stopReason === "error" and throughput wording; 40 tok/s control → stop.
• packages/coding-agent/test/suite/retry-fallback-throughput-degraded.test.ts: one primary call, then retry_fallback_applied; empty chain → no second model, error still names tok/s.

[code-yeongyu]

Fixed by #1743, merged as a7dee82bb15f216d099c1fd50aa2878ce39e4d59.

What changed. A slow-but-alive stream is now a first-class failure instead of an invisible one. packages/agent/src/stream-throughput-watchdog.ts measures streamed units after the first event and, once the grace period has passed and a full observation window of streaming has been measured, aborts the request when the sustained rate sits below the floor. Shipped defaults: floor 8 units/second, 20 s window, 5 s grace, and a 16-unit minimum before any rate is judged, so a short heartbeat or a single long reasoning pause cannot trip it. Setting the floor or the window to 0 disables the check. Only text and thinking deltas count, and time spent on pending local work is excluded.

Why it is not just another stall. The verdict is deliberately kept out of the provider-stall pattern and gets its own predicate. A stall is silence, which a same-model retry can genuinely fix; a crawling stream is an upstream that is answering too slowly for replaying the same payload to help. So the degraded verdict skips the same-model retry budget and consults the fallback chain immediately, and the fallback model starts with a fresh budget as on any other hop. When no chain candidate exists, the turn ends with the measured rate in the message and emits stream_throughput_degraded carrying chainConfigured, mirroring how an absent fallback chain is already surfaced — the session never sits silently on the trickle. The interactive working status now shows the live rate, so a crawl is visible while it is happening rather than only measurable afterwards.

Evidence. RED at the test-only commit d046741: the decisive case failed with expected 'stop' to be 'error' while three control cases (healthy stream with batched deltas, a stream that ends before the window closes, and a zero floor) passed at the same commit — the test discriminates rather than failing on everything. GREEN at ced1dad: all four throughput cases pass, and every pre-existing stream-start case still passes, including lets slow-but-alive streams finish despite gaps, which was adjusted to sit above the new floor rather than deleted. A second test pins the routing: one primary call, then the fallback applied, and with fallback disabled no second model is called while the user-visible error still names the measured rate. Tests ran on a second machine; CI on the merge head reported no failing and no pending jobs.

Known imprecision, tracked for a follow-up. The unit estimate is ceil(characters / 4), so the message says tok/s while measuring an approximation. It is proportional to the threshold and does not affect the verdict at the shipped floor, but the wording claims more precision than the measurement has.