feat(combo): per-combo attempt first-byte deadline so a stalled target hops before the global 200s connectTimeoutMs

[Veritas-7]

Area

Proxy and routing

What are you trying to accomplish?

I run a combo (strategy: round-robin, three targets that point at the same upstream family through different keys) as a resilient lane for an agent client launched with ocx claude. When one target stalls (the upstream accepts the connection but never sends response headers), the request should hop to the next target within a bounded, combo-specific time so the agent's turn survives instead of timing out on the client side.

What prevents this today?

• Every combo attempt inherits the global connectTimeoutMs (default 200000) as its header-arrival budget — src/server/chat-native.ts:171, src/server/claude-messages.ts:380, src/server/responses/core.ts:3820 (v2.38.0 line numbers). A stalled target therefore burns ~200 s before the combo loop reaches comboFailureDecision → advanceComboAfterFailure and hops.
• OcxComboConfig (src/types/config.ts) has no per-attempt or per-request time budget: only targets, strategy, stickyLimit, defaultEffort, imageInput. I checked main on 2026-09-02 and this is still the case after v2.40.0.
• Lowering connectTimeoutMs globally is not a safe workaround: it also governs every non-combo provider, including ones whose first byte legitimately arrives late (long non-streaming reasoning).

Observed on one proxy over a 24 h window (v2.38.0, ~4.8k requests to a 3-target glm-5.3-flash combo while the upstream was rate-limiting and stalling):

metric	value
requests where a target attempt failed with 502 only after ~200 s of header wait	11
of those, requests where the client cancelled (499) during the second attempt	6
wall time of those cancelled requests	343–359 s
healthy first-byte latency on the same proxy, same period (combo / openai / xai targets)	p95 ≤ 24 s, max 53 s

So the 200 s wait is almost never a legitimate slow start for these targets; it is dead time that the 60 s target cooldown cannot recover because the cooldown only starts after the attempt fails.

Log shape (from the proxy console):

[combo] glm-round: glm-a/glm-5.3-flash failed with 502 after 200010ms
[combo] glm-round: glm-b/glm-5.3-flash failed with 500 after 180245ms
[combo] glm-round: glm-c/glm-5.3-flash failed with 502 after 200013ms

What should OpenCodex do?

1. Accept an optional per-combo attempt first-byte deadline (name open, e.g. attemptFirstByteTimeoutMs). If a combo attempt has not received upstream response headers within that time, treat it exactly like a hop-able upstream failure: cool the target (existing coolComboTarget), log the [combo] … failed with … after …ms line with a distinct reason (e.g. first-byte-timeout), and advance to the next eligible target.
2. Optionally accept a per-combo total request budget (e.g. requestBudgetMs). When the sum of attempts would exceed it, return the last failure instead of starting another attempt.
3. Default for both: unset → current behaviour (inherit connectTimeoutMs, no total budget), so existing configs are unchanged.
4. Expose both fields in the dashboard combo editor next to stickyLimit.

Scope note: this is not #486 (cooldown length; that issue was auto-closed for missing detail). Cooldown semantics stay as they are — this proposal only bounds the attempt itself, which the cooldown cannot do.

Example usage or interface

"combos": {
  "glm-round": {
    "strategy": "round-robin",
    "attemptFirstByteTimeoutMs": 60000,
    "requestBudgetMs": 180000,
    "targets": [
      { "provider": "glm-a", "model": "glm-5.3-flash" },
      { "provider": "glm-b", "model": "glm-5.3-flash" },
      { "provider": "glm-c", "model": "glm-5.3-flash" }
    ]
  }
}

Expected behaviour with that config:

[combo] glm-round: glm-a/glm-5.3-flash failed with first-byte-timeout after 60008ms
[combo] glm-round: glm-b/glm-5.3-flash ok after 4210ms

i.e. the hop happens after ~60 s instead of ~200 s, and the whole request can never exceed ~180 s of upstream time.

Alternatives or workarounds

• Lower connectTimeoutMs globally — rejected, it changes behaviour for every non-combo provider.
• Patch src/server/responses/core.ts locally — works, but the patch is lost on every ocx update.
• Rely on the client's own turn timeout — the client does cancel (499), but only after the proxy has already spent 200 s+ on the stalled target, so the turn is lost either way.

Additional context

• Related, but none of them bound the attempt's header wait: [Feature]: Make failover cooldown duration configurable (currently hardcoded 60s) #486 (cooldown knob), [Feature]: re-evaluate a routing profile when the selected candidate fails terminally #1572 (re-evaluate routing profile on terminal failure), fix(routing): treat a live full burst window as exhausted, not unknown (#3029) #3110 (burst window, v2.39.0), [Bug]: failover combos stop on zero-output SSE terminal failures and model-EOL HTTP 410 responses #2431 / fix(combo): fail over zero-output transport incompletes (carry of #3227) #3236 (zero-output failover, v2.40.0).
• Happy to open a PR if the shape above (or a different field name / placement) is acceptable.

Checks

• I searched existing issues and documentation.
• This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
• I removed secrets and personal data.

[lidge-jun]

리뷰 · 우선순위 48 / 80

이 이슈는 combo 레인에서 한 타깃이 TCP/연결은 받아 두고 응답 헤더(첫 바이트)를 안 보내는 멈춤(stall)일 때, 전역 connectTimeoutMs(기본 200초)를 다 태우기 전에 다음 타깃으로 넘어가게 하자는 제안이다. 작성자 시나리오는 strategy: round-robin 콤보를 ocx claude 에이전트 앞에 두고, 키만 다른 같은 패밀리 업스트림 세 개를 넣는 형태다. 지금 구조에서는 멈춘 타깃 하나가 턴 전체를 약 200초 잡아먹고, 그제야 comboFailureDecision → advanceComboAfterFailure 로 넘어간다.

현재 dev HEAD 272ff6b11(패키지 2.41.0) 기준으로 본문이 가리킨 코드 경로는 맞다. src/server/chat-native.ts 171줄, src/server/claude-messages.ts 380줄, src/server/responses/core.ts 3925줄 부근에서 시도마다 config.connectTimeoutMs ?? 200_000 을 헤더 도착 예산으로 쓴다. OcxComboConfig(src/types/config.ts 808–840줄) 필드에는 targets · strategy · stickyLimit · defaultEffort · reasoningEffortMode · imageInput · alias · nativeAlias · displayName 만 있고, 시도당 first-byte 데드라인 필드는 없다. 전역 connectTimeoutMs 를 줄이는 우회는 콤보 아닌 단일 제공자 요청까지 같이 짧아져서, 작성자가 말한 대로 안전하지 않다.

이미 가까운 기차가 하나 있다. #3236 이 콤보 스트림의 zero-output incomplete 를 페일오버한다(src/server/responses/combo-stream-preflight.ts). 그건 헤더는 왔는데 본문이 비는 쪽이고, 이 이슈는 헤더 자체가 안 오는 stall 이다. 겹치지만 같은 패치가 아니다. 그래서 zero-output 머지를 이유로 이 이슈를 닫으면 안 된다. types.ts/config.ts 분할 캠페인과도 직접 충돌하지 않는다. 새 필드를 OcxComboConfig 에 넣으면 설정 스키마 쪽이 조금 건드려질 수는 있다.

우선순위 48인 이유. 실사용 고통(에이전트 턴 타임아웃)은 분명하고 재현 서술이 구체적이다. 다만 설계 선택이 남아 있다. 시도당 예산을 콤보 필드에 둘지, 라우팅 프로필에 둘지, 헤더 대기만 자를지 connect 전체와 공유할지, 스트림 경로(chat-native/claude-messages/responses/core) 세 곳에 같은 계약을 어떻게 심을지. 기본값을 전역보다 짧게 두면 느린 정상 업스트림을 오탐으로 넘길 수 있다. enhancement + proxy 라벨은 맞다. 지금 당장 릴리즈 블로커는 아니다.

경로 src/types/config.ts OcxComboConfig - 시도당 first-byte/헤더 예산 필드가 없다. 전역 connectTimeoutMs 만으로는 콤보 홉을 짧게 못 만든다.
경로 src/server/chat-native.ts:171 · claude-messages.ts:380 · responses/core.ts:3925 부근 - 시도마다 전역 200s 헤더 예산을 그대로 쓴다. stall 한 번이 턴을 잠근다.
경로 comboFailureDecision / advanceComboAfterFailure - 실패 판정 후에야 다음 타깃으로 간다. 예산이 길면 홉도 늦다.
경로 #3236 zero-output 페일오버 - 헤더 이후 빈 본문 문제. 이 이슈(헤더 미도착 stall)와 겹치되 대체재가 아니다.
경로 전역 connectTimeoutMs 축소 - 콤보 밖 요청까지 같이 짧아져 우회로 부적합하다.

메인테이너의 판단이 필요한 지점

• 새 필드를 OcxComboConfig 에 둘지(attemptHeaderTimeoutMs 같은 이름) 전역 옆에 콤보 전용 오버라이드만 둘지
• 기본값을 얼마로 할지(예: 5–15s) 그리고 sticky RR 배치와 어떻게 맞출지
• chat / claude-messages / responses 세 경로에 동일 계약을 한 패치로 심을지, 가장 아픈 경로만 먼저 할지
• help wanted 로 열어 둘지, 메인테이너가 스키마부터 짧게 스펙을 박고 받을지
• types/config 분할 진행 중이라 스키마 PR은 close-don't-rebase 대상이 될 수 있는지(필드 추가면 보통 독립 가능)

너의 추천
열어 두고 needs-design 성격으로 짧게 스펙을 먼저 박는 편이 좋다. 최소 스펙 예시는 OcxComboConfig 에 optional 시도당 헤더 예산(전역보다 짧을 때만 적용) + 세 서버 경로에서 그 값을 connect 예산 상한으로 쓰기 + stall/timeout을 기존 combo failure 경로로 넘기기. #3236 과 이슈를 합치지 마라. 중복·분할 무효로 닫을 대상이 아니다. 당장 머지할 PR은 없다.

이 댓글은 grok-bot이 작성했습니다

[Veritas-7]

Proposed spec for #3266 (separate from #3236):

Add two optional per-combo fields:

• attemptFirstByteTimeoutMs: maximum time for one target to produce final response headers.
• requestBudgetMs: optional wall-clock budget shared by all target attempts in one logical combo request.

Both are positive integer milliseconds (1..600000). When omitted, behavior is unchanged. If both are present, require requestBudgetMs >= attemptFirstByteTimeoutMs.

For each target, the effective header deadline is:

min(global connectTimeoutMs, attemptFirstByteTimeoutMs, remaining requestBudgetMs)

The request budget starts once when combo execution begins and is never reset on hop or same-target recovery. If it is exhausted before a send, no further target is dispatched. A header deadline produces synthetic HTTP 504 with error.code = "first_byte_timeout"; this is target-local and therefore enters the existing combo hop path. That path records the failed attempt, cools the target through advanceComboAfterFailure/coolComboTarget, and selects the next eligible target. Client cancellation remains 499 and never hops.

Suggested log line:

[combo] <comboId>: <provider/model> first-byte-timeout after <elapsed>ms; hopping

Implementation should reuse the existing header-timeout machinery across Chat, Anthropic Messages, Responses passthrough, and adapter paths, with only minimal CODEOWNED core.ts plumbing. Tests should cover validation/default compatibility, deadline precedence, stalled-header failover/cooldown, all transport paths, and budget exhaustion before a later dispatch.

[Veritas-7]

Measured data now argues against our own earlier proposal of a global per-attempt first-byte deadline.

What we measured

• Source: six days of local proxy attempt records, covering 134,716 attempts.
• Stall criterion: zero first byte plus duration clustered at the global timeout (200.008–200.019 s).

Measure	Result
Timeout-shaped stalls	23 attempts (0.17 per 1000); 4 of 6 days
Attempts in the timeout band	28; failover rescued 5
Successful attempts at 190 s or more to first byte	44 (0.33 per 1000)
Successful attempts at 30–120 s to first byte	12,249 (9.1%)
p99 time-to-first-byte	57 s

Why the proposed fix is net-negative

A 190 s first-byte deadline would abort 44 healthy successful responses to rescue 23 stalled ones, roughly a 1.9:1 loss. At 120 s, the trade-off is about 7:1 against. The mechanism cannot distinguish a dead target from a model still thinking until the first byte arrives; connection-level failures overwhelmingly fail fast (under 5 s), so the 200 s stall is a rare tail.

Alternatives

A. Hedged failover

After a configurable no-first-byte delay, start the next eligible target in parallel without aborting the first; whichever produces the first byte wins and the loser is aborted.
Default off, with the delay set well above p99 first-byte time; duplicated upstream usage is bounded by aborting the loser, and a healthy slow response is never killed.

B. Per-target opt-in deadline

Allow each combo target to declare its own first-byte deadline, absent by default; the measurement makes this more defensible for a fast non-reasoning target than as a global policy.
Long-but-healthy first-byte times concentrated on reasoning-heavy targets, so a target-specific policy avoids imposing their latency on every target.

Would you rather see the connect/TLS phase get its own shorter timeout? That is a cheaper change, but it would not catch these particular stalls because the connection had already been accepted before the silence began.

We are happy to implement whichever direction you prefer; the branch for the original design is parked, not abandoned.

[Veritas-7]

A correction to a number in my earlier comment, found while writing this up internally.

I wrote that 12,249 attempts (9.1%) took 30 to 120 seconds to first byte and succeeded. That label is wrong. Re-running the same sample — the first 134,716 attempts in the log, ending 2026-09-03T13:31Z — separates cleanly:

measure, successful attempts in the 30-120 s band	count	share
time to first byte	2,057	1.5%
total attempt duration	12,249	9.1%

So 12,249 was a total-duration figure presented as a first-byte one. The underlying cause is that the log carries both a request-level and an attempt-level record with the same field names, and I conflated the two.

The conclusion is unaffected, and if anything the corrected figure argues for it more strongly. The decision rests on how many healthy responses a first-byte deadline would abort versus how many stalls it would rescue. On first-byte timing at that cutoff: 2,057 successful attempts waited 30-120 seconds for their first byte, against 21 attempts that produced no first byte at all and ran to the 200-second timeout. I originally gave the stall count as 23; the number moves between 21 and 23 depending on how tightly "at the timeout" is banded, since the durations cluster between 198 and 204 seconds rather than landing on 200000 ms exactly — no record matches that value exactly, which is worth knowing for anyone reproducing this.

Either way the ratio is roughly two orders of magnitude against the deadline, so the recommendation stands: hedging a second attempt without aborting the first, or a per-target opt-in deadline, rather than a global first-byte cutoff.

[Veritas-7]

Second correction — opencodex#3266

This is my second correction to the measurements I posted on this issue. The issue is lidge-jun/opencodex#3266, titled feat(combo): per-combo attempt first-byte deadline so a stalled target hops before the global 200s connectTimeoutMs; the proposed feature is combo-route-only.

I apologize for reversing myself a second time on this thread. The numbers below were re-measured from scratch on the same sample, and the full reproduction script and its output are included so the figures can be checked independently rather than taken on trust.

The file-order prefix of the first 134,716 attempts — the same sample size as the original post — yields 19 timeout-shaped stalls. On this same prefix: 23 is the count of all attempts with 198,000 <= durationMs < 200,000; 28 is the count of no-first-byte attempts with 198,000 <= durationMs < 202,000; the 21 I gave in my previous correction is the count of no-first-byte attempts with 199,500 <= durationMs < 200,500; and 19 is the count of no-first-byte attempts in the tight 200,000 <= durationMs < 200,020 window that the original comment actually described (200.008–200.019 s). The published counts are band-sensitive, and that accounts for most of the spread I reported. I am not claiming those bands are the populations the original numbers came from: the 23-count is 20 successful (status 200) attempts and 3 with status 502, and 19 of the 23 recorded a first byte — 14 of them under 60 seconds, the earliest at 1,178 ms — so that set is not "timeout-shaped stalls" under any reading. What does line up is the rescue count: both the 28 band and the tight window contain exactly 5 attempts whose parent request still ended 200, matching the "failover rescued 5" I published. I am standardizing on the tight 200,000 <= durationMs < 200,020 window because it is the criterion the original comment stated (200.008-200.019 s), and I am reporting the band sensitivity beside it rather than claiming the earlier counts are reconstructed. The prior cutoff label was also wrong: 2026-09-03T13:31Z was Korean local time written with a Z suffix. The measured boundary is 2026-09-03T04:31:35.971000Z (epoch 1788409895971, = 13:31:35.971+09:00).

Values in the published (verbatim) column are copied from issue comments 5520470506 and 5534802008; the four rows labelled as my previous correction preserve the latter comment's published claims. Corrected values carry an explicit population label; the new route decomposition is measured on the same 134,716-attempt prefix.

measure	published (verbatim)	corrected	corrected population
Timeout-shaped stalls	23 attempts (0.17 per 1000); 4 of 6 days	19 (status 502 = 18, status 504 = 1); 0.141 per 1000; 3 of 6 days	all attempts
Attempts in the timeout band	28; failover rescued 5	19 in the tight 200,000 <= durationMs < 200,020 window; failover rescued 5; every attempt in that window had no first byte. Widening the window to 199,900-200,100 gives 21 no-first-byte attempts (20 of them combo); the published 28 corresponds to no-first-byte attempts in 198,000-202,000.	all attempts
Timeout-shaped stalls (my previous correction)	21 attempts that produced no first byte at all and ran to the 200-second timeout	reproduces as no-first-byte attempts with 199,500 <= durationMs < 200,500; the tight 200,000-200,020 window gives 19	all attempts
Stall-count stability (my previous correction)	the number moves between 21 and 23 depending on how tightly "at the timeout" is banded	confirmed; each of 23/28/21/19 reproduces as a band on this prefix, though I cannot confirm which band the original counts used	all attempts
Duration spread (my previous correction)	durations cluster between 198 and 204 seconds rather than landing on 200000 ms exactly	confirmed: 0 records at exactly 200,000 ms; 30 no-first-byte attempts fall in 198-204 s, of which 19 fall in 200.008-200.019 s	all attempts
Recommendation (my previous correction)	the recommendation stands: hedging ... rather than a global first-byte cutoff	retracted as stated: the cost side that produced it is non-combo, so it cannot decide a combo-only feature	published versus corrected route attribution
Successful attempts at 190 s or more to first byte	44 (0.33 per 1000)	44 (status 200 with a recorded first byte); route split: explicit-provider 42, native 2, combo 0	all successful attempts with status 200 and firstOutputMs >= 190,000
Successful attempts at 30–120 s to first byte	12,249 (9.1%)	2,057 (status 200 with a recorded first byte; 1.5% of all 134,716 attempts); 12,249 is the total-duration count, not the first-byte count	all successful attempts with status 200 and firstOutputMs in 30,000–120,000 ms
p99 time-to-first-byte	57 s	56,582.820 ms (linear interpolation) / 56,664 ms (nearest-rank); both round to 57 s	all attempts with status 200 and a recorded first byte
190-second conclusion	A 190 s first-byte deadline would abort 44 healthy successful responses to rescue 23 stalled ones, roughly a 1.9:1 loss.	Invalid as a combo-route cost estimate: the 44 are non-combo, while the feature is combo-route-only.	published comparison versus corrected route attribution
120-second conclusion	At 120 s, the trade-off is about 7:1 against.	Combo-only: 1 success (status 200 with a recorded first byte) at or above 120 s versus 18 stalls; this is not 7:1 against.	combo only

Why the original loss calculation was invalid

The original 190-second argument used 44 slow successful attempts as the cost side. On the corrected prefix, those 44 are explicit-provider 42 + native 2 + combo 0. The proposed change is combo-route-only, so those 44 responses are outside the feature's reach. I therefore imported the cost from routes the feature cannot affect.

Combo-only trade-off

For the route the feature can actually change, the corrected prefix contains 14,400 combo attempts, 18 combo stalls in the tight 200,000 <= durationMs < 200,020 window, and 0 combo successes with first byte at or after 190 seconds. Every combo stall ended at 200,008–200,019 ms, so a 190-second deadline would have an observed cost of 0 combo successes and would save only 10.008–10.019 s per stall, 180.195 s across the six-day sample. This is a small benefit, not the earlier claimed rescue-versus-loss trade-off. Because the stalls are in the tight window, 190 s leaves about 10 s of the 200 s cap before the observed stall boundary.

The anti-deadline recommendation in my previous correction is withdrawn: its cost side was non-combo, so it cannot decide a combo-only feature. Threshold choice is the material decision: at 60 s, 8 combo successes are in the observed danger zone while a stalled attempt would be cut roughly 140 s earlier; at 120 s, 1 combo success is in that zone; at 190 s, 0 are. These measurements leave that policy choice to the maintainer rather than presenting a global 44-versus-23 loss calculation that does not apply to combo.

Method

• The reproducible sample is the file-order prefix. The 134,716th attempt is on 1-indexed log line 155,335, with boundary timestamp 1788409895971 (2026-09-03T04:31:35.971Z).
• routeDecision.routeKind is a request-level field; it is absent from all 134,716 attempt objects, so each attempt inherits routeKind from its parent request record.
• An attempt record is an element of the attempts array on a request-level line. Lines without an attempts array are compact request records and are excluded.
• A missing firstOutputMs is a measurement gap, not a stall: 45,604 of the 49,074 no-first-byte attempts in this prefix (93%) ended with status 200. Only the tight window is all-5xx (502 x 18, 504 x 1); the wider bands mix in completed successes.
• The timeout-shaped stall criterion is no first byte plus 200,000 <= durationMs < 200,020. The band-sensitivity controls additionally count all attempts in 198,000 <= durationMs < 200,000, no-first-byte attempts in 198,000 <= durationMs < 202,000, and no-first-byte attempts in 199,500 <= durationMs < 200,500. The first-byte controls use successful attempts with firstOutputMs; the total-duration control uses successful attempts with durationMs.
• p99 is reported by both methods: linear interpolation at 0.99 * (n - 1) and nearest-rank at index ceil(0.99 * n) - 1. The method is named beside every p99 result.
• The output block below was generated by redirecting the corresponding command to a file and then diffed against the block.

Reproduction

The required source-document reproduction script was run with the following command. It reads only request-level attempts array elements and excludes compact records without that array.

python3 - "$HOME/.opencodex/usage.jsonl" 134716 <<'PY'
import collections
import datetime
import json
import math
import statistics
import sys

path, target_attempts = sys.argv[1], int(sys.argv[2])
attempts = []
last_line = None
last_timestamp = None
with open(path, encoding="utf-8") as stream:
    for line_number, line in enumerate(stream, 1):
        record = json.loads(line)
        attempt_field = record.get("attempts")
        if not isinstance(attempt_field, list):
            continue
        route_kind = (record.get("routeDecision") or {}).get("routeKind")
        if not isinstance(route_kind, str):
            raise SystemExit(f"missing parent routeKind at log line {line_number}")
        for attempt in attempt_field:
            attempts.append((attempt, record))
            last_line = line_number
            last_timestamp = record.get("timestamp")
            if len(attempts) == target_attempts:
                break
        if len(attempts) == target_attempts:
            break

if len(attempts) != target_attempts:
    raise SystemExit(f"expected {target_attempts} attempts, got {len(attempts)}")

attempt_values = [attempt for attempt, _ in attempts]

def succeeded(attempt):
    return attempt.get("status") == 200

def has_first_output(attempt):
    return attempt.get("firstOutputMs") is not None

def is_number(value):
    return isinstance(value, (int, float)) and not isinstance(value, bool)

no_first_byte_near_timeout = [
    (attempt, record) for attempt, record in attempts
    if not has_first_output(attempt)
    and is_number(attempt.get("durationMs"))
    and 200_000 <= attempt["durationMs"] < 200_020
]
first_byte_successes = sorted(
    attempt["firstOutputMs"]
    for attempt in attempt_values
    if succeeded(attempt) and has_first_output(attempt)
)

p99_position = 0.99 * (len(first_byte_successes) - 1)
p99_lower_index = math.floor(p99_position)
p99_linear_ms = (
    first_byte_successes[p99_lower_index]
    + (first_byte_successes[p99_lower_index + 1] - first_byte_successes[p99_lower_index])
    * (p99_position - p99_lower_index)
)
p99_nearest_rank_ms = first_byte_successes[math.ceil(0.99 * len(first_byte_successes)) - 1]

band_attempts = [
    (attempt, record) for attempt, record in attempts
    if is_number(attempt.get("durationMs"))
    and 200_000 <= attempt["durationMs"] < 200_020
]
all_attempts_198000_200000 = [
    (attempt, record) for attempt, record in attempts
    if is_number(attempt.get("durationMs"))
    and 198_000 <= attempt["durationMs"] < 200_000
]
no_first_byte_198000_202000 = [
    (attempt, record) for attempt, record in attempts
    if not has_first_output(attempt)
    and is_number(attempt.get("durationMs"))
    and 198_000 <= attempt["durationMs"] < 202_000
]
no_first_byte_199500_200500 = [
    (attempt, record) for attempt, record in attempts
    if not has_first_output(attempt)
    and is_number(attempt.get("durationMs"))
    and 199_500 <= attempt["durationMs"] < 200_500
]
no_first_byte_198000_204000 = [
    (attempt, record) for attempt, record in attempts
    if not has_first_output(attempt)
    and is_number(attempt.get("durationMs"))
    and 198_000 <= attempt["durationMs"] < 204_000
]
no_first_byte_199900_200100 = [
    (attempt, record) for attempt, record in attempts
    if not has_first_output(attempt)
    and is_number(attempt.get("durationMs"))
    and 199_900 <= attempt["durationMs"] < 200_100
]
combo_no_first_byte_199900_200100 = [
    (attempt, record) for attempt, record in no_first_byte_199900_200100
    if (record.get("routeDecision") or {}).get("routeKind") == "combo"
]
no_first_byte_all_prefix = [
    (attempt, record) for attempt, record in attempts
    if not has_first_output(attempt)
]
no_first_byte_status_counts = collections.Counter(
    attempt.get("status") for attempt, _ in no_first_byte_all_prefix
)
all_attempts_198000_200000_status_counts = collections.Counter(
    attempt.get("status") for attempt, _ in all_attempts_198000_200000
)
all_attempts_198000_200000_route_counts = collections.Counter(
    (record.get("routeDecision") or {}).get("routeKind")
    for _, record in all_attempts_198000_200000
)
all_attempts_198000_200000_first_byte_values = [
    attempt["firstOutputMs"] for attempt, _ in all_attempts_198000_200000
    if has_first_output(attempt)
]
all_attempts_198000_200000_first_byte_under_60_seconds = sum(
    value < 60_000 for value in all_attempts_198000_200000_first_byte_values
)
failover_rescued = [
    (attempt, record) for attempt, record in band_attempts
    if attempt.get("status") != 200 and record.get("status") == 200
]
failover_rescued_wide = [
    (attempt, record) for attempt, record in no_first_byte_198000_202000
    if attempt.get("status") != 200 and record.get("status") == 200
]
combo_successes = sorted(
    attempt["firstOutputMs"]
    for attempt, record in attempts
    if (record.get("routeDecision") or {}).get("routeKind") == "combo"
    and succeeded(attempt)
    and has_first_output(attempt)
)
combo_stalls = [
    (attempt, record)
    for attempt, record in no_first_byte_near_timeout
    if (record.get("routeDecision") or {}).get("routeKind") == "combo"
]
combo_savings_ms = [attempt["durationMs"] - 190_000 for attempt, _ in combo_stalls]
combo_p99_position = 0.99 * (len(combo_successes) - 1)
combo_p99_lower_index = math.floor(combo_p99_position)
combo_p99_linear_ms = (
    combo_successes[combo_p99_lower_index]
    + (combo_successes[combo_p99_lower_index + 1] - combo_successes[combo_p99_lower_index])
    * (combo_p99_position - combo_p99_lower_index)
)
stall_days_utc = {
    datetime.datetime.fromtimestamp(record["timestamp"] / 1000, datetime.timezone.utc).date().isoformat()
    for attempt, record in no_first_byte_near_timeout
    if record.get("timestamp") is not None
}
client_cancel_attempts_2 = sum(
    record.get("closeReason") == "client_cancel"
    and len(record.get("attempts") or []) == 2
    for attempt, record in no_first_byte_near_timeout
)
error_status_stalls = sum(
    record.get("status") in (429, 502, 504)
    for attempt, record in no_first_byte_near_timeout
)
parent_outcomes = {}
for attempt, record in no_first_byte_near_timeout:
    key = (
        record.get("status"),
        record.get("closeReason"),
        len(record.get("attempts") or []),
    )
    parent_outcomes[key] = parent_outcomes.get(key, 0) + 1

print("attempts", len(attempt_values))
print("prefix_boundary_1_indexed_line", last_line)
print("prefix_boundary_timestamp_ms", last_timestamp)
print("route_kind_source", "request_parent")
print("route_kind_counts")
for route_kind in ("native", "explicit-provider", "combo", "default-provider"):
    print(route_kind, sum(
        (record.get("routeDecision") or {}).get("routeKind") == route_kind
        for _, record in attempts
    ))
print("attempt_route_kind_fields", sum("routeKind" in attempt for attempt in attempt_values))
print("timeout_band_attempts", len(band_attempts))
print("all_attempts_198000_200000", len(all_attempts_198000_200000))
print("all_attempts_198000_200000_status_counts", dict(sorted(all_attempts_198000_200000_status_counts.items())))
print("all_attempts_198000_200000_route_counts", dict(sorted(all_attempts_198000_200000_route_counts.items())))
print("all_attempts_198000_200000_first_byte_count", len(all_attempts_198000_200000_first_byte_values))
print("all_attempts_198000_200000_first_byte_under_60_seconds", all_attempts_198000_200000_first_byte_under_60_seconds)
print("all_attempts_198000_200000_first_byte_min_ms", min(all_attempts_198000_200000_first_byte_values))
print("all_attempts_exact_200000", sum(
    is_number(a.get("durationMs")) and a["durationMs"] == 200_000
    for a, _ in attempts
))
print("no_first_byte_all_prefix", len(no_first_byte_all_prefix))
print("no_first_byte_status_counts", dict(sorted(no_first_byte_status_counts.items())))
print("no_first_byte_198000_202000", len(no_first_byte_198000_202000))
print("no_first_byte_198000_202000_failover_rescued", len(failover_rescued_wide))
print("no_first_byte_199500_200500", len(no_first_byte_199500_200500))
print("no_first_byte_200000_200020", len(no_first_byte_near_timeout))
print("no_first_byte_198000_204000", len(no_first_byte_198000_204000))
print("no_first_byte_199900_200100", len(no_first_byte_199900_200100))
print("combo_no_first_byte_199900_200100", len(combo_no_first_byte_199900_200100))
print("stalls", len(no_first_byte_near_timeout))
print("stall_status_502", sum(a.get("status") == 502 for a, _ in no_first_byte_near_timeout))
print("stall_status_504", sum(a.get("status") == 504 for a, _ in no_first_byte_near_timeout))
print("combo_stalls", len(combo_stalls))
print("combo_stall_status_502", sum(a.get("status") == 502 for a, _ in combo_stalls))
print("combo_stall_status_504", sum(a.get("status") == 504 for a, _ in combo_stalls))
print("band_failover_rescued", len(failover_rescued))
print("tight_failover_rescued", len(failover_rescued))
print("slow_successes_at_or_after_190_seconds", sum(
    succeeded(a) and has_first_output(a) and a["firstOutputMs"] >= 190_000
    for a in attempt_values
))
print("route_kind_cross_table slow_successes_at_or_after_190_seconds stalls")
for route_kind in ("native", "explicit-provider", "combo", "default-provider"):
    print(route_kind, sum(
        (record.get("routeDecision") or {}).get("routeKind") == route_kind
        and succeeded(a) and has_first_output(a) and a["firstOutputMs"] >= 190_000
        for a, record in attempts
    ), sum(
        (record.get("routeDecision") or {}).get("routeKind") == route_kind
        for a, record in no_first_byte_near_timeout
    ))
print("successes_by_first_byte_between_30_and_120_seconds", sum(
    succeeded(a) and has_first_output(a)
    and 30_000 <= a["firstOutputMs"] <= 120_000
    for a in attempt_values
))
print("successes_by_total_duration_between_30_and_120_seconds", sum(
    succeeded(a) and is_number(a.get("durationMs"))
    and 30_000 <= a["durationMs"] <= 120_000
    for a in attempt_values
))
print("successful_attempts_with_first_byte_status_200_recorded", len(first_byte_successes))
print("p99_first_byte_linear_interpolation_ms", f"{p99_linear_ms:.3f}")
print("p99_first_byte_nearest_rank_ms", p99_nearest_rank_ms)
print("combo_successes_with_first_output_status_200_recorded", len(combo_successes))
print("combo_p99_first_byte_linear_interpolation_ms", f"{combo_p99_linear_ms:.3f}")
print("combo_p99_first_byte_nearest_rank_ms", combo_successes[math.ceil(0.99 * len(combo_successes)) - 1])
print("combo_max_first_byte_ms", max(combo_successes))
for threshold_seconds in (60, 120, 190):
    print(f"combo_successes_at_or_after_seconds_{threshold_seconds}", sum(
        value >= threshold_seconds * 1000 for value in combo_successes
    ))
print("combo_stall_duration_min_ms", min(a["durationMs"] for a, _ in combo_stalls))
print("combo_stall_duration_max_ms", max(a["durationMs"] for a, _ in combo_stalls))
print("combo_savings_vs_190_seconds_min", f"{min(combo_savings_ms) / 1000:.3f}")
print("combo_savings_vs_190_seconds_max", f"{max(combo_savings_ms) / 1000:.3f}")
print("combo_savings_vs_190_seconds_total", f"{sum(combo_savings_ms) / 1000:.3f}")
print("stall_days_utc", sorted(stall_days_utc))
print("client_cancel_parent_attempts_2", client_cancel_attempts_2)
print("error_status_stalls", error_status_stalls)
print("stall_parent_outcomes", sorted((str(key), value) for key, value in parent_outcomes.items()))
print("boundary_utc", datetime.datetime.fromtimestamp(last_timestamp / 1000, datetime.timezone.utc).isoformat().replace("+00:00", "Z"))
PY

The status fields sit at two levels. stall_status_502 = 18 and stall_status_504 = 1 are the attempt-level statuses of the 19 stalled attempts. stall_parent_outcomes is the parent request outcome for those same 19: 5 were rescued by failover (parent 200), 11 were cancelled by the client (parent 499) on requests that all carried exactly two attempts, meaning the stalled attempt had already hit the 200 s cap and failover had already started a second attempt before the client gave up, and 3 ended in an error status (429/502/504). The client cancellations follow the stall; they do not replace it.

Output from the script above was generated by redirecting its stdout to a file and then diffed against this block.

attempts 134716
prefix_boundary_1_indexed_line 155335
prefix_boundary_timestamp_ms 1788409895971
route_kind_source request_parent
route_kind_counts
native 97290
explicit-provider 22980
combo 14400
default-provider 46
attempt_route_kind_fields 0
timeout_band_attempts 19
all_attempts_198000_200000 23
all_attempts_198000_200000_status_counts {200: 20, 502: 3}
all_attempts_198000_200000_route_counts {'combo': 5, 'explicit-provider': 6, 'native': 12}
all_attempts_198000_200000_first_byte_count 19
all_attempts_198000_200000_first_byte_under_60_seconds 14
all_attempts_198000_200000_first_byte_min_ms 1178
all_attempts_exact_200000 0
no_first_byte_all_prefix 49074
no_first_byte_status_counts {200: 45604, 400: 8, 429: 1715, 499: 62, 500: 4, 502: 1671, 503: 5, 504: 3, 520: 2}
no_first_byte_198000_202000 28
no_first_byte_198000_202000_failover_rescued 5
no_first_byte_199500_200500 21
no_first_byte_200000_200020 19
no_first_byte_198000_204000 30
no_first_byte_199900_200100 21
combo_no_first_byte_199900_200100 20
stalls 19
stall_status_502 18
stall_status_504 1
combo_stalls 18
combo_stall_status_502 18
combo_stall_status_504 0
band_failover_rescued 5
tight_failover_rescued 5
slow_successes_at_or_after_190_seconds 44
route_kind_cross_table slow_successes_at_or_after_190_seconds stalls
native 2 0
explicit-provider 42 1
combo 0 18
default-provider 0 0
successes_by_first_byte_between_30_and_120_seconds 2057
successes_by_total_duration_between_30_and_120_seconds 12249
successful_attempts_with_first_byte_status_200_recorded 85083
p99_first_byte_linear_interpolation_ms 56582.820
p99_first_byte_nearest_rank_ms 56664
combo_successes_with_first_output_status_200_recorded 6144
combo_p99_first_byte_linear_interpolation_ms 25727.730
combo_p99_first_byte_nearest_rank_ms 25895
combo_max_first_byte_ms 155929
combo_successes_at_or_after_seconds_60 8
combo_successes_at_or_after_seconds_120 1
combo_successes_at_or_after_seconds_190 0
combo_stall_duration_min_ms 200008
combo_stall_duration_max_ms 200019
combo_savings_vs_190_seconds_min 10.008
combo_savings_vs_190_seconds_max 10.019
combo_savings_vs_190_seconds_total 180.195
stall_days_utc ['2026-08-30', '2026-09-02', '2026-09-03']
client_cancel_parent_attempts_2 11
error_status_stalls 3
stall_parent_outcomes [('(200, None, 2)', 5), ("(429, 'non_stream', 2)", 1), ("(499, 'client_cancel', 2)", 11), ("(502, 'non_stream', 1)", 1), ("(504, 'non_stream', 1)", 1)]
boundary_utc 2026-09-03T04:31:35.971000Z

The output block above is the same stdout produced by the command. Its route counts, cross-table, combo thresholds, combo p99 values, combo maximum, and 190-second savings are all generated values; no output was hand-entered.

The p99 estimators are explicit: the all-attempt successful-first-byte p99 (status 200 with a recorded first byte) is linear interpolation at 0.99 * (n - 1) = 56,582.820 ms and nearest-rank at ceil(0.99 * n) - 1 = 56,664 ms; the combo-only p99 (status 200 with a recorded first byte) uses the same two methods and is 25,727.730 ms (linear interpolation) / 25,895 ms (nearest-rank). Both all-attempt estimators round to the published 57 s.

The published 23, 28, and 21 counts are copied verbatim from the issue comments above; their populations are now identified by the reproduced bands, and the tight-window 19 is the standardized criterion.

[lidge-jun]

Closing this as a maintainer decision on the size of the win, and I want to be explicit that it is my call rather than a concession you made — your final comment withdrew the earlier anti-deadline argument precisely because its cost data was non-combo.

Thank you for correcting the measurements publicly twice rather than letting the first numbers stand. That is unusually careful reporting, and the route decomposition is what made this decidable.

The final figures put timeout-shaped stalls at 19 in 134,716 attempts (0.141 per 1000, across 3 of 6 days), and your combo-only decomposition narrows that to 18 stalls against 14,400 combo attempts. Failover rescued 5; the other 14 ran the full 200 s cap.

On combo-only data a 190 s deadline would abort 0 healthy responses, but every observed stall landed at 200.008–200.019 s, so it would save about 10 s per stall — roughly 180 s across a six-day sample. A tighter threshold buys more and starts costing real responses: 1 combo success sits at or above 120 s and 8 at or above 60 s. Against that, a second per-attempt deadline alongside connectTimeoutMs — documented as a header-arrival budget at src/types/config.ts:672 and again at :1058 — leaves operators with two interacting deadlines to reason about for a 0.13% event that already partially self-heals. That trade is not worth the configuration surface today.

If the stall rate changes materially — a provider that regularly holds connections without sending headers, or a window where failover rescues few of them — please reopen with the new sample. The measurement methodology you built here would make that a quick decision.