diff --git a/devlog/_plan/260915_godfile_round5/070_core_outcome.md b/devlog/_plan/260915_godfile_round5/070_core_outcome.md new file mode 100644 index 0000000000..3595d9ea21 --- /dev/null +++ b/devlog/_plan/260915_godfile_round5/070_core_outcome.md @@ -0,0 +1,118 @@ +# 070 core.ts 편입 (계획 변경 기록) + +000_plan.md 은 `src/server/responses/core.ts` 를 라운드5 범위 밖으로 선언했다. 별도 워크트리에서 +다른 에이전트가 담당하고 있었기 때문이다. 그 작업이 완료돼 이 라운드로 편입했으므로 그 선언을 정정한다. + +## 무엇이 들어왔는가 + +`src/server/responses/core.ts` 9,386 -> 210줄. 리프 24개가 `src/server/responses/` 하위에 생겼다. +가장 큰 리프는 `passthrough-dispatch.ts` 1,476줄이고 전부 2,000줄 아래다. + +앞선 네 단위와 결정적으로 다른 점이 하나 있다. 나머지는 순수 이동이었지만 이것은 아니다. +`handleResponsesInner` 는 약 5,600줄짜리 단일 함수였고, 그 본문을 13개 처리 구간으로 나눴다. +계정 교체나 재시도 후에도 같은 상태를 보아야 하는 값들은 복사하지 않고 원래 지역 변수에 연결된 +getter/setter 로 넘긴다: 전송 예산, adapter, 인증 snapshot, 도구 별칭, 취소 상태, continuation 재시도 횟수. + +이 결정이 라운드5 의 `serveOptions` 추출과 같은 성질이다. 거기서도 가변 캡처 3개를 구조 분해하면 +스냅샷이 되어 조용히 깨졌다. 여기서는 그 대상이 6종이고, 대상이 하나라도 값 복사로 새면 계정 교체 +직후의 재시도가 이전 계정의 예산과 adapter 를 들고 돌아간다. + +## 오라클 처리 방식이 더 낫다 + +라운드5 는 오라클을 손으로 재지정했고 두 번 놓쳤다. bridge 에서는 경로가 조립돼 있어서 리터럴 검색이 +못 봤고(CI 에서 `Received value does not have a length property: null`), server/index 에서는 같은 파일 +안 세 번째 describe 를 시뮬레이션이 빠뜨렸다. + +core.ts 쪽은 `tests/helpers/responses-core-source.ts` 에 모듈 목록을 상수로 두고 +`readResponsesCoreSource()` 가 그 전부를 이어 읽는다. 그리고 `tests/responses/responses-core-modules.test.ts` 가 +그 목록이 실제 소스 import 그래프와 일치하는지 단언한다. 리프를 추가하고 목록에 넣지 않으면 그 테스트가 +실패하므로, 오라클이 조용히 vacuous 해지는 경로가 닫힌다. 다음 라운드는 이 방식을 먼저 쓴다. + +## 이 라운드의 최종 상태 + +| 파일 | 이전 | 이후 | +| --- | ---: | ---: | +| src/adapters/openai-responses.ts | 2,627 | 6 | +| src/bridge.ts | 2,206 | 7 | +| src/server/index.ts | 3,400 | 893 | +| src/server/responses/core.ts | 9,386 | 210 | + +이로써 `src/` 의 산출물 제외 2,000줄 이상 파일은 0개가 된다. 산출물은 +`src/adapters/cursor/gen/agent_pb.ts`(15,274) 하나이고 ratchet 의 generated 목록에 있다. + +## 남은 것 + +`handleResponsesInner` 는 사라졌지만 그 자리에 1,476줄짜리 `passthrough-dispatch.ts` 가 있다. +2,000줄 게이트는 통과하지만 한 파일이 하나의 일을 한다고 말하기는 어렵다. 다음 라운드의 후보는 +줄 수가 아니라 이런 "게이트는 통과하는데 여전히 큰" 리프들이다. + + +## 이 산출물에 대한 편입 검토 + +읽기만 하고 판정한 평가를 남긴다. 편입을 결정한 근거이자, 다음 라운드가 무엇을 고칠지의 목록이다. + +설계는 이 라운드의 다른 네 건보다 어렵고 결과도 낫다. 나머지는 전부 순수 이동이었고 이것은 +저장소에서 가장 위험한 핫 경로를 실제로 재구성했다. `handleResponsesInner` 가 85줄 파이프라인이 됐고 +각 단계가 상태 객체 아니면 `Response` 를 반환해서 `if (x instanceof Response) return x` 한 줄로 원본의 +조기 반환을 보존한다. 예외로 흐름을 바꾸는 방식을 택하지 않았고, admission lease 의 바깥쪽 `finally` 도 +최상위에 그대로 남아 있다. + +가변 상태 처리가 특히 정확하다. getter/setter 의 타입을 새로 적지 않고 `typeof rateLimitRetries` 처럼 +원래 지역 변수에 묶어 썼다. 타입을 따로 적어두면 나중에 원본만 바뀌어 조용히 어긋난다. 라운드5 의 +`serveOptions` 추출이 같은 함정을 만났고, 이쪽이 더 깔끔하다. + +`responses-core-modules.test.ts` 는 이 라운드에서 가장 값어치 있는 장치다. `core.ts` 에서 형제 import 를 따라 +그래프를 걷고, 발견된 소유자 집합이 선언된 목록과 양방향으로 같은지 단언하고, 각 모듈이 2,000줄 미만인지 +확인하고, 그래프가 비순환인지까지 본다. 라운드5 는 오라클을 손으로 재지정하다 두 번 놓쳤다(bridge 는 +CI 가, server/index 는 감사자가 잡았다). 이 방식은 그 경로를 구조적으로 닫는다. + +새 모듈 24개에 타입 검사나 린트를 끄는 주석이 하나도 없다. 억제로 통과시킨 자리가 없다는 뜻이다. + +### 걸리는 것 두 가지 + +단계 함수가 위치 인자를 최대 8개 받는다. `deliverAdapterResponse(requestContext, requestState, +transportState, sidecarState, responseEffects, completionPolicy, adapterExchange, continuationState)` +같은 모양이고, 타입이 겹치는 인접 인자 두 개가 바뀌어도 컴파일된다. 라운드4 계획이 제안했던 단일 +`ResponsesTurnState` 객체라면 이 위험이 없다. "상태가 인자 목록으로 샌다" 는 비용을 실제로 지불한 자리다. + +`passthrough-dispatch.ts` 가 1,476줄이다. 게이트는 통과하지만 한 파일이 한 가지 일을 한다고 말하기 +어렵고, 덩어리가 `core.ts` 에서 그 옆으로 옮겨간 면이 있다. 이름도 두 계열로 갈린다. +`request-prepare`, `passthrough-delivery` 는 책임으로 지었고 `core-auth`, `core-errors`, +`core-normalize` 는 "예전에 core.ts 에 있었다" 는 출처 표시일 뿐이다. 후자는 시간이 지나면 의미가 없다. + +### 편입 과정에서 고친 것 + +`bun x tsc --noEmit` 을 실제로 돌리니 `TS4058` 한 건이 나왔다. `passthrough-dispatch.ts:143` 의 +`preparePassthroughExchange` 가 export 되면서 추론 반환 타입에 `NamespacedTool` 이 노출되는데, 그 인터페이스는 +`src/server/responses-image-gen-repair.ts` 에서 export 되지 않아 이름을 지을 수 없었다. 인터페이스를 +export 해서 해결했다. 원본이 한 파일이었을 때는 그 타입이 모듈 밖으로 나가지 않아 드러나지 않던 종류다. + +이 오류는 그 워크트리에 `node_modules` 가 없어 진짜 typecheck 를 못 돌린 탓이고, 담당 에이전트가 +"테스트·타입체크·빌드는 실행하지 않았다" 고 먼저 밝혔다. 편입 쪽에서 주 체크아웃의 `node_modules` 를 +링크해 실제 typecheck 를 돌려 잡았다. 다음 라운드는 이 링크를 먼저 걸고 시작한다 — CI 한 바퀴가 +로컬 30초보다 비싸다. + + +## 편입 검증 결과 + +독립 감사자가 읽기 전용으로 네 항목을 재측정해 전부 통과했다. 기록할 값어치가 있는 부분만 남긴다. + +가변 상태는 실제로 accessor 로 연결돼 있다. 선언이 모두 소유 함수 안의 `let` 이고 반환 객체의 accessor 가 +그 바인딩을 닫는다. 전송 예산은 `request-send-budget.ts` 의 `pendingHopPermit` get/set 이고 리프 write 는 +`passthrough-dispatch.ts` 1142-1144 다. adapter 와 OAuth snapshot, failover 카운터는 `request-transport.ts` +91-111 선언 / 652-733 get/set 이고 리프가 `transportState.anthropicPoolFailovers += 1` 처럼 쓴다. +continuation 재시도 카운터는 `adapter-dispatch.ts` 345 의 `let rateLimitRetries` 로 recovery loop **바깥**에 +있고 934-938 get/set 을 통해 `adapter-continuation.ts` 266 이 증가시킨다. 루프 안쪽에 있었다면 재시도마다 +0 으로 돌아가 무한 재시도가 된다. 구조 분해 후 대입하는 위험 패턴은 해당 필드에 없다. + +값 순환도 없다. 리프 24개와 `core.ts` 그래프에 `from "./core"` 가 값·타입 모두 없다. +`compact.ts` 와 `policy-fallback.ts` 가 파사드를 값으로 import 하지만 `core.ts` 가 그 둘을 import 하지 +않으므로 단방향이다. combo 재진입은 `core.ts` 182 에서 만든 `requestDispatchers` 를 주입받아 +`request-prepare.ts` 214 와 `core-combo.ts` 478 이 호출한다. + +admission lease 는 두 owner 가 분리돼 있다. 바깥 finally 는 `core.ts` 174-178, native 이관은 +`passthrough-execution.ts` 26-27 에서 `pendingHostAdmissionLease` 를 native 쪽으로 옮기고 null 로 비운 뒤 +48-52 의 native finally 가 받는다. adapter/runTurn 경로는 pending 을 비우지 않으므로 바깥만 해제한다. +`releaseUpstreamHostAdmission` 이 `activeLeaseIds` 불일치 시 no-op 이고 probe 해제도 id 불일치면 return +하므로 이중 해제 경로가 아니다. + diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 72508aaf6f..885d2abcd4 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -167,6 +167,7 @@ } }, "explicit": { + "responses-core-modules.test.ts": "responses", "chat-responses-control-integration.test.ts": "responses", "coding-agent-tool-result-images.test.ts": "adapters", "hub-usage.test.ts": "server", diff --git a/src/server/responses-image-gen-repair.ts b/src/server/responses-image-gen-repair.ts index 11eb2a0be8..dd680c1b85 100644 --- a/src/server/responses-image-gen-repair.ts +++ b/src/server/responses-image-gen-repair.ts @@ -2,7 +2,7 @@ import { collectResponsesToolGroups } from "../responses/tool-groups"; import { relaySseWithPayloadRewrite, type SsePayloadRewrite } from "./sse-payload-rewrite"; import type { TranslatorBudget } from "../lib/translator-budget"; -interface NamespacedTool { +export interface NamespacedTool { namespace: string; name: string; } diff --git a/src/server/responses/adapter-continuation.ts b/src/server/responses/adapter-continuation.ts new file mode 100644 index 0000000000..5201af2d54 --- /dev/null +++ b/src/server/responses/adapter-continuation.ts @@ -0,0 +1,509 @@ +import type { ResponsesRequestContext } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResponsesSidecarAuth } from "./request-sidecar-auth"; +import type { ResponsesSendBudget } from "./request-send-budget"; +import type { AdapterExchange } from "./adapter-dispatch"; +import type { OcxParsedRequest, AdapterEvent } from "../../types"; +import type { AttemptRecoveryKind } from "../../usage/log"; +import type { AdapterRequest } from "../../adapters/base"; +import { + recordAdapterReasoning, + recordAdapterTier, + noteAttemptSend, + sealRequestAttemptIdentity, + recordAttemptCredentialSource, +} from "../request-log"; +import { waitForProviderRequestSlot } from "../../providers/request-pacing"; +import { providerFetch, fetchWithHeaderTimeout, safeHostLabel } from "./fetch-helpers"; +import { + transientRetryPolicyFor, + rateLimitRetryDelayMs, + hasKeyPoolFailover, + rotateProviderTransportOn429, +} from "../../providers/key-failover"; +import { + fetchWithTransientRetry, + fetchWithResetRetry, + applyUpstreamRecoveryInit, + prepareSameTarget429Wait, +} from "../../lib/upstream-retry"; +import { redactSecretString } from "../../lib/redact"; +import { resolveWireProtocolOverride } from "../adapter-resolve"; +import { bindRouteReasoningReplayScope } from "./core-replay"; +import { + ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST, + rotateAnthropicAccountOn429, + getAnthropicPoolAccessSnapshot, + formatAnthropicProviderForLog, +} from "../../oauth/anthropic-routing"; +import { + GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST, + isGenericOAuthFailoverEnabled, + rotateGenericOAuthAccountOn429, + failoverAccountSnapshot, +} from "../../oauth/generic-account-failover"; +import { shouldAttemptImageTierRetry } from "../image-retry"; +import { readDisplaySafeErrorText, normalizeUpstreamErrorText } from "./core-errors"; +import { isCyberPolicyCode, CYBER_POLICY_FALLBACK_MESSAGE, CYBER_POLICY_ERROR_CODE } from "../../lib/errors"; +import { cancelBodyOnAbort } from "../../lib/abort"; +import { guardTerminalEventStream } from "./terminal-guard"; + +/** One responsibility of the Responses request pipeline; state owners are explicit. */ +export function createAdapterContinuations( + requestContext: Pick, + requestState: Pick< + PreparedResponsesRequest, + | "route" + | "selectedForwardHeaders" + | "translatorBudget" + | "inboundWire" + | "parsed" + >, + transportState: Pick< + ResponsesTransport, + | "activeAdapter" + | "sameTargetRequest" + | "sameTargetParsed" + | "sameTargetToken" + | "transportToken" + | "imageTierBias" + | "oauthDispatch" + | "invalidateSameTargetRequest" + | "resolveSelectionAdapter" + | "anthropicPoolAccountId" + | "anthropicPoolFailovers" + | "anthropicSessionKey" + | "commitResolvedOAuthSelection" + | "genericFailoverAccountId" + | "genericFailovers" + | "applyFailoverSnapshot" + >, + sidecarState: Pick, + sendBudgetState: Pick< + ResponsesSendBudget, + | "adapterSendBudget" + | "noteAdapterPhysicalSend" + | "remainingTransientSendBudget" + | "noteTransientSends" + | "reserveCredentialHop" + >, + adapterExchange: Pick< + AdapterExchange, + | "upstream" + | "connectMs" + | "rateLimitPolicy" + | "rateLimitRetries" + | "stallTimeoutMs" + >, +) { + const { options, logCtx, config } = requestContext; + const { + oauthDispatch, + invalidateSameTargetRequest, + resolveSelectionAdapter, + anthropicSessionKey, + commitResolvedOAuthSelection, + applyFailoverSnapshot, + } = transportState; + const { route, translatorBudget, inboundWire, parsed } = requestState; + const { routedCompaction } = sidecarState; + const { upstream, connectMs, rateLimitPolicy, stallTimeoutMs } = adapterExchange; + const { + adapterSendBudget, + noteAdapterPhysicalSend, + remainingTransientSendBudget, + noteTransientSends, + reserveCredentialHop, + } = sendBudgetState; + + + // One bounded internal continuation re-ask for clean end_turn turns that announced an edit + // without emitting a tool call. Anthropic gets this by default; openai-chat providers opt in + // per-provider via `terminalContinuationGuard` (the heuristic was tuned on Anthropic turns, + // so it stays off for the shared openai-chat adapter unless a provider enables it). + const terminalGuardEnabled = (transportState.activeAdapter.name === "anthropic" + || (transportState.activeAdapter.name === "openai-chat" && route.provider.terminalContinuationGuard === true)) + && !options.comboAttempt && !routedCompaction; + /** + * One bounded internal re-ask for Anthropic end_turn-without-tool-call turns. Replays the + * continuation on a 429 with the same-key retry budget (hoisted per request), then falls + * back to key/account failover; a failure becomes an in-stream adapter error so the client + * never sees a second hidden HTTP response or an unbounded retry loop. + */ + const fetchTerminalGuardContinuation = async function* ( + nextParsed: OcxParsedRequest, + initialRecoveryKind?: AttemptRecoveryKind, + ): AsyncGenerator { + let response: Response | undefined; + // One-shot recovery label for the next top-of-loop continuation send after a failover rotation. + let nextContinuationRecoveryKind: AttemptRecoveryKind | undefined = initialRecoveryKind; + /** + * Build and fetch one terminal-guard continuation. `recoveryKind` tags same-target and + * failover sends (`empty-completion`, `rate-limit-429`, `key-429`, + * `anthropic-oauth-429`, `image-413`); the + * adapter rebuild is deterministic for the same parsed request (tests assert byte-identical + * replays). + */ + const fetchContinuation = async (recoveryKind?: AttemptRecoveryKind): Promise => { + let continuationRequest: AdapterRequest | undefined; + if (transportState.sameTargetRequest !== undefined && transportState.sameTargetParsed === nextParsed && transportState.sameTargetToken === transportState.transportToken) { + // Same target (key/adapter/parsed/tier unchanged): replay the exact cached request. + continuationRequest = transportState.sameTargetRequest; + } else { + try { + continuationRequest = await transportState.activeAdapter.buildRequest(nextParsed, { + headers: requestState.selectedForwardHeaders, + translatorBudget, + ...(transportState.imageTierBias > 0 ? { imageTierBias: transportState.imageTierBias } : {}), + }); + recordAdapterReasoning(logCtx, continuationRequest); + recordAdapterTier(logCtx, continuationRequest); + } catch (err) { + // The main body is already streaming, so there is no HTTP error surface: release + // any partial body observation and surface the failure as an in-stream error via + // the outer catch (no upstream.abort() — that would kill the live body stream). + continuationRequest?.releaseBodyObservation?.(); + throw err; + } + transportState.sameTargetRequest = continuationRequest; + transportState.sameTargetParsed = nextParsed; + transportState.sameTargetToken = transportState.transportToken; + } + // Both branches assign the request (the build catch rethrows), so capture it in a + // const for the fetch callback and finally below — a `let` read inside a nested + // function keeps its undefined half, which would break the byte-identical replay. + const builtContinuationRequest = continuationRequest; + const continuationEstimate = typeof builtContinuationRequest.usageLog?.inputTokens === "number" + ? builtContinuationRequest.usageLog.inputTokens + : undefined; + if (continuationEstimate !== undefined) logCtx.usageLogInputTokens = continuationEstimate; + // Optional recovery label for same-target / failover continuation sends. + const replayKind: AttemptRecoveryKind | undefined = recoveryKind; + try { + if (transportState.activeAdapter.fetchResponse) { + noteAttemptSend(logCtx.activeAttempt, continuationEstimate, replayKind); + await waitForProviderRequestSlot(route.providerName, route.provider, nextParsed.modelId, upstream.signal); + return await transportState.activeAdapter.fetchResponse(builtContinuationRequest, { + abortSignal: upstream.signal, + timeoutMs: connectMs, + sendBudget: adapterSendBudget, + onPhysicalSend: send => noteAdapterPhysicalSend(continuationEstimate, send), + stream: nextParsed.stream, + executor: providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(builtContinuationRequest, nextParsed), + providerName: route.providerName, + modelId: nextParsed.modelId, + }), + }); + } + // Same #1851 scope guard as the initial send: transient-5xx retry only for direct + // Google AI Studio; every other adapter keeps reset-only semantics here. + const continuationTransientPolicy = transientRetryPolicyFor(route.provider); + const fetchContinuationWithRetryPolicy = (route.provider.adapter === "google" || continuationTransientPolicy) + ? fetchWithTransientRetry + : fetchWithResetRetry; + return await fetchContinuationWithRetryPolicy( + recovery => { + noteAttemptSend(logCtx.activeAttempt, continuationEstimate, recovery ?? replayKind); + return fetchWithHeaderTimeout( + builtContinuationRequest.url, + applyUpstreamRecoveryInit({ + method: builtContinuationRequest.method, + headers: builtContinuationRequest.headers, + body: builtContinuationRequest.body, + }, recovery), + upstream.signal, + connectMs, + nextParsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(builtContinuationRequest, nextParsed), + providerName: route.providerName, + modelId: nextParsed.modelId, + }), + ); + }, + { + abortSignal: upstream.signal, + label: safeHostLabel(builtContinuationRequest.url), + // Same request-scoped budget as the initial send and the 429/rotation refetches: + // a terminal-guard continuation is another leg of ONE request, so handing it a + // fresh `attempts` would let one request exceed the configured total-send ceiling. + ...(continuationTransientPolicy + ? { + attempts: remainingTransientSendBudget(continuationTransientPolicy.attempts), + onSendsConsumed: noteTransientSends, + } + : {}), + }, + ); + } finally { + builtContinuationRequest.releaseBodyObservation?.(); + } + }; + while (true) { + try { + const recoveryKind = nextContinuationRecoveryKind; + nextContinuationRecoveryKind = undefined; + response = await fetchContinuation(recoveryKind); + } catch (error) { + if (options.abortSignal?.aborted || upstream.signal.aborted) { + yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; + } else { + yield { type: "error", message: `Provider continuation failed: ${redactSecretString(error instanceof Error ? error.message : String(error))}` }; + } + return; + } + + // Same-target 429 wait-and-retry (opt-in `retryOn429`) before key/account failover: + // a primary-key rate-limit blip replays on the SAME key, matching the main recovery + // loop; only after the attempts are exhausted does the continuation fail over. + while ( + response.status === 429 + && rateLimitPolicy !== null + && adapterExchange.rateLimitRetries < rateLimitPolicy.attempts + ) { + adapterExchange.rateLimitRetries += 1; + // Release unread body + heartbeat-fed wait via the shared same-target helper. + const retryAfterHeader = response.headers.get("retry-after"); + try { + yield* prepareSameTarget429Wait({ + body: response.body, + // Listen on the upstream signal: once the SSE body is being streamed, a client + // cancel aborts `upstream` through the bridge, and upstream is also linked from + // options.abortSignal — so this covers both cancellation paths. + signal: upstream.signal, + delayMs: rateLimitRetryDelayMs(rateLimitPolicy, retryAfterHeader, Date.now()), + heartbeatIntervalMs: Math.min(10_000, Math.max(250, stallTimeoutMs / 2)), + }); + } catch { + if (options.abortSignal?.aborted || upstream.signal.aborted) { + yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; + } else { + yield { type: "error", message: "Provider continuation failed: retry wait interrupted" }; + } + return; + } + // Client cancellation wins over any stale timer edge: re-check before dispatching the + // replay so the continuation never starts work for a request the client abandoned. + if (options.abortSignal?.aborted || upstream.signal.aborted) { + yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; + return; + } + try { + response = await fetchContinuation("rate-limit-429"); + } catch (error) { + if (options.abortSignal?.aborted || upstream.signal.aborted) { + yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; + } else { + yield { type: "error", message: `Provider continuation failed: ${redactSecretString(error instanceof Error ? error.message : String(error))}` }; + } + return; + } + } + + if (response.status === 429 && hasKeyPoolFailover(route.provider)) { + const rotated = rotateProviderTransportOn429(config, route.providerName, route.provider, { + retryAfter: response.headers.get("retry-after"), + now: Date.now(), + attemptedKey: route.provider.apiKey, + promptCacheKey: nextParsed.options.promptCacheKey, + }); + if (rotated) { + try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } + route.provider = rotated; + invalidateSameTargetRequest(); + transportState.activeAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + bindRouteReasoningReplayScope({ + parsed: nextParsed, + providerName: route.providerName, + provider: route.provider, + adapterName: transportState.activeAdapter.name, + }); + // Response persistence closes over the outer parsed request; keep its owner binding in + // sync with the terminal-guard clone that builds the rotated continuation request. + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: route.provider, + adapterName: transportState.activeAdapter.name, + }); + nextContinuationRecoveryKind = "key-429"; + continue; + } + } + if ( + response.status === 429 + && transportState.anthropicPoolAccountId + && transportState.anthropicPoolFailovers < ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST + ) { + const nextAccountId = rotateAnthropicAccountOn429( + config, + transportState.anthropicPoolAccountId, + response.headers.get("retry-after"), + anthropicSessionKey, + Date.now(), + response.headers, + ); + if (nextAccountId) { + try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } + try { + const admitted = await commitResolvedOAuthSelection(await getAnthropicPoolAccessSnapshot(nextAccountId)); + if (!admitted) throw new Error("OAuth selection changed during recovery"); + transportState.anthropicPoolAccountId = admitted.accountId; + transportState.anthropicPoolFailovers += 1; + route.provider = { ...route.provider, apiKey: admitted.accessToken }; + invalidateSameTargetRequest(); + logCtx.provider = formatAnthropicProviderForLog("anthropic", admitted.accountId, config); + transportState.activeAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, transportState.activeAdapter.name, logCtx.accountLogLabel); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, transportState.activeAdapter.name); + nextContinuationRecoveryKind = "anthropic-oauth-429"; + continue; + } catch { + // fall through to emit continuation error below + } + } + } + // Generic OAuth rotation for the continuation loop. The streaming loop grew this arm with + // #2568 and this one did not, so an xAI/Cursor/Kimi/Copilot/Antigravity/Nous continuation + // 429 stayed terminal even with failover fully active -- the same class of divergence the + // two sidecars already produced once. Request-local state is shared with the other arms so + // the per-request bound cannot be silently re-armed by reaching a different loop. + if ( + response.status === 429 + && transportState.genericFailoverAccountId + && transportState.genericFailovers < GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST + && isGenericOAuthFailoverEnabled(config, route.providerName) + ) { + // Intersection with the shared request budget. The continuation loop re-sends the + // turn, so without this the per-request bound could be re-armed simply by reaching a + // different loop -- which is the divergence the comment above already warns about. + const hop = reserveCredentialHop( + "auth-recovery", + `${route.providerName}|${route.modelId}|continuation-oauth-429`, + ); + const nextAccountId = hop.allowed + ? rotateGenericOAuthAccountOn429( + config, + route.providerName, + transportState.genericFailoverAccountId, + response.headers.get("retry-after"), + ) + : null; + if (!nextAccountId) hop.permit?.release(); + if (nextAccountId) { + try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } + try { + // The FULL snapshot through the shared helper, never a bare bearer: Antigravity + // pairs an account-matched projectId with its token and Kiro carries routing + // metadata, so a token-only swap would mix one account's credential with another's + // routing data. + const snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); + transportState.genericFailovers += 1; + const applied = await applyFailoverSnapshot(snapshot, nextParsed); + if (!applied) hop.permit?.release(); + if (applied) { + invalidateSameTargetRequest(); + transportState.activeAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, transportState.activeAdapter.name, logCtx.accountLogLabel); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, transportState.activeAdapter.name); + nextContinuationRecoveryKind = "oauth-account-429"; + continue; + } + } catch { + // fall through to emit continuation error below + } + } + } + if (shouldAttemptImageTierRetry({ + status: response.status, + adapterName: transportState.activeAdapter.name, + parsed: nextParsed, + alreadyAttempted: transportState.imageTierBias > 0, + })) { + transportState.imageTierBias = 1; + invalidateSameTargetRequest(); + try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } + nextContinuationRecoveryKind = "image-413"; + continue; + } + break; + } + + if (!response.ok) { + const errorText = await readDisplaySafeErrorText(response, upstream.signal, "unknown error"); + const normalized = normalizeUpstreamErrorText(errorText, "unknown error"); + yield { + type: "error", + status: normalized.cyberPolicy ? 400 : response.status, + message: normalized.cyberPolicy + ? normalized.message + ?? (isCyberPolicyCode(normalized.code) ? CYBER_POLICY_FALLBACK_MESSAGE : normalized.safeText) + : `Provider continuation error ${response.status}: ${normalized.safeText}`, + ...(normalized.cyberPolicy + ? { + errorType: normalized.type ?? CYBER_POLICY_ERROR_CODE, + code: CYBER_POLICY_ERROR_CODE, + retryable: false, + } + : {}), + }; + return; + } + + try { + // Protect the continuation body against a client abort landing between fetch resolution and + // reader attach, exactly as the initial response is guarded above (#390/366e3053). Without + // this, a client cancel during the continuation reopens the Bun fetch-to-reader abort race. + const detachContinuationBodyGuard = cancelBodyOnAbort(response.body, upstream.signal); + try { + if (nextParsed.stream) { + yield* transportState.activeAdapter.parseStream(response, translatorBudget, logCtx.activeTierMetadata); + } else if (transportState.activeAdapter.parseResponse) { + yield* await transportState.activeAdapter.parseResponse(response, translatorBudget, logCtx.activeTierMetadata); + } else { + yield { type: "error", message: "Provider continuation does not support response parsing" }; + } + } finally { + detachContinuationBodyGuard(); + } + } catch (error) { + if (options.abortSignal?.aborted) { + yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; + } else { + yield { type: "error", message: `Provider continuation parse failed: ${redactSecretString(error instanceof Error ? error.message : String(error))}` }; + } + } + }; + + const fetchGuardedEmptyCompletionRetry = (): AsyncIterable => { + const retryEvents = fetchTerminalGuardContinuation(parsed, "empty-completion"); + return terminalGuardEnabled + ? guardTerminalEventStream({ + parsed, + firstEvents: retryEvents, + adapterName: transportState.activeAdapter.name, + maxAutoContinuations: 1, + continuation: fetchTerminalGuardContinuation, + }) + : retryEvents; + }; + + return { + terminalGuardEnabled, + fetchTerminalGuardContinuation, + fetchGuardedEmptyCompletionRetry, + }; +} + +export type AdapterContinuations = Exclude, Response>; diff --git a/src/server/responses/adapter-delivery.ts b/src/server/responses/adapter-delivery.ts new file mode 100644 index 0000000000..a46d1faf8b --- /dev/null +++ b/src/server/responses/adapter-delivery.ts @@ -0,0 +1,214 @@ +import type { ResponsesRequestContext } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResponsesSidecarAuth } from "./request-sidecar-auth"; +import type { ResponsesEffects } from "./response-effects"; +import type { ResponsesCompletionPolicy } from "./completion-policy"; +import type { AdapterExchange } from "./adapter-dispatch"; +import type { AdapterContinuations } from "./adapter-continuation"; +import { guardTerminalEventStream } from "./terminal-guard"; +import { guardEmptyCompletionEventStream } from "./empty-completion-guard"; +import { bridgeToResponsesSSE, buildResponseJSON, formatErrorResponse } from "../../bridge"; +import type { OcxProviderContinuationState, AdapterEvent } from "../../types"; +import { rememberResponseState } from "../../responses/state"; +import { trackStreamLifetime } from "../lifecycle"; +import { awaitThoughtSignatureDurability } from "../../responses/thought-signature-replay"; +import { adapterResponseReachedServingTerminal } from "./core-replay"; + +/** One responsibility of the Responses request pipeline; state owners are explicit. */ +export async function deliverAdapterResponse( + requestContext: Pick, + requestState: Pick< + PreparedResponsesRequest, + | "parsed" + | "translatorBudget" + | "toolBridgeMaps" + | "rememberKiroDeliveredFinalAnswer" + | "responseStateOptions" + >, + transportState: Pick, + sidecarState: Pick, + responseEffects: Pick< + ResponsesEffects, + | "cancelResponseCompletion" + | "commitReasoningReplayServingRoute" + | "continuationStateForResponse" + | "notifyResponseComplete" + >, + completionPolicy: Pick, + adapterExchange: Pick, + continuationState: Pick, +): Promise { + const { logCtx, options, config } = requestContext; + const { + parsed, + translatorBudget, + toolBridgeMaps, + rememberKiroDeliveredFinalAnswer, + responseStateOptions, + } = requestState; + const { upstreamResponse, upstream, cleanupUpstreamAbort } = adapterExchange; + const { + terminalGuardEnabled, + fetchTerminalGuardContinuation, + fetchGuardedEmptyCompletionRetry, + } = continuationState; + const { emptyCompletionGuardEnabled } = completionPolicy; + const { + cancelResponseCompletion, + commitReasoningReplayServingRoute, + continuationStateForResponse, + notifyResponseComplete, + } = responseEffects; + const { routedCompaction } = sidecarState; + + + if (parsed.stream) { + const initialEventStream = transportState.activeAdapter.parseStream( + upstreamResponse, + translatorBudget, + logCtx.activeTierMetadata, + ); + const eventStream = terminalGuardEnabled + ? guardTerminalEventStream({ + parsed, + firstEvents: initialEventStream, + adapterName: transportState.activeAdapter.name, + maxAutoContinuations: 1, + continuation: fetchTerminalGuardContinuation, + }) + : initialEventStream; + // The empty-completion guard sits OUTSIDE the terminal guard: a completed + // turn with no text and no tool call is retried with the IDENTICAL request + // (fetchTerminalGuardContinuation(parsed) replays the cached byte-identical + // request — same body, same headers, same signal). + const guardedEventStream = emptyCompletionGuardEnabled + ? guardEmptyCompletionEventStream({ + firstEvents: eventStream, + continuation: fetchGuardedEmptyCompletionRetry, + }) + : eventStream; + const { toolNsMap, declaredToolNames, toolParameterSchemas, freeformToolNames, toolSearchToolNames } = toolBridgeMaps; + const sseStream = bridgeToResponsesSSE( + guardedEventStream, parsed._responseModelId ?? parsed.modelId, toolNsMap, freeformToolNames, toolSearchToolNames, + () => { cancelResponseCompletion(); upstream.abort(); }, 2_000, + { + translatorBudget, + replayCacheScope: parsed._reasoningReplayScope, + ...(options.forceEmptyResponseId ? { responseId: "" } : {}), + stallTimeoutSec: config.stallTimeoutSec, + hideThinkingSummary: parsed.options.hideThinkingSummary, + declaredToolNames, + toolParameterSchemas, + ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), + ...(routedCompaction ? { compaction: true } : {}), + // Same grok-surface split as the runTurn branch above. + ...(logCtx.surface === "grok" ? { heartbeatStyle: "comment" as const } : {}), + onUsage: usage => { + // Raw adapter usage, pre wire-normalization (see the runTurn branch above). + logCtx.usageFromBridge = true; + if (usage) { + logCtx.usage = usage; + if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; + } + }, + onCompletedResponse: (response: Record, providerState?: OcxProviderContinuationState) => { + commitReasoningReplayServingRoute(); + rememberKiroDeliveredFinalAnswer(transportState.activeAdapter.name, response); + // Compaction turns must NOT enter the continuation cache: _rawBody still holds the full + // PRE-compaction history, and a later previous_response_id expansion would rehydrate the + // giant stale chain Codex just replaced. + if (!routedCompaction) { + rememberResponseState( + parsed._rawBody, + response, + continuationStateForResponse(providerState), + responseStateOptions(transportState.activeAdapter.name === "kiro"), + ); + } + notifyResponseComplete(response); + }, + }, + ); + const bridgeTurnAc = new AbortController(); + const trackedSse = trackStreamLifetime(sseStream, bridgeTurnAc, cleanupUpstreamAbort, options.turnAdmissionLease); + return new Response(trackedSse, { + headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache", "Connection": "keep-alive", "X-Accel-Buffering": "no" }, + }); + } + + if (transportState.activeAdapter.parseResponse) { + let events: AdapterEvent[]; + try { + const initialEvents = await transportState.activeAdapter.parseResponse( + upstreamResponse, + translatorBudget, + logCtx.activeTierMetadata, + ); + let guardedEvents: AdapterEvent[]; + if (terminalGuardEnabled) { + guardedEvents = []; + for await (const event of guardTerminalEventStream({ + parsed, + firstEvents: (async function* () { yield* initialEvents; })(), + adapterName: transportState.activeAdapter.name, + maxAutoContinuations: 1, + continuation: fetchTerminalGuardContinuation, + })) guardedEvents.push(event); + } else { + guardedEvents = initialEvents; + } + if (emptyCompletionGuardEnabled) { + events = []; + for await (const event of guardEmptyCompletionEventStream({ + firstEvents: (async function* () { yield* guardedEvents; })(), + continuation: fetchGuardedEmptyCompletionRetry, + })) events.push(event); + } else { + events = guardedEvents; + } + } finally { + cleanupUpstreamAbort(); + } + const { toolNsMap, declaredToolNames, toolParameterSchemas, freeformToolNames, toolSearchToolNames } = toolBridgeMaps; + let providerState: OcxProviderContinuationState | undefined; + const json = buildResponseJSON(events, parsed._responseModelId ?? parsed.modelId, { + translatorBudget, + replayCacheScope: parsed._reasoningReplayScope, + hideThinkingSummary: parsed.options.hideThinkingSummary, + toolNsMap, + declaredToolNames, + toolParameterSchemas, + freeformToolNames, + toolSearchToolNames, + ...(routedCompaction ? { compaction: true } : {}), + onProviderState: state => { providerState = state; }, + onUsage: usage => { + logCtx.usageFromBridge = true; + if (usage) { + logCtx.usage = usage; + if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; + } + }, + }); + // See the streaming branch: compaction turns skip the continuation cache. + if (!routedCompaction) { + rememberKiroDeliveredFinalAnswer(transportState.activeAdapter.name, json); + rememberResponseState( + parsed._rawBody, + json, + continuationStateForResponse(providerState), + responseStateOptions(transportState.activeAdapter.name === "kiro"), + ); + } + // #1926 gap 2: same buffered-path durability bound as the primary branch. + await awaitThoughtSignatureDurability(); + if (adapterResponseReachedServingTerminal(events, json)) { + commitReasoningReplayServingRoute(); + } + notifyResponseComplete(json); + return new Response(JSON.stringify(json), { headers: { "Content-Type": "application/json" } }); + } + + return formatErrorResponse(400, "invalid_request_error", "Non-streaming not supported by this adapter"); +} diff --git a/src/server/responses/adapter-dispatch.ts b/src/server/responses/adapter-dispatch.ts new file mode 100644 index 0000000000..e1757186cf --- /dev/null +++ b/src/server/responses/adapter-dispatch.ts @@ -0,0 +1,943 @@ +import type { ResponsesRequestContext, ResponsesAdmissionState } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResponsesEffects } from "./response-effects"; +import type { ResponsesSendBudget } from "./request-send-budget"; +import { linkAbortSignal } from "./core-lifetime"; +import type { AdapterRequest } from "../../adapters/base"; +import type { AdapterEvent } from "../../types"; +import { bridgeToResponsesSSE, buildResponseJSON, formatErrorResponse } from "../../bridge"; +import { trackStreamLifetime } from "../lifecycle"; +import { + recordAdapterReasoning, + recordAdapterTier, + noteAttemptSend, + sealRequestAttemptIdentity, + recordAttemptCredentialSource, +} from "../request-log"; +import { clientCancelledResponse, readDisplaySafeErrorText, normalizeUpstreamErrorText } from "./core-errors"; +import { redactSecretString } from "../../lib/redact"; +import { waitForProviderRequestSlot } from "../../providers/request-pacing"; +import { providerFetch, fetchWithHeaderTimeout, safeHostLabel } from "./fetch-helpers"; +import { + transientRetryPolicyFor, + rateLimitRetryPolicyFor, + hasKeyPoolFailover, + rotateProviderTransportOn401, + rateLimitRetryDelayMs, + rotateProviderTransportOn429, +} from "../../providers/key-failover"; +import { + fetchWithTransientRetry, + fetchWithResetRetry, + applyUpstreamRecoveryInit, + SendBudgetExhaustedError, + prepareSameTarget429Wait, + sleepWithAbort, +} from "../../lib/upstream-retry"; +import { describeUpstreamConnectFailure } from "./upstream-error"; +import type { OpaqueBlobRecoveryGuard } from "./core-opaque-recovery"; +import type { AttemptRecoveryKind } from "../../usage/log"; +import type { OAuthAccessSnapshot } from "../../oauth"; +import { publicOAuthAuthenticationErrorMessage } from "../../oauth"; +import { resolveProviderTransport } from "../../providers/xai-transport"; +import { resolveCopilotApiBaseUrl } from "../../oauth/github-copilot"; +import { resolveWireProtocolOverride } from "../adapter-resolve"; +import { bindRouteReasoningReplayScope } from "./core-replay"; +import { + ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST, + rotateAnthropicAccountOn429, + getAnthropicPoolAccessSnapshot, + formatAnthropicProviderForLog, +} from "../../oauth/anthropic-routing"; +import { + GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST, + isGenericOAuthFailoverEnabled, + rotateGenericOAuthAccountOn429, + failoverAccountSnapshot, +} from "../../oauth/generic-account-failover"; +import { + attemptOpaqueBlobRecovery, + consoleGoUploadRejectionBody, + CONSOLE_GO_UPLOAD_RETRY_DELAY_MS, + reasoningEffortRejectionText, +} from "./core-opaque-recovery"; +import { shouldAttemptImageTierRetry } from "../image-retry"; +import { + isTransientConsoleGoUploadRejection, + enrichOpenCodeZenUpstreamMessage, +} from "../../providers/opencode-zen-rate-limit"; +import { planReasoningEffortDowngrade } from "../../providers/reasoning-metadata"; +import { consumeComboFailure } from "./core-combo-failure"; +import { streamingContextOverflowResponse, jsonContextOverflowResponse } from "./context-overflow"; +import { isFixedCodexAccount } from "./core-codex-account"; +import { recordSubagentQuotaFailureForThreadSpawn } from "../../codex/subagent-model-fallback"; +import { isCyberPolicyCode, CYBER_POLICY_FALLBACK_MESSAGE, CYBER_POLICY_ERROR_CODE } from "../../lib/errors"; +import { resolveClientRetryAfter } from "../../lib/retry-after"; +import { cancelBodyOnAbort } from "../../lib/abort"; + +/** One responsibility of the Responses request pipeline; state owners are explicit. */ +export async function prepareAdapterExchange( + requestContext: Pick, + admissionState: ResponsesAdmissionState, + requestState: Pick< + PreparedResponsesRequest, + | "parsed" + | "toolBridgeMaps" + | "translatorBudget" + | "selectedForwardHeaders" + | "route" + | "inboundWire" + | "clientRequestedStream" + | "subagentQuotaFailureModel" + | "subagentFallbackAccountId" + >, + transportState: Pick< + ResponsesTransport, + | "activeAdapter" + | "adapter" + | "sameTargetRequest" + | "sameTargetParsed" + | "sameTargetToken" + | "transportToken" + | "oauthDispatch" + | "imageTierBias" + | "isOAuth401ReplayProvider" + | "sentOAuthSnapshot" + | "refreshResolvedOAuthSelection" + | "replayOAuthCredentialSnapshot" + | "invalidateSameTargetRequest" + | "resolveSelectionAdapter" + | "anthropicPoolAccountId" + | "anthropicPoolFailovers" + | "anthropicSessionKey" + | "commitResolvedOAuthSelection" + | "genericFailoverAccountId" + | "genericFailovers" + | "applyFailoverSnapshot" + >, + responseEffects: Pick, + sendBudgetState: Pick< + ResponsesSendBudget, + | "adapterSendBudget" + | "noteAdapterPhysicalSend" + | "remainingTransientSendBudget" + | "noteTransientSends" + | "recoverySendAllowance" + | "recoveryClassFor" + | "sendBudgetExhausted" + | "reserveCredentialHop" + >, +) { + const { options, config, logCtx, req } = requestContext; + const { + oauthDispatch, + isOAuth401ReplayProvider, + refreshResolvedOAuthSelection, + invalidateSameTargetRequest, + resolveSelectionAdapter, + anthropicSessionKey, + commitResolvedOAuthSelection, + applyFailoverSnapshot, + } = transportState; + const { + parsed, + toolBridgeMaps, + translatorBudget, + route, + inboundWire, + clientRequestedStream, + subagentQuotaFailureModel, + } = requestState; + const { cancelResponseCompletion, notifyResponseComplete, refreshRequestToolAliases } = responseEffects; + const { + adapterSendBudget, + noteAdapterPhysicalSend, + remainingTransientSendBudget, + noteTransientSends, + recoverySendAllowance, + recoveryClassFor, + sendBudgetExhausted, + reserveCredentialHop, + } = sendBudgetState; + + + const upstream = new AbortController(); + const cleanupUpstreamAbort = linkAbortSignal(upstream, options.abortSignal); + const connectMs = config.connectTimeoutMs ?? 200_000; + // Bridge stall budget (seconds of silence before upstream_stall_timeout); the retry backoff + // heartbeat interval is derived from it so the watchdog is always fed during deliberate waits. + const stallTimeoutMs = typeof config.stallTimeoutSec === "number" && Number.isFinite(config.stallTimeoutSec) && config.stallTimeoutSec > 0 + ? Math.floor(config.stallTimeoutSec * 1000) + : 300_000; + transportState.activeAdapter = transportState.adapter; + + // One immutable, body-safe outbound request per same-target sequence (URL, serialized body, + // auth headers, generated compat headers). Same-target 429 replays reuse it verbatim; the + // builder runs again only after a key/account/adapter rotation, an oauth refresh, or an + // image-tier bias change (transportToken bump). `body` is always a serialized string, so + // reuse is safe, and releaseBodyObservation is idempotent per build. + let initialRequest: AdapterRequest | undefined; + let inputTokenEstimate: number | undefined; + // An adapter may know the turn needs no inference at all — Kiro's replayed history ending in a + // delivered final answer. Answer it locally: no build (so no token estimate), no send (so + // sendCount stays 0), and crucially no empty-completion guard, which treats an outputless + // terminal as a failed turn and re-invokes the identical request. Routing this through the + // ordinary event path would therefore reinstate the loop it exists to end. + const localTerminal = transportState.activeAdapter.localTerminal?.(parsed); + if (localTerminal) { + logCtx.localTerminalReason = localTerminal.reason; + // Mark the physical attempt too, not just the parent row. `finishRequestAttempt` finalizes the + // attempt through the same estimated-provider path, so without this the row reads exact while + // its own attempt still claims an estimate — the detailed accounting a maintainer actually + // reads for a zero-send turn. + if (logCtx.activeAttempt) logCtx.activeAttempt.locallyAnswered = true; + cleanupUpstreamAbort(); + upstream.abort(); + const terminalEvents: AdapterEvent[] = [{ + type: "done", + endTurn: true, + usage: { inputTokens: 0, outputTokens: 0, totalTokens: 0 }, + }]; + if (parsed.stream) { + const localSse = bridgeToResponsesSSE( + (async function* () { yield* terminalEvents; })(), + parsed._responseModelId ?? parsed.modelId, + toolBridgeMaps.toolNsMap, + toolBridgeMaps.freeformToolNames, + toolBridgeMaps.toolSearchToolNames, + cancelResponseCompletion, + 2_000, + { + translatorBudget, + onCompletedResponse: notifyResponseComplete, + ...(options.forceEmptyResponseId ? { responseId: "" } : {}), + ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), + }, + ); + // Same lifetime tracking as every other streaming return in this function: the turn + // admission lease is released when the body finishes or the client disconnects. Returning + // the raw stream would hold a lease for a turn that already has all of its output. + const localTurnAc = new AbortController(); + return new Response( + trackStreamLifetime(localSse, localTurnAc, undefined, options.turnAdmissionLease), + { + headers: { + "Content-Type": "text/event-stream", + "Cache-Control": "no-cache", + "Connection": "keep-alive", + "X-Accel-Buffering": "no", + }, + }, + ); + } + const json = buildResponseJSON(terminalEvents, parsed._responseModelId ?? parsed.modelId, { translatorBudget }); + notifyResponseComplete(json); + return new Response(JSON.stringify(json), { headers: { "Content-Type": "application/json" } }); + } + try { + initialRequest = await transportState.activeAdapter.buildRequest(parsed, { headers: requestState.selectedForwardHeaders, translatorBudget }); + refreshRequestToolAliases(initialRequest); + recordAdapterReasoning(logCtx, initialRequest); + recordAdapterTier(logCtx, initialRequest); + inputTokenEstimate = typeof initialRequest.usageLog?.inputTokens === "number" + ? initialRequest.usageLog.inputTokens + : undefined; + if (inputTokenEstimate !== undefined) logCtx.usageLogInputTokens = inputTokenEstimate; + } catch (err) { + // A throwing buildRequest never returned a request; if a post-build step threw, release + // the serialized-body observation (idempotent) so the translator budget is not leaked. + // The build runs after linkAbortSignal, so a failure must also tear the link down and + // abort the upstream controller instead of escaping handleResponses unmapped. + initialRequest?.releaseBodyObservation?.(); + cleanupUpstreamAbort(); + upstream.abort(); + if (options.abortSignal?.aborted) return clientCancelledResponse(); + const msg = err instanceof Error ? err.message : String(err); + return formatErrorResponse(400, "invalid_request_error", redactSecretString(msg)); + } + // The catch path above always returns, so the request is definitely assigned here. + // Capture it in a const so the fetch callbacks read a narrowed, immutable value + // (TypeScript drops narrowing for a `let` captured by a nested function). + const builtInitialRequest = initialRequest; + transportState.sameTargetRequest = builtInitialRequest; + transportState.sameTargetParsed = parsed; + transportState.sameTargetToken = transportState.transportToken; + /** + * Invalidate the same-target request cache. Every credential/adapter/parsed mutation MUST + * go through here: the cache keys on `parsed` REFERENCE identity, so an in-place mutation + * is invisible to it and a missed bump would replay a request built with a stale key. + */ + + let upstreamResponse: Response; + try { + if (transportState.activeAdapter.fetchResponse) { + noteAttemptSend(logCtx.activeAttempt, inputTokenEstimate); + await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, upstream.signal); + upstreamResponse = await transportState.activeAdapter.fetchResponse(builtInitialRequest, { + abortSignal: upstream.signal, + timeoutMs: connectMs, + sendBudget: adapterSendBudget, + onPhysicalSend: send => noteAdapterPhysicalSend(inputTokenEstimate, send), + stream: parsed.stream, + executor: providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(builtInitialRequest), + providerName: route.providerName, + modelId: route.modelId, + }), + }); + } else { + // #1851 scope guard: transient-5xx retry on this generic adapter path is opt-in for + // direct Google AI Studio only (Vertex/Antigravity use fetchResponse above). Other + // adapters keep reset-only retry so combo failover still hops on the first 5xx + // instead of burning ~1.2s of same-target retries per hop. + // #2643: an opted-in key-auth openai-chat provider also gets transient-5xx retry. The + // legacy direct-Google exception is preserved exactly; every other adapter still keeps + // reset-only semantics so combo failover hops on the first 5xx. + const transientPolicy = transientRetryPolicyFor(route.provider); + const fetchWithRetryPolicy = (route.provider.adapter === "google" || transientPolicy) + ? fetchWithTransientRetry + : fetchWithResetRetry; + upstreamResponse = await fetchWithRetryPolicy( + recovery => { + noteAttemptSend(logCtx.activeAttempt, inputTokenEstimate, recovery); + return fetchWithHeaderTimeout(builtInitialRequest.url, applyUpstreamRecoveryInit({ + method: builtInitialRequest.method, + headers: builtInitialRequest.headers, + body: builtInitialRequest.body, + }, recovery), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(builtInitialRequest), + providerName: route.providerName, + modelId: route.modelId, + })); + }, + { + abortSignal: upstream.signal, + label: safeHostLabel(builtInitialRequest.url), + ...(transientPolicy + // Draws the remainder, not the raw policy. A combo child inherits the parent's + // holder but used to take a fresh full allowance on its own first send, so the + // shared counter was inherited without ever being read as a limit. + ? { + attempts: remainingTransientSendBudget(transientPolicy.attempts), + onSendsConsumed: noteTransientSends, + } + : {}), + }, + ); + } + } catch (err) { + cleanupUpstreamAbort(); + upstream.abort(); + if (options.abortSignal?.aborted) return clientCancelledResponse(); + const msg = describeUpstreamConnectFailure(err, connectMs); + return formatErrorResponse(502, "upstream_error", msg); + } finally { + builtInitialRequest.releaseBodyObservation?.(); + } + + // Same-target 429 retry budget is per REQUEST: it lives OUTSIDE the recovery loop (so a 413/401 + // replay that comes back 429 cannot silently re-arm a fresh budget) and is SHARED with the + // terminal-guard continuation below, so the main loop + one continuation can never exceed + // `attempts` same-key replays in total (bounded per request). + const rateLimitPolicy = rateLimitRetryPolicyFor(route.provider); + let rateLimitRetries = 0; + // Shared with the terminal-guard continuation below: an image-tier reduction that let the + // main request clear a 413 must not be forgotten on the very next continuation build. + if (!upstreamResponse.ok) { + // Recovery loop: multi-key 429 failover + at most ONE opaque-state rebuild and ONE + // anthropic 413 tightened retry + // (devlog/260714_image_normalization_pipeline/030). One mutable activeAdapter serves + // both paths so a 429→413 sequence never rebuilds against a stale pre-rotation + // adapter, and imageTierBias — once armed — rides EVERY subsequent rebuild so a + // 413→429 rotation cannot silently undo the tightening. + let imageRetryAttempted = false; + const opaqueBlobRecoveryGuard: OpaqueBlobRecoveryGuard = { attempted: false }; + // Console Go answers a transient 400 "Invalid upload request." for bodies it accepts + // moments later; at most one byte-identical replay is allowed per request. + const consoleGoUploadRetryGuard: { attempted: boolean } = { attempted: false }; + let oauth401ReplayAttempted = false; + // At most one reasoning-effort downgrade per request. This sits outside the recovery loop + // below for the same reason the two guards above do: a guard declared inside it is reset by + // every `continue recovery`, which would let one turn walk the whole ladder down. + const reasoningEffortDowngradeGuard: { attempted: boolean } = { attempted: false }; + /** + * Rebuild the request from the current parsed input (and any image-tier bias) and refetch + * it once, tagging the attempt with the given recovery kind. Rebuilds are deterministic + * for the same parsed request, so same-target replays stay byte-identical. + */ + const rebuildAndRefetch = async ( + recovery: AttemptRecoveryKind, + ): Promise => { + let retryRequest: AdapterRequest; + if (transportState.sameTargetRequest !== undefined && transportState.sameTargetParsed === parsed && transportState.sameTargetToken === transportState.transportToken) { + // Same target (key/adapter/parsed/tier unchanged): replay the exact cached request. + retryRequest = transportState.sameTargetRequest; + } else { + try { + retryRequest = await transportState.activeAdapter.buildRequest(parsed, { + headers: requestState.selectedForwardHeaders, + translatorBudget, + ...(transportState.imageTierBias > 0 ? { imageTierBias: transportState.imageTierBias } : {}), + }); + recordAdapterReasoning(logCtx, retryRequest); + recordAdapterTier(logCtx, retryRequest); + } catch (err) { + // A rotated/rebuilt adapter build failure is a request-shaping error, not an + // upstream connect failure: tear the abort link down and map it as 400 (no 413 + // translator-budget mapping here — that stays with parseRequest/buildToolBridgeMaps). + cleanupUpstreamAbort(); + upstream.abort(); + if (options.abortSignal?.aborted) return { failed: clientCancelledResponse() }; + const msg = err instanceof Error ? err.message : String(err); + return { failed: formatErrorResponse(400, "invalid_request_error", redactSecretString(msg)) }; + } + transportState.sameTargetRequest = retryRequest; + transportState.sameTargetParsed = parsed; + transportState.sameTargetToken = transportState.transportToken; + } + refreshRequestToolAliases(retryRequest); + const retryEstimate = typeof retryRequest.usageLog?.inputTokens === "number" + ? retryRequest.usageLog.inputTokens + : undefined; + if (retryEstimate !== undefined) logCtx.usageLogInputTokens = retryEstimate; + logCtx.providerAdapter = transportState.activeAdapter.name; + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, transportState.activeAdapter.name, logCtx.accountLogLabel); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, transportState.activeAdapter.name); + noteAttemptSend(logCtx.activeAttempt, retryEstimate, recovery); + try { + try { + if (transportState.activeAdapter.fetchResponse) { + await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, upstream.signal); + return await transportState.activeAdapter.fetchResponse(retryRequest, { + abortSignal: upstream.signal, + timeoutMs: connectMs, + sendBudget: adapterSendBudget, + onPhysicalSend: send => noteAdapterPhysicalSend(retryEstimate, send), + stream: parsed.stream, + executor: providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(retryRequest), + providerName: route.providerName, + modelId: route.modelId, + }), + }); + } + // #2643 review: this leg used to call fetchWithHeaderTimeout directly, so an + // opted-in provider's transient-5xx policy applied to the initial send and to + // native chat but was silently bypassed here — a 429 that recovered into a + // retryable 503 got no retry on the Responses path. Route it through the same + // selection, and pass what is LEFT of the request-scoped budget rather than a + // fresh one, so a recovery loop cannot multiply total upstream sends. + const refetchTransientPolicy = transientRetryPolicyFor(route.provider); + const refetchWithPolicy = (route.provider.adapter === "google" || refetchTransientPolicy) + ? fetchWithTransientRetry + : fetchWithResetRetry; + // Same rule as the passthrough rebuild: spend the base allowance first, then the one + // shared final-recovery reserve, so a recovery that follows a spent streak still gets + // its single send instead of dying at three. + const refetchAllowance = refetchTransientPolicy + ? recoverySendAllowance( + refetchTransientPolicy.attempts, + recoveryClassFor(recovery), + `${route.providerName}|${route.modelId}|${recovery}`, + ) + : undefined; + try { + return await refetchWithPolicy( + recoveryKind => { + if (refetchAllowance?.permit && !refetchAllowance.permit.use()) { + throw new SendBudgetExhaustedError(safeHostLabel(retryRequest.url)); + } + return fetchWithHeaderTimeout(retryRequest.url, + applyUpstreamRecoveryInit({ + method: retryRequest.method, headers: retryRequest.headers, body: retryRequest.body, + }, recoveryKind), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(retryRequest), + providerName: route.providerName, + modelId: route.modelId, + })); + }, + { + abortSignal: upstream.signal, + label: safeHostLabel(retryRequest.url), + ...(refetchAllowance + ? { + attempts: refetchAllowance.attempts, + onSendsConsumed: noteTransientSends, + } + : {}), + }, + ); + } finally { + // Refunds only a reservation whose send never happened -- an abort settled before + // the thunk ran. A used or externally settled permit ignores this. + refetchAllowance?.permit?.release(); + } + } finally { + retryRequest.releaseBodyObservation?.(); + } + } catch (err) { + cleanupUpstreamAbort(); + upstream.abort(); + if (options.abortSignal?.aborted) { + return { failed: clientCancelledResponse() }; + } + const msg = describeUpstreamConnectFailure(err, connectMs); + return { failed: formatErrorResponse(502, "upstream_error", msg) }; + } + }; + // Keep recovery kinds in sync with the native Responses `passthroughRecovery:` loop above. + recovery: for (;;) { + if ( + upstreamResponse.status === 401 + && isOAuth401ReplayProvider + && transportState.sentOAuthSnapshot + && !oauth401ReplayAttempted + && !sendBudgetExhausted() + ) { + oauth401ReplayAttempted = true; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + let refreshed: OAuthAccessSnapshot; + try { + refreshed = await refreshResolvedOAuthSelection(transportState.sentOAuthSnapshot); + } catch (err) { + cleanupUpstreamAbort(); + return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(err)); + } + if (route.provider.googleMode === "cloud-code-assist" && !refreshed.projectId) { + cleanupUpstreamAbort(); + return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(new Error("Cloud Code Assist project is required"))); + } + transportState.sentOAuthSnapshot = refreshed; + transportState.replayOAuthCredentialSnapshot = { + accountId: refreshed.accountId, + generation: refreshed.generation, + }; + if (route.providerName === "kiro") { + parsed._kiroAuthContext = { ...(refreshed.kiro ?? {}) }; + } + const refreshedProvider = resolveProviderTransport( + route.providerName, + { + ...route.provider, + apiKey: refreshed.accessToken, + ...(refreshed.projectId ? { project: refreshed.projectId } : {}), + }, + parsed.options.promptCacheKey, + route.providerName === "github-copilot" + ? resolveCopilotApiBaseUrl(refreshed.apiBaseUrl) + : undefined, + ); + route.provider = refreshedProvider; + invalidateSameTargetRequest(); + transportState.activeAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, refreshedProvider, inboundWire), + config.cacheRetention, + ); + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: refreshedProvider, + adapterName: transportState.activeAdapter.name, + oauthCredentialSnapshot: transportState.replayOAuthCredentialSnapshot, + }); + const result = await rebuildAndRefetch("oauth-401"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue recovery; + } + + // Static API-key pools can recover a credential-scoped 401 without abandoning the + // provider: one revoked or mistyped key says nothing about its siblings. OAuth providers + // refresh above and never enter here — `hasKeyPoolFailover` rejects oauth/forward modes. + // Runs after the OAuth replay so a refreshable token is never treated as a dead key. + while (upstreamResponse.status === 401 && hasKeyPoolFailover(route.provider)) { + const rotated = rotateProviderTransportOn401(config, route.providerName, route.provider, { + now: Date.now(), + attemptedKey: route.provider.apiKey, + promptCacheKey: parsed.options.promptCacheKey, + }); + if (!rotated) break; + // Release the failed response's socket before retrying; unread bodies otherwise linger + // until runtime cleanup (one per rotated key). + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + route.provider = rotated; + invalidateSameTargetRequest(); + transportState.activeAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: route.provider, + adapterName: transportState.activeAdapter.name, + }); + const result = await rebuildAndRefetch("key-401"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + } + + // Same-target 429 wait-and-retry (opt-in `retryOn429`, issue #487). Codex never retries + // 429 itself (it retries 5xx only), and single-key pools cannot use the failover below, + // so wait (Retry-After or the fixed interval) and replay the IDENTICAL request on the + // same key first. Pre-stream only: a 429 arrives before any bytes are relayed, so the + // replay is lossless. Runs before key failover so "primary-first" setups keep the same + // key on rate-limit blips; only after the attempts are exhausted does failover run. + while ( + upstreamResponse.status === 429 + && rateLimitPolicy !== null + && rateLimitRetries < rateLimitPolicy.attempts + && !sendBudgetExhausted() + ) { + rateLimitRetries += 1; + // Release unread body + deliberate wait via the shared same-target helper. + const retryAfterHeader = upstreamResponse.headers.get("retry-after"); + try { + for await (const _ of prepareSameTarget429Wait({ + body: upstreamResponse.body, + signal: options.abortSignal, + delayMs: rateLimitRetryDelayMs(rateLimitPolicy, retryAfterHeader, Date.now()), + })) { + // pre-stream: no stall watchdog to feed + } + } catch { + cleanupUpstreamAbort(); + upstream.abort(); + return clientCancelledResponse(); + } + // Client cancellation wins over any stale timer edge: re-check before dispatching the + // replay so an adapter never starts work for a request the client already abandoned. + if (options.abortSignal?.aborted || upstream.signal.aborted) { + cleanupUpstreamAbort(); + upstream.abort(); + return clientCancelledResponse(); + } + const result = await rebuildAndRefetch("rate-limit-429"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + } + + // Multi-key 429 failover: rotate to the next pool key (cooldown-aware) and retry the + // SAME request once per remaining key. OAuth/forward providers and single-key pools + // return null immediately, so this stays a no-op for them (src/providers/key-failover.ts). + while (upstreamResponse.status === 429 && hasKeyPoolFailover(route.provider)) { + const rotated = rotateProviderTransportOn429(config, route.providerName, route.provider, { + retryAfter: upstreamResponse.headers.get("retry-after"), + now: Date.now(), + attemptedKey: route.provider.apiKey, + promptCacheKey: parsed.options.promptCacheKey, + }); + if (!rotated) break; + // Release the failed response's socket before retrying; unread bodies otherwise linger + // until runtime cleanup (one per rotated key under a rate-limit storm). + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + route.provider = rotated; + invalidateSameTargetRequest(); + transportState.activeAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: route.provider, + adapterName: transportState.activeAdapter.name, + }); + const result = await rebuildAndRefetch("key-429"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + } + + // Opt-in Anthropic OAuth account pool (#294): cool the failed account and retry + // with another eligible OAuth account (bounded per request). Disabled by default. + while ( + upstreamResponse.status === 429 + && transportState.anthropicPoolAccountId + && transportState.anthropicPoolFailovers < ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST + ) { + const nextAccountId = rotateAnthropicAccountOn429( + config, + transportState.anthropicPoolAccountId, + upstreamResponse.headers.get("retry-after"), + anthropicSessionKey, + Date.now(), + upstreamResponse.headers, + ); + if (!nextAccountId) break; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + try { + const admitted = await commitResolvedOAuthSelection(await getAnthropicPoolAccessSnapshot(nextAccountId)); + if (!admitted) throw new Error("OAuth selection changed during recovery"); + transportState.anthropicPoolAccountId = admitted.accountId; + transportState.anthropicPoolFailovers += 1; + route.provider = { ...route.provider, apiKey: admitted.accessToken }; + invalidateSameTargetRequest(); + logCtx.provider = formatAnthropicProviderForLog("anthropic", admitted.accountId, config); + transportState.activeAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, transportState.activeAdapter.name, logCtx.accountLogLabel); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, transportState.activeAdapter.name); + const result = await rebuildAndRefetch("anthropic-oauth-429"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + } catch { + break; + } + } + // Generic OAuth account failover (#2568) for providers with no pool of their own. + // Presence is consent since #2568d: rotation is ON by default once two or more eligible + // accounts are stored for the provider, because a second deliberate login is read as the + // operator asking for it. A single-account install is still a strict no-op, and an + // explicit `oauthAccountFailover.enabled: false` (global or per provider) still wins -- + // see isGenericOAuthFailoverEnabled in src/oauth/generic-account-failover.ts. Codex and + // Anthropic are excluded by isGenericFailoverProvider: their pools own quota scopes, + // probe leases and affinity that this must not reimplement. + while ( + upstreamResponse.status === 429 + && transportState.genericFailoverAccountId + && transportState.genericFailovers < GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST + && isGenericOAuthFailoverEnabled(config, route.providerName) + ) { + // Intersection with the shared request budget. This arm re-sends through + // rebuildAndRefetch, so the roster cap alone would let one request walk the roster on + // an allowance the rest of the request cannot see. A refusal ends the ladder with the + // real 429 already in hand, which is the decided exhaustion contract. + const hop = reserveCredentialHop( + "auth-recovery", + `${route.providerName}|${route.modelId}|adapter-recovery-oauth-429`, + ); + if (!hop.allowed) break; + const nextAccountId = rotateGenericOAuthAccountOn429( + config, + route.providerName, + transportState.genericFailoverAccountId, + upstreamResponse.headers.get("retry-after"), + ); + if (!nextAccountId) { + hop.permit?.release(); + break; + } + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + try { + // The FULL snapshot, not just the bearer: Antigravity pairs an account-matched + // projectId with its token and Kiro carries routing metadata, so a token-only swap + // would mix one account's credential with another's routing data. + const snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); + transportState.genericFailovers += 1; + if (!await applyFailoverSnapshot(snapshot)) { + hop.permit?.release(); + break; + } + invalidateSameTargetRequest(); + transportState.activeAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, transportState.activeAdapter.name, logCtx.accountLogLabel); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, transportState.activeAdapter.name); + const result = await rebuildAndRefetch("oauth-account-429"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + } catch { + break; + } + } + // Unknown provenance is deliberately fail-soft in pre-flight: after a restart, TTL expiry, + // or LRU eviction, a valid same-backend blob must survive. A decoder's own 4xx identity is + // the missing authoritative signal. Rebuild once through the same sanitation path used by a + // known route switch; invalidating is mandatory because `parsed` mutates in place and the + // same-target cache would otherwise replay the rejected bytes verbatim. + const opaqueBlobRecovery = await attemptOpaqueBlobRecovery({ + response: upstreamResponse, + outboundBody: transportState.sameTargetRequest?.body, + adapterName: transportState.activeAdapter.name, + parsed, + guard: opaqueBlobRecoveryGuard, + signal: upstream.signal, + }, recovery => { + invalidateSameTargetRequest(); + return rebuildAndRefetch(recovery); + }); + if (opaqueBlobRecovery.kind === "failed") return opaqueBlobRecovery.response; + if (opaqueBlobRecovery.kind === "recovered") { + upstreamResponse = opaqueBlobRecovery.response; + continue recovery; + } + // Anthropic 413 request_too_large: rebuild once with every image one tier lower + // (spiral guard: single attempt). The biased response re-enters the 429 check above. + if (shouldAttemptImageTierRetry({ + status: upstreamResponse.status, + adapterName: transportState.activeAdapter.name, + parsed, + alreadyAttempted: imageRetryAttempted, + })) { + imageRetryAttempted = true; + transportState.imageTierBias = 1; + invalidateSameTargetRequest(); + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + const result = await rebuildAndRefetch("image-413"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue recovery; + } + // Console Go (opencode-zen / opencode-go) intermittently rejects a body it accepts seconds + // later with 400 invalid_request_error / "Invalid upload request." Replay the + // byte-identical request once after the exact gateway rejection. + if (!consoleGoUploadRetryGuard.attempted) { + const uploadRejectionBody = await consoleGoUploadRejectionBody( + upstreamResponse, + consoleGoUploadRetryGuard.attempted, + upstream.signal, + ); + if (uploadRejectionBody !== undefined + && isTransientConsoleGoUploadRejection({ + status: upstreamResponse.status, + errorBody: uploadRejectionBody, + outboundUrl: transportState.sameTargetRequest?.url, + })) { + consoleGoUploadRetryGuard.attempted = true; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + if (!upstream.signal.aborted) { + try { + await sleepWithAbort(CONSOLE_GO_UPLOAD_RETRY_DELAY_MS, upstream.signal); + } catch { cleanupUpstreamAbort(); return clientCancelledResponse(); } + } + if (upstream.signal.aborted) { cleanupUpstreamAbort(); return clientCancelledResponse(); } + const result = await rebuildAndRefetch("console-go-upload-retry"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue recovery; + } + } + // Reasoning-effort downgrade, mirroring the passthroughRecovery loop above: learn the + // refused rung, then replay once at the next published one. + if (!reasoningEffortDowngradeGuard.attempted) { + const rejectionText = await reasoningEffortRejectionText( + upstreamResponse, + reasoningEffortDowngradeGuard.attempted, + upstream.signal, + ); + const downgrade = rejectionText === undefined + ? undefined + : planReasoningEffortDowngrade({ + provider: route.provider, + modelId: parsed.modelId, + requested: parsed.options.reasoning, + rejectionText, + }); + if (downgrade) { + reasoningEffortDowngradeGuard.attempted = true; + parsed.options.reasoning = downgrade.effort; + // The same-target cache keys on parsed identity, so a mutated effort needs a token bump. + invalidateSameTargetRequest(); + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + const result = await rebuildAndRefetch("reasoning-effort-downgrade"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue recovery; + } + } + break; + } + if (!upstreamResponse.ok) { + if (options.comboAttempt) { + // No pre-read guard: `consumeComboFailure` -> `readBoundedResponseBody` reads + // `response.body` itself with the abort signal threaded through, and the combo + // contract is that this body's getter is touched exactly once. A guard here would be + // a second `.body` access for no gain, since the bounded reader owns settlement. + const failure = await consumeComboFailure(upstreamResponse, options.abortSignal) + .finally(cleanupUpstreamAbort); + options.onConsumedComboFailure?.(failure); + return failure.response; + } + let errorText: string; + try { + errorText = await readDisplaySafeErrorText( + upstreamResponse, + upstream.signal, + "unknown error", + ); + } finally { + cleanupUpstreamAbort(); + } + if (upstreamResponse.status === 413) { + return clientRequestedStream + ? streamingContextOverflowResponse(parsed._responseModelId ?? parsed.modelId, translatorBudget) + : jsonContextOverflowResponse(); + } + if (!isFixedCodexAccount(admissionState.authCtx)) { + recordSubagentQuotaFailureForThreadSpawn( + req.headers, + subagentQuotaFailureModel, + upstreamResponse.status === 429 || upstreamResponse.status === 402 + ? upstreamResponse.status + : `Provider error ${upstreamResponse.status}: ${redactSecretString(errorText.slice(0, 500))}`, + config, + requestState.subagentFallbackAccountId, + ); + } + // Upstreams occasionally echo request details in error bodies — scrub token-shaped + // material before it reaches the client-facing error surface. + const upstreamRetryAfter = upstreamResponse.headers.get("retry-after"); + const normalized = normalizeUpstreamErrorText(errorText, "unknown error"); + const message = normalized.cyberPolicy + ? normalized.message + ?? (isCyberPolicyCode(normalized.code) ? CYBER_POLICY_FALLBACK_MESSAGE : normalized.safeText) + : enrichOpenCodeZenUpstreamMessage( + `Provider error ${upstreamResponse.status}: ${normalized.safeText}`, + { + status: upstreamResponse.status, + providerName: route.providerName, + baseUrl: route.provider.baseUrl, + adapter: route.provider.adapter, + authMode: route.provider.authMode, + hasApiKey: Boolean(route.provider.apiKey?.trim()), + upstreamRetryAfter, + // This recovery path is the HTTP Responses wire; custom runTurn transports + // never reach enrichOpenCodeZenUpstreamMessage here. + supportsHttpSameKeyRetry: true, + }, + ); + const retryAfter = normalized.cyberPolicy + ? undefined + : resolveClientRetryAfter({ + status: upstreamResponse.status, + message, + upstreamRetryAfter, + }); + return formatErrorResponse( + upstreamResponse.status, + normalized.cyberPolicy ? (normalized.type ?? CYBER_POLICY_ERROR_CODE) : "upstream_error", + message, + { + ...(normalized.cyberPolicy ? { code: CYBER_POLICY_ERROR_CODE } : {}), + ...(retryAfter !== undefined ? { retryAfter } : {}), + }, + ); + } + } + + cancelBodyOnAbort(upstreamResponse.body, upstream.signal); + + return { + upstream, + cleanupUpstreamAbort, + connectMs, + stallTimeoutMs, + upstreamResponse, + rateLimitPolicy, + get rateLimitRetries(): typeof rateLimitRetries { + return rateLimitRetries; + }, + set rateLimitRetries(value: typeof rateLimitRetries) { + rateLimitRetries = value; + }, + }; +} + +export type AdapterExchange = Exclude>, Response>; diff --git a/src/server/responses/completion-policy.ts b/src/server/responses/completion-policy.ts new file mode 100644 index 0000000000..653f94d065 --- /dev/null +++ b/src/server/responses/completion-policy.ts @@ -0,0 +1,33 @@ +import type { ResponsesRequestContext } from "./core-options"; +import type { ResponsesSidecarAuth } from "./request-sidecar-auth"; +import { emptyCompletionRetryEnabled } from "./empty-completion-guard"; + +/** One responsibility of the Responses request pipeline; state owners are explicit. */ +export function createResponsesCompletionPolicy( + requestContext: Pick, + sidecarState: Pick, +) { + const { config, options } = requestContext; + const { routedCompaction } = sidecarState; + + + // Empty-completion guard (codex-router PR #145 port): a 200 that completes with no output + // text and no tool call is a failure the client cannot see — it silently records the turn as + // done. The guard holds pre-content adapter events, suppresses the terminal of an empty + // turn, retries the IDENTICAL request once, and surfaces a stated error when the retry is + // empty or fails. This is a top-level config opt-in; OCX_EMPTY_COMPLETION_RETRY=0 is a + // disable-only emergency override. Compaction turns and combo attempts keep their own + // machinery (the combo preflight already handles empty streams). Native Chat-to-Chat + // requests return from handleChatCompletions before entering Responses core, so they are + // intentionally outside this guard and retain their existing one-send wire behavior. + const emptyCompletionGuardEnabled = + emptyCompletionRetryEnabled(config) + && !options.comboAttempt + && !routedCompaction; + + return { + emptyCompletionGuardEnabled, + }; +} + +export type ResponsesCompletionPolicy = Exclude, Response>; diff --git a/src/server/responses/core-auth.ts b/src/server/responses/core-auth.ts new file mode 100644 index 0000000000..b8d62be240 --- /dev/null +++ b/src/server/responses/core-auth.ts @@ -0,0 +1,527 @@ +import type { OcxProviderConfig, OcxConfig } from "../../types"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import type { CodexAuthContext } from "../../codex/auth-context"; +import type { RouteResult } from "../../router"; +import type { HandleResponsesOptions } from "./core-options"; +import { + hasForwardableCodexBearer, + validateForwardAdmissionCredential, + isProxyAdmissionSecret, + ForwardAdmissionCredentialError, +} from "../auth-cors"; +import { + providerConsumesCallerAuthorization, + captureCallerDirectAuth, +} from "../../providers/caller-authorization"; +import { inspectChatGptDomainClaim } from "../../oauth/chatgpt"; +import { + resolveCodexAuthContext, + CodexMainProfileDrainingError, + materializeCodexUpstreamAuthAsync, + headersForCodexAuthContext, + isCodexAuthContextUsable, + releaseCodexAuthContextProbeLease, + CodexAuthContextError, + applyCodexAuthContextToProvider, + stripCodexRuntimeProviderFields, +} from "../../codex/auth-context"; +import { codexAccountSelectionForTurn, tryClaimNativeMainProfileForTurn } from "../lifecycle"; +import { isNativeMainTrafficBlocked } from "../../codex/native-profile-startup"; +import { formatErrorResponse } from "../../bridge"; +import { clientCancelledResponse } from "./core-errors"; +import { formatCodexProviderForLog, handOffThreadAffinityGeneration } from "../../codex/routing"; +import { mapCodexAuthContextErrorToResponse, nativeMainRefreshFailureResponse } from "./codex-auth-error"; +import { + isTerminalCodexPoolRefreshFailure, + forceRefreshCodexPoolToken, + capturePoolQuotaWriter, +} from "../../codex/account-store"; +import type { RequestLogContext } from "../request-log"; +import { markLocalRequestLogRefusal } from "../request-log"; +import { CODEX_POOL_REFRESH_INCOMPLETE_LOG_REASON } from "../../codex/pool-refresh-backoff"; +import { codexAuthContextLogLabel } from "../../codex/account-label"; +import { forceRefreshMainAccountToken } from "../../codex/main-account"; + +/** Keep synthesized Claude identity out of request headers reused by policy/combo fallback. */ +export function withClaudeNativeSession(headers: Headers, provider: OcxProviderConfig, sessionId?: string): Headers { + if (!sessionId || !isCanonicalOpenAiForwardProvider(provider) + || headers.has("session_id") || headers.has("session-id") || headers.has("thread-id")) return headers; + const forwarded = new Headers(headers); + forwarded.set("session_id", sessionId); + return forwarded; +} + + +export type ResponsesAuthResolution = + | { ok: true; authCtx: CodexAuthContext; headers: Headers; callerAuthHeaders: Headers; substituteMainCredential: boolean } + | { ok: false; response: Response }; + + +/** + * The caller credential the final Codex auth resolution will be given, as far as the ROUTE + * decides it: a route change that may cross a credential domain drops the raw caller credential, + * and a trusted Claude-main handoff replaces it. + * + * Shared with the lineage preview in `handleResponsesInner`, which has to read a conversation's + * family under the same authenticated scope the resolution will record it under -- that scope is + * an HMAC of exactly this Authorization header. Two copies of this rule would put preview and + * final auth in different scopes the first time one of them changed. + */ +export function codexRouteCredentialDomainHeaders( + req: Request, + route: RouteResult, + options: HandleResponsesOptions, + credentialDomainWasRewritten: boolean, +): Headers { + const trustedClaudeMainForFinalRoute = options.stripClaudeMainAuthForNoncanonicalForward === true + && isCanonicalOpenAiForwardProvider(route.provider) + ? options.trustedClaudeMainAuth : undefined; + if (trustedClaudeMainForFinalRoute) { + const claudeMainHeaders = new Headers(req.headers); + claudeMainHeaders.set("authorization", trustedClaudeMainForFinalRoute.authorization); + if (trustedClaudeMainForFinalRoute.chatgptAccountId) { + claudeMainHeaders.set("chatgpt-account-id", trustedClaudeMainForFinalRoute.chatgptAccountId); + } else { + claudeMainHeaders.delete("chatgpt-account-id"); + } + return claudeMainHeaders; + } + // Route-changing recursion retains typed admission, never an unscoped raw + // caller credential. Bearer admission is substituted or stripped below. + const routeMayChangeCredentialDomain = options.comboAttempt === true + || route.routeKind === "policy" + || credentialDomainWasRewritten; + if (routeMayChangeCredentialDomain && options.admission?.source !== "bearer") { + const scoped = new Headers(req.headers); + scoped.delete("authorization"); + scoped.delete("chatgpt-account-id"); + return scoped; + } + return req.headers; +} + + +/** + * Does this route substitute OUR stored main credential, and does the caller own the credential + * this request will authenticate with? + * + * Both answers are needed twice: by the resolution below, and by the lineage preview, which must + * not follow a Pool family binding for a request whose credential never enters Pool state. One + * implementation, because two copies of this predicate disagreeing is the divergence the preview + * gate exists to prevent. The reasoning behind the substitution test itself is at its use site + * below (#1686, #2132). + */ +export function codexRouteCredentialOwnership( + authInputHeaders: Headers, + config: OcxConfig, + route: RouteResult, + options: HandleResponsesOptions, +): { substituteMainCredential: boolean; requestScopedMainCredential: boolean } { + const substituteMainCredential = options.admission?.source === "bearer" + && (route.codexAccountMode !== undefined || isCanonicalOpenAiForwardProvider(route.provider)); + return { + substituteMainCredential, + requestScopedMainCredential: route.codexAccountMode !== undefined + && !substituteMainCredential + && hasForwardableCodexBearer(authInputHeaders, config), + }; +} + + +/** + * Resolve Codex auth for a route. On unusable contexts, releases any probe lease + * before returning the 401 (nothing reaches upstream). + */ +export async function resolveResponsesCodexAuth( + req: Request, + config: OcxConfig, + route: RouteResult, + options: HandleResponsesOptions, + credentialDomainWasRewritten = false, +): Promise { + try { + let authInputHeaders = codexRouteCredentialDomainHeaders( + req, + route, + options, + credentialDomainWasRewritten, + ); + // A caller-auth transport that is not canonical OpenAI (keyless Cursor) consumes the + // caller's Authorization as its own upstream token. Keep that contract only for a clean + // single bearer with NO ChatGPT-domain marker. A bearer marked for the ChatGPT domain — + // whether its marker is valid or malformed/conflicting — a combined/malformed value, or + // the captured explicit OpenAI pair is never a Cursor token; a foreign JWT carrying only + // a generic organizations claim is not ChatGPT-marked and keeps the legacy contract. + // chatgpt-account-id has no meaning outside the ChatGPT domain. + if (!isCanonicalOpenAiForwardProvider(route.provider) + && providerConsumesCallerAuthorization(route.provider)) { + const rawAuth = authInputHeaders.get("authorization")?.trim(); + const singleBearer = /^Bearer[\t ]+([^\s,]+)$/i.exec(rawAuth ?? "")?.[1]; + const domainClaim = singleBearer ? inspectChatGptDomainClaim(singleBearer) : { kind: "absent" as const }; + const dropBearer = options.nativeCallerAuth != null || domainClaim.kind !== "absent" + || (rawAuth !== undefined && singleBearer === undefined); + if (dropBearer || authInputHeaders.has("chatgpt-account-id")) { + const scoped = new Headers(authInputHeaders); + if (dropBearer) scoped.delete("authorization"); + scoped.delete("chatgpt-account-id"); + authInputHeaders = scoped; + } + } + // The caller's own Direct credential may cross an internal route change only to the + // canonical OpenAI transport, under a predicate deliberately STRICTER than plain + // unchanged-route Direct forwarding: a clean non-proxy bearer whose ChatGPT-domain + // marker is valid, with any explicit account header matching that marker. Unchanged + // routes keep their legacy rules; sidecar enrichment grants no primary authority. + if (options.callerDirectAuth && isCanonicalOpenAiForwardProvider(route.provider)) { + const directHeaders = new Headers({ + authorization: options.callerDirectAuth.authorization, + ...(options.callerDirectAuth.chatgptAccountId + ? { "chatgpt-account-id": options.callerDirectAuth.chatgptAccountId } : {}), + }); + if (captureCallerDirectAuth(directHeaders, config)) { + authInputHeaders = new Headers(authInputHeaders); + authInputHeaders.set("authorization", options.callerDirectAuth.authorization); + if (options.callerDirectAuth.chatgptAccountId) { + authInputHeaders.set("chatgpt-account-id", options.callerDirectAuth.chatgptAccountId); + } else { + authInputHeaders.delete("chatgpt-account-id"); + } + } + } + // #1686: a caller that proved admission with a BEARER presented one of our own secrets. + // Refusing it here is what made the codex-cli `env_key` contract unusable against Direct. + // Admitting it is only safe because the stored main credential is substituted below, so + // the admission secret still never leaves this process. + // + // #2132: substitution answers "does THIS ROUTE need our stored ChatGPT credential", not + // "how did the caller authenticate". Only a native Codex route reaches the ChatGPT backend + // and can consume that credential; a key-authenticated routed provider carries its own and + // never touches it. Keying on the caller alone made an install that deliberately never + // logged into ChatGPT fail every routed request with "No usable Codex main credential". + // + // But ask that question the way the ADAPTER asks it. `codexAccountMode` is derived from the + // provider NAME (`providerCodexAccountMode`), while the passthrough adapter decides whether + // to forward caller credentials from the TRANSPORT — adapter, auth mode, and base URL + // (`isCanonicalOpenAiForwardProvider`). A row the operator named anything other than + // `openai`, pointed at the canonical ChatGPT backend with `authMode: "forward"`, satisfies + // the adapter's test and fails this one, so substitution was skipped and the adapter then + // forwarded our own admission secret upstream. Two predicates answering one question is the + // bug; the transport is the authority, because the transport is what actually carries the + // header. A key-authenticated routed provider is still not canonical-forward, so #2132's + // no-ChatGPT-login install keeps working. + const { substituteMainCredential, requestScopedMainCredential } = codexRouteCredentialOwnership( + authInputHeaders, + config, + route, + options, + ); + const stripAuthorization = options.admission?.source === "bearer" && !substituteMainCredential; + if (route.codexAccountMode === "direct" && !substituteMainCredential) { + validateForwardAdmissionCredential(authInputHeaders, config); + } + let authCtx: CodexAuthContext; + if (route.codexAccountMode) { + authCtx = await resolveCodexAuthContext(authInputHeaders, config, route.codexAccountMode, { + admission: options.admission, + codexAuthPolicy: options.codexAuthPolicy, + accountId: route.codexAccountId, + modelId: route.modelId, + substituteMainCredentialForDirect: substituteMainCredential, + requestScopedMainCredential, + beginCodexAccountSelection: codexAccountSelectionForTurn(options.turnAdmissionLease), + resolveCodexModelEntitlements: options.resolveCodexModelEntitlements, + signal: options.abortSignal, + nativeMainRefreshDependencies: options.nativeMainRefreshDependencies, + }); + options.onCodexAuthContextResolved?.(authCtx); + } else { + // A custom-named canonical-forward provider has no Codex account mode, but an + // admission bearer still substitutes the stored main credential below. Claim the + // same physical profile before synthesizing the main context so transport-based + // substitution cannot bypass a switch drain. + if ( + substituteMainCredential + && ( + isNativeMainTrafficBlocked() + || !tryClaimNativeMainProfileForTurn(options.turnAdmissionLease) + || isNativeMainTrafficBlocked() + ) + ) { + throw new CodexMainProfileDrainingError(); + } + authCtx = { kind: "main", accountId: null }; + options.onCodexAuthContextResolved?.(undefined); + } + // This resolver also builds a synthetic main context for unrelated keyed routes. Only + // the actual Codex-forward transport consumes main quota; provider names are not proof + // (custom-named canonical-forward providers must retain the same protection). + const mainPolicyConfig = isCanonicalOpenAiForwardProvider(route.provider) + ? options.codexAuthPolicy ?? config : undefined; + const headers = await materializeCodexUpstreamAuthAsync(authInputHeaders, authCtx, { + admission: options.admission, + config: mainPolicyConfig, + modelId: route.modelId, + beginCodexAccountSelection: codexAccountSelectionForTurn(options.turnAdmissionLease), + substituteMainCredential, + signal: options.abortSignal, + nativeMainRefreshDependencies: options.nativeMainRefreshDependencies, + }); + // Awaiting even a cached materialization yields. Preserve the policy error if the live + // quota/config changed during that yield, before usability could mislabel it as reauth. + headersForCodexAuthContext(headers, authCtx, mainPolicyConfig, route.modelId, options.admission); + if (!isCodexAuthContextUsable(authCtx, config)) { + releaseCodexAuthContextProbeLease(authCtx); + return { + ok: false, + response: formatErrorResponse(401, "authentication_error", "Selected Codex account needs reauthentication"), + }; + } + if (stripAuthorization) { + headers.delete("authorization"); + headers.delete("chatgpt-account-id"); + } + if (providerConsumesCallerAuthorization(route.provider) && options.admission?.source !== undefined + && options.admission.source !== "loopback") { + validateForwardAdmissionCredential(headers, config); + } else { + // Even adapters that ignore caller auth must not retain a proxy secret for + // a later internal hop or a future transport change. + const bearer = headers.get("authorization")?.replace(/^Bearer\s+/i, "").trim(); + if (bearer && isProxyAdmissionSecret(bearer, config)) { + headers.delete("authorization"); + headers.delete("chatgpt-account-id"); + } + } + return { + ok: true, + authCtx, + headers, + callerAuthHeaders: new Headers(authInputHeaders), + substituteMainCredential, + }; + } catch (err) { + if (options.abortSignal?.aborted || req.signal.aborted) { + return { ok: false, response: clientCancelledResponse() }; + } + if (err instanceof CodexAuthContextError) { + const safeAccountLabel = route.codexAccountNamespace + ? `${route.providerName}-${route.codexAccountNamespace}` + : formatCodexProviderForLog(route.providerName, err.accountId, config); + console.error(`[codex-auth] Pool account ${safeAccountLabel} token failed; reauthentication required`); + } + if (err instanceof ForwardAdmissionCredentialError) { + return { ok: false, response: formatErrorResponse(401, "authentication_error", err.message) }; + } + const response = mapCodexAuthContextErrorToResponse(err, { + accountSelector: route.codexAccountNamespace, + now: Date.now(), + }); + if (response) return { ok: false, response }; + throw err; + } +} + + +/** + * Terminal means the grant itself is dead and no retry can help. Everything else — + * an untyped network failure, a token-endpoint 5xx surfacing as `unknown`, an abort, + * refresh capacity, lock contention, a superseded flight — is transient, and treating + * it as terminal would quarantine a healthy account on an upstream blip, which is the + * defect this path exists to fix (#2887). + */ +export function isTerminalPoolRefreshFailure(error: unknown): boolean { + // Delegated so "terminal" has ONE definition. A missing record or a missing refresh-grant + // fingerprint is permanent -- retrying cannot conjure a credential -- and used to be a bare + // Error, which fell through to the retryable 503 and told the operator to keep retrying a + // request that could never succeed. + return isTerminalCodexPoolRefreshFailure(error); +} + + +/** + * The refusal an operator meets when a stored pool credential's forced refresh does not complete. + * + * A bare "retry this request" reads as a transient fault in the proxy, which is how #4212's + * reporter spent an afternoon concluding OpenCodex had broken while one of their own accounts was + * the thing that needed them. It stays a retryable 503 and stays non-quarantining, because the + * refresh genuinely may succeed and a token-endpoint 5xx must not retire a healthy account + * (#2887). What it adds is the account and the exit: when retrying stops helping, that account + * has to be signed in again. + * + * The label is a public account selector when the request carried one, otherwise the durable + * `p`-prefixed log label — never the raw pool id and never the email. Those are the identifiers + * `responses-compaction-routing.test.ts` and `codex-auth-context.test.ts` already assert must not + * reach an operator-facing surface, and an error body travels further than a log line, not less. + * When neither is resolvable the sentence degrades to "the selected Codex pool account" rather + * than naming something opaque, because a wrong name is worse than no name. + * + * The wording says "sign in to that account again" and deliberately does NOT say + * "reauthentication". `classifyError` runs `isAuthenticationMessage` before it reaches the + * `status === 503` arm, and that check is status-blind on the bare substring "authentication", + * which "reauthentication" contains. A body carrying that word is reclassified to + * `authentication_error` / `invalid_api_key` even though the HTTP status stays 503 — and Codex + * applies retry-after backoff only for `server_is_overloaded`, so the friendlier sentence would + * have quietly disabled the retry this refusal exists to ask for. `options.code` cannot buy the + * classification back; only the wording can. + */ +export function poolCredentialRefreshIncompleteResponse(args: { + authCtx: CodexAuthContext; + config: Pick; + accountSelector?: string; + logCtx?: RequestLogContext; +}): Response { + // The wire contract below is unchanged on purpose, so the record has to carry the origin + // instead. Without it an operator reads this sentence under a field named "Upstream reason" + // and goes looking at the provider's status page for a refusal that never left this process. + if (args.logCtx) markLocalRequestLogRefusal(args.logCtx, CODEX_POOL_REFRESH_INCOMPLETE_LOG_REASON); + const label = args.accountSelector ?? codexAuthContextLogLabel(args.authCtx, args.config); + const account = label ? `Codex pool account ${label}` : "the selected Codex pool account"; + const response = formatErrorResponse( + 503, + "server_busy", + `Codex credential refresh did not complete for ${account}; retry this request. ` + + "If it keeps failing, sign in to that account again.", + ); + const headers = new Headers(response.headers); + headers.set("Retry-After", "1"); + return new Response(response.body, { status: response.status, headers }); +} + + +/** + * One forced refresh and one same-account rebuild for a stored pool credential that + * upstream rejected with a pre-stream 401. `quarantine` distinguishes a dead grant, + * which must retire the account, from a transient failure, which must not. + */ +export async function refreshPoolForwardAuth(args: { + logCtx?: RequestLogContext; + req: Request; + config: OcxConfig; + route: RouteResult; + authCtx: CodexAuthContext & { kind: "pool" }; + substituteMainCredential: boolean; + options: HandleResponsesOptions; +}): Promise< + | { ok: true; authCtx: CodexAuthContext; provider: OcxProviderConfig; headers: Headers } + | { ok: false; response: Response; quarantine: boolean; quarantineGeneration?: number } +> { + const { req, config, route, authCtx, substituteMainCredential, options } = args; + try { + const refreshed = await forceRefreshCodexPoolToken(authCtx.accountId, { + rejectedGeneration: authCtx.generation, + rejectedAccessToken: authCtx.accessToken, + signal: options.abortSignal, + }); + if (!refreshed.rotated) { + // The store resolved to the same bearer upstream just rejected. Replaying it + // would spend another upstream call to earn the identical 401. Upstream can do + // this on a SUCCESSFUL response by rotating only the refresh grant, so the + // credential generation may already have moved — quarantine has to be fenced on + // where the credential actually is, not on the generation we started from. + return { + ok: false, + quarantine: true, + quarantineGeneration: refreshed.generation, + response: formatErrorResponse(401, "authentication_error", "Selected Codex account needs reauthentication"), + }; + } + // Only a CAS this request performed itself proves the new credential descends from + // the rejected one. Somebody else's replacement may be a different identity, and + // its affinity must be retired rather than inherited. + if (refreshed.selfRefreshed) { + handOffThreadAffinityGeneration(authCtx.accountId, authCtx.generation, refreshed.generation); + } + const refreshedAuthCtx: CodexAuthContext = { + ...authCtx, + accessToken: refreshed.accessToken, + chatgptAccountId: refreshed.chatgptAccountId, + generation: refreshed.generation, + poolQuotaWriter: capturePoolQuotaWriter(authCtx.accountId, refreshed), + }; + const provider = applyCodexAuthContextToProvider( + stripCodexRuntimeProviderFields(route.provider), + refreshedAuthCtx, + route.codexAccountMode, + ); + const headers = await materializeCodexUpstreamAuthAsync(req.headers, refreshedAuthCtx, { + admission: options.admission, + config: options.codexAuthPolicy ?? config, + modelId: route.modelId, + substituteMainCredential, + signal: options.abortSignal, + nativeMainRefreshDependencies: options.nativeMainRefreshDependencies, + }); + return { ok: true, authCtx: refreshedAuthCtx, provider, headers }; + } catch (error) { + if (isTerminalPoolRefreshFailure(error)) { + return { + ok: false, + quarantine: true, + response: formatErrorResponse(401, "authentication_error", "Selected Codex account needs reauthentication"), + }; + } + return { + ok: false, + quarantine: false, + response: poolCredentialRefreshIncompleteResponse({ + authCtx, + config, + accountSelector: route.codexAccountNamespace, + logCtx: args.logCtx, + }), + }; + } +} + + +export async function refreshNativeMainForwardAuth(args: { + req: Request; + config: OcxConfig; + route: RouteResult; + authCtx: CodexAuthContext; + substituteMainCredential: boolean; + options: HandleResponsesOptions; +}): Promise< + | { ok: true; authCtx: CodexAuthContext; provider: OcxProviderConfig; headers: Headers } + | { ok: false; response: Response } +> { + const { req, config, route, authCtx, substituteMainCredential, options } = args; + if (authCtx.kind !== "main-pool") { + return { ok: false, response: formatErrorResponse(401, "authentication_error", "No native main credential to refresh") }; + } + try { + const refreshed = await forceRefreshMainAccountToken(authCtx.accessToken, { + signal: options.abortSignal, + ...(options.nativeMainRefreshDependencies ?? {}), + }); + if (!refreshed) { + return { ok: false, response: formatErrorResponse(401, "authentication_error", "Codex main account needs reauthentication") }; + } + const refreshedAuthCtx: CodexAuthContext = { + ...authCtx, + accessToken: refreshed.accessToken, + chatgptAccountId: refreshed.chatgptAccountId, + }; + const provider = applyCodexAuthContextToProvider( + stripCodexRuntimeProviderFields(route.provider), + refreshedAuthCtx, + route.codexAccountMode, + ); + const headers = await materializeCodexUpstreamAuthAsync(req.headers, refreshedAuthCtx, { + admission: options.admission, + config: options.codexAuthPolicy ?? config, + modelId: route.modelId, + substituteMainCredential, + signal: options.abortSignal, + nativeMainRefreshDependencies: options.nativeMainRefreshDependencies, + }); + return { ok: true, authCtx: refreshedAuthCtx, provider, headers }; + } catch (error) { + if (options.abortSignal?.aborted || req.signal.aborted) { + return { ok: false, response: clientCancelledResponse() }; + } + return { ok: false, response: mapCodexAuthContextErrorToResponse(error, { + now: Date.now(), accountSelector: route.codexAccountNamespace, + }) ?? nativeMainRefreshFailureResponse(error) }; + } +} diff --git a/src/server/responses/core-codex-account.ts b/src/server/responses/core-codex-account.ts new file mode 100644 index 0000000000..5d0fc50109 --- /dev/null +++ b/src/server/responses/core-codex-account.ts @@ -0,0 +1,859 @@ +import type { OcxConfig, OcxProviderConfig, OcxParsedRequest } from "../../types"; +import type { CodexAuthContext, CodexAuthPolicyConfig } from "../../codex/auth-context"; +import type { CodexUpstreamOutcome } from "../../codex/routing"; +import { + recordCodexUpstreamOutcome, + computeQuotaCooldown, + formatCodexProviderForLog, +} from "../../codex/routing"; +import type { CodexWsQuotaObserver } from "./codex-ws-metadata"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import { isCodexAccountGenerationLive } from "../../codex/account-store"; +import { applyAccountQuotaFromUpstreamHeaders as applyCapturedCodexQuota } from "../../codex/quota"; +import type { RouteResult } from "../../router"; +import { + normalizeUpstreamHostCircuitThreshold, + upstreamHostHealthKey, + resetUpstreamHostHealth, +} from "../../codex/upstream-host-health"; +import { safeOriginLabel, fetchWithHeaderTimeout, providerFetch } from "./fetch-helpers"; +import { formatErrorResponse } from "../../bridge"; +import { readBoundedResponseBody } from "../../lib/bounded-body"; +import { upstreamErrorMessageFromPayload, isRateLimitOrQuotaFailureMessage } from "../../lib/errors"; +import { isNonReplayableResponse, isTransientUpstreamStatus } from "../../lib/upstream-retry"; +import type { RequestLogContext } from "../request-log"; +import type { DataPlaneAdmission } from "../auth-cors"; +import type { InboundWire } from "../../providers/registry"; +import type { BunRuntimeGateInput } from "./ws-upstream"; +import type { TranslatorBudget } from "../../lib/translator-budget"; +import type { AdmissionLease } from "../../lib/admission"; +import { + resolveCodexModelEntitlements, + invalidateCodexModelEntitlementsForAccount, + entitledCodexAccountIdsForModel, +} from "../../codex/model-entitlements"; +import type { TransientSendBudget } from "../../lib/upstream-retry"; +import { resolveAdapter, resolveWireProtocolOverride } from "../adapter-resolve"; +import { codexAccountSelectionForTurn } from "../lifecycle"; +import { isNativeMainTrafficBlocked } from "../../codex/native-profile-startup"; +import { MAIN_CODEX_ACCOUNT_ID } from "../../codex/main-account"; +import { slugsEquivalent } from "../../providers/slug-codec"; +import { + codexProbeLeaseId, + codexProbeQuotaScope, + releaseCodexAuthContextProbeLease, + resolveCodexAuthContext, + CodexPoolAuthenticationError, + CodexAuthContextError, + CodexAccountCooldownError, + CodexMainProfileDrainingError, + headersForCodexAuthContext, + applyCodexAuthContextToProvider, + stripCodexRuntimeProviderFields, + createCodexReserveDispatchGuard, +} from "../../codex/auth-context"; +import { ACCOUNT_GATED_NATIVE_OPENAI_MODELS } from "../../codex/catalog/native-models"; +import { isRequestExecutionBudget } from "../../lib/request-execution-budget"; +import type { SingleUseDispatchPermit } from "../../lib/request-execution-budget"; +import { hasForwardableCodexBearer } from "../auth-cors"; +import { bindRouteReasoningReplayScope } from "./core-replay"; +import { + conversationStateBindingFromAuth, + applyAccountChangeConversationStateScrub, +} from "./account-change-state"; +import { + recordAdapterReasoning, + recordAdapterTier, + sealRequestAttemptIdentity, + recordAttemptCredentialSource, + noteAttemptSend, +} from "../request-log"; +import { codexAuthContextLogLabel } from "../../codex/account-label"; +import { chargeWorkflowSends } from "../../lib/workflow-budget"; +import type { ResponsesTerminalStatus } from "../../bridge"; + +export function sidecarOutcomeRecorder( + config: OcxConfig, + authCtx: CodexAuthContext, +): ((outcome: CodexUpstreamOutcome) => void) | undefined { + return authCtx.kind === "pool" || authCtx.kind === "main-pool" + ? outcome => recordCodexUpstreamOutcome(config, authCtx.accountId, outcome, { + threadId: authCtx.affinityKey, + fixedAccount: authCtx.fixedAccount, + probeLeaseId: authCtx.probeLeaseId, + probeQuotaScope: authCtx.probeQuotaScope, + writerGeneration: authCtx.writerGeneration, + // A vision or web-search sidecar can return 401/403, and that is evidence about the exact + // stored credential it used. Without the generation it becomes an account-wide quarantine + // that a replacement inherits (#2892 gap 4). `main-pool` has no stored-record generation, so + // it keeps the unfenced account-wide semantics. + ...(authCtx.kind === "pool" ? { credentialGeneration: authCtx.generation } : {}), + }) + : undefined; +} + + + + +export function codexLogAccountId(authCtx: CodexAuthContext): string | null { + return authCtx.kind === "pool" || authCtx.kind === "main-pool" ? authCtx.accountId : null; +} + + +export function isFixedCodexAccount(authCtx: CodexAuthContext): boolean { + return (authCtx.kind === "pool" || authCtx.kind === "main-pool") + && authCtx.fixedAccount === true; +} + + +export function usesCodexForwardPoolAuth( + authCtx: CodexAuthContext, + provider: OcxProviderConfig, +): authCtx is Extract { + return (authCtx.kind === "pool" || authCtx.kind === "main-pool") + && provider.authMode === "forward" && provider.adapter === "openai-responses"; +} + + +export function codexWsQuotaObserver(authCtx: CodexAuthContext, provider: OcxProviderConfig, modelId?: string): CodexWsQuotaObserver | undefined { + if (!isCanonicalOpenAiForwardProvider(provider) || !usesCodexForwardPoolAuth(authCtx, provider)) return undefined; + const { accountId, writerGeneration } = authCtx; + const credentialGeneration = authCtx.kind === "pool" ? authCtx.generation : undefined; + const mainWriter = authCtx.kind === "main-pool" ? authCtx.mainQuotaWriter : undefined; + return headers => { + if (credentialGeneration !== undefined && !isCodexAccountGenerationLive(accountId, credentialGeneration)) return; + applyCapturedCodexQuota(accountId, headers, writerGeneration, mainWriter, { modelId, poolWriter: authCtx.kind === "pool" ? authCtx.poolQuotaWriter : undefined }); + }; +} + + +export function preAuthUpstreamHostCircuitKey( + route: Pick, + config: OcxConfig, + options: { requireResponsesAdapter?: boolean } = {}, +): string | null { + if ( + normalizeUpstreamHostCircuitThreshold(config.upstreamHostCircuitThreshold) === 0 + || route.codexAccountMode !== "pool" + || route.codexAccountId !== undefined + || route.provider.authMode !== "forward" + || (options.requireResponsesAdapter !== false && route.provider.adapter !== "openai-responses") + ) return null; + return upstreamHostHealthKey(route.providerName, safeOriginLabel(route.provider.baseUrl ?? "")); +} + + +export function upstreamHostCircuitOpenResponse(retryAfterSeconds: number): Response { + return formatErrorResponse( + 503, + "upstream_host_circuit_open", + "Provider host is temporarily unavailable", + { retryAfter: String(retryAfterSeconds) }, + ); +} + + +export function normalizeCodexUnsupportedModelDetail(value: string): string { + return value.trim().replace(/\s+/gu, " ").toLocaleLowerCase("en-US"); +} + + +export function isAllowListedCodexAccountModel400( + status: number, + bodyText: string, + modelId: string, +): boolean { + if (status !== 400) return false; + try { + const payload = JSON.parse(bodyText) as unknown; + if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; + const detail = (payload as { detail?: unknown }).detail; + if (typeof detail !== "string") return false; + const expected = `The '${modelId}' model is not supported when using Codex with a ChatGPT account.`; + return normalizeCodexUnsupportedModelDetail(detail) + === normalizeCodexUnsupportedModelDetail(expected); + } catch { + return false; + } +} + + +export async function shouldRetryCodexPoolAccountModel400( + response: Response, + modelId: string, + signal?: AbortSignal, +): Promise { + if (response.status !== 400) return false; + try { + const body = await readBoundedResponseBody(response.clone(), { signal }); + return body.displaySafe + && !body.truncated + && isAllowListedCodexAccountModel400(response.status, body.text, modelId); + } catch { + return false; + } +} + + +/** Pre-stream quota/billing rejections that warrant one alternate-account attempt (#584). */ +export function codexQuotaFailureMessage(body: string): string | undefined { + try { + const payload = JSON.parse(body) as unknown; + const canonical = upstreamErrorMessageFromPayload(payload); + if (canonical !== undefined) return canonical; + if (typeof payload === "string") return payload; + if (!payload || typeof payload !== "object" || Array.isArray(payload)) return undefined; + const record = payload as Record; + if (typeof record.message === "string") return record.message; + return typeof record.error === "string" ? record.error : undefined; + } catch { + // Plain-text gateways remain supported. Valid JSON is inspected only at recognized + // message fields so echoed request content elsewhere cannot trigger account cooldown. + return body; + } +} + + +export async function shouldRetryCodexPoolAccountQuota( + response: Response, + signal?: AbortSignal, +): Promise { + // A post-send WebSocket gateway status must not become a second account's send; the + // body carries no quota evidence either, but the marker is the contract, not the prose. + if (isNonReplayableResponse(response)) return false; + if (response.status === 402 || response.status === 429) return true; + if (response.status < 500 || response.status >= 600) return false; + try { + // Reject malformed UTF-8 instead of matching quota words around replacement characters. + const body = await readBoundedResponseBody(response.clone(), { signal, fatalUtf8: true }); + const message = body.displaySafe && !body.truncated + ? codexQuotaFailureMessage(body.text) + : undefined; + return message !== undefined + && isRateLimitOrQuotaFailureMessage(message); + } catch { + return false; + } +} + + +/** + * A pre-stream upstream 5xx another Codex account may still be able to serve. + * + * `server_is_overloaded` is the shape this exists for. The ChatGPT backend refuses in a few + * hundred milliseconds, the body carries no quota evidence, and nothing in that exchange is + * account health — so the pool keeps choosing the same account and every request fails on it + * while the other accounts sit idle. That is what an operator sees as the pool refusing to move. + * + * The status stays exactly as upstream sent it. `classifyCodexUpstreamOutcome` maps 5xx to the + * transient class, so the account earns an ordinary failure streak and `upstreamFailoverThreshold` + * decides when it is soft-avoided, rather than a quota cooldown it never earned. + * + * Deliberately narrow. {@link isNonReplayableResponse} still refuses: a post-send WebSocket + * gateway status means the body already reached the origin, so sending it from a second account + * could duplicate a turn the origin may still be running. A 5xx whose body confirms quota is not + * routed here either — {@link shouldRetryCodexPoolAccountQuota} classifies that one first and + * carries the cooldown with it. + */ +export function shouldRetryCodexPoolAccountTransient(response: Response): boolean { + return !isNonReplayableResponse(response) && isTransientUpstreamStatus(response.status); +} + + +export interface CodexPoolAccountRetryArgs { + /** Sanitized caller input, before any selected Pool credential was materialized. */ + callerAuthHeaders: Headers; + config: OcxConfig; + route: { providerName: string; modelId: string; provider: OcxProviderConfig }; + parsed: OcxParsedRequest; + logCtx: RequestLogContext; + options: { + admission?: DataPlaneAdmission; + codexAuthPolicy?: CodexAuthPolicyConfig; + visionDescribeTerminal?: boolean; + abortSignal?: AbortSignal; + onCodexAuthContextResolved?: (ctx: CodexAuthContext) => void; + deferCodexResetDerivedCooldown?: boolean; + // Narrowed subset of HandleResponsesOptions: the retry rebuilds the adapter, so it + // needs the inbound scope or the retry could land on a different wire than the + // first attempt. + inboundWire?: InboundWire; + codexWsRuntimeIdentity?: BunRuntimeGateInput; + translatorBudget: TranslatorBudget; + turnAdmissionLease?: AdmissionLease; + resolveCodexModelEntitlements?: typeof resolveCodexModelEntitlements; + /** The logical request's execution budget: the account move is its fourth send. */ + sendBudget?: TransientSendBudget; + /** Root workflow this turn belongs to, so the move is charged there as well. */ + workflowRootId?: string; + }; + firstAuthCtx: Extract; + firstResponse: Response; + outcomeStatus: number; + /** + * Forbid resolving a DIFFERENT account for this retry. + * + * Set when a stored Pool 401 already spent this logical request's account budget on its own + * refresh and replay. The same-account gated-model retry above stays available, because it + * sends to the account that was already paying; only the alternate-account resolution below is + * out of budget. + */ + sameAccountOnly?: boolean; + upstream: AbortController; + connectMs: number; + passthroughEstimate?: number; + stream: boolean; + onResponse?: ( + response: Response, + authCtx: CodexAuthContext, + request: Awaited["buildRequest"]>>, + ) => void; +} + + +export type CodexPoolAccountRetryResult = + | { + kind: "retried"; + authCtx: CodexAuthContext; + request: Awaited["buildRequest"]>>; + upstreamResponse: Response; + selectedForwardHeaders: Headers; + } + | { kind: "no-alternate" } + | { + kind: "transport"; + error: unknown; + authCtx: CodexAuthContext; + }; + + +/** Keep retry-stage entitlement snapshots inside the native-main selection fence. */ +export async function resolveCodexRetryModelEntitlements( + config: OcxConfig, + resolver: typeof resolveCodexModelEntitlements, + turnAdmissionLease?: AdmissionLease, +): Promise>> { + // The initial auth selection has already released its admission before the first + // response arrives. Re-enter for every refresh so profile switching cannot overlap + // credential discovery, and omit main entirely when a drain or recovery owns it. + const selectionAdmission = codexAccountSelectionForTurn(turnAdmissionLease)?.(); + const nativeMainReadsForbidden = isNativeMainTrafficBlocked() + || selectionAdmission?.mainProfileDraining === true; + try { + return await resolver(config, { + excludeAccountIds: nativeMainReadsForbidden + ? new Set([MAIN_CODEX_ACCOUNT_ID]) + : undefined, + }); + } finally { + selectionAdmission?.release(); + } +} + + +export const CODEX_ACCOUNT_GATED_CANONICAL_WIRE_MODELS: ReadonlyMap = new Map([ + // The authenticated catalog currently advertises Daybreak Blue, while successful responses + // identify the serving model as gpt-5.6-sol. Sending the selector itself is shard-dependent: + // live traffic can receive the exact unsupported-model 400 repeatedly from the same entitled + // account. Keep Daybreak as the admission/catalog identity, but use the stable serving id on + // the credential-bearing wire after entitlement selection has completed. + ["gpt-daybreak-blue-latest", "gpt-5.6-sol"], +]); + + +export function codexAccountGatedCanonicalWireModel(modelId: string): string | undefined { + const exact = CODEX_ACCOUNT_GATED_CANONICAL_WIRE_MODELS.get(modelId); + if (exact) return exact; + for (const [selector, wireModel] of CODEX_ACCOUNT_GATED_CANONICAL_WIRE_MODELS) { + if (slugsEquivalent(modelId, selector)) return wireModel; + } + return undefined; +} + + +export function applyCodexAccountGatedWireNormalization(parsed: OcxParsedRequest, route: RouteResult, logCtx?: RequestLogContext): void { + if (!isCanonicalOpenAiForwardProvider(route.provider)) return; + const wireModel = codexAccountGatedCanonicalWireModel(route.modelId); + if (!wireModel) return; + + if (logCtx) { + logCtx.preserveResolvedModelFromRoute = true; + delete logCtx.resolvedModel; + } + parsed.modelId = wireModel; + if (!parsed._rawBody || typeof parsed._rawBody !== "object") return; + const raw = parsed._rawBody as Record; + raw.model = wireModel; + // Daybreak's authenticated catalog does not advertise retention support, and the upstream + // rejects this optional Codex hint before model execution. Removing it preserves request + // semantics while avoiding an otherwise terminal pre-stream 400. + delete raw.prompt_cache_retention; +} + + +/** + * Workspace-denial evidence for a 403, read from the upstream body. + * + * #1789: a valid K12 credential gets 403 `codex_workspace_access_denied` on a routed prompt. + * Without this the account is quarantined for reauthentication, which cannot fix a workspace + * grant and loops forever. Fails closed: an unreadable body keeps the historical handling. + */ +export async function codexDenialOutcomeMeta(response: Response): Promise<{ denial?: "workspace" | "entitlement" }> { + if (response.status !== 403) return {}; + const { classifyCodexPreStreamRejection } = await import("../../codex/quota-rejection"); + const rejection = await classifyCodexPreStreamRejection(response); + return rejection.denial ? { denial: rejection.denial } : {}; +} + + +export function codexQuotaOutcomeMeta(response: Response): { + retryAfter: string | null; + resetAt: string[]; +} { + return { + retryAfter: response.headers.get("retry-after"), + resetAt: [ + response.headers.get("x-codex-primary-reset-at"), + response.headers.get("x-codex-secondary-reset-at"), + response.headers.get("x-codex-tertiary-reset-at"), + ].filter((value): value is string => !!value), + }; +} + + +/** + * A reset timestamp describes a quota window, not an explicit instruction to + * stop using the whole account. A combo may therefore try a later model in the + * same request, while Retry-After and headerless quota failures remain blocking. + */ +export function shouldDeferCodexResetDerivedCooldown(response: Response, enabled?: boolean): boolean { + return enabled === true + && (response.status === 429 || response.status === 402) + && computeQuotaCooldown(codexQuotaOutcomeMeta(response)).source === "reset-derived"; +} + + +/** + * One bounded alternate-account retry for Codex pool auth. Used for allow-listed + * model-400 and for pre-stream 429/402 quota failures (#584). + */ +export async function retryCodexPoolOnAlternateAccount( + args: CodexPoolAccountRetryArgs, +): Promise { + const { + callerAuthHeaders, config, route, parsed, logCtx, options, firstAuthCtx, firstResponse, + outcomeStatus, upstream, connectMs, passthroughEstimate, stream, + } = args; + const inboundWire = options.inboundWire ?? "responses"; + const entitlementResolver = options.resolveCodexModelEntitlements ?? resolveCodexModelEntitlements; + let retryAuthCtx: CodexAuthContext | undefined; + // A transient 5xx must record even when this request cannot move: the ordinary terminal + // recorder only fires for an OK event-stream body, so a pre-stream refusal would otherwise + // leave the account looking healthy no matter how many times it refused, and the pool would + // keep handing it the next request. + const recordUnmovedTransientOutcome = (): void => { + if (!isTransientUpstreamStatus(outcomeStatus)) return; + recordCodexUpstreamOutcome(config, firstAuthCtx.accountId, outcomeStatus, { + threadId: firstAuthCtx.affinityKey, + fixedAccount: firstAuthCtx.fixedAccount, + modelId: route.modelId, + probeLeaseId: codexProbeLeaseId(firstAuthCtx), + probeQuotaScope: codexProbeQuotaScope(firstAuthCtx), + writerGeneration: firstAuthCtx.writerGeneration, + }); + }; + if (outcomeStatus === 400 && ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(route.modelId)) { + invalidateCodexModelEntitlementsForAccount(firstAuthCtx.accountId); + let refreshed; + try { + refreshed = await resolveCodexRetryModelEntitlements( + config, + entitlementResolver, + options.turnAdmissionLease, + ); + } catch (error) { + await firstResponse.body?.cancel().catch(() => undefined); + releaseCodexAuthContextProbeLease(firstAuthCtx); + throw error; + } + if (entitledCodexAccountIdsForModel(refreshed, route.modelId)?.has(firstAuthCtx.accountId)) { + // The authenticated roster still grants this exact model. Retry on the same account: + // upstream shards can briefly disagree during a gated-model rollout, but a pre-stream 400 + // proves no output was committed and keeps this replay bounded. + retryAuthCtx = firstAuthCtx; + } + } + // Exact account selectors may retry the same confirmed account above, but must never resolve + // an alternate. Quota failures and a refreshed entitlement miss remain terminal. + if (!retryAuthCtx && (firstAuthCtx.fixedAccount || args.sameAccountOnly === true)) { + recordUnmovedTransientOutcome(); + return { kind: "no-alternate" }; + } + // An account move is the guarded profile's fourth send and draws the single shared + // final-recovery reserve. Nothing bounded it per request before: `excludeAccountId` excludes + // only the account that just failed, and the caller's recovery loop can return here after the + // alternate fails too, so one request could walk the pool an account at a time. The permit is + // consumed immediately before the physical send, so a resolution that finds no alternate + // costs nothing. + const executionBudget = isRequestExecutionBudget(args.options.sendBudget) + ? args.options.sendBudget + : undefined; + let accountMovePermit: SingleUseDispatchPermit | undefined; + if (!retryAuthCtx && executionBudget) { + const decision = executionBudget.reserveDispatch({ + sendClass: "account-failover", + targetKey: `${route.providerName}|${route.modelId}|alternate-account`, + }); + if (!decision.allowed) { + recordUnmovedTransientOutcome(); + return { kind: "no-alternate" }; + } + accountMovePermit = decision.permit; + } + try { + retryAuthCtx ??= await resolveCodexAuthContext( + callerAuthHeaders, + config, + "pool", + { + excludeAccountId: firstAuthCtx.accountId, + admission: options.admission, + codexAuthPolicy: options.codexAuthPolicy, + modelId: route.modelId, + requestScopedMainCredential: hasForwardableCodexBearer(callerAuthHeaders, config), + beginCodexAccountSelection: codexAccountSelectionForTurn(options.turnAdmissionLease), + resolveCodexModelEntitlements: entitlementResolver, + }, + ); + } catch (error) { + const unexpectedRetryError = + !(error instanceof CodexPoolAuthenticationError) + && !(error instanceof CodexAuthContextError) + && !(error instanceof CodexAccountCooldownError) + && !(error instanceof CodexMainProfileDrainingError); + if (unexpectedRetryError) { + // The reservation is the charge now, so an abandoned move has to hand its send back. + accountMovePermit?.release(); + await firstResponse.body?.cancel().catch(() => undefined); + releaseCodexAuthContextProbeLease(firstAuthCtx); + throw error; + } + } + // A validated request-owned main bearer is a real alternate when the failed credential was a + // stored Pool account. It has no Pool account id to promote or cool, but it can own this one + // bounded replay. The resolver already refuses it when main itself is the excluded credential. + if ( + retryAuthCtx?.kind !== "pool" + && retryAuthCtx?.kind !== "main-pool" + && retryAuthCtx?.kind !== "main" + ) { + // A body-confirmed quota response may arrive under HTTP 5xx. Without an alternate, + // the ordinary terminal recorder sees only that wire status and would misclassify it + // as transient, leaving the exhausted account immediately selectable next turn. + if (outcomeStatus !== firstResponse.status && (outcomeStatus === 429 || outcomeStatus === 402)) { + recordCodexUpstreamOutcome(config, firstAuthCtx.accountId, outcomeStatus, { + ...codexQuotaOutcomeMeta(firstResponse), + threadId: firstAuthCtx.affinityKey, + modelId: route.modelId, + probeLeaseId: codexProbeLeaseId(firstAuthCtx), + probeQuotaScope: codexProbeQuotaScope(firstAuthCtx), + writerGeneration: firstAuthCtx.writerGeneration, + }); + } + // No usable alternate was resolved, so the reserved move never becomes a send. + accountMovePermit?.release(); + recordUnmovedTransientOutcome(); + return { kind: "no-alternate" }; + } + + const quotaMeta = { ...codexQuotaOutcomeMeta(firstResponse), ...(await codexDenialOutcomeMeta(firstResponse)) }; + if (outcomeStatus === 429 || outcomeStatus === 402) { + const { applyAccountQuotaFromUpstreamHeaders } = await import("../../codex/auth-api"); + applyAccountQuotaFromUpstreamHeaders( + firstAuthCtx.accountId, + firstResponse.headers, + firstAuthCtx.writerGeneration, + firstAuthCtx.kind === "main-pool" ? firstAuthCtx.mainQuotaWriter : undefined, + { modelId: route.modelId, poolWriter: firstAuthCtx.kind === "pool" ? firstAuthCtx.poolQuotaWriter : undefined }, + ); + } + const deferFirstOutcome = shouldDeferCodexResetDerivedCooldown( + firstResponse, + options.deferCodexResetDerivedCooldown, + ); + const recordFirstOutcome = (): void => { + recordCodexUpstreamOutcome(config, firstAuthCtx.accountId, outcomeStatus, { + ...quotaMeta, + threadId: firstAuthCtx.affinityKey, + modelId: route.modelId, + probeLeaseId: codexProbeLeaseId(firstAuthCtx), + probeQuotaScope: codexProbeQuotaScope(firstAuthCtx), + writerGeneration: firstAuthCtx.writerGeneration, + // Retry already advanced the RR ring via excludeAccountId — reuse for promotion. + ...(retryAuthCtx.accountId ? { promoteAccountId: retryAuthCtx.accountId } : {}), + }); + }; + // Only a combo reset-derived outcome is deferred. Retry-After, defaults, and + // ordinary requests must block the first account before the alternate send. + if (!deferFirstOutcome) recordFirstOutcome(); + const retryHeaders = headersForCodexAuthContext(callerAuthHeaders, retryAuthCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission); + const retryProvider = applyCodexAuthContextToProvider( + stripCodexRuntimeProviderFields(route.provider), + retryAuthCtx, + "pool", + ); + const retryAdapter = resolveAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, retryProvider, inboundWire), + config.cacheRetention, + route.providerName, + ); + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: retryProvider, + adapterName: retryAdapter.name, + codexAuthContext: retryAuthCtx, + forwardHeaders: retryHeaders, + }); + { + const binding = conversationStateBindingFromAuth( + retryAuthCtx, + firstAuthCtx.kind === "pool" || firstAuthCtx.kind === "main-pool" + ? firstAuthCtx.affinityKey + : undefined, + ); + if (binding) { + applyAccountChangeConversationStateScrub({ + body: parsed._rawBody, + parsed, + bindingKey: binding.bindingKey, + servingAccountId: binding.accountId, + priorAccountId: firstAuthCtx.accountId, + logCtx, + }); + } + } + const request = await retryAdapter.buildRequest(parsed, { + headers: retryHeaders, + translatorBudget: options.translatorBudget, + }); + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + + await firstResponse.body?.cancel().catch(() => undefined); + options.onCodexAuthContextResolved?.(retryAuthCtx); + route.provider = retryProvider; + logCtx.provider = formatCodexProviderForLog( + route.providerName, + retryAuthCtx.accountId, + config, + ); + logCtx.accountLogLabel = codexAuthContextLogLabel(retryAuthCtx, config); + sealRequestAttemptIdentity( + logCtx.activeAttempt, + logCtx.provider, + retryAdapter.name, + logCtx.accountLogLabel, + ); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, retryAdapter.name); + + const retrySameConfirmedAccount = outcomeStatus === 400 + && ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(route.modelId) + && retryAuthCtx.accountId === firstAuthCtx.accountId; + // Live Daybreak traffic has produced long runs of unsupported-model 400s from different + // upstream shards even while the authenticated roster continues to grant the model. Permit + // seven additional same-account sends (eight total including the original), re-checking the + // exact allow-listed body and fresh entitlement before every later send. Alternate-account and + // quota recovery retain their historical one-send bound. + // + // Two different bounds, and the effective one is the smaller. `maxRetrySends` answers "how + // many times is it worth re-asking THIS account for a model its roster still grants"; the + // shared budget answers "how many times may this LOGICAL REQUEST reach upstream in total, + // across every layer that can re-send". A ladder of eight layered on sends the request had + // already made is exactly the per-request multiplication #4546 is about, so the ladder is + // capped at what the request has left. The floor of one keeps the single retry this function + // was called to make -- the move already paid for itself with its own permit -- and each rung + // past the first reserves its own send below, so a refusal stops the ladder with the last + // upstream answer intact. + // The ladder replays to the SAME account, so it must reserve under the same target key the + // other legs use. Folding the account id in made every rung read as a target change, which + // spent the one cross-account slot a real move needs on a same-account replay. + const ladderTargetKey = `${route.providerName}|${route.modelId}`; + // The ladder keeps its OWN bound rather than drawing on what the request has left. Clamping it + // to the shared total looked right and broke a working, pinned path: #2097 fixes this recovery + // at eight same-account dispatches (tests/server/server-auth.test.ts), and a request that has + // already spent sends would silently stop short of it. Reconciling an eight-send same-account + // ladder with a four-send request total is a policy decision, not a clamp to add in passing. + // What this diff does fix is that the rungs are now CHARGED instead of free. + const maxRetrySends = retrySameConfirmedAccount ? 7 : 1; + let retrySendCount = 0; + let upstreamResponse: Response; + try { + while (true) { + // The same-account gated-model 400 ladder below keeps its own `maxRetrySends` bound and + // does not take the reserve again; only the move itself does. + if (accountMovePermit) { + const charged = accountMovePermit.use(); + accountMovePermit = undefined; + if (!charged) { + recordUnmovedTransientOutcome(); + return { kind: "no-alternate" }; + } + // The move is a physical send like any other, so the root workflow is charged too. + chargeWorkflowSends(args.options.workflowRootId, 1); + } + noteAttemptSend(logCtx.activeAttempt, passthroughEstimate); + try { + upstreamResponse = await fetchWithHeaderTimeout( + request.url, + { + method: request.method, + headers: request.headers, + body: request.body, + }, + upstream.signal, + connectMs, + stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + providerName: route.providerName, + modelId: route.modelId, + onCodexWsQuota: codexWsQuotaObserver(retryAuthCtx, route.provider, route.modelId), + beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) + ? createCodexReserveDispatchGuard(retryAuthCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, + }), + // Credential-bearing forward send: never follow a redirect into a + // dead-host rejection after the credential was seen (#914). + route.provider.authMode === "forward", + ); + } catch (error) { + // Only the forward send is a transport boundary. Entitlement resolver throws below are + // deliberately outside this catch so programming errors retain their original path. + return { kind: "transport", error, authCtx: retryAuthCtx }; + } + retrySendCount += 1; + args.onResponse?.(upstreamResponse, retryAuthCtx, request); + if (!retrySameConfirmedAccount || retrySendCount >= maxRetrySends) break; + // Caller-owned main is an alternate-account replay and can never enter the bounded + // same-stored-account 400 loop above. Keep that invariant explicit for the account-id reads. + if (retryAuthCtx.kind === "main") break; + if (!await shouldRetryCodexPoolAccountModel400( + upstreamResponse, + route.modelId, + options.abortSignal, + )) break; + invalidateCodexModelEntitlementsForAccount(retryAuthCtx.accountId); + let refreshed: Awaited>; + try { + refreshed = await resolveCodexRetryModelEntitlements( + config, + entitlementResolver, + options.turnAdmissionLease, + ); + } catch (error) { + await upstreamResponse.body?.cancel().catch(() => undefined); + await firstResponse.body?.cancel().catch(() => undefined); + releaseCodexAuthContextProbeLease(firstAuthCtx); + releaseCodexAuthContextProbeLease(retryAuthCtx); + throw error; + } + if (!entitledCodexAccountIdsForModel(refreshed, route.modelId)?.has(retryAuthCtx.accountId)) break; + // The next rung is another physical send of this logical request: a same-account, + // same-target replay, charged as an ordinary transient send rather than as a move. + // Reserved here, immediately before looping back, so a refusal stops the ladder with the + // last upstream 400 intact instead of spending a send it cannot make. + // Every rung is CHARGED, and a refusal does not end the ladder. That asymmetry is + // deliberate and it is the one place the shared cap yields. This is a same-account, + // same-target replay of a model-gating 400 whose own bound is eight dispatches, pinned by + // #2097; letting a spent request budget cut it to four would break a recovery that works + // today, which is precisely the mistake 040_send_budget.md warns a flat ceiling makes. + // The request total still governs everything that changes target or credential. + if (executionBudget) { + const rung = executionBudget.reserveDispatch({ + sendClass: "transient", + targetKey: ladderTargetKey, + }); + if (rung.allowed) rung.permit.use(); + chargeWorkflowSends(args.options.workflowRootId, 1); + } + await upstreamResponse.body?.cancel().catch(() => undefined); + } + } finally { + request.releaseBodyObservation?.(); + } + // A real HTTP response proves the host was reached (#914). + const retryHostKey = upstreamHostHealthKey(route.providerName, safeOriginLabel(request.url)); + if (normalizeUpstreamHostCircuitThreshold(config.upstreamHostCircuitThreshold) > 0) { + resetUpstreamHostHealth(retryHostKey, null); + } else { + resetUpstreamHostHealth(retryHostKey); + } + if (deferFirstOutcome && upstreamResponse.ok) { + // Deferral keeps the first account eligible for a later combo model while an + // alternate attempt is still fallible. Commit its quota outcome only once the + // alternate account returns a successful HTTP response; otherwise the combo may + // still need the first account for its next target. + recordFirstOutcome(); + } + return { + kind: "retried", + authCtx: retryAuthCtx, + request, + upstreamResponse, + selectedForwardHeaders: retryHeaders, + }; +} + + + + +export function codexForwardTerminalOutcomeRecorder( + config: OcxConfig, + authCtx: CodexAuthContext, + provider: OcxProviderConfig, + modelId?: string, + logCtx?: RequestLogContext, +): ((status: ResponsesTerminalStatus, httpStatusOverride?: number) => void) | undefined { + if (!usesCodexForwardPoolAuth(authCtx, provider)) return undefined; + return (status, httpStatusOverride) => { + const quotaStatus = [httpStatusOverride, logCtx?.terminalHttpStatus] + .find(value => value === 429 || value === 402); + if (status === "incomplete" && quotaStatus === undefined) { + // Normal limit/content-filter/stall terminal — the account served the + // request. Don't penalize account health; record success to clear any + // prior soft-avoid so a healthy account isn't stuck avoided. + recordCodexUpstreamOutcome(config, authCtx.accountId, 200, { + threadId: authCtx.affinityKey, + fixedAccount: authCtx.fixedAccount, + modelId, + probeLeaseId: codexProbeLeaseId(authCtx), + probeQuotaScope: codexProbeQuotaScope(authCtx), + writerGeneration: authCtx.writerGeneration, + }); + return; + } + // status === "completed" or "failed": use the semantic HTTP status derived + // from the terminal SSE error payload (httpStatusFromTerminalError in + // request-log inspection) instead of collapsing every non-completed terminal + // to 502. A 400 invalid_request_error must not soft-avoid the account or + // rebind threads — only genuine transport/5xx failures should trigger + // transient health recording. + // httpStatusOverride: the combo WS path inspects SSE payloads into the parent + // logCtx, but this recorder closes over the child logCtx. The caller passes + // the parent's terminalHttpStatus so the semantic status is not lost. + const outcome = status === "completed" + ? 200 + : (quotaStatus ?? httpStatusOverride ?? logCtx?.terminalHttpStatus ?? 502); + recordCodexUpstreamOutcome(config, authCtx.accountId, outcome, { + threadId: authCtx.affinityKey, + fixedAccount: authCtx.fixedAccount, + modelId, + probeLeaseId: codexProbeLeaseId(authCtx), + probeQuotaScope: codexProbeQuotaScope(authCtx), + writerGeneration: authCtx.writerGeneration, + // A mid-stream terminal can carry a semantic 401 long after the credential was + // replaced. It is never replayed — the client already saw output — but it must + // not retire the replacement either (#2887). + ...(authCtx.kind === "pool" ? { credentialGeneration: authCtx.generation } : {}), + }); + }; +} diff --git a/src/server/responses/core-combo-failure.ts b/src/server/responses/core-combo-failure.ts new file mode 100644 index 0000000000..c35fc90a3d --- /dev/null +++ b/src/server/responses/core-combo-failure.ts @@ -0,0 +1,210 @@ +import { parseRetryAfterMs } from "../../combos"; +import type { ConsumedComboFailure, HandleResponsesOptions } from "./core-options"; +import type { OcxUsage } from "../../types"; +import { readBoundedResponseBody } from "../../lib/bounded-body"; +import { codexQuotaFailureMessage, codexQuotaOutcomeMeta } from "./core-codex-account"; +import { + isRateLimitOrQuotaFailureMessage, + isCyberPolicyCode, + isCyberPolicyMessage, + CYBER_POLICY_ERROR_CODE, + CYBER_POLICY_FALLBACK_MESSAGE, +} from "../../lib/errors"; +import { normalizeUpstreamErrorText } from "./core-errors"; +import { resolveClientRetryAfter } from "../../lib/retry-after"; +import { formatErrorResponse } from "../../bridge"; +import { usageFromResponsesPayload } from "../request-log"; +import type { ResponsesTerminalStatus } from "../../bridge"; + +export function sanitizedRetryAfter(value: string | null, now: number): string | undefined { + const trimmed = value?.trim(); + if (!trimmed || trimmed.length > 128) return undefined; + return parseRetryAfterMs(trimmed, now) !== undefined ? trimmed : undefined; +} + + + + +export async function consumeComboFailure( + response: Response, + signal?: AbortSignal, + now = Date.now(), +): Promise { + const fallback = `Provider error ${response.status}`; + let classificationText = fallback; + let usage: OcxUsage | undefined; + let upstreamCode: string | undefined; + let upstreamMessage: string | undefined; + let upstreamType: string | undefined; + // Whether the body itself confirms a quota/rate-limit refusal, computed on the SAME read as + // the classification below. `shouldRetryCodexPoolAccountQuota` cannot be called here without + // a second body read, so this mirrors its normalization: raw 402/429, or a 5xx whose intact, + // display-safe body carries a recognized quota message. + let quotaConfirmedByBody = false; + try { + const body = await readBoundedResponseBody(response, { + signal, + // Match shouldRetryCodexPoolAccountQuota before treating a 5xx body as quota evidence. + fatalUtf8: response.status >= 500 && response.status < 600, + }); + usage = usageFromComboFailureText(body.text); + if ( + response.status >= 500 && response.status < 600 + && body.displaySafe && !body.truncated + ) { + const quotaMessage = codexQuotaFailureMessage(body.text); + quotaConfirmedByBody = quotaMessage !== undefined + && isRateLimitOrQuotaFailureMessage(quotaMessage); + } + if (body.displaySafe) { + const normalized = normalizeUpstreamErrorText(body.text, fallback); + classificationText = normalized.safeText; + upstreamCode = normalized.code; + upstreamMessage = normalized.message; + upstreamType = normalized.type; + } + } catch (error) { + if (signal?.aborted) throw error; + classificationText = fallback; + } + const cyberFailure = isCyberPolicyCode(upstreamCode) || isCyberPolicyMessage(classificationText); + const normalizedUpstreamCode = cyberFailure ? CYBER_POLICY_ERROR_CODE : upstreamCode; + const message = cyberFailure + ? upstreamMessage + ?? (isCyberPolicyCode(upstreamCode) ? CYBER_POLICY_FALLBACK_MESSAGE : classificationText) + : classificationText === fallback + ? fallback + : `${fallback}: ${classificationText}`; + const upstreamRetryAfter = response.headers.get("retry-after"); + // Past HTTP dates are an immediate retry directive, just like the numeric value zero. + // Normalize before the client helper discards them and substitutes a default delay. + const effectiveRetryAfter = parseRetryAfterMs(upstreamRetryAfter, now) === undefined + && parseRetryAfterMs(upstreamRetryAfter, now, { preserveImmediate: true }) !== undefined + ? "0" + : upstreamRetryAfter; + // Client response may get the synthetic "2" fallback; cooldown metadata must not — + // otherwise coolComboTarget treats it as a 2s cooldown instead of the 60s default. + const clientRetryAfter = resolveClientRetryAfter({ + status: response.status, + message, + upstreamRetryAfter: effectiveRetryAfter, + now, + }); + const cooldownRetryAfter = resolveClientRetryAfter({ + status: response.status, + message, + upstreamRetryAfter: effectiveRetryAfter, + now, + includeDefault: false, + }); + return { + response: formatErrorResponse( + response.status, + cyberFailure ? (upstreamType ?? CYBER_POLICY_ERROR_CODE) : "upstream_error", + message, + { + ...(normalizedUpstreamCode !== undefined ? { code: normalizedUpstreamCode } : {}), + ...(clientRetryAfter !== undefined ? { retryAfter: clientRetryAfter } : {}), + }, + ), + classificationText, + ...(normalizedUpstreamCode !== undefined ? { upstreamCode: normalizedUpstreamCode } : {}), + ...(!cyberFailure && cooldownRetryAfter !== undefined ? { retryAfter: cooldownRetryAfter } : {}), + // The EFFECTIVE classification decides, not the raw status. An upstream that wraps a quota + // refusal in a 5xx still carries `x-codex-*-reset-at`, and gating on 402/429 alone threw + // those away, so the combo target came back up immediately instead of waiting for the + // window it was told about. `cyberFailure` stays excluded: a policy block is not a quota. + ...(!cyberFailure + && (response.status === 429 || response.status === 402 || quotaConfirmedByBody) + ? { resetAt: codexQuotaOutcomeMeta(response).resetAt } + : {}), + ...(usage ? { usage } : {}), + }; +} + + + + +export function usageFromComboFailureText(text: string): OcxUsage | undefined { + try { + const payload = JSON.parse(text) as Record; + const nested = payload.response; + const source = nested && typeof nested === "object" && !Array.isArray(nested) + ? nested as Record + : payload; + return usageFromResponsesPayload(source.usage); + } catch { + return undefined; + } +} + + + + +export function createChildPassthroughCallbackGate(options: HandleResponsesOptions) { + type Pending = + | { kind: "terminal"; status: ResponsesTerminalStatus } + | { kind: "cancel" }; + let state: "pending" | "committed" | "discarded" = "pending"; + let pending: Pending | undefined; + let accepted = false; + let pendingModel: string | undefined; + let completionAccepted = false; + let completionRejected = false; + const publish = (value: Pending): void => { + if (value.kind === "terminal") options.onNativePassthroughTerminal?.(value.status); + else options.onNativePassthroughCancel?.(); + }; + const publishCompletion = (): void => { + if (state !== "committed" || completionRejected || pendingModel === undefined) return; + const model = pendingModel; + pendingModel = undefined; + options.onResponseComplete?.(model); + }; + const receive = (value: Pending): void => { + if (state === "discarded" || accepted) return; + accepted = true; + if (value.kind === "cancel" || value.status !== "completed") { + completionRejected = true; + pendingModel = undefined; + } + if (state === "committed") return publish(value); + pending ??= value; + }; + return { + onTerminal: (status: ResponsesTerminalStatus) => receive({ kind: "terminal", status }), + onCancel: () => receive({ kind: "cancel" }), + onResponseComplete: (model: string) => { + if (state === "discarded" || completionRejected || completionAccepted || !model.trim()) return; + completionAccepted = true; + pendingModel = model; + publishCompletion(); + }, + commit: () => { + if (state !== "pending") return; + state = "committed"; + if (pending) publish(pending); + pending = undefined; + publishCompletion(); + }, + discard: () => { + state = "discarded"; + pending = undefined; + pendingModel = undefined; + }, + }; +} + + + +export function buildComboChildHeaders(parentHeaders: HeadersInit): Headers { + const childHeaders = new Headers(parentHeaders); + // A provisional caller credential is not authoritative for a Combo child. + childHeaders.delete("authorization"); + childHeaders.delete("chatgpt-account-id"); + // Combo children re-serialize already-decoded JSON. Keeping transport metadata from + // the parent would make the child decoder treat plain JSON as compressed bytes. + childHeaders.delete("content-length"); + childHeaders.delete("content-encoding"); + return childHeaders; +} diff --git a/src/server/responses/core-combo.ts b/src/server/responses/core-combo.ts new file mode 100644 index 0000000000..43863fc294 --- /dev/null +++ b/src/server/responses/core-combo.ts @@ -0,0 +1,707 @@ +import { + CODEX_TEXT_GUARDED_BUDGET_POLICY, + createRequestExecutionBudget, + isRequestExecutionBudget, +} from "../../lib/request-execution-budget"; +import type { + RequestExecutionBudgetPolicy, + RequestExecutionBudget, +} from "../../lib/request-execution-budget"; +import type { OcxConfig } from "../../types"; +import type { RequestLogContext } from "../request-log"; +import type { HandleResponsesOptions, ResponsesDispatchers, ConsumedComboFailure } from "./core-options"; +import type { TranslatorBudget } from "../../lib/translator-budget"; +import { + getCombo, + comboRequestHasImageInput, + pickComboTargetWithWait, + targetKey, + concreteComboRequestBody, + comboDefaultEffort, + isComboTargetInCooldown, + noteComboSuccess, + comboFailureDecision, + advanceComboAfterFailure, + comboFailureCooldownScope, +} from "../../combos"; +import { formatErrorResponse } from "../../bridge"; +import { + expandPreviousResponseInput, + previousResponseScopeMismatch, + previousResponseReplayFailure, + previousResponseProviderState, +} from "../../responses/state"; +import { hasUnreadableEncryptedAgentTask } from "./encrypted-payload"; +import { routeConcreteModel, comboRouteDecisionTrace } from "../../router"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import type { AgentTaskRecoveryFailureReason } from "./agent-task-recovery"; +import { + agentTaskRecoveryConfig, + discardEncryptedAgentTaskRecovery, + recoverEncryptedAgentTaskWithResult, +} from "./agent-task-recovery"; +import { isThreadSpawnRequest, supportedLadderFor } from "../effort-policy"; +import { + clientCancelledResponse, + comboUnavailable, + unreadableEncryptedAgentTaskResponse, +} from "./core-errors"; +import { + buildComboChildHeaders, + createChildPassthroughCallbackGate, + consumeComboFailure, +} from "./core-combo-failure"; +import { linkRequestSessionLane, sessionLaneIdFromRequest } from "../request-log-conversation"; +import type { CodexAuthContext } from "../../codex/auth-context"; +import type { ResponsesTerminalStatus } from "../../bridge"; +import { beginRequestAttempt, sealRequestAttemptIdentity, finishRequestAttempt } from "../request-log"; +import { rememberComboForLane } from "./combo-session-recall"; +import { runTurnAdapterSseResponses } from "./core-lifetime"; +import { + isNativePassthroughSseResponse, + isEagerRelaySseResponse, + markNativePassthroughSseResponse, + markEagerRelaySseResponse, +} from "../relay"; +import { preflightComboStreamResponse } from "./combo-stream-preflight"; +import { streamingContextOverflowResponse, jsonContextOverflowResponse } from "./context-overflow"; + +/** + * Sends one combo target may run on its own before the ladder moves on. A target is a whole + * request as far as its own provider is concerned, so this is the guarded profile's base + * allowance rather than a separate number to keep in sync. + */ +export const COMBO_TARGET_BASE_SENDS = CODEX_TEXT_GUARDED_BUDGET_POLICY.baseSendAllowance; + + +/** + * A combo's execution policy is DECLARED by the combo, not inherited from the single-target + * profile. + * + * `maxTargetTransitions: 1` and `maxAlternateTargetSends: 1` describe an account move, and + * applying them to a combo would refuse the second hop of a three-target combo -- which is why + * combo was left off `reserveDispatch` when the per-request split landed. The transitions a + * combo may make are exactly the targets it declares minus the one it starts on. What stays + * capped is the TOTAL: the first target's full ladder, one send for every further declared + * target, and the one shared final-recovery reserve. A one-target combo reduces to the guarded + * profile exactly, and a three-target combo whose every target fails hard reaches upstream six + * times instead of the twelve #4546 measured. + */ +export function comboExecutionBudgetPolicy(declaredTargets: number): RequestExecutionBudgetPolicy { + const targets = Math.max(1, Math.trunc(declaredTargets)); + const hops = targets - 1; + const reserve = CODEX_TEXT_GUARDED_BUDGET_POLICY.finalRecoveryAllowance; + const total = COMBO_TARGET_BASE_SENDS + hops + reserve; + return { + maxTotalModelSends: total, + baseSendAllowance: total - reserve, + finalRecoveryAllowance: reserve, + maxAlternateTargetSends: Math.max(1, hops), + maxTargetTransitions: Math.max(1, hops), + }; +} + + +/** + * A budget scope that keeps its own recovery ledgers but spends the SAME request-wide counter. + * + * `used` is redefined as an accessor onto the parent because the factory reads it back off this + * object -- `remainingBaseSends` and the total check both do -- so a copied number would let a + * combo target run its ladder against a stale total, which is precisely the per-layer counting + * this work exists to remove. The reserve, alternate-target and transition ledgers stay + * per-scope on purpose: a combo target's account failover is its own recovery decision, while + * the request total still bounds every target together. + */ +export function deriveSendBudgetScope( + parent: RequestExecutionBudget, + policy: RequestExecutionBudgetPolicy, +): RequestExecutionBudget { + const scope = createRequestExecutionBudget(policy, parent.logicalRequestId); + Object.defineProperty(scope, "used", { + get: () => parent.used, + set: (value: number) => { parent.used = value; }, + enumerable: true, + configurable: true, + }); + return scope; +} + + +/** + * The ladder one combo target may run, expressed as an allowance on the request-wide counter. + * + * `used + COMBO_TARGET_BASE_SENDS` gives this target its own ladder from wherever the request + * already stands, and the clamp holds back one send for each target still declared after it: a + * first target that 5xx-streaks must not eat the send the last declared target is entitled to. + * That guarantee is the difference between a per-target policy and a shared pool the first + * target drains. + */ +export function comboTargetSendBudget( + comboScope: RequestExecutionBudget, + targetsDeclaredAfterThisOne: number, +): RequestExecutionBudget { + const policy = comboScope.policy; + const heldForLaterTargets = Math.max(0, targetsDeclaredAfterThisOne); + const ceiling = Math.max(1, policy.maxTotalModelSends - heldForLaterTargets); + return deriveSendBudgetScope(comboScope, { + maxTotalModelSends: policy.maxTotalModelSends, + baseSendAllowance: Math.min(ceiling, comboScope.used + COMBO_TARGET_BASE_SENDS), + finalRecoveryAllowance: policy.finalRecoveryAllowance, + // Within one target the account-move shape is unchanged: three same-account sends plus one + // alternate is the recovery live traffic depends on, and a combo does not widen it. + maxAlternateTargetSends: CODEX_TEXT_GUARDED_BUDGET_POLICY.maxAlternateTargetSends, + maxTargetTransitions: CODEX_TEXT_GUARDED_BUDGET_POLICY.maxTargetTransitions, + }); +} + + +export async function executeComboResponses( + req: Request, + rawBody: unknown, + comboId: string, + config: OcxConfig, + logCtx: RequestLogContext, + options: HandleResponsesOptions & { translatorBudget: TranslatorBudget }, + requestDispatchers: ResponsesDispatchers, +): Promise { + const requestedModel = typeof (rawBody as { model?: unknown } | null)?.model === "string" + ? (rawBody as { model: string }).model + : `combo/${comboId}`; + Object.assign(logCtx, { + requestedModel, + model: requestedModel, + provider: "combo", + comboId, + }); + const combo = getCombo(config, comboId); + if (!combo) { + return formatErrorResponse(404, "invalid_request_error", `Unknown combo: ${comboId}`); + } + // The ladder's own scope, derived from what this combo DECLARES. It shares the request-wide + // counter with the holder that arrived on options -- a combo child already inherited that + // counter, but nothing read it as a limit across targets -- while its transition and + // alternate-target ledgers come from the target list rather than from the single-target + // account-move profile (#4546). + const comboSendScope = isRequestExecutionBudget(options.sendBudget) + ? deriveSendBudgetScope(options.sendBudget, comboExecutionBudgetPolicy(combo.targets.length)) + : undefined; + // Expand previous_response_id before image policy and child dispatch so a + // continuation that only references prior images still fails closed when + // imageInput is disabled (and so targets see the full replayed input). + const inboundClientThreadId = req.headers.get("x-codex-parent-thread-id")?.trim() || undefined; + const body = expandPreviousResponseInput(rawBody, inboundClientThreadId); + const scopeMismatch = previousResponseScopeMismatch(body); + if (scopeMismatch) { + console.warn("[opencodex] dropped a previous_response_id with a mismatched client task scope; continuing fresh"); + } + if (previousResponseReplayFailure(body)) { + return formatErrorResponse( + 400, + "previous_response_not_found", + "Continuation state is unavailable or corrupt; resend the full conversation without previous_response_id.", + ); + } + // Missing state returns the original body without a failure marker. Reject + // that unresolved continuation for image-disabled combos so a target cannot + // resolve prior images out of band. A successful expansion yields a new + // object (still carrying previous_response_id) and must not be treated as + // unresolved — text-only stored continuations remain allowed. + const requestedPreviousId = typeof (rawBody as { previous_response_id?: unknown } | null)?.previous_response_id === "string" + ? (rawBody as { previous_response_id: string }).previous_response_id.trim() + : ""; + const unresolvedPrevious = requestedPreviousId.length > 0 && body === rawBody; + if (combo.imageInput === "disabled" && unresolvedPrevious) { + return formatErrorResponse( + 400, + "previous_response_not_found", + "Continuation state is unavailable or corrupt; resend the full conversation without previous_response_id.", + ); + } + if (combo.imageInput === "disabled" && comboRequestHasImageInput(body)) { + return formatErrorResponse(400, "invalid_request_error", `Combo "${comboId}" does not accept image input`); + } + const comboReplaySnapshot = { + sourceBody: body, + previousResponseInputExpanded: body !== rawBody + && typeof (body as { previous_response_id?: unknown }).previous_response_id === "string", + providerContinuation: !scopeMismatch && body !== rawBody && requestedPreviousId + ? previousResponseProviderState(requestedPreviousId) + : undefined, + recoveredPlaintext: false, + }; + const adoptFailedChildLog = (childLog: RequestLogContext): void => { + // Attempts remain the complete physical history; the logical row mirrors the most recent + // failed target so an exhausted combo still has useful top-level reasoning diagnostics. + Object.assign(logCtx, childLog, { + requestedModel, + model: requestedModel, + provider: "combo", + comboId, + routeDecision: logCtx.routeDecision, + attempts: logCtx.attempts, + activeAttempt: undefined, + activeAttemptStartedAt: undefined, + }); + }; + + const unreadableEncryptedAgentTask = hasUnreadableEncryptedAgentTask( + (body as { input?: unknown } | undefined)?.input, + ); + const canDecryptUnreadableAgentTask = (target: (typeof combo.targets)[number]): boolean => { + const provider = config.providers[target.provider]; + if (!provider || provider.disabled === true) return false; + try { + const route = routeConcreteModel(config, `${target.provider}/${target.model}`); + return isCanonicalOpenAiForwardProvider(route.provider); + } catch { + return false; + } + }; + let comboPayloadReadable = false; + const payloadEligible = (target: (typeof combo.targets)[number]): boolean => + comboPayloadReadable || !unreadableEncryptedAgentTask || canDecryptUnreadableAgentTask(target); + let encryptedTaskRecoveryAttempted = false; + let recoveryFailureReason: AgentTaskRecoveryFailureReason | undefined; + let storedPool401ReplayDispatched = false; + const recoverUnreadableEncryptedTask = async (): Promise => { + if (encryptedTaskRecoveryAttempted) return false; + encryptedTaskRecoveryAttempted = true; + const recovery = agentTaskRecoveryConfig(config); + if ( + (options.inboundWire ?? "responses") !== "responses" + || !isThreadSpawnRequest(req.headers) + || !recovery + || options.comboAttempt + ) { + discardEncryptedAgentTaskRecovery( + req, + (body as { input?: unknown } | undefined)?.input, + config, + { parentThreadId: inboundClientThreadId }, + ); + return false; + } + let recovered = false; + try { + const result = await recoverEncryptedAgentTaskWithResult( + req, + (body as { input?: unknown } | undefined)?.input, + recovery, + config, + { parentThreadId: inboundClientThreadId, abortSignal: options.abortSignal }, + ); + recovered = result.recovered; + recoveryFailureReason = result.recovered ? undefined : result.reason; + } catch { + recovered = false; + recoveryFailureReason = undefined; + } + // Recovery has the same in-place input mutation contract as the direct routed path. + if ( + !recovered + || hasUnreadableEncryptedAgentTask((body as { input?: unknown } | undefined)?.input) + ) { + discardEncryptedAgentTaskRecovery( + req, + (body as { input?: unknown } | undefined)?.input, + config, + { parentThreadId: inboundClientThreadId }, + ); + return false; + } + comboPayloadReadable = true; + comboReplaySnapshot.recoveredPlaintext = true; + return true; + }; + const initialNow = Date.now(); + const pickWithWait = (pickOptions: { + exclude?: Iterable; + eligible?: (target: NonNullable["targets"][number]) => boolean; + now?: number; + }) => pickComboTargetWithWait(config, comboId, { + ...pickOptions, + waitForCooldownMs: combo.waitForCooldownMs, + abortSignal: options.abortSignal, + }); + let pick = await pickWithWait({ + eligible: payloadEligible, + now: initialNow, + }); + + if (unreadableEncryptedAgentTask && !pick) { + pick = await pickWithWait({ now: initialNow }); + if (!pick) { + discardEncryptedAgentTaskRecovery( + req, + (body as { input?: unknown } | undefined)?.input, + config, + { parentThreadId: inboundClientThreadId }, + ); + return options.abortSignal?.aborted + ? clientCancelledResponse() + : comboUnavailable(comboId); + } + if (!(await recoverUnreadableEncryptedTask())) { + return options.abortSignal?.aborted + ? clientCancelledResponse() + : unreadableEncryptedAgentTaskResponse(recoveryFailureReason); + } + } + + if (!pick) { + return options.abortSignal?.aborted + ? clientCancelledResponse() + : comboUnavailable(comboId); + } + // One immutable combo selection trace, before any child dispatch; child + // adoption below must never replace it with a concrete child route trace. + logCtx.routeDecision = comboRouteDecisionTrace(config, comboId, pick, requestedModel); + + let lastFailure: Response | null = null; + // Dispatched targets, not attempted picks: it indexes the declared target list so the clamp + // below can tell how many targets are still entitled to a send. + let comboTargetsDispatched = 0; + // The child log behind `lastFailure`. The natural end of the ladder adopts it inside the + // no-more-targets branch; a budget refusal ends the ladder one iteration later, where that + // iteration's own `childLog` is already out of scope. + let lastFailedChildLog: RequestLogContext | undefined; + // The exhausted-combo mapping below runs outside the loop, where `failure.upstreamCode` + // is gone, so carry the loop's own classification decision instead of re-deriving a + // weaker one from the status alone (#4149). + let lastFailureClassifiesOverflow = false; + while (pick) { + if (options.abortSignal?.aborted) return clientCancelledResponse(); + const firstComboTarget = comboTargetsDispatched === 0; + // The first target seeds the ledger's target identity and charges nothing; every later one + // is a real transition, refused once the declared hops, the alternate-target ledger or the + // request total are spent. `countedExternally` is required: the child charges its own + // physical sends, and charging here as well would halve the cap without saying so. + const hopDecision = comboSendScope?.reserveDispatch({ + sendClass: firstComboTarget ? "initial" : "combo-failover", + targetKey: `${pick.target.provider}/${pick.target.model}`, + countedExternally: true, + }); + if (hopDecision && hopDecision.allowed) hopDecision.permit.use(); + else if (hopDecision && !firstComboTarget) { + // Out of budget is not this target's failure. The established exhaustion contract is to + // return the last real upstream answer with its status, headers and any quota body + // intact rather than to mint a synthetic error, and a later target only exists because + // an earlier one already recorded one. + if (lastFailedChildLog) adoptFailedChildLog(lastFailedChildLog); + break; + } + const targetSendBudget = comboSendScope + ? comboTargetSendBudget(comboSendScope, combo.targets.length - 1 - comboTargetsDispatched) + : options.sendBudget; + comboTargetsDispatched += 1; + const childLog: RequestLogContext = { + model: pick.target.model, + provider: pick.target.provider, + ...(logCtx.conversationId ? { conversationId: logCtx.conversationId } : {}), + ...(logCtx.surface ? { surface: logCtx.surface } : {}), + }; + const targetRoute = routeConcreteModel(config, `${pick.target.provider}/${pick.target.model}`); + const childBody = concreteComboRequestBody( + body, + pick.target, + comboDefaultEffort(config, comboId), + supportedLadderFor({ provider: targetRoute.provider, modelId: targetRoute.modelId }), + combo.reasoningEffortMode, + ); + const childHeaders = buildComboChildHeaders(req.headers); + const childRequest = new Request(req.url, { + method: req.method, + headers: childHeaders, + body: JSON.stringify(childBody), + }); + linkRequestSessionLane(req, childRequest); + let resolvedAuth: CodexAuthContext | undefined; + let terminalRecorder: ((status: ResponsesTerminalStatus, httpStatusOverride?: number) => void) | undefined; + const started = Date.now(); + const attempt = beginRequestAttempt( + (logCtx.attempts?.length ?? 0) + 1, + pick.target.provider, + pick.target.model, + config.providers[pick.target.provider]!.adapter, + ); + childLog.activeAttempt = attempt; + let attemptRetained = false; + const retainCancelledAttempt = (): void => { + if (attemptRetained) return; + sealRequestAttemptIdentity( + attempt, + childLog.provider, + childLog.providerAdapter ?? attempt.adapter, + childLog.accountLogLabel, + ); + finishRequestAttempt(attempt, 499, Date.now() - started, childLog.usage); + (logCtx.attempts ??= []).push(attempt); + attemptRetained = true; + }; + const completedTarget = { provider: pick.target.provider, model: pick.target.model }; + const writerGeneration = pick.writerGeneration; + let consumedChildFailure: ConsumedComboFailure | undefined; + const callbackGate = createChildPassthroughCallbackGate({ + ...options, + onResponseComplete: model => { + // The live config can change while the child is streaming. Never retain credentials. + const currentCombo = getCombo(config, comboId); + const provider = config.providers[completedTarget.provider]; + if (Object.hasOwn(config.providers, completedTarget.provider) + && provider && provider.disabled !== true + && currentCombo?.targets.some(target => targetKey(target) === targetKey(completedTarget))) { + rememberComboForLane(sessionLaneIdFromRequest(req.headers), comboId, completedTarget, model, writerGeneration); + } + options.onResponseComplete?.(model); + }, + onNativePassthroughTerminal: status => { + // A committed stream can acquire terminal metadata after preflight copied + // the child log. Publish it before the outer logger finalizes, but only + // through the gate: discarded attempts must never affect the parent. + // Undefined child fields must preserve metadata already inspected by WS. + if (childLog.terminalHttpStatus !== undefined) logCtx.terminalHttpStatus = childLog.terminalHttpStatus; + if (childLog.terminalIncompleteReason !== undefined) logCtx.terminalIncompleteReason = childLog.terminalIncompleteReason; + if (childLog.terminalErrorCode !== undefined) logCtx.terminalErrorCode = childLog.terminalErrorCode; + if (childLog.upstreamError !== undefined) logCtx.upstreamError = childLog.upstreamError; + options.onNativePassthroughTerminal?.(status); + }, + }); + let response: Response; + try { + const currentTargetProvider = pick.target.provider; + const deferCodexResetDerivedCooldown = combo.strategy === "failover" + && combo.targets.slice(pick.targetIndex + 1).some(target => + target.provider === currentTargetProvider + && payloadEligible(target) + && !isComboTargetInCooldown(comboId, target), + ); + response = await requestDispatchers.handleResponses(childRequest, config, childLog, { + ...options, + // After the spread: the child must run on THIS target's ladder, not on the holder the + // parent arrived with. + sendBudget: targetSendBudget, + comboAttempt: true, + comboReplaySnapshot, + deferCodexResetDerivedCooldown, + // Attempt-relative TTFT is recorded HERE (not via childLog.firstOutputMs — a later + // Object.assign(logCtx, childLog) would overwrite the request-relative value). + onFirstOutput: () => { + if (attempt.firstOutputMs === undefined) { + attempt.firstOutputMs = Math.max(0, Date.now() - started); + } + options.onFirstOutput?.(); + }, + onCodexAuthContextResolved: value => { resolvedAuth = value; }, + setTerminalOutcomeRecorder: value => { terminalRecorder = value; }, + onConsumedComboFailure: value => { consumedChildFailure = value; }, + onStoredPool401ReplayDispatched: () => { storedPool401ReplayDispatched = true; }, + onNativePassthroughTerminal: callbackGate.onTerminal, + onNativePassthroughCancel: callbackGate.onCancel, + onResponseComplete: callbackGate.onResponseComplete, + }); + } catch (error) { + callbackGate.discard(); + if (options.abortSignal?.aborted) { + retainCancelledAttempt(); + return clientCancelledResponse(); + } + throw error; + } + + if (options.abortSignal?.aborted) { + callbackGate.discard(); + retainCancelledAttempt(); + return clientCancelledResponse(); + } + + if (response.ok && !runTurnAdapterSseResponses.has(response)) { + const nativePassthrough = isNativePassthroughSseResponse(response); + const eagerRelay = isEagerRelaySseResponse(response); + let preflight; + try { + preflight = await preflightComboStreamResponse(response, childLog); + } catch (error) { + callbackGate.discard(); + if (options.abortSignal?.aborted) { + retainCancelledAttempt(); + return clientCancelledResponse(); + } + throw error; + } + if (preflight.kind === "failed") { + callbackGate.discard(); + terminalRecorder?.("failed", preflight.response.status); + response = preflight.response; + } else { + response = preflight.response; + if (nativePassthrough) markNativePassthroughSseResponse(response); + if (eagerRelay) markEagerRelaySseResponse(response); + } + } + + if (response.ok) { + sealRequestAttemptIdentity( + attempt, + childLog.provider, + childLog.providerAdapter ?? attempt.adapter, + childLog.accountLogLabel, + ); + (logCtx.attempts ??= []).push(attempt); + attemptRetained = true; + noteComboSuccess(comboId, combo, pick.target, pick.writerGeneration); + Object.assign(logCtx, childLog, { + requestedModel, + model: requestedModel, + provider: "combo", + comboId, + routeDecision: logCtx.routeDecision, + attempts: logCtx.attempts, + activeAttempt: attempt, + activeAttemptStartedAt: started, + resolvedModel: childLog.resolvedModel ?? childLog.model, + }); + options.onCodexAuthContextResolved?.(resolvedAuth); + options.setTerminalOutcomeRecorder?.(terminalRecorder); + callbackGate.commit(); + return response; + } + + callbackGate.discard(); + if (response.status === 499) { + retainCancelledAttempt(); + return clientCancelledResponse(); + } + let failure: ConsumedComboFailure; + try { + failure = consumedChildFailure + ?? await consumeComboFailure(response, options.abortSignal); + } catch (error) { + if (options.abortSignal?.aborted) { + retainCancelledAttempt(); + return clientCancelledResponse(); + } + throw error; + } + if (options.abortSignal?.aborted) { + retainCancelledAttempt(); + return clientCancelledResponse(); + } + sealRequestAttemptIdentity( + attempt, + childLog.provider, + childLog.providerAdapter ?? attempt.adapter, + childLog.accountLogLabel, + ); + finishRequestAttempt( + attempt, + failure.response.status, + Date.now() - started, + failure.usage, + ); + (logCtx.attempts ??= []).push(attempt); + attemptRetained = true; + lastFailure = failure.response; + lastFailedChildLog = childLog; + const failureDecision = comboFailureDecision(failure.response.status, failure.classificationText, { + code: failure.upstreamCode, + }); + const wantsStream = (rawBody as { stream?: unknown } | null)?.stream === true; + // Local byte admission has its own diagnostic; do not relabel it as an upstream refusal. + const classifyOverflow = failure.response.status === 413 + && (wantsStream || (failure.upstreamCode !== "outbound_body_too_large" + && failure.upstreamCode !== "translation_buffer_limit")); + lastFailureClassifiesOverflow = classifyOverflow; + if (storedPool401ReplayDispatched) { + if (failureDecision === "hop" && unreadableEncryptedAgentTask && !comboPayloadReadable) { + const recoveredTarget = await pickWithWait({ + exclude: pick.attempted, + eligible: target => { + try { + const route = routeConcreteModel(config, `${target.provider}/${target.model}`); + return route.codexAccountMode === undefined + && !isCanonicalOpenAiForwardProvider(route.provider); + } catch { + return false; + } + }, + }); + if (options.abortSignal?.aborted) return clientCancelledResponse(); + if (recoveredTarget && await recoverUnreadableEncryptedTask()) { + pick = recoveredTarget; + continue; + } + if (options.abortSignal?.aborted) return clientCancelledResponse(); + } + // Keep the spent Pool budget sticky even after a recovered routed child: + // no later failure may reopen ordinary combo/native account hopping. + adoptFailedChildLog(childLog); + if (classifyOverflow && failureDecision === "stop") { + return wantsStream + ? streamingContextOverflowResponse(requestedModel, options.translatorBudget) + : jsonContextOverflowResponse(); + } + return lastFailure; + } + if (failureDecision === "stop") { + adoptFailedChildLog(childLog); + if (classifyOverflow) { + return wantsStream + ? streamingContextOverflowResponse(requestedModel, options.translatorBudget) + : jsonContextOverflowResponse(); + } + return lastFailure; + } + console.warn( + `[combo] ${comboId}: ${targetKey(pick.target)} failed with ${failure.response.status} after ${Date.now() - started}ms`, + ); + const failureNow = Date.now(); + const attemptedTargets = pick.attempted; + const nextPick = advanceComboAfterFailure(config, pick, { + retryAfter: failure.retryAfter, + resetAt: failure.resetAt, + cooldownMs: combo.cooldownMs, + now: failureNow, + cooldownScope: comboFailureCooldownScope(failure.response.status, failure.classificationText, { + code: failure.upstreamCode, + }), + eligible: payloadEligible, + status: failure.response.status, + code: failure.upstreamCode, + message: failure.classificationText, + }); + if (nextPick) { + pick = nextPick; + } else { + pick = await pickWithWait({ + exclude: pick.attempted, + eligible: payloadEligible, + now: failureNow, + }); + } + if (!pick) { + if (options.abortSignal?.aborted) return clientCancelledResponse(); + if (unreadableEncryptedAgentTask && !comboPayloadReadable) { + const recoveredTarget = await pickWithWait({ + exclude: attemptedTargets, + now: failureNow, + }); + if (recoveredTarget && await recoverUnreadableEncryptedTask()) { + pick = recoveredTarget; + continue; + } + } + // Waiting or recovery may have observed cancellation after the check above. + if (options.abortSignal?.aborted) return clientCancelledResponse(); + adoptFailedChildLog(childLog); + } + } + if ( + lastFailure?.status === 413 + && lastFailureClassifiesOverflow + ) { + return (rawBody as { stream?: unknown } | null)?.stream === true + ? streamingContextOverflowResponse(requestedModel, options.translatorBudget) + : jsonContextOverflowResponse(); + } + return lastFailure!; +} diff --git a/src/server/responses/core-errors.ts b/src/server/responses/core-errors.ts new file mode 100644 index 0000000000..5e90d7fa02 --- /dev/null +++ b/src/server/responses/core-errors.ts @@ -0,0 +1,152 @@ +import { readBoundedResponseBody } from "../../lib/bounded-body"; +import { redactSecretString } from "../../lib/redact"; +import { isCyberPolicyMessage, isCyberPolicyCode } from "../../lib/errors"; +import { isTranslatorBudgetExceededError } from "../../lib/translator-budget"; +import { formatErrorResponse } from "../../bridge"; +import { + UnsupportedContentEncodingError, + DecompressedBodyTooLargeError, + describeInboundBodyRefusal, +} from "../request-decompress"; +import { comboCooldownRetryAfterSeconds } from "../../combos"; +import type { AgentTaskRecoveryFailureReason } from "./agent-task-recovery"; + +/** + * Materialize an upstream error body only when the bounded reader observed a complete, + * display-safe payload. Partial timeout and over-limit prefixes are attacker-controlled, + * so callers keep their existing status-only fallback instead. + */ +export async function readDisplaySafeErrorText( + response: Response, + signal: AbortSignal, + fallback: string, +): Promise { + try { + const body = await readBoundedResponseBody(response, { signal }); + return body.displaySafe ? body.text : fallback; + } catch { + // Preserve the former Response.text().catch(fallback) contract. Request-abort + // classification remains owned by the surrounding response pipeline. + return fallback; + } +} + + +export interface NormalizedUpstreamErrorText { + safeText: string; + message?: string; + type?: string; + code?: string; + cyberPolicy: boolean; +} + + +/** + * Extract the structured provider error envelope without making `error.type` authoritative. + * Policy identity comes from the dedicated code (or the legacy message fallback); a credible + * upstream type is only carried through so callers do not erase provider diagnostics. + */ +export function normalizeUpstreamErrorText(text: string, fallback: string): NormalizedUpstreamErrorText { + const safeText = redactSecretString(text).slice(0, 500).trim() || fallback; + let message: string | undefined; + let type: string | undefined; + let code: string | undefined; + try { + const parsed = JSON.parse(text) as Record; + const response = parsed.response && typeof parsed.response === "object" && !Array.isArray(parsed.response) + ? parsed.response as Record + : undefined; + const candidates = [parsed.error, response?.error, response?.last_error, parsed.last_error, parsed]; + const source = candidates.find((candidate): candidate is Record => { + if (candidate === null || typeof candidate !== "object" || Array.isArray(candidate)) return false; + const record = candidate as Record; + return [record.message, record.type, record.code].some(value => typeof value === "string"); + }); + if (!source) return { safeText, cyberPolicy: isCyberPolicyMessage(safeText) }; + if (typeof source.message === "string" && source.message.trim()) { + message = redactSecretString(source.message.trim()).slice(0, 500); + } + if (typeof source.type === "string" && source.type.trim()) type = source.type.trim(); + if (typeof source.code === "string" && source.code.trim()) code = source.code.trim(); + } catch { + /* non-JSON upstream body — retain the bounded display-safe text */ + } + const cyberPolicy = isCyberPolicyCode(code) || isCyberPolicyMessage(message ?? safeText); + return { safeText, message, type, code, cyberPolicy }; +} + + + + +export function decodeRequestErrorResponse(err: unknown, label: string): Response { + if (isTranslatorBudgetExceededError(err)) { + return formatErrorResponse(413, "request_too_large", "request translation buffer exceeded the safe limit", { + code: "translation_buffer_limit", + }); + } + if (err instanceof UnsupportedContentEncodingError) { + return formatErrorResponse(415, "invalid_request_error", err.message); + } + if (err instanceof DecompressedBodyTooLargeError) { + return formatErrorResponse(413, "inbound_body_too_large", describeInboundBodyRefusal(err)); + } + console.warn(`[${label}] request body decode/parse failed: ${err instanceof Error ? `${err.name}: ${err.message}` : String(err)}`); + return formatErrorResponse(400, "invalid_request_error", "Invalid JSON body"); +} + + + + +export function comboUnavailableResponse( + message: string, + options?: { retryAfter?: string | null }, +): Response { + const headers = new Headers({ "Content-Type": "application/json" }); + const retryAfter = options?.retryAfter?.trim(); + if (retryAfter && retryAfter.length > 0 && retryAfter.length <= 128) { + headers.set("Retry-After", retryAfter); + } + return new Response( + JSON.stringify({ + error: { message, type: "server_error", code: "combo_unavailable" }, + }), + { status: 503, headers }, + ); +} + + +export function comboUnavailable(comboId: string, now = Date.now()): Response { + return comboUnavailableResponse(`No available targets for combo: ${comboId}`, { + retryAfter: comboCooldownRetryAfterSeconds(comboId, now), + }); +} + + + + +/** + * Build the 499 JSON error the proxy returns when the client disconnects before the + * response completes (`client_cancelled`). + */ +export function clientCancelledResponse(): Response { + return formatErrorResponse(499, "client_cancelled", "Client cancelled request"); +} + + +export const UNREADABLE_ENCRYPTED_AGENT_TASK_MESSAGE = + "Routed V2 worker task is encrypted for the native ChatGPT backend and cannot be read by the selected provider. Use plaintext V2 agent-message delivery or select a native ChatGPT model."; + + +export function unreadableEncryptedAgentTaskResponse(reason?: AgentTaskRecoveryFailureReason): Response { + return new Response( + JSON.stringify({ + error: { + message: UNREADABLE_ENCRYPTED_AGENT_TASK_MESSAGE, + type: "invalid_request_error", + code: "unreadable_encrypted_agent_task", + ...(reason === undefined ? {} : { recovery_reason: reason }), + }, + }), + { status: 400, headers: { "Content-Type": "application/json" } }, + ); +} diff --git a/src/server/responses/core-lifetime.ts b/src/server/responses/core-lifetime.ts new file mode 100644 index 0000000000..3bc4f43cbe --- /dev/null +++ b/src/server/responses/core-lifetime.ts @@ -0,0 +1,95 @@ +import type { TranslatorBudget } from "../../lib/translator-budget"; +import { + isNativePassthroughSseResponse, + markNativePassthroughSseResponse, + isEagerRelaySseResponse, + markEagerRelaySseResponse, +} from "../relay"; + +// runTurn adapters own an event queue and perform their combo preflight before +// bridging. A second byte-stream reader would reinterpret that transport's +// already-committed event boundary and can replay custom adapter work. +export const runTurnAdapterSseResponses = new WeakSet(); + + +// Whole-body policy for non-streaming upstream JSON responses (see the application/json +// branch of the passthrough return path). 32 MiB matches the continuation snapshot read +// bound and is far above any legitimate non-streaming completion, including base64 image +// payloads. The stall deadlines only govern the body transfer — generation time before +// the response headers is untouched. Generation after early/chunked headers but before +// the first body byte previously used the 30-second inactivity deadline; this call site +// gives it the full body deadline instead. +export const MAX_UPSTREAM_JSON_BODY_BYTES = 32 * 1024 * 1024; + +export const UPSTREAM_JSON_BODY_TOTAL_TIMEOUT_MS = 180_000; + +export const UPSTREAM_JSON_BODY_INACTIVITY_TIMEOUT_MS = 30_000; + +export const UPSTREAM_JSON_BODY_READ_OPTIONS = { + maxBytes: MAX_UPSTREAM_JSON_BODY_BYTES, + totalTimeoutMs: UPSTREAM_JSON_BODY_TOTAL_TIMEOUT_MS, + inactivityTimeoutMs: UPSTREAM_JSON_BODY_INACTIVITY_TIMEOUT_MS, + firstByteTimeoutMs: UPSTREAM_JSON_BODY_TOTAL_TIMEOUT_MS, +}; + + + + +export function finalizeOwnedTranslatorBudget(response: Response, budget: TranslatorBudget): Response { + if (!response.body) { + budget.dispose(); + return response; + } + const reader = response.body.getReader(); + let finalized = false; + const finalize = () => { + if (finalized) return; + finalized = true; + budget.dispose(); + }; + const body = new ReadableStream({ + async pull(controller) { + try { + const result = await reader.read(); + if (result.done) { + finalize(); + controller.close(); + } else { + controller.enqueue(result.value); + } + } catch (error) { + finalize(); + controller.error(error); + } + }, + async cancel(reason) { + try { await reader.cancel(reason); } finally { finalize(); } + }, + }); + const finalizedResponse = new Response(body, { + status: response.status, + statusText: response.statusText, + headers: response.headers, + }); + if (isNativePassthroughSseResponse(response)) { + markNativePassthroughSseResponse(finalizedResponse); + } + if (isEagerRelaySseResponse(response)) { + markEagerRelaySseResponse(finalizedResponse); + } + return finalizedResponse; +} + + + + +export function linkAbortSignal(upstream: AbortController, signal?: AbortSignal): () => void { + if (!signal) return () => {}; + if (signal.aborted) { + upstream.abort(signal.reason); + return () => {}; + } + const onAbort = () => upstream.abort(signal.reason); + signal.addEventListener("abort", onAbort, { once: true }); + return () => signal.removeEventListener("abort", onAbort); +} diff --git a/src/server/responses/core-normalize.ts b/src/server/responses/core-normalize.ts new file mode 100644 index 0000000000..777fd7364c --- /dev/null +++ b/src/server/responses/core-normalize.ts @@ -0,0 +1,350 @@ +import { sanitizeLogMetadataString } from "../../lib/redact"; +import type { RouteResult } from "../../router"; +import type { InboundWire } from "../../providers/registry"; +import { resolveWireProtocolOverride } from "../adapter-resolve"; +import type { OcxConfig, OcxParsedRequest, OcxProviderConfig, TierDecision } from "../../types"; +import { + resolveCodexModelEntitlements, + entitledCodexAccountIdsForModel, +} from "../../codex/model-entitlements"; +import type { SubagentModelEligibleAccountIds } from "../../codex/subagent-model-fallback"; +import { subagentFallbackNeedsModelEntitlements } from "../../codex/subagent-model-fallback"; +import { MAIN_CODEX_ACCOUNT_ID } from "../../codex/main-account"; +import type { RequestLogContext } from "../request-log"; +import type { HandleResponsesOptions } from "./core-options"; +import { prepareEffortNormalization } from "../effort-policy"; +import { providerModelResponsesUpstreamStreaming } from "../../providers/registry"; +import { resolveOpenCodeGoTransport } from "../../providers/opencode-go-transport"; +import { getOrAllocateRequestSessionLane } from "../request-log-conversation"; +import { shouldPreparePlaintextV2AgentMessages } from "../../responses/plaintext-v2-agent-messages"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import { applyOpenAiVirtualModel } from "../../providers/openai-virtual-models"; +import { + fastPolicyForModel, + serviceTierSupportFromPolicy, + SERVICE_TIER_ADAPTERS, +} from "../../providers/service-tier"; +import { + tierObservationContext, + decideTier, + tierValueAfterDecision, + canonicalFastTierMarker, +} from "../../providers/fastwire"; +import { multiAgentGuidanceText, injectDeveloperMessage, collabSurface } from "./collaboration"; +import { multiAgentGuidanceEnabled } from "../../config"; +import { isInjectionDebugEnabled } from "../../lib/debug-settings"; +import { injectionDebugLog } from "../../lib/injection-debug-log"; +import { recordAttemptRequestedEffort } from "../request-log"; +import type { ResolvedFastPolicy } from "../../providers/fastwire"; + +export const MAX_FAST_WIRE_CAPABILITY_WARNINGS = 256; + +export const warnedFastWireCapabilityGaps = new Set(); + + +export function warnFastWireCapabilityGap(providerName: string, modelId: string): void { + const safeProvider = sanitizeLogMetadataString(providerName) ?? "unknown"; + const safeModel = sanitizeLogMetadataString(modelId) ?? "unknown"; + const key = `${safeProvider}\0${safeModel}`; + if (warnedFastWireCapabilityGaps.has(key)) return; + if (warnedFastWireCapabilityGaps.size >= MAX_FAST_WIRE_CAPABILITY_WARNINGS) { + const oldest = warnedFastWireCapabilityGaps.values().next().value; + if (oldest !== undefined) warnedFastWireCapabilityGaps.delete(oldest); + } + warnedFastWireCapabilityGaps.add(key); + console.warn( + `[opencodex] Fast policy for ${safeProvider}/${safeModel} has service-tier capability but no Fast wire; preserving only caller-permitted tier behavior`, + ); +} + + +/** + * Keep this trust boundary deliberately narrow: only a key-auth Responses route may consume + * opaque child-task ciphertext, and the model's final wire override must still be Responses. + * Callers keep combo attempts on their existing native-only recovery/fail-closed behavior. + */ +export function canPassThroughEncryptedV2AgentTask( + route: RouteResult, + inboundWire: InboundWire, +): boolean { + if (route.combo !== undefined) return false; + const provider = route.provider; + if ( + inboundWire !== "responses" + || provider.allowEncryptedV2AgentTasks !== true + || (provider.authMode ?? "key") !== "key" + ) return false; + + return resolveWireProtocolOverride( + route.providerName, + route.modelId, + provider, + inboundWire, + ).adapter === "openai-responses"; +} + + +export async function resolveSubagentFallbackModelEligibility(args: { + config: OcxConfig; + fallbackChain: readonly string[] | null; + nativeMainReadsForbidden: boolean; + resolver: typeof resolveCodexModelEntitlements; +}): Promise { + if (!subagentFallbackNeedsModelEntitlements(args.fallbackChain, args.config)) return undefined; + const excludeAccountIds = args.nativeMainReadsForbidden + ? new Set([MAIN_CODEX_ACCOUNT_ID]) + : undefined; + const snapshot = await args.resolver(args.config, { excludeAccountIds }); + return (modelId) => { + const entitledAccountIds = entitledCodexAccountIdsForModel(snapshot, modelId); + return entitledAccountIds + ? new Set([...entitledAccountIds].filter(accountId => !excludeAccountIds?.has(accountId))) + : undefined; + }; +} + + +/** + * Apply every route-dependent request mutation against the final selected route. + * Must run only after subagent fallback has settled the model/provider. + */ +export async function applyFinalRouteRequestNormalization(args: { + parsed: OcxParsedRequest; + route: RouteResult; + config: OcxConfig; + req: Request; + logCtx: RequestLogContext; + inboundWire: InboundWire; + inboundTransport?: "websocket"; + claudeGoAffinity?: HandleResponsesOptions["claudeGoAffinity"]; +}): Promise { + const { parsed, route, config, req, logCtx, inboundWire, inboundTransport } = args; + const effortSelector = prepareEffortNormalization(parsed, route); + + // Only Anthropic message routes retain the Codex-facing selector. Other providers must keep + // their existing response.model contract even when their public and wire model ids differ. + const responseModelId = parsed.modelId; + const preserveAnthropicResponseModel = route.providerName === "anthropic" + || route.provider.adapter === "anthropic"; + + // Apply the routed model id upstream: routing may strip a "/" namespace. + if (route.modelId !== parsed.modelId) { + if (parsed._rawBody && typeof parsed._rawBody === "object") { + (parsed._rawBody as { model?: string }).model = route.modelId; + } + parsed.modelId = route.modelId; + } + // Transport-neutral reliability policy (#875): applies to any Responses + // upstream whose final adapter is openai-responses, not only WS turns. + const responsesUpstreamStreaming = providerModelResponsesUpstreamStreaming( + route.providerName, + route.provider, + route.modelId, + ); + + // Settle the wire once so logging, fast-mode, auth, and sidecars read the adapter + // this request will actually use (#404). + route.provider = resolveOpenCodeGoTransport(route.provider, + args.claudeGoAffinity ? args.claudeGoAffinity.sessionLane : getOrAllocateRequestSessionLane(req)); + route.provider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire); + parsed._plaintextV2AgentMessages = shouldPreparePlaintextV2AgentMessages({ + enabled: config.plaintextV2AgentMessages === true, + inboundWire, + canonicalChatGpt: isCanonicalOpenAiForwardProvider(route.provider), + requestBody: parsed._rawBody, + }); + // Recompute from the original wire preference on every route, including fallback. + // A provider default never converts raw reasoning into a summary. + if (inboundWire === "responses" && parsed._rawBody) { + const summary = (parsed._rawBody as { reasoning?: { summary?: unknown } }).reasoning?.summary; + parsed.options.hideThinkingSummary = summary === "none" + || (!summary && route.provider.showThinkingSummary !== true); + } + if (preserveAnthropicResponseModel) parsed._responseModelId = responseModelId; + logCtx.model = route.modelId; + logCtx.provider = route.providerName; + logCtx.providerAdapter = route.provider.adapter; + logCtx.routeDecision = route.routeDecision; + if (route.routeReason === "model-alias" || route.modelId !== responseModelId && responseModelId.includes("/")) logCtx.requestedAlias = responseModelId; + + if (responsesUpstreamStreaming === false && route.provider.adapter === "openai-responses") { + parsed.stream = false; + if (parsed._rawBody && typeof parsed._rawBody === "object") { + (parsed._rawBody as Record).stream = false; + } + } + + // Generic Responses clients (e.g. AI-SDK apps) omit `store`, but the canonical + // forward Codex backend rejects a native request without an explicit store:false. + // Default it only there — every other Responses upstream (key-auth providers and + // custom forward gateways) intentionally keeps the omitted-store server-side + // default for previous_response_id reuse — and never override an explicit value. + if ( + isCanonicalOpenAiForwardProvider(route.provider) + && parsed._rawBody && typeof parsed._rawBody === "object" + && (parsed._rawBody as Record).store === undefined + ) { + (parsed._rawBody as Record).store = false; + } + + // Final selected model before virtual wire-model rewriting (Pro aliases). + const finalSelectedModelId = route.modelId; + + // Virtual model rewriting: Pro aliases → base model + reasoning.mode="pro". + applyOpenAiVirtualModel(parsed, route, logCtx); + if (parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId) { + logCtx.resolvedModel = route.modelId; + logCtx.preserveResolvedModelFromRoute = true; + } + + // Resolve Fast policy after the final route/wire settles. A1 records the decision on parsed + // options; the Responses adapter owns the final outbound body write. + const fastPolicy = fastPolicyForModel( + route.provider, + route.modelId, + route.providerName, + inboundWire, + config.providers[route.providerName], + ); + const modelServiceTierSupport = serviceTierSupportFromPolicy(fastPolicy); + const callerTier = parsed.options.serviceTier; + // The ChatGPT-internal Codex backend echoes `service_tier: "default"` even on turns it + // scheduled as priority, so its echo cannot confirm OR deny Fast. Believing it reported every + // Fast request as `response-declined` (#2558). The public API's echo stays authoritative. + parsed.options.tierObservation = tierObservationContext( + fastPolicy, + config.fastMode, + callerTier, + isCanonicalOpenAiForwardProvider(route.provider) ? false : undefined, + ); + parsed.options.tierDecision = decideTier(fastPolicy, config.fastMode, callerTier); + parsed.options.serviceTier = tierValueAfterDecision(parsed.options.tierDecision, callerTier); + if (fastPolicy.capability === true && fastPolicy.fastWire === null) { + warnFastWireCapabilityGap(route.providerName, route.modelId); + } + applyServiceTierGate( + route.provider, + parsed._rawBody, + parsed.options, + route.modelId, + route.providerName, + inboundWire, + fastPolicy, + ); + if (modelServiceTierSupport === false) { + logCtx.requestedServiceTier = undefined; + logCtx.requestedSpeedLabel = undefined; + } + + { + const guidance = await multiAgentGuidanceText(parsed, { + multiAgentGuidanceEnabled: config.multiAgentGuidanceEnabled, + codexAccountNamespace: route.codexAccountNamespace, + injectionModel: config.injectionModel, + injectionEffort: config.injectionEffort, + subagentModels: config.subagentModels, + subagentModelFallback: config.subagentModelFallback, + injectionPrompt: config.injectionPrompt, + }); + if (guidance) { + injectDeveloperMessage(parsed, guidance); + if (isInjectionDebugEnabled()) { + injectionDebugLog(`[opencodex] ${route.modelId}: multi-agent guidance injected (surface=${collabSurface(parsed)}, guidanceEnabled=${multiAgentGuidanceEnabled(config)}, ${guidance.length} chars)`); + } + } else if (isInjectionDebugEnabled() && collabSurface(parsed) !== null) { + injectionDebugLog(`[opencodex] ${route.modelId}: collab surface=${collabSurface(parsed)}, guidance silent (effort=${parsed.options.reasoning ?? "unset"}, injectionModel=${config.injectionModel ?? "unset"})`); + } + } + + { + const { applyPinnedEffort } = await import("../effort-policy"); + const pinned = applyPinnedEffort(parsed, route, config, effortSelector); + if (pinned) { + logCtx.requestedEffort = pinned.from ? `${pinned.from}->${pinned.to}` : pinned.to; + if (isInjectionDebugEnabled()) { + injectionDebugLog(`[opencodex] ${route.modelId}: pinned reasoning effort applied (${pinned.from ?? "none"} -> ${pinned.to})`); + } + } + } + + { + const { applyEffortCap, effortCapAppliesTo, supportedLadderFor } = await import("../effort-policy"); + const surface = collabSurface(parsed); + if (effortCapAppliesTo(surface, req.headers, config, parsed._compactionRequest === true)) { + const capped = applyEffortCap(parsed, req.headers, config, supportedLadderFor(route)); + if (capped) { + logCtx.requestedEffort = `${capped.from}->${capped.to}`; + if (isInjectionDebugEnabled()) { + injectionDebugLog(`[opencodex] ${route.modelId}: effort cap applied (${capped.from} -> ${capped.to}, ${capped.subagent ? "sub-agent" : "main"} turn)`); + } + } + } else if (isInjectionDebugEnabled() && (config.effortCap || config.subagentEffortCap)) { + injectionDebugLog(`[opencodex] ${route.modelId}: effort cap skipped (surface=${surface ?? "none"}, v2 feature only)`); + } + } + + { + const { nativeEffortClamp, shouldApplyNativeEffortClamp } = await import("../../codex/catalog"); + const clamped = shouldApplyNativeEffortClamp(route.providerName, route.provider, finalSelectedModelId) + ? nativeEffortClamp(route.modelId, parsed.options.reasoning) + : null; + if (clamped) { + parsed.options.reasoning = clamped; + const raw = parsed._rawBody as { reasoning?: { effort?: string } } | undefined; + if (raw?.reasoning && typeof raw.reasoning === "object") raw.reasoning.effort = clamped; + logCtx.requestedEffort = `${logCtx.requestedEffort ?? "max"}->${clamped}`; + } + } + recordAttemptRequestedEffort(logCtx); + logCtx.modelSupportsServiceTier = SERVICE_TIER_ADAPTERS.has(route.provider.adapter) + ? modelServiceTierSupport + : undefined; +} + + +/** + * Service-tier capability gate, applied after the final route/wire is settled. A + * provider explicitly documented as NOT supporting `service_tier` must never + * receive it: strip the field and clear the logging value even when the caller + * supplied one (fail closed). A policy-produced canonical Fast decision has + * already passed capability validation and cannot be vetoed by Chat's caller + * forwarding permission. On unclassified routes every caller tier remains subject + * to `forwardCallerTier`. + */ +export function applyServiceTierGate( + provider: OcxProviderConfig, + rawBody: unknown, + options: { serviceTier?: string; tierDecision?: TierDecision }, + modelId?: string, + providerName?: string, + inbound: InboundWire = "responses", + resolvedPolicy?: ResolvedFastPolicy, +): void { + // A direct unit caller without a model id retains the historical tri-state behavior for + // adapters outside the OpenAI service-tier family. Once a model is known, resolve the final + // model adapter as well: an explicit override to Anthropic (or another non-OpenAI wire) must + // not carry a caller-supplied `service_tier` through a route that cannot forward it. + if (modelId === undefined && !SERVICE_TIER_ADAPTERS.has(provider.adapter)) return; + const policy = modelId === undefined + ? undefined + : resolvedPolicy ?? fastPolicyForModel(provider, modelId, providerName, inbound); + const forwardCallerTier = modelId === undefined + ? provider.supportsServiceTier !== false + : policy!.forwardCallerTier; + const rawTier = rawBody && typeof rawBody === "object" + ? (rawBody as Record).service_tier + : undefined; + const canonicalDecision = options.tierDecision?.kind === "set"; + const callerTierIsForeign = rawTier !== undefined + && (typeof rawTier !== "string" || canonicalFastTierMarker(rawTier) === undefined); + const dropForeignCallerTier = policy?.capability === true + && policy.fastWire?.kind === "service-tier" + && policy.fastWire?.foreignCallerTiers === "drop" + && callerTierIsForeign; + if (policy && policy.capability !== false && canonicalDecision) return; + if (forwardCallerTier && !dropForeignCallerTier) return; + if (rawBody && typeof rawBody === "object") { + delete (rawBody as Record).service_tier; + } + options.serviceTier = undefined; +} diff --git a/src/server/responses/core-opaque-recovery.ts b/src/server/responses/core-opaque-recovery.ts new file mode 100644 index 0000000000..bc7bbe40a1 --- /dev/null +++ b/src/server/responses/core-opaque-recovery.ts @@ -0,0 +1,380 @@ +import { ENCRYPTED_FUNCTION_OUTPUT_REJECTION, upstreamErrorMessageFromPayload } from "../../lib/errors"; +import { readBoundedResponseBody } from "../../lib/bounded-body"; +import { isReasoningEffortRejection } from "../../providers/reasoning-metadata"; +import { isNonReplayableResponse } from "../../lib/upstream-retry"; +import type { OcxParsedRequest } from "../../types"; +import type { RequestLogContext } from "../request-log"; +import type { AttemptRecoveryKind } from "../../usage/log"; +import { rememberReasoningReplayOpaqueBlobRejection } from "../../responses/reasoning-replay-cache"; + +export const OPAQUE_RESPONSES_INPUT_TYPES = new Set([ + "reasoning", + "compaction", + "compaction_summary", + "context_compaction", +]); + +export const FUNCTION_OUTPUT_TYPES = new Set(["function_call_output", "custom_tool_call_output"]); + +// codex-app subagent results replay as agent_message items whose content parts may carry +// backend-minted encrypted_content; the ChatGPT backend decrypts them in its function-output +// path, so a cross-identity replay of those parts produces ENCRYPTED_FUNCTION_OUTPUT_REJECTION. +export const AGENT_MESSAGE_TYPE = "agent_message"; + + +export function encryptedFunctionOutputParts(output: unknown): boolean { + return Array.isArray(output) && output.some(part => ( + part !== null + && typeof part === "object" + && !Array.isArray(part) + && (part as { type?: unknown }).type === "encrypted_content" + && typeof (part as { encrypted_content?: unknown }).encrypted_content === "string" + && (part as { encrypted_content: string }).encrypted_content.length > 0 + )); +} + + +export function outboundResponsesInput(bodyText: string | undefined): unknown[] | undefined { + if (!bodyText) return undefined; + try { + const body = JSON.parse(bodyText) as unknown; + if (!body || typeof body !== "object" || Array.isArray(body)) return undefined; + const input = (body as { input?: unknown }).input; + return Array.isArray(input) ? input : undefined; + } catch { + return undefined; + } +} + + +export function outboundResponsesBodyCarriesEncryptedFunctionOutput(bodyText: string | undefined): boolean { + const input = outboundResponsesInput(bodyText); + if (!input) return false; + return input.some(item => { + if (item === null || typeof item !== "object" || Array.isArray(item)) return false; + const candidate = item as { type?: unknown; output?: unknown; content?: unknown }; + const type = String(candidate.type ?? ""); + if (FUNCTION_OUTPUT_TYPES.has(type) && encryptedFunctionOutputParts(candidate.output)) return true; + return type === AGENT_MESSAGE_TYPE && encryptedFunctionOutputParts(candidate.content); + }); +} + + +export function outboundResponsesBodyCarriesOpaqueBlob(bodyText: string | undefined): boolean { + const input = outboundResponsesInput(bodyText); + if (!input) return false; + return input.some(item => { + if (!item || typeof item !== "object" || Array.isArray(item)) return false; + const candidate = item as { type?: unknown; encrypted_content?: unknown; output?: unknown }; + if ( + typeof candidate.type === "string" + && OPAQUE_RESPONSES_INPUT_TYPES.has(candidate.type) + && typeof candidate.encrypted_content === "string" + && candidate.encrypted_content.length > 0 + ) return true; + if ( + typeof candidate.type === "string" + && FUNCTION_OUTPUT_TYPES.has(candidate.type) + && encryptedFunctionOutputParts(candidate.output) + ) return true; + return candidate.type === AGENT_MESSAGE_TYPE + && encryptedFunctionOutputParts((candidate as { content?: unknown }).content); + }); +} + + +export function isEncryptedFunctionOutputRejection(bodyText: string): boolean { + if (bodyText.trim() === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) return true; + try { + const payload = JSON.parse(bodyText) as unknown; + if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; + const record = payload as { detail?: unknown; message?: unknown; error?: unknown }; + if (record.detail === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) return true; + if (record.message === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) return true; + if (record.error === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) return true; + return record.error !== null + && typeof record.error === "object" + && !Array.isArray(record.error) + && (record.error as { message?: unknown }).message === ENCRYPTED_FUNCTION_OUTPUT_REJECTION; + } catch { + return false; + } +} + + +/** + * #4469: reasoning encrypted_content is minted per caller identity, so replaying it under a + * different caller is rejected with "reasoning `encrypted_content` was not issued to this + * caller". Substring checks tolerate the optional backticks and a leading or trailing + * sentence, while the "was not issued to this caller" anchor plus an encrypted-content or + * reasoning subject keep unrelated invalid_request_error prose from gaining a hidden resend. + */ +export function isReasoningBlobCallerMismatchMessage(message: string): boolean { + if (!message.includes("was not issued to this caller")) return false; + return message.includes("encrypted_content") || message.includes("reasoning"); +} + + +export function isSelfIdentifiedOpaqueBlobRejection(bodyText: string): boolean { + if (isEncryptedFunctionOutputRejection(bodyText)) return true; + try { + if (upstreamErrorMessageFromPayload(JSON.parse(bodyText) as unknown) === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) { + return true; + } + } catch { + /* invalid JSON bodies fall through to the exact nested envelope checks */ + } + try { + const payload = JSON.parse(bodyText) as unknown; + if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; + const record = payload as { code?: unknown; type?: unknown; message?: unknown; error?: unknown }; + + if (record.error && typeof record.error === "object" && !Array.isArray(record.error)) { + const error = record.error as { type?: unknown; code?: unknown; message?: unknown }; + if (error.type === "invalid_request_error") { + if (error.code === "invalid_encrypted_content") return true; + if ( + (error.code === null || error.code === undefined) + && typeof error.message === "string" + && error.message.startsWith("The encrypted content ") + && error.message.endsWith( + " could not be verified. Reason: Encrypted content could not be decrypted or parsed.", + ) + ) return true; + // #4469: the caller-mismatch wording arrives without a dedicated code, so the + // message itself is the identity. It is not gated on code being null — the upstream + // may attach a generic code — because the anchored phrase is already specific. + if (typeof error.message === "string" && isReasoningBlobCallerMismatchMessage(error.message)) { + return true; + } + } + } + + // The flat stream-error envelope carries type/message at the top level rather than under + // an error object; the same anchored identity applies there. + if ( + record.type === "invalid_request_error" + && typeof record.message === "string" + && isReasoningBlobCallerMismatchMessage(record.message) + ) return true; + + if (record.code !== "invalid-argument" || typeof record.error !== "string") return false; + return record.error.startsWith("Could not decode the compaction blob") + || record.error.startsWith("Could not decrypt the provided encrypted_content"); + } catch { + return false; + } +} + + +/** + * Whether an upstream Responses 4xx authoritatively rejected opaque replay state. + * + * The outbound-body check is intentional: the inbound transcript may contain a proxy envelope or + * compaction blob that the adapter already lowered, in which case a replay would be byte-identical. + * OpenAI usually exposes a dedicated nested code; ChatGPT also emits one exact code-less + * unverifiable-ciphertext message, and #4469 added the anchored caller-mismatch wording for + * reasoning blobs minted under a different caller. xAI's code is generic, so its two concrete + * decoder error identities are also required. Unrelated error prose must never gain a hidden resend. + */ +export function shouldAttemptOpaqueBlobRecovery(args: { + status: number; + adapterName: string; + outboundBody?: string; + errorBody: string; + alreadyAttempted: boolean; +}): boolean { + const acceptedStatus = (args.status >= 400 && args.status < 500) + || ( + args.status === 502 + && outboundResponsesBodyCarriesEncryptedFunctionOutput(args.outboundBody) + && isEncryptedFunctionOutputRejection(args.errorBody) + ); + return acceptedStatus + && args.adapterName === "openai-responses" + && !args.alreadyAttempted + && outboundResponsesBodyCarriesOpaqueBlob(args.outboundBody) + && isSelfIdentifiedOpaqueBlobRejection(args.errorBody); +} + + +/** + * Peek the upstream error body for the reasoning-effort downgrade. Only 400/403 are considered + * and the body must be complete and display-safe, the same contract the other rejection peeks + * use. The match is deliberately narrow: the upstream has to name reasoning effort, so an + * unrelated 400 never triggers a replay. + */ +export async function reasoningEffortRejectionText( + response: Response, + alreadyAttempted: boolean, + signal: AbortSignal, +): Promise { + if (alreadyAttempted) return undefined; + if (response.status !== 400 && response.status !== 403) return undefined; + try { + const body = await readBoundedResponseBody(response.clone(), { signal }); + if (!body.displaySafe || body.truncated) return undefined; + return isReasoningEffortRejection(body.text) ? body.text : undefined; + } catch { + return undefined; + } +} + + +export async function opaqueBlobRejectionBodyForRecovery( + response: Response, + outboundBody: string | undefined, + adapterName: string, + alreadyAttempted: boolean, + signal: AbortSignal, +): Promise { + if ( + isNonReplayableResponse(response) + || response.status < 400 + || (response.status >= 500 && response.status !== 502) + || adapterName !== "openai-responses" + || alreadyAttempted + || !outboundResponsesBodyCarriesOpaqueBlob(outboundBody) + ) return undefined; + try { + const body = await readBoundedResponseBody(response.clone(), { signal }); + return body.displaySafe && !body.truncated ? body.text : undefined; + } catch { + return undefined; + } +} + + +/** + * Backoff for the single exact-request replay after a canonical Console upload rejection. + */ +export const CONSOLE_GO_UPLOAD_RETRY_DELAY_MS = 800; + + +/** + * Peek the upstream error body for the Console Go transient-400 recovery. Only a complete, + * display-safe body may drive a retry decision (same contract as + * opaqueBlobRejectionBodyForRecovery), and reading a clone leaves the original response intact + * for the caller's own error surface when no retry is taken. + */ +export async function consoleGoUploadRejectionBody( + response: Response, + alreadyAttempted: boolean, + signal: AbortSignal, +): Promise { + if (isNonReplayableResponse(response) || response.status !== 400 || alreadyAttempted) return undefined; + try { + const body = await readBoundedResponseBody(response.clone(), { signal }); + return body.displaySafe && !body.truncated ? body.text : undefined; + } catch { + return undefined; + } +} + + +export function prepareOpaqueBlobRecovery(parsed: OcxParsedRequest): void { + parsed._stripReasoningEncryptedContent = true; + const rawBody = parsed._rawBody; + if (!rawBody || typeof rawBody !== "object" || Array.isArray(rawBody)) return; + const input = (rawBody as { input?: unknown }).input; + if (!Array.isArray(input)) return; + const stripEncryptedParts = (parts: unknown[]): unknown[] => { + let changed = false; + const stripped = parts.map(part => { + if ( + part !== null + && typeof part === "object" + && !Array.isArray(part) + && (part as { type?: unknown }).type === "encrypted_content" + && typeof (part as { encrypted_content?: unknown }).encrypted_content === "string" + && (part as { encrypted_content: string }).encrypted_content.length > 0 + ) { + changed = true; + return { type: "input_text", text: "[encrypted content omitted]" }; + } + return part; + }); + return changed ? stripped : parts; + }; + const strippedInput = input.map(item => { + if (!item || typeof item !== "object" || Array.isArray(item)) return item; + const record = item as Record; + const type = String(record.type ?? ""); + if (FUNCTION_OUTPUT_TYPES.has(type) && Array.isArray(record.output)) { + const output = stripEncryptedParts(record.output); + return output !== record.output ? { ...record, output } : item; + } + if (type === AGENT_MESSAGE_TYPE && Array.isArray(record.content)) { + const content = stripEncryptedParts(record.content); + return content !== record.content ? { ...record, content } : item; + } + return item; + }); + Object.assign(rawBody, { input: strippedInput }); +} + + +export function resetStreamedOpaqueBlobLogContext(logCtx: RequestLogContext): void { + delete logCtx.upstreamError; + delete logCtx.terminalHttpStatus; + delete logCtx.terminalErrorCode; + delete logCtx.terminalIncompleteReason; +} + + +export type OpaqueBlobRecoveryGuard = { attempted: boolean }; + + +export type OpaqueBlobRecoveryResult = + | { kind: "skipped" } + | { kind: "recovered"; response: Response } + | { kind: "failed"; response: Response }; + + +export async function attemptOpaqueBlobRecovery( + args: { + response: Response; + outboundBody?: string; + adapterName: string; + parsed: OcxParsedRequest; + guard: OpaqueBlobRecoveryGuard; + signal: AbortSignal; + }, + rebuild: (kind: AttemptRecoveryKind) => Promise, +): Promise { + const errorBody = await opaqueBlobRejectionBodyForRecovery( + args.response, + args.outboundBody, + args.adapterName, + args.guard.attempted, + args.signal, + ); + if (errorBody === undefined || !shouldAttemptOpaqueBlobRecovery({ + status: args.response.status, + adapterName: args.adapterName, + outboundBody: args.outboundBody, + errorBody, + alreadyAttempted: args.guard.attempted, + })) { + return { kind: "skipped" }; + } + + args.guard.attempted = true; + const rejectedScope = args.parsed._reasoningReplayScope + ? { + clientThreadId: args.parsed._reasoningReplayScope.clientThreadId, + ...(args.parsed._reasoningReplayScope.current + ? { current: { ...args.parsed._reasoningReplayScope.current } } + : {}), + } + : undefined; + prepareOpaqueBlobRecovery(args.parsed); + try { void args.response.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + const result = await rebuild("opaque-blob-rejection"); + if (!("failed" in result) && result.ok) { + rememberReasoningReplayOpaqueBlobRejection(rejectedScope); + } + return "failed" in result + ? { kind: "failed", response: result.failed } + : { kind: "recovered", response: result }; +} diff --git a/src/server/responses/core-options.ts b/src/server/responses/core-options.ts new file mode 100644 index 0000000000..19adf73412 --- /dev/null +++ b/src/server/responses/core-options.ts @@ -0,0 +1,159 @@ +import type { OcxUsage, OcxProviderContinuationState, OcxConfig } from "../../types"; +import type { CodexAuthPolicyConfig, CodexAuthContext } from "../../codex/auth-context"; +import type { AdmissionLease } from "../../lib/admission"; +import type { DataPlaneAdmission } from "../auth-cors"; +import { resolveCodexModelEntitlements } from "../../codex/model-entitlements"; +import type { ResponsesTerminalStatus } from "../../bridge"; +import type { ResponsesTerminalRepairScheduler } from "../responses-terminal-repair"; +import type { BunRuntimeGateInput } from "./ws-upstream"; +import type { NativeMainRefreshDependencies } from "../../codex/main-account"; +import type { InboundWire } from "../../providers/registry"; +import type { ExplicitOpenAiCallerAuth } from "../../providers/openai-sidecar"; +import type { CallerDirectAuth } from "../../providers/caller-authorization"; +import type { TranslatorBudget } from "../../lib/translator-budget"; +import type { TransientSendBudget } from "../../lib/upstream-retry"; +import type { RequestLogContext } from "../request-log"; +import type { UpstreamHostAdmissionLease } from "../../codex/upstream-host-health"; + +export interface ConsumedComboFailure { + response: Response; + classificationText: string; + /** Structured upstream `error.code` when present in the failure body. */ + upstreamCode?: string; + /** Valid numeric/date value used only for cooldown calculation. */ + retryAfter?: string; + /** Upstream Codex quota-window reset timestamps used for combo cooldowns. */ + resetAt?: string[]; + /** Reserved for 040 usage attribution without adding another body read. */ + usage?: OcxUsage; +} + + + + +export interface HandleResponsesOptions { + /** Internal Claude replay identity; consumed only by the final canonical Go transport. */ + claudeGoAffinity?: { sessionLane?: string }; + /** Validated Claude metadata identity; projected only into final canonical attempt headers. */ + claudeNativeSessionId?: string; + /** Original live policy owner; separate from caller-specific routing/sidecar snapshots. */ + codexAuthPolicy?: CodexAuthPolicyConfig; + turnAdmissionLease?: AdmissionLease; + /** + * How the caller proved data-plane admission (#1686). + * + * A bearer-presented admission secret is one of OUR OWN secrets, so a Direct turn must + * SUBSTITUTE the stored main credential rather than forward it. Without this fact at the + * decision point, Direct cannot tell an admission bearer from the user own ChatGPT bearer, + * which is why it refused the whole env_key flow instead of serving it. + */ + admission?: DataPlaneAdmission; + /** Called at most once after the complete client body is read and accepted for dispatch. */ + onRequestBodyRead?: () => void; + forceEmptyResponseId?: boolean; + abortSignal?: AbortSignal; + /** One-shot TTFT callback: first non-empty model output observed (WP4). */ + onFirstOutput?: () => void; + onCodexAuthContextResolved?: (context: CodexAuthContext | undefined) => void; + /** Internal deterministic seam for account-gated native fallback tests. */ + resolveCodexModelEntitlements?: typeof resolveCodexModelEntitlements; + /** Internal: validated final client-visible model, after completed terminal success only. */ + onResponseComplete?: (model: string) => void; + recordTerminalOutcomes?: boolean; + setTerminalOutcomeRecorder?: (recorder: ((status: ResponsesTerminalStatus, httpStatusOverride?: number) => void) | undefined) => void; + onNativePassthroughTerminal?: (status: ResponsesTerminalStatus) => void; + onNativePassthroughCancel?: () => void; + /** Internal deterministic clock/timer seam for provider terminal repair. */ + responsesTerminalRepairScheduler?: ResponsesTerminalRepairScheduler; + /** Internal deterministic runtime-identity seam for Codex upstream WS selection tests. */ + codexWsRuntimeIdentity?: BunRuntimeGateInput; + /** Test seam for native main refresh without live OAuth traffic. */ + nativeMainRefreshDependencies?: NativeMainRefreshDependencies; + /** + * When true, body `prompt_cache_key` is a Claude Desktop shared cache cohort + * (system/tools hash), not a per-session id — do not use it for Anthropic pool affinity. + */ + promptCacheKeyIsSharedCohort?: boolean; + /** + * Wire protocol the ORIGINAL client spoke. The Chat and Anthropic surfaces translate + * their body into a Responses shape and replay through this function, so without an + * explicit value the replay would look like a native Responses request and an + * inbound-scoped registry wire default would fire for a client that never asked for + * it. Omitted means a genuine Responses inbound. + */ + inboundWire?: InboundWire; + /** Internal transport identity for route-scoped upstream compatibility policy. */ + inboundTransport?: "websocket"; + /** + * Claude replay may add native-main auth so OpenAI sidecars remain available. + * Strip only that internal credential when the final route is a noncanonical + * forward/caller-auth destination; final routing can differ from Claude's preflight route. + */ + stripClaudeMainAuthForNoncanonicalForward?: boolean; + /** In-memory credential proven by Claude's native-main turn claim; never persist or log. */ + trustedClaudeMainAuth?: { authorization: string; chatgptAccountId?: string }; + /** Sidecar-only auth captured before route changes; null means no usable original pair. */ + openAiSidecarAuth?: ExplicitOpenAiCallerAuth | null; + /** Internal Chat bridge permission to obtain claimed stored auth only for a final Direct sidecar. */ + allowStoredOpenAiSidecarAuth?: boolean; + /** Original caller-owned native pair; separate from any claimed sidecar enrichment. */ + nativeCallerAuth?: ExplicitOpenAiCallerAuth | null; + /** Caller Direct credential under Direct\'s own predicate; restored only for the canonical OpenAI final route. */ + callerDirectAuth?: CallerDirectAuth | null; + /** Internal recursion guard; callers outside this module must not set it. */ + comboAttempt?: boolean; + /** Internal combo handoff for one parent-validated continuation snapshot. */ + comboReplaySnapshot?: { + sourceBody: unknown; + previousResponseInputExpanded: boolean; + providerContinuation: OcxProviderContinuationState | undefined; + recoveredPlaintext: boolean; + }; + /** Internal combo handoff: allow a later same-provider model after a reset-derived 429/402. */ + deferCodexResetDerivedCooldown?: boolean; + /** 030-owned handoff when a child consumed the original failure under bounds. */ + onConsumedComboFailure?: (failure: ConsumedComboFailure) => void; + /** A stored Pool credential was refreshed and its one allowed same-account replay was sent. */ + onStoredPool401ReplayDispatched?: () => void; + /** Caller-owned for Chat/Claude replay; omitted only at genuine Responses ingress. */ + translatorBudget?: TranslatorBudget; + /** + * Transient sends already spent by this logical request. Combo children inherit the parent's + * holder through the options spread, so a fan-out shares one allowance instead of taking a + * fresh one per target (#4546). + */ + sendBudget?: TransientSendBudget; + /** + * Terminal vision-describe marker (roadmap 180): true when the inbound + * request IS the vision sidecar's own loopback describe call. The plan site + * then STRIPS images instead of planning another describe — a depth cap of 1 + * that holds under predicate drift and combo re-resolution. The Chat surface + * detects the raw `x-opencodex-vision-describe` header before its bridge + * rebuilds headers and carries the fact through this flag. + */ + visionDescribeTerminal?: boolean; +} + +/** Values shared by the call, not a bag of mutable pipeline state. */ +export interface ResponsesRequestContext { + req: Request; + config: OcxConfig; + logCtx: RequestLogContext; + options: HandleResponsesOptions & { translatorBudget: TranslatorBudget }; +} + +/** Admission leases remain owned by the outer finally until explicitly transferred. */ +export interface ResponsesAdmissionState { + pendingHostAdmissionLease: UpstreamHostAdmissionLease | null; + authCtx: CodexAuthContext; +} + +export interface PassthroughAdmissionState { + lease: UpstreamHostAdmissionLease | null; +} + +/** Recursive combo children enter the same ingress without a runtime import cycle. */ +export interface ResponsesDispatchers { + handleResponses(req: Request, config: OcxConfig, logCtx: RequestLogContext, options?: HandleResponsesOptions): Promise; + handleComboResponses(req: Request, body: unknown, comboId: string, config: OcxConfig, logCtx: RequestLogContext, options: HandleResponsesOptions & { translatorBudget: TranslatorBudget }): Promise; +} diff --git a/src/server/responses/core-replay.ts b/src/server/responses/core-replay.ts new file mode 100644 index 0000000000..c28cb1bac0 --- /dev/null +++ b/src/server/responses/core-replay.ts @@ -0,0 +1,225 @@ +import type { + OcxProviderContinuationOwner, + OcxProviderContinuationState, + OcxParsedRequest, + OcxProviderConfig, + OcxReasoningReplayIdentity, + AdapterEvent, +} from "../../types"; +import { + isValidProviderContinuationOwner, + sameProviderContinuationOwner, + providerContinuationOwnerFromReplayIdentity, + providerContinuationRouteScope, +} from "../../responses/provider-continuation"; +import { + reasoningReplayDestinationIdentity, + reasoningReplayOAuthCredentialIdentity, + durableReplayCredentialIdentity, + reasoningReplayCodexCredentialIdentity, + reasoningReplayKeyCredentialIdentity, + durableReplayDestinationIdentity, + bindReasoningReplayScope, + reasoningReplayServingIdentityChanged, + reasoningReplayOpaqueBlobRejectionMemoized, +} from "../../responses/reasoning-replay-cache"; +import type { OAuthAccessSnapshot } from "../../oauth"; +import type { CodexAuthContext } from "../../codex/auth-context"; +import { thoughtSignatureReplaySalt } from "../../responses/thought-signature-replay"; +import { randomUUID } from "node:crypto"; + +/** + * Adapters whose continuation state must survive Codex's store:false requests. + */ +export function adapterNeedsForcedContinuation(name: string): boolean { + return name === "kiro" || name === "cursor"; +} + + +export type ContinuationOwnerRead = + | { kind: "missing" } + | { kind: "invalid" } + | { kind: "valid"; owner: OcxProviderContinuationOwner }; + + +export function readProviderContinuationOwner( + state: OcxProviderContinuationState | undefined, +): ContinuationOwnerRead { + if (!state || state.__ocxOwner === undefined) return { kind: "missing" }; + const owner = state.__ocxOwner; + if (!isValidProviderContinuationOwner(owner)) return { kind: "invalid" }; + return { kind: "valid", owner: { ...owner } }; +} + + +export function providerContinuationPayload( + state: OcxProviderContinuationState | undefined, +): OcxProviderContinuationState | undefined { + if (!state) return undefined; + const cloned = structuredClone(state); + delete cloned.__ocxOwner; + return Object.keys(cloned).length > 0 ? cloned : undefined; +} + + +export function bindProviderContinuationForRoute( + parsed: OcxParsedRequest, + currentOwner: OcxProviderContinuationOwner | undefined, +): void { + const candidate = parsed._providerContinuationCandidate; + const storedOwner = readProviderContinuationOwner(candidate); + const mayRestore = storedOwner.kind === "valid" + && !!currentOwner + && sameProviderContinuationOwner(storedOwner.owner, currentOwner); + const restored = mayRestore ? providerContinuationPayload(candidate) : undefined; + if (restored) parsed._providerContinuation = restored; + else delete parsed._providerContinuation; + const cursorConversationId = restored?.cursor?.conversationId; + if (cursorConversationId) parsed._cursorConversationId = cursorConversationId; + else delete parsed._cursorConversationId; + if (currentOwner) parsed._providerContinuationOwner = { ...currentOwner }; + else delete parsed._providerContinuationOwner; +} + + +export function providerContinuationDestinationIdentity( + parsed: OcxParsedRequest, + provider: OcxProviderConfig, +): string | undefined { + const kiroContext = parsed._kiroAuthContext; + return reasoningReplayDestinationIdentity(JSON.stringify([ + provider.baseUrl.trim().replace(/\/+$/, ""), + provider.responsesPath ?? "", + kiroContext?.profileArn ?? "", + kiroContext?.apiRegion ?? "", + kiroContext?.ssoRegion ?? "", + ])); +} + + +export function bindRouteReasoningReplayScope(args: { + parsed: OcxParsedRequest; + providerName: string; + provider: OcxProviderConfig; + adapterName: string; + oauthCredentialSnapshot?: Pick; + codexAuthContext?: CodexAuthContext; + forwardHeaders?: Headers; +}): void { + const { parsed, providerName, provider, adapterName } = args; + let credentialIdentity: string | undefined; + let credentialDurableIdentity: string | undefined; + const durableSalt = thoughtSignatureReplaySalt(); + if (provider.authMode === "oauth") { + credentialIdentity = reasoningReplayOAuthCredentialIdentity( + args.oauthCredentialSnapshot, + provider.headers, + ); + // The persisted account-slot id survives token refresh and restarts; the rotating + // generation deliberately does NOT participate (#1926 design: rotation-safe). + credentialDurableIdentity = durableReplayCredentialIdentity( + "oauth", + args.oauthCredentialSnapshot?.accountId, + provider.headers, + durableSalt, + ); + } else if (provider.authMode === "forward") { + const poolContext = args.codexAuthContext?.kind === "pool" + || args.codexAuthContext?.kind === "main-pool" + ? args.codexAuthContext + : undefined; + credentialIdentity = reasoningReplayCodexCredentialIdentity({ + authorization: poolContext + ? `Bearer ${poolContext.accessToken}` + : args.forwardHeaders?.get("authorization"), + chatgptAccountId: poolContext?.chatgptAccountId + ?? args.forwardHeaders?.get("chatgpt-account-id"), + accountId: poolContext?.accountId, + credentialGeneration: poolContext?.kind === "pool" + ? poolContext.generation + : undefined, + writerGeneration: poolContext?.writerGeneration, + headers: provider.headers, + }); + // Durable identity requires a STABLE, TRUSTED account handle. Pool context comes from + // our own account store; a client-supplied chatgpt-account-id header is attacker + // -influenceable bucket selection and a bearer alone is rotating material — both are + // refused, so direct-forward turns get no durable scope (fail closed; the in-process + // cache still covers same-process replay). + const codexDurableHandle = poolContext?.accountId + ?? poolContext?.chatgptAccountId + ?? undefined; + credentialDurableIdentity = durableReplayCredentialIdentity( + "codex", + codexDurableHandle ?? undefined, + provider.headers, + durableSalt, + ); + } else if (provider.authMode !== "local") { + credentialIdentity = reasoningReplayKeyCredentialIdentity(provider); + credentialDurableIdentity = durableReplayCredentialIdentity( + "key", + nonEmptyProviderApiKey(provider), + provider.headers, + durableSalt, + ); + } + const providerDestinationIdentity = reasoningReplayDestinationIdentity(provider.baseUrl); + const replayIdentity: OcxReasoningReplayIdentity | undefined = credentialIdentity && providerDestinationIdentity + ? { + providerName, + providerDestinationIdentity, + providerDestinationDurableIdentity: durableReplayDestinationIdentity(provider.baseUrl), + adapterName, + modelId: parsed.modelId, + credentialIdentity, + ...(credentialDurableIdentity ? { credentialDurableIdentity } : {}), + } + : undefined; + const continuationDestinationIdentity = providerContinuationDestinationIdentity(parsed, provider); + const continuationOwner = providerContinuationOwnerFromReplayIdentity( + replayIdentity && continuationDestinationIdentity + ? { ...replayIdentity, providerDestinationIdentity: continuationDestinationIdentity } + : undefined, + ); + if (adapterName === "cursor") { + // The final route owner is authoritative for Cursor and supersedes the account-derived + // seed assigned before route binding. A Cursor conversation must be scoped to the exact + // provider/destination/adapter/model/credential that serves it. + if (continuationOwner) parsed._cursorIdentityScope = providerContinuationRouteScope(continuationOwner); + else if (!parsed._cursorIdentityScope?.startsWith("cursor-unowned:")) { + // Prevent the adapter's token-only fallback from recreating a provider-private id after the + // route owner failed closed. The sentinel is per parsed request and contains no credential. + parsed._cursorIdentityScope = `cursor-unowned:${randomUUID()}`; + } + } + bindReasoningReplayScope( + parsed._reasoningReplayScope, + replayIdentity, + ); + // Keep this sticky for the whole outbound request: a later auth/key rebind may compare equal + // after the first mismatch, but it cannot make history minted by the prior route decodable. + if (reasoningReplayServingIdentityChanged(parsed._reasoningReplayScope)) { + parsed._stripReasoningEncryptedContent = true; + } + if (reasoningReplayOpaqueBlobRejectionMemoized(parsed._reasoningReplayScope)) { + parsed._stripReasoningEncryptedContent = true; + } + bindProviderContinuationForRoute(parsed, continuationOwner); +} + + +export function adapterResponseReachedServingTerminal( + events: readonly AdapterEvent[], + response: Readonly>, +): boolean { + return (response.status === "completed" || response.status === "incomplete") + && events.some(event => event.type === "done" || event.type === "incomplete"); +} + + +export function nonEmptyProviderApiKey(provider: OcxProviderConfig): string | undefined { + return typeof provider.apiKey === "string" && provider.apiKey.trim().length > 0 + ? provider.apiKey + : undefined; +} diff --git a/src/server/responses/core.ts b/src/server/responses/core.ts index 7e2562b84e..1bef92779a 100644 --- a/src/server/responses/core.ts +++ b/src/server/responses/core.ts @@ -1,9386 +1,210 @@ -import { capturePoolQuotaWriter } from "../../codex/account-store"; -import { CODEX_POOL_REFRESH_INCOMPLETE_LOG_REASON } from "../../codex/pool-refresh-backoff"; -import type { Server } from "bun"; -import { recordContextSessionOwner } from "../../codex/context-owner"; -import { contextRelayActivated } from "../../codex/context-compat"; -import { randomUUID } from "node:crypto"; -import { bridgeToResponsesSSE, buildResponseJSON, formatErrorResponse, type ResponsesTerminalStatus } from "../../bridge"; -import { formatPassthroughUpstreamError } from "./passthrough-error"; -import { - createResponsesFieldBackfillBlockRewrite, - backfillResponsesFieldsJson, -} from "./responses-field-backfill"; -import { checkInputAdmission } from "./input-admission"; -import { - checkOutboundBodySize, - describeOutboundBodyRefusal, -} from "./outbound-body-guard"; -import { nativeContextLimits } from "../../codex/catalog"; -import { describeUpstreamConnectFailure } from "./upstream-error"; -import type { CodexWsQuotaObserver } from "./codex-ws-metadata"; -import { applyAccountQuotaFromUpstreamHeaders as applyCapturedCodexQuota } from "../../codex/quota"; -import { isCodexAccountGenerationLive } from "../../codex/account-store"; -import { isCodexWsQuotaObservedResponse } from "./ws-upstream"; -import { - multiAgentGuidanceEnabled, - resolveEnvValue, -} from "../../config"; -import { parseRequest } from "../../responses/parser"; -import { - bindReasoningReplayScope, - commitReasoningReplayServingIdentity, - reasoningReplayCodexCredentialIdentity, - reasoningReplayDestinationIdentity, - durableReplayDestinationIdentity, - durableReplayCredentialIdentity, - reasoningReplayKeyCredentialIdentity, - reasoningReplayOpaqueBlobRejectionMemoized, - reasoningReplayOAuthCredentialIdentity, - reasoningReplayServingIdentityChanged, - rememberReasoningReplayOpaqueBlobRejection, -} from "../../responses/reasoning-replay-cache"; -import { awaitThoughtSignatureDurability, thoughtSignatureReplaySalt } from "../../responses/thought-signature-replay"; -import { buildCompactV1Output, COMPACT_PROMPT, decodeCompactionSummary, extractCompactUserMessages } from "../../responses/compaction"; -import { FORWARD_HEADERS, sanitizeReasoningInputContent } from "../../adapters/openai-responses"; -import { XaiToolSchemaCompatibilityError } from "../../adapters/xai-tool-schema"; -import { - copyPreviousResponseReplayProvenance, - expandPreviousResponseInput, - markBodyNonPersistable, - previousResponseProviderState, - previousResponseReplayFailure, - previousResponseScopeMismatch, - rememberResponseState, -} from "../../responses/state"; -import { - bindTurnTerminationScope, - rememberDeliveredFinalAnswer, -} from "../../responses/turn-termination"; -import { - isValidProviderContinuationOwner, - mergeProviderContinuationPayload, - providerContinuationOwnerFromReplayIdentity, - providerContinuationRouteScope, - sameProviderContinuationOwner, -} from "../../responses/provider-continuation"; -import { - rememberComboForLane, - recallComboForLane, -} from "./combo-session-recall"; -import { - comboRouteDecisionTrace, - NoEligiblePolicyCandidateError, - routeCompactionModel, - routeConcreteModel, - routeModel, - type RouteResult, -} from "../../router"; -import { evidenceFromBody } from "../../routing/request-evidence"; -import { resolvePassiveRouteSubjectId } from "../passive-route-linker"; -import { - advanceComboAfterFailure, - comboCooldownRetryAfterSeconds, - comboDefaultEffort, - comboFailureCooldownScope, - comboFailureDecision, - comboIdFromRawBody, - comboRequestHasImageInput, - concreteComboRequestBody, - getCombo, - resolveComboId, - isComboTargetInCooldown, - NoAvailableComboTargetsError, - noteComboSuccess, - parseRetryAfterMs, - pickComboTarget, - pickComboTargetWithWait, - targetKey, -} from "../../combos"; -import { isInjectionDebugEnabled } from "../../lib/debug-settings"; -import { - CYBER_POLICY_ERROR_CODE, - CYBER_POLICY_FALLBACK_MESSAGE, - adapterFailureFromMessage, - isCyberPolicyCode, - isCyberPolicyMessage, -} from "../../lib/errors"; -import { injectionDebugLog } from "../../lib/injection-debug-log"; -import { resolveClientRetryAfter } from "../../lib/retry-after"; -import { - enrichOpenCodeZenUpstreamMessage, - isTransientConsoleGoUploadRejection, -} from "../../providers/opencode-zen-rate-limit"; -import { CODE_MODE_EXEC_TOOL_NAME, modelInList, namespacedToolName } from "../../types"; +import type { OcxConfig } from "../../types"; +import type { RequestLogContext } from "../request-log"; import type { - AdapterEvent, - OcxConfig, - OcxParsedRequest, - OcxProviderConfig, - OcxProviderContinuationOwner, - OcxProviderContinuationState, - OcxReasoningReplayIdentity, - OcxUsage, - TierDecision, -} from "../../types"; -import { - forceRefreshOAuthAccessSnapshot, - getValidAccessTokenForAccount, - getValidAccessSnapshotForAccount, - getValidAccessTokenSnapshot, - publicOAuthAuthenticationErrorMessage, - type OAuthAccessSnapshot, - UnsupportedOAuthProviderError, -} from "../../oauth"; -import { captureOAuthAccountSelection, commitOAuthAccountSelection, credentialGeneration, getAccountCredentialWithStatus } from "../../oauth/store"; -import { - ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST, - anthropicSessionKeyFromParts, - commitAnthropicSelectionRouting, - formatAnthropicProviderForLog, - getAnthropicPoolAccessSnapshot, - getAnthropicPoolRetryAfterSeconds, - isAnthropicAccountPoolEnabled, - hasAnthropicFailoverQuorum, - resolveAnthropicAccountForSession, - rotateAnthropicAccountOn429, - type AnthropicAccountSelectionReason, -} from "../../oauth/anthropic-routing"; -import { stampOAuthAccountLabel } from "../../providers/label"; -import { - failoverAccountSnapshot, - forgetGenericFailoverRoster, - GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST, - isGenericFailoverProvider, - isGenericOAuthFailoverEnabled, - noteGenericPoolSelection, - preferredInitialAccount, - rotateGenericOAuthAccountOn429, -} from "../../oauth/generic-account-failover"; -import { resolveCopilotApiBaseUrl } from "../../oauth/github-copilot"; -import { buildWebSearchTool, planWebSearch, runWithWebSearch, shouldResolveOpenAiWebSearchSidecar } from "../../web-search"; -import { - createPassthroughWebSearchBridgeExecutor, - createPassthroughWebSearchBridgeStream, - planPassthroughWebSearchBridge, - resolvePassthroughWebSearchBridgeAuth, - shouldResolveOpenAiPassthroughWebSearchBridge, -} from "../../web-search/passthrough-bridge"; -import { buildImageTool, buildVideoTool, planImageBridge, planVideoBridge, runWithImageBridge, clampImageMaxRounds, IMAGE_GEN_TOOL_NAME, VIDEO_GEN_TOOL_NAME } from "../../images"; -import { describeImagesInPlace, planVisionSidecar, requiresVisionPreprocessing, resolveOpenAiVisionModel, shouldResolveOpenAiVisionSidecar, stripImagesInPlace } from "../../vision"; -import { createAdapterEventQueue, preflightAdapterEvents, type AdapterEventQueue } from "../../adapters/run-turn-queue"; -import { - applyCodexAuthContextToProvider, - createCodexReserveDispatchGuard, - unwrapUpstreamRetryEvidenceError, - codexPoolAffinityKey, - previewCodexPoolLineage, - CodexAccountCooldownError, - CodexAuthContextError, - CodexMainProfileDrainingError, - CodexPoolAuthenticationError, - CodexThreadAffinityExpiredError, - headersForCodexAuthContext, - materializeCodexUpstreamAuthAsync, - isCodexAuthContextUsable, - resolveCodexAuthContext, - codexProbeLeaseId, - codexProbeQuotaScope, - releaseCodexAuthContextProbeLease, - stripCodexRuntimeProviderFields, - type CodexAuthContext, - type CodexAuthPolicyConfig, -} from "../../codex/auth-context"; -import { - entitledCodexAccountIdsForModel, - invalidateCodexModelEntitlementsForAccount, - resolveCodexModelEntitlements, -} from "../../codex/model-entitlements"; -import { ACCOUNT_GATED_NATIVE_OPENAI_MODELS } from "../../codex/catalog/native-models"; -import { - MAIN_CODEX_ACCOUNT_ID, - forceRefreshMainAccountToken, - type NativeMainRefreshDependencies, -} from "../../codex/main-account"; -import { captureCodexAffinityDiagnostic } from "../../codex/affinity-debug"; -import { - computeQuotaCooldown, - codexQuotaScopeForModel, - formatCodexProviderForLog, - handOffThreadAffinityGeneration, - previewCodexAccountForRequest, - recordCodexUpstreamOutcome, - type CodexUpstreamOutcome, -} from "../../codex/routing"; -import { - TokenRefreshError, - isTerminalCodexPoolRefreshFailure, - forceRefreshCodexPoolToken, - readCodexAccountRecord, -} from "../../codex/account-store"; -import { codexAuthContextLogLabel } from "../../codex/account-label"; -import { - applyUpstreamRecoveryInit, - fetchWithResetRetry, - fetchWithTransientRetry, - isNonReplayableResponse, - isTransientUpstreamStatus, - prepareSameTarget429Wait, - sleepWithAbort, - TRANSIENT_RETRY_MAX_ATTEMPTS, - SendBudgetExhaustedError, - type TransientSendBudget, -} from "../../lib/upstream-retry"; -import { - createRequestExecutionBudget, - isRequestExecutionBudget, - CODEX_TEXT_GUARDED_BUDGET_POLICY, - type RequestExecutionBudget, - type RequestExecutionBudgetPolicy, - type SendClass, - type SingleUseDispatchPermit, -} from "../../lib/request-execution-budget"; -import { - chargeWorkflowSends, - workflowSendCeilingReached, -} from "../../lib/workflow-budget"; -import { workflowRefusalResponse } from "../workflow-refusal"; -import { - ForwardAdmissionCredentialError, - hasForwardableCodexBearer, - isProxyAdmissionSecret, - validateForwardAdmissionCredential, -} from "../auth-cors"; -import { resolveContextPrincipal } from "../auth-cors"; -import type { DataPlaneAdmission } from "../auth-cors"; -import { createTranslatorBudget, isTranslatorBudgetExceededError, type TranslatorBudget } from "../../lib/translator-budget"; -import { captureExplicitOpenAiCallerAuth, listOpenAiForwardSidecarCandidates, resolveFirstUsableOpenAiSidecar, type ExplicitOpenAiCallerAuth, type ResolvedOpenAiForwardSidecar } from "../../providers/openai-sidecar"; -import { inspectChatGptDomainClaim } from "../../oauth/chatgpt"; -import { captureCallerDirectAuth, providerConsumesCallerAuthorization, type CallerDirectAuth } from "../../providers/caller-authorization"; -import { isCanonicalOpenAiForwardProvider, OPENAI_CODEX_PROVIDER_ID } from "../../providers/openai-tiers"; -import { CODEX_RESERVE_HELPER_UNSUPPORTED_MESSAGE, isCodexReserveHelperUnsupported } from "../../codex/loopback-target"; -import { providerContextCap } from "../../providers/context-cap"; -import { - fastPolicyForModel, - serviceTierSupportFromPolicy, - SERVICE_TIER_ADAPTERS, -} from "../../providers/service-tier"; -import { - canonicalFastTierMarker, - decideTier, - tierObservationContext, - tierValueAfterDecision, - type ResolvedFastPolicy, -} from "../../providers/fastwire"; -import { - RequestPacingQueueOverloadError, - waitForProviderRequestSlot, -} from "../../providers/request-pacing"; -import { slugsEquivalent } from "../../providers/slug-codec"; -import { isMuseSubscriptionUsagePayload, parseMuseSubscriptionUsage } from "../../providers/muse-subscription-usage"; -import { hasPassiveAccountQuota, recordAnthropicAccountQuotaFromHeaders, recordPassiveAccountQuota } from "../../providers/quota"; -import { captureConfigGeneration } from "../../lib/state-store-sweeper"; -import { applyOpenAiVirtualModel, resolveOpenAiCompactModel } from "../../providers/openai-virtual-models"; -import { isUsageDebugEnabled } from "../../usage/debug"; -import { - readJsonRequestBody, - describeInboundBodyRefusal, - resolveInboundBodyLimitBytes, - DecompressedBodyTooLargeError, - UnsupportedContentEncodingError, -} from "../request-decompress"; -import { resolveAdapter, resolveWireProtocolOverride } from "../adapter-resolve"; -import { - providerModelResponsesTerminalRepair, - providerModelResponsesUpstreamStreaming, - type InboundWire, -} from "../../providers/registry"; -import type { AdapterRequest, ProviderAdapter } from "../../adapters/base"; -import { providerApiKeySelectionIsCurrent, resolveCurrentProviderApiKeyTransport } from "../../providers/api-key-selection"; -import { - hasKeyPoolFailover, - selectProactiveApiKeyTransport, - rateLimitRetryDelayMs, - rateLimitRetryPolicyFor, - rotateProviderTransportOn429, - rotateProviderTransportOn401, - transientRetryPolicyFor, -} from "../../providers/key-failover"; -import { shouldAttemptImageTierRetry } from "../image-retry"; -import { isXaiResponsesDestination, resolveProviderTransport } from "../../providers/xai-transport"; -import { resolveOpenCodeGoTransport } from "../../providers/opencode-go-transport"; -import type { WsData } from "../ws-bridge"; -import { - codexAccountSelectionForTurn, - registerTurn, - trackStreamLifetime, - tryClaimNativeMainProfileForTurn, - unregisterTurn, -} from "../lifecycle"; -import { redactSecretString, sanitizeLogMetadataString } from "../../lib/redact"; -import { readBoundedResponseBody } from "../../lib/bounded-body"; -import { isReasoningEffortRejection, planReasoningEffortDowngrade } from "../../providers/reasoning-metadata"; -import { - ENCRYPTED_FUNCTION_OUTPUT_REJECTION, - isRateLimitOrQuotaFailureMessage, - upstreamErrorMessageFromPayload, -} from "../../lib/errors"; -import type { AdmissionLease } from "../../lib/admission"; -import { tryClaimNativeMainProfileForTurn as tryClaimStoredSidecarMainProfile } from "../../codex/native-main-admission"; -import { prepareEffortNormalization, supportedLadderFor } from "../effort-policy"; -import { isThreadSpawnRequest } from "../effort-policy"; -import { - applySubagentModelFallback, - maybePrimeSubagentQuota, - recordSubagentQuotaFailureForThreadSpawn, - resolveSubagentFallbackChain, - subagentFallbackNeedsModelEntitlements, - type SubagentModelEligibleAccountIds, - type SubagentPoolAccountPreview, -} from "../../codex/subagent-model-fallback"; -import { isNativeMainTrafficBlocked } from "../../codex/native-profile-startup"; -import { - beginRequestAttempt, - finishRequestAttempt, - inspectResponseLogJson, - noteAttemptSend, - readConfiguredCodexServiceTier, - recordAdapterReasoning, - recordAdapterTier, - recordAdapterTierMetadata, - recordAttemptRequestedEffort, - requestLogSpeedLabel, - sealRequestAttemptIdentity, - recordAttemptCredentialSource, - usageFromResponsesPayload, - type RequestLogContext, - markLocalRequestLogRefusal, -} from "../request-log"; -import { - conversationIdFromResponsesRequest, - getOrAllocateRequestSessionLane, - linkRequestSessionLane, - normalizeLogConversationId, - reasoningReplayConversationIdFromResponsesRequest, - sessionLaneIdFromRequest, - sessionIdHeaderFromRequest, -} from "../request-log-conversation"; -import type { AttemptRecoveryKind } from "../../usage/log"; -import { - consumeForInspection, - consumeForResponseLogMetadata, - createSseInspector, - terminalStatusFromParsed, - isEagerRelaySseResponse, - isNativePassthroughSseResponse, - markEagerRelaySseResponse, - markNativePassthroughSseResponse, - relaySseWithFailedTail, - codexSafetyBufferingFilterOptions, - relayWithAbort, - sanitizePassthroughHeaders, -} from "../relay"; -import { - agentTaskRecoveryConfig, - discardEncryptedAgentTaskRecovery, - recoverEncryptedAgentTaskWithResult, - restoreCachedEncryptedAgentTasks, - type AgentTaskRecoveryFailureReason, -} from "./agent-task-recovery"; -import { relaySseEagerBounded } from "../relay-eager"; -import { - relayResponsesSseWithTerminalRepair, - type ResponsesTerminalRepairScheduler, -} from "../responses-terminal-repair"; -import { isWin32EagerRewrite, selectEagerPath } from "../../lib/bun-stream-caps"; -import { cancelBodyOnAbort } from "../../lib/abort"; -import { isCodexWsUpstreamResponse, type BunRuntimeGateInput } from "./ws-upstream"; -import { readCodexWsStage } from "./codex-ws-wire"; -import { - createResponsesItemIdPayloadRewrite, - hasResponsesItemIdRepair, - repairResponsesJsonItemIds, -} from "../responses-item-id-repair"; -import { - createImageGenCallRestoreRewrite, - imageGenToolCallAliases, - restoreImageGenCallsInJson, -} from "../responses-image-gen-repair"; -import { createResponsesModelPayloadRewrite, rewriteResponsesModelJson } from "../responses-model-rewrite"; -import { parseRequestEffortRowId } from "../effort-row"; -import { parseSyntheticRowId } from "../fast-row"; -import { - collectSelfNamedNamespaceScrubAuthorization, - createSelfNamedToolCallNamespaceScrubRewrite, - scrubSelfNamedToolCallNamespaceInJson, -} from "../responses-self-named-namespace-scrub"; -import type { EffectiveSubagentRoster, SpawnAgentSurface } from "../../codex/catalog"; + HandleResponsesOptions, + ResponsesRequestContext, + ResponsesAdmissionState, + ResponsesDispatchers, +} from "./core-options"; +import { createTranslatorBudget } from "../../lib/translator-budget"; +import { captureExplicitOpenAiCallerAuth } from "../../providers/openai-sidecar"; +import { captureCallerDirectAuth } from "../../providers/caller-authorization"; +import { createRequestExecutionBudget } from "../../lib/request-execution-budget"; +import { finalizeOwnedTranslatorBudget } from "./core-lifetime"; +import type { TranslatorBudget } from "../../lib/translator-budget"; +import { executeComboResponses } from "./core-combo"; +import { prepareResponsesRequest } from "./request-prepare"; +import { prepareResponsesTransport } from "./request-transport"; +import { prepareResponsesSidecarAuth } from "./request-sidecar-auth"; +import { createResponsesEffects } from "./response-effects"; +import { createResponsesSendBudget } from "./request-send-budget"; +import { executePassthroughResponse } from "./passthrough-execution"; +import { executeResponsesSidecars } from "./sidecar-execution"; +import { createResponsesCompletionPolicy } from "./completion-policy"; +import { executeResponsesRunTurn } from "./run-turn-execution"; +import { prepareAdapterExchange } from "./adapter-dispatch"; +import { createAdapterContinuations } from "./adapter-continuation"; +import { deliverAdapterResponse } from "./adapter-delivery"; +import { releaseUpstreamHostAdmission } from "../../codex/upstream-host-health"; +import { releaseCodexAuthContextProbeLease } from "../../codex/auth-context"; + +/** Public Responses entry and compatibility exports. Implementations live with their owners. */ -import { buildToolBridgeMaps, collabSurface, injectDeveloperMessage, multiAgentGuidanceText } from "./collaboration"; -import { mapCodexAuthContextErrorToResponse, nativeMainRefreshFailureResponse } from "./codex-auth-error"; -import { hasUnreadableEncryptedAgentTask, looksLikeBackendCiphertext, sanitizeEncryptedContentInPlace, stripAgentMessageCiphertextInPlace } from "./encrypted-payload"; -import { - applyAccountChangeConversationStateScrub, - conversationStateBindingFromAuth, - rememberServingConversationStateIssuer, -} from "./account-change-state"; -import { fetchWithHeaderTimeout, providerFetch, safeHostLabel, safeOriginLabel, storedPoolReplayDispatchNotifier, type ProviderFetchOptions } from "./fetch-helpers"; -import { classifyTransportFailureKind, transportErrorCode } from "../../lib/upstream-reachability"; -import { - acquireUpstreamHostAdmission, - disableUpstreamHostCircuitForKey, - normalizeUpstreamHostCircuitThreshold, - recordUpstreamHostFailure, - releaseUpstreamHostAdmission, - resetUpstreamHostHealth, - upstreamHostHealthKey, - type UpstreamHostAdmissionLease, -} from "../../codex/upstream-host-health"; -import { createGrokResponsesSparseTerminalBlockRewrite } from "../grok-responses-snapshot-repair"; -import { createGrokResponsesControlFrameBlockRewrite } from "../grok-responses-control-frame"; -import { - createResponsesSnapshotBlockRewrite, - hasResponsesSnapshotRepair, - repairResponsesSnapshotJson, -} from "../responses-snapshot-repair"; -import { - composeSseBlockRewrites, - composeSsePayloadRewrites, - payloadRewriteAsBlockRewrite, - relaySseWithBlockRewrite, -} from "../sse-payload-rewrite"; -import { hasUnmappedRoutedCustomToolOutput, restoreRoutedCustomCalls, restoreRoutedCustomCallsInJson } from "../../responses/custom-tool-compat"; -import { createRoutedCustomToolRestoreBlockRewrite } from "../responses-custom-tool-repair"; -import { collectFunctionCallRepairSchemas, repairFunctionCallsInJson } from "../../responses/function-call-compat"; -import { createResponsesFunctionToolRepairBlockRewrite } from "../responses-function-tool-repair"; -import { restoreRoutedToolSearchCallsInJson } from "../../responses/tool-search-compat"; -import { createRoutedToolSearchRestoreBlockRewrite } from "../responses-tool-search-repair"; -import { - createRoutedNamespaceCallRestoreRewrite, - NamespaceToolCollisionError, - restoreRoutedNamespaceCalls, - restoreRoutedNamespaceCallsInJson, - type RoutedNamespaceToolAliases, -} from "../../responses/namespace-tool-compat"; -import { - createPlaintextV2AgentMessageCallRestoreRewrite, - PLAINTEXT_V2_AGENT_MESSAGE_RESTORE_OVERFLOW_MESSAGE, - restorePlaintextV2AgentMessageCalls, - restorePlaintextV2AgentMessageCallsInJsonResult, - shouldPreparePlaintextV2AgentMessages, -} from "../../responses/plaintext-v2-agent-messages"; -import { - createMuseToolNameRestoreRewrite, - restoreMuseToolNames, - restoreMuseToolNamesInJson, - type MuseToolNameAliases, -} from "../../responses/muse-tool-name-alias"; -import { - collectDeclaredBareWireToolNames, - collectDeclaredNamelessClientCallTypes, - collectDeclaredWireToolNames, - collectProviderExecutedCallTypes, - createUndeclaredToolCallGuardBlockRewrite, - normalizeDefaultNamespaceInJson, - normalizeDefaultNamespaceInResponse, - currentTurnWireToolCatalogBody, - hasExplicitWireToolCatalog, - undeclaredToolCallMessage, - undeclaredToolCallName, - undeclaredToolCallNameInResponse, - type ProviderExecutedCallType, -} from "../responses-undeclared-tool-guard"; -import { createGithubCopilotResponsesBlockRewrite } from "../github-copilot-responses-repair"; -import { responsesJsonToSseStream } from "../responses-json-events"; -import { jsonContextOverflowResponse, streamingContextOverflowResponse } from "./context-overflow"; -import { guardTerminalEventStream } from "./terminal-guard"; -import { - emptyCompletionRetryEnabled, - emptyCompletionNotice, - observeEmptyCompletion, - guardEmptyCompletionEventStream, -} from "./empty-completion-guard"; -import { preflightComboStreamResponse } from "./combo-stream-preflight"; - -// runTurn adapters own an event queue and perform their combo preflight before -// bridging. A second byte-stream reader would reinterpret that transport's -// already-committed event boundary and can replay custom adapter work. -const runTurnAdapterSseResponses = new WeakSet(); /** - * Adapters whose continuation state must survive Codex's store:false requests. + * Route one `/v1/responses` request through the adapter pipeline: recovery loop, passthrough + * wire, image/web-search bridges, and the terminal-guard continuation. */ -export function adapterNeedsForcedContinuation(name: string): boolean { - return name === "kiro" || name === "cursor"; -} - -export function sidecarOutcomeRecorder( +export async function handleResponses( + req: Request, config: OcxConfig, - authCtx: CodexAuthContext, -): ((outcome: CodexUpstreamOutcome) => void) | undefined { - return authCtx.kind === "pool" || authCtx.kind === "main-pool" - ? outcome => recordCodexUpstreamOutcome(config, authCtx.accountId, outcome, { - threadId: authCtx.affinityKey, - fixedAccount: authCtx.fixedAccount, - probeLeaseId: authCtx.probeLeaseId, - probeQuotaScope: authCtx.probeQuotaScope, - writerGeneration: authCtx.writerGeneration, - // A vision or web-search sidecar can return 401/403, and that is evidence about the exact - // stored credential it used. Without the generation it becomes an account-wide quarantine - // that a replacement inherits (#2892 gap 4). `main-pool` has no stored-record generation, so - // it keeps the unfenced account-wide semantics. - ...(authCtx.kind === "pool" ? { credentialGeneration: authCtx.generation } : {}), - }) - : undefined; -} - - - -import { isShadowSourceModel, shadowSourceModelPrefix, shouldInterceptShadowCall } from "../../lib/shadow-call"; - -export { DEFAULT_SHADOW_SOURCE_MODELS, isShadowSourceModel, shadowSourceModels } from "../../lib/shadow-call"; - - - -export function codexLogAccountId(authCtx: CodexAuthContext): string | null { - return authCtx.kind === "pool" || authCtx.kind === "main-pool" ? authCtx.accountId : null; -} - -type ContinuationOwnerRead = - | { kind: "missing" } - | { kind: "invalid" } - | { kind: "valid"; owner: OcxProviderContinuationOwner }; - -function readProviderContinuationOwner( - state: OcxProviderContinuationState | undefined, -): ContinuationOwnerRead { - if (!state || state.__ocxOwner === undefined) return { kind: "missing" }; - const owner = state.__ocxOwner; - if (!isValidProviderContinuationOwner(owner)) return { kind: "invalid" }; - return { kind: "valid", owner: { ...owner } }; -} - -function providerContinuationPayload( - state: OcxProviderContinuationState | undefined, -): OcxProviderContinuationState | undefined { - if (!state) return undefined; - const cloned = structuredClone(state); - delete cloned.__ocxOwner; - return Object.keys(cloned).length > 0 ? cloned : undefined; -} - -function bindProviderContinuationForRoute( - parsed: OcxParsedRequest, - currentOwner: OcxProviderContinuationOwner | undefined, -): void { - const candidate = parsed._providerContinuationCandidate; - const storedOwner = readProviderContinuationOwner(candidate); - const mayRestore = storedOwner.kind === "valid" - && !!currentOwner - && sameProviderContinuationOwner(storedOwner.owner, currentOwner); - const restored = mayRestore ? providerContinuationPayload(candidate) : undefined; - if (restored) parsed._providerContinuation = restored; - else delete parsed._providerContinuation; - const cursorConversationId = restored?.cursor?.conversationId; - if (cursorConversationId) parsed._cursorConversationId = cursorConversationId; - else delete parsed._cursorConversationId; - if (currentOwner) parsed._providerContinuationOwner = { ...currentOwner }; - else delete parsed._providerContinuationOwner; -} - -function providerContinuationDestinationIdentity( - parsed: OcxParsedRequest, - provider: OcxProviderConfig, -): string | undefined { - const kiroContext = parsed._kiroAuthContext; - return reasoningReplayDestinationIdentity(JSON.stringify([ - provider.baseUrl.trim().replace(/\/+$/, ""), - provider.responsesPath ?? "", - kiroContext?.profileArn ?? "", - kiroContext?.apiRegion ?? "", - kiroContext?.ssoRegion ?? "", - ])); -} - -function bindRouteReasoningReplayScope(args: { - parsed: OcxParsedRequest; - providerName: string; - provider: OcxProviderConfig; - adapterName: string; - oauthCredentialSnapshot?: Pick; - codexAuthContext?: CodexAuthContext; - forwardHeaders?: Headers; -}): void { - const { parsed, providerName, provider, adapterName } = args; - let credentialIdentity: string | undefined; - let credentialDurableIdentity: string | undefined; - const durableSalt = thoughtSignatureReplaySalt(); - if (provider.authMode === "oauth") { - credentialIdentity = reasoningReplayOAuthCredentialIdentity( - args.oauthCredentialSnapshot, - provider.headers, - ); - // The persisted account-slot id survives token refresh and restarts; the rotating - // generation deliberately does NOT participate (#1926 design: rotation-safe). - credentialDurableIdentity = durableReplayCredentialIdentity( - "oauth", - args.oauthCredentialSnapshot?.accountId, - provider.headers, - durableSalt, - ); - } else if (provider.authMode === "forward") { - const poolContext = args.codexAuthContext?.kind === "pool" - || args.codexAuthContext?.kind === "main-pool" - ? args.codexAuthContext - : undefined; - credentialIdentity = reasoningReplayCodexCredentialIdentity({ - authorization: poolContext - ? `Bearer ${poolContext.accessToken}` - : args.forwardHeaders?.get("authorization"), - chatgptAccountId: poolContext?.chatgptAccountId - ?? args.forwardHeaders?.get("chatgpt-account-id"), - accountId: poolContext?.accountId, - credentialGeneration: poolContext?.kind === "pool" - ? poolContext.generation - : undefined, - writerGeneration: poolContext?.writerGeneration, - headers: provider.headers, - }); - // Durable identity requires a STABLE, TRUSTED account handle. Pool context comes from - // our own account store; a client-supplied chatgpt-account-id header is attacker - // -influenceable bucket selection and a bearer alone is rotating material — both are - // refused, so direct-forward turns get no durable scope (fail closed; the in-process - // cache still covers same-process replay). - const codexDurableHandle = poolContext?.accountId - ?? poolContext?.chatgptAccountId - ?? undefined; - credentialDurableIdentity = durableReplayCredentialIdentity( - "codex", - codexDurableHandle ?? undefined, - provider.headers, - durableSalt, - ); - } else if (provider.authMode !== "local") { - credentialIdentity = reasoningReplayKeyCredentialIdentity(provider); - credentialDurableIdentity = durableReplayCredentialIdentity( - "key", - nonEmptyProviderApiKey(provider), - provider.headers, - durableSalt, - ); - } - const providerDestinationIdentity = reasoningReplayDestinationIdentity(provider.baseUrl); - const replayIdentity: OcxReasoningReplayIdentity | undefined = credentialIdentity && providerDestinationIdentity - ? { - providerName, - providerDestinationIdentity, - providerDestinationDurableIdentity: durableReplayDestinationIdentity(provider.baseUrl), - adapterName, - modelId: parsed.modelId, - credentialIdentity, - ...(credentialDurableIdentity ? { credentialDurableIdentity } : {}), - } - : undefined; - const continuationDestinationIdentity = providerContinuationDestinationIdentity(parsed, provider); - const continuationOwner = providerContinuationOwnerFromReplayIdentity( - replayIdentity && continuationDestinationIdentity - ? { ...replayIdentity, providerDestinationIdentity: continuationDestinationIdentity } - : undefined, - ); - if (adapterName === "cursor") { - // The final route owner is authoritative for Cursor and supersedes the account-derived - // seed assigned before route binding. A Cursor conversation must be scoped to the exact - // provider/destination/adapter/model/credential that serves it. - if (continuationOwner) parsed._cursorIdentityScope = providerContinuationRouteScope(continuationOwner); - else if (!parsed._cursorIdentityScope?.startsWith("cursor-unowned:")) { - // Prevent the adapter's token-only fallback from recreating a provider-private id after the - // route owner failed closed. The sentinel is per parsed request and contains no credential. - parsed._cursorIdentityScope = `cursor-unowned:${randomUUID()}`; - } - } - bindReasoningReplayScope( - parsed._reasoningReplayScope, - replayIdentity, - ); - // Keep this sticky for the whole outbound request: a later auth/key rebind may compare equal - // after the first mismatch, but it cannot make history minted by the prior route decodable. - if (reasoningReplayServingIdentityChanged(parsed._reasoningReplayScope)) { - parsed._stripReasoningEncryptedContent = true; - } - if (reasoningReplayOpaqueBlobRejectionMemoized(parsed._reasoningReplayScope)) { - parsed._stripReasoningEncryptedContent = true; - } - bindProviderContinuationForRoute(parsed, continuationOwner); -} - -function adapterResponseReachedServingTerminal( - events: readonly AdapterEvent[], - response: Readonly>, -): boolean { - return (response.status === "completed" || response.status === "incomplete") - && events.some(event => event.type === "done" || event.type === "incomplete"); -} - -const OPAQUE_RESPONSES_INPUT_TYPES = new Set([ - "reasoning", - "compaction", - "compaction_summary", - "context_compaction", -]); -const FUNCTION_OUTPUT_TYPES = new Set(["function_call_output", "custom_tool_call_output"]); -// codex-app subagent results replay as agent_message items whose content parts may carry -// backend-minted encrypted_content; the ChatGPT backend decrypts them in its function-output -// path, so a cross-identity replay of those parts produces ENCRYPTED_FUNCTION_OUTPUT_REJECTION. -const AGENT_MESSAGE_TYPE = "agent_message"; - -function encryptedFunctionOutputParts(output: unknown): boolean { - return Array.isArray(output) && output.some(part => ( - part !== null - && typeof part === "object" - && !Array.isArray(part) - && (part as { type?: unknown }).type === "encrypted_content" - && typeof (part as { encrypted_content?: unknown }).encrypted_content === "string" - && (part as { encrypted_content: string }).encrypted_content.length > 0 - )); -} - -function outboundResponsesInput(bodyText: string | undefined): unknown[] | undefined { - if (!bodyText) return undefined; - try { - const body = JSON.parse(bodyText) as unknown; - if (!body || typeof body !== "object" || Array.isArray(body)) return undefined; - const input = (body as { input?: unknown }).input; - return Array.isArray(input) ? input : undefined; - } catch { - return undefined; - } -} - -function outboundResponsesBodyCarriesEncryptedFunctionOutput(bodyText: string | undefined): boolean { - const input = outboundResponsesInput(bodyText); - if (!input) return false; - return input.some(item => { - if (item === null || typeof item !== "object" || Array.isArray(item)) return false; - const candidate = item as { type?: unknown; output?: unknown; content?: unknown }; - const type = String(candidate.type ?? ""); - if (FUNCTION_OUTPUT_TYPES.has(type) && encryptedFunctionOutputParts(candidate.output)) return true; - return type === AGENT_MESSAGE_TYPE && encryptedFunctionOutputParts(candidate.content); - }); -} - -function outboundResponsesBodyCarriesOpaqueBlob(bodyText: string | undefined): boolean { - const input = outboundResponsesInput(bodyText); - if (!input) return false; - return input.some(item => { - if (!item || typeof item !== "object" || Array.isArray(item)) return false; - const candidate = item as { type?: unknown; encrypted_content?: unknown; output?: unknown }; - if ( - typeof candidate.type === "string" - && OPAQUE_RESPONSES_INPUT_TYPES.has(candidate.type) - && typeof candidate.encrypted_content === "string" - && candidate.encrypted_content.length > 0 - ) return true; - if ( - typeof candidate.type === "string" - && FUNCTION_OUTPUT_TYPES.has(candidate.type) - && encryptedFunctionOutputParts(candidate.output) - ) return true; - return candidate.type === AGENT_MESSAGE_TYPE - && encryptedFunctionOutputParts((candidate as { content?: unknown }).content); - }); -} - -function isEncryptedFunctionOutputRejection(bodyText: string): boolean { - if (bodyText.trim() === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) return true; - try { - const payload = JSON.parse(bodyText) as unknown; - if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; - const record = payload as { detail?: unknown; message?: unknown; error?: unknown }; - if (record.detail === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) return true; - if (record.message === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) return true; - if (record.error === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) return true; - return record.error !== null - && typeof record.error === "object" - && !Array.isArray(record.error) - && (record.error as { message?: unknown }).message === ENCRYPTED_FUNCTION_OUTPUT_REJECTION; - } catch { - return false; - } -} - -/** - * #4469: reasoning encrypted_content is minted per caller identity, so replaying it under a - * different caller is rejected with "reasoning `encrypted_content` was not issued to this - * caller". Substring checks tolerate the optional backticks and a leading or trailing - * sentence, while the "was not issued to this caller" anchor plus an encrypted-content or - * reasoning subject keep unrelated invalid_request_error prose from gaining a hidden resend. - */ -function isReasoningBlobCallerMismatchMessage(message: string): boolean { - if (!message.includes("was not issued to this caller")) return false; - return message.includes("encrypted_content") || message.includes("reasoning"); -} - -function isSelfIdentifiedOpaqueBlobRejection(bodyText: string): boolean { - if (isEncryptedFunctionOutputRejection(bodyText)) return true; - try { - if (upstreamErrorMessageFromPayload(JSON.parse(bodyText) as unknown) === ENCRYPTED_FUNCTION_OUTPUT_REJECTION) { - return true; - } - } catch { - /* invalid JSON bodies fall through to the exact nested envelope checks */ - } - try { - const payload = JSON.parse(bodyText) as unknown; - if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; - const record = payload as { code?: unknown; type?: unknown; message?: unknown; error?: unknown }; - - if (record.error && typeof record.error === "object" && !Array.isArray(record.error)) { - const error = record.error as { type?: unknown; code?: unknown; message?: unknown }; - if (error.type === "invalid_request_error") { - if (error.code === "invalid_encrypted_content") return true; - if ( - (error.code === null || error.code === undefined) - && typeof error.message === "string" - && error.message.startsWith("The encrypted content ") - && error.message.endsWith( - " could not be verified. Reason: Encrypted content could not be decrypted or parsed.", - ) - ) return true; - // #4469: the caller-mismatch wording arrives without a dedicated code, so the - // message itself is the identity. It is not gated on code being null — the upstream - // may attach a generic code — because the anchored phrase is already specific. - if (typeof error.message === "string" && isReasoningBlobCallerMismatchMessage(error.message)) { - return true; - } - } - } - - // The flat stream-error envelope carries type/message at the top level rather than under - // an error object; the same anchored identity applies there. - if ( - record.type === "invalid_request_error" - && typeof record.message === "string" - && isReasoningBlobCallerMismatchMessage(record.message) - ) return true; - - if (record.code !== "invalid-argument" || typeof record.error !== "string") return false; - return record.error.startsWith("Could not decode the compaction blob") - || record.error.startsWith("Could not decrypt the provided encrypted_content"); - } catch { - return false; - } -} - -/** - * Whether an upstream Responses 4xx authoritatively rejected opaque replay state. - * - * The outbound-body check is intentional: the inbound transcript may contain a proxy envelope or - * compaction blob that the adapter already lowered, in which case a replay would be byte-identical. - * OpenAI usually exposes a dedicated nested code; ChatGPT also emits one exact code-less - * unverifiable-ciphertext message, and #4469 added the anchored caller-mismatch wording for - * reasoning blobs minted under a different caller. xAI's code is generic, so its two concrete - * decoder error identities are also required. Unrelated error prose must never gain a hidden resend. - */ -export function shouldAttemptOpaqueBlobRecovery(args: { - status: number; - adapterName: string; - outboundBody?: string; - errorBody: string; - alreadyAttempted: boolean; -}): boolean { - const acceptedStatus = (args.status >= 400 && args.status < 500) - || ( - args.status === 502 - && outboundResponsesBodyCarriesEncryptedFunctionOutput(args.outboundBody) - && isEncryptedFunctionOutputRejection(args.errorBody) - ); - return acceptedStatus - && args.adapterName === "openai-responses" - && !args.alreadyAttempted - && outboundResponsesBodyCarriesOpaqueBlob(args.outboundBody) - && isSelfIdentifiedOpaqueBlobRejection(args.errorBody); -} - -/** - * Peek the upstream error body for the reasoning-effort downgrade. Only 400/403 are considered - * and the body must be complete and display-safe, the same contract the other rejection peeks - * use. The match is deliberately narrow: the upstream has to name reasoning effort, so an - * unrelated 400 never triggers a replay. - */ -async function reasoningEffortRejectionText( - response: Response, - alreadyAttempted: boolean, - signal: AbortSignal, -): Promise { - if (alreadyAttempted) return undefined; - if (response.status !== 400 && response.status !== 403) return undefined; - try { - const body = await readBoundedResponseBody(response.clone(), { signal }); - if (!body.displaySafe || body.truncated) return undefined; - return isReasoningEffortRejection(body.text) ? body.text : undefined; - } catch { - return undefined; - } -} - -async function opaqueBlobRejectionBodyForRecovery( - response: Response, - outboundBody: string | undefined, - adapterName: string, - alreadyAttempted: boolean, - signal: AbortSignal, -): Promise { - if ( - isNonReplayableResponse(response) - || response.status < 400 - || (response.status >= 500 && response.status !== 502) - || adapterName !== "openai-responses" - || alreadyAttempted - || !outboundResponsesBodyCarriesOpaqueBlob(outboundBody) - ) return undefined; - try { - const body = await readBoundedResponseBody(response.clone(), { signal }); - return body.displaySafe && !body.truncated ? body.text : undefined; - } catch { - return undefined; - } -} - -/** - * Backoff for the single exact-request replay after a canonical Console upload rejection. - */ -const CONSOLE_GO_UPLOAD_RETRY_DELAY_MS = 800; - -/** - * Peek the upstream error body for the Console Go transient-400 recovery. Only a complete, - * display-safe body may drive a retry decision (same contract as - * opaqueBlobRejectionBodyForRecovery), and reading a clone leaves the original response intact - * for the caller's own error surface when no retry is taken. - */ -async function consoleGoUploadRejectionBody( - response: Response, - alreadyAttempted: boolean, - signal: AbortSignal, -): Promise { - if (isNonReplayableResponse(response) || response.status !== 400 || alreadyAttempted) return undefined; - try { - const body = await readBoundedResponseBody(response.clone(), { signal }); - return body.displaySafe && !body.truncated ? body.text : undefined; - } catch { - return undefined; - } -} - -/** - * Materialize an upstream error body only when the bounded reader observed a complete, - * display-safe payload. Partial timeout and over-limit prefixes are attacker-controlled, - * so callers keep their existing status-only fallback instead. - */ -export async function readDisplaySafeErrorText( - response: Response, - signal: AbortSignal, - fallback: string, -): Promise { - try { - const body = await readBoundedResponseBody(response, { signal }); - return body.displaySafe ? body.text : fallback; - } catch { - // Preserve the former Response.text().catch(fallback) contract. Request-abort - // classification remains owned by the surrounding response pipeline. - return fallback; - } -} - -interface NormalizedUpstreamErrorText { - safeText: string; - message?: string; - type?: string; - code?: string; - cyberPolicy: boolean; -} - -/** - * Extract the structured provider error envelope without making `error.type` authoritative. - * Policy identity comes from the dedicated code (or the legacy message fallback); a credible - * upstream type is only carried through so callers do not erase provider diagnostics. - */ -function normalizeUpstreamErrorText(text: string, fallback: string): NormalizedUpstreamErrorText { - const safeText = redactSecretString(text).slice(0, 500).trim() || fallback; - let message: string | undefined; - let type: string | undefined; - let code: string | undefined; + logCtx: RequestLogContext, + options: HandleResponsesOptions = {}, +): Promise { + const ownsBudget = options.translatorBudget === undefined; + const translatorBudget = options.translatorBudget ?? createTranslatorBudget(); try { - const parsed = JSON.parse(text) as Record; - const response = parsed.response && typeof parsed.response === "object" && !Array.isArray(parsed.response) - ? parsed.response as Record - : undefined; - const candidates = [parsed.error, response?.error, response?.last_error, parsed.last_error, parsed]; - const source = candidates.find((candidate): candidate is Record => { - if (candidate === null || typeof candidate !== "object" || Array.isArray(candidate)) return false; - const record = candidate as Record; - return [record.message, record.type, record.code].some(value => typeof value === "string"); - }); - if (!source) return { safeText, cyberPolicy: isCyberPolicyMessage(safeText) }; - if (typeof source.message === "string" && source.message.trim()) { - message = redactSecretString(source.message.trim()).slice(0, 500); - } - if (typeof source.type === "string" && source.type.trim()) type = source.type.trim(); - if (typeof source.code === "string" && source.code.trim()) code = source.code.trim(); - } catch { - /* non-JSON upstream body — retain the bounded display-safe text */ - } - const cyberPolicy = isCyberPolicyCode(code) || isCyberPolicyMessage(message ?? safeText); - return { safeText, message, type, code, cyberPolicy }; -} - -function prepareOpaqueBlobRecovery(parsed: OcxParsedRequest): void { - parsed._stripReasoningEncryptedContent = true; - const rawBody = parsed._rawBody; - if (!rawBody || typeof rawBody !== "object" || Array.isArray(rawBody)) return; - const input = (rawBody as { input?: unknown }).input; - if (!Array.isArray(input)) return; - const stripEncryptedParts = (parts: unknown[]): unknown[] => { - let changed = false; - const stripped = parts.map(part => { - if ( - part !== null - && typeof part === "object" - && !Array.isArray(part) - && (part as { type?: unknown }).type === "encrypted_content" - && typeof (part as { encrypted_content?: unknown }).encrypted_content === "string" - && (part as { encrypted_content: string }).encrypted_content.length > 0 - ) { - changed = true; - return { type: "input_text", text: "[encrypted content omitted]" }; - } - return part; + const response = await handleResponsesInner(req, config, logCtx, { + ...options, + openAiSidecarAuth: options.openAiSidecarAuth === undefined + ? captureExplicitOpenAiCallerAuth(req.headers, config) : options.openAiSidecarAuth, + nativeCallerAuth: options.nativeCallerAuth === undefined + ? captureExplicitOpenAiCallerAuth(req.headers, config) : options.nativeCallerAuth, + callerDirectAuth: options.callerDirectAuth === undefined + ? captureCallerDirectAuth(req.headers, config) : options.callerDirectAuth, + // Capture before combo replay rebuilds the Request headers; children carry options. + visionDescribeTerminal: options.visionDescribeTerminal === true + || req.headers.get("x-opencodex-vision-describe") === "1", + translatorBudget, + // Created once at genuine ingress; a combo child arrives with the parent's holder already + // in options and must not start a fresh allowance. + sendBudget: options.sendBudget ?? createRequestExecutionBudget(), }); - return changed ? stripped : parts; - }; - const strippedInput = input.map(item => { - if (!item || typeof item !== "object" || Array.isArray(item)) return item; - const record = item as Record; - const type = String(record.type ?? ""); - if (FUNCTION_OUTPUT_TYPES.has(type) && Array.isArray(record.output)) { - const output = stripEncryptedParts(record.output); - return output !== record.output ? { ...record, output } : item; - } - if (type === AGENT_MESSAGE_TYPE && Array.isArray(record.content)) { - const content = stripEncryptedParts(record.content); - return content !== record.content ? { ...record, content } : item; - } - return item; - }); - Object.assign(rawBody, { input: strippedInput }); -} - -function resetStreamedOpaqueBlobLogContext(logCtx: RequestLogContext): void { - delete logCtx.upstreamError; - delete logCtx.terminalHttpStatus; - delete logCtx.terminalErrorCode; - delete logCtx.terminalIncompleteReason; -} - -type OpaqueBlobRecoveryGuard = { attempted: boolean }; - -type OpaqueBlobRecoveryResult = - | { kind: "skipped" } - | { kind: "recovered"; response: Response } - | { kind: "failed"; response: Response }; - -async function attemptOpaqueBlobRecovery( - args: { - response: Response; - outboundBody?: string; - adapterName: string; - parsed: OcxParsedRequest; - guard: OpaqueBlobRecoveryGuard; - signal: AbortSignal; - }, - rebuild: (kind: AttemptRecoveryKind) => Promise, -): Promise { - const errorBody = await opaqueBlobRejectionBodyForRecovery( - args.response, - args.outboundBody, - args.adapterName, - args.guard.attempted, - args.signal, - ); - if (errorBody === undefined || !shouldAttemptOpaqueBlobRecovery({ - status: args.response.status, - adapterName: args.adapterName, - outboundBody: args.outboundBody, - errorBody, - alreadyAttempted: args.guard.attempted, - })) { - return { kind: "skipped" }; - } - - args.guard.attempted = true; - const rejectedScope = args.parsed._reasoningReplayScope - ? { - clientThreadId: args.parsed._reasoningReplayScope.clientThreadId, - ...(args.parsed._reasoningReplayScope.current - ? { current: { ...args.parsed._reasoningReplayScope.current } } - : {}), - } - : undefined; - prepareOpaqueBlobRecovery(args.parsed); - try { void args.response.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - const result = await rebuild("opaque-blob-rejection"); - if (!("failed" in result) && result.ok) { - rememberReasoningReplayOpaqueBlobRejection(rejectedScope); + return ownsBudget ? finalizeOwnedTranslatorBudget(response, translatorBudget) : response; + } catch (error) { + if (ownsBudget) translatorBudget.dispose(); + throw error; } - return "failed" in result - ? { kind: "failed", response: result.failed } - : { kind: "recovered", response: result }; -} - -function nonEmptyProviderApiKey(provider: OcxProviderConfig): string | undefined { - return typeof provider.apiKey === "string" && provider.apiKey.trim().length > 0 - ? provider.apiKey - : undefined; -} - -function isFixedCodexAccount(authCtx: CodexAuthContext): boolean { - return (authCtx.kind === "pool" || authCtx.kind === "main-pool") - && authCtx.fixedAccount === true; } -export function usesCodexForwardPoolAuth( - authCtx: CodexAuthContext, - provider: OcxProviderConfig, -): authCtx is Extract { - return (authCtx.kind === "pool" || authCtx.kind === "main-pool") - && provider.authMode === "forward" && provider.adapter === "openai-responses"; -} - -function codexWsQuotaObserver(authCtx: CodexAuthContext, provider: OcxProviderConfig, modelId?: string): CodexWsQuotaObserver | undefined { - if (!isCanonicalOpenAiForwardProvider(provider) || !usesCodexForwardPoolAuth(authCtx, provider)) return undefined; - const { accountId, writerGeneration } = authCtx; - const credentialGeneration = authCtx.kind === "pool" ? authCtx.generation : undefined; - const mainWriter = authCtx.kind === "main-pool" ? authCtx.mainQuotaWriter : undefined; - return headers => { - if (credentialGeneration !== undefined && !isCodexAccountGenerationLive(accountId, credentialGeneration)) return; - applyCapturedCodexQuota(accountId, headers, writerGeneration, mainWriter, { modelId, poolWriter: authCtx.kind === "pool" ? authCtx.poolQuotaWriter : undefined }); - }; -} - -export function preAuthUpstreamHostCircuitKey( - route: Pick, +export async function handleComboResponses( + req: Request, + rawBody: unknown, + comboId: string, config: OcxConfig, - options: { requireResponsesAdapter?: boolean } = {}, -): string | null { - if ( - normalizeUpstreamHostCircuitThreshold(config.upstreamHostCircuitThreshold) === 0 - || route.codexAccountMode !== "pool" - || route.codexAccountId !== undefined - || route.provider.authMode !== "forward" - || (options.requireResponsesAdapter !== false && route.provider.adapter !== "openai-responses") - ) return null; - return upstreamHostHealthKey(route.providerName, safeOriginLabel(route.provider.baseUrl ?? "")); -} - -export function upstreamHostCircuitOpenResponse(retryAfterSeconds: number): Response { - return formatErrorResponse( - 503, - "upstream_host_circuit_open", - "Provider host is temporarily unavailable", - { retryAfter: String(retryAfterSeconds) }, + logCtx: RequestLogContext, + options: HandleResponsesOptions & { translatorBudget: TranslatorBudget }, +): Promise { + return executeComboResponses( + req, + rawBody, + comboId, + config, + logCtx, + options, + requestDispatchers, ); } -function normalizeCodexUnsupportedModelDetail(value: string): string { - return value.trim().replace(/\s+/gu, " ").toLocaleLowerCase("en-US"); -} - -function isAllowListedCodexAccountModel400( - status: number, - bodyText: string, - modelId: string, -): boolean { - if (status !== 400) return false; - try { - const payload = JSON.parse(bodyText) as unknown; - if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; - const detail = (payload as { detail?: unknown }).detail; - if (typeof detail !== "string") return false; - const expected = `The '${modelId}' model is not supported when using Codex with a ChatGPT account.`; - return normalizeCodexUnsupportedModelDetail(detail) - === normalizeCodexUnsupportedModelDetail(expected); - } catch { - return false; - } -} - -async function shouldRetryCodexPoolAccountModel400( - response: Response, - modelId: string, - signal?: AbortSignal, -): Promise { - if (response.status !== 400) return false; - try { - const body = await readBoundedResponseBody(response.clone(), { signal }); - return body.displaySafe - && !body.truncated - && isAllowListedCodexAccountModel400(response.status, body.text, modelId); - } catch { - return false; - } -} - -/** Pre-stream quota/billing rejections that warrant one alternate-account attempt (#584). */ -function codexQuotaFailureMessage(body: string): string | undefined { - try { - const payload = JSON.parse(body) as unknown; - const canonical = upstreamErrorMessageFromPayload(payload); - if (canonical !== undefined) return canonical; - if (typeof payload === "string") return payload; - if (!payload || typeof payload !== "object" || Array.isArray(payload)) return undefined; - const record = payload as Record; - if (typeof record.message === "string") return record.message; - return typeof record.error === "string" ? record.error : undefined; - } catch { - // Plain-text gateways remain supported. Valid JSON is inspected only at recognized - // message fields so echoed request content elsewhere cannot trigger account cooldown. - return body; - } -} - -export async function shouldRetryCodexPoolAccountQuota( - response: Response, - signal?: AbortSignal, -): Promise { - // A post-send WebSocket gateway status must not become a second account's send; the - // body carries no quota evidence either, but the marker is the contract, not the prose. - if (isNonReplayableResponse(response)) return false; - if (response.status === 402 || response.status === 429) return true; - if (response.status < 500 || response.status >= 600) return false; - try { - // Reject malformed UTF-8 instead of matching quota words around replacement characters. - const body = await readBoundedResponseBody(response.clone(), { signal, fatalUtf8: true }); - const message = body.displaySafe && !body.truncated - ? codexQuotaFailureMessage(body.text) - : undefined; - return message !== undefined - && isRateLimitOrQuotaFailureMessage(message); - } catch { - return false; - } -} - -/** - * A pre-stream upstream 5xx another Codex account may still be able to serve. - * - * `server_is_overloaded` is the shape this exists for. The ChatGPT backend refuses in a few - * hundred milliseconds, the body carries no quota evidence, and nothing in that exchange is - * account health — so the pool keeps choosing the same account and every request fails on it - * while the other accounts sit idle. That is what an operator sees as the pool refusing to move. - * - * The status stays exactly as upstream sent it. `classifyCodexUpstreamOutcome` maps 5xx to the - * transient class, so the account earns an ordinary failure streak and `upstreamFailoverThreshold` - * decides when it is soft-avoided, rather than a quota cooldown it never earned. - * - * Deliberately narrow. {@link isNonReplayableResponse} still refuses: a post-send WebSocket - * gateway status means the body already reached the origin, so sending it from a second account - * could duplicate a turn the origin may still be running. A 5xx whose body confirms quota is not - * routed here either — {@link shouldRetryCodexPoolAccountQuota} classifies that one first and - * carries the cooldown with it. - */ -export function shouldRetryCodexPoolAccountTransient(response: Response): boolean { - return !isNonReplayableResponse(response) && isTransientUpstreamStatus(response.status); -} - -interface CodexPoolAccountRetryArgs { - /** Sanitized caller input, before any selected Pool credential was materialized. */ - callerAuthHeaders: Headers; - config: OcxConfig; - route: { providerName: string; modelId: string; provider: OcxProviderConfig }; - parsed: OcxParsedRequest; - logCtx: RequestLogContext; - options: { - admission?: DataPlaneAdmission; - codexAuthPolicy?: CodexAuthPolicyConfig; - visionDescribeTerminal?: boolean; - abortSignal?: AbortSignal; - onCodexAuthContextResolved?: (ctx: CodexAuthContext) => void; - deferCodexResetDerivedCooldown?: boolean; - // Narrowed subset of HandleResponsesOptions: the retry rebuilds the adapter, so it - // needs the inbound scope or the retry could land on a different wire than the - // first attempt. - inboundWire?: InboundWire; - codexWsRuntimeIdentity?: BunRuntimeGateInput; - translatorBudget: TranslatorBudget; - turnAdmissionLease?: AdmissionLease; - resolveCodexModelEntitlements?: typeof resolveCodexModelEntitlements; - /** The logical request's execution budget: the account move is its fourth send. */ - sendBudget?: TransientSendBudget; - /** Root workflow this turn belongs to, so the move is charged there as well. */ - workflowRootId?: string; - }; - firstAuthCtx: Extract; - firstResponse: Response; - outcomeStatus: number; - /** - * Forbid resolving a DIFFERENT account for this retry. - * - * Set when a stored Pool 401 already spent this logical request's account budget on its own - * refresh and replay. The same-account gated-model retry above stays available, because it - * sends to the account that was already paying; only the alternate-account resolution below is - * out of budget. - */ - sameAccountOnly?: boolean; - upstream: AbortController; - connectMs: number; - passthroughEstimate?: number; - stream: boolean; - onResponse?: ( - response: Response, - authCtx: CodexAuthContext, - request: Awaited["buildRequest"]>>, - ) => void; -} - -type CodexPoolAccountRetryResult = - | { - kind: "retried"; - authCtx: CodexAuthContext; - request: Awaited["buildRequest"]>>; - upstreamResponse: Response; - selectedForwardHeaders: Headers; - } - | { kind: "no-alternate" } - | { - kind: "transport"; - error: unknown; - authCtx: CodexAuthContext; - }; - -/** Keep retry-stage entitlement snapshots inside the native-main selection fence. */ -async function resolveCodexRetryModelEntitlements( +/** Compose request phases while retaining the original admission-finally ownership. */ +async function handleResponsesInner( + req: Request, config: OcxConfig, - resolver: typeof resolveCodexModelEntitlements, - turnAdmissionLease?: AdmissionLease, -): Promise>> { - // The initial auth selection has already released its admission before the first - // response arrives. Re-enter for every refresh so profile switching cannot overlap - // credential discovery, and omit main entirely when a drain or recovery owns it. - const selectionAdmission = codexAccountSelectionForTurn(turnAdmissionLease)?.(); - const nativeMainReadsForbidden = isNativeMainTrafficBlocked() - || selectionAdmission?.mainProfileDraining === true; - try { - return await resolver(config, { - excludeAccountIds: nativeMainReadsForbidden - ? new Set([MAIN_CODEX_ACCOUNT_ID]) - : undefined, - }); - } finally { - selectionAdmission?.release(); - } -} - -const CODEX_ACCOUNT_GATED_CANONICAL_WIRE_MODELS: ReadonlyMap = new Map([ - // The authenticated catalog currently advertises Daybreak Blue, while successful responses - // identify the serving model as gpt-5.6-sol. Sending the selector itself is shard-dependent: - // live traffic can receive the exact unsupported-model 400 repeatedly from the same entitled - // account. Keep Daybreak as the admission/catalog identity, but use the stable serving id on - // the credential-bearing wire after entitlement selection has completed. - ["gpt-daybreak-blue-latest", "gpt-5.6-sol"], -]); - -export function codexAccountGatedCanonicalWireModel(modelId: string): string | undefined { - const exact = CODEX_ACCOUNT_GATED_CANONICAL_WIRE_MODELS.get(modelId); - if (exact) return exact; - for (const [selector, wireModel] of CODEX_ACCOUNT_GATED_CANONICAL_WIRE_MODELS) { - if (slugsEquivalent(modelId, selector)) return wireModel; - } - return undefined; -} - -function applyCodexAccountGatedWireNormalization(parsed: OcxParsedRequest, route: RouteResult, logCtx?: RequestLogContext): void { - if (!isCanonicalOpenAiForwardProvider(route.provider)) return; - const wireModel = codexAccountGatedCanonicalWireModel(route.modelId); - if (!wireModel) return; - - if (logCtx) { - logCtx.preserveResolvedModelFromRoute = true; - delete logCtx.resolvedModel; - } - parsed.modelId = wireModel; - if (!parsed._rawBody || typeof parsed._rawBody !== "object") return; - const raw = parsed._rawBody as Record; - raw.model = wireModel; - // Daybreak's authenticated catalog does not advertise retention support, and the upstream - // rejects this optional Codex hint before model execution. Removing it preserves request - // semantics while avoiding an otherwise terminal pre-stream 400. - delete raw.prompt_cache_retention; -} - -/** - * Workspace-denial evidence for a 403, read from the upstream body. - * - * #1789: a valid K12 credential gets 403 `codex_workspace_access_denied` on a routed prompt. - * Without this the account is quarantined for reauthentication, which cannot fix a workspace - * grant and loops forever. Fails closed: an unreadable body keeps the historical handling. - */ -async function codexDenialOutcomeMeta(response: Response): Promise<{ denial?: "workspace" | "entitlement" }> { - if (response.status !== 403) return {}; - const { classifyCodexPreStreamRejection } = await import("../../codex/quota-rejection"); - const rejection = await classifyCodexPreStreamRejection(response); - return rejection.denial ? { denial: rejection.denial } : {}; -} - -function codexQuotaOutcomeMeta(response: Response): { - retryAfter: string | null; - resetAt: string[]; -} { - return { - retryAfter: response.headers.get("retry-after"), - resetAt: [ - response.headers.get("x-codex-primary-reset-at"), - response.headers.get("x-codex-secondary-reset-at"), - response.headers.get("x-codex-tertiary-reset-at"), - ].filter((value): value is string => !!value), - }; -} - -/** - * A reset timestamp describes a quota window, not an explicit instruction to - * stop using the whole account. A combo may therefore try a later model in the - * same request, while Retry-After and headerless quota failures remain blocking. - */ -function shouldDeferCodexResetDerivedCooldown(response: Response, enabled?: boolean): boolean { - return enabled === true - && (response.status === 429 || response.status === 402) - && computeQuotaCooldown(codexQuotaOutcomeMeta(response)).source === "reset-derived"; -} - -/** - * One bounded alternate-account retry for Codex pool auth. Used for allow-listed - * model-400 and for pre-stream 429/402 quota failures (#584). - */ -async function retryCodexPoolOnAlternateAccount( - args: CodexPoolAccountRetryArgs, -): Promise { - const { - callerAuthHeaders, config, route, parsed, logCtx, options, firstAuthCtx, firstResponse, - outcomeStatus, upstream, connectMs, passthroughEstimate, stream, - } = args; - const inboundWire = options.inboundWire ?? "responses"; - const entitlementResolver = options.resolveCodexModelEntitlements ?? resolveCodexModelEntitlements; - let retryAuthCtx: CodexAuthContext | undefined; - // A transient 5xx must record even when this request cannot move: the ordinary terminal - // recorder only fires for an OK event-stream body, so a pre-stream refusal would otherwise - // leave the account looking healthy no matter how many times it refused, and the pool would - // keep handing it the next request. - const recordUnmovedTransientOutcome = (): void => { - if (!isTransientUpstreamStatus(outcomeStatus)) return; - recordCodexUpstreamOutcome(config, firstAuthCtx.accountId, outcomeStatus, { - threadId: firstAuthCtx.affinityKey, - fixedAccount: firstAuthCtx.fixedAccount, - modelId: route.modelId, - probeLeaseId: codexProbeLeaseId(firstAuthCtx), - probeQuotaScope: codexProbeQuotaScope(firstAuthCtx), - writerGeneration: firstAuthCtx.writerGeneration, - }); - }; - if (outcomeStatus === 400 && ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(route.modelId)) { - invalidateCodexModelEntitlementsForAccount(firstAuthCtx.accountId); - let refreshed; - try { - refreshed = await resolveCodexRetryModelEntitlements( - config, - entitlementResolver, - options.turnAdmissionLease, - ); - } catch (error) { - await firstResponse.body?.cancel().catch(() => undefined); - releaseCodexAuthContextProbeLease(firstAuthCtx); - throw error; - } - if (entitledCodexAccountIdsForModel(refreshed, route.modelId)?.has(firstAuthCtx.accountId)) { - // The authenticated roster still grants this exact model. Retry on the same account: - // upstream shards can briefly disagree during a gated-model rollout, but a pre-stream 400 - // proves no output was committed and keeps this replay bounded. - retryAuthCtx = firstAuthCtx; - } - } - // Exact account selectors may retry the same confirmed account above, but must never resolve - // an alternate. Quota failures and a refreshed entitlement miss remain terminal. - if (!retryAuthCtx && (firstAuthCtx.fixedAccount || args.sameAccountOnly === true)) { - recordUnmovedTransientOutcome(); - return { kind: "no-alternate" }; - } - // An account move is the guarded profile's fourth send and draws the single shared - // final-recovery reserve. Nothing bounded it per request before: `excludeAccountId` excludes - // only the account that just failed, and the caller's recovery loop can return here after the - // alternate fails too, so one request could walk the pool an account at a time. The permit is - // consumed immediately before the physical send, so a resolution that finds no alternate - // costs nothing. - const executionBudget = isRequestExecutionBudget(args.options.sendBudget) - ? args.options.sendBudget - : undefined; - let accountMovePermit: SingleUseDispatchPermit | undefined; - if (!retryAuthCtx && executionBudget) { - const decision = executionBudget.reserveDispatch({ - sendClass: "account-failover", - targetKey: `${route.providerName}|${route.modelId}|alternate-account`, - }); - if (!decision.allowed) { - recordUnmovedTransientOutcome(); - return { kind: "no-alternate" }; - } - accountMovePermit = decision.permit; - } - try { - retryAuthCtx ??= await resolveCodexAuthContext( - callerAuthHeaders, - config, - "pool", - { - excludeAccountId: firstAuthCtx.accountId, - admission: options.admission, - codexAuthPolicy: options.codexAuthPolicy, - modelId: route.modelId, - requestScopedMainCredential: hasForwardableCodexBearer(callerAuthHeaders, config), - beginCodexAccountSelection: codexAccountSelectionForTurn(options.turnAdmissionLease), - resolveCodexModelEntitlements: entitlementResolver, - }, - ); - } catch (error) { - const unexpectedRetryError = - !(error instanceof CodexPoolAuthenticationError) - && !(error instanceof CodexAuthContextError) - && !(error instanceof CodexAccountCooldownError) - && !(error instanceof CodexMainProfileDrainingError); - if (unexpectedRetryError) { - // The reservation is the charge now, so an abandoned move has to hand its send back. - accountMovePermit?.release(); - await firstResponse.body?.cancel().catch(() => undefined); - releaseCodexAuthContextProbeLease(firstAuthCtx); - throw error; - } - } - // A validated request-owned main bearer is a real alternate when the failed credential was a - // stored Pool account. It has no Pool account id to promote or cool, but it can own this one - // bounded replay. The resolver already refuses it when main itself is the excluded credential. - if ( - retryAuthCtx?.kind !== "pool" - && retryAuthCtx?.kind !== "main-pool" - && retryAuthCtx?.kind !== "main" - ) { - // A body-confirmed quota response may arrive under HTTP 5xx. Without an alternate, - // the ordinary terminal recorder sees only that wire status and would misclassify it - // as transient, leaving the exhausted account immediately selectable next turn. - if (outcomeStatus !== firstResponse.status && (outcomeStatus === 429 || outcomeStatus === 402)) { - recordCodexUpstreamOutcome(config, firstAuthCtx.accountId, outcomeStatus, { - ...codexQuotaOutcomeMeta(firstResponse), - threadId: firstAuthCtx.affinityKey, - modelId: route.modelId, - probeLeaseId: codexProbeLeaseId(firstAuthCtx), - probeQuotaScope: codexProbeQuotaScope(firstAuthCtx), - writerGeneration: firstAuthCtx.writerGeneration, - }); - } - // No usable alternate was resolved, so the reserved move never becomes a send. - accountMovePermit?.release(); - recordUnmovedTransientOutcome(); - return { kind: "no-alternate" }; - } - - const quotaMeta = { ...codexQuotaOutcomeMeta(firstResponse), ...(await codexDenialOutcomeMeta(firstResponse)) }; - if (outcomeStatus === 429 || outcomeStatus === 402) { - const { applyAccountQuotaFromUpstreamHeaders } = await import("../../codex/auth-api"); - applyAccountQuotaFromUpstreamHeaders( - firstAuthCtx.accountId, - firstResponse.headers, - firstAuthCtx.writerGeneration, - firstAuthCtx.kind === "main-pool" ? firstAuthCtx.mainQuotaWriter : undefined, - { modelId: route.modelId, poolWriter: firstAuthCtx.kind === "pool" ? firstAuthCtx.poolQuotaWriter : undefined }, - ); - } - const deferFirstOutcome = shouldDeferCodexResetDerivedCooldown( - firstResponse, - options.deferCodexResetDerivedCooldown, - ); - const recordFirstOutcome = (): void => { - recordCodexUpstreamOutcome(config, firstAuthCtx.accountId, outcomeStatus, { - ...quotaMeta, - threadId: firstAuthCtx.affinityKey, - modelId: route.modelId, - probeLeaseId: codexProbeLeaseId(firstAuthCtx), - probeQuotaScope: codexProbeQuotaScope(firstAuthCtx), - writerGeneration: firstAuthCtx.writerGeneration, - // Retry already advanced the RR ring via excludeAccountId — reuse for promotion. - ...(retryAuthCtx.accountId ? { promoteAccountId: retryAuthCtx.accountId } : {}), - }); - }; - // Only a combo reset-derived outcome is deferred. Retry-After, defaults, and - // ordinary requests must block the first account before the alternate send. - if (!deferFirstOutcome) recordFirstOutcome(); - const retryHeaders = headersForCodexAuthContext(callerAuthHeaders, retryAuthCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission); - const retryProvider = applyCodexAuthContextToProvider( - stripCodexRuntimeProviderFields(route.provider), - retryAuthCtx, - "pool", - ); - const retryAdapter = resolveAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, retryProvider, inboundWire), - config.cacheRetention, - route.providerName, - ); - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: retryProvider, - adapterName: retryAdapter.name, - codexAuthContext: retryAuthCtx, - forwardHeaders: retryHeaders, - }); - { - const binding = conversationStateBindingFromAuth( - retryAuthCtx, - firstAuthCtx.kind === "pool" || firstAuthCtx.kind === "main-pool" - ? firstAuthCtx.affinityKey - : undefined, + logCtx: RequestLogContext, + options: HandleResponsesOptions & { translatorBudget: TranslatorBudget }, +): Promise { + const requestContext: ResponsesRequestContext = { req, config, logCtx, options }; + const admissionState: ResponsesAdmissionState = { + pendingHostAdmissionLease: null, + authCtx: { kind: "main", accountId: null }, + }; + try { + const requestState = await prepareResponsesRequest(requestContext, admissionState, requestDispatchers); + if (requestState instanceof Response) return requestState; + const transportState = await prepareResponsesTransport(requestContext, admissionState, requestState); + if (transportState instanceof Response) return transportState; + const sidecarState = await prepareResponsesSidecarAuth(requestContext, requestState, transportState); + if (sidecarState instanceof Response) return sidecarState; + const responseEffects = createResponsesEffects( + requestContext, + admissionState, + requestState, + sidecarState, + ); + const sendBudgetState = createResponsesSendBudget(requestContext); + if (sendBudgetState instanceof Response) return sendBudgetState; + if ("passthrough" in transportState.adapter && transportState.adapter.passthrough && !sidecarState.routedCompaction) { + return await executePassthroughResponse( + requestContext, + admissionState, + requestState, + transportState, + sidecarState, + responseEffects, + sendBudgetState, + ); + } + const sidecarPlans = await executeResponsesSidecars( + requestContext, + requestState, + transportState, + sidecarState, + responseEffects, + sendBudgetState, + ); + if (sidecarPlans instanceof Response) return sidecarPlans; + const completionPolicy = createResponsesCompletionPolicy(requestContext, sidecarState); + if (transportState.adapter.runTurn) return await executeResponsesRunTurn( + requestContext, + admissionState, + requestState, + transportState, + sidecarState, + responseEffects, + sendBudgetState, + completionPolicy, + ); + const adapterExchange = await prepareAdapterExchange( + requestContext, + admissionState, + requestState, + transportState, + responseEffects, + sendBudgetState, + ); + if (adapterExchange instanceof Response) return adapterExchange; + const continuationState = createAdapterContinuations( + requestContext, + requestState, + transportState, + sidecarState, + sendBudgetState, + adapterExchange, + ); + return await deliverAdapterResponse( + requestContext, + requestState, + transportState, + sidecarState, + responseEffects, + completionPolicy, + adapterExchange, + continuationState, ); - if (binding) { - applyAccountChangeConversationStateScrub({ - body: parsed._rawBody, - parsed, - bindingKey: binding.bindingKey, - servingAccountId: binding.accountId, - priorAccountId: firstAuthCtx.accountId, - logCtx, - }); - } - } - const request = await retryAdapter.buildRequest(parsed, { - headers: retryHeaders, - translatorBudget: options.translatorBudget, - }); - recordAdapterReasoning(logCtx, request); - recordAdapterTier(logCtx, request); - - await firstResponse.body?.cancel().catch(() => undefined); - options.onCodexAuthContextResolved?.(retryAuthCtx); - route.provider = retryProvider; - logCtx.provider = formatCodexProviderForLog( - route.providerName, - retryAuthCtx.accountId, - config, - ); - logCtx.accountLogLabel = codexAuthContextLogLabel(retryAuthCtx, config); - sealRequestAttemptIdentity( - logCtx.activeAttempt, - logCtx.provider, - retryAdapter.name, - logCtx.accountLogLabel, - ); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, retryAdapter.name); - - const retrySameConfirmedAccount = outcomeStatus === 400 - && ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(route.modelId) - && retryAuthCtx.accountId === firstAuthCtx.accountId; - // Live Daybreak traffic has produced long runs of unsupported-model 400s from different - // upstream shards even while the authenticated roster continues to grant the model. Permit - // seven additional same-account sends (eight total including the original), re-checking the - // exact allow-listed body and fresh entitlement before every later send. Alternate-account and - // quota recovery retain their historical one-send bound. - // - // Two different bounds, and the effective one is the smaller. `maxRetrySends` answers "how - // many times is it worth re-asking THIS account for a model its roster still grants"; the - // shared budget answers "how many times may this LOGICAL REQUEST reach upstream in total, - // across every layer that can re-send". A ladder of eight layered on sends the request had - // already made is exactly the per-request multiplication #4546 is about, so the ladder is - // capped at what the request has left. The floor of one keeps the single retry this function - // was called to make -- the move already paid for itself with its own permit -- and each rung - // past the first reserves its own send below, so a refusal stops the ladder with the last - // upstream answer intact. - // The ladder replays to the SAME account, so it must reserve under the same target key the - // other legs use. Folding the account id in made every rung read as a target change, which - // spent the one cross-account slot a real move needs on a same-account replay. - const ladderTargetKey = `${route.providerName}|${route.modelId}`; - // The ladder keeps its OWN bound rather than drawing on what the request has left. Clamping it - // to the shared total looked right and broke a working, pinned path: #2097 fixes this recovery - // at eight same-account dispatches (tests/server/server-auth.test.ts), and a request that has - // already spent sends would silently stop short of it. Reconciling an eight-send same-account - // ladder with a four-send request total is a policy decision, not a clamp to add in passing. - // What this diff does fix is that the rungs are now CHARGED instead of free. - const maxRetrySends = retrySameConfirmedAccount ? 7 : 1; - let retrySendCount = 0; - let upstreamResponse: Response; - try { - while (true) { - // The same-account gated-model 400 ladder below keeps its own `maxRetrySends` bound and - // does not take the reserve again; only the move itself does. - if (accountMovePermit) { - const charged = accountMovePermit.use(); - accountMovePermit = undefined; - if (!charged) { - recordUnmovedTransientOutcome(); - return { kind: "no-alternate" }; - } - // The move is a physical send like any other, so the root workflow is charged too. - chargeWorkflowSends(args.options.workflowRootId, 1); - } - noteAttemptSend(logCtx.activeAttempt, passthroughEstimate); - try { - upstreamResponse = await fetchWithHeaderTimeout( - request.url, - { - method: request.method, - headers: request.headers, - body: request.body, - }, - upstream.signal, - connectMs, - stream, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - providerName: route.providerName, - modelId: route.modelId, - onCodexWsQuota: codexWsQuotaObserver(retryAuthCtx, route.provider, route.modelId), - beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) - ? createCodexReserveDispatchGuard(retryAuthCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, - }), - // Credential-bearing forward send: never follow a redirect into a - // dead-host rejection after the credential was seen (#914). - route.provider.authMode === "forward", - ); - } catch (error) { - // Only the forward send is a transport boundary. Entitlement resolver throws below are - // deliberately outside this catch so programming errors retain their original path. - return { kind: "transport", error, authCtx: retryAuthCtx }; - } - retrySendCount += 1; - args.onResponse?.(upstreamResponse, retryAuthCtx, request); - if (!retrySameConfirmedAccount || retrySendCount >= maxRetrySends) break; - // Caller-owned main is an alternate-account replay and can never enter the bounded - // same-stored-account 400 loop above. Keep that invariant explicit for the account-id reads. - if (retryAuthCtx.kind === "main") break; - if (!await shouldRetryCodexPoolAccountModel400( - upstreamResponse, - route.modelId, - options.abortSignal, - )) break; - invalidateCodexModelEntitlementsForAccount(retryAuthCtx.accountId); - let refreshed: Awaited>; - try { - refreshed = await resolveCodexRetryModelEntitlements( - config, - entitlementResolver, - options.turnAdmissionLease, - ); - } catch (error) { - await upstreamResponse.body?.cancel().catch(() => undefined); - await firstResponse.body?.cancel().catch(() => undefined); - releaseCodexAuthContextProbeLease(firstAuthCtx); - releaseCodexAuthContextProbeLease(retryAuthCtx); - throw error; - } - if (!entitledCodexAccountIdsForModel(refreshed, route.modelId)?.has(retryAuthCtx.accountId)) break; - // The next rung is another physical send of this logical request: a same-account, - // same-target replay, charged as an ordinary transient send rather than as a move. - // Reserved here, immediately before looping back, so a refusal stops the ladder with the - // last upstream 400 intact instead of spending a send it cannot make. - // Every rung is CHARGED, and a refusal does not end the ladder. That asymmetry is - // deliberate and it is the one place the shared cap yields. This is a same-account, - // same-target replay of a model-gating 400 whose own bound is eight dispatches, pinned by - // #2097; letting a spent request budget cut it to four would break a recovery that works - // today, which is precisely the mistake 040_send_budget.md warns a flat ceiling makes. - // The request total still governs everything that changes target or credential. - if (executionBudget) { - const rung = executionBudget.reserveDispatch({ - sendClass: "transient", - targetKey: ladderTargetKey, - }); - if (rung.allowed) rung.permit.use(); - chargeWorkflowSends(args.options.workflowRootId, 1); - } - await upstreamResponse.body?.cancel().catch(() => undefined); - } - } finally { - request.releaseBodyObservation?.(); - } - // A real HTTP response proves the host was reached (#914). - const retryHostKey = upstreamHostHealthKey(route.providerName, safeOriginLabel(request.url)); - if (normalizeUpstreamHostCircuitThreshold(config.upstreamHostCircuitThreshold) > 0) { - resetUpstreamHostHealth(retryHostKey, null); - } else { - resetUpstreamHostHealth(retryHostKey); - } - if (deferFirstOutcome && upstreamResponse.ok) { - // Deferral keeps the first account eligible for a later combo model while an - // alternate attempt is still fallible. Commit its quota outcome only once the - // alternate account returns a successful HTTP response; otherwise the combo may - // still need the first account for its next target. - recordFirstOutcome(); - } - return { - kind: "retried", - authCtx: retryAuthCtx, - request, - upstreamResponse, - selectedForwardHeaders: retryHeaders, - }; -} - - - -export function codexForwardTerminalOutcomeRecorder( - config: OcxConfig, - authCtx: CodexAuthContext, - provider: OcxProviderConfig, - modelId?: string, - logCtx?: RequestLogContext, -): ((status: ResponsesTerminalStatus, httpStatusOverride?: number) => void) | undefined { - if (!usesCodexForwardPoolAuth(authCtx, provider)) return undefined; - return (status, httpStatusOverride) => { - const quotaStatus = [httpStatusOverride, logCtx?.terminalHttpStatus] - .find(value => value === 429 || value === 402); - if (status === "incomplete" && quotaStatus === undefined) { - // Normal limit/content-filter/stall terminal — the account served the - // request. Don't penalize account health; record success to clear any - // prior soft-avoid so a healthy account isn't stuck avoided. - recordCodexUpstreamOutcome(config, authCtx.accountId, 200, { - threadId: authCtx.affinityKey, - fixedAccount: authCtx.fixedAccount, - modelId, - probeLeaseId: codexProbeLeaseId(authCtx), - probeQuotaScope: codexProbeQuotaScope(authCtx), - writerGeneration: authCtx.writerGeneration, - }); - return; - } - // status === "completed" or "failed": use the semantic HTTP status derived - // from the terminal SSE error payload (httpStatusFromTerminalError in - // request-log inspection) instead of collapsing every non-completed terminal - // to 502. A 400 invalid_request_error must not soft-avoid the account or - // rebind threads — only genuine transport/5xx failures should trigger - // transient health recording. - // httpStatusOverride: the combo WS path inspects SSE payloads into the parent - // logCtx, but this recorder closes over the child logCtx. The caller passes - // the parent's terminalHttpStatus so the semantic status is not lost. - const outcome = status === "completed" - ? 200 - : (quotaStatus ?? httpStatusOverride ?? logCtx?.terminalHttpStatus ?? 502); - recordCodexUpstreamOutcome(config, authCtx.accountId, outcome, { - threadId: authCtx.affinityKey, - fixedAccount: authCtx.fixedAccount, - modelId, - probeLeaseId: codexProbeLeaseId(authCtx), - probeQuotaScope: codexProbeQuotaScope(authCtx), - writerGeneration: authCtx.writerGeneration, - // A mid-stream terminal can carry a semantic 401 long after the credential was - // replaced. It is never replayed — the client already saw output — but it must - // not retire the replacement either (#2887). - ...(authCtx.kind === "pool" ? { credentialGeneration: authCtx.generation } : {}), - }); - }; -} - - - -export function decodeRequestErrorResponse(err: unknown, label: string): Response { - if (isTranslatorBudgetExceededError(err)) { - return formatErrorResponse(413, "request_too_large", "request translation buffer exceeded the safe limit", { - code: "translation_buffer_limit", - }); - } - if (err instanceof UnsupportedContentEncodingError) { - return formatErrorResponse(415, "invalid_request_error", err.message); - } - if (err instanceof DecompressedBodyTooLargeError) { - return formatErrorResponse(413, "inbound_body_too_large", describeInboundBodyRefusal(err)); - } - console.warn(`[${label}] request body decode/parse failed: ${err instanceof Error ? `${err.name}: ${err.message}` : String(err)}`); - return formatErrorResponse(400, "invalid_request_error", "Invalid JSON body"); -} - - - -export function comboUnavailableResponse( - message: string, - options?: { retryAfter?: string | null }, -): Response { - const headers = new Headers({ "Content-Type": "application/json" }); - const retryAfter = options?.retryAfter?.trim(); - if (retryAfter && retryAfter.length > 0 && retryAfter.length <= 128) { - headers.set("Retry-After", retryAfter); - } - return new Response( - JSON.stringify({ - error: { message, type: "server_error", code: "combo_unavailable" }, - }), - { status: 503, headers }, - ); -} - -function comboUnavailable(comboId: string, now = Date.now()): Response { - return comboUnavailableResponse(`No available targets for combo: ${comboId}`, { - retryAfter: comboCooldownRetryAfterSeconds(comboId, now), - }); -} - - - -export interface ConsumedComboFailure { - response: Response; - classificationText: string; - /** Structured upstream `error.code` when present in the failure body. */ - upstreamCode?: string; - /** Valid numeric/date value used only for cooldown calculation. */ - retryAfter?: string; - /** Upstream Codex quota-window reset timestamps used for combo cooldowns. */ - resetAt?: string[]; - /** Reserved for 040 usage attribution without adding another body read. */ - usage?: OcxUsage; -} - - - -export interface HandleResponsesOptions { - /** Internal Claude replay identity; consumed only by the final canonical Go transport. */ - claudeGoAffinity?: { sessionLane?: string }; - /** Validated Claude metadata identity; projected only into final canonical attempt headers. */ - claudeNativeSessionId?: string; - /** Original live policy owner; separate from caller-specific routing/sidecar snapshots. */ - codexAuthPolicy?: CodexAuthPolicyConfig; - turnAdmissionLease?: AdmissionLease; - /** - * How the caller proved data-plane admission (#1686). - * - * A bearer-presented admission secret is one of OUR OWN secrets, so a Direct turn must - * SUBSTITUTE the stored main credential rather than forward it. Without this fact at the - * decision point, Direct cannot tell an admission bearer from the user own ChatGPT bearer, - * which is why it refused the whole env_key flow instead of serving it. - */ - admission?: DataPlaneAdmission; - /** Called at most once after the complete client body is read and accepted for dispatch. */ - onRequestBodyRead?: () => void; - forceEmptyResponseId?: boolean; - abortSignal?: AbortSignal; - /** One-shot TTFT callback: first non-empty model output observed (WP4). */ - onFirstOutput?: () => void; - onCodexAuthContextResolved?: (context: CodexAuthContext | undefined) => void; - /** Internal deterministic seam for account-gated native fallback tests. */ - resolveCodexModelEntitlements?: typeof resolveCodexModelEntitlements; - /** Internal: validated final client-visible model, after completed terminal success only. */ - onResponseComplete?: (model: string) => void; - recordTerminalOutcomes?: boolean; - setTerminalOutcomeRecorder?: (recorder: ((status: ResponsesTerminalStatus, httpStatusOverride?: number) => void) | undefined) => void; - onNativePassthroughTerminal?: (status: ResponsesTerminalStatus) => void; - onNativePassthroughCancel?: () => void; - /** Internal deterministic clock/timer seam for provider terminal repair. */ - responsesTerminalRepairScheduler?: ResponsesTerminalRepairScheduler; - /** Internal deterministic runtime-identity seam for Codex upstream WS selection tests. */ - codexWsRuntimeIdentity?: BunRuntimeGateInput; - /** Test seam for native main refresh without live OAuth traffic. */ - nativeMainRefreshDependencies?: NativeMainRefreshDependencies; - /** - * When true, body `prompt_cache_key` is a Claude Desktop shared cache cohort - * (system/tools hash), not a per-session id — do not use it for Anthropic pool affinity. - */ - promptCacheKeyIsSharedCohort?: boolean; - /** - * Wire protocol the ORIGINAL client spoke. The Chat and Anthropic surfaces translate - * their body into a Responses shape and replay through this function, so without an - * explicit value the replay would look like a native Responses request and an - * inbound-scoped registry wire default would fire for a client that never asked for - * it. Omitted means a genuine Responses inbound. - */ - inboundWire?: InboundWire; - /** Internal transport identity for route-scoped upstream compatibility policy. */ - inboundTransport?: "websocket"; - /** - * Claude replay may add native-main auth so OpenAI sidecars remain available. - * Strip only that internal credential when the final route is a noncanonical - * forward/caller-auth destination; final routing can differ from Claude's preflight route. - */ - stripClaudeMainAuthForNoncanonicalForward?: boolean; - /** In-memory credential proven by Claude's native-main turn claim; never persist or log. */ - trustedClaudeMainAuth?: { authorization: string; chatgptAccountId?: string }; - /** Sidecar-only auth captured before route changes; null means no usable original pair. */ - openAiSidecarAuth?: ExplicitOpenAiCallerAuth | null; - /** Internal Chat bridge permission to obtain claimed stored auth only for a final Direct sidecar. */ - allowStoredOpenAiSidecarAuth?: boolean; - /** Original caller-owned native pair; separate from any claimed sidecar enrichment. */ - nativeCallerAuth?: ExplicitOpenAiCallerAuth | null; - /** Caller Direct credential under Direct\'s own predicate; restored only for the canonical OpenAI final route. */ - callerDirectAuth?: CallerDirectAuth | null; - /** Internal recursion guard; callers outside this module must not set it. */ - comboAttempt?: boolean; - /** Internal combo handoff for one parent-validated continuation snapshot. */ - comboReplaySnapshot?: { - sourceBody: unknown; - previousResponseInputExpanded: boolean; - providerContinuation: OcxProviderContinuationState | undefined; - recoveredPlaintext: boolean; - }; - /** Internal combo handoff: allow a later same-provider model after a reset-derived 429/402. */ - deferCodexResetDerivedCooldown?: boolean; - /** 030-owned handoff when a child consumed the original failure under bounds. */ - onConsumedComboFailure?: (failure: ConsumedComboFailure) => void; - /** A stored Pool credential was refreshed and its one allowed same-account replay was sent. */ - onStoredPool401ReplayDispatched?: () => void; - /** Caller-owned for Chat/Claude replay; omitted only at genuine Responses ingress. */ - translatorBudget?: TranslatorBudget; - /** - * Transient sends already spent by this logical request. Combo children inherit the parent's - * holder through the options spread, so a fan-out shares one allowance instead of taking a - * fresh one per target (#4546). - */ - sendBudget?: TransientSendBudget; - /** - * Terminal vision-describe marker (roadmap 180): true when the inbound - * request IS the vision sidecar's own loopback describe call. The plan site - * then STRIPS images instead of planning another describe — a depth cap of 1 - * that holds under predicate drift and combo re-resolution. The Chat surface - * detects the raw `x-opencodex-vision-describe` header before its bridge - * rebuilds headers and carries the fact through this flag. - */ - visionDescribeTerminal?: boolean; -} - - - -/** - * Build the 499 JSON error the proxy returns when the client disconnects before the - * response completes (`client_cancelled`). - */ -export function clientCancelledResponse(): Response { - return formatErrorResponse(499, "client_cancelled", "Client cancelled request"); -} - - - -export function sanitizedRetryAfter(value: string | null, now: number): string | undefined { - const trimmed = value?.trim(); - if (!trimmed || trimmed.length > 128) return undefined; - return parseRetryAfterMs(trimmed, now) !== undefined ? trimmed : undefined; -} - - - -export async function consumeComboFailure( - response: Response, - signal?: AbortSignal, - now = Date.now(), -): Promise { - const fallback = `Provider error ${response.status}`; - let classificationText = fallback; - let usage: OcxUsage | undefined; - let upstreamCode: string | undefined; - let upstreamMessage: string | undefined; - let upstreamType: string | undefined; - // Whether the body itself confirms a quota/rate-limit refusal, computed on the SAME read as - // the classification below. `shouldRetryCodexPoolAccountQuota` cannot be called here without - // a second body read, so this mirrors its normalization: raw 402/429, or a 5xx whose intact, - // display-safe body carries a recognized quota message. - let quotaConfirmedByBody = false; - try { - const body = await readBoundedResponseBody(response, { - signal, - // Match shouldRetryCodexPoolAccountQuota before treating a 5xx body as quota evidence. - fatalUtf8: response.status >= 500 && response.status < 600, - }); - usage = usageFromComboFailureText(body.text); - if ( - response.status >= 500 && response.status < 600 - && body.displaySafe && !body.truncated - ) { - const quotaMessage = codexQuotaFailureMessage(body.text); - quotaConfirmedByBody = quotaMessage !== undefined - && isRateLimitOrQuotaFailureMessage(quotaMessage); - } - if (body.displaySafe) { - const normalized = normalizeUpstreamErrorText(body.text, fallback); - classificationText = normalized.safeText; - upstreamCode = normalized.code; - upstreamMessage = normalized.message; - upstreamType = normalized.type; - } - } catch (error) { - if (signal?.aborted) throw error; - classificationText = fallback; - } - const cyberFailure = isCyberPolicyCode(upstreamCode) || isCyberPolicyMessage(classificationText); - const normalizedUpstreamCode = cyberFailure ? CYBER_POLICY_ERROR_CODE : upstreamCode; - const message = cyberFailure - ? upstreamMessage - ?? (isCyberPolicyCode(upstreamCode) ? CYBER_POLICY_FALLBACK_MESSAGE : classificationText) - : classificationText === fallback - ? fallback - : `${fallback}: ${classificationText}`; - const upstreamRetryAfter = response.headers.get("retry-after"); - // Past HTTP dates are an immediate retry directive, just like the numeric value zero. - // Normalize before the client helper discards them and substitutes a default delay. - const effectiveRetryAfter = parseRetryAfterMs(upstreamRetryAfter, now) === undefined - && parseRetryAfterMs(upstreamRetryAfter, now, { preserveImmediate: true }) !== undefined - ? "0" - : upstreamRetryAfter; - // Client response may get the synthetic "2" fallback; cooldown metadata must not — - // otherwise coolComboTarget treats it as a 2s cooldown instead of the 60s default. - const clientRetryAfter = resolveClientRetryAfter({ - status: response.status, - message, - upstreamRetryAfter: effectiveRetryAfter, - now, - }); - const cooldownRetryAfter = resolveClientRetryAfter({ - status: response.status, - message, - upstreamRetryAfter: effectiveRetryAfter, - now, - includeDefault: false, - }); - return { - response: formatErrorResponse( - response.status, - cyberFailure ? (upstreamType ?? CYBER_POLICY_ERROR_CODE) : "upstream_error", - message, - { - ...(normalizedUpstreamCode !== undefined ? { code: normalizedUpstreamCode } : {}), - ...(clientRetryAfter !== undefined ? { retryAfter: clientRetryAfter } : {}), - }, - ), - classificationText, - ...(normalizedUpstreamCode !== undefined ? { upstreamCode: normalizedUpstreamCode } : {}), - ...(!cyberFailure && cooldownRetryAfter !== undefined ? { retryAfter: cooldownRetryAfter } : {}), - // The EFFECTIVE classification decides, not the raw status. An upstream that wraps a quota - // refusal in a 5xx still carries `x-codex-*-reset-at`, and gating on 402/429 alone threw - // those away, so the combo target came back up immediately instead of waiting for the - // window it was told about. `cyberFailure` stays excluded: a policy block is not a quota. - ...(!cyberFailure - && (response.status === 429 || response.status === 402 || quotaConfirmedByBody) - ? { resetAt: codexQuotaOutcomeMeta(response).resetAt } - : {}), - ...(usage ? { usage } : {}), - }; -} - - - -export function usageFromComboFailureText(text: string): OcxUsage | undefined { - try { - const payload = JSON.parse(text) as Record; - const nested = payload.response; - const source = nested && typeof nested === "object" && !Array.isArray(nested) - ? nested as Record - : payload; - return usageFromResponsesPayload(source.usage); - } catch { - return undefined; - } -} - - - -export function createChildPassthroughCallbackGate(options: HandleResponsesOptions) { - type Pending = - | { kind: "terminal"; status: ResponsesTerminalStatus } - | { kind: "cancel" }; - let state: "pending" | "committed" | "discarded" = "pending"; - let pending: Pending | undefined; - let accepted = false; - let pendingModel: string | undefined; - let completionAccepted = false; - let completionRejected = false; - const publish = (value: Pending): void => { - if (value.kind === "terminal") options.onNativePassthroughTerminal?.(value.status); - else options.onNativePassthroughCancel?.(); - }; - const publishCompletion = (): void => { - if (state !== "committed" || completionRejected || pendingModel === undefined) return; - const model = pendingModel; - pendingModel = undefined; - options.onResponseComplete?.(model); - }; - const receive = (value: Pending): void => { - if (state === "discarded" || accepted) return; - accepted = true; - if (value.kind === "cancel" || value.status !== "completed") { - completionRejected = true; - pendingModel = undefined; - } - if (state === "committed") return publish(value); - pending ??= value; - }; - return { - onTerminal: (status: ResponsesTerminalStatus) => receive({ kind: "terminal", status }), - onCancel: () => receive({ kind: "cancel" }), - onResponseComplete: (model: string) => { - if (state === "discarded" || completionRejected || completionAccepted || !model.trim()) return; - completionAccepted = true; - pendingModel = model; - publishCompletion(); - }, - commit: () => { - if (state !== "pending") return; - state = "committed"; - if (pending) publish(pending); - pending = undefined; - publishCompletion(); - }, - discard: () => { - state = "discarded"; - pending = undefined; - pendingModel = undefined; - }, - }; -} - - -export function buildComboChildHeaders(parentHeaders: HeadersInit): Headers { - const childHeaders = new Headers(parentHeaders); - // A provisional caller credential is not authoritative for a Combo child. - childHeaders.delete("authorization"); - childHeaders.delete("chatgpt-account-id"); - // Combo children re-serialize already-decoded JSON. Keeping transport metadata from - // the parent would make the child decoder treat plain JSON as compressed bytes. - childHeaders.delete("content-length"); - childHeaders.delete("content-encoding"); - return childHeaders; -} - -const UNREADABLE_ENCRYPTED_AGENT_TASK_MESSAGE = - "Routed V2 worker task is encrypted for the native ChatGPT backend and cannot be read by the selected provider. Use plaintext V2 agent-message delivery or select a native ChatGPT model."; - -// Whole-body policy for non-streaming upstream JSON responses (see the application/json -// branch of the passthrough return path). 32 MiB matches the continuation snapshot read -// bound and is far above any legitimate non-streaming completion, including base64 image -// payloads. The stall deadlines only govern the body transfer — generation time before -// the response headers is untouched. Generation after early/chunked headers but before -// the first body byte previously used the 30-second inactivity deadline; this call site -// gives it the full body deadline instead. -const MAX_UPSTREAM_JSON_BODY_BYTES = 32 * 1024 * 1024; -const UPSTREAM_JSON_BODY_TOTAL_TIMEOUT_MS = 180_000; -const UPSTREAM_JSON_BODY_INACTIVITY_TIMEOUT_MS = 30_000; -const MAX_FAST_WIRE_CAPABILITY_WARNINGS = 256; -const warnedFastWireCapabilityGaps = new Set(); - -function warnFastWireCapabilityGap(providerName: string, modelId: string): void { - const safeProvider = sanitizeLogMetadataString(providerName) ?? "unknown"; - const safeModel = sanitizeLogMetadataString(modelId) ?? "unknown"; - const key = `${safeProvider}\0${safeModel}`; - if (warnedFastWireCapabilityGaps.has(key)) return; - if (warnedFastWireCapabilityGaps.size >= MAX_FAST_WIRE_CAPABILITY_WARNINGS) { - const oldest = warnedFastWireCapabilityGaps.values().next().value; - if (oldest !== undefined) warnedFastWireCapabilityGaps.delete(oldest); - } - warnedFastWireCapabilityGaps.add(key); - console.warn( - `[opencodex] Fast policy for ${safeProvider}/${safeModel} has service-tier capability but no Fast wire; preserving only caller-permitted tier behavior`, - ); -} -export const UPSTREAM_JSON_BODY_READ_OPTIONS = { - maxBytes: MAX_UPSTREAM_JSON_BODY_BYTES, - totalTimeoutMs: UPSTREAM_JSON_BODY_TOTAL_TIMEOUT_MS, - inactivityTimeoutMs: UPSTREAM_JSON_BODY_INACTIVITY_TIMEOUT_MS, - firstByteTimeoutMs: UPSTREAM_JSON_BODY_TOTAL_TIMEOUT_MS, -}; - -function unreadableEncryptedAgentTaskResponse(reason?: AgentTaskRecoveryFailureReason): Response { - return new Response( - JSON.stringify({ - error: { - message: UNREADABLE_ENCRYPTED_AGENT_TASK_MESSAGE, - type: "invalid_request_error", - code: "unreadable_encrypted_agent_task", - ...(reason === undefined ? {} : { recovery_reason: reason }), - }, - }), - { status: 400, headers: { "Content-Type": "application/json" } }, - ); -} - -/** - * Keep this trust boundary deliberately narrow: only a key-auth Responses route may consume - * opaque child-task ciphertext, and the model's final wire override must still be Responses. - * Callers keep combo attempts on their existing native-only recovery/fail-closed behavior. - */ -function canPassThroughEncryptedV2AgentTask( - route: RouteResult, - inboundWire: InboundWire, -): boolean { - if (route.combo !== undefined) return false; - const provider = route.provider; - if ( - inboundWire !== "responses" - || provider.allowEncryptedV2AgentTasks !== true - || (provider.authMode ?? "key") !== "key" - ) return false; - - return resolveWireProtocolOverride( - route.providerName, - route.modelId, - provider, - inboundWire, - ).adapter === "openai-responses"; -} - -/** Keep synthesized Claude identity out of request headers reused by policy/combo fallback. */ -function withClaudeNativeSession(headers: Headers, provider: OcxProviderConfig, sessionId?: string): Headers { - if (!sessionId || !isCanonicalOpenAiForwardProvider(provider) - || headers.has("session_id") || headers.has("session-id") || headers.has("thread-id")) return headers; - const forwarded = new Headers(headers); - forwarded.set("session_id", sessionId); - return forwarded; -} - -type ResponsesAuthResolution = - | { ok: true; authCtx: CodexAuthContext; headers: Headers; callerAuthHeaders: Headers; substituteMainCredential: boolean } - | { ok: false; response: Response }; - -/** - * The caller credential the final Codex auth resolution will be given, as far as the ROUTE - * decides it: a route change that may cross a credential domain drops the raw caller credential, - * and a trusted Claude-main handoff replaces it. - * - * Shared with the lineage preview in `handleResponsesInner`, which has to read a conversation's - * family under the same authenticated scope the resolution will record it under -- that scope is - * an HMAC of exactly this Authorization header. Two copies of this rule would put preview and - * final auth in different scopes the first time one of them changed. - */ -function codexRouteCredentialDomainHeaders( - req: Request, - route: RouteResult, - options: HandleResponsesOptions, - credentialDomainWasRewritten: boolean, -): Headers { - const trustedClaudeMainForFinalRoute = options.stripClaudeMainAuthForNoncanonicalForward === true - && isCanonicalOpenAiForwardProvider(route.provider) - ? options.trustedClaudeMainAuth : undefined; - if (trustedClaudeMainForFinalRoute) { - const claudeMainHeaders = new Headers(req.headers); - claudeMainHeaders.set("authorization", trustedClaudeMainForFinalRoute.authorization); - if (trustedClaudeMainForFinalRoute.chatgptAccountId) { - claudeMainHeaders.set("chatgpt-account-id", trustedClaudeMainForFinalRoute.chatgptAccountId); - } else { - claudeMainHeaders.delete("chatgpt-account-id"); - } - return claudeMainHeaders; - } - // Route-changing recursion retains typed admission, never an unscoped raw - // caller credential. Bearer admission is substituted or stripped below. - const routeMayChangeCredentialDomain = options.comboAttempt === true - || route.routeKind === "policy" - || credentialDomainWasRewritten; - if (routeMayChangeCredentialDomain && options.admission?.source !== "bearer") { - const scoped = new Headers(req.headers); - scoped.delete("authorization"); - scoped.delete("chatgpt-account-id"); - return scoped; - } - return req.headers; -} - -/** - * Does this route substitute OUR stored main credential, and does the caller own the credential - * this request will authenticate with? - * - * Both answers are needed twice: by the resolution below, and by the lineage preview, which must - * not follow a Pool family binding for a request whose credential never enters Pool state. One - * implementation, because two copies of this predicate disagreeing is the divergence the preview - * gate exists to prevent. The reasoning behind the substitution test itself is at its use site - * below (#1686, #2132). - */ -function codexRouteCredentialOwnership( - authInputHeaders: Headers, - config: OcxConfig, - route: RouteResult, - options: HandleResponsesOptions, -): { substituteMainCredential: boolean; requestScopedMainCredential: boolean } { - const substituteMainCredential = options.admission?.source === "bearer" - && (route.codexAccountMode !== undefined || isCanonicalOpenAiForwardProvider(route.provider)); - return { - substituteMainCredential, - requestScopedMainCredential: route.codexAccountMode !== undefined - && !substituteMainCredential - && hasForwardableCodexBearer(authInputHeaders, config), - }; -} - -/** - * Resolve Codex auth for a route. On unusable contexts, releases any probe lease - * before returning the 401 (nothing reaches upstream). - */ -async function resolveResponsesCodexAuth( - req: Request, - config: OcxConfig, - route: RouteResult, - options: HandleResponsesOptions, - credentialDomainWasRewritten = false, -): Promise { - try { - let authInputHeaders = codexRouteCredentialDomainHeaders( - req, - route, - options, - credentialDomainWasRewritten, - ); - // A caller-auth transport that is not canonical OpenAI (keyless Cursor) consumes the - // caller's Authorization as its own upstream token. Keep that contract only for a clean - // single bearer with NO ChatGPT-domain marker. A bearer marked for the ChatGPT domain — - // whether its marker is valid or malformed/conflicting — a combined/malformed value, or - // the captured explicit OpenAI pair is never a Cursor token; a foreign JWT carrying only - // a generic organizations claim is not ChatGPT-marked and keeps the legacy contract. - // chatgpt-account-id has no meaning outside the ChatGPT domain. - if (!isCanonicalOpenAiForwardProvider(route.provider) - && providerConsumesCallerAuthorization(route.provider)) { - const rawAuth = authInputHeaders.get("authorization")?.trim(); - const singleBearer = /^Bearer[\t ]+([^\s,]+)$/i.exec(rawAuth ?? "")?.[1]; - const domainClaim = singleBearer ? inspectChatGptDomainClaim(singleBearer) : { kind: "absent" as const }; - const dropBearer = options.nativeCallerAuth != null || domainClaim.kind !== "absent" - || (rawAuth !== undefined && singleBearer === undefined); - if (dropBearer || authInputHeaders.has("chatgpt-account-id")) { - const scoped = new Headers(authInputHeaders); - if (dropBearer) scoped.delete("authorization"); - scoped.delete("chatgpt-account-id"); - authInputHeaders = scoped; - } - } - // The caller's own Direct credential may cross an internal route change only to the - // canonical OpenAI transport, under a predicate deliberately STRICTER than plain - // unchanged-route Direct forwarding: a clean non-proxy bearer whose ChatGPT-domain - // marker is valid, with any explicit account header matching that marker. Unchanged - // routes keep their legacy rules; sidecar enrichment grants no primary authority. - if (options.callerDirectAuth && isCanonicalOpenAiForwardProvider(route.provider)) { - const directHeaders = new Headers({ - authorization: options.callerDirectAuth.authorization, - ...(options.callerDirectAuth.chatgptAccountId - ? { "chatgpt-account-id": options.callerDirectAuth.chatgptAccountId } : {}), - }); - if (captureCallerDirectAuth(directHeaders, config)) { - authInputHeaders = new Headers(authInputHeaders); - authInputHeaders.set("authorization", options.callerDirectAuth.authorization); - if (options.callerDirectAuth.chatgptAccountId) { - authInputHeaders.set("chatgpt-account-id", options.callerDirectAuth.chatgptAccountId); - } else { - authInputHeaders.delete("chatgpt-account-id"); - } - } - } - // #1686: a caller that proved admission with a BEARER presented one of our own secrets. - // Refusing it here is what made the codex-cli `env_key` contract unusable against Direct. - // Admitting it is only safe because the stored main credential is substituted below, so - // the admission secret still never leaves this process. - // - // #2132: substitution answers "does THIS ROUTE need our stored ChatGPT credential", not - // "how did the caller authenticate". Only a native Codex route reaches the ChatGPT backend - // and can consume that credential; a key-authenticated routed provider carries its own and - // never touches it. Keying on the caller alone made an install that deliberately never - // logged into ChatGPT fail every routed request with "No usable Codex main credential". - // - // But ask that question the way the ADAPTER asks it. `codexAccountMode` is derived from the - // provider NAME (`providerCodexAccountMode`), while the passthrough adapter decides whether - // to forward caller credentials from the TRANSPORT — adapter, auth mode, and base URL - // (`isCanonicalOpenAiForwardProvider`). A row the operator named anything other than - // `openai`, pointed at the canonical ChatGPT backend with `authMode: "forward"`, satisfies - // the adapter's test and fails this one, so substitution was skipped and the adapter then - // forwarded our own admission secret upstream. Two predicates answering one question is the - // bug; the transport is the authority, because the transport is what actually carries the - // header. A key-authenticated routed provider is still not canonical-forward, so #2132's - // no-ChatGPT-login install keeps working. - const { substituteMainCredential, requestScopedMainCredential } = codexRouteCredentialOwnership( - authInputHeaders, - config, - route, - options, - ); - const stripAuthorization = options.admission?.source === "bearer" && !substituteMainCredential; - if (route.codexAccountMode === "direct" && !substituteMainCredential) { - validateForwardAdmissionCredential(authInputHeaders, config); - } - let authCtx: CodexAuthContext; - if (route.codexAccountMode) { - authCtx = await resolveCodexAuthContext(authInputHeaders, config, route.codexAccountMode, { - admission: options.admission, - codexAuthPolicy: options.codexAuthPolicy, - accountId: route.codexAccountId, - modelId: route.modelId, - substituteMainCredentialForDirect: substituteMainCredential, - requestScopedMainCredential, - beginCodexAccountSelection: codexAccountSelectionForTurn(options.turnAdmissionLease), - resolveCodexModelEntitlements: options.resolveCodexModelEntitlements, - signal: options.abortSignal, - nativeMainRefreshDependencies: options.nativeMainRefreshDependencies, - }); - options.onCodexAuthContextResolved?.(authCtx); - } else { - // A custom-named canonical-forward provider has no Codex account mode, but an - // admission bearer still substitutes the stored main credential below. Claim the - // same physical profile before synthesizing the main context so transport-based - // substitution cannot bypass a switch drain. - if ( - substituteMainCredential - && ( - isNativeMainTrafficBlocked() - || !tryClaimNativeMainProfileForTurn(options.turnAdmissionLease) - || isNativeMainTrafficBlocked() - ) - ) { - throw new CodexMainProfileDrainingError(); - } - authCtx = { kind: "main", accountId: null }; - options.onCodexAuthContextResolved?.(undefined); - } - // This resolver also builds a synthetic main context for unrelated keyed routes. Only - // the actual Codex-forward transport consumes main quota; provider names are not proof - // (custom-named canonical-forward providers must retain the same protection). - const mainPolicyConfig = isCanonicalOpenAiForwardProvider(route.provider) - ? options.codexAuthPolicy ?? config : undefined; - const headers = await materializeCodexUpstreamAuthAsync(authInputHeaders, authCtx, { - admission: options.admission, - config: mainPolicyConfig, - modelId: route.modelId, - beginCodexAccountSelection: codexAccountSelectionForTurn(options.turnAdmissionLease), - substituteMainCredential, - signal: options.abortSignal, - nativeMainRefreshDependencies: options.nativeMainRefreshDependencies, - }); - // Awaiting even a cached materialization yields. Preserve the policy error if the live - // quota/config changed during that yield, before usability could mislabel it as reauth. - headersForCodexAuthContext(headers, authCtx, mainPolicyConfig, route.modelId, options.admission); - if (!isCodexAuthContextUsable(authCtx, config)) { - releaseCodexAuthContextProbeLease(authCtx); - return { - ok: false, - response: formatErrorResponse(401, "authentication_error", "Selected Codex account needs reauthentication"), - }; - } - if (stripAuthorization) { - headers.delete("authorization"); - headers.delete("chatgpt-account-id"); - } - if (providerConsumesCallerAuthorization(route.provider) && options.admission?.source !== undefined - && options.admission.source !== "loopback") { - validateForwardAdmissionCredential(headers, config); - } else { - // Even adapters that ignore caller auth must not retain a proxy secret for - // a later internal hop or a future transport change. - const bearer = headers.get("authorization")?.replace(/^Bearer\s+/i, "").trim(); - if (bearer && isProxyAdmissionSecret(bearer, config)) { - headers.delete("authorization"); - headers.delete("chatgpt-account-id"); - } - } - return { - ok: true, - authCtx, - headers, - callerAuthHeaders: new Headers(authInputHeaders), - substituteMainCredential, - }; - } catch (err) { - if (options.abortSignal?.aborted || req.signal.aborted) { - return { ok: false, response: clientCancelledResponse() }; - } - if (err instanceof CodexAuthContextError) { - const safeAccountLabel = route.codexAccountNamespace - ? `${route.providerName}-${route.codexAccountNamespace}` - : formatCodexProviderForLog(route.providerName, err.accountId, config); - console.error(`[codex-auth] Pool account ${safeAccountLabel} token failed; reauthentication required`); - } - if (err instanceof ForwardAdmissionCredentialError) { - return { ok: false, response: formatErrorResponse(401, "authentication_error", err.message) }; - } - const response = mapCodexAuthContextErrorToResponse(err, { - accountSelector: route.codexAccountNamespace, - now: Date.now(), - }); - if (response) return { ok: false, response }; - throw err; - } -} - -/** - * Terminal means the grant itself is dead and no retry can help. Everything else — - * an untyped network failure, a token-endpoint 5xx surfacing as `unknown`, an abort, - * refresh capacity, lock contention, a superseded flight — is transient, and treating - * it as terminal would quarantine a healthy account on an upstream blip, which is the - * defect this path exists to fix (#2887). - */ -function isTerminalPoolRefreshFailure(error: unknown): boolean { - // Delegated so "terminal" has ONE definition. A missing record or a missing refresh-grant - // fingerprint is permanent -- retrying cannot conjure a credential -- and used to be a bare - // Error, which fell through to the retryable 503 and told the operator to keep retrying a - // request that could never succeed. - return isTerminalCodexPoolRefreshFailure(error); -} - -/** - * The refusal an operator meets when a stored pool credential's forced refresh does not complete. - * - * A bare "retry this request" reads as a transient fault in the proxy, which is how #4212's - * reporter spent an afternoon concluding OpenCodex had broken while one of their own accounts was - * the thing that needed them. It stays a retryable 503 and stays non-quarantining, because the - * refresh genuinely may succeed and a token-endpoint 5xx must not retire a healthy account - * (#2887). What it adds is the account and the exit: when retrying stops helping, that account - * has to be signed in again. - * - * The label is a public account selector when the request carried one, otherwise the durable - * `p`-prefixed log label — never the raw pool id and never the email. Those are the identifiers - * `responses-compaction-routing.test.ts` and `codex-auth-context.test.ts` already assert must not - * reach an operator-facing surface, and an error body travels further than a log line, not less. - * When neither is resolvable the sentence degrades to "the selected Codex pool account" rather - * than naming something opaque, because a wrong name is worse than no name. - * - * The wording says "sign in to that account again" and deliberately does NOT say - * "reauthentication". `classifyError` runs `isAuthenticationMessage` before it reaches the - * `status === 503` arm, and that check is status-blind on the bare substring "authentication", - * which "reauthentication" contains. A body carrying that word is reclassified to - * `authentication_error` / `invalid_api_key` even though the HTTP status stays 503 — and Codex - * applies retry-after backoff only for `server_is_overloaded`, so the friendlier sentence would - * have quietly disabled the retry this refusal exists to ask for. `options.code` cannot buy the - * classification back; only the wording can. - */ -export function poolCredentialRefreshIncompleteResponse(args: { - authCtx: CodexAuthContext; - config: Pick; - accountSelector?: string; - logCtx?: RequestLogContext; -}): Response { - // The wire contract below is unchanged on purpose, so the record has to carry the origin - // instead. Without it an operator reads this sentence under a field named "Upstream reason" - // and goes looking at the provider's status page for a refusal that never left this process. - if (args.logCtx) markLocalRequestLogRefusal(args.logCtx, CODEX_POOL_REFRESH_INCOMPLETE_LOG_REASON); - const label = args.accountSelector ?? codexAuthContextLogLabel(args.authCtx, args.config); - const account = label ? `Codex pool account ${label}` : "the selected Codex pool account"; - const response = formatErrorResponse( - 503, - "server_busy", - `Codex credential refresh did not complete for ${account}; retry this request. ` - + "If it keeps failing, sign in to that account again.", - ); - const headers = new Headers(response.headers); - headers.set("Retry-After", "1"); - return new Response(response.body, { status: response.status, headers }); -} - -/** - * One forced refresh and one same-account rebuild for a stored pool credential that - * upstream rejected with a pre-stream 401. `quarantine` distinguishes a dead grant, - * which must retire the account, from a transient failure, which must not. - */ -async function refreshPoolForwardAuth(args: { - logCtx?: RequestLogContext; - req: Request; - config: OcxConfig; - route: RouteResult; - authCtx: CodexAuthContext & { kind: "pool" }; - substituteMainCredential: boolean; - options: HandleResponsesOptions; -}): Promise< - | { ok: true; authCtx: CodexAuthContext; provider: OcxProviderConfig; headers: Headers } - | { ok: false; response: Response; quarantine: boolean; quarantineGeneration?: number } -> { - const { req, config, route, authCtx, substituteMainCredential, options } = args; - try { - const refreshed = await forceRefreshCodexPoolToken(authCtx.accountId, { - rejectedGeneration: authCtx.generation, - rejectedAccessToken: authCtx.accessToken, - signal: options.abortSignal, - }); - if (!refreshed.rotated) { - // The store resolved to the same bearer upstream just rejected. Replaying it - // would spend another upstream call to earn the identical 401. Upstream can do - // this on a SUCCESSFUL response by rotating only the refresh grant, so the - // credential generation may already have moved — quarantine has to be fenced on - // where the credential actually is, not on the generation we started from. - return { - ok: false, - quarantine: true, - quarantineGeneration: refreshed.generation, - response: formatErrorResponse(401, "authentication_error", "Selected Codex account needs reauthentication"), - }; - } - // Only a CAS this request performed itself proves the new credential descends from - // the rejected one. Somebody else's replacement may be a different identity, and - // its affinity must be retired rather than inherited. - if (refreshed.selfRefreshed) { - handOffThreadAffinityGeneration(authCtx.accountId, authCtx.generation, refreshed.generation); - } - const refreshedAuthCtx: CodexAuthContext = { - ...authCtx, - accessToken: refreshed.accessToken, - chatgptAccountId: refreshed.chatgptAccountId, - generation: refreshed.generation, - poolQuotaWriter: capturePoolQuotaWriter(authCtx.accountId, refreshed), - }; - const provider = applyCodexAuthContextToProvider( - stripCodexRuntimeProviderFields(route.provider), - refreshedAuthCtx, - route.codexAccountMode, - ); - const headers = await materializeCodexUpstreamAuthAsync(req.headers, refreshedAuthCtx, { - admission: options.admission, - config: options.codexAuthPolicy ?? config, - modelId: route.modelId, - substituteMainCredential, - signal: options.abortSignal, - nativeMainRefreshDependencies: options.nativeMainRefreshDependencies, - }); - return { ok: true, authCtx: refreshedAuthCtx, provider, headers }; - } catch (error) { - if (isTerminalPoolRefreshFailure(error)) { - return { - ok: false, - quarantine: true, - response: formatErrorResponse(401, "authentication_error", "Selected Codex account needs reauthentication"), - }; - } - return { - ok: false, - quarantine: false, - response: poolCredentialRefreshIncompleteResponse({ - authCtx, - config, - accountSelector: route.codexAccountNamespace, - logCtx: args.logCtx, - }), - }; - } -} - -async function refreshNativeMainForwardAuth(args: { - req: Request; - config: OcxConfig; - route: RouteResult; - authCtx: CodexAuthContext; - substituteMainCredential: boolean; - options: HandleResponsesOptions; -}): Promise< - | { ok: true; authCtx: CodexAuthContext; provider: OcxProviderConfig; headers: Headers } - | { ok: false; response: Response } -> { - const { req, config, route, authCtx, substituteMainCredential, options } = args; - if (authCtx.kind !== "main-pool") { - return { ok: false, response: formatErrorResponse(401, "authentication_error", "No native main credential to refresh") }; - } - try { - const refreshed = await forceRefreshMainAccountToken(authCtx.accessToken, { - signal: options.abortSignal, - ...(options.nativeMainRefreshDependencies ?? {}), - }); - if (!refreshed) { - return { ok: false, response: formatErrorResponse(401, "authentication_error", "Codex main account needs reauthentication") }; - } - const refreshedAuthCtx: CodexAuthContext = { - ...authCtx, - accessToken: refreshed.accessToken, - chatgptAccountId: refreshed.chatgptAccountId, - }; - const provider = applyCodexAuthContextToProvider( - stripCodexRuntimeProviderFields(route.provider), - refreshedAuthCtx, - route.codexAccountMode, - ); - const headers = await materializeCodexUpstreamAuthAsync(req.headers, refreshedAuthCtx, { - admission: options.admission, - config: options.codexAuthPolicy ?? config, - modelId: route.modelId, - substituteMainCredential, - signal: options.abortSignal, - nativeMainRefreshDependencies: options.nativeMainRefreshDependencies, - }); - return { ok: true, authCtx: refreshedAuthCtx, provider, headers }; - } catch (error) { - if (options.abortSignal?.aborted || req.signal.aborted) { - return { ok: false, response: clientCancelledResponse() }; - } - return { ok: false, response: mapCodexAuthContextErrorToResponse(error, { - now: Date.now(), accountSelector: route.codexAccountNamespace, - }) ?? nativeMainRefreshFailureResponse(error) }; - } -} - -async function resolveSubagentFallbackModelEligibility(args: { - config: OcxConfig; - fallbackChain: readonly string[] | null; - nativeMainReadsForbidden: boolean; - resolver: typeof resolveCodexModelEntitlements; -}): Promise { - if (!subagentFallbackNeedsModelEntitlements(args.fallbackChain, args.config)) return undefined; - const excludeAccountIds = args.nativeMainReadsForbidden - ? new Set([MAIN_CODEX_ACCOUNT_ID]) - : undefined; - const snapshot = await args.resolver(args.config, { excludeAccountIds }); - return (modelId) => { - const entitledAccountIds = entitledCodexAccountIdsForModel(snapshot, modelId); - return entitledAccountIds - ? new Set([...entitledAccountIds].filter(accountId => !excludeAccountIds?.has(accountId))) - : undefined; - }; -} - -/** - * Apply every route-dependent request mutation against the final selected route. - * Must run only after subagent fallback has settled the model/provider. - */ -async function applyFinalRouteRequestNormalization(args: { - parsed: OcxParsedRequest; - route: RouteResult; - config: OcxConfig; - req: Request; - logCtx: RequestLogContext; - inboundWire: InboundWire; - inboundTransport?: "websocket"; - claudeGoAffinity?: HandleResponsesOptions["claudeGoAffinity"]; -}): Promise { - const { parsed, route, config, req, logCtx, inboundWire, inboundTransport } = args; - const effortSelector = prepareEffortNormalization(parsed, route); - - // Only Anthropic message routes retain the Codex-facing selector. Other providers must keep - // their existing response.model contract even when their public and wire model ids differ. - const responseModelId = parsed.modelId; - const preserveAnthropicResponseModel = route.providerName === "anthropic" - || route.provider.adapter === "anthropic"; - - // Apply the routed model id upstream: routing may strip a "/" namespace. - if (route.modelId !== parsed.modelId) { - if (parsed._rawBody && typeof parsed._rawBody === "object") { - (parsed._rawBody as { model?: string }).model = route.modelId; - } - parsed.modelId = route.modelId; - } - // Transport-neutral reliability policy (#875): applies to any Responses - // upstream whose final adapter is openai-responses, not only WS turns. - const responsesUpstreamStreaming = providerModelResponsesUpstreamStreaming( - route.providerName, - route.provider, - route.modelId, - ); - - // Settle the wire once so logging, fast-mode, auth, and sidecars read the adapter - // this request will actually use (#404). - route.provider = resolveOpenCodeGoTransport(route.provider, - args.claudeGoAffinity ? args.claudeGoAffinity.sessionLane : getOrAllocateRequestSessionLane(req)); - route.provider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire); - parsed._plaintextV2AgentMessages = shouldPreparePlaintextV2AgentMessages({ - enabled: config.plaintextV2AgentMessages === true, - inboundWire, - canonicalChatGpt: isCanonicalOpenAiForwardProvider(route.provider), - requestBody: parsed._rawBody, - }); - // Recompute from the original wire preference on every route, including fallback. - // A provider default never converts raw reasoning into a summary. - if (inboundWire === "responses" && parsed._rawBody) { - const summary = (parsed._rawBody as { reasoning?: { summary?: unknown } }).reasoning?.summary; - parsed.options.hideThinkingSummary = summary === "none" - || (!summary && route.provider.showThinkingSummary !== true); - } - if (preserveAnthropicResponseModel) parsed._responseModelId = responseModelId; - logCtx.model = route.modelId; - logCtx.provider = route.providerName; - logCtx.providerAdapter = route.provider.adapter; - logCtx.routeDecision = route.routeDecision; - if (route.routeReason === "model-alias" || route.modelId !== responseModelId && responseModelId.includes("/")) logCtx.requestedAlias = responseModelId; - - if (responsesUpstreamStreaming === false && route.provider.adapter === "openai-responses") { - parsed.stream = false; - if (parsed._rawBody && typeof parsed._rawBody === "object") { - (parsed._rawBody as Record).stream = false; - } - } - - // Generic Responses clients (e.g. AI-SDK apps) omit `store`, but the canonical - // forward Codex backend rejects a native request without an explicit store:false. - // Default it only there — every other Responses upstream (key-auth providers and - // custom forward gateways) intentionally keeps the omitted-store server-side - // default for previous_response_id reuse — and never override an explicit value. - if ( - isCanonicalOpenAiForwardProvider(route.provider) - && parsed._rawBody && typeof parsed._rawBody === "object" - && (parsed._rawBody as Record).store === undefined - ) { - (parsed._rawBody as Record).store = false; - } - - // Final selected model before virtual wire-model rewriting (Pro aliases). - const finalSelectedModelId = route.modelId; - - // Virtual model rewriting: Pro aliases → base model + reasoning.mode="pro". - applyOpenAiVirtualModel(parsed, route, logCtx); - if (parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId) { - logCtx.resolvedModel = route.modelId; - logCtx.preserveResolvedModelFromRoute = true; - } - - // Resolve Fast policy after the final route/wire settles. A1 records the decision on parsed - // options; the Responses adapter owns the final outbound body write. - const fastPolicy = fastPolicyForModel( - route.provider, - route.modelId, - route.providerName, - inboundWire, - config.providers[route.providerName], - ); - const modelServiceTierSupport = serviceTierSupportFromPolicy(fastPolicy); - const callerTier = parsed.options.serviceTier; - // The ChatGPT-internal Codex backend echoes `service_tier: "default"` even on turns it - // scheduled as priority, so its echo cannot confirm OR deny Fast. Believing it reported every - // Fast request as `response-declined` (#2558). The public API's echo stays authoritative. - parsed.options.tierObservation = tierObservationContext( - fastPolicy, - config.fastMode, - callerTier, - isCanonicalOpenAiForwardProvider(route.provider) ? false : undefined, - ); - parsed.options.tierDecision = decideTier(fastPolicy, config.fastMode, callerTier); - parsed.options.serviceTier = tierValueAfterDecision(parsed.options.tierDecision, callerTier); - if (fastPolicy.capability === true && fastPolicy.fastWire === null) { - warnFastWireCapabilityGap(route.providerName, route.modelId); - } - applyServiceTierGate( - route.provider, - parsed._rawBody, - parsed.options, - route.modelId, - route.providerName, - inboundWire, - fastPolicy, - ); - if (modelServiceTierSupport === false) { - logCtx.requestedServiceTier = undefined; - logCtx.requestedSpeedLabel = undefined; - } - - { - const guidance = await multiAgentGuidanceText(parsed, { - multiAgentGuidanceEnabled: config.multiAgentGuidanceEnabled, - codexAccountNamespace: route.codexAccountNamespace, - injectionModel: config.injectionModel, - injectionEffort: config.injectionEffort, - subagentModels: config.subagentModels, - subagentModelFallback: config.subagentModelFallback, - injectionPrompt: config.injectionPrompt, - }); - if (guidance) { - injectDeveloperMessage(parsed, guidance); - if (isInjectionDebugEnabled()) { - injectionDebugLog(`[opencodex] ${route.modelId}: multi-agent guidance injected (surface=${collabSurface(parsed)}, guidanceEnabled=${multiAgentGuidanceEnabled(config)}, ${guidance.length} chars)`); - } - } else if (isInjectionDebugEnabled() && collabSurface(parsed) !== null) { - injectionDebugLog(`[opencodex] ${route.modelId}: collab surface=${collabSurface(parsed)}, guidance silent (effort=${parsed.options.reasoning ?? "unset"}, injectionModel=${config.injectionModel ?? "unset"})`); - } - } - - { - const { applyPinnedEffort } = await import("../effort-policy"); - const pinned = applyPinnedEffort(parsed, route, config, effortSelector); - if (pinned) { - logCtx.requestedEffort = pinned.from ? `${pinned.from}->${pinned.to}` : pinned.to; - if (isInjectionDebugEnabled()) { - injectionDebugLog(`[opencodex] ${route.modelId}: pinned reasoning effort applied (${pinned.from ?? "none"} -> ${pinned.to})`); - } - } - } - - { - const { applyEffortCap, effortCapAppliesTo, supportedLadderFor } = await import("../effort-policy"); - const surface = collabSurface(parsed); - if (effortCapAppliesTo(surface, req.headers, config, parsed._compactionRequest === true)) { - const capped = applyEffortCap(parsed, req.headers, config, supportedLadderFor(route)); - if (capped) { - logCtx.requestedEffort = `${capped.from}->${capped.to}`; - if (isInjectionDebugEnabled()) { - injectionDebugLog(`[opencodex] ${route.modelId}: effort cap applied (${capped.from} -> ${capped.to}, ${capped.subagent ? "sub-agent" : "main"} turn)`); - } - } - } else if (isInjectionDebugEnabled() && (config.effortCap || config.subagentEffortCap)) { - injectionDebugLog(`[opencodex] ${route.modelId}: effort cap skipped (surface=${surface ?? "none"}, v2 feature only)`); - } - } - - { - const { nativeEffortClamp, shouldApplyNativeEffortClamp } = await import("../../codex/catalog"); - const clamped = shouldApplyNativeEffortClamp(route.providerName, route.provider, finalSelectedModelId) - ? nativeEffortClamp(route.modelId, parsed.options.reasoning) - : null; - if (clamped) { - parsed.options.reasoning = clamped; - const raw = parsed._rawBody as { reasoning?: { effort?: string } } | undefined; - if (raw?.reasoning && typeof raw.reasoning === "object") raw.reasoning.effort = clamped; - logCtx.requestedEffort = `${logCtx.requestedEffort ?? "max"}->${clamped}`; - } - } - recordAttemptRequestedEffort(logCtx); - logCtx.modelSupportsServiceTier = SERVICE_TIER_ADAPTERS.has(route.provider.adapter) - ? modelServiceTierSupport - : undefined; -} - - - -/** - * Sends one combo target may run on its own before the ladder moves on. A target is a whole - * request as far as its own provider is concerned, so this is the guarded profile's base - * allowance rather than a separate number to keep in sync. - */ -const COMBO_TARGET_BASE_SENDS = CODEX_TEXT_GUARDED_BUDGET_POLICY.baseSendAllowance; - -/** - * A combo's execution policy is DECLARED by the combo, not inherited from the single-target - * profile. - * - * `maxTargetTransitions: 1` and `maxAlternateTargetSends: 1` describe an account move, and - * applying them to a combo would refuse the second hop of a three-target combo -- which is why - * combo was left off `reserveDispatch` when the per-request split landed. The transitions a - * combo may make are exactly the targets it declares minus the one it starts on. What stays - * capped is the TOTAL: the first target's full ladder, one send for every further declared - * target, and the one shared final-recovery reserve. A one-target combo reduces to the guarded - * profile exactly, and a three-target combo whose every target fails hard reaches upstream six - * times instead of the twelve #4546 measured. - */ -function comboExecutionBudgetPolicy(declaredTargets: number): RequestExecutionBudgetPolicy { - const targets = Math.max(1, Math.trunc(declaredTargets)); - const hops = targets - 1; - const reserve = CODEX_TEXT_GUARDED_BUDGET_POLICY.finalRecoveryAllowance; - const total = COMBO_TARGET_BASE_SENDS + hops + reserve; - return { - maxTotalModelSends: total, - baseSendAllowance: total - reserve, - finalRecoveryAllowance: reserve, - maxAlternateTargetSends: Math.max(1, hops), - maxTargetTransitions: Math.max(1, hops), - }; -} - -/** - * A budget scope that keeps its own recovery ledgers but spends the SAME request-wide counter. - * - * `used` is redefined as an accessor onto the parent because the factory reads it back off this - * object -- `remainingBaseSends` and the total check both do -- so a copied number would let a - * combo target run its ladder against a stale total, which is precisely the per-layer counting - * this work exists to remove. The reserve, alternate-target and transition ledgers stay - * per-scope on purpose: a combo target's account failover is its own recovery decision, while - * the request total still bounds every target together. - */ -function deriveSendBudgetScope( - parent: RequestExecutionBudget, - policy: RequestExecutionBudgetPolicy, -): RequestExecutionBudget { - const scope = createRequestExecutionBudget(policy, parent.logicalRequestId); - Object.defineProperty(scope, "used", { - get: () => parent.used, - set: (value: number) => { parent.used = value; }, - enumerable: true, - configurable: true, - }); - return scope; -} - -/** - * The ladder one combo target may run, expressed as an allowance on the request-wide counter. - * - * `used + COMBO_TARGET_BASE_SENDS` gives this target its own ladder from wherever the request - * already stands, and the clamp holds back one send for each target still declared after it: a - * first target that 5xx-streaks must not eat the send the last declared target is entitled to. - * That guarantee is the difference between a per-target policy and a shared pool the first - * target drains. - */ -function comboTargetSendBudget( - comboScope: RequestExecutionBudget, - targetsDeclaredAfterThisOne: number, -): RequestExecutionBudget { - const policy = comboScope.policy; - const heldForLaterTargets = Math.max(0, targetsDeclaredAfterThisOne); - const ceiling = Math.max(1, policy.maxTotalModelSends - heldForLaterTargets); - return deriveSendBudgetScope(comboScope, { - maxTotalModelSends: policy.maxTotalModelSends, - baseSendAllowance: Math.min(ceiling, comboScope.used + COMBO_TARGET_BASE_SENDS), - finalRecoveryAllowance: policy.finalRecoveryAllowance, - // Within one target the account-move shape is unchanged: three same-account sends plus one - // alternate is the recovery live traffic depends on, and a combo does not widen it. - maxAlternateTargetSends: CODEX_TEXT_GUARDED_BUDGET_POLICY.maxAlternateTargetSends, - maxTargetTransitions: CODEX_TEXT_GUARDED_BUDGET_POLICY.maxTargetTransitions, - }); -} - -export async function handleComboResponses( - req: Request, - rawBody: unknown, - comboId: string, - config: OcxConfig, - logCtx: RequestLogContext, - options: HandleResponsesOptions & { translatorBudget: TranslatorBudget }, -): Promise { - const requestedModel = typeof (rawBody as { model?: unknown } | null)?.model === "string" - ? (rawBody as { model: string }).model - : `combo/${comboId}`; - Object.assign(logCtx, { - requestedModel, - model: requestedModel, - provider: "combo", - comboId, - }); - const combo = getCombo(config, comboId); - if (!combo) { - return formatErrorResponse(404, "invalid_request_error", `Unknown combo: ${comboId}`); - } - // The ladder's own scope, derived from what this combo DECLARES. It shares the request-wide - // counter with the holder that arrived on options -- a combo child already inherited that - // counter, but nothing read it as a limit across targets -- while its transition and - // alternate-target ledgers come from the target list rather than from the single-target - // account-move profile (#4546). - const comboSendScope = isRequestExecutionBudget(options.sendBudget) - ? deriveSendBudgetScope(options.sendBudget, comboExecutionBudgetPolicy(combo.targets.length)) - : undefined; - // Expand previous_response_id before image policy and child dispatch so a - // continuation that only references prior images still fails closed when - // imageInput is disabled (and so targets see the full replayed input). - const inboundClientThreadId = req.headers.get("x-codex-parent-thread-id")?.trim() || undefined; - const body = expandPreviousResponseInput(rawBody, inboundClientThreadId); - const scopeMismatch = previousResponseScopeMismatch(body); - if (scopeMismatch) { - console.warn("[opencodex] dropped a previous_response_id with a mismatched client task scope; continuing fresh"); - } - if (previousResponseReplayFailure(body)) { - return formatErrorResponse( - 400, - "previous_response_not_found", - "Continuation state is unavailable or corrupt; resend the full conversation without previous_response_id.", - ); - } - // Missing state returns the original body without a failure marker. Reject - // that unresolved continuation for image-disabled combos so a target cannot - // resolve prior images out of band. A successful expansion yields a new - // object (still carrying previous_response_id) and must not be treated as - // unresolved — text-only stored continuations remain allowed. - const requestedPreviousId = typeof (rawBody as { previous_response_id?: unknown } | null)?.previous_response_id === "string" - ? (rawBody as { previous_response_id: string }).previous_response_id.trim() - : ""; - const unresolvedPrevious = requestedPreviousId.length > 0 && body === rawBody; - if (combo.imageInput === "disabled" && unresolvedPrevious) { - return formatErrorResponse( - 400, - "previous_response_not_found", - "Continuation state is unavailable or corrupt; resend the full conversation without previous_response_id.", - ); - } - if (combo.imageInput === "disabled" && comboRequestHasImageInput(body)) { - return formatErrorResponse(400, "invalid_request_error", `Combo "${comboId}" does not accept image input`); - } - const comboReplaySnapshot = { - sourceBody: body, - previousResponseInputExpanded: body !== rawBody - && typeof (body as { previous_response_id?: unknown }).previous_response_id === "string", - providerContinuation: !scopeMismatch && body !== rawBody && requestedPreviousId - ? previousResponseProviderState(requestedPreviousId) - : undefined, - recoveredPlaintext: false, - }; - const adoptFailedChildLog = (childLog: RequestLogContext): void => { - // Attempts remain the complete physical history; the logical row mirrors the most recent - // failed target so an exhausted combo still has useful top-level reasoning diagnostics. - Object.assign(logCtx, childLog, { - requestedModel, - model: requestedModel, - provider: "combo", - comboId, - routeDecision: logCtx.routeDecision, - attempts: logCtx.attempts, - activeAttempt: undefined, - activeAttemptStartedAt: undefined, - }); - }; - - const unreadableEncryptedAgentTask = hasUnreadableEncryptedAgentTask( - (body as { input?: unknown } | undefined)?.input, - ); - const canDecryptUnreadableAgentTask = (target: (typeof combo.targets)[number]): boolean => { - const provider = config.providers[target.provider]; - if (!provider || provider.disabled === true) return false; - try { - const route = routeConcreteModel(config, `${target.provider}/${target.model}`); - return isCanonicalOpenAiForwardProvider(route.provider); - } catch { - return false; - } - }; - let comboPayloadReadable = false; - const payloadEligible = (target: (typeof combo.targets)[number]): boolean => - comboPayloadReadable || !unreadableEncryptedAgentTask || canDecryptUnreadableAgentTask(target); - let encryptedTaskRecoveryAttempted = false; - let recoveryFailureReason: AgentTaskRecoveryFailureReason | undefined; - let storedPool401ReplayDispatched = false; - const recoverUnreadableEncryptedTask = async (): Promise => { - if (encryptedTaskRecoveryAttempted) return false; - encryptedTaskRecoveryAttempted = true; - const recovery = agentTaskRecoveryConfig(config); - if ( - (options.inboundWire ?? "responses") !== "responses" - || !isThreadSpawnRequest(req.headers) - || !recovery - || options.comboAttempt - ) { - discardEncryptedAgentTaskRecovery( - req, - (body as { input?: unknown } | undefined)?.input, - config, - { parentThreadId: inboundClientThreadId }, - ); - return false; - } - let recovered = false; - try { - const result = await recoverEncryptedAgentTaskWithResult( - req, - (body as { input?: unknown } | undefined)?.input, - recovery, - config, - { parentThreadId: inboundClientThreadId, abortSignal: options.abortSignal }, - ); - recovered = result.recovered; - recoveryFailureReason = result.recovered ? undefined : result.reason; - } catch { - recovered = false; - recoveryFailureReason = undefined; - } - // Recovery has the same in-place input mutation contract as the direct routed path. - if ( - !recovered - || hasUnreadableEncryptedAgentTask((body as { input?: unknown } | undefined)?.input) - ) { - discardEncryptedAgentTaskRecovery( - req, - (body as { input?: unknown } | undefined)?.input, - config, - { parentThreadId: inboundClientThreadId }, - ); - return false; - } - comboPayloadReadable = true; - comboReplaySnapshot.recoveredPlaintext = true; - return true; - }; - const initialNow = Date.now(); - const pickWithWait = (pickOptions: { - exclude?: Iterable; - eligible?: (target: NonNullable["targets"][number]) => boolean; - now?: number; - }) => pickComboTargetWithWait(config, comboId, { - ...pickOptions, - waitForCooldownMs: combo.waitForCooldownMs, - abortSignal: options.abortSignal, - }); - let pick = await pickWithWait({ - eligible: payloadEligible, - now: initialNow, - }); - - if (unreadableEncryptedAgentTask && !pick) { - pick = await pickWithWait({ now: initialNow }); - if (!pick) { - discardEncryptedAgentTaskRecovery( - req, - (body as { input?: unknown } | undefined)?.input, - config, - { parentThreadId: inboundClientThreadId }, - ); - return options.abortSignal?.aborted - ? clientCancelledResponse() - : comboUnavailable(comboId); - } - if (!(await recoverUnreadableEncryptedTask())) { - return options.abortSignal?.aborted - ? clientCancelledResponse() - : unreadableEncryptedAgentTaskResponse(recoveryFailureReason); - } - } - - if (!pick) { - return options.abortSignal?.aborted - ? clientCancelledResponse() - : comboUnavailable(comboId); - } - // One immutable combo selection trace, before any child dispatch; child - // adoption below must never replace it with a concrete child route trace. - logCtx.routeDecision = comboRouteDecisionTrace(config, comboId, pick, requestedModel); - - let lastFailure: Response | null = null; - // Dispatched targets, not attempted picks: it indexes the declared target list so the clamp - // below can tell how many targets are still entitled to a send. - let comboTargetsDispatched = 0; - // The child log behind `lastFailure`. The natural end of the ladder adopts it inside the - // no-more-targets branch; a budget refusal ends the ladder one iteration later, where that - // iteration's own `childLog` is already out of scope. - let lastFailedChildLog: RequestLogContext | undefined; - // The exhausted-combo mapping below runs outside the loop, where `failure.upstreamCode` - // is gone, so carry the loop's own classification decision instead of re-deriving a - // weaker one from the status alone (#4149). - let lastFailureClassifiesOverflow = false; - while (pick) { - if (options.abortSignal?.aborted) return clientCancelledResponse(); - const firstComboTarget = comboTargetsDispatched === 0; - // The first target seeds the ledger's target identity and charges nothing; every later one - // is a real transition, refused once the declared hops, the alternate-target ledger or the - // request total are spent. `countedExternally` is required: the child charges its own - // physical sends, and charging here as well would halve the cap without saying so. - const hopDecision = comboSendScope?.reserveDispatch({ - sendClass: firstComboTarget ? "initial" : "combo-failover", - targetKey: `${pick.target.provider}/${pick.target.model}`, - countedExternally: true, - }); - if (hopDecision && hopDecision.allowed) hopDecision.permit.use(); - else if (hopDecision && !firstComboTarget) { - // Out of budget is not this target's failure. The established exhaustion contract is to - // return the last real upstream answer with its status, headers and any quota body - // intact rather than to mint a synthetic error, and a later target only exists because - // an earlier one already recorded one. - if (lastFailedChildLog) adoptFailedChildLog(lastFailedChildLog); - break; - } - const targetSendBudget = comboSendScope - ? comboTargetSendBudget(comboSendScope, combo.targets.length - 1 - comboTargetsDispatched) - : options.sendBudget; - comboTargetsDispatched += 1; - const childLog: RequestLogContext = { - model: pick.target.model, - provider: pick.target.provider, - ...(logCtx.conversationId ? { conversationId: logCtx.conversationId } : {}), - ...(logCtx.surface ? { surface: logCtx.surface } : {}), - }; - const targetRoute = routeConcreteModel(config, `${pick.target.provider}/${pick.target.model}`); - const childBody = concreteComboRequestBody( - body, - pick.target, - comboDefaultEffort(config, comboId), - supportedLadderFor({ provider: targetRoute.provider, modelId: targetRoute.modelId }), - combo.reasoningEffortMode, - ); - const childHeaders = buildComboChildHeaders(req.headers); - const childRequest = new Request(req.url, { - method: req.method, - headers: childHeaders, - body: JSON.stringify(childBody), - }); - linkRequestSessionLane(req, childRequest); - let resolvedAuth: CodexAuthContext | undefined; - let terminalRecorder: ((status: ResponsesTerminalStatus, httpStatusOverride?: number) => void) | undefined; - const started = Date.now(); - const attempt = beginRequestAttempt( - (logCtx.attempts?.length ?? 0) + 1, - pick.target.provider, - pick.target.model, - config.providers[pick.target.provider]!.adapter, - ); - childLog.activeAttempt = attempt; - let attemptRetained = false; - const retainCancelledAttempt = (): void => { - if (attemptRetained) return; - sealRequestAttemptIdentity( - attempt, - childLog.provider, - childLog.providerAdapter ?? attempt.adapter, - childLog.accountLogLabel, - ); - finishRequestAttempt(attempt, 499, Date.now() - started, childLog.usage); - (logCtx.attempts ??= []).push(attempt); - attemptRetained = true; - }; - const completedTarget = { provider: pick.target.provider, model: pick.target.model }; - const writerGeneration = pick.writerGeneration; - let consumedChildFailure: ConsumedComboFailure | undefined; - const callbackGate = createChildPassthroughCallbackGate({ - ...options, - onResponseComplete: model => { - // The live config can change while the child is streaming. Never retain credentials. - const currentCombo = getCombo(config, comboId); - const provider = config.providers[completedTarget.provider]; - if (Object.hasOwn(config.providers, completedTarget.provider) - && provider && provider.disabled !== true - && currentCombo?.targets.some(target => targetKey(target) === targetKey(completedTarget))) { - rememberComboForLane(sessionLaneIdFromRequest(req.headers), comboId, completedTarget, model, writerGeneration); - } - options.onResponseComplete?.(model); - }, - onNativePassthroughTerminal: status => { - // A committed stream can acquire terminal metadata after preflight copied - // the child log. Publish it before the outer logger finalizes, but only - // through the gate: discarded attempts must never affect the parent. - // Undefined child fields must preserve metadata already inspected by WS. - if (childLog.terminalHttpStatus !== undefined) logCtx.terminalHttpStatus = childLog.terminalHttpStatus; - if (childLog.terminalIncompleteReason !== undefined) logCtx.terminalIncompleteReason = childLog.terminalIncompleteReason; - if (childLog.terminalErrorCode !== undefined) logCtx.terminalErrorCode = childLog.terminalErrorCode; - if (childLog.upstreamError !== undefined) logCtx.upstreamError = childLog.upstreamError; - options.onNativePassthroughTerminal?.(status); - }, - }); - let response: Response; - try { - const currentTargetProvider = pick.target.provider; - const deferCodexResetDerivedCooldown = combo.strategy === "failover" - && combo.targets.slice(pick.targetIndex + 1).some(target => - target.provider === currentTargetProvider - && payloadEligible(target) - && !isComboTargetInCooldown(comboId, target), - ); - response = await handleResponses(childRequest, config, childLog, { - ...options, - // After the spread: the child must run on THIS target's ladder, not on the holder the - // parent arrived with. - sendBudget: targetSendBudget, - comboAttempt: true, - comboReplaySnapshot, - deferCodexResetDerivedCooldown, - // Attempt-relative TTFT is recorded HERE (not via childLog.firstOutputMs — a later - // Object.assign(logCtx, childLog) would overwrite the request-relative value). - onFirstOutput: () => { - if (attempt.firstOutputMs === undefined) { - attempt.firstOutputMs = Math.max(0, Date.now() - started); - } - options.onFirstOutput?.(); - }, - onCodexAuthContextResolved: value => { resolvedAuth = value; }, - setTerminalOutcomeRecorder: value => { terminalRecorder = value; }, - onConsumedComboFailure: value => { consumedChildFailure = value; }, - onStoredPool401ReplayDispatched: () => { storedPool401ReplayDispatched = true; }, - onNativePassthroughTerminal: callbackGate.onTerminal, - onNativePassthroughCancel: callbackGate.onCancel, - onResponseComplete: callbackGate.onResponseComplete, - }); - } catch (error) { - callbackGate.discard(); - if (options.abortSignal?.aborted) { - retainCancelledAttempt(); - return clientCancelledResponse(); - } - throw error; - } - - if (options.abortSignal?.aborted) { - callbackGate.discard(); - retainCancelledAttempt(); - return clientCancelledResponse(); - } - - if (response.ok && !runTurnAdapterSseResponses.has(response)) { - const nativePassthrough = isNativePassthroughSseResponse(response); - const eagerRelay = isEagerRelaySseResponse(response); - let preflight; - try { - preflight = await preflightComboStreamResponse(response, childLog); - } catch (error) { - callbackGate.discard(); - if (options.abortSignal?.aborted) { - retainCancelledAttempt(); - return clientCancelledResponse(); - } - throw error; - } - if (preflight.kind === "failed") { - callbackGate.discard(); - terminalRecorder?.("failed", preflight.response.status); - response = preflight.response; - } else { - response = preflight.response; - if (nativePassthrough) markNativePassthroughSseResponse(response); - if (eagerRelay) markEagerRelaySseResponse(response); - } - } - - if (response.ok) { - sealRequestAttemptIdentity( - attempt, - childLog.provider, - childLog.providerAdapter ?? attempt.adapter, - childLog.accountLogLabel, - ); - (logCtx.attempts ??= []).push(attempt); - attemptRetained = true; - noteComboSuccess(comboId, combo, pick.target, pick.writerGeneration); - Object.assign(logCtx, childLog, { - requestedModel, - model: requestedModel, - provider: "combo", - comboId, - routeDecision: logCtx.routeDecision, - attempts: logCtx.attempts, - activeAttempt: attempt, - activeAttemptStartedAt: started, - resolvedModel: childLog.resolvedModel ?? childLog.model, - }); - options.onCodexAuthContextResolved?.(resolvedAuth); - options.setTerminalOutcomeRecorder?.(terminalRecorder); - callbackGate.commit(); - return response; - } - - callbackGate.discard(); - if (response.status === 499) { - retainCancelledAttempt(); - return clientCancelledResponse(); - } - let failure: ConsumedComboFailure; - try { - failure = consumedChildFailure - ?? await consumeComboFailure(response, options.abortSignal); - } catch (error) { - if (options.abortSignal?.aborted) { - retainCancelledAttempt(); - return clientCancelledResponse(); - } - throw error; - } - if (options.abortSignal?.aborted) { - retainCancelledAttempt(); - return clientCancelledResponse(); - } - sealRequestAttemptIdentity( - attempt, - childLog.provider, - childLog.providerAdapter ?? attempt.adapter, - childLog.accountLogLabel, - ); - finishRequestAttempt( - attempt, - failure.response.status, - Date.now() - started, - failure.usage, - ); - (logCtx.attempts ??= []).push(attempt); - attemptRetained = true; - lastFailure = failure.response; - lastFailedChildLog = childLog; - const failureDecision = comboFailureDecision(failure.response.status, failure.classificationText, { - code: failure.upstreamCode, - }); - const wantsStream = (rawBody as { stream?: unknown } | null)?.stream === true; - // Local byte admission has its own diagnostic; do not relabel it as an upstream refusal. - const classifyOverflow = failure.response.status === 413 - && (wantsStream || (failure.upstreamCode !== "outbound_body_too_large" - && failure.upstreamCode !== "translation_buffer_limit")); - lastFailureClassifiesOverflow = classifyOverflow; - if (storedPool401ReplayDispatched) { - if (failureDecision === "hop" && unreadableEncryptedAgentTask && !comboPayloadReadable) { - const recoveredTarget = await pickWithWait({ - exclude: pick.attempted, - eligible: target => { - try { - const route = routeConcreteModel(config, `${target.provider}/${target.model}`); - return route.codexAccountMode === undefined - && !isCanonicalOpenAiForwardProvider(route.provider); - } catch { - return false; - } - }, - }); - if (options.abortSignal?.aborted) return clientCancelledResponse(); - if (recoveredTarget && await recoverUnreadableEncryptedTask()) { - pick = recoveredTarget; - continue; - } - if (options.abortSignal?.aborted) return clientCancelledResponse(); - } - // Keep the spent Pool budget sticky even after a recovered routed child: - // no later failure may reopen ordinary combo/native account hopping. - adoptFailedChildLog(childLog); - if (classifyOverflow && failureDecision === "stop") { - return wantsStream - ? streamingContextOverflowResponse(requestedModel, options.translatorBudget) - : jsonContextOverflowResponse(); - } - return lastFailure; - } - if (failureDecision === "stop") { - adoptFailedChildLog(childLog); - if (classifyOverflow) { - return wantsStream - ? streamingContextOverflowResponse(requestedModel, options.translatorBudget) - : jsonContextOverflowResponse(); - } - return lastFailure; - } - console.warn( - `[combo] ${comboId}: ${targetKey(pick.target)} failed with ${failure.response.status} after ${Date.now() - started}ms`, - ); - const failureNow = Date.now(); - const attemptedTargets = pick.attempted; - const nextPick = advanceComboAfterFailure(config, pick, { - retryAfter: failure.retryAfter, - resetAt: failure.resetAt, - cooldownMs: combo.cooldownMs, - now: failureNow, - cooldownScope: comboFailureCooldownScope(failure.response.status, failure.classificationText, { - code: failure.upstreamCode, - }), - eligible: payloadEligible, - status: failure.response.status, - code: failure.upstreamCode, - message: failure.classificationText, - }); - if (nextPick) { - pick = nextPick; - } else { - pick = await pickWithWait({ - exclude: pick.attempted, - eligible: payloadEligible, - now: failureNow, - }); - } - if (!pick) { - if (options.abortSignal?.aborted) return clientCancelledResponse(); - if (unreadableEncryptedAgentTask && !comboPayloadReadable) { - const recoveredTarget = await pickWithWait({ - exclude: attemptedTargets, - now: failureNow, - }); - if (recoveredTarget && await recoverUnreadableEncryptedTask()) { - pick = recoveredTarget; - continue; - } - } - // Waiting or recovery may have observed cancellation after the check above. - if (options.abortSignal?.aborted) return clientCancelledResponse(); - adoptFailedChildLog(childLog); - } - } - if ( - lastFailure?.status === 413 - && lastFailureClassifiesOverflow - ) { - return (rawBody as { stream?: unknown } | null)?.stream === true - ? streamingContextOverflowResponse(requestedModel, options.translatorBudget) - : jsonContextOverflowResponse(); - } - return lastFailure!; -} - - - -function finalizeOwnedTranslatorBudget(response: Response, budget: TranslatorBudget): Response { - if (!response.body) { - budget.dispose(); - return response; - } - const reader = response.body.getReader(); - let finalized = false; - const finalize = () => { - if (finalized) return; - finalized = true; - budget.dispose(); - }; - const body = new ReadableStream({ - async pull(controller) { - try { - const result = await reader.read(); - if (result.done) { - finalize(); - controller.close(); - } else { - controller.enqueue(result.value); - } - } catch (error) { - finalize(); - controller.error(error); - } - }, - async cancel(reason) { - try { await reader.cancel(reason); } finally { finalize(); } - }, - }); - const finalizedResponse = new Response(body, { - status: response.status, - statusText: response.statusText, - headers: response.headers, - }); - if (isNativePassthroughSseResponse(response)) { - markNativePassthroughSseResponse(finalizedResponse); - } - if (isEagerRelaySseResponse(response)) { - markEagerRelaySseResponse(finalizedResponse); - } - return finalizedResponse; -} - -/** - * Service-tier capability gate, applied after the final route/wire is settled. A - * provider explicitly documented as NOT supporting `service_tier` must never - * receive it: strip the field and clear the logging value even when the caller - * supplied one (fail closed). A policy-produced canonical Fast decision has - * already passed capability validation and cannot be vetoed by Chat's caller - * forwarding permission. On unclassified routes every caller tier remains subject - * to `forwardCallerTier`. - */ -export function applyServiceTierGate( - provider: OcxProviderConfig, - rawBody: unknown, - options: { serviceTier?: string; tierDecision?: TierDecision }, - modelId?: string, - providerName?: string, - inbound: InboundWire = "responses", - resolvedPolicy?: ResolvedFastPolicy, -): void { - // A direct unit caller without a model id retains the historical tri-state behavior for - // adapters outside the OpenAI service-tier family. Once a model is known, resolve the final - // model adapter as well: an explicit override to Anthropic (or another non-OpenAI wire) must - // not carry a caller-supplied `service_tier` through a route that cannot forward it. - if (modelId === undefined && !SERVICE_TIER_ADAPTERS.has(provider.adapter)) return; - const policy = modelId === undefined - ? undefined - : resolvedPolicy ?? fastPolicyForModel(provider, modelId, providerName, inbound); - const forwardCallerTier = modelId === undefined - ? provider.supportsServiceTier !== false - : policy!.forwardCallerTier; - const rawTier = rawBody && typeof rawBody === "object" - ? (rawBody as Record).service_tier - : undefined; - const canonicalDecision = options.tierDecision?.kind === "set"; - const callerTierIsForeign = rawTier !== undefined - && (typeof rawTier !== "string" || canonicalFastTierMarker(rawTier) === undefined); - const dropForeignCallerTier = policy?.capability === true - && policy.fastWire?.kind === "service-tier" - && policy.fastWire?.foreignCallerTiers === "drop" - && callerTierIsForeign; - if (policy && policy.capability !== false && canonicalDecision) return; - if (forwardCallerTier && !dropForeignCallerTier) return; - if (rawBody && typeof rawBody === "object") { - delete (rawBody as Record).service_tier; - } - options.serviceTier = undefined; -} - -/** - * Route one `/v1/responses` request through the adapter pipeline: recovery loop, passthrough - * wire, image/web-search bridges, and the terminal-guard continuation. - */ -export async function handleResponses( - req: Request, - config: OcxConfig, - logCtx: RequestLogContext, - options: HandleResponsesOptions = {}, -): Promise { - const ownsBudget = options.translatorBudget === undefined; - const translatorBudget = options.translatorBudget ?? createTranslatorBudget(); - try { - const response = await handleResponsesInner(req, config, logCtx, { - ...options, - openAiSidecarAuth: options.openAiSidecarAuth === undefined - ? captureExplicitOpenAiCallerAuth(req.headers, config) : options.openAiSidecarAuth, - nativeCallerAuth: options.nativeCallerAuth === undefined - ? captureExplicitOpenAiCallerAuth(req.headers, config) : options.nativeCallerAuth, - callerDirectAuth: options.callerDirectAuth === undefined - ? captureCallerDirectAuth(req.headers, config) : options.callerDirectAuth, - // Capture before combo replay rebuilds the Request headers; children carry options. - visionDescribeTerminal: options.visionDescribeTerminal === true - || req.headers.get("x-opencodex-vision-describe") === "1", - translatorBudget, - // Created once at genuine ingress; a combo child arrives with the parent's holder already - // in options and must not start a fresh allowance. - sendBudget: options.sendBudget ?? createRequestExecutionBudget(), - }); - return ownsBudget ? finalizeOwnedTranslatorBudget(response, translatorBudget) : response; - } catch (error) { - if (ownsBudget) translatorBudget.dispose(); - throw error; - } -} - -/** - * Inner implementation of `handleResponses`; owns the pre-stream recovery loop and the - * per-request same-target 429 retry budgets. - */ -async function handleResponsesInner( - req: Request, - config: OcxConfig, - logCtx: RequestLogContext, - options: HandleResponsesOptions & { translatorBudget: TranslatorBudget }, -): Promise { - let pendingHostAdmissionLease: UpstreamHostAdmissionLease | null = null; - let authCtx: CodexAuthContext = { kind: "main", accountId: null }; - try { - // The Chat and Anthropic surfaces replay through here with a Responses-shaped body, - // so an omitted value means a genuine Responses inbound. - const inboundWire = options.inboundWire ?? "responses"; - const translatorBudget = options.translatorBudget; - const agentTaskRecovery = agentTaskRecoveryConfig(config); - let body: unknown; - try { - body = await readJsonRequestBody(req, translatorBudget, resolveInboundBodyLimitBytes(config.maxInboundBodyBytes)); - } catch (err) { - if (options.abortSignal?.aborted || req.signal.aborted) { - return clientCancelledResponse(); - } - return decodeRequestErrorResponse(err, "responses"); - } - // An effort row naming a table-less combo (`combo/x--high`) must reach the combo dispatcher - // as its base id, so the selector is normalized here, before comboIdFromRawBody reads model. - const comboRows = !options.comboAttempt && body && typeof body === "object" && !Array.isArray(body) - && typeof (body as { model?: unknown }).model === "string" - // One parse for both grammars, from the selector as the client sent it. Parsing them - // separately made the outcome depend on which ran first. - ? parseSyntheticRowId((body as { model: string }).model, config) - : { fastRow: null, effortRow: null }; - const comboEffortRow = comboRows.effortRow; - if (comboRows.fastRow) { - // Same reason as the effort row above: the combo dispatcher reads `model` next, so the - // selector has to be normalized before it, or a combo child is built from a synthetic id. - const raw = body as Record; - raw.model = comboRows.fastRow.baseId; - // A caller INTENT, not a decision. decideTier still rules on eligibility downstream, so - // fastMode:false and an ineligible route both still suppress it. - raw.service_tier = "priority"; - } - if (comboEffortRow) { - const raw = body as Record; - raw.model = comboEffortRow.baseId; - const rawReasoning = raw.reasoning; - raw.reasoning = { - ...(rawReasoning && typeof rawReasoning === "object" && !Array.isArray(rawReasoning) - ? rawReasoning as Record - : {}), - effort: comboEffortRow.effort, - }; - } - // Compaction may send the last client-visible bare model after a combo switch. - // Configured selectors take precedence; otherwise recall before combo dispatch (#3891). - if (!options.comboAttempt && body && typeof body === "object" && !Array.isArray(body)) { - const rawModel = (body as { model?: unknown }).model; - const rawInput = (body as { input?: unknown }).input; - const isCompactionTrigger = Array.isArray(rawInput) - && rawInput.some((item: unknown) => - typeof item === "object" && item !== null && (item as { type?: string }).type === "compaction_trigger"); - if (typeof rawModel === "string" && !rawModel.includes("/") && isCompactionTrigger - && !comboRows.fastRow && !comboEffortRow - && !resolveComboId(config, rawModel)) { - const recalledComboId = recallComboForLane(config, sessionLaneIdFromRequest(req.headers), rawModel); - if (recalledComboId) { - (body as Record).model = `combo/${recalledComboId}`; - } - } - } - // A shadow-call replacement that names a COMBO is routing policy, not the identity of any - // one pick. The late intercept site below resolves it through routeModel/tryPickComboModel, - // which collapses the table to a single target while still tagging `routeKind: "combo"`, so - // the combo gate on the next line never fires, handleComboResponses never runs, and 429/5xx - // hops — which only exist inside that loop — are unreachable (#4129). Rewrite the selector - // here instead, before comboIdFromRawBody reads `model`, and identify the combo by CONFIG - // LOOKUP so the check can never observe a one-candidate collapse. - if (!options.comboAttempt && body && typeof body === "object" && !Array.isArray(body)) { - const shadowIntercept = config.shadowCallIntercept; - const rawShadowModel = (body as { model?: unknown }).model; - if (shadowIntercept?.enabled && shadowIntercept.model && typeof rawShadowModel === "string" - && isShadowSourceModel(rawShadowModel, shadowIntercept.sourceModels)) { - const shadowComboId = resolveComboId(config, shadowIntercept.model); - if (shadowComboId && Object.hasOwn(config.combos ?? {}, shadowComboId)) { - (body as Record).model = shadowIntercept.model; - // Same rule as the late intercept site: record the operator-configured prefix that - // matched, never the caller's raw model string. Matching is by prefix, so the raw - // value is caller-controlled and reaches usage.jsonl and /api/logs. - logCtx.shadowCallRewrittenFrom = sanitizeLogMetadataString( - shadowSourceModelPrefix(rawShadowModel, shadowIntercept.sourceModels), - ); - } - } - } - const comboId = !options.comboAttempt ? comboIdFromRawBody(body, config) : null; - if (comboId && Object.hasOwn(config.combos ?? {}, comboId)) { - options.onRequestBodyRead?.(); - return handleComboResponses(req, body, comboId, config, logCtx, { - ...options, - // The original request body was accepted above. Combo children are synthetic - // replays and must not repeat the caller-owned timeout transition. - onRequestBodyRead: undefined, - }); - } - let unreadableEncryptedAgentTask = hasUnreadableEncryptedAgentTask( - (body as { input?: unknown } | undefined)?.input, - ); - const inboundClientThreadId = req.headers.get("x-codex-parent-thread-id")?.trim() || undefined; - const cursorClientThreadId = codexPoolAffinityKey(req.headers); - const originalBody = body; - if (options.comboReplaySnapshot) { - copyPreviousResponseReplayProvenance(options.comboReplaySnapshot.sourceBody, body); - } else { - body = expandPreviousResponseInput(body, inboundClientThreadId); - if (previousResponseScopeMismatch(body)) { - console.warn("[opencodex] dropped a previous_response_id with a mismatched client task scope; continuing fresh"); - } - if (previousResponseReplayFailure(body)) { - return formatErrorResponse( - 400, - "previous_response_not_found", - "Continuation state is unavailable or corrupt; resend the full conversation without previous_response_id.", - ); - } - } - const previousResponseInputExpanded = options.comboReplaySnapshot?.previousResponseInputExpanded - ?? (body !== originalBody - && typeof (body as { previous_response_id?: unknown }).previous_response_id === "string"); - - // Spawn-message compatibility (both directions): agent_message task payloads ride in - // encrypted_content slots as plaintext. Rewrite them to input_text on the RAW body BEFORE - // parsing so every consumer sees the payload: parseRequest (routed/translated providers read - // the parsed messages) and the native passthrough (_rawBody is this same object, serialized - // verbatim). Genuine backend ciphertext is left byte-identical (looksLikeBackendCiphertext). - { - const rewritten = sanitizeEncryptedContentInPlace( - (body as { input?: unknown } | undefined)?.input, - ); - if (rewritten > 0) - console.warn( - `[opencodex] rewrote ${rewritten} plaintext encrypted_content part(s) to input_text (spawn-message compatibility)`, - ); - } - - let parsed: OcxParsedRequest; - let toolBridgeMaps: ReturnType; - try { - parsed = parseRequest(body); - parsed._promptCacheKeyIsSharedCohort = options.promptCacheKeyIsSharedCohort; - // Captured before any parser mutates it, so both grammars see the client's id. - const { fastRow, effortRow } = parseSyntheticRowId(parsed.modelId, config); - if (fastRow) { - parsed.modelId = fastRow.baseId; - parsed.options.serviceTier = "priority"; - const raw = parsed._rawBody as Record; - raw.model = fastRow.baseId; - raw.service_tier = "priority"; - } - if (effortRow) { - parsed.modelId = effortRow.baseId; - parsed.options.reasoning = effortRow.effort; - const raw = parsed._rawBody as Record; - const rawReasoning = raw.reasoning; - raw.model = effortRow.baseId; - raw.reasoning = { - ...(rawReasoning && typeof rawReasoning === "object" && !Array.isArray(rawReasoning) - ? rawReasoning as Record - : {}), - effort: effortRow.effort, - }; - } - if (options.comboReplaySnapshot?.recoveredPlaintext) { - markBodyNonPersistable(parsed._rawBody); - } - toolBridgeMaps = buildToolBridgeMaps(parsed, translatorBudget); - if (previousResponseInputExpanded) parsed._previousResponseInputExpanded = true; - const providerContinuationCandidate = options.comboReplaySnapshot - ? options.comboReplaySnapshot.providerContinuation - : previousResponseProviderState(parsed.previousResponseId); - if (providerContinuationCandidate) parsed._providerContinuationCandidate = providerContinuationCandidate; - if (inboundClientThreadId) { - parsed._clientThreadId = inboundClientThreadId; - } else if ( - options.inboundWire === "anthropic" - && options.promptCacheKeyIsSharedCohort !== true - && typeof parsed.options.promptCacheKey === "string" - && parsed.options.promptCacheKey.trim().length > 0 - ) { - // Claude Code has no Codex parent-thread header, but its metadata.user_id is - // translated into a stable per-session prompt_cache_key. Use it as the replay - // thread identity so Gemini thought signatures are remembered by call_id for - // Anthropic Messages clients too (#1735/#1926). Keep `_clientThreadId` unset so - // existing provider session-id derivation (first-user-text fallback) is unchanged. - // Normalize through anthropicSessionKeyFromParts so overlong keys are hashed and - // trimming matches the affinity/session-key path exactly (no raw >128-char ids). - const normalizedCacheKey = anthropicSessionKeyFromParts({ - promptCacheKey: parsed.options.promptCacheKey, - // The enclosing branch already proves this is not the shared cohort. - promptCacheKeyIsSharedCohort: false, - }); - if (normalizedCacheKey) { - parsed._reasoningReplayScope = { clientThreadId: normalizedCacheKey }; - } - } - if (cursorClientThreadId) parsed._cursorClientThreadId = cursorClientThreadId; - } catch (err) { - if (isTranslatorBudgetExceededError(err)) { - return formatErrorResponse(413, "request_too_large", "request translation buffer exceeded the safe limit", { - code: "translation_buffer_limit", - }); - } - return formatErrorResponse(400, "invalid_request_error", err instanceof Error ? err.message : String(err)); - } - options.onRequestBodyRead?.(); - const responseStateOptions = (force = false): { force?: boolean; clientThreadId?: string } => ({ - ...(force ? { force: true } : {}), - ...(parsed._clientThreadId ? { clientThreadId: parsed._clientThreadId } : {}), - }); - const resolvedConversationId = conversationIdFromResponsesRequest({ - clientThreadId: parsed._clientThreadId, - sessionIdHeader: sessionIdHeaderFromRequest(req.headers), - threadIdHeader: req.headers.get("thread-id"), - cursorConversationId: parsed._cursorConversationId, - }); - bindTurnTerminationScope(parsed, resolvedConversationId); - const rememberKiroDeliveredFinalAnswer = (adapterName: string, response: unknown): void => { - if (adapterName === "kiro") rememberDeliveredFinalAnswer(parsed, response); - }; - // _clientThreadId remains the routing/continuation identity supplied by Codex. Replay state uses - // a dedicated raw conversation namespace so mixed headers that carry the same identity still - // match, and a shared/synthetic session_id cannot coalesce distinct thread/Cursor conversations. - // Keep an Anthropic prompt_cache_key scope already bound above (#1735/#1926). - if (!parsed._reasoningReplayScope) { - const reasoningReplayConversationId = reasoningReplayConversationIdFromResponsesRequest({ - clientThreadId: parsed._clientThreadId, - threadIdHeader: req.headers.get("thread-id"), - cursorConversationId: parsed._cursorConversationId, - sessionIdHeader: sessionIdHeaderFromRequest(req.headers), - }); - if (reasoningReplayConversationId) { - parsed._reasoningReplayScope = { clientThreadId: reasoningReplayConversationId }; - } - } - // Prefer a pre-populated id (routed Claude) over Responses headers that may be - // absent or synthetically injected (session_id from prompt_cache_key). - if (!logCtx.conversationId) { - logCtx.conversationId = resolvedConversationId; - } - logCtx.requestedModel = parsed.modelId; - logCtx.requestedEffort = parsed.options.reasoning; - logCtx.callerServiceTier = sanitizeLogMetadataString(parsed.options.serviceTier); - logCtx.requestedServiceTier = parsed.options.serviceTier; - logCtx.requestedSpeedLabel = requestLogSpeedLabel(parsed.options.serviceTier); - logCtx.configuredServiceTier = readConfiguredCodexServiceTier(); - logCtx.configuredSpeedLabel = requestLogSpeedLabel(logCtx.configuredServiceTier); - - let route: RouteResult; - let credentialDomainWasRewritten = false; - try { - // A `compaction_trigger` turn may name a bare native model the operator has - // no canonical OpenAI route for (#2901). Only the initial compaction route - // may fall back to the configured default provider; combo attempts and the - // later fallback/recovery re-routes keep the ordinary reservation. - const resolveRoute = (modelId: string) => options.comboAttempt - ? routeConcreteModel(config, modelId) - : parsed._compactionRequest === true - ? routeCompactionModel(config, modelId, evidenceFromBody(parsed._rawBody)) - : routeModel(config, modelId, evidenceFromBody(parsed._rawBody)); - const _sci = config.shadowCallIntercept; - let shadowRoute: RouteResult | undefined; - if (_sci?.enabled && _sci.model && isShadowSourceModel(parsed.modelId, _sci.sourceModels)) { - const sourcePrefix = shadowSourceModelPrefix(parsed.modelId, _sci.sourceModels)!; - let sourceIdentity = { providerName: OPENAI_CODEX_PROVIDER_ID, modelId: sourcePrefix }; - try { - const resolvedSource = routeConcreteModel(config, parsed.modelId); - sourceIdentity = { providerName: resolvedSource.providerName, modelId: sourcePrefix }; - } catch { /* Native Codex helper calls remain OpenAI-owned without an enabled OpenAI route. */ } - const targetRoute = resolveRoute(_sci.model); - if (shouldInterceptShadowCall(parsed.modelId, _sci.sourceModels, sourceIdentity, targetRoute)) { - credentialDomainWasRewritten = true; - const _sciOriginal = parsed.modelId; - parsed.modelId = _sci.model; - if (parsed._rawBody && typeof parsed._rawBody === "object") { - (parsed._rawBody as { model?: string }).model = _sci.model; - } - // Record the operator-configured prefix that matched, NOT the caller's raw model string. - // Matching is by prefix, so a caller can append arbitrary text and still intercept; that - // raw value would then land in usage.jsonl and /api/logs behind a pattern-based redactor - // that does not recognize every credential family. The prefix is a value the operator - // configured, so no caller-controlled string is persisted. - logCtx.shadowCallRewrittenFrom = sanitizeLogMetadataString( - shadowSourceModelPrefix(_sciOriginal, _sci.sourceModels), - ); - // Helpers must not resume/append into the parent thread's Cursor conversation. - parsed._cursorIsolateConversation = true; - shadowRoute = targetRoute; - } - } - if (parsed._compactionRequest === true) parsed._cursorIsolateConversation = true; - route = shadowRoute ?? resolveRoute(parsed.modelId); - logCtx.routeDecision = route.routeDecision; - } catch (err) { - if (err instanceof NoAvailableComboTargetsError) { - return comboUnavailable(err.comboId); - } - if (err instanceof NoEligiblePolicyCandidateError) { - // Persist the evaluation trace (per-candidate exclusions + the - // no-eligible reason) so failed policy requests stay auditable. - logCtx.routeDecision = err.trace; - } - return formatErrorResponse(404, "invalid_request_error", err instanceof Error ? err.message : String(err)); - } - - const hasUnexpandedPreviousResponse = !!parsed.previousResponseId - && parsed._previousResponseInputExpanded !== true; - // Exact account selectors are isolated from Pool-wide quota work. A canonical replay miss must - // also fail closed without polling quota upstream. Cached fallback state can still select a - // provider with native continuation support below. - const threadSpawn = isThreadSpawnRequest(req.headers); - const initialSubagentFallbackChain = threadSpawn && !options.comboAttempt - ? resolveSubagentFallbackChain(parsed, config) - : null; - const previewSelectionAdmission = threadSpawn - && !options.comboAttempt - && (route.codexAccountId === undefined || initialSubagentFallbackChain !== null) - ? codexAccountSelectionForTurn(options.turnAdmissionLease)?.() - : undefined; - const nativeMainRecoveryBlocked = isNativeMainTrafficBlocked(); - const nativeMainReadsForbidden = nativeMainRecoveryBlocked - || previewSelectionAdmission?.mainProfileDraining === true; - const previewSelectionOptions = { - nativeMainSelectionOnly: !nativeMainRecoveryBlocked - && previewSelectionAdmission?.mainProfileDraining === true, - }; - let selectedForwardHeaders = req.headers; - let subagentFallbackAccountId = config.activeCodexAccountId ?? null; - let subagentFallbackAccountPreview: SubagentPoolAccountPreview | undefined; - let subagentFallbackModelEligibleAccountIdsForModel: SubagentModelEligibleAccountIds | undefined; - let subagentQuotaFailureModel = parsed.modelId; - const parentThreadId = req.headers.get("x-codex-parent-thread-id")?.trim() ?? null; - const poolAffinityKey = codexPoolAffinityKey(req.headers) ?? null; - // Preview has to see the same lineage resolve does. Without it, a child's first turn is - // previewed as a cold pick and resolved onto the family account, and the subagent fallback - // then decides model eligibility against an account the request will never use. - // - // "The same" means both halves of the question the final resolution asks. The Authorization - // it will be given, because the lineage scope is an HMAC of exactly that header; and its own - // Pool-state predicate, because a fixed account selector and a request-owned credential - // deliberately create no affinity at all -- previewing a family binding for one of those would - // hand model fallback an account this request can never authenticate as. Read-only: the record - // is written by the resolution that binds, never by a preview that may own no Pool state. - const previewAuthHeaders = codexRouteCredentialDomainHeaders( - req, - route, - options, - credentialDomainWasRewritten, - ); - const poolLineage = previewCodexPoolLineage(previewAuthHeaders, options.codexAuthPolicy ?? config, { - accountId: route.codexAccountId, - modelId: route.modelId, - admission: options.admission, - requestScopedMainCredential: codexRouteCredentialOwnership( - previewAuthHeaders, - config, - route, - options, - ).requestScopedMainCredential, - }); - - try { - if ( - threadSpawn - && route.codexAccountId === undefined - && !(hasUnexpandedPreviousResponse && isCanonicalOpenAiForwardProvider(route.provider)) - ) { - await maybePrimeSubagentQuota(config, Date.now(), { nativeMainReadsForbidden }); - } - - // Subagent fallback must settle the final model/provider BEFORE route-dependent - // normalization (virtual models, effort caps, service tier, wire protocol). - // Preview the preferred Codex account without acquiring a probe lease or refreshing - // tokens — auth is resolved only after the final route is selected. - if ( - threadSpawn - && !options.comboAttempt - && (route.codexAccountId === undefined || initialSubagentFallbackChain !== null) - ) { - // The final resolveCodexAuthContext binds under codexQuotaScopeForModel(route.modelId), - // so the preview must read the same scope slot — an undefined scope would map to the - // "legacy" affinity bucket and never find a binding made under "shared" or a native - // model scope, making the preview diverge from the account that actually authenticates. - const fallbackChain = initialSubagentFallbackChain; - subagentFallbackModelEligibleAccountIdsForModel = await resolveSubagentFallbackModelEligibility({ - config, - fallbackChain, - nativeMainReadsForbidden, - resolver: options.resolveCodexModelEntitlements ?? resolveCodexModelEntitlements, - }); - const fallbackNow = Date.now(); - subagentFallbackAccountPreview = (modelId, previewNow, modelEligibleAccountIds) => previewCodexAccountForRequest( - poolAffinityKey, - config, - previewNow, - codexQuotaScopeForModel(modelId), - { ...previewSelectionOptions, modelEligibleAccountIds }, - modelId, - poolLineage, - ); - const previewAccountId = route.codexAccountId ?? subagentFallbackAccountPreview( - route.modelId, - fallbackNow, - subagentFallbackModelEligibleAccountIdsForModel?.(route.modelId), - ); - subagentFallbackAccountId = previewAccountId ?? config.activeCodexAccountId ?? null; - const fallback = applySubagentModelFallback( - parsed, - req.headers, - config, - previewAccountId, - fallbackNow, - unreadableEncryptedAgentTask, - previewSelectionOptions, - subagentFallbackAccountPreview, - subagentFallbackModelEligibleAccountIdsForModel, - fallbackChain, - candidateRoute => canPassThroughEncryptedV2AgentTask(candidateRoute, inboundWire), - ); - if (fallback) { - (logCtx as unknown as Record).subagentModelFallbackFrom = fallback.from; - (logCtx as unknown as Record).subagentModelFallbackTo = fallback.to; - if (isInjectionDebugEnabled()) { - injectionDebugLog(`[opencodex] subagent model fallback ${fallback.from} -> ${fallback.to}`); - } - } - subagentQuotaFailureModel = fallback?.to ?? parsed.modelId; - - if (fallback?.to && !slugsEquivalent(fallback.to, route.modelId)) { - try { - route = routeModel(config, fallback.to, evidenceFromBody(parsed._rawBody)); - credentialDomainWasRewritten = true; - logCtx.routeDecision = route.routeDecision; - } catch (err) { - if (err instanceof NoAvailableComboTargetsError) { - return comboUnavailable(err.comboId); - } - if (err instanceof NoEligiblePolicyCandidateError) { - logCtx.routeDecision = err.trace; - } - return formatErrorResponse(404, "invalid_request_error", err instanceof Error ? err.message : String(err)); - } - } - } - } finally { - previewSelectionAdmission?.release(); - } - - let recoveryFailureReason: AgentTaskRecoveryFailureReason | undefined; - // Native fallback and explicitly trusted direct Responses routes can consume ciphertext, - // so recover only after final route selection. - // - // Deliberately NOT gated on `threadSpawn` (#4089). Switching a live thread from a native - // ChatGPT model to a routed provider replays a backend-minted encrypted agent message on every - // later turn, and a model switch is not a spawn, so the spawn requirement failed the thread - // closed permanently without ever attempting recovery. The trust boundary is - // `recoveryAdmission()` in ./agent-task-recovery -- Codex originator, live native ChatGPT - // bearer, matching chatgpt-account-id, no inbound API key, no proxy-admission secret -- which - // admits only the owner of the session that would be spent. `threadSpawn` narrowed which of - // that owner's own requests could use their own session; it kept nobody else out. The combo - // gate above keeps its spawn requirement: that path has its own native-target filtering and - // per-attempt failover, and the reported defect is on this path. - if ( - inboundWire === "responses" - && agentTaskRecovery - && !isCanonicalOpenAiForwardProvider(route.provider) - && !options.comboAttempt - && !canPassThroughEncryptedV2AgentTask(route, inboundWire) - ) { - let recovered = restoreCachedEncryptedAgentTasks( - req, (body as { input?: unknown } | undefined)?.input, config, { parentThreadId }, - ) > 0; - unreadableEncryptedAgentTask = hasUnreadableEncryptedAgentTask( - (body as { input?: unknown } | undefined)?.input, - ); - if (unreadableEncryptedAgentTask) try { - const result = await recoverEncryptedAgentTaskWithResult( - req, - (body as { input?: unknown } | undefined)?.input, - agentTaskRecovery, - config, - { parentThreadId, abortSignal: options.abortSignal }, - ); - recovered = result.recovered; - recoveryFailureReason = result.recovered ? undefined : result.reason; - } catch { - recovered = false; - recoveryFailureReason = undefined; - } - if (recovered) { - unreadableEncryptedAgentTask = hasUnreadableEncryptedAgentTask( - (body as { input?: unknown } | undefined)?.input, - ); - if (!unreadableEncryptedAgentTask) { - try { - const reparsed = parseRequest(body); - const kept: Array = [ - "_previousResponseInputExpanded", - "_providerContinuation", - "_providerContinuationCandidate", - "_providerContinuationOwner", - "_cursorConversationId", - "_clientThreadId", - "_promptCacheKeyIsSharedCohort", - "_cursorClientThreadId", - "_reasoningReplayScope", - "_cursorIsolateConversation", - ]; - for (const key of kept) { - if (parsed[key] !== undefined) { - (reparsed as unknown as Record)[key] = parsed[key]; - } - } - bindTurnTerminationScope(reparsed, resolvedConversationId); - parsed = reparsed; - // The recovery mutated `body.input` in place, so `_rawBody` now carries decrypted task - // text. Bar it from the continuation cache before any recording path can reach it — - // that cache is persisted to disk, which would defeat the recovery cache's TTL. - markBodyNonPersistable(parsed._rawBody); - - // The ciphertext-only pass intentionally excludes routed candidates. Once recovery - // makes the assignment readable, run selection again with the full configured chain - // and keep the route in sync with any newly selected fallback. - const recoverySelectionAdmission = codexAccountSelectionForTurn(options.turnAdmissionLease)?.(); - const fallback = (() => { - try { - const recoveryNativeMainBlocked = isNativeMainTrafficBlocked(); - const recoverySelectionOptions = { - nativeMainSelectionOnly: !recoveryNativeMainBlocked - && recoverySelectionAdmission?.mainProfileDraining === true, - }; - const recoveryNow = Date.now(); - // Carry the entitlement filter through recovery too (#2509/#2623). The scope was - // already re-previewed per candidate here; the ELIGIBLE-ACCOUNT set was not, so a - // recovered assignment could select an account that is not entitled to the model - // and then fail closed at final auth — the same class of stale-selection bug as - // the quota scope, one layer over. - subagentFallbackAccountPreview = (modelId, previewNow, modelEligibleAccountIds) => previewCodexAccountForRequest( - poolAffinityKey, - config, - previewNow, - codexQuotaScopeForModel(modelId), - { ...recoverySelectionOptions, modelEligibleAccountIds }, - modelId, - poolLineage, - ); - const recoveryPreviewAccountId = subagentFallbackAccountPreview( - parsed.modelId, - recoveryNow, - subagentFallbackModelEligibleAccountIdsForModel?.(parsed.modelId), - ); - return applySubagentModelFallback( - parsed, - req.headers, - config, - recoveryPreviewAccountId, - recoveryNow, - false, - recoverySelectionOptions, - subagentFallbackAccountPreview, - subagentFallbackModelEligibleAccountIdsForModel, - ); - } finally { - recoverySelectionAdmission?.release(); - } - })(); - if (fallback) { - (logCtx as unknown as Record).subagentModelFallbackFrom = fallback.from; - (logCtx as unknown as Record).subagentModelFallbackTo = fallback.to; - if (isInjectionDebugEnabled()) { - injectionDebugLog(`[opencodex] subagent model fallback ${fallback.from} -> ${fallback.to}`); - } - } - subagentQuotaFailureModel = fallback?.to ?? parsed.modelId; - - if (fallback?.to && !slugsEquivalent(fallback.to, route.modelId)) { - try { - route = routeModel(config, fallback.to, evidenceFromBody(parsed._rawBody)); - credentialDomainWasRewritten = true; - logCtx.routeDecision = route.routeDecision; - } catch (err) { - if (err instanceof NoAvailableComboTargetsError) { - return comboUnavailable(err.comboId); - } - if (err instanceof NoEligiblePolicyCandidateError) { - logCtx.routeDecision = err.trace; - } - return formatErrorResponse( - 404, - "invalid_request_error", - err instanceof Error ? err.message : String(err), - ); - } - } - } catch { - unreadableEncryptedAgentTask = true; - } - } - } - } - - if (options.abortSignal?.aborted) return clientCancelledResponse(); - - // Encrypted child tasks may reach the canonical native backend or an explicitly trusted - // direct Responses route. This runs against the FINAL route so native-only fallback can - // rescue an incompatible primary without weakening combo behavior. - const finalRouteCanPassThroughEncryptedTask = !options.comboAttempt - && canPassThroughEncryptedV2AgentTask(route, inboundWire); - if ( - (route.combo !== undefined || !isCanonicalOpenAiForwardProvider(route.provider)) - && !finalRouteCanPassThroughEncryptedTask - && unreadableEncryptedAgentTask - ) { - return unreadableEncryptedAgentTaskResponse(recoveryFailureReason); - } - - // The guard above asks whether the CURRENT worker task is readable, and it only inspects the - // tail item. An `agent_message` that mixes readable text with backend ciphertext answers - // "readable" to that question at every position, so it passed -- and then - // `normalizeRoutedAgentMessages` refused to lower it, because lowering requires every part to - // be representable. The raw Responses passthrough serialized the private item as it stood, so - // backend ciphertext and an item type only the Codex backend declares reached a third-party - // provider, which answered `422 unknown item type "agent_message"` (#4454). - // - // The opaque-blob path already knows the repair: replace the undecryptable part with an - // omission marker, which leaves the item lowerable. It applied that repair only AFTER an - // upstream rejection. For a destination that cannot accept the private item under any - // circumstances, that round trip was never going to succeed and sent the ciphertext to find - // out, so do the repair here instead. Recovery above has already had its chance to turn the - // same bytes into real plaintext; only what it could not rescue reaches this. - if (inboundWire === "responses" && !finalRouteCanPassThroughEncryptedTask) { - // Only the raw Responses passthrough puts input items on the wire verbatim, so that is the - // only wire this has to repair: translated wires rebuild the body from parsed messages, where - // `inputContentParts` drops an encrypted part instead of forwarding it. The exemption is the - // canonical Codex backend alone, because it is the one destination that minted these bytes and - // can read them. `authMode: "forward"` is NOT that test -- a noncanonical forward gateway is - // somebody else's server that happens to be configured for passthrough, and it receives the - // ciphertext like any other third party. - // - // Combo children run this too. Each child carries its own `structuredClone` of the body - // (`concreteComboRequestBody`) and its own concrete route, so a sibling's repair is invisible - // here and a target that resolves to a routed Responses wire would otherwise send the - // ciphertext that the parent's own dispatch no longer does. - const wireProvider = resolveWireProtocolOverride( - route.providerName, - route.modelId, - route.provider, - inboundWire, - ); - if (wireProvider.adapter === "openai-responses" && !isCanonicalOpenAiForwardProvider(wireProvider)) { - const repaired = stripAgentMessageCiphertextInPlace((body as { input?: unknown } | undefined)?.input); - if (repaired > 0) { - console.warn( - `[opencodex] replaced ciphertext in ${repaired} replayed agent message(s) with an omission marker; the selected provider cannot read native ChatGPT ciphertext`, - ); - } - } - } - - // The canonical ChatGPT backend rejects previous_response_id, so a local replay miss leaves no - // safe way to recover the omitted history. Fail before auth, adapter construction, or upstream - // I/O instead of stripping the id and silently forwarding a context-free delta (#702). - // Codex recognizes previous_response_not_found on WebSocket errors and reconnects with its - // full input. A generic invalid_request_error instead terminates the task after cache expiry. - if ( - hasUnexpandedPreviousResponse - && isCanonicalOpenAiForwardProvider(route.provider) - ) { - return formatErrorResponse( - 400, - "previous_response_not_found", - "OpenAI forward continuation state is unavailable or expired; resend the full conversation without previous_response_id.", - ); - } - - if (hasUnexpandedPreviousResponse) { - const continuationProvider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire); - // Stateless destinations cannot resolve the omitted prefix. Stateful destinations may, - // but a lowered custom result still needs its call to recover the original wire type. - // Native function/custom continuations without lowering keep their upstream-owned state. - if (continuationProvider.adapter === "openai-responses" - && (continuationProvider.statelessResponses === true - || hasUnmappedRoutedCustomToolOutput(parsed._rawBody, continuationProvider.supportsResponsesCustomTools))) { - return formatErrorResponse( - 400, - "previous_response_not_found", - "Routed continuation requires unavailable local history; resend the full conversation without previous_response_id.", - ); - } - } - - // Captured before normalization: whether the CLIENT asked for SSE. The - // transport-neutral upstream-streaming policy below may force a bounded JSON - // upstream for reliability (#875); the answer must then be reframed to SSE - // for streaming clients. - const clientRequestedStream = parsed.stream; - await applyFinalRouteRequestNormalization({ - parsed, - route, - config, - req, - logCtx, - inboundWire, - inboundTransport: options.inboundTransport, - claudeGoAffinity: options.claudeGoAffinity, - }); - // Attribute local auth/cooldown failures to the public selector too; exact auth may fail before - // the normal post-resolution provider label is assigned. - if (route.codexAccountNamespace) { - logCtx.provider = `${route.providerName}-${route.codexAccountNamespace}`; - } - - if (options.abortSignal?.aborted) return clientCancelledResponse(); - // Resolve aliases/combo children before refusing helpers; do not spend main auth or host budget. - if (isCanonicalOpenAiForwardProvider(route.provider) - && isCodexReserveHelperUnsupported(options.codexAuthPolicy ?? config, route.modelId, - options.admission, options.visionDescribeTerminal === true)) { - return formatErrorResponse(400, "invalid_request_error", CODEX_RESERVE_HELPER_UNSUPPORTED_MESSAGE); - } - // Refuse an input that cannot plausibly fit the model context window before spending auth, - // circuit budget, or upstream bandwidth on a turn the provider will reject anyway (#1412). - // - // Compaction turns are exempt: Codex sends compaction_trigger BECAUSE context is full, so - // refusing the turn that shrinks the context would deadlock the client against the very - // limit this gate reports — it would be told to compact and then denied the compaction. - if (parsed._compactionRequest !== true) { - const inputAdmission = checkInputAdmission(parsed, route.provider, route.providerName, parsed.modelId, nativeContextLimits(config)); - if (!inputAdmission.admitted) { - // #1524: this is a LOCAL preflight refusal, not an upstream verdict. A policy or combo - // fallback must be able to skip this candidate and try one whose context window fits, - // instead of treating the first incompatible candidate as the end of the chain. The - // distinct code is what lets the fallback layer tell the two apart -- an upstream - // `context_length_exceeded` still stops, because retrying it elsewhere is guesswork. - if (clientRequestedStream && !options.comboAttempt) { - return streamingContextOverflowResponse( - parsed._responseModelId ?? parsed.modelId, - translatorBudget, - ); - } - return formatErrorResponse( - 413, - "input_admission_refused", - `Estimated input (~${inputAdmission.estimatedTokens} tokens) is far past the context window ` - + `of ${parsed.modelId} (${inputAdmission.ceiling} tokens). Start a new session or choose a ` - + `model with a larger context window.`, - ); - } - } - const preAuthHostKey = preAuthUpstreamHostCircuitKey(route, config); - if (preAuthHostKey) { - const admission = acquireUpstreamHostAdmission( - preAuthHostKey, - config.upstreamHostCircuitThreshold, - ); - if (admission.kind === "blocked") { - return upstreamHostCircuitOpenResponse(admission.retryAfterSeconds); - } - pendingHostAdmissionLease = admission.lease; - } - - let substituteMainCredential = false; - let callerAuthHeaders: Headers; - { - const finalAuth = await resolveResponsesCodexAuth(req, config, route, options, credentialDomainWasRewritten); - if (!finalAuth.ok) return finalAuth.response; - authCtx = finalAuth.authCtx; - selectedForwardHeaders = withClaudeNativeSession(finalAuth.headers, route.provider, options.claudeNativeSessionId); - callerAuthHeaders = withClaudeNativeSession(finalAuth.callerAuthHeaders, route.provider, options.claudeNativeSessionId); - substituteMainCredential = finalAuth.substituteMainCredential; - } - - route.provider = applyCodexAuthContextToProvider(route.provider, authCtx, route.codexAccountMode); - applyCodexAccountGatedWireNormalization(parsed, route, logCtx); - logCtx.provider = route.codexAccountNamespace - ? `${route.providerName}-${route.codexAccountNamespace}` - : formatCodexProviderForLog(route.providerName, codexLogAccountId(authCtx), config); - logCtx.accountLogLabel = codexAuthContextLogLabel(authCtx, config); - // A move is the expensive event: it discards the prefix warmed on the previous account. Record - // it as an event with its cause, so the operator reads it off one line instead of inferring it - // from account labels across many (#4546). - if (authCtx.kind === "pool" && authCtx.affinityDecision) { - logCtx.affinity = authCtx.affinityDecision.move; - logCtx.affinityReason = authCtx.affinityDecision.reason; - } - { - const binding = conversationStateBindingFromAuth(authCtx, poolAffinityKey); - if (binding) { - applyAccountChangeConversationStateScrub({ - body: parsed._rawBody, - parsed, - bindingKey: binding.bindingKey, - servingAccountId: binding.accountId, - logCtx, - }); - } - } - // Seed an account-derived scope before final adapter binding. Cursor never treats it as - // authoritative: bindRouteReasoningReplayScope replaces it with the exact route owner or a - // per-request fail-closed sentinel after the final provider and credential are known. - const identityScope = codexLogAccountId(authCtx); - if (identityScope) parsed._cursorIdentityScope = identityScope; - subagentFallbackAccountId = authCtx.kind === "pool" || authCtx.kind === "main-pool" - ? authCtx.accountId - : config.activeCodexAccountId ?? null; - - // OAuth providers: swap in a fresh access token (auto-refreshed) as the Bearer key, so the - // existing openai-chat / anthropic adapters authenticate with no change. - const isOAuth401ReplayProvider = ( - route.providerName === "xai" - || route.providerName === "github-copilot" - || route.providerName === "kiro" - || route.providerName === "google-antigravity" - || route.providerName === "orcarouter-oauth" - ) && route.provider.authMode === "oauth"; - let sentOAuthSnapshot: OAuthAccessSnapshot | undefined; - let replayOAuthCredentialSnapshot: Pick | undefined; - let anthropicPoolAccountId: string | null = null; - let anthropicPoolFailovers = 0; - // Generic OAuth rotation (#2568) for providers with no pool of their own. Bound to the account - // the request actually used, so a concurrent rotation cannot cool an innocent replacement. - let genericFailoverAccountId: string | null = null; - let genericFailovers = 0; - let oauthSelection = route.provider.authMode === "oauth" - ? captureOAuthAccountSelection(route.providerName) : null; - let servingOAuthSnapshot: OAuthAccessSnapshot | undefined; - // These owners also serve early passthrough and sidecar sends. A dispatch-time - // rebuild must update every later builder, without entering a later block's TDZ. - let adapter: ProviderAdapter; - let activeAdapter: ProviderAdapter; - let runTurnAdapter: ProviderAdapter; - let sameTargetRequest: AdapterRequest | undefined; - let sameTargetParsed: OcxParsedRequest | undefined; - let sameTargetToken = 0; - let transportToken = 0; - let imageTierBias = 0; - const invalidateSameTargetRequest = (): void => { transportToken += 1; }; - type DispatchBinding = - | { kind: "oauth"; selection: NonNullable; snapshot: OAuthAccessSnapshot } - | { kind: "api-key"; provider: OcxProviderConfig }; - const requestBindings = new WeakMap(); - const adapterBindings = new WeakMap(); - const rawRunTurns = new WeakMap>(); - const commitResolvedOAuthSelection = async ( - candidate: OAuthAccessSnapshot, - proactive = false, - anthropicReason?: AnthropicAccountSelectionReason, - ): Promise => { - const maxSelectionAttempts = 3; - for (let attempt = 0; attempt < maxSelectionAttempts; attempt++) { - if (!oauthSelection) return null; - const proactiveEnabled = route.providerName === "anthropic" - ? isAnthropicAccountPoolEnabled(config) - : (config.providers[route.providerName]?.oauthAccountFailover?.enabled - ?? config.oauthAccountFailover?.enabled) === true; - if (proactive && candidate.accountId !== oauthSelection.accountId && !proactiveEnabled) { - oauthSelection = captureOAuthAccountSelection(route.providerName); - if (!oauthSelection) return null; - candidate = route.providerName === "anthropic" - ? await getAnthropicPoolAccessSnapshot(oauthSelection.accountId) - : await getValidAccessSnapshotForAccount(route.providerName, oauthSelection.accountId, { requireUsableAccount: true }); - } - const committed = await commitOAuthAccountSelection(route.providerName, candidate.accountId, { - expectedSelection: oauthSelection, - expectedCredentialGeneration: candidate.generation, - requireUsableAccount: true, - }); - if (committed) { - if (route.providerName === "anthropic" && !commitAnthropicSelectionRouting( - candidate.accountId, oauthSelection, committed, - { config, sessionKey: anthropicSessionKey, reason: anthropicReason, expectedCredentialGeneration: candidate.generation }, - )) return null; - oauthSelection = committed; - servingOAuthSnapshot = candidate; - forgetGenericFailoverRoster(route.providerName); - return candidate; - } - // A newer manual choice wins over this request's old proposal, including A→B→A. - // Resolve that choice, not the rejected candidate, before trying admission again. - oauthSelection = captureOAuthAccountSelection(route.providerName); - if (!oauthSelection) return null; - candidate = route.providerName === "anthropic" - ? await getAnthropicPoolAccessSnapshot(oauthSelection.accountId) - : await getValidAccessSnapshotForAccount(route.providerName, oauthSelection.accountId, { requireUsableAccount: true }); - if (route.provider.googleMode === "cloud-code-assist" && !candidate.projectId) return null; - } - return null; - }; - const refreshResolvedOAuthSelection = async (sent: OAuthAccessSnapshot): Promise => { - const current = captureOAuthAccountSelection(route.providerName); - const unchanged = current?.accountId === oauthSelection?.accountId - && current?.revision === oauthSelection?.revision; - const candidate = unchanged ? await forceRefreshOAuthAccessSnapshot(sent) : sent; - const admitted = await commitResolvedOAuthSelection(candidate); - if (!admitted) throw new Error("OAuth selection changed during credential recovery"); - genericFailoverAccountId = admitted.accountId; - stampOAuthAccountLabel(logCtx, route.providerName, route.provider, admitted.accountId); - return admitted; - }; - /** - * Config generation captured where the serving credential is RESOLVED, not where the - * quota is written. A streaming turn is a long await, so a generation captured at write - * time cannot see a config or account change that happened earlier in the same turn — - * the case the fence exists for. Stays 0 for every provider without a passive quota. - */ - let passiveQuotaWriterGeneration = 0; - /** - * Apply a rotated account's FULL credential snapshot to the live route (#2568d). - * - * One helper for all three rotation sites on purpose. Each site used to inline the same four - * lines, and the divergence that produced was the bug: `apiKey` was swapped while the routing - * metadata paired with it stayed behind. - * - * Returns false when the snapshot cannot be used safely, and the caller must then abandon the - * rotation rather than send a half-applied identity: - * - * - Copilot pins its bearer to an account-scoped regional origin, so transport is re-resolved - * with the new account's `apiBaseUrl` instead of inheriting the previous account's host. The - * snapshot value is RESOLVED first: `rotatedProvider` is a clone of the FAILED account's - * provider, so passing a bare `undefined` origin let the transport resolver fall through its - * own `?? validateCopilotApiBaseUrl(provider.baseUrl)` step to the previous account's host — - * pairing B's bearer with A's accepted origin. Login and refresh always persist a resolved - * origin, so this fallback protects malformed or manually seeded credentials. - * - A Cloud Code Assist provider needs an account-matched project. Antigravity's refresh path - * tolerates project discovery failing, so a stored account can legitimately have no project; - * sending that account's bearer with the FAILED account's project is worse than not rotating. - */ - const applyFailoverSnapshot = async ( - snapshot: OAuthAccessSnapshot, - retryParsed: OcxParsedRequest = parsed, - ): Promise => { - if (route.provider.googleMode === "cloud-code-assist" && !snapshot.projectId) return false; - const committed = await commitResolvedOAuthSelection(snapshot); - if (!committed) return false; - snapshot = committed; - let rotatedProvider: OcxProviderConfig = { ...route.provider, apiKey: snapshot.accessToken }; - if (route.providerName === "github-copilot") { - rotatedProvider = resolveProviderTransport( - route.providerName, - rotatedProvider, - parsed.options.promptCacheKey, - resolveCopilotApiBaseUrl(snapshot.apiBaseUrl), - ) as OcxProviderConfig; - } - if (snapshot.projectId) rotatedProvider = { ...rotatedProvider, project: snapshot.projectId }; - route.provider = rotatedProvider; - if (route.providerName === "kiro") { - const kiroContext = { ...(snapshot.kiro ?? {}) }; - // Terminal-guard continuations are rebuilt from a shallow clone. Updating only the - // outer request pairs the new bearer with the failed account's region/profile on - // the retry. Keep both owners synchronized; for ordinary paths they are identical. - parsed._kiroAuthContext = kiroContext; - if (retryParsed !== parsed) retryParsed._kiroAuthContext = { ...kiroContext }; - } - // Re-stamp: a request that rotated accounts must be attributed to the account that actually - // served it. All three rotation sites funnel through here, so this is the only re-stamp - // needed -- and putting it anywhere else would let one of the three drift. - stampOAuthAccountLabel(logCtx, route.providerName, route.provider, snapshot.accountId); - if (route.providerName === "anthropic") { - anthropicPoolAccountId = snapshot.accountId; - logCtx.provider = formatAnthropicProviderForLog("anthropic", snapshot.accountId, config); - } else { - genericFailoverAccountId = snapshot.accountId; - } - sentOAuthSnapshot = snapshot; - replayOAuthCredentialSnapshot = { accountId: snapshot.accountId, generation: snapshot.generation }; - return true; - }; - const selectionIsCurrent = (binding: DispatchBinding | undefined): boolean => { - if (route.provider.authMode === "forward") return true; - if (!binding) return false; - if (binding.kind === "api-key") return providerApiKeySelectionIsCurrent(config, route.providerName, binding.provider); - const selected = captureOAuthAccountSelection(route.providerName); - const row = getAccountCredentialWithStatus(route.providerName, binding.snapshot.accountId); - return selected?.accountId === binding.selection.accountId && selected?.revision === binding.selection.revision - && !!row && !row.needsReauth && row.credential.expires > Date.now() - && credentialGeneration(row.credential) === binding.snapshot.generation; - }; - const resolveSelectionAdapter = (provider: OcxProviderConfig, retention = config.cacheRetention): ProviderAdapter => { - const resolved = resolveAdapter(provider, retention, route.providerName); - if (route.provider.authMode === "forward") return resolved; - const binding: DispatchBinding | undefined = route.provider.authMode === "oauth" - ? oauthSelection && servingOAuthSnapshot - ? { kind: "oauth", selection: { ...oauthSelection }, snapshot: servingOAuthSnapshot } - : undefined - : { kind: "api-key", provider: { ...route.provider } }; - if (binding) adapterBindings.set(resolved, binding); - const build = resolved.buildRequest.bind(resolved); - resolved.buildRequest = async (requestParsed, incoming) => { - const request = await build(requestParsed, incoming); - // Capture at adapter creation, never from mutable serving state after an await. - if (binding) requestBindings.set(request, binding); - return request; - }; - if (resolved.runTurn) { - rawRunTurns.set(resolved, resolved.runTurn.bind(resolved)); - resolved.runTurn = (requestParsed, incoming, emit) => runSelectedTurn(resolved, requestParsed, incoming, emit); - } - return resolved; - }; - const refreshDispatchAdapter = async (requestParsed: OcxParsedRequest): Promise => { - if (route.provider.authMode === "oauth") { - if (!servingOAuthSnapshot || !await applyFailoverSnapshot(servingOAuthSnapshot, requestParsed)) { - throw new Error("OAuth account selection changed before dispatch"); - } - } else { - const current = resolveCurrentProviderApiKeyTransport(config, route.providerName, route.provider); - if (!current) throw new Error("API key selection is unavailable before dispatch"); - route.provider = current; - } - adapter = activeAdapter = runTurnAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - ); - invalidateSameTargetRequest(); - return adapter; - }; - const refreshRunTurnAdapter = async (requestParsed: OcxParsedRequest): Promise => { - requestParsed._cursorIdentityScope = undefined; - requestParsed._cursorConversationId = undefined; - if (requestParsed._providerContinuation?.cursor) { - const { cursor: _oldCursor, ...rest } = requestParsed._providerContinuation; - requestParsed._providerContinuation = rest; - } - return refreshDispatchAdapter(requestParsed); - }; - const runSelectedTurn = async ( - selectedAdapter: ProviderAdapter, - ...[requestParsed, incoming, emit]: Parameters> - ): Promise => { - for (let attempt = 0; attempt < 3; attempt++) { - if (!selectionIsCurrent(adapterBindings.get(selectedAdapter))) selectedAdapter = await refreshRunTurnAdapter(requestParsed); - const binding = adapterBindings.get(selectedAdapter); - const run = rawRunTurns.get(selectedAdapter); - if (!run) throw new Error("Selected provider no longer supports this turn transport"); - let sent = false; - let refused = false; - // Both main and image-loop callers already acquired the initial pacing slot. - // Subsequent physical messages retain this adapter/credential and are paced normally. - const fetch = providerFetch(route.provider, options.codexWsRuntimeIdentity, { - providerName: route.providerName, modelId: route.modelId, pacingSlotAcquired: true, - beforeDispatch: () => { - if (sent) return; - if (!selectionIsCurrent(binding)) { - refused = true; - throw new Error("Account selection changed before the first turn dispatch"); - } - sent = true; - }, - }); - try { - await run(requestParsed, { ...incoming, providerFetch: fetch }, event => { if (!refused) emit(event); }); - } catch (error) { - if (!refused) throw error; - } - if (!refused) return; - // The adapter may map the guard's exception to an error event. Neither that - // event nor a refused send may escape before retrying the newly selected account. - selectedAdapter = await refreshRunTurnAdapter(requestParsed); - } - throw new Error("Account selection changed repeatedly before turn dispatch"); - }; - const oauthDispatch = (wireRequest: AdapterRequest, requestParsed = parsed): ProviderFetchOptions["dispatchOverride"] => { - if (route.provider.authMode === "forward") return undefined; - return async (input, init, execute) => { - let destination = input; - let dispatchInit = init; - for (let attempt = 0; attempt < 3; attempt++) { - if (selectionIsCurrent(requestBindings.get(wireRequest))) { - const fetchImpl = (route.provider as OcxProviderConfig & { fetch?: typeof globalThis.fetch }).fetch ?? execute; - const binding = requestBindings.get(wireRequest); - const snapshot = route.providerName === "anthropic" && anthropicPoolAccountId && binding?.kind === "oauth" - ? binding.snapshot : undefined; - const writerGeneration = snapshot ? captureConfigGeneration() : 0; - const sentHeaders = snapshot ? new Headers(dispatchInit.headers) : undefined; - const ownsBearer = snapshot !== undefined - && sentHeaders?.get("authorization") === `Bearer ${snapshot.accessToken}` - && !sentHeaders?.has("x-api-key"); - // Reselection can choose a provider override instead of the supplied executor. - const response = await fetchImpl(destination, { ...dispatchInit, redirect: "manual" }); - // Observe each physical response before retries replace it. The binding belongs to - // this dispatch, so a manual switch cannot file A's headers against B. Header - // overrides and credential replacement make ownership unprovable: skip those writes. - if (ownsBearer && snapshot) { - try { - const current = getAccountCredentialWithStatus("anthropic", snapshot.accountId); - if (current && !current.needsReauth && credentialGeneration(current.credential) === snapshot.generation) { - recordAnthropicAccountQuotaFromHeaders(snapshot.accountId, response.headers, writerGeneration); - } - } catch { /* best-effort observation cannot fail the response */ } - } - return response; - } - const nextAdapter = await refreshDispatchAdapter(requestParsed); - const rebuilt = await nextAdapter.buildRequest(requestParsed, { - headers: selectedForwardHeaders, translatorBudget, - ...(imageTierBias > 0 ? { imageTierBias } : {}), - }); - const bodySize = checkOutboundBodySize(rebuilt.body, config.maxUpstreamBodyBytes); - if (!bodySize.admitted) { - rebuilt.releaseBodyObservation?.(); - return formatErrorResponse(413, "outbound_body_too_large", describeOutboundBodyRefusal(bodySize)); - } - const headers = new Headers(dispatchInit.headers); - for (const name of Object.keys(wireRequest.headers)) headers.delete(name); - for (const [name, value] of Object.entries(rebuilt.headers)) headers.set(name, value); - wireRequest.releaseBodyObservation?.(); - Object.assign(wireRequest, rebuilt); - const binding = requestBindings.get(rebuilt); - if (binding) requestBindings.set(wireRequest, binding); - else requestBindings.delete(wireRequest); - sameTargetRequest = wireRequest; - sameTargetParsed = requestParsed; - sameTargetToken = transportToken; - destination = rebuilt.url; - dispatchInit = { ...dispatchInit, method: rebuilt.method, headers, body: rebuilt.body }; - bindRouteReasoningReplayScope({ parsed: requestParsed, providerName: route.providerName, provider: route.provider, - adapterName: nextAdapter.name, oauthCredentialSnapshot: replayOAuthCredentialSnapshot }); - // The next iteration validates synchronously and calls fetch in that same turn. - } - throw new Error("OAuth account selection changed repeatedly before dispatch"); - }; - }; - const anthropicSessionKey = route.providerName === "anthropic" && route.provider.authMode === "oauth" - ? anthropicSessionKeyFromParts({ - sessionIdHeader: sessionIdHeaderFromRequest(req.headers), - threadIdHeader: req.headers.get("thread-id"), - promptCacheKey: typeof parsed.options.promptCacheKey === "string" ? parsed.options.promptCacheKey : null, - clientThreadId: typeof parsed._clientThreadId === "string" ? parsed._clientThreadId : null, - promptCacheKeyIsSharedCohort: options.promptCacheKeyIsSharedCohort === true, - }) - : null; - if (route.provider.authMode === "oauth") { - try { - if (route.providerName === "anthropic" && isAnthropicAccountPoolEnabled(config)) { - const selection = resolveAnthropicAccountForSession(anthropicSessionKey, config); - if (!selection.accountId) { - if (selection.reason === "all-cooled") { - const retryAfterSec = getAnthropicPoolRetryAfterSeconds(); - return formatErrorResponse( - 429, - "rate_limit_error", - "All Anthropic OAuth accounts are temporarily rate-limited", - retryAfterSec !== null ? { retryAfter: String(retryAfterSec) } : undefined, - ); - } - return formatErrorResponse(401, "authentication_error", "No eligible Anthropic OAuth account available"); - } - const admitted = await commitResolvedOAuthSelection(await getAnthropicPoolAccessSnapshot(selection.accountId), true, selection.reason); - if (!admitted) return formatErrorResponse(409, "conflict_error", "OAuth account selection changed; retry the request"); - anthropicPoolAccountId = admitted.accountId; - route.provider = { ...route.provider, apiKey: admitted.accessToken }; - logCtx.provider = formatAnthropicProviderForLog("anthropic", admitted.accountId, config); - } else { - // Prefer the account with known headroom BEFORE the first attempt. Rotation alone - // only reacts to a 429, so a turn could open on an account a previous probe already - // measured as spent. A null answer means "use the active account", so every provider - // without quota evidence keeps the resolution it has today. - const preferredAccountId = isGenericFailoverProvider(route.providerName, route.provider) - ? preferredInitialAccount(config, route.providerName) - : null; - // Resolved account-scoped, NOT through failoverAccountSnapshot: that helper marks a - // rotation site, and rotation sites must apply their credential through - // applyFailoverSnapshot's pairing rules. This is initial resolution — the code below - // already pairs the snapshot's Kiro metadata, Copilot origin and Antigravity project - // with this same bearer, exactly as it does for the active account. - let usedPreferredAccount = preferredAccountId !== null; - let resolved: OAuthAccessSnapshot; - if (preferredAccountId) { - try { - // `requireUsableAccount` makes a removed OR reauth-flagged account throw from - // inside the resolver's own store read. Without it a revoked account resolves - // successfully — its credential is still readable — and the request would - // dispatch on an account already known to need a fresh login. - resolved = await getValidAccessSnapshotForAccount( - route.providerName, - preferredAccountId, - { requireUsableAccount: true }, - ); - } catch { - // The roster is read behind a short TTL, so a preferred account can be removed - // or flagged for reauth in the window after it was cached. Resolving it then - // throws, and a PREFERENCE that turns a healthy request into a 401 is worse - // than no preference at all — the active account is still perfectly usable. - // Drop the stale roster so the next request re-reads it, and carry on. - forgetGenericFailoverRoster(route.providerName); - usedPreferredAccount = false; - resolved = await getValidAccessTokenSnapshot(route.providerName); - } - } else { - resolved = await getValidAccessTokenSnapshot(route.providerName); - } - // A Cloud Code Assist account needs its own project. Antigravity's refresh path - // tolerates project discovery failing, so a stored account can legitimately have - // none — and a PREFERENCE must never turn a working request into an error. Fall - // back to the ordinary active-account resolution instead, which is exactly what - // would have happened had the preference never existed. - if (usedPreferredAccount && route.provider.googleMode === "cloud-code-assist" && !resolved.projectId) { - resolved = await getValidAccessTokenSnapshot(route.providerName); - usedPreferredAccount = false; - } - const admitted = await commitResolvedOAuthSelection(resolved, true); - if (!admitted) return formatErrorResponse(409, "conflict_error", "OAuth account selection changed; retry the request"); - if (admitted.accountId !== resolved.accountId) usedPreferredAccount = true; - resolved = admitted; - replayOAuthCredentialSnapshot = { - accountId: resolved.accountId, - generation: resolved.generation, - }; - if (isOAuth401ReplayProvider) sentOAuthSnapshot = resolved; - route.provider = { ...route.provider, apiKey: resolved.accessToken }; - // Attribution is independent of failover (#2699): stamped from the resolved snapshot - // itself, not from inside the `isGenericFailoverProvider` branch below, so a future - // narrowing of that predicate cannot silently switch attribution off. - stampOAuthAccountLabel(logCtx, route.providerName, route.provider, resolved.accountId); - // Remember which account actually served this request so a 429 cools THAT one, not - // whichever account is active by the time the response comes back (#2568). - if (isGenericFailoverProvider(route.providerName, route.provider)) { - genericFailoverAccountId = resolved.accountId; - // Advance the pool cursor only now that this account is actually admitted. The - // helper returns immediately unless the kernel is on AND the strategy is - // round-robin, so quota and fill-first pools reach it without being touched. - noteGenericPoolSelection(config, route.providerName, resolved.accountId); - } - // Anthropic is excluded from isGenericFailoverProvider -- its own pool owns affinity and - // a fail-closed local-cli credential rule -- so without this stamp its identity is - // dropped whenever the pool flag is off, and a later 429 has no account to cool. Reactive - // failover needs only the id: no affinity bind, no promotion, no quota-ranked pick. Those - // are proactive and stay behind anthropicAccountPool.enabled. - if (route.providerName === "anthropic" && hasAnthropicFailoverQuorum()) { - anthropicPoolAccountId = resolved.accountId; - } - // Captured beside the account it fences, so the two can never disagree. - if (hasPassiveAccountQuota(route.providerName)) { - passiveQuotaWriterGeneration = captureConfigGeneration(); - } - if (route.providerName === "kiro") { - // `{}` is intentional: this is an account-scoped request with no stored routing metadata. - // Only genuinely accountless adapter calls leave the context undefined and use local/env fallback. - parsed._kiroAuthContext = { ...(resolved.kiro ?? {}) }; - } - // Project identity belongs to the admitted account on EVERY request, including - // the request after a pool transition made that account the persisted active one. - if (route.provider.googleMode === "cloud-code-assist") { - if (!resolved.projectId) return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(new Error("Cloud Code Assist account project is unavailable"))); - route.provider = { ...route.provider, project: resolved.projectId }; - } - } - } catch (err) { - if (err instanceof UnsupportedOAuthProviderError) { - const safeProviderName = redactSecretString(route.providerName); - return formatErrorResponse( - 400, - "invalid_request_error", - `${redactSecretString(err.message)}. Remove or reconfigure provider '${safeProviderName}' in the OpenCodex configuration.`, - ); - } - return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(err)); - } - } - // Key-auth twin of the OAuth preference above: pick a warm key BEFORE the first attempt when - // the committed one is already cooling, instead of spending the request earning a 429 the - // runtime could already predict. The picker refuses to override a healthy committed key and - // returns null without a configured strategy, so an ordinary install evaluates one predicate. - // - // It RETURNS a rebuilt route rather than mutating one, and the assignment has to land here -- - // ahead of the transport pin below, the adapterProvider copy that follows it, and the request - // the HTTP path bakes later. The image bridge and web search read route.provider directly and - // have no stale-selection re-read to save them, so ordering is the whole correctness argument. - // - // The Transport variant, not the bare picker: the picker answers with the PERSISTED row, and - // a built-in provider stored in its valid minimal form would lose the adapter id, base URL - // and static headers registry backfill supplies, throwing `Unknown adapter: undefined`. - const proactiveKeyProvider = selectProactiveApiKeyTransport( - config, - route.providerName, - route.provider, - parsed.options.promptCacheKey, - ); - if (proactiveKeyProvider) route.provider = proactiveKeyProvider; - route.provider = resolveProviderTransport( - route.providerName, - route.provider, - parsed.options.promptCacheKey, - route.providerName === "github-copilot" && route.provider.authMode === "oauth" - ? resolveCopilotApiBaseUrl(sentOAuthSnapshot?.apiBaseUrl) - : undefined, - ); - let adapterProvider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire); - const stripClaudeMainAuth = options.stripClaudeMainAuthForNoncanonicalForward === true - && !isCanonicalOpenAiForwardProvider(adapterProvider) - && ((adapterProvider.adapter === "openai-responses" && adapterProvider.authMode === "forward") - || providerConsumesCallerAuthorization(adapterProvider)); - if (stripClaudeMainAuth) { - releaseCodexAuthContextProbeLease(authCtx); - authCtx = { kind: "main", accountId: null }; - route.provider = stripCodexRuntimeProviderFields(route.provider); - adapterProvider = stripCodexRuntimeProviderFields(adapterProvider); - selectedForwardHeaders = new Headers(selectedForwardHeaders); - selectedForwardHeaders.delete("authorization"); - selectedForwardHeaders.delete("chatgpt-account-id"); - delete route.codexAccountMode; - delete route.codexAccountId; - delete route.codexAccountNamespace; - logCtx.provider = route.providerName; - delete logCtx.accountLogLabel; - } - adapter = resolveSelectionAdapter(adapterProvider, config.cacheRetention); - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: adapterProvider, - adapterName: adapter.name, - oauthCredentialSnapshot: replayOAuthCredentialSnapshot, - codexAuthContext: authCtx, - forwardHeaders: selectedForwardHeaders, - }); - if (!logCtx.conversationId && parsed._cursorConversationId) { - logCtx.conversationId = normalizeLogConversationId(parsed._cursorConversationId); - } - logCtx.providerAdapter = adapter.name; - // Ordinary requests receive one durable attempt only after their final initial - // adapter is resolved. Combo children own their attempt and retries keep it. - if (!options.comboAttempt && !logCtx.activeAttempt) { - const attempt = beginRequestAttempt( - (logCtx.attempts?.length ?? 0) + 1, - logCtx.provider, - route.modelId, - adapter.name, - ); - logCtx.activeAttempt = attempt; - logCtx.activeAttemptStartedAt = Date.now(); - (logCtx.attempts ??= []).push(attempt); - } - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, adapter.name, logCtx.accountLogLabel); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, adapterProvider, adapter.name); - runTurnAdapter = adapter; - if (adapter.runTurn) { - recordAdapterTierMetadata(logCtx, adapter.tierLogForRunTurn?.(parsed)); - } - // Optional route-identity linkage for attempt correlation (CL-09 consumes it). The slot - // resolves to null unless an opt-in subsystem registered a linker, so an install without - // routing profiles does no work here and loads no additional module. The non-throwing - // guarantee lives in the slot helper. - if (logCtx.activeAttempt && !logCtx.activeAttempt.labRouteSubjectId) { - const passiveSubjectId = resolvePassiveRouteSubjectId( - config, - route.providerName, - route.modelId, - route.provider, - inboundWire, - ); - if (passiveSubjectId) logCtx.activeAttempt.labRouteSubjectId = passiveSubjectId; - } - const isPassthrough = "passthrough" in adapter && !!adapter.passthrough; - - const rawInput = (parsed._rawBody as { input?: unknown }).input; - if (!isPassthrough && Array.isArray(rawInput) && rawInput.some( - item => item !== null && typeof item === "object" && item.type === "computer_call_output", - )) { - return formatErrorResponse( - 400, - "invalid_request_error", - "computer_call_output requires a Responses passthrough route; send screenshots as user input_image content on translated routes.", - ); - } - - if (adapter.name === "kiro" && parsed.previousResponseId && !parsed._previousResponseInputExpanded) { - return formatErrorResponse( - 400, - "invalid_request_error", - "Kiro continuation state is missing; start a new session instead of reusing this previous_response_id.", - ); - } - - let openAiSidecar: ResolvedOpenAiForwardSidecar | undefined; - const visionDescribeTerminal = options.visionDescribeTerminal === true; - const routedCompaction = parsed._compactionRequest === true - && !isCanonicalOpenAiForwardProvider(route.provider); - const needsOpenAiVision = !visionDescribeTerminal - && shouldResolveOpenAiVisionSidecar(config, route.provider, route.modelId, parsed, route.providerName); - const needsOpenAiSearch = !routedCompaction && !adapter.runTurn - && (shouldResolveOpenAiWebSearchSidecar(config, parsed, isPassthrough) - || shouldResolveOpenAiPassthroughWebSearchBridge(route.provider, parsed, isPassthrough)); - if (needsOpenAiVision || needsOpenAiSearch) { - try { - const candidates = listOpenAiForwardSidecarCandidates(config); - let sidecarAuth = options.openAiSidecarAuth; - if (!sidecarAuth && options.allowStoredOpenAiSidecarAuth === true - && route.codexAccountId === undefined - && candidates.some(candidate => candidate.accountMode === "direct") - && tryClaimStoredSidecarMainProfile(options.turnAdmissionLease)) { - // Request-local helper authority only: never promote this pair to caller, primary, - // or retry credentials. Claim before reading so profile switches remain fenced. - try { - const { getMainAccountToken } = await import("../../codex/main-account"); - const token = getMainAccountToken(); - if (token) sidecarAuth = captureExplicitOpenAiCallerAuth(new Headers({ - authorization: `Bearer ${token.accessToken}`, "chatgpt-account-id": token.chatgptAccountId, - }), config); - } catch { /* stored enrichment is optional */ } - } - // Preserve explicit OpenAI helper auth across route changes without returning it to - // primary-provider headers or alternate-main retry. The resolver revalidates scope. - const sidecarHeaders = new Headers(req.headers); - sidecarHeaders.delete("authorization"); - sidecarHeaders.delete("chatgpt-account-id"); - if (sidecarAuth) { - sidecarHeaders.set("authorization", sidecarAuth.authorization); - sidecarHeaders.set("chatgpt-account-id", sidecarAuth.chatgptAccountId); - } - openAiSidecar = await resolveFirstUsableOpenAiSidecar( - candidates, - sidecarHeaders, - config, - { - admission: options.admission, - codexAuthPolicy: options.codexAuthPolicy, - // Account-qualified native routes are passthrough, so their in-turn helper is vision. - // Scope its cooldown and outcome to the helper model, not the routed text model. - ...(route.codexAccountId !== undefined - ? { exactAccount: { accountId: route.codexAccountId, modelId: resolveOpenAiVisionModel(config) } } - : {}), - beginCodexAccountSelection: codexAccountSelectionForTurn(options.turnAdmissionLease), - }, - ); - } catch (err) { - // Sidecars are optional helpers for an otherwise independent routed turn. - // An unavailable/cooling/expired Multi credential disables the helper; it - // must not turn a valid routed-provider request into a Codex-auth failure. - if ( - !(err instanceof CodexPoolAuthenticationError) - && !(err instanceof CodexAuthContextError) - && !(err instanceof CodexAccountCooldownError) - && !(err instanceof CodexThreadAffinityExpiredError) - && !(err instanceof CodexMainProfileDrainingError) - ) throw err; - } - } - - // Vision sidecar: the routed model can't see images (provider.noVisionModels). Describe each - // attached image through the selected sidecar backend and replace it with text BEFORE the main - // call, so the text-only model can reason about it. - // Terminal describe fence (roadmap 180): the sidecar's OWN loopback describe - // call must never plan another describe. The flag arrives from the Chat - // surface (whose bridge rebuilds headers) or as the raw header for native - // Responses callers. Marked + text-only routed model → strip, depth cap 1. - const visionPlan = visionDescribeTerminal - ? undefined - : planVisionSidecar(config, route.provider, route.modelId, parsed, openAiSidecar, { - admission: options.admission, codexAuthPolicy: options.codexAuthPolicy, providerName: route.providerName, - }); - const recordSidecarOutcome = openAiSidecar?.recordOutcome; - if (visionPlan) { - await describeImagesInPlace( - parsed, - visionPlan, - openAiSidecar?.headers ?? selectedForwardHeaders, - options.abortSignal, - recordSidecarOutcome, - translatorBudget, - ); - } else if (requiresVisionPreprocessing(config, route.provider, route.modelId, route.providerName)) { - // Image capability is not positively proven but no sidecar plan is dispatchable: fail closed. - // Never forward raw image bytes to an unverified upstream. - stripImagesInPlace(parsed, translatorBudget); - } - - const recordTerminalOutcomes = options.recordTerminalOutcomes !== false; - let responseCompletionNotified = false; - let responseCompletionCancelled = false; - const cancelResponseCompletion = (): void => { responseCompletionCancelled = true; }; - const notifyResponseComplete = (response: { status?: unknown; model?: unknown }): void => { - if (responseCompletionNotified || responseCompletionCancelled - || options.abortSignal?.aborted || req.signal.aborted - || response.status !== "completed" - || typeof response.model !== "string" || !response.model.trim()) return; - responseCompletionNotified = true; - options.onResponseComplete?.(response.model); - }; - - const continuationStateForResponse = ( - emitted?: OcxProviderContinuationState, - ): OcxProviderContinuationState | undefined => { - const cursorConversationId = parsed._cursorConversationId; - const inherited = providerContinuationPayload(parsed._providerContinuation); - const emittedPayload = providerContinuationPayload(emitted); - if (!emittedPayload && !inherited && !cursorConversationId) return undefined; - const merged = mergeProviderContinuationPayload( - inherited ?? {}, - emittedPayload ?? {}, - ) as OcxProviderContinuationState; - if (cursorConversationId) { - merged.cursor = { ...(merged.cursor ?? {}), conversationId: cursorConversationId }; - } - return parsed._providerContinuationOwner - ? { ...merged, __ocxOwner: { ...parsed._providerContinuationOwner } } - : merged; - }; - - // Remote compaction v2 on a ROUTED model: Codex sent `compaction_trigger` and requires exactly - // one `{type:"compaction"}` output item (codex-rs compact_remote_v2.rs). Passthrough handles it - // natively upstream; here we run the routed model as a plain summarizer — no tools, no web-search - // sidecar — and the bridge appends the synthetic compaction item (src/responses/compaction.ts). - // A Responses-shaped wire does not imply support for Codex's private - // `compaction_trigger` item — only the canonical ChatGPT backend speaks that - // contract. An API-key gateway would receive the trigger, answer with an ordinary - // message, and leave Codex fataling on a missing compaction item (#422). - const commitReasoningReplayServingRoute = (outboundHeaders?: HeadersInit): void => { - commitReasoningReplayServingIdentity(parsed._reasoningReplayScope); - rememberServingConversationStateIssuer(authCtx, poolAffinityKey); - // History has no model namespace. Record the account that actually accepted this - // final attempt, after refresh/failover, rather than guessing from mutable affinity. - // Recording is relay state. With the feature off there is no relay, so building an owner - // registry for it is out of scope for this request. - if (outboundHeaders && isCanonicalOpenAiForwardProvider(route.provider) && contextRelayActivated()) { - recordContextSessionOwner(resolveContextPrincipal(req, config, options.admission), req.headers, - route.provider.baseUrl, authCtx, new Headers(outboundHeaders), substituteMainCredential); - } - }; - if (routedCompaction) { - delete parsed.context.tools; - delete parsed._webSearch; - delete parsed.options.toolChoice; - delete parsed.options.parallelToolCalls; - // The compaction turn is a plain prose summary; a surviving structured-output format - // would force schema-constrained JSON into the synthetic compaction item. The flag and - // the raw `text` control go too: the key-mode openai-responses adapter builds from - // _rawBody, so a surviving format there would still reach the upstream. (The Kiro - // guard no longer reads _rawBody.text; it refuses structured output only.) - delete parsed.options.textFormat; - delete parsed._structuredOutput; - if (parsed._rawBody && typeof parsed._rawBody === "object") { - delete (parsed._rawBody as Record).text; - } - parsed.context.messages.push({ role: "user", content: COMPACT_PROMPT, timestamp: Date.now() }); - } - - let routedNamespaceToolAliases: RoutedNamespaceToolAliases = new Map(); - let plaintextV2AgentMessageToolNames: ReadonlySet = new Set(); - let plaintextV2AgentMessageAliasedToolNames: ReadonlySet = new Set(); - let routedMuseToolNameAliases: MuseToolNameAliases = new Map(); - const refreshRequestToolAliases = (builtRequest: AdapterRequest): void => { - routedNamespaceToolAliases = builtRequest.convertedRoutedNamespaceToolAliases ?? new Map(); - plaintextV2AgentMessageToolNames = builtRequest.plaintextV2AgentMessageToolNames ?? new Set(); - plaintextV2AgentMessageAliasedToolNames = builtRequest.plaintextV2AgentMessageAliasedToolNames ?? new Set(); - routedMuseToolNameAliases = builtRequest.convertedMuseToolNameAliases ?? new Map(); - }; - - // One transient-retry budget for the whole LOGICAL request, read ABOVE the passthrough branch - // so that branch shares it too. It used to be a local declared below, which put it in the - // temporal dead zone for the passthrough sends and left each recovery leg taking the helper's - // fresh default of 3. It is now a holder carried on options, so a combo child inherits the - // parent's spend instead of starting over per target -- both halves of the measured - // amplification in #4546. - const sendBudget = options.sendBudget ?? createRequestExecutionBudget(); - // The root workflow is the user-visible task. A per-request cap cannot bound a fan-out that - // sends once per child seven hundred times, so every send charged to the request is charged - // to the root as well (#4546). - const workflowRootId = req.headers.get("x-codex-parent-thread-id")?.trim() || undefined; - const noteTransientSends = (used: number): void => { - const charged = Math.max(0, used); - sendBudget.used += charged; - chargeWorkflowSends(workflowRootId, charged); - }; - // Refused before any dispatch, and deliberately not by evicting the root's ledger entry: - // dropping the record to make room would hand the fan-out a fresh allowance, which is the - // laundering this ceiling exists to stop. The client is told the task needs a new grant - // rather than being given a synthetic upstream error. - if (workflowSendCeilingReached(workflowRootId)) { - // A log context exists here, unlike at HTTP admission, so the row this request writes is - // marked synthetic rather than reading as a request that vanished with zero sends. - return workflowRefusalResponse("workflow-sends-exhausted", logCtx, undefined, workflowRootId); - } - // No floor. Math.max(1, ...) meant an exhausted request still funded one send on every - // recovery leg, so a bounded per-leg allowance never became a bounded per-request one. - const remainingTransientSendBudget = (budget: number): number => - isRequestExecutionBudget(sendBudget) - ? sendBudget.remainingBaseSends(budget) - : Math.max(0, budget - sendBudget.used); - // The adapter contract needs the full budget, not just the counter. options.sendBudget is - // typed as the narrow holder so a caller that predates this can still pass one, so narrow it - // once here rather than asserting at each adapter call site. - const adapterSendBudget = isRequestExecutionBudget(sendBudget) ? sendBudget : undefined; - /** - * Records an adapter's OWN inner retries against this attempt. - * - * Ordinal 1 is the send each call site already recorded through `noteAttemptSend`, so only - * the extra physical sends are added here and an adapter that does not retry internally - * leaves its log byte-for-byte as it was. Kiro reaches roughly eighteen sends per call and - * Cursor re-sends a whole turn, and both reported one; a count that cannot be observed - * cannot be pinned by a regression, which is why the instrumentation precedes the cap. - */ - const noteAdapterPhysicalSend = ( - inputTokens: number | undefined, - send: { ordinal: number; recovery?: AttemptRecoveryKind }, - ): void => { - if (send.ordinal <= 1) return; - noteAttemptSend(logCtx.activeAttempt, inputTokens, send.recovery); - }; - const sendBudgetExhausted = (): boolean => - remainingTransientSendBudget(TRANSIENT_RETRY_MAX_ATTEMPTS) === 0; - /** - * A credential hop reserves the send its own replay will make, and that replay is a recovery - * leg. The leg must SPEND the hop's reservation instead of taking a second one: the - * final-recovery reserve is single, so a rebuild that reserved on top of a hop would be - * refused and the request would answer with a synthetic 502 in place of the real 429 the hop - * was recovering from. - */ - let pendingHopPermit: SingleUseDispatchPermit | undefined; - /** - * How many sends a recovery leg may make, and the permit that authorises the last one. - * - * The base allowance is spent first. Once it is gone a recovery class may still draw the - * single shared final-recovery reserve -- which is what keeps the validated sanitized rebuild - * after a 5xx streak alive at four total sends -- but an account move and a rebuild cannot - * each take one. `countedExternally` is set because these legs run through the retry helper, - * which reports the same send again through `onSendsConsumed`. - */ - const recoverySendAllowance = ( - cap: number, - sendClass: SendClass, - targetKey: string, - ): { attempts: number; permit?: SingleUseDispatchPermit } => { - const base = remainingTransientSendBudget(cap); - if (base > 0) return { attempts: base }; - if (pendingHopPermit) { - const hopPermit = pendingHopPermit; - pendingHopPermit = undefined; - return { attempts: 1, permit: hopPermit }; - } - if (!isRequestExecutionBudget(sendBudget)) return { attempts: 0 }; - const decision = sendBudget.reserveDispatch({ sendClass, targetKey, countedExternally: true }); - return decision.allowed ? { attempts: 1, permit: decision.permit } : { attempts: 0 }; - }; - /** - * One credential hop of this logical request, admitted by the INTERSECTION of two bounds. - * - * `GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST` and `ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST` - * stay exactly as they are: they bound rotation within one credential roster. What neither - * can see is everything else this request already sent, so three hops layered on a spent - * budget still reached upstream three more times. A hop now happens only when its own layer - * cap AND the shared budget both permit it, and the smaller of the two wins. - * - * `countedExternally` is for the hops whose replay goes out through the retry helper, which - * reports the same physical send through `onSendsConsumed`; the others are charged here and - * nowhere else. A refusal is not an error: the caller keeps the real upstream response -- - * status, `Retry-After`, quota body -- because return-the-last-answer is the exhaustion - * contract this unit settled on. - */ - /** - * A credential rotation inside ONE provider's roster is "auth-recovery", not - * "account-failover". The distinction is load-bearing: "account-failover" sets - * `isAlternateTarget` unconditionally, so under `maxAlternateTargetSends: 1` the first - * rotation would refuse every later one AND consume the single slot a genuine cross-pool - * move needs -- a roster whose first two accounts are both 429'd would return the 429 - * while a free third account sat unused. The roster cap bounds how far rotation walks; - * the shared total bounds how many sends the request makes. Reserve "account-failover" - * for a real move between pools. - */ - const reserveCredentialHop = ( - sendClass: SendClass, - targetKey: string, - countedExternally = false, - ): { allowed: boolean; permit?: SingleUseDispatchPermit } => { - if (!isRequestExecutionBudget(sendBudget)) return { allowed: true }; - const decision = sendBudget.reserveDispatch({ sendClass, targetKey, countedExternally }); - return decision.allowed ? { allowed: true, permit: decision.permit } : { allowed: false }; - }; - /** - * Both classes share the one reserve, so this only changes what the decision is called -- - * but a recovery event that says "repair" when a credential refresh drove it is the kind of - * mislabelled evidence #4592 existed to stop. - */ - const recoveryClassFor = (recovery: AttemptRecoveryKind): SendClass => - /401|429|oauth|rate-limit|key/.test(recovery) ? "auth-recovery" : "repair"; - - if ("passthrough" in adapter && adapter.passthrough && !routedCompaction) { - let hostAdmissionLease = pendingHostAdmissionLease; - pendingHostAdmissionLease = null; - try { - const codexSafetyBufferingOptions = isCanonicalOpenAiForwardProvider(route.provider) - ? codexSafetyBufferingFilterOptions(config) - : undefined; - const imageGenCallAliases = route.provider.authMode === "forward" - ? new Map() - : imageGenToolCallAliases(toolBridgeMaps.toolNsMap, parsed._rawBody, translatorBudget); - const routedCustomToolNames = new Set(); - const routedCustomToolRepairNames = new Set(); - const routedToolSearchNames = new Set(); - // Local continuation cache for the ChatGPT passthrough. Codex WS turns chain with - // previous_response_id, ocx converts them to internal HTTP requests, and the ChatGPT Codex - // REST backend rejects the parameter — the adapter strips it in forward mode, so the ONLY - // way a chained turn keeps its earlier context is the local replay expansion. Record - // completed passthrough responses (force bypasses Codex's blanket store:false) so the next - // turn's expansion hits. Never record a body whose own previous_response_id failed to - // expand: its input is a delta, and storing it would replay a truncated conversation. - // Compaction turns are excluded: _rawBody still carries the full pre-compaction history and - // recording it would let a later expansion rehydrate the chain Codex just replaced. - const passthroughRecordEligible = parsed._compactionRequest !== true - && (!parsed.previousResponseId || parsed._previousResponseInputExpanded === true); - const rememberPassthroughResponse = passthroughRecordEligible - ? (response: { id?: unknown; output?: unknown; status?: unknown }) => - rememberResponseState(parsed._rawBody, response, undefined, responseStateOptions(true)) - : undefined; - if (parsed.previousResponseId && !parsed._previousResponseInputExpanded) { - console.warn( - `[responses] previous_response_id ${parsed.previousResponseId} not found in local replay state ` - + `(model ${parsed.modelId}); forwarding without it — earlier turns may be missing from this request`, - ); - } - // Preserve the caller's readable catalog boundary before provider-specific normalization can - // remove an unsupported final entry (for example xAI cached-only web search). - const replayedInputPrefixLength = parsed._replayPrefixLen ?? 0; - const clientToolAuthorizationBody = currentTurnWireToolCatalogBody( - parsed._rawBody, - replayedInputPrefixLength, - ); - const selfNamedNamespaceScrubAuthorization = collectSelfNamedNamespaceScrubAuthorization( - clientToolAuthorizationBody, - toolBridgeMaps.bareCustomToolNames, - toolBridgeMaps.bareFunctionToolNames, - ); - const clientExplicitWireToolCatalog = hasExplicitWireToolCatalog(clientToolAuthorizationBody); - const clientDeclaredWireToolNames = collectDeclaredWireToolNames(clientToolAuthorizationBody); - const clientDeclaredBareWireToolNames = collectDeclaredBareWireToolNames(clientToolAuthorizationBody); - const clientDeclaredNamelessCallTypes = collectDeclaredNamelessClientCallTypes( - clientToolAuthorizationBody, - ); - // Hosted calls the PROVIDER runs itself. Gated on the destination actually being xAI, so a - // declaration alone cannot buy the exemption on some other upstream that never serves it. - // Provider-executed declarations are authorized from the actual outbound body, after the - // adapter has applied destination-specific injection and normalization. Client-executed tool - // authority remains bounded to the caller-owned catalog above. - const providerExecutedCallTypes = new Set(); - let request: Awaited>; - try { - request = await adapter.buildRequest(parsed, { headers: selectedForwardHeaders, translatorBudget }); - } catch (error) { - releaseCodexAuthContextProbeLease(authCtx); - // A tool catalog this proxy cannot lower onto one wire namespace is a client input error, and - // the rotation-rebuild and bridged paths already answer 400 for the identical throw. Rethrowing - // it here escaped every catch up to the Bun handler, so the same request produced an - // unstructured 500 — and no request log — depending only on whether a rotation ran first. - // Same shape for a tool_choice this proxy cannot honor: the destination rejects a schema the - // catalog had to drop, so the selector naming it is a client input error, not a 500. - if (error instanceof NamespaceToolCollisionError || error instanceof XaiToolSchemaCompatibilityError) { - return formatErrorResponse(400, "invalid_request_error", redactSecretString(error.message)); - } - throw error; - } - const functionRepairSchemas = isCanonicalOpenAiForwardProvider(route.provider) - ? new Map() - : collectFunctionCallRepairSchemas(clientToolAuthorizationBody); - if (!isCanonicalOpenAiForwardProvider(route.provider)) { - for (const name of request.convertedRoutedCustomToolNames ?? []) { - if ( - toolBridgeMaps.freeformToolNames.has(name) - || toolBridgeMaps.toolNsMap.get(name)?.freeform === true - ) routedCustomToolNames.add(name); - } - for (const name of request.routedCustomToolRepairNames ?? []) { - if ( - toolBridgeMaps.freeformToolNames.has(name) - || toolBridgeMaps.toolNsMap.get(name)?.freeform === true - ) routedCustomToolRepairNames.add(name); - } - } - for (const name of request.convertedRoutedToolSearchNames ?? []) { - // The adapter already keeps this set empty when tool_choice forbids the private search. - // Its wire name may be collision-aliased, so comparing it to the caller-facing name here - // would incorrectly disable restoration for the exact ambiguous-name case the alias fixes. - routedToolSearchNames.add(name); - } - refreshRequestToolAliases(request); - // #1700: the bridged paths refuse a call to a tool the request never declared - // (`declaredToolNames`, src/bridge.ts). The passthrough had no equivalent, so a routed - // provider's top-level `apply_patch` — which under Codex code mode exists only as a nested - // `tools.apply_patch(...)` helper inside `exec`, never as a wire tool — reached Codex as a - // call it cannot execute, and the turn showed a bare `aborted` with the file untouched. - // Forward auth is the canonical ChatGPT backend speaking Codex's own protocol rather than a - // routed provider, so it keeps passing through unguarded, as it does for the rewrites above. - // The guard needs a catalog to compare against, so it stands down when the request omits one. - // An explicit empty catalog is still authoritative: it declares that no client tools may be - // called. A passthrough request can legitimately omit `tools` entirely and still receive a call - // the client understands — `tests/providers/github-copilot/github-copilot-stream-contract.test.ts` sends - // `{model, input, stream}` with no tools and Copilot answers with a `custom_tool_call` for - // `apply_patch`. Policing an absent catalog truncates that turn. An unreadable body lands there - // too because the proxy cannot establish the caller's declared authorization boundary. - const parseOutboundRequestBody = (bodyText: string): Record | undefined => { - try { - const body = JSON.parse(bodyText) as unknown; - return body && typeof body === "object" && !Array.isArray(body) - ? body as Record - : undefined; - } catch { - return undefined; - } - }; - let outboundRequestBody: Record | undefined; - const declaredWireToolNames = new Set(); - const declaredBareWireToolNames = new Set(); - const declaredNamelessClientCallTypes = new Set(); - // `buildToolBridgeMaps` creates a bare alias only when the caller selected exactly one - // namespaced tool through a bare tool_choice. Restore that request-bounded identity before - // authorization checks instead of admitting the bare name into the declared set: for `exec`, - // the latter would also authorize the unrelated code-mode helper names. - const authorizedBareNamespaceToolAliases: RoutedNamespaceToolAliases = new Map( - [...toolBridgeMaps.toolNsMap].flatMap(([alias, identity]) => - alias === identity.name - ? [[alias, { - namespace: identity.namespace, - name: identity.name, - kind: identity.freeform ? "custom" as const : "function" as const, - }] as const] - : [] - ), - ); - const restoreAuthorizedBareNamespaceToolCalls = (value: unknown): unknown => - restoreRoutedNamespaceCalls(value, authorizedBareNamespaceToolAliases).value; - const normalizeFunctionCompletionJson = (text: string): string => { - const snapshot = hasResponsesSnapshotRepair(route.provider.responsesSnapshotRepair) - ? repairResponsesSnapshotJson(text, outboundRequestBody) - : text; - // Sparse gateways need completion status inferred before schema repair can - // distinguish completed arguments from in-progress placeholders. - return repairFunctionCallsInJson(backfillResponsesFieldsJson(snapshot), functionRepairSchemas); - }; - let undeclaredToolGuardActive = false; - const refreshUndeclaredToolGuard = (builtRequest: AdapterRequest): void => { - outboundRequestBody = parseOutboundRequestBody(builtRequest.body); - providerExecutedCallTypes.clear(); - if (isXaiResponsesDestination(route.provider)) { - // Preserve the caller-declared authorization recognized by the original classifier, then - // add adapter-injected declarations from the actual current-turn outbound catalog. - for (const callType of collectProviderExecutedCallTypes(clientToolAuthorizationBody)) { - providerExecutedCallTypes.add(callType); - } - const currentOutboundCatalog = currentTurnWireToolCatalogBody( - outboundRequestBody, - replayedInputPrefixLength, - ); - for (const callType of collectProviderExecutedCallTypes(currentOutboundCatalog)) { - providerExecutedCallTypes.add(callType); - } - } - declaredWireToolNames.clear(); - // With no replay prefix the full outbound body belongs to this turn and its normalized - // aliases are authoritative. A continuation's outbound body still contains historical - // catalogs (and may promote historical tool-search definitions), so it can never widen the - // current caller snapshot captured above. - declaredBareWireToolNames.clear(); - if (replayedInputPrefixLength === 0) { - for (const name of collectDeclaredWireToolNames(outboundRequestBody)) { - declaredWireToolNames.add(name); - } - for (const name of collectDeclaredBareWireToolNames(outboundRequestBody)) { - declaredBareWireToolNames.add(name); - } - } - for (const name of clientDeclaredWireToolNames) declaredWireToolNames.add(name); - for (const name of clientDeclaredBareWireToolNames) declaredBareWireToolNames.add(name); - declaredNamelessClientCallTypes.clear(); - if (replayedInputPrefixLength === 0) { - for (const callType of collectDeclaredNamelessClientCallTypes(outboundRequestBody)) { - declaredNamelessClientCallTypes.add(callType); - } - } - for (const callType of clientDeclaredNamelessCallTypes) { - declaredNamelessClientCallTypes.add(callType); - } - // On an ordinary request these maps capture caller-catalog identities that normalization may - // replace on the outbound wire (for example a client image tool becoming hosted). On replay, - // however, the parsed maps also contain historical catalog entries, so only the bounded - // current-turn wire snapshot above may authorize a call. - if (replayedInputPrefixLength === 0) { - for (const name of toolBridgeMaps.declaredToolNames) { - // `buildToolBridgeMaps` also aliases a namespaced tool under its bare name when the - // caller's `tool_choice` selected it unambiguously, which the bridge needs to route the - // call back. For `exec` alone that alias would also switch on nested-helper - // normalization and re-authorize `exec_command`/`shell_command`/`apply_patch`/`view_image`, so it is - // admitted here only when the caller's own catalog declared a bare `exec`. Selecting an - // MCP `exec` is not a declaration of the code-mode shell tool. - if ( - name === CODE_MODE_EXEC_TOOL_NAME - && !clientDeclaredWireToolNames.has(CODE_MODE_EXEC_TOOL_NAME) - ) continue; - declaredWireToolNames.add(name); - } - } - undeclaredToolGuardActive = ( - declaredWireToolNames.size > 0 - || clientDeclaredNamelessCallTypes.size > 0 - || clientExplicitWireToolCatalog - ) && route.provider.authMode !== "forward"; - }; - refreshUndeclaredToolGuard(request); - // A refused turn must not seed `previous_response_id` replay. The inspection branch reads the - // untouched upstream stream, so it can still observe a `response.completed` the client never - // received; checking the payload itself rather than a flag shared with the client relay keeps - // this free of tee ordering races. - // - // Checking only the terminal snapshot is not enough. An upstream can announce the undeclared - // call in `response.output_item.added`, which trips the client guard, and then close with a - // `response.completed` whose `output` is empty. The client gets `response.failed`, the terminal - // check sees nothing undeclared, and the refused turn enters continuation state anyway. So the - // rejection is sticky for the whole turn, set from every parsed payload on the inspection side. - let inspectionSawUndeclaredTool = false; - let inspectedTerminal: ResponsesTerminalStatus | null = null; - let inspectedCompletionSeen = false; - let firstTerminalAllowsRecall = false; - const passiveQuotaObserved = hasPassiveAccountQuota(route.providerName) - && route.provider.authMode === "oauth"; - const noteInspectedPayload = (payload: unknown) => { - // First terminal stays authoritative even in metadata-only inspection, which - // intentionally continues parsing after a failed/incomplete terminal. - const terminal = terminalStatusFromParsed(payload); - if (inspectedTerminal === null && terminal !== null) { - inspectedTerminal = terminal; - // The client boundary accepts a terminal by event type, even without a - // response object. Such a terminal must permanently decline recall. - if (terminal === "completed" && payload && typeof payload === "object" - && "response" in payload && payload.response && typeof payload.response === "object" - && !Array.isArray(payload.response) && "model" in payload.response) { - firstTerminalAllowsRecall = typeof payload.response.model === "string" - && payload.response.model.trim().length > 0; - } - } - // Meta reports subscription usage ONLY as an in-stream event; there is no endpoint - // to poll (003 §E probed 17 paths, all 404). Observed here rather than behind a - // dedicated inspector handler because onParsedPayload already reaches every - // passthrough shape -- eager relay and both tee consumers -- through this one - // function. - // - // Placed BEFORE the undeclared-tool early return below, which is load-bearing: that - // guard latches for the rest of the turn once it fires, and a turn that tripped it - // still legitimately reports usage. - if (passiveQuotaObserved && isMuseSubscriptionUsagePayload(payload)) { - const quota = parseMuseSubscriptionUsage(payload); - // Read at EVENT time, not at handler construction: failover rebinds this, and the - // quota belongs to the account that actually served the turn. - const servingAccountId = genericFailoverAccountId; - if (quota && servingAccountId) { - recordPassiveAccountQuota(route.providerName, servingAccountId, quota, passiveQuotaWriterGeneration); - } - } - // Gated on the same flag as the guard itself: with no readable catalog (or a forward-auth - // provider) every name looks undeclared, and flipping this would stop recording continuation - // state for exactly the passthrough traffic the guard deliberately stands down for. - if (undeclaredToolGuardActive && !inspectionSawUndeclaredTool && undeclaredToolCallName( - restoreAuthorizedBareNamespaceToolCalls( - restoreMuseToolNames(payload, routedMuseToolNameAliases).value, - ), - declaredWireToolNames, - declaredNamelessClientCallTypes, - providerExecutedCallTypes, - declaredBareWireToolNames, - ) !== undefined) { - inspectionSawUndeclaredTool = true; - } - // The snapshot callback opts the inspector into output reconstruction. Compaction - // has no continuation cache, so use the parsed terminal here without adding retention. - if (plaintextV2AgentMessageToolNames.size === 0 && !rememberPassthroughResponse && payload && typeof payload === "object" - && "type" in payload && payload.type === "response.completed" - && "response" in payload && payload.response && typeof payload.response === "object" - && !Array.isArray(payload.response)) { - rememberPassthroughResponseChecked(payload.response as Record); - } - }; - const rememberPassthroughResponseChecked = ( - response: { id?: unknown; output?: unknown; status?: unknown; model?: unknown }, - ) => { - if (inspectionSawUndeclaredTool) return; - const restored = restoreRoutedCustomCalls( - restoreAuthorizedBareNamespaceToolCalls( - restoreRoutedNamespaceCalls( - restoreMuseToolNames(response, routedMuseToolNameAliases).value, - routedNamespaceToolAliases, - ).value, - ), - routedCustomToolNames, - routedCustomToolRepairNames, - declaredWireToolNames, - ).value; - const normalizedResponse = (functionRepairSchemas.size > 0 - ? JSON.parse(normalizeFunctionCompletionJson(JSON.stringify(restored))) - : restored) as { id?: unknown; output?: unknown; status?: unknown }; - const plaintextRestore = restorePlaintextV2AgentMessageCalls( - normalizedResponse, plaintextV2AgentMessageToolNames, plaintextV2AgentMessageAliasedToolNames, - ); - if (plaintextRestore.overflowed) return; - const restoredResponse = plaintextRestore.value as typeof normalizedResponse; - // Replay overlap compares the items the client echoes, including visible reasoning shape. - const replayResponse = restoredResponse; - if ( - undeclaredToolGuardActive - && undeclaredToolCallNameInResponse( - restoredResponse, - declaredWireToolNames, - declaredNamelessClientCallTypes, - providerExecutedCallTypes, - declaredBareWireToolNames, - ) !== undefined - ) { - return; - } - const normalizedReplayResponse = (undeclaredToolGuardActive - ? normalizeDefaultNamespaceInResponse( - replayResponse, - declaredWireToolNames, - declaredBareWireToolNames, - ).value - : replayResponse) as typeof replayResponse; - rememberPassthroughResponse?.(normalizedReplayResponse); - const firstCompletion = !inspectedCompletionSeen; - inspectedCompletionSeen = true; - if (firstCompletion && (inspectedTerminal === null || firstTerminalAllowsRecall)) { - // A model-less first completion permanently declines recall; later terminal - // frames are hidden by the client boundary and cannot supply its identity. - // Native inspection sees the pre-rewrite model. Only an actual terminal - // model can seed recall; an absent model never falls back to the pick. - if (typeof response.model === "string" && response.model.trim()) { - notifyResponseComplete({ - status: response.status, - model: parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId - ? parsed._responseModelId : response.model, - }); - } - } - }; - recordAdapterReasoning(logCtx, request); - recordAdapterTier(logCtx, request); - const actualHostKey = upstreamHostHealthKey( - route.providerName, - safeOriginLabel(request.url), - ); - const hostKey = route.provider.authMode === "forward" - ? actualHostKey - : null; - const hostCircuitEnabled = hostKey !== null - && normalizeUpstreamHostCircuitThreshold(config.upstreamHostCircuitThreshold) > 0; - if (hostKey !== null && !hostCircuitEnabled) { - disableUpstreamHostCircuitForKey(actualHostKey); - } - if (hostAdmissionLease && hostAdmissionLease.key !== hostKey) { - return formatErrorResponse(502, "upstream_error", "Provider host changed after circuit admission"); - } - if (options.abortSignal?.aborted) { - releaseCodexAuthContextProbeLease(authCtx); - return clientCancelledResponse(); - } - if (!hostAdmissionLease && hostCircuitEnabled) { - const admission = acquireUpstreamHostAdmission( - hostKey!, - config.upstreamHostCircuitThreshold, - ); - if (admission.kind === "blocked") { - releaseCodexAuthContextProbeLease(authCtx); - return upstreamHostCircuitOpenResponse(admission.retryAfterSeconds); - } - hostAdmissionLease = admission.lease; - } - const settleObservedHostResponse = (): void => { - if (hostCircuitEnabled) { - resetUpstreamHostHealth(actualHostKey, hostAdmissionLease); - } else { - resetUpstreamHostHealth(actualHostKey); - } - hostAdmissionLease = null; - }; - /** - * #4191: a Codex WS exchange pins its content-free stage record on the - * Response it resolves (markCodexWsStage). Adopting the record here, at - * the single funnel every physical upstream response passes through, - * binds it to the attempt that actually served it — including the 502/504 - * pre-response JSON settles that never reach the SSE relay. - */ - const adoptCodexWsStage = (response: Response): void => { - const stage = readCodexWsStage(response); - if (stage && logCtx.activeAttempt) logCtx.activeAttempt.codexWsStage = stage; - }; - const adoptObservedResponse = (response: T): T => { - settleObservedHostResponse(); - adoptCodexWsStage(response); - return response; - }; - let passthroughEstimate = typeof request.usageLog?.inputTokens === "number" - ? request.usageLog.inputTokens - : undefined; - if (passthroughEstimate !== undefined) { - logCtx.usageLogInputTokens = passthroughEstimate; - } - // Abort the upstream if the client disconnects. A directly-relayed body does not propagate the - // consumer's cancel to a signalled fetch, so we pass the signal and relay through relayWithAbort, - // whose cancel() aborts the upstream — preventing leaked connections (RC2, passthrough path). - const upstream = new AbortController(); - linkAbortSignal(upstream, options.abortSignal); - const connectMs = config.connectTimeoutMs ?? 200_000; - let upstreamResponse: Response; - /** - * Refuse a built body that exceeds the operator's configured ceiling, before it is sent. - * - * Unconfigured this measures nothing and returns undefined, so an unset proxy behaves - * exactly as it does today. Runs at every point a body is built or rebuilt, because a - * rebuild can produce a payload the initial check never saw. - */ - const refuseOversizedOutboundBody = ( - builtRequest: AdapterRequest, - refusalAuthCtx: CodexAuthContext = authCtx, - ): Response | undefined => { - const result = checkOutboundBodySize(builtRequest.body, config.maxUpstreamBodyBytes); - if (result.admitted) return undefined; - - // This returns before the surrounding fetch/finally owns the observation, so release - // it here or one refused body holds translator budget for the process lifetime. - builtRequest.releaseBodyObservation?.(); - upstream.abort(); - releaseUpstreamHostAdmission(hostAdmissionLease); - hostAdmissionLease = null; - releaseCodexAuthContextProbeLease(refusalAuthCtx); - logCtx.errorCode = "outbound_body_too_large"; - console.warn( - `[responses] refused an oversized outbound body: bytes=${result.bytes} limit=${result.limit} ` - + `input_images=${result.imageCount} image_bytes=${result.imageBytes} ` - + `model=${JSON.stringify(parsed.modelId)}`, - ); - // A streaming client treats HTTP 413 as a retryable transport error and resends the same - // oversized body — the reconnect loop #3177 exists to stop. Terminal overflow is the - // honest shape, and it is what the upstream-413 path already returns. - if (clientRequestedStream) { - return streamingContextOverflowResponse( - parsed._responseModelId ?? parsed.modelId, - translatorBudget, - ); - } - return formatErrorResponse( - 413, - "outbound_body_too_large", - describeOutboundBodyRefusal(result), - ); - }; - const transportFailureResponse = (err: unknown): Response => { - upstream.abort(); - if (options.abortSignal?.aborted) { - releaseUpstreamHostAdmission(hostAdmissionLease); - hostAdmissionLease = null; - releaseCodexAuthContextProbeLease(authCtx); - return clientCancelledResponse(); - } - // A budget refusal is a proxy decision, not an upstream fault. Reporting it as - // 502 upstream_error would blame the provider for a limit this process applied, and - // would record a fake reachability failure against the account's health. - if (err instanceof SendBudgetExhaustedError) { - releaseUpstreamHostAdmission(hostAdmissionLease); - hostAdmissionLease = null; - releaseCodexAuthContextProbeLease(authCtx); - return formatErrorResponse(429, "request_send_budget_exhausted", err.message); - } - const localRefusal = mapCodexAuthContextErrorToResponse(unwrapUpstreamRetryEvidenceError(err), { - now: Date.now(), accountSelector: route.codexAccountNamespace, - }); - if (localRefusal) { - releaseUpstreamHostAdmission(hostAdmissionLease); - hostAdmissionLease = null; - releaseCodexAuthContextProbeLease(authCtx); - return localRefusal; - } - const outcome = classifyTransportFailureKind(err); - // Host-level evidence stands regardless of pool membership: a direct - // forward send has no pool accounting, but the reachability failure is - // still host-wide, not account evidence (#914 review). - if (outcome === "connect_neutral") { - if (hostCircuitEnabled) { - recordUpstreamHostFailure(actualHostKey, { - code: transportErrorCode(err), - threshold: config.upstreamHostCircuitThreshold, - lease: hostAdmissionLease, - }); - } else { - recordUpstreamHostFailure(actualHostKey, { code: transportErrorCode(err) }); - } - hostAdmissionLease = null; - } else { - releaseUpstreamHostAdmission(hostAdmissionLease); - hostAdmissionLease = null; - } - if (usesCodexForwardPoolAuth(authCtx, route.provider)) { - recordCodexUpstreamOutcome(config, authCtx.accountId, outcome, { - threadId: authCtx.affinityKey, - fixedAccount: authCtx.fixedAccount, - modelId: route.modelId, - probeLeaseId: codexProbeLeaseId(authCtx), - probeQuotaScope: codexProbeQuotaScope(authCtx), - writerGeneration: authCtx.writerGeneration, - }); - } - const msg = outcome === "timeout" - ? `Provider connect timeout after ${connectMs}ms` - : describeUpstreamConnectFailure(err, connectMs); - return formatErrorResponse(502, "upstream_error", msg); - }; - const initialBodyRefusal = refuseOversizedOutboundBody(request); - if (initialBodyRefusal) return initialBodyRefusal; - try { - // Transient-5xx pre-stream retry (devlog/_plan/260716_claudecode_hardening/010): - // the ChatGPT backend emits transient 502/520s that an immediate retry absorbs. - // Body is a replayable string; nothing has streamed to the client yet. - upstreamResponse = await fetchWithTransientRetry( - recovery => { - noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, recovery); - return fetchWithHeaderTimeout(request.url, applyUpstreamRecoveryInit({ - method: request.method, - headers: request.headers, - body: request.body, - }, recovery), upstream.signal, connectMs, parsed.stream, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(request), - providerName: route.providerName, - modelId: route.modelId, - onCodexWsQuota: codexWsQuotaObserver(authCtx, route.provider, route.modelId), - beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) - ? createCodexReserveDispatchGuard(authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, - }), - route.provider.authMode === "forward") - // Every real attempt response — including an intermediate 5xx the - // retry wrapper replaces — proves the host was reached (#914 review). - .then(adoptObservedResponse); - }, - { abortSignal: upstream.signal, label: safeHostLabel(request.url), attempts: remainingTransientSendBudget(TRANSIENT_RETRY_MAX_ATTEMPTS), onSendsConsumed: noteTransientSends }, - ); - } catch (err) { - return transportFailureResponse(err); - } finally { - request.releaseBodyObservation?.(); - } - - const opaqueBlobRecoveryGuard: OpaqueBlobRecoveryGuard = { attempted: false }; - // At most one reasoning-effort downgrade per request. - const reasoningEffortDowngradeGuard: { attempted: boolean } = { attempted: false }; - let oauth401ReplayAttempted = false; - let codex401ReplayKind: "main" | "stored" | null = null; - // Console Go answers a transient 400 "Invalid upload request." for bodies it accepts - // moments later; at most one byte-identical replay is allowed per request. - const consoleGoUploadRetryGuard: { attempted: boolean } = { attempted: false }; - const rateLimitPolicy = rateLimitRetryPolicyFor(route.provider); - let rateLimitRetries = 0; - const rebuildAndRefetch = async ( - recovery: AttemptRecoveryKind, - ): Promise => { - const retryAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - if (!("passthrough" in retryAdapter) || !retryAdapter.passthrough) { - upstream.abort(); - return { failed: formatErrorResponse(502, "upstream_error", "Recovery changed the provider wire unexpectedly") }; - } - try { - if (recovery !== "console-go-upload-retry") { - request = await retryAdapter.buildRequest(parsed, { - headers: selectedForwardHeaders, - translatorBudget, - }); - } - refreshRequestToolAliases(request); - recordAdapterReasoning(logCtx, request); - recordAdapterTier(logCtx, request); - } catch (err) { - upstream.abort(); - if (options.abortSignal?.aborted) return { failed: clientCancelledResponse() }; - const msg = err instanceof Error ? err.message : String(err); - return { failed: formatErrorResponse(400, "invalid_request_error", redactSecretString(msg)) }; - } - passthroughEstimate = typeof request.usageLog?.inputTokens === "number" - ? request.usageLog.inputTokens - : undefined; - if (passthroughEstimate !== undefined) logCtx.usageLogInputTokens = passthroughEstimate; - refreshUndeclaredToolGuard(request); - logCtx.providerAdapter = retryAdapter.name; - sealRequestAttemptIdentity( - logCtx.activeAttempt, - logCtx.provider, - retryAdapter.name, - logCtx.accountLogLabel, - ); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, retryAdapter.name); - const rebuiltBodyRefusal = refuseOversizedOutboundBody(request); - if (rebuiltBodyRefusal) return { failed: rebuiltBodyRefusal }; - // The base allowance is spent first; once it is gone this leg may still draw the one - // shared final-recovery reserve, which is what keeps a validated sanitized rebuild - // after a 5xx streak alive at four total sends instead of dying at three. Reserved - // outside the try so the finally can hand it back if the leg never reached its send. - const allowance = recoverySendAllowance( - TRANSIENT_RETRY_MAX_ATTEMPTS, - recoveryClassFor(recovery), - `${route.providerName}|${route.modelId}|${recovery}`, - ); - try { - return await fetchWithTransientRetry( - innerRecovery => { - // Gated on the return, not fire-and-forget: a consumed permit means this leg - // already sent once, and letting the second call through would be a free send. - if (allowance.permit && !allowance.permit.use()) { - throw new SendBudgetExhaustedError(safeHostLabel(request.url)); - } - noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, innerRecovery ?? recovery); - return fetchWithHeaderTimeout(request.url, applyUpstreamRecoveryInit({ - method: request.method, - headers: request.headers, - body: request.body, - }, innerRecovery), upstream.signal, connectMs, parsed.stream, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(request), - providerName: route.providerName, - modelId: route.modelId, - onCodexWsQuota: codexWsQuotaObserver(authCtx, route.provider, route.modelId), - beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) - ? createCodexReserveDispatchGuard(authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, - }), - route.provider.authMode === "forward") - .then(adoptObservedResponse); - }, - { abortSignal: upstream.signal, label: safeHostLabel(request.url), attempts: allowance.attempts, onSendsConsumed: noteTransientSends }, - ); - } catch (err) { - return { failed: transportFailureResponse(err) }; - } finally { - // A no-op once the permit was used or once onSendsConsumed settled it; it only refunds - // a reservation whose send never happened. - allowance.permit?.release(); - request.releaseBodyObservation?.(); - } - }; - - // Keep recovery kinds in sync with the generic `recovery:` loop below. - passthroughRecovery: for (;;) { - - if ( - upstreamResponse.status === 401 - && (authCtx.kind === "main-pool" || authCtx.kind === "pool") - && usesCodexForwardPoolAuth(authCtx, route.provider) - && codex401ReplayKind === null - ) { - codex401ReplayKind = authCtx.kind === "pool" ? "stored" : "main"; - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed */ } - const poolAuthCtx = authCtx.kind === "pool" ? authCtx : undefined; - const poolReplay = poolAuthCtx - ? await refreshPoolForwardAuth({ req, config, route, authCtx: poolAuthCtx, substituteMainCredential, options, logCtx }) - : undefined; - const replay = poolReplay - ?? await refreshNativeMainForwardAuth({ req, config, route, authCtx, substituteMainCredential, options }); - if (!replay.ok) { - // Compact already records this; core historically returned without recording, - // so a dead grant stayed selectable and every request repeated the same doomed - // refresh. Fenced by the generation the 401 belongs to (#2887). - if (poolAuthCtx && poolReplay && !poolReplay.ok && poolReplay.quarantine) { - recordCodexUpstreamOutcome(config, poolAuthCtx.accountId, 401, { - threadId: poolAuthCtx.affinityKey, - fixedAccount: poolAuthCtx.fixedAccount, - modelId: route.modelId, - writerGeneration: poolAuthCtx.writerGeneration, - credentialGeneration: poolReplay.quarantineGeneration ?? poolAuthCtx.generation, - }); - } - upstream.abort(); - releaseCodexAuthContextProbeLease(authCtx); - return replay.response; - } - authCtx = replay.authCtx; - route.provider = replay.provider; - selectedForwardHeaders = withClaudeNativeSession(replay.headers, replay.provider, options.claudeNativeSessionId); - const replayAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, replay.provider, inboundWire), - config.cacheRetention, - ); - if (!("passthrough" in replayAdapter) || !replayAdapter.passthrough) { - upstream.abort(); - return formatErrorResponse(502, "upstream_error", "Native main refresh changed the provider wire unexpectedly"); - } - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: replay.provider, - adapterName: replayAdapter.name, - codexAuthContext: authCtx, - forwardHeaders: selectedForwardHeaders, - }); - logCtx.providerAdapter = replayAdapter.name; - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, replayAdapter.name, logCtx.accountLogLabel); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, replayAdapter.name); - try { - request = await replayAdapter.buildRequest(parsed, { - headers: selectedForwardHeaders, - translatorBudget, - }); - refreshRequestToolAliases(request); - recordAdapterReasoning(logCtx, request); - recordAdapterTier(logCtx, request); - refreshUndeclaredToolGuard(request); - // The 401 replay rebuilds the body before sending, so it needs the same ceiling as - // every other build site; a replay is exactly when a grown payload reappears. - const replayBodyRefusal = refuseOversizedOutboundBody(request); - if (replayBodyRefusal) return replayBodyRefusal; - noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, "oauth-401"); - upstreamResponse = await fetchWithHeaderTimeout( - request.url, - { method: request.method, headers: request.headers, body: request.body }, - upstream.signal, - connectMs, - parsed.stream, - // The replay-dispatched signal is what bounds the rest of this logical request, so it - // has to describe a send that actually happened. fetchWithHeaderTimeout awaits pacing - // admission BEFORE calling the executor, so signalling at the call site would spend the - // budget even when a rejected pacing wait means nothing reaches the network. Wrapping - // the executor moves the signal to the last moment before the send, where a throw from - // here on is a genuine transport attempt. - storedPoolReplayDispatchNotifier( - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(request), - providerName: route.providerName, - modelId: route.modelId, - onCodexWsQuota: codexWsQuotaObserver(authCtx, route.provider, route.modelId), - beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) - ? createCodexReserveDispatchGuard(authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, - }), - codex401ReplayKind === "stored" ? options.onStoredPool401ReplayDispatched : undefined, - ), - route.provider.authMode === "forward", - ).then(adoptObservedResponse); - } catch (err) { - return transportFailureResponse(err); - } finally { - request.releaseBodyObservation?.(); - } - continue passthroughRecovery; - } - - if (codex401ReplayKind !== null && upstreamResponse.status === 401) break; - - // Native Responses providers return before the generic adapter recovery loop below. Keep - // their OAuth contract identical: one pre-stream 401 forces a credential refresh and one - // rebuilt replay. xAI's current subscription models use this branch now that their official - // Grok CLI catalog declares the Responses backend. - if ( - upstreamResponse.status === 401 - && isOAuth401ReplayProvider - && sentOAuthSnapshot - && !oauth401ReplayAttempted - // Refused here, before the 401 body is cancelled: once it is gone the request can only - // answer with a synthetic 502, which would report a proxy budget decision as an upstream - // fault and throw away the credential evidence the client needs. - && !sendBudgetExhausted() - ) { - oauth401ReplayAttempted = true; - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - let refreshed: OAuthAccessSnapshot; - try { - refreshed = await refreshResolvedOAuthSelection(sentOAuthSnapshot); - } catch (err) { - upstream.abort(); - releaseCodexAuthContextProbeLease(authCtx); - return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(err)); - } - if (route.provider.googleMode === "cloud-code-assist" && !refreshed.projectId) { - upstream.abort(); - releaseCodexAuthContextProbeLease(authCtx); - return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(new Error("Cloud Code Assist project is required"))); - } - sentOAuthSnapshot = refreshed; - replayOAuthCredentialSnapshot = { - accountId: refreshed.accountId, - generation: refreshed.generation, - }; - if (route.providerName === "kiro") { - parsed._kiroAuthContext = { ...(refreshed.kiro ?? {}) }; - } - const refreshedProvider = resolveProviderTransport( - route.providerName, - { - ...route.provider, - apiKey: refreshed.accessToken, - ...(refreshed.projectId ? { project: refreshed.projectId } : {}), - }, - parsed.options.promptCacheKey, - route.providerName === "github-copilot" - ? resolveCopilotApiBaseUrl(refreshed.apiBaseUrl) - : undefined, - ); - route.provider = refreshedProvider; - const refreshedAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, refreshedProvider, inboundWire), - config.cacheRetention, - ); - if (!("passthrough" in refreshedAdapter) || !refreshedAdapter.passthrough) { - upstream.abort(); - return formatErrorResponse(502, "upstream_error", "OAuth refresh changed the provider wire unexpectedly"); - } - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: refreshedProvider, - adapterName: refreshedAdapter.name, - oauthCredentialSnapshot: replayOAuthCredentialSnapshot, - }); - logCtx.providerAdapter = refreshedAdapter.name; - sealRequestAttemptIdentity( - logCtx.activeAttempt, - logCtx.provider, - refreshedAdapter.name, - logCtx.accountLogLabel, - ); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, refreshedAdapter.name); - try { - request = await refreshedAdapter.buildRequest(parsed, { - headers: selectedForwardHeaders, - translatorBudget, - }); - refreshRequestToolAliases(request); - recordAdapterReasoning(logCtx, request); - recordAdapterTier(logCtx, request); - } catch (err) { - upstream.abort(); - if (options.abortSignal?.aborted) return clientCancelledResponse(); - const msg = err instanceof Error ? err.message : String(err); - return formatErrorResponse(400, "invalid_request_error", redactSecretString(msg)); - } - refreshUndeclaredToolGuard(request); - const refreshedBodyRefusal = refuseOversizedOutboundBody(request); - if (refreshedBodyRefusal) return refreshedBodyRefusal; - try { - upstreamResponse = await fetchWithTransientRetry( - recovery => { - noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, recovery ?? "oauth-401"); - return fetchWithHeaderTimeout(request.url, applyUpstreamRecoveryInit({ - method: request.method, - headers: request.headers, - body: request.body, - }, recovery), upstream.signal, connectMs, parsed.stream, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(request), - providerName: route.providerName, - modelId: route.modelId, - onCodexWsQuota: codexWsQuotaObserver(authCtx, route.provider, route.modelId), - beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) - ? createCodexReserveDispatchGuard(authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, - }), - route.provider.authMode === "forward") - .then(adoptObservedResponse); - }, - { abortSignal: upstream.signal, label: safeHostLabel(request.url), attempts: remainingTransientSendBudget(TRANSIENT_RETRY_MAX_ATTEMPTS), onSendsConsumed: noteTransientSends }, - ); - } catch (err) { - return transportFailureResponse(err); - } finally { - request.releaseBodyObservation?.(); - } - } - - // Native Responses returns before the generic adapter's OAuth rotation loop. Keep - // the same quorum, cooldown and request budget here, before any client bytes flow. - if ( - upstreamResponse.status === 429 - && genericFailoverAccountId - && genericFailovers < GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST - && isGenericOAuthFailoverEnabled(config, route.providerName) - ) { - // The roster cap above is one half of the bound; the request's shared budget is the - // other. A refused hop leaves the real 429 -- body, Retry-After and any quota evidence - // -- exactly as upstream sent it. - const hop = reserveCredentialHop( - "auth-recovery", - `${route.providerName}|${route.modelId}|oauth-account-429`, - true, - ); - if (hop.allowed) { - const nextAccountId = rotateGenericOAuthAccountOn429( - config, route.providerName, genericFailoverAccountId, - upstreamResponse.headers.get("retry-after"), - ); - let snapshot: OAuthAccessSnapshot | undefined; - if (nextAccountId) { - try { snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); } - catch { /* Keep the original 429 body readable when the next credential is unavailable. */ } - } - if (snapshot && await applyFailoverSnapshot(snapshot)) { - genericFailovers += 1; - route.provider = resolveProviderTransport( - route.providerName, route.provider, parsed.options.promptCacheKey, sentOAuthSnapshot?.apiBaseUrl, - ); - bindRouteReasoningReplayScope({ - parsed, providerName: route.providerName, provider: route.provider, - adapterName: "openai-responses", oauthCredentialSnapshot: replayOAuthCredentialSnapshot, - }); - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already closed */ } - // The replay IS this hop's send, so the rebuild spends the reservation instead of - // asking for one of its own. - pendingHopPermit = hop.permit; - const result = await rebuildAndRefetch("oauth-account-429"); - pendingHopPermit = undefined; - if ("failed" in result) return result.failed; - upstreamResponse = result; - continue passthroughRecovery; - } - // No credential moved, so the reservation costs nothing. - hop.permit?.release(); - } - } - - // Same-target 429 wait-and-retry (opt-in `retryOn429`) for key-auth providers on the - // passthrough wire. This branch returns before the recovery loop below, so Responses-shaped - // key-auth gateways (e.g. the built-in DeepSeek preset) would otherwise surface 429 - // immediately with no same-key replay. Pre-stream only — nothing has been relayed yet, so - // the replay is lossless (same invariant as the recovery loop). Forward/OAuth providers - // keep their pool logic below (rateLimitRetryPolicyFor returns null for them). - while ( - upstreamResponse.status === 429 - && rateLimitPolicy !== null - && rateLimitRetries < rateLimitPolicy.attempts - // Checked here rather than inside the helper: prepareSameTarget429Wait releases the 429 - // body, so a refusal discovered after the wait can no longer return the real rate-limit - // answer and would surface a synthetic 502 instead. - && !sendBudgetExhausted() - ) { - rateLimitRetries += 1; - // Release unread body + deliberate wait via the shared same-target helper. - const retryAfterHeader = upstreamResponse.headers.get("retry-after"); - try { - for await (const _ of prepareSameTarget429Wait({ - body: upstreamResponse.body, - signal: options.abortSignal, - delayMs: rateLimitRetryDelayMs(rateLimitPolicy, retryAfterHeader, Date.now()), - })) { - // pre-stream: no stall watchdog to feed - } - } catch { - upstream.abort(); - return clientCancelledResponse(); - } - // Client cancellation wins over any stale timer edge: re-check before dispatching the - // replay so the wire never starts work for a request the client already abandoned. - if (options.abortSignal?.aborted || upstream.signal.aborted) { - upstream.abort(); - return clientCancelledResponse(); - } - try { - upstreamResponse = await fetchWithTransientRetry( - recovery => { - // The first send of every replay is itself a rate-limit retry; inner transient-5xx - // recoveries keep their own label (recovery is provided for those). - noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, recovery ?? "rate-limit-429"); - return fetchWithHeaderTimeout(request.url, applyUpstreamRecoveryInit({ - method: request.method, - headers: request.headers, - body: request.body, - }, recovery), upstream.signal, connectMs, parsed.stream, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(request), - providerName: route.providerName, - modelId: route.modelId, - onCodexWsQuota: codexWsQuotaObserver(authCtx, route.provider, route.modelId), - beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) - ? createCodexReserveDispatchGuard(authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, - }), - route.provider.authMode === "forward") - .then(adoptObservedResponse); - }, - { abortSignal: upstream.signal, label: safeHostLabel(request.url), attempts: remainingTransientSendBudget(TRANSIENT_RETRY_MAX_ATTEMPTS), onSendsConsumed: noteTransientSends }, - ); - } catch (err) { - return transportFailureResponse(err); - } - } - - const captureAffinityResponse = ( - response: Response, - captureAuthCtx: CodexAuthContext = authCtx, - captureRequest: Awaited> = request, - credentialSubstituted = substituteMainCredential - || captureAuthCtx.kind === "pool" - || captureAuthCtx.kind === "main-pool", - ): void => { - if (!isCanonicalOpenAiForwardProvider(route.provider)) return; - captureCodexAffinityDiagnostic({ - inboundHeaders: req.headers, - outboundHeaders: captureRequest.headers, - authKind: captureAuthCtx.kind, - accountMode: route.codexAccountMode, - fixedAccount: isFixedCodexAccount(captureAuthCtx), - credentialSubstituted, - accountGatedModel: ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(route.modelId), - wireModelNormalized: parsed.modelId !== route.modelId, - status: response.status, - }); - }; - captureAffinityResponse(upstreamResponse); - - if (usesCodexForwardPoolAuth(authCtx, route.provider)) { - let poolRetryOutcome: number | undefined; - if (await shouldRetryCodexPoolAccountModel400( - upstreamResponse, - route.modelId, - options.abortSignal, - )) { - poolRetryOutcome = 400; - } else if (!authCtx.fixedAccount && await shouldRetryCodexPoolAccountQuota( - upstreamResponse, - options.abortSignal, - )) { - // Pre-stream only: once SSE has begun, mid-stream quota stays terminal. - // ChatGPT sometimes wraps quota exhaustion in a generic 5xx. Normalize only - // body-confirmed cases to quota evidence so cooldown and rotation both apply. - poolRetryOutcome = upstreamResponse.status >= 500 ? 429 : upstreamResponse.status; - } else if (!authCtx.fixedAccount && shouldRetryCodexPoolAccountTransient(upstreamResponse)) { - // A plain transient 5xx the same-account retry layer could not absorb. Keep the real - // status so it records as transient rather than quota. - poolRetryOutcome = upstreamResponse.status; - } - - if (poolRetryOutcome !== undefined) { - // A stored Pool 401 spent this request's account budget on its own refresh and replay, so - // nothing afterwards may be paid for out of a DIFFERENT account. One flag carries that, - // rather than a status check here as well: a quota failure has no same-account move, so - // `sameAccountOnly` makes it terminal by refusing the alternate; the gated-model 400 - // ladder does have one — retrying the account the refreshed roster still grants — and - // keeps it. An earlier revision also broke here on a non-400 outcome, which no test could - // justify because this flag already produced the identical result. - const storedReplaySpent = codex401ReplayKind === "stored"; - const retry = await retryCodexPoolOnAlternateAccount({ - callerAuthHeaders, - config, - route, - parsed, - logCtx, - options: { ...options, workflowRootId }, - firstAuthCtx: authCtx, - firstResponse: upstreamResponse, - outcomeStatus: poolRetryOutcome, - sameAccountOnly: storedReplaySpent, - upstream, - connectMs, - passthroughEstimate, - stream: parsed.stream, - onResponse: (response, retryAuthCtx, retryRequest) => { - adoptCodexWsStage(response); - captureAffinityResponse( - response, - retryAuthCtx, - retryRequest, - retryAuthCtx.kind !== "main", - ); - }, - }); - if (retry.kind === "transport") { - authCtx = retry.authCtx; - return transportFailureResponse(retry.error); - } - if (retry.kind === "retried") { - authCtx = retry.authCtx; - request = retry.request; - refreshRequestToolAliases(request); - refreshUndeclaredToolGuard(request); - upstreamResponse = retry.upstreamResponse; - selectedForwardHeaders = retry.selectedForwardHeaders; - // Keep subagent quota-failure health keyed to the account that actually served. - subagentFallbackAccountId = retry.authCtx.accountId; - } - } - } - // The deterministic route record cannot classify history it never observed (restart, expiry, - // eviction, or an older transcript). Inspect only a bounded clone of a 4xx whose exact outbound - // Responses body still carries opaque state, then rebuild once through the ordinary adapter - // sanitation path. A second rejection falls through unchanged because the guard stays armed. - const opaqueBlobRecovery = await attemptOpaqueBlobRecovery({ - response: upstreamResponse, - outboundBody: request.body, - adapterName: adapter.name, - parsed, - guard: opaqueBlobRecoveryGuard, - signal: upstream.signal, - }, rebuildAndRefetch); - if (opaqueBlobRecovery.kind === "failed") return opaqueBlobRecovery.response; - if (opaqueBlobRecovery.kind === "recovered") { - upstreamResponse = opaqueBlobRecovery.response; - continue passthroughRecovery; - } - - const recoveryContentType = upstreamResponse.headers.get("content-type")?.toLowerCase() ?? ""; - const streamedFunctionOutputCandidate = upstreamResponse.ok - && !!upstreamResponse.body - && (recoveryContentType.includes("text/event-stream") || (!recoveryContentType && parsed.stream)) - && !opaqueBlobRecoveryGuard.attempted - && outboundResponsesBodyCarriesEncryptedFunctionOutput(request.body); - if (streamedFunctionOutputCandidate) { - const preflightLog: RequestLogContext = { model: logCtx.model, provider: logCtx.provider }; - const preflight = await preflightComboStreamResponse(upstreamResponse, preflightLog, - payload => { - if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; - const type = (payload as { type?: unknown }).type; - return (type === "error" || type === "response.failed" || type === "response.incomplete") - && upstreamErrorMessageFromPayload(payload) === ENCRYPTED_FUNCTION_OUTPUT_REJECTION; - }, { - allowMissingContentType: !recoveryContentType && parsed.stream, - replayReadErrors: true, - }); - if (options.abortSignal?.aborted) return transportFailureResponse(options.abortSignal.reason); - upstreamResponse = preflight.response; - if (preflight.kind === "failed") { - const streamedOpaqueRecovery = await attemptOpaqueBlobRecovery({ - response: upstreamResponse, - outboundBody: request.body, - adapterName: adapter.name, - parsed, - guard: opaqueBlobRecoveryGuard, - signal: upstream.signal, - }, rebuildAndRefetch); - if (streamedOpaqueRecovery.kind === "failed") return streamedOpaqueRecovery.response; - if (streamedOpaqueRecovery.kind === "recovered") { - resetStreamedOpaqueBlobLogContext(logCtx); - upstreamResponse = streamedOpaqueRecovery.response; - continue passthroughRecovery; - } - logCtx.upstreamError = preflightLog.upstreamError; - logCtx.terminalHttpStatus = preflightLog.terminalHttpStatus; - logCtx.terminalErrorCode = preflightLog.terminalErrorCode; - logCtx.terminalIncompleteReason = preflightLog.terminalIncompleteReason; - } - } - // Console Go (opencode-zen / opencode-go) intermittently rejects a body it accepts seconds - // later with 400 invalid_request_error / "Invalid upload request." Replay the byte-identical - // request once after the exact gateway rejection. Single-shot guard. - // This recovery reuses the captured request; other recovery kinds still rebuild. - if (!consoleGoUploadRetryGuard.attempted) { - const uploadRejectionBody = await consoleGoUploadRejectionBody( - upstreamResponse, - consoleGoUploadRetryGuard.attempted, - upstream.signal, - ); - if (uploadRejectionBody !== undefined - && isTransientConsoleGoUploadRejection({ - status: upstreamResponse.status, - errorBody: uploadRejectionBody, - outboundUrl: request.url, - })) { - consoleGoUploadRetryGuard.attempted = true; - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - if (!upstream.signal.aborted) { - try { - await sleepWithAbort(CONSOLE_GO_UPLOAD_RETRY_DELAY_MS, upstream.signal); - } catch { return clientCancelledResponse(); } - } - if (upstream.signal.aborted) return clientCancelledResponse(); - const result = await rebuildAndRefetch("console-go-upload-retry"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - continue passthroughRecovery; - } - } - // Reasoning-effort downgrade: a rung the catalog still advertises can be refused upstream -- - // the metadata records the model's ladder, not this account's entitlement (a Muse Code - // subscription gates max on muse-spark-1.3-contributor, for example). Learn the refusal so - // later turns clamp before dispatch, then replay once at the next lower published rung - // instead of failing the turn; requestedEffort/effectiveEffort keep both values in usage. - if (!reasoningEffortDowngradeGuard.attempted) { - const rejectionText = await reasoningEffortRejectionText( - upstreamResponse, - reasoningEffortDowngradeGuard.attempted, - upstream.signal, - ); - const downgrade = rejectionText === undefined - ? undefined - : planReasoningEffortDowngrade({ - provider: route.provider, - modelId: parsed.modelId, - requested: parsed.options.reasoning, - rejectionText, - }); - if (downgrade) { - reasoningEffortDowngradeGuard.attempted = true; - parsed.options.reasoning = downgrade.effort; - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - const result = await rebuildAndRefetch("reasoning-effort-downgrade"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - continue passthroughRecovery; - } - } - break; - } - const headers = sanitizePassthroughHeaders(upstreamResponse.headers, codexSafetyBufferingOptions); - const resolvedModel = headers.get("openai-model")?.trim(); - if (resolvedModel && !logCtx.preserveResolvedModelFromRoute) logCtx.resolvedModel = resolvedModel; - if (isUsageDebugEnabled()) { - const upstreamContentType = upstreamResponse.headers.get("content-type"); - if (upstreamContentType) logCtx.usageDebugContentType = upstreamContentType; - } - // The chatgpt backend may omit Content-Type on SSE responses. Fall back to - // treating a successful body as SSE when the caller requested streaming. - const passthroughCt = headers.get("content-type")?.toLowerCase(); - const isEventStream = passthroughCt?.includes("text/event-stream") - || (plaintextV2AgentMessageToolNames.size === 0 && upstreamResponse.ok && !!upstreamResponse.body && !passthroughCt && parsed.stream); - const recordTerminalOutcome = codexForwardTerminalOutcomeRecorder( - config, - authCtx, - route.provider, - route.modelId, - logCtx, - ); - let terminalOutcomeRecorded = false; - const terminalRecorder = recordTerminalOutcome - ? (status: ResponsesTerminalStatus, httpStatusOverride?: number): void => { - if (terminalOutcomeRecorded) return; - terminalOutcomeRecorded = true; - recordTerminalOutcome(status, httpStatusOverride); - } - : undefined; - const terminalBodyWillRecord = !!terminalRecorder && upstreamResponse.ok && isEventStream; - // Capture quota from upstream response for multi-account tracking - if (usesCodexForwardPoolAuth(authCtx, route.provider)) { - // primary was the 5h window; it now carries weekly data for GPT plans. - // Prefer primary when present, fall back to secondary for compatibility. - const quotaMeta = { ...codexQuotaOutcomeMeta(upstreamResponse), ...(await codexDenialOutcomeMeta(upstreamResponse)) }; - const { applyAccountQuotaFromUpstreamHeaders } = await import("../../codex/auth-api"); - if (!isCodexWsQuotaObservedResponse(upstreamResponse)) { - applyAccountQuotaFromUpstreamHeaders(authCtx.accountId, upstreamResponse.headers, - authCtx.writerGeneration, authCtx.kind === "main-pool" ? authCtx.mainQuotaWriter : undefined, - { modelId: route.modelId, poolWriter: authCtx.kind === "pool" ? authCtx.poolQuotaWriter : undefined }); - } - if (terminalBodyWillRecord) { - options.setTerminalOutcomeRecorder?.((status, httpStatusOverride) => { - terminalRecorder(status, httpStatusOverride); - if (status === "failed" || status === "incomplete") { - const quotaFailureMessage = [httpStatusOverride, logCtx.terminalHttpStatus] - .find(value => value === 429 || value === 402); - if (!isFixedCodexAccount(authCtx) && quotaFailureMessage !== undefined) { - recordSubagentQuotaFailureForThreadSpawn( - req.headers, - subagentQuotaFailureModel, - quotaFailureMessage, - config, - subagentFallbackAccountId, - ); - } - } - options.onNativePassthroughTerminal?.(status); - }); - } else if (!shouldDeferCodexResetDerivedCooldown( - upstreamResponse, - options.deferCodexResetDerivedCooldown, - )) { - recordCodexUpstreamOutcome(config, authCtx.accountId, upstreamResponse.status, { - ...quotaMeta, - threadId: authCtx.affinityKey, - fixedAccount: authCtx.fixedAccount, - modelId: route.modelId, - probeLeaseId: codexProbeLeaseId(authCtx), - probeQuotaScope: codexProbeQuotaScope(authCtx), - writerGeneration: authCtx.writerGeneration, - // Includes a replay's second 401, which is the case that actually retires the - // account — fence it on the credential the request was holding. - ...(authCtx.kind === "pool" ? { credentialGeneration: authCtx.generation } : {}), - }); - } - } - - // Non-2xx passthrough failures must never reach Codex as an empty body — - // Codex renders that as the opaque "Unknown error" (#452). Combo attempts - // keep their typed failure envelope. Except for the classified 413 below, - // non-empty bodies are relayed verbatim - // (headers included) so pool-retry Activation B/D and client diagnostics stay intact. - // Manual-redirect policy (#914): a 3xx is relayed as-is (Location preserved - // through sanitizePassthroughHeaders) so a redirect to a dead host can never - // masquerade as a pre-connection failure after the credential was seen. - // The numeric outcome above already classified it neutral — no streak. - if (upstreamResponse.status >= 300 && upstreamResponse.status < 400) { - return new Response(upstreamResponse.body, { - status: upstreamResponse.status, - statusText: upstreamResponse.statusText, - headers: sanitizePassthroughHeaders(upstreamResponse.headers, codexSafetyBufferingOptions), - }); - } - if (!upstreamResponse.ok) { - if (options.comboAttempt) { - // No pre-read guard here: `consumeComboFailure` -> `readBoundedResponseBody` reads - // `response.body` itself and already threads the abort signal through its own read, - // and the combo contract is that this body's getter is touched exactly once (pinned by - // "captures passthrough failed usage from its original bounded body exactly once"). - // Attaching a guard would be a second `.body` access and break that contract for no - // gain, since the bounded reader owns settlement on this path. - const failure = await consumeComboFailure(upstreamResponse, options.abortSignal); - options.onConsumedComboFailure?.(failure); - return failure.response; - } - // The bounded reader owns the original body, deadline, abort settlement, and lock. - // Unsafe partial data falls back to #452's non-empty status-only JSON. - const errorText = await readDisplaySafeErrorText(upstreamResponse, upstream.signal, ""); - if (upstreamResponse.status === 413) { - return clientRequestedStream - ? streamingContextOverflowResponse(parsed._responseModelId ?? parsed.modelId, translatorBudget) - : jsonContextOverflowResponse(); - } - return formatPassthroughUpstreamError(upstreamResponse.status, errorText, { - statusText: upstreamResponse.statusText, - headers, - }); - } - - // Bun#32111 workaround: passthrough SSE uses tee()+native relay to avoid the - // async-pull segfault on Windows. Branch[0] goes directly to the Response (Bun - // native relay, never enters JS Sink.write); branch[1] is consumed in the - // background for terminal-outcome/quota inspection only. - // #314 alternative shape: win32 no-rewrite traffic follows the runtime/config - // gate; darwin no-rewrite traffic joins it only for explicit - // `streamMode: "eager-relay"` opt-in. Darwin `auto` always stays tee. The - // eager shape skips tee and uses one bounded reader with inline inspection - // (src/server/relay-eager.ts; policy: - // devlog/_fin/260731_macos_rss_retention/100_darwin_eager_optin.md). - // The bundled known-bad runtime remains on tee by default on both platforms. - if (isEventStream && upstreamResponse.body) { - // For streamed passthrough, a successful terminal response means non-error upstream status - // before relay starts. Waiting for SSE completion would retain request state across the whole - // stream; a later body failure does not undo that this destination accepted and served the turn. - commitReasoningReplayServingRoute(request.headers); - const terminalRepairPolicy = providerModelResponsesTerminalRepair( - route.providerName, - route.provider, - route.modelId, - ); - // #3761: opt-in hosted-web-search bridge. Codex always declares the hosted web_search tool, - // and this branch relays that declaration on the assumption the destination executes it. - // A KEY-auth gateway that does not (Ollama Cloud GLM) answers with a function_call named - // web_search that nothing runs, and the undeclared-tool guard below ends the turn. When the - // provider opts in, the bridge intercepts that one call, runs the search, continues the - // conversation upstream, and hands back ordinary Responses SSE — so every rewrite below, - // including the guard itself, still inspects the client-facing stream. Default OFF: without - // the opt-in this is one planner call and the relay is byte-identical to before. - const webSearchBridgeAuth = resolvePassthroughWebSearchBridgeAuth( - route.provider.webSearchBridge?.backend, - config, - openAiSidecar, - ); - const webSearchBridgePlan = planPassthroughWebSearchBridge(parsed, route.provider, { - providerName: route.providerName, - isPassthrough: true, - stream: parsed.stream === true, - auth: webSearchBridgeAuth, - }); - // Capture the binding that actually served the first leg, after its permitted reselection. - const webSearchBridgeBinding = requestBindings.get(request); - // The bridge wraps the RAW upstream body, so terminal repair below still owns the single - // client-facing terminal — the bridge drops the terminal of every intercepted leg. - const upstreamSseBody = webSearchBridgePlan - ? createPassthroughWebSearchBridgeStream({ - plan: webSearchBridgePlan, - firstLeg: upstreamResponse.body, - requestBody: request.body, - // Continuation legs replay the same built request with the executed search appended. - // The first leg already passed the recovery ladder, the outbound size ceiling, and the - // host circuit; a KEY-auth destination has no OAuth refresh to replay on a later leg. - send: (continuationBody: string) => fetchWithHeaderTimeout( - request.url, - { method: request.method, headers: request.headers, body: continuationBody }, - upstream.signal, - connectMs, - true, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - // Pacing can outlive a manual selection change. A continuation must retain the - // first leg's key and appended search result, never rebuild from the original turn. - beforeDispatch: () => { - if (webSearchBridgeBinding?.kind !== "api-key" - || !providerApiKeySelectionIsCurrent(config, route.providerName, webSearchBridgeBinding.provider)) { - throw new Error("API key selection changed during a web-search continuation"); - } - }, - providerName: route.providerName, - modelId: route.modelId, - }), - false, - ), - execute: createPassthroughWebSearchBridgeExecutor(webSearchBridgePlan, { - providerApiKey: route.provider.apiKey ?? "", - auth: webSearchBridgeAuth, - hostedTool: parsed._webSearch, - describeImages: requiresVisionPreprocessing(config, route.provider, route.modelId, route.providerName), - sidecar: config.webSearchSidecar, - }), - // Appending a search result can push the continuation past the ceiling the first leg - // was admitted under, so the same limit is re-applied before every later send. - checkOutboundBody: (continuationBody: string) => { - const result = checkOutboundBodySize(continuationBody, config.maxUpstreamBodyBytes); - return result.admitted ? undefined : describeOutboundBodyRefusal(result); - }, - signal: upstream.signal, - }) - : upstreamResponse.body; - const passthroughSseBody = terminalRepairPolicy - ? relayResponsesSseWithTerminalRepair( - upstreamSseBody, - upstream, - terminalRepairPolicy, - translatorBudget, - options.responsesTerminalRepairScheduler, - ) - : upstreamSseBody; - const repairConfig = route.provider.responsesItemIdRepair; - // Grok Build renders deltas live but reconstructs its durable assistant - // turn from the completed response snapshot. Native Responses streams - // may instead carry the complete items in output_item.done, so the - // explicit Grok compatibility marker enables strict client compatibility rewrites. - // The provider's broader snapshot/lifecycle repair remains opt-in. - const grokClientCompatibilityEnabled = logCtx.surface === "grok"; - const snapshotRepairEnabled = hasResponsesSnapshotRepair(route.provider.responsesSnapshotRepair); - const githubCopilotRepairEnabled = route.providerName === "github-copilot"; - const responseModelRewrite = parsed._responseModelId !== undefined - && parsed._responseModelId !== parsed.modelId - ? createResponsesModelPayloadRewrite(parsed._responseModelId) - : undefined; - // Compose opt-in payload rewrites into one parse/stringify pass (image-gen restore first). - const payloadRewrites = [ - createImageGenCallRestoreRewrite(imageGenCallAliases), - // #3217: a call whose namespace repeats its own name is unroutable in codex-rs. - createSelfNamedToolCallNamespaceScrubRewrite(selfNamedNamespaceScrubAuthorization), - routedMuseToolNameAliases.size > 0 - ? createMuseToolNameRestoreRewrite(routedMuseToolNameAliases) - : undefined, - routedNamespaceToolAliases.size > 0 - ? createRoutedNamespaceCallRestoreRewrite(routedNamespaceToolAliases) - : undefined, - authorizedBareNamespaceToolAliases.size > 0 - ? createRoutedNamespaceCallRestoreRewrite(authorizedBareNamespaceToolAliases) - : undefined, - hasResponsesItemIdRepair(repairConfig) - ? createResponsesItemIdPayloadRewrite(repairConfig!, translatorBudget) - : undefined, - responseModelRewrite, - ].filter((rewrite): rewrite is NonNullable => rewrite !== undefined); - // #893: sparse-snapshot gateways get field backfills AND lifecycle event - // injection at the block level, after payload rewrites. Defaults come - // from the finalized OUTBOUND body — the normalized internal tool shapes - // are not the Responses wire shapes the snapshot must mirror. - // Only validated client blocks may publish plaintext continuation state. - // Raw inspection precedes rewriting on eager relays, so it cannot own this write. - const plaintextInspector = plaintextV2AgentMessageToolNames.size > 0 - ? createSseInspector({ onCompletedResponse: rememberPassthroughResponseChecked }) - : undefined; - const plaintextEncoder = plaintextInspector ? new TextEncoder() : undefined; - const rememberPlaintextBlock = plaintextInspector - ? Object.assign((block: string): readonly string[] => { - plaintextInspector.feed(plaintextEncoder!.encode(`${block}\n\n`)); - return [block]; - }, { dispose: () => plaintextInspector.dispose() }) - : undefined; - const blockRewrites = [ - payloadRewrites.length > 0 - ? payloadRewriteAsBlockRewrite(composeSsePayloadRewrites(...payloadRewrites)) - : undefined, - routedCustomToolNames.size > 0 || routedCustomToolRepairNames.size > 0 - ? createRoutedCustomToolRestoreBlockRewrite( - routedCustomToolNames, - translatorBudget, - routedCustomToolRepairNames, - declaredWireToolNames, - ) - : undefined, - routedToolSearchNames.size > 0 - ? createRoutedToolSearchRestoreBlockRewrite(routedToolSearchNames, translatorBudget) - : undefined, - githubCopilotRepairEnabled - ? createGithubCopilotResponsesBlockRewrite(translatorBudget) - : undefined, - grokClientCompatibilityEnabled - ? createGrokResponsesControlFrameBlockRewrite() - : undefined, - grokClientCompatibilityEnabled - ? createGrokResponsesSparseTerminalBlockRewrite(translatorBudget) - : undefined, - snapshotRepairEnabled - ? createResponsesSnapshotBlockRewrite(outboundRequestBody, translatorBudget) - : undefined, - plaintextV2AgentMessageToolNames.size > 0 - ? payloadRewriteAsBlockRewrite(createPlaintextV2AgentMessageCallRestoreRewrite( - plaintextV2AgentMessageToolNames, plaintextV2AgentMessageAliasedToolNames, - )) - : undefined, - createResponsesFieldBackfillBlockRewrite(), - functionRepairSchemas.size > 0 - ? createResponsesFunctionToolRepairBlockRewrite(functionRepairSchemas, translatorBudget) - : undefined, - // Last: every rewrite above can still rename or reshape a call item, so the guard must - // compare the names the client will actually receive against the declared catalog. - undeclaredToolGuardActive - ? createUndeclaredToolCallGuardBlockRewrite( - declaredWireToolNames, - declaredNamelessClientCallTypes, - providerExecutedCallTypes, - declaredBareWireToolNames, - ) - : undefined, - rememberPlaintextBlock, - ].filter((rewrite): rewrite is NonNullable => rewrite !== undefined); - const clientBlockRewrite = blockRewrites.length > 0 - ? composeSseBlockRewrites(...blockRewrites) - : undefined; - const needsClientRewrite = clientBlockRewrite !== undefined; - // #864: win32 rewrite traffic must never enter the tee()+JS-pull chain - // (Bun#32111 JS-sink segfault — text frames pass, the terminal block is - // lost). The eager single reader applies the same rewrites inline. - const win32EagerRewrite = isWin32EagerRewrite(process.platform, needsClientRewrite); - const eagerPath = selectEagerPath( - process.platform, - needsClientRewrite, - config.streamMode ?? "auto", - ); - // A successful Codex WS upgrade is a push source. If it entered tee(), - // the inspection branch could drain continuously while the slow client - // branch retained bytes without a bound. Force the existing bounded, - // single-reader relay before tee; HTTP fallback responses stay unmarked. - const forceCodexWsEagerRelay = isCodexWsUpstreamResponse(upstreamResponse); - const inlineEagerRewrite = needsClientRewrite - && (forceCodexWsEagerRelay || win32EagerRewrite || eagerPath?.useEagerRelay === true); - if (forceCodexWsEagerRelay || eagerPath?.useEagerRelay || win32EagerRewrite) { - const turnAc = new AbortController(); - linkAbortSignal(upstream, turnAc.signal); - registerTurn(turnAc, options.turnAdmissionLease); - const reportNativeTerminal = recordTerminalOutcomes - ? (status: ResponsesTerminalStatus, httpStatusOverride?: number) => { - terminalRecorder?.(status, httpStatusOverride); - if (status === "failed" || status === "incomplete") { - const quotaFailureMessage = [httpStatusOverride, logCtx.terminalHttpStatus] - .find(value => value === 429 || value === 402); - if (!isFixedCodexAccount(authCtx) && quotaFailureMessage !== undefined) { - recordSubagentQuotaFailureForThreadSpawn( - req.headers, - subagentQuotaFailureModel, - quotaFailureMessage, - config, - subagentFallbackAccountId, - ); - } - } - options.onNativePassthroughTerminal?.(status); - } - : undefined; - const inspector = createSseInspector({ - onTerminal: reportNativeTerminal, - logCtx, - onCompletedResponse: rememberPassthroughResponse && plaintextV2AgentMessageToolNames.size === 0 ? rememberPassthroughResponseChecked : undefined, - onParsedPayload: noteInspectedPayload, - onFirstOutput: options.onFirstOutput, - pinCompletedResponseIdToFirstSeen: githubCopilotRepairEnabled, - }); - const eagerBody = relaySseEagerBounded(passthroughSseBody, turnAc, { - inspectChunk: chunk => inspector.feed(chunk), - finishInspection: () => inspector.finish(), - disposeInspection: () => inspector.dispose(), - // Stream lifetime follows the protocol terminal even when this request - // has no outcome callback configured (reported() would stay false). - sawTerminal: () => inspector.terminalSeen(), - ...(clientBlockRewrite - ? { rewriteBlocks: clientBlockRewrite } - : {}), - onSynthetic: (kind, reason) => { - if (!reportNativeTerminal) return; - if (kind === "incomplete") { - logCtx.terminalSource = "synthetic"; - reportNativeTerminal("incomplete"); - } else if (reason === "upstream_error") { - logCtx.terminalSource = "synthetic"; - reportNativeTerminal("failed", logCtx.terminalHttpStatus ?? 502); - } else { - logCtx.transportPhase = "mid_stream"; - logCtx.terminalSource = "synthetic"; - if (logCtx.activeAttempt) logCtx.activeAttempt.streamAborted = true; - reportNativeTerminal("failed", 502); - } - }, - onClientCancel: () => { - responseCompletionCancelled = true; - options.onNativePassthroughCancel?.(); - }, - onDone: () => unregisterTurn(turnAc), - }, { - clientGoneSignal: options.abortSignal, - terminalBoundary: codexSafetyBufferingOptions, - ...(inlineEagerRewrite ? { rewriteBudget: translatorBudget } : {}), - ...(logCtx.upstreamError === undefined ? {} : { upstreamError: logCtx.upstreamError }), - }); - // When selected, this relay closes response.completed even if upstream - // keeps the connection alive. Marked Codex WS traffic, Windows - // forced-rewrite traffic, and Darwin explicit eager traffic apply - // client rewrites inline rather than via the tee()+JS-pull chain. - if (!headers.has("content-type")) headers.set("content-type", "text/event-stream"); - return markEagerRelaySseResponse( - markNativePassthroughSseResponse(new Response(eagerBody, { - status: upstreamResponse.status, - headers, - })), - ); - } - const [nativeBody, inspectBody] = passthroughSseBody.tee(); - const turnAc = new AbortController(); - const clientGone = new AbortController(); - linkAbortSignal(upstream, turnAc.signal); - registerTurn(turnAc, options.turnAdmissionLease); - const inspectionConsumerOptions = { - // Request abort can reject the fetch body before the response cancel hook runs. - clientGoneSignal: options.abortSignal - ? AbortSignal.any([clientGone.signal, options.abortSignal]) - : clientGone.signal, - drainBounds: { ms: 15_000, bytes: 32 * 1024 * 1024 }, - upstream, - pinCompletedResponseIdToFirstSeen: githubCopilotRepairEnabled, - onParsedPayload: noteInspectedPayload, - }; - if (recordTerminalOutcomes) { - // A real terminal was parsed from the (teed) inspection stream — record it as the outcome - // even if the client has already disconnected: the turn genuinely reached that terminal, so - // it must log as completed/failed, not be dropped or downgraded to a cancel (#44). A pure - // client-cancel (no terminal seen) is finalized separately via consumeForInspection's onCancel. - const reportNativeTerminal = (status: ResponsesTerminalStatus, httpStatusOverride?: number) => { - terminalRecorder?.(status, httpStatusOverride); - if (status === "failed" || status === "incomplete") { - const quotaFailureMessage = [httpStatusOverride, logCtx.terminalHttpStatus] - .find(value => value === 429 || value === 402); - if (!isFixedCodexAccount(authCtx) && quotaFailureMessage !== undefined) { - recordSubagentQuotaFailureForThreadSpawn( - req.headers, - subagentQuotaFailureModel, - quotaFailureMessage, - config, - subagentFallbackAccountId, - ); - } - } - options.onNativePassthroughTerminal?.(status); - }; - consumeForInspection( - inspectBody, - reportNativeTerminal, - turnAc.signal, - () => unregisterTurn(turnAc), - logCtx, - () => { - responseCompletionCancelled = true; - options.onNativePassthroughCancel?.(); - }, - rememberPassthroughResponse && plaintextV2AgentMessageToolNames.size === 0 ? rememberPassthroughResponseChecked : undefined, - options.onFirstOutput, - inspectionConsumerOptions, - ); - } else { - consumeForResponseLogMetadata( - inspectBody, - logCtx, - turnAc.signal, - () => unregisterTurn(turnAc), - rememberPassthroughResponse && plaintextV2AgentMessageToolNames.size === 0 ? rememberPassthroughResponseChecked : undefined, - options.onFirstOutput, - inspectionConsumerOptions, - ); - } - if (!headers.has("content-type")) headers.set("content-type", "text/event-stream"); - // Windows was handled by the eager terminal-aware branch above. Remaining - // tee traffic can use the JS relay to close on a protocol terminal and to - // convert a mid-stream reset into a clean response.failed event. - const rewrittenBody = clientBlockRewrite !== undefined - ? relaySseWithBlockRewrite(nativeBody, clientBlockRewrite, translatorBudget) - : nativeBody; - const clientBody = relaySseWithFailedTail( - rewrittenBody, - upstream, - reason => { - responseCompletionCancelled = true; - clientGone.abort(reason); - }, - { upstreamError: logCtx.upstreamError, terminalBoundary: codexSafetyBufferingOptions }, - ); - return markNativePassthroughSseResponse(new Response(clientBody, { - status: upstreamResponse.status, - headers, - })); - } - if (headers.get("content-type")?.toLowerCase().includes("application/json")) { - // Bounded whole-body read: a non-streaming upstream JSON body is fully materialized - // here (and again by the request-log finalizer and the WebSocket bridge's reframing), - // so an unbounded .text() would let a hostile or stuck upstream grow proxy memory - // without limit. This path is no longer rare — WebSocket turns for models whose - // streaming terminal event is unreliable are deliberately answered with bounded JSON. - // Oversize and stall deadlines both fail closed; a partial body is never parsed. - const bounded = await readBoundedResponseBody(upstreamResponse, UPSTREAM_JSON_BODY_READ_OPTIONS); - if (bounded.oversized) { - return formatErrorResponse(502, "upstream_error", "upstream JSON response exceeded the safe body limit"); - } - if (bounded.truncated) { - return formatErrorResponse(502, "upstream_error", "upstream JSON response stalled before completing"); - } - const text = bounded.text; - inspectResponseLogJson(logCtx, text); - let plaintextV2RestoreFailed = false; - let clientJson = (() => { - const restoredNamespace = restoreRoutedNamespaceCallsInJson( - scrubSelfNamedToolCallNamespaceInJson( - restoreMuseToolNamesInJson( - restoreImageGenCallsInJson(text, imageGenCallAliases), - routedMuseToolNameAliases, - ), - selfNamedNamespaceScrubAuthorization, - ), - routedNamespaceToolAliases, - ); - const restoredAuthorizedBareNamespace = restoreRoutedNamespaceCallsInJson( - restoredNamespace, - authorizedBareNamespaceToolAliases, - ); - const restored = restoreRoutedCustomCallsInJson( - restoredAuthorizedBareNamespace, - routedCustomToolNames, - routedCustomToolRepairNames, - declaredWireToolNames, - ); - const restoredToolSearch = restoreRoutedToolSearchCallsInJson( - restored, - routedToolSearchNames, - ); - const normalizedJson = normalizeFunctionCompletionJson(restoredToolSearch); - const plaintextRestore = restorePlaintextV2AgentMessageCallsInJsonResult( - normalizedJson, plaintextV2AgentMessageToolNames, plaintextV2AgentMessageAliasedToolNames, - ); - plaintextV2RestoreFailed = plaintextRestore.overflowed; - const repaired = plaintextRestore.value; - const modelRewritten = parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId - ? rewriteResponsesModelJson(repaired, parsed._responseModelId) - : repaired; - return modelRewritten; - })(); - if (plaintextV2RestoreFailed) { - return formatErrorResponse(502, "upstream_error", PLAINTEXT_V2_AGENT_MESSAGE_RESTORE_OVERFLOW_MESSAGE); - } - // #1700: same fail-closed policy as the SSE relay above. Both the plain JSON answer and - // the reframed-SSE branch below are built from this body, so one check covers them. This - // runs BEFORE the continuation cache write below: a refused turn must not become state a - // later `previous_response_id` replay can expand from. - if (undeclaredToolGuardActive) { - const undeclared = (() => { - try { - return undeclaredToolCallNameInResponse( - JSON.parse(clientJson), - declaredWireToolNames, - declaredNamelessClientCallTypes, - providerExecutedCallTypes, - declaredBareWireToolNames, - ); - } catch { - return undefined; - } - })(); - if (undeclared !== undefined) { - return formatErrorResponse(502, "upstream_error", undeclaredToolCallMessage(undeclared)); - } - clientJson = normalizeDefaultNamespaceInJson( - clientJson, - declaredWireToolNames, - declaredBareWireToolNames, - ); - } - commitReasoningReplayServingRoute(request.headers); - try { - rememberPassthroughResponseChecked( - JSON.parse(text) as { id?: unknown; output?: unknown; status?: unknown; model?: unknown }, - ); - } catch { /* non-JSON despite content-type; recording is best-effort */ } - // #875: the transport-neutral reliability policy forced a bounded JSON - // upstream for a client that asked for SSE. Reframe the completed JSON - // as the canonical terminal SSE sequence (created → output_item.done → - // terminal → [DONE]) so Codex commits the turn instead of hanging on a - // stream that never closes. Non-streaming clients keep the plain JSON. - if (clientRequestedStream === true - && options.inboundTransport !== "websocket" - && providerModelResponsesUpstreamStreaming(route.providerName, route.provider, route.modelId) === false - && route.provider.adapter === "openai-responses") { - let completed: Record | undefined; - try { - const parsedCompleted = JSON.parse(clientJson) as unknown; - if (!parsedCompleted || typeof parsedCompleted !== "object" || Array.isArray(parsedCompleted)) { - throw new TypeError("bounded Responses JSON is not an object"); - } - let candidate = parsedCompleted as Record; - // The bounded-JSON answer bypasses the SSE relay, so it also bypasses - // the SSE item-id rewrite. Apply the same client-facing normalization - // here or this policy would silently disable id repair for the very - // providers that need it (raw record already happened above). - if (hasResponsesItemIdRepair(route.provider.responsesItemIdRepair)) { - candidate = repairResponsesJsonItemIds(candidate, route.provider.responsesItemIdRepair!, translatorBudget); - } - completed = candidate; - } catch { - // Non-JSON despite content-type: fall through to the plain relay. - } - if (completed) { - let stream: ReadableStream; - try { - stream = responsesJsonToSseStream(completed); - } catch (error) { - if (error instanceof RangeError) { - return formatErrorResponse( - 502, - "upstream_error", - "upstream JSON response exceeded the synthesized SSE item limit", - ); - } - throw error; - } - const sseHeaders = sanitizePassthroughHeaders(headers, codexSafetyBufferingOptions); - sseHeaders.set("content-type", "text/event-stream"); - sseHeaders.set("cache-control", "no-store"); - return new Response(stream, { - status: upstreamResponse.status, - statusText: upstreamResponse.statusText, - headers: sseHeaders, - }); - } - } - // WS turns reframe this JSON into events in the bridge, which is the - // other relay-free path — normalize ids so both bounded-JSON paths agree. - const outboundJson = options.inboundTransport === "websocket" - && providerModelResponsesUpstreamStreaming(route.providerName, route.provider, route.modelId) === false - && hasResponsesItemIdRepair(route.provider.responsesItemIdRepair) - ? (() => { - try { - return JSON.stringify(repairResponsesJsonItemIds( - JSON.parse(clientJson) as Record, - route.provider.responsesItemIdRepair!, - translatorBudget, - )); - } catch { - return clientJson; - } - })() - : clientJson; - return new Response(outboundJson, { - status: upstreamResponse.status, - statusText: upstreamResponse.statusText, - headers, - }); - } - if (plaintextV2AgentMessageToolNames.size > 0) { - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already closed */ } - return formatErrorResponse(502, "upstream_error", "plaintext V2 agent-message response used an unsupported content type"); - } - // An unclassified passthrough body is relayed directly and has no bounded completion observer; - // use the same non-error-status success boundary as SSE instead of retaining per-stream state. - commitReasoningReplayServingRoute(request.headers); - const body = relayWithAbort(upstreamResponse.body, upstream); - const turnAc = new AbortController(); - const tracked = body ? trackStreamLifetime(body, turnAc, undefined, options.turnAdmissionLease) : null; - return new Response(tracked, { - status: upstreamResponse.status, - headers, - }); - } finally { - if (hostAdmissionLease) { - releaseUpstreamHostAdmission(hostAdmissionLease); - releaseCodexAuthContextProbeLease(authCtx); - } - } - } - - // Tool results are PAIRED by call_id. parseRequest writes it into OcxToolResultMessage.toolCallId - // (parser.ts:738/752) without validating it, because inputItemSchema's permissive catch-all - // (schema.ts:106) accepts a tool item whose strict schema failed only for a missing call_id. A - // translating adapter then consumes `toolCallId: string` holding undefined: kiro-wire.ts:32 - // TypeErrors, ollama-native.ts:334 throws, and anthropic.ts:775 sends - // "[tool_result without adjacent tool_use: undefined]" upstream (issue #3259). - // - // This CANNOT move into the schema. parseRequest (:2812) runs before the passthrough branch - // (:3719), so a parse-time rejection would also kill forward/key passthrough and routed - // compaction — paths that never read context.messages, build from _rawBody, and already - // degrade an unpaired output to "[tool output for unknown call]" on their own. - // - // Keyed on the adapter, not on position: routedCompaction skips the passthrough branch above - // yet still builds from _rawBody (see the :3703 comment). - if (!("passthrough" in adapter && adapter.passthrough)) { - const unpaired = parsed.context.messages.find( - message => message.role === "toolResult" - && (typeof (message as { toolCallId?: unknown }).toolCallId !== "string" - || (message as { toolCallId: string }).toolCallId.length === 0), - ); - if (unpaired) { - // Never interpolate the tool output: this message reaches the client and the logs. - return formatErrorResponse( - 400, - "invalid_request_error", - "tool result requires a non-empty string call_id", - ); - } - } - - // Image / web-search sidecars: plan once, then dispatch with runTurn-aware priority. - // Routed-compaction turns must NOT hit the image bridge: compaction clears tools/_webSearch but - // leaves _imageGeneration, so planImageBridge would activate and return a normal Responses - // completion instead of the synthetic compaction item Codex expects (#424). - // - // Web-search's loop only supports buildRequest/fetch/parseStream — NOT adapter.runTurn. Sending - // Cursor/runTurn requests into runWithWebSearch produces empty HTTP failures. So: - // - non-runTurn: web-search wins over image when both eligible (documented priority) - // - runTurn: image bridge may run (it supports runTurn); web-search is skipped so runTurn - // can proceed for web-search-only turns - const wsPlan = !routedCompaction - ? planWebSearch(config, parsed, false, route.provider, route.modelId, openAiSidecar, { - admission: options.admission, codexAuthPolicy: options.codexAuthPolicy, providerName: route.providerName, - }) - : undefined; - const imgPlan = !routedCompaction ? await planImageBridge(config, parsed, route.provider) : undefined; - const vidPlan = !routedCompaction ? await planVideoBridge(config, parsed, route.provider) : undefined; - const canRunWebSearch = !!wsPlan && !adapter.runTurn; - const rotateSidecarProviderOn429 = async ( - retryAfter: string | null, - responseHeaders?: Headers, - ): Promise => { - const rotated = rotateProviderTransportOn429(config, route.providerName, route.provider, { - retryAfter, - now: Date.now(), - attemptedKey: route.provider.apiKey, - promptCacheKey: parsed.options.promptCacheKey, - }); - if (rotated) { - route.provider = rotated; - } else if ( - // A POSITIVE gate, not an early return. An early `return null` here made every later arm - // unreachable: Anthropic never has a genericFailoverAccountId (isGenericFailoverProvider - // excludes it), so its sidecar 429s died on this guard before the Anthropic arm below - // could ever be considered. - genericFailoverAccountId - && genericFailovers < GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST - && isGenericOAuthFailoverEnabled(config, route.providerName) - ) { - // Intersection with the request's shared budget. The sidecar replay is dispatched by the - // web-search/image loop and never reaches `onSendsConsumed`, so this reservation is the - // charge; a refusal returns null and the caller keeps the real 429 it already has. - const hop = reserveCredentialHop( - "auth-recovery", - `${route.providerName}|${route.modelId}|sidecar-oauth-429`, - ); - if (!hop.allowed) return null; - const nextAccountId = rotateGenericOAuthAccountOn429( - config, - route.providerName, - genericFailoverAccountId, - retryAfter, - ); - if (!nextAccountId) { - hop.permit?.release(); - return null; - } - try { - const snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); - genericFailovers += 1; - if (!await applyFailoverSnapshot(snapshot)) { - hop.permit?.release(); - return null; - } - } catch { - hop.permit?.release(); - return null; - } - hop.permit?.use(); - } else if ( - // Anthropic's pool is excluded from generic failover, so without this arm a 429 inside a - // web-search or image-bridge turn was terminal even with the pool fully enabled -- while - // the very same 429 on the main response path rotated. - anthropicPoolAccountId - && anthropicPoolFailovers < ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST - ) { - // Same intersection for the Anthropic roster: its own per-request bound still applies, - // and the shared budget decides whether this request may spend another send at all. - const hop = reserveCredentialHop( - "auth-recovery", - `${route.providerName}|${route.modelId}|sidecar-anthropic-429`, - ); - if (!hop.allowed) return null; - const nextAccountId = rotateAnthropicAccountOn429( - config, - anthropicPoolAccountId, - retryAfter, - anthropicSessionKey, - Date.now(), - responseHeaders, - ); - if (!nextAccountId) { - hop.permit?.release(); - return null; - } - try { - // Deliberately NOT applyFailoverSnapshot: that helper exists to pair per-account routing - // metadata (Copilot origin, Antigravity project, Kiro context) with its bearer. Anthropic - // carries none, and getAnthropicPoolAccessToken is what enforces its fail-closed - // local-cli credential rule. Both existing Anthropic rotation sites apply the token the - // same way. - const admitted = await commitResolvedOAuthSelection(await getAnthropicPoolAccessSnapshot(nextAccountId)); - if (!admitted) throw new Error("OAuth selection changed during recovery"); - anthropicPoolAccountId = admitted.accountId; - anthropicPoolFailovers += 1; - route.provider = { ...route.provider, apiKey: admitted.accessToken }; - logCtx.provider = formatAnthropicProviderForLog("anthropic", admitted.accountId, config); - } catch { - hop.permit?.release(); - return null; - } - hop.permit?.use(); - } else { - // No key pool, no generic OAuth roster, no Anthropic pool could produce a replacement - // credential. The 429 is terminal for this sidecar turn. - return null; - } - const rotatedAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: route.provider, - adapterName: rotatedAdapter.name, - }); - return rotatedAdapter; - }; - if ((imgPlan || vidPlan) && canRunWebSearch) { - // Web search takes priority when both are active — the media bridge cannot run - // alongside runWithWebSearch. Surface a runtime signal so the user knows their - // configured video/image bridge was skipped for this turn, rather than silently - // dropping a paid capability. - if (vidPlan) console.warn("[videos] video bridge skipped: web search is active for this turn"); - if (imgPlan) console.warn("[images] image bridge skipped: web search is active for this turn"); - } - if ((imgPlan || vidPlan) && (!wsPlan || adapter.runTurn)) { - // The image bridge detects a hosted image_generation tool and requires streaming. - // The video bridge activates from config and injects a tool — it also needs streaming - // (the loop returns SSE). For video-only (no imgPlan) on a non-streaming request, skip - // the bridge entirely so enabling the feature doesn't break ordinary non-streaming traffic. - if (!parsed.stream) { - if (imgPlan) { - return formatErrorResponse(400, "invalid_request_error", "image bridge requires stream=true"); - } - // Video-only: skip bridge for non-streaming requests - } else { - // Replace any pre-existing image_gen/video_gen aliases instead of appending duplicate wire names. - const priorTools = parsed.context.tools ?? []; - const bridgeTools = [...priorTools.filter(t => { - if (t.imageGeneration) return false; - if (t.videoGeneration) return false; - if (imgPlan && imgPlan.toolNames.has(t.name)) return false; - if (imgPlan && t.namespace && imgPlan.toolNames.has(namespacedToolName(t.namespace, t.name))) return false; - // Only strip unnamespaced video_gen aliases — a namespaced MCP video_gen is left alone. - if (vidPlan && !t.namespace && vidPlan.toolNames.has(t.name)) return false; - return true; - })]; - const existingNames = new Set(bridgeTools.map(t => t.name)); - if (imgPlan && !existingNames.has(IMAGE_GEN_TOOL_NAME)) bridgeTools.push(buildImageTool()); - if (vidPlan && !existingNames.has(VIDEO_GEN_TOOL_NAME)) bridgeTools.push(buildVideoTool()); - parsed.context.tools = bridgeTools; - // Hosted image_generation tool_choice / allowed_tools must target the synthetic function name. - // Gate on imgPlan — in a video-only turn buildImageTool() was never injected, so rewriting - // image_generation/image_gen aliases would add an undeclared tool that strict upstreams reject. - const tc = parsed.options.toolChoice; - if (imgPlan && tc && typeof tc === "object" && "allowedTools" in tc && Array.isArray(tc.allowedTools)) { - const mapped = tc.allowedTools.map(name => - name === "image_generation" || name === "image_gen" || (imgPlan.toolNames.has(name) ?? false) - ? IMAGE_GEN_TOOL_NAME - : name, - ); - parsed.options.toolChoice = { ...tc, allowedTools: [...new Set(mapped)] }; - } else if (imgPlan && tc && typeof tc === "object" && "name" in tc && typeof tc.name === "string" - && (tc.name === "image_generation" || imgPlan.toolNames.has(tc.name))) { - parsed.options.toolChoice = { ...tc, name: IMAGE_GEN_TOOL_NAME }; - } - const imageProviderFetch = providerFetch( - route.provider, - options.codexWsRuntimeIdentity, - { providerName: route.providerName, modelId: route.modelId }, - ); - const imgResponse = await runWithImageBridge({ - parsed, adapter, - incomingMeta: { headers: selectedForwardHeaders, abortSignal: options.abortSignal, translatorBudget }, - ...(imgPlan ? { plan: imgPlan } : {}), - ...(vidPlan ? { videoPlan: vidPlan } : {}), - forwardHeaders: selectedForwardHeaders, - onAttemptSend: (recovery?: AttemptRecoveryKind) => - noteAttemptSend(logCtx.activeAttempt, logCtx.usageLogInputTokens, recovery), - abortSignal: options.abortSignal, - maxRounds: imgPlan && vidPlan - ? clampImageMaxRounds(Math.min(config.images?.maxRounds ?? 3, config.images?.videoMaxRounds ?? 2)) - : imgPlan - ? clampImageMaxRounds(config.images?.maxRounds) - : clampImageMaxRounds(config.images?.videoMaxRounds ?? 2), - connectTimeoutMs: config.connectTimeoutMs ?? 200_000, - stallTimeoutSec: config.stallTimeoutSec, - waitForRequestSlot: imageProviderFetch.waitForPacing, - fetchImpl: imageProviderFetch.unpacedFetch ?? imageProviderFetch, - fetchForRequest: (request, iterParsed) => { - const fetch = providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(request, iterParsed), - providerName: route.providerName, modelId: route.modelId, - }); - return fetch.unpacedFetch ?? fetch; - }, - onRequestBuilt: request => { - recordAdapterReasoning(logCtx, request); - recordAdapterTier(logCtx, request); - }, - ...(vidPlan?.timeoutMs ? { videoTimeoutMs: vidPlan.timeoutMs } : {}), - onUsage: usage => { - // Cursor may assign _cursorConversationId inside the image loop's first runTurn; - // backfill so Logs can filter/total that opening request (parity with the normal - // runTurn branch). - if (!logCtx.conversationId && parsed._cursorConversationId) { - logCtx.conversationId = normalizeLogConversationId(parsed._cursorConversationId); - } - logCtx.usageFromBridge = true; - if (usage) { - logCtx.usage = usage; - if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; - } - }, - on429: rotateSidecarProviderOn429, - retryOn429Policy: rateLimitRetryPolicyFor(route.provider), - ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), - ...(options.forceEmptyResponseId ? { forceEmptyResponseId: true } : {}), - onCompletedResponse: (response, providerState) => { - commitReasoningReplayServingRoute(); - rememberKiroDeliveredFinalAnswer(adapter.name, response); - rememberResponseState( - parsed._rawBody, - response, - continuationStateForResponse(providerState), - responseStateOptions(adapterNeedsForcedContinuation(adapter.name)), - ); - notifyResponseComplete(response); - }, - }); - if (imgResponse.body) { - const imgTurnAc = new AbortController(); - imgTurnAc.signal.addEventListener("abort", cancelResponseCompletion, { once: true }); - return new Response(trackStreamLifetime(imgResponse.body, imgTurnAc, undefined, options.turnAdmissionLease), { - status: imgResponse.status, - headers: imgResponse.headers, - }); - } - return imgResponse; - } // end else (streaming bridge) - } - - // Web-search sidecar: Codex enabled web_search but this is a routed (non-OpenAI) model that can't - // run it server-side. Expose web_search as a function tool and run searches via the gpt-mini sidecar - // through the ChatGPT passthrough, looping until the model answers. Otherwise take the normal path. - // Placed BEFORE the runTurn early-return for non-runTurn adapters so dual-tool turns dispatch - // through web-search instead of being swallowed. runTurn adapters never enter this branch. - if (canRunWebSearch && wsPlan) { - parsed.context.tools = [...(parsed.context.tools ?? []), buildWebSearchTool()]; - // Resolve the mutable route at send time: a 429 rotation replaces route.provider, so retaining - // one pre-rotation providerFetch would keep the old credential and transport pin. - const routedProviderFetch = ((input: Parameters[0], init?: RequestInit) => - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - providerName: route.providerName, - modelId: route.modelId, - })(input, init)) as typeof globalThis.fetch; - const wsResponse = await runWithWebSearch({ - parsed, adapter, - fetchForRequest: (request, iterParsed) => providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(request, iterParsed), - providerName: route.providerName, modelId: route.modelId, - }), - incomingMeta: { - headers: selectedForwardHeaders, - abortSignal: options.abortSignal, - translatorBudget, - providerFetch: routedProviderFetch, - }, - backend: wsPlan.backend, - forwardProvider: wsPlan.forwardSidecar?.provider, - anthropicSidecar: wsPlan.anthropicSidecar, - xaiSidecar: wsPlan.xaiSidecar, - geminiSidecar: wsPlan.geminiSidecar, - xaiSearchOptions: wsPlan.xaiSearchOptions, - // The exa key never rides the plan: read it from config at unpack time (L9). - ...(wsPlan.exaConfigured ? { exaApiKey: config.webSearchSidecar?.exaApiKey } : {}), - hostedTool: wsPlan.hostedTool, - selectedForwardHeaders: wsPlan.forwardSidecar?.headers ?? selectedForwardHeaders, - settings: wsPlan.settings, - maxSearches: wsPlan.maxSearches, - forceEmptyResponseId: true, - abortSignal: options.abortSignal, - ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), - onRequestBuilt: request => { - recordAdapterReasoning(logCtx, request); - recordAdapterTier(logCtx, request); - }, - onAttemptSend: (recovery?: AttemptRecoveryKind) => - noteAttemptSend(logCtx.activeAttempt, logCtx.usageLogInputTokens, recovery), - onUsage: usage => { - logCtx.usageFromBridge = true; - if (usage) { - logCtx.usage = usage; - if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; - } - }, - recordSidecarOutcome: wsPlan.forwardSidecar?.recordOutcome, - connectTimeoutMs: config.connectTimeoutMs ?? 200_000, - routedModelStallTimeoutMs: wsPlan.routedModelStallTimeoutMs, - stallTimeoutSec: wsPlan.stallTimeoutSec, - streamRoutedModelOutput: wsPlan.streamRoutedModelOutput, - on429: rotateSidecarProviderOn429, - retryOn429Policy: rateLimitRetryPolicyFor(route.provider), - onCompletedResponse: response => { - commitReasoningReplayServingRoute(); - notifyResponseComplete(response); - }, - }); - // Register the sidecar stream as an active turn so drainAndShutdown waits for (or aborts) - // in-flight web-search turns instead of skipping them during graceful shutdown. - if (wsResponse.body) { - const wsTurnAc = new AbortController(); - wsTurnAc.signal.addEventListener("abort", cancelResponseCompletion, { once: true }); - return new Response(trackStreamLifetime(wsResponse.body, wsTurnAc, undefined, options.turnAdmissionLease), { - status: wsResponse.status, - headers: wsResponse.headers, - }); - } - return wsResponse; - } - - // Empty-completion guard (codex-router PR #145 port): a 200 that completes with no output - // text and no tool call is a failure the client cannot see — it silently records the turn as - // done. The guard holds pre-content adapter events, suppresses the terminal of an empty - // turn, retries the IDENTICAL request once, and surfaces a stated error when the retry is - // empty or fails. This is a top-level config opt-in; OCX_EMPTY_COMPLETION_RETRY=0 is a - // disable-only emergency override. Compaction turns and combo attempts keep their own - // machinery (the combo preflight already handles empty streams). Native Chat-to-Chat - // requests return from handleChatCompletions before entering Responses core, so they are - // intentionally outside this guard and retain their existing one-send wire behavior. - const emptyCompletionGuardEnabled = - emptyCompletionRetryEnabled(config) - && !options.comboAttempt - && !routedCompaction; - - if (adapter.runTurn) { - const runTurnAbort = new AbortController(); - const cleanupRunTurnAbort = linkAbortSignal(runTurnAbort, options.abortSignal); - const queue = createAdapterEventQueue({ - onBacklogExceeded: () => runTurnAbort.abort(), - }); - const refreshRunTurnSelection = async (): Promise => { - if (selectionIsCurrent(adapterBindings.get(runTurnAdapter))) return; - await refreshRunTurnAdapter(parsed); - bindRouteReasoningReplayScope({ parsed, providerName: route.providerName, provider: route.provider, - adapterName: runTurnAdapter.name, oauthCredentialSnapshot: replayOAuthCredentialSnapshot }); - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, runTurnAdapter.name, logCtx.accountLogLabel); - }; - // Initial admission must settle before the streaming Response commits HTTP 200. - // Let the outer Responses facade preserve the local retryable-429 contract. - try { - await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, runTurnAbort.signal); - } catch (error) { - cleanupRunTurnAbort(); - queue.close(); - throw error; - } - // One attempt of the runTurn transport, against an explicit queue. The - // empty-completion guard re-invokes the IDENTICAL turn (same parsed request, - // same forwarded headers, same abort signal) through a fresh queue, so the - // attempt body must not capture the first queue. Each attempt consumes its - // own provider pacing slot (#1584): retries are paced like first attempts. - const runTurnAttempt = async ( - targetQueue: AdapterEventQueue, - recovery?: AttemptRecoveryKind, - pacingSlotAcquired = false, - ): Promise => { - try { - if (!pacingSlotAcquired) { - await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, runTurnAbort.signal); - } - await refreshRunTurnSelection(); - noteAttemptSend(logCtx.activeAttempt, logCtx.usageLogInputTokens, recovery); - const runTurnProviderFetch = providerFetch( - route.provider, - options.codexWsRuntimeIdentity, - { - providerName: route.providerName, - modelId: route.modelId, - // runTurnAttempt acquired this logical turn's first physical-request slot above. - // Cursor HTTP/1.1 consumes it for RunSSE; every BidiAppend and redial then waits on - // the same provider queue through this stateful wrapper. - pacingSlotAcquired: true, - }, - ); - await runTurnAdapter.runTurn?.( - parsed, - { - headers: selectedForwardHeaders, - abortSignal: runTurnAbort.signal, - translatorBudget, - providerFetch: runTurnProviderFetch, - // The only way the request budget reaches a transport the adapter owns. Without it - // a Cursor turn's inner ladder was three physical sends the cap read as one. - ...(adapterSendBudget ? { sendBudget: adapterSendBudget } : {}), - }, - targetQueue.push, - ); - } catch (err) { - targetQueue.push(err instanceof RequestPacingQueueOverloadError - ? { - type: "error", - status: 429, - errorType: "rate_limit_error", - retryable: true, - message: err.message, - } - : { - type: "error", - message: err instanceof Error ? err.message : String(err), - }); - } finally { - // Cursor assigns a stable conversation id inside runTurn on the first headerless - // turn; backfill so Logs can filter/total that opening request (#330 / #522). - if (!logCtx.conversationId && parsed._cursorConversationId) { - logCtx.conversationId = normalizeLogConversationId(parsed._cursorConversationId); - } - targetQueue.close(); - } - }; - const runTurn = async (): Promise => runTurnAttempt(queue, undefined, true); - const rotateRunTurnAdapterOnPreflight429 = async ( - error: Extract, - ): Promise => { - const status = error.status ?? adapterFailureFromMessage(error.message).httpStatus; - if ( - status !== 429 - || !genericFailoverAccountId - || genericFailovers >= GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST - || !isGenericOAuthFailoverEnabled(config, route.providerName) - ) return false; - // Intersection with the request's shared budget: the roster bound above answers "may this - // credential set rotate again", this answers "may this request send again at all". The - // replayed turn is dispatched by runTurnAttempt and never reaches `onSendsConsumed`, so - // this reservation is the charge. Refusing returns false, which leaves the preflight 429 - // to reach the client exactly as the adapter produced it. - const hop = reserveCredentialHop( - "auth-recovery", - `${route.providerName}|${route.modelId}|runturn-oauth-429`, - ); - if (!hop.allowed) return false; - const nextAccountId = rotateGenericOAuthAccountOn429( - config, - route.providerName, - genericFailoverAccountId, - null, - ); - if (!nextAccountId) { - hop.permit?.release(); - return false; - } - try { - const snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); - genericFailovers += 1; - if (!await applyFailoverSnapshot(snapshot)) { - hop.permit?.release(); - return false; - } - // A Cursor conversation/checkpoint is credential-scoped. The failed attempt emitted no - // client-visible bytes, so replay is safe, but carrying its account identity into the next - // account would not be. Let the rotated adapter derive a fresh identity and conversation. - parsed._cursorIdentityScope = undefined; - parsed._cursorConversationId = undefined; - if (parsed._providerContinuation?.cursor) { - const { cursor: _discardedCursor, ...otherProviderState } = parsed._providerContinuation; - parsed._providerContinuation = otherProviderState; - } - const rotatedProvider = resolveWireProtocolOverride( - route.providerName, - route.modelId, - route.provider, - inboundWire, - ); - const rotatedAdapter = resolveSelectionAdapter(rotatedProvider, config.cacheRetention); - if (!rotatedAdapter.runTurn) { - hop.permit?.release(); - return false; - } - runTurnAdapter = rotatedAdapter; - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: rotatedProvider, - adapterName: rotatedAdapter.name, - oauthCredentialSnapshot: { accountId: snapshot.accountId, generation: snapshot.generation }, - codexAuthContext: authCtx, - forwardHeaders: selectedForwardHeaders, - }); - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, rotatedAdapter.name, logCtx.accountLogLabel); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, rotatedAdapter.name); - // The caller replays the turn on this rotation, so the reservation is now confirmed. - hop.permit?.use(); - return true; - } catch { - hop.permit?.release(); - return false; - } - }; - const preflightRunTurnFailover = async ( - firstSource: AsyncIterable, - ): Promise> => { - let source = firstSource; - while (true) { - const preflight = await preflightAdapterEvents(source); - if (!preflight.error || !(await rotateRunTurnAdapterOnPreflight429(preflight.error))) { - return preflight.stream; - } - const retryQueue = createAdapterEventQueue({ - onBacklogExceeded: () => runTurnAbort.abort(), - }); - void runTurnAttempt(retryQueue, "oauth-account-429"); - source = retryQueue.stream(); - } - }; - // The empty-completion retry re-runs the turn against a fresh queue: the - // first queue is closed once its attempt settles, and pushing into it after - // close is a silent no-op. - const runTurnRetrySource = (): AsyncIterable => { - const retryQueue = createAdapterEventQueue({ - onBacklogExceeded: () => runTurnAbort.abort(), - }); - void runTurnAttempt(retryQueue, "empty-completion"); - return retryQueue.stream(); - }; - - const { toolNsMap, declaredToolNames, toolParameterSchemas, freeformToolNames, toolSearchToolNames } = toolBridgeMaps; - if (parsed.stream) { - void runTurn(); - let eventSource: AsyncIterable = queue.stream(); - if (route.provider.authMode === "oauth" || (genericFailoverAccountId && isGenericOAuthFailoverEnabled(config, route.providerName))) { - // Preflight holds only heartbeats and the first meaningful event. A first-event 429 can be - // replayed transparently; after any output reaches the bridge, a later error stays terminal. - eventSource = await preflightRunTurnFailover(eventSource); - } - if (options.comboAttempt) { - const preflight = await preflightAdapterEvents(eventSource); - if (preflight.error || preflight.empty) { - runTurnAbort.abort(); - queue.close(); - const message = preflight.error?.message ?? "Adapter ended before producing a response"; - return formatErrorResponse(502, "upstream_error", redactSecretString(message)); - } - eventSource = preflight.stream; - } - const guardedSource = emptyCompletionGuardEnabled - ? guardEmptyCompletionEventStream({ - firstEvents: eventSource, - // Identical-turn retry: same parsed request, same headers, same - // signal — run the adapter transport again against a fresh queue. - continuation: runTurnRetrySource, - }) - // Guard off (the default): leave the stream alone, but record that the turn ended - // empty so the user has something to correlate instead of an unexplained blank - // result (#2472). Retrying by default would re-send a turn that may already have had - // billable side effects, so the honest default is observability, not recovery. - : observeEmptyCompletion(eventSource, () => { - console.warn(emptyCompletionNotice(route.providerName, route.modelId)); - }); - const sseStream = bridgeToResponsesSSE( - guardedSource, parsed._responseModelId ?? parsed.modelId, toolNsMap, freeformToolNames, toolSearchToolNames, - () => { - cancelResponseCompletion(); - runTurnAbort.abort(); - queue.close(); - }, 2_000, - { - translatorBudget, - replayCacheScope: parsed._reasoningReplayScope, - ...(options.forceEmptyResponseId ? { responseId: "" } : {}), - stallTimeoutSec: config.stallTimeoutSec, - hideThinkingSummary: parsed.options.hideThinkingSummary, - declaredToolNames, - toolParameterSchemas, - ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), - ...(routedCompaction ? { compaction: true } : {}), - // grok-build's strict decoder dies on the typed response.heartbeat frame; its - // eventsource layer tolerates comment keep-alives. Codex needs the opposite. - ...(logCtx.surface === "grok" ? { heartbeatStyle: "comment" as const } : {}), - onUsage: usage => { - // Raw adapter usage, pre wire-normalization: the bridged SSE now always carries - // zero-default detail objects, so provenance must come from here (cache_detail_missing). - logCtx.usageFromBridge = true; - if (usage) { - logCtx.usage = usage; - if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; - } - }, - onCompletedResponse: (response: Record, providerState?: OcxProviderContinuationState) => { - commitReasoningReplayServingRoute(); - rememberKiroDeliveredFinalAnswer(adapter.name, response); - if (!routedCompaction) { - rememberResponseState( - parsed._rawBody, - response, - continuationStateForResponse(providerState), - responseStateOptions(adapterNeedsForcedContinuation(adapter.name)), - ); - } - notifyResponseComplete(response); - }, - }, - ); - const bridgeTurnAc = new AbortController(); - const trackedSse = trackStreamLifetime(sseStream, bridgeTurnAc, undefined, options.turnAdmissionLease); - const response = new Response(trackedSse, { - headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache", "Connection": "keep-alive", "X-Accel-Buffering": "no" }, - }); - runTurnAdapterSseResponses.add(response); - return response; - } - - await runTurn(); - const firstAttemptEvents = await queue.collect(); - let runTurnEvents: AdapterEvent[] = firstAttemptEvents; - if (route.provider.authMode === "oauth" || (genericFailoverAccountId && isGenericOAuthFailoverEnabled(config, route.providerName))) { - runTurnEvents = []; - for await (const event of await preflightRunTurnFailover( - (async function* () { yield* firstAttemptEvents; })(), - )) runTurnEvents.push(event); - } - let events: AdapterEvent[]; - if (emptyCompletionGuardEnabled) { - events = []; - for await (const event of guardEmptyCompletionEventStream({ - firstEvents: (async function* () { yield* runTurnEvents; })(), - continuation: runTurnRetrySource, - })) events.push(event); - } else { - events = runTurnEvents; - } - if (options.comboAttempt) { - const firstMeaningful = events.find(event => event.type !== "heartbeat"); - if (!firstMeaningful || firstMeaningful.type === "error") { - const message = firstMeaningful?.type === "error" - ? firstMeaningful.message - : "Adapter ended before producing a response"; - return formatErrorResponse(502, "upstream_error", redactSecretString(message)); - } - } - let providerState: OcxProviderContinuationState | undefined; - const json = buildResponseJSON(events, parsed._responseModelId ?? parsed.modelId, { - translatorBudget, - replayCacheScope: parsed._reasoningReplayScope, - hideThinkingSummary: parsed.options.hideThinkingSummary, - toolNsMap, - declaredToolNames, - toolParameterSchemas, - freeformToolNames, - toolSearchToolNames, - ...(routedCompaction ? { compaction: true } : {}), - onProviderState: state => { providerState = state; }, - onUsage: usage => { - logCtx.usageFromBridge = true; - if (usage) { - logCtx.usage = usage; - if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; - } - }, - }); - if (!routedCompaction) { - rememberKiroDeliveredFinalAnswer(adapter.name, json); - rememberResponseState( - parsed._rawBody, - json, - continuationStateForResponse(providerState), - responseStateOptions(adapterNeedsForcedContinuation(adapter.name)), - ); - } - // #1926 gap 2: the buffered path queued its signature persists inside - // buildResponseJSON; bound the durability window before the JSON becomes - // externally visible. - await awaitThoughtSignatureDurability(); - if (adapterResponseReachedServingTerminal(events, json)) { - commitReasoningReplayServingRoute(); - } - notifyResponseComplete(json); - return new Response(JSON.stringify(json), { headers: { "Content-Type": "application/json" } }); - } - - const upstream = new AbortController(); - const cleanupUpstreamAbort = linkAbortSignal(upstream, options.abortSignal); - const connectMs = config.connectTimeoutMs ?? 200_000; - // Bridge stall budget (seconds of silence before upstream_stall_timeout); the retry backoff - // heartbeat interval is derived from it so the watchdog is always fed during deliberate waits. - const stallTimeoutMs = typeof config.stallTimeoutSec === "number" && Number.isFinite(config.stallTimeoutSec) && config.stallTimeoutSec > 0 - ? Math.floor(config.stallTimeoutSec * 1000) - : 300_000; - activeAdapter = adapter; - - // One immutable, body-safe outbound request per same-target sequence (URL, serialized body, - // auth headers, generated compat headers). Same-target 429 replays reuse it verbatim; the - // builder runs again only after a key/account/adapter rotation, an oauth refresh, or an - // image-tier bias change (transportToken bump). `body` is always a serialized string, so - // reuse is safe, and releaseBodyObservation is idempotent per build. - let initialRequest: AdapterRequest | undefined; - let inputTokenEstimate: number | undefined; - // An adapter may know the turn needs no inference at all — Kiro's replayed history ending in a - // delivered final answer. Answer it locally: no build (so no token estimate), no send (so - // sendCount stays 0), and crucially no empty-completion guard, which treats an outputless - // terminal as a failed turn and re-invokes the identical request. Routing this through the - // ordinary event path would therefore reinstate the loop it exists to end. - const localTerminal = activeAdapter.localTerminal?.(parsed); - if (localTerminal) { - logCtx.localTerminalReason = localTerminal.reason; - // Mark the physical attempt too, not just the parent row. `finishRequestAttempt` finalizes the - // attempt through the same estimated-provider path, so without this the row reads exact while - // its own attempt still claims an estimate — the detailed accounting a maintainer actually - // reads for a zero-send turn. - if (logCtx.activeAttempt) logCtx.activeAttempt.locallyAnswered = true; - cleanupUpstreamAbort(); - upstream.abort(); - const terminalEvents: AdapterEvent[] = [{ - type: "done", - endTurn: true, - usage: { inputTokens: 0, outputTokens: 0, totalTokens: 0 }, - }]; - if (parsed.stream) { - const localSse = bridgeToResponsesSSE( - (async function* () { yield* terminalEvents; })(), - parsed._responseModelId ?? parsed.modelId, - toolBridgeMaps.toolNsMap, - toolBridgeMaps.freeformToolNames, - toolBridgeMaps.toolSearchToolNames, - cancelResponseCompletion, - 2_000, - { - translatorBudget, - onCompletedResponse: notifyResponseComplete, - ...(options.forceEmptyResponseId ? { responseId: "" } : {}), - ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), - }, - ); - // Same lifetime tracking as every other streaming return in this function: the turn - // admission lease is released when the body finishes or the client disconnects. Returning - // the raw stream would hold a lease for a turn that already has all of its output. - const localTurnAc = new AbortController(); - return new Response( - trackStreamLifetime(localSse, localTurnAc, undefined, options.turnAdmissionLease), - { - headers: { - "Content-Type": "text/event-stream", - "Cache-Control": "no-cache", - "Connection": "keep-alive", - "X-Accel-Buffering": "no", - }, - }, - ); - } - const json = buildResponseJSON(terminalEvents, parsed._responseModelId ?? parsed.modelId, { translatorBudget }); - notifyResponseComplete(json); - return new Response(JSON.stringify(json), { headers: { "Content-Type": "application/json" } }); - } - try { - initialRequest = await activeAdapter.buildRequest(parsed, { headers: selectedForwardHeaders, translatorBudget }); - refreshRequestToolAliases(initialRequest); - recordAdapterReasoning(logCtx, initialRequest); - recordAdapterTier(logCtx, initialRequest); - inputTokenEstimate = typeof initialRequest.usageLog?.inputTokens === "number" - ? initialRequest.usageLog.inputTokens - : undefined; - if (inputTokenEstimate !== undefined) logCtx.usageLogInputTokens = inputTokenEstimate; - } catch (err) { - // A throwing buildRequest never returned a request; if a post-build step threw, release - // the serialized-body observation (idempotent) so the translator budget is not leaked. - // The build runs after linkAbortSignal, so a failure must also tear the link down and - // abort the upstream controller instead of escaping handleResponses unmapped. - initialRequest?.releaseBodyObservation?.(); - cleanupUpstreamAbort(); - upstream.abort(); - if (options.abortSignal?.aborted) return clientCancelledResponse(); - const msg = err instanceof Error ? err.message : String(err); - return formatErrorResponse(400, "invalid_request_error", redactSecretString(msg)); - } - // The catch path above always returns, so the request is definitely assigned here. - // Capture it in a const so the fetch callbacks read a narrowed, immutable value - // (TypeScript drops narrowing for a `let` captured by a nested function). - const builtInitialRequest = initialRequest; - sameTargetRequest = builtInitialRequest; - sameTargetParsed = parsed; - sameTargetToken = transportToken; - /** - * Invalidate the same-target request cache. Every credential/adapter/parsed mutation MUST - * go through here: the cache keys on `parsed` REFERENCE identity, so an in-place mutation - * is invisible to it and a missed bump would replay a request built with a stale key. - */ - - let upstreamResponse: Response; - try { - if (activeAdapter.fetchResponse) { - noteAttemptSend(logCtx.activeAttempt, inputTokenEstimate); - await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, upstream.signal); - upstreamResponse = await activeAdapter.fetchResponse(builtInitialRequest, { - abortSignal: upstream.signal, - timeoutMs: connectMs, - sendBudget: adapterSendBudget, - onPhysicalSend: send => noteAdapterPhysicalSend(inputTokenEstimate, send), - stream: parsed.stream, - executor: providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(builtInitialRequest), - providerName: route.providerName, - modelId: route.modelId, - }), - }); - } else { - // #1851 scope guard: transient-5xx retry on this generic adapter path is opt-in for - // direct Google AI Studio only (Vertex/Antigravity use fetchResponse above). Other - // adapters keep reset-only retry so combo failover still hops on the first 5xx - // instead of burning ~1.2s of same-target retries per hop. - // #2643: an opted-in key-auth openai-chat provider also gets transient-5xx retry. The - // legacy direct-Google exception is preserved exactly; every other adapter still keeps - // reset-only semantics so combo failover hops on the first 5xx. - const transientPolicy = transientRetryPolicyFor(route.provider); - const fetchWithRetryPolicy = (route.provider.adapter === "google" || transientPolicy) - ? fetchWithTransientRetry - : fetchWithResetRetry; - upstreamResponse = await fetchWithRetryPolicy( - recovery => { - noteAttemptSend(logCtx.activeAttempt, inputTokenEstimate, recovery); - return fetchWithHeaderTimeout(builtInitialRequest.url, applyUpstreamRecoveryInit({ - method: builtInitialRequest.method, - headers: builtInitialRequest.headers, - body: builtInitialRequest.body, - }, recovery), upstream.signal, connectMs, parsed.stream, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(builtInitialRequest), - providerName: route.providerName, - modelId: route.modelId, - })); - }, - { - abortSignal: upstream.signal, - label: safeHostLabel(builtInitialRequest.url), - ...(transientPolicy - // Draws the remainder, not the raw policy. A combo child inherits the parent's - // holder but used to take a fresh full allowance on its own first send, so the - // shared counter was inherited without ever being read as a limit. - ? { - attempts: remainingTransientSendBudget(transientPolicy.attempts), - onSendsConsumed: noteTransientSends, - } - : {}), - }, - ); - } - } catch (err) { - cleanupUpstreamAbort(); - upstream.abort(); - if (options.abortSignal?.aborted) return clientCancelledResponse(); - const msg = describeUpstreamConnectFailure(err, connectMs); - return formatErrorResponse(502, "upstream_error", msg); - } finally { - builtInitialRequest.releaseBodyObservation?.(); - } - - // Same-target 429 retry budget is per REQUEST: it lives OUTSIDE the recovery loop (so a 413/401 - // replay that comes back 429 cannot silently re-arm a fresh budget) and is SHARED with the - // terminal-guard continuation below, so the main loop + one continuation can never exceed - // `attempts` same-key replays in total (bounded per request). - const rateLimitPolicy = rateLimitRetryPolicyFor(route.provider); - let rateLimitRetries = 0; - // Shared with the terminal-guard continuation below: an image-tier reduction that let the - // main request clear a 413 must not be forgotten on the very next continuation build. - if (!upstreamResponse.ok) { - // Recovery loop: multi-key 429 failover + at most ONE opaque-state rebuild and ONE - // anthropic 413 tightened retry - // (devlog/260714_image_normalization_pipeline/030). One mutable activeAdapter serves - // both paths so a 429→413 sequence never rebuilds against a stale pre-rotation - // adapter, and imageTierBias — once armed — rides EVERY subsequent rebuild so a - // 413→429 rotation cannot silently undo the tightening. - let imageRetryAttempted = false; - const opaqueBlobRecoveryGuard: OpaqueBlobRecoveryGuard = { attempted: false }; - // Console Go answers a transient 400 "Invalid upload request." for bodies it accepts - // moments later; at most one byte-identical replay is allowed per request. - const consoleGoUploadRetryGuard: { attempted: boolean } = { attempted: false }; - let oauth401ReplayAttempted = false; - // At most one reasoning-effort downgrade per request. This sits outside the recovery loop - // below for the same reason the two guards above do: a guard declared inside it is reset by - // every `continue recovery`, which would let one turn walk the whole ladder down. - const reasoningEffortDowngradeGuard: { attempted: boolean } = { attempted: false }; - /** - * Rebuild the request from the current parsed input (and any image-tier bias) and refetch - * it once, tagging the attempt with the given recovery kind. Rebuilds are deterministic - * for the same parsed request, so same-target replays stay byte-identical. - */ - const rebuildAndRefetch = async ( - recovery: AttemptRecoveryKind, - ): Promise => { - let retryRequest: AdapterRequest; - if (sameTargetRequest !== undefined && sameTargetParsed === parsed && sameTargetToken === transportToken) { - // Same target (key/adapter/parsed/tier unchanged): replay the exact cached request. - retryRequest = sameTargetRequest; - } else { - try { - retryRequest = await activeAdapter.buildRequest(parsed, { - headers: selectedForwardHeaders, - translatorBudget, - ...(imageTierBias > 0 ? { imageTierBias } : {}), - }); - recordAdapterReasoning(logCtx, retryRequest); - recordAdapterTier(logCtx, retryRequest); - } catch (err) { - // A rotated/rebuilt adapter build failure is a request-shaping error, not an - // upstream connect failure: tear the abort link down and map it as 400 (no 413 - // translator-budget mapping here — that stays with parseRequest/buildToolBridgeMaps). - cleanupUpstreamAbort(); - upstream.abort(); - if (options.abortSignal?.aborted) return { failed: clientCancelledResponse() }; - const msg = err instanceof Error ? err.message : String(err); - return { failed: formatErrorResponse(400, "invalid_request_error", redactSecretString(msg)) }; - } - sameTargetRequest = retryRequest; - sameTargetParsed = parsed; - sameTargetToken = transportToken; - } - refreshRequestToolAliases(retryRequest); - const retryEstimate = typeof retryRequest.usageLog?.inputTokens === "number" - ? retryRequest.usageLog.inputTokens - : undefined; - if (retryEstimate !== undefined) logCtx.usageLogInputTokens = retryEstimate; - logCtx.providerAdapter = activeAdapter.name; - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, activeAdapter.name, logCtx.accountLogLabel); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, activeAdapter.name); - noteAttemptSend(logCtx.activeAttempt, retryEstimate, recovery); - try { - try { - if (activeAdapter.fetchResponse) { - await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, upstream.signal); - return await activeAdapter.fetchResponse(retryRequest, { - abortSignal: upstream.signal, - timeoutMs: connectMs, - sendBudget: adapterSendBudget, - onPhysicalSend: send => noteAdapterPhysicalSend(retryEstimate, send), - stream: parsed.stream, - executor: providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(retryRequest), - providerName: route.providerName, - modelId: route.modelId, - }), - }); - } - // #2643 review: this leg used to call fetchWithHeaderTimeout directly, so an - // opted-in provider's transient-5xx policy applied to the initial send and to - // native chat but was silently bypassed here — a 429 that recovered into a - // retryable 503 got no retry on the Responses path. Route it through the same - // selection, and pass what is LEFT of the request-scoped budget rather than a - // fresh one, so a recovery loop cannot multiply total upstream sends. - const refetchTransientPolicy = transientRetryPolicyFor(route.provider); - const refetchWithPolicy = (route.provider.adapter === "google" || refetchTransientPolicy) - ? fetchWithTransientRetry - : fetchWithResetRetry; - // Same rule as the passthrough rebuild: spend the base allowance first, then the one - // shared final-recovery reserve, so a recovery that follows a spent streak still gets - // its single send instead of dying at three. - const refetchAllowance = refetchTransientPolicy - ? recoverySendAllowance( - refetchTransientPolicy.attempts, - recoveryClassFor(recovery), - `${route.providerName}|${route.modelId}|${recovery}`, - ) - : undefined; - try { - return await refetchWithPolicy( - recoveryKind => { - if (refetchAllowance?.permit && !refetchAllowance.permit.use()) { - throw new SendBudgetExhaustedError(safeHostLabel(retryRequest.url)); - } - return fetchWithHeaderTimeout(retryRequest.url, - applyUpstreamRecoveryInit({ - method: retryRequest.method, headers: retryRequest.headers, body: retryRequest.body, - }, recoveryKind), upstream.signal, connectMs, parsed.stream, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(retryRequest), - providerName: route.providerName, - modelId: route.modelId, - })); - }, - { - abortSignal: upstream.signal, - label: safeHostLabel(retryRequest.url), - ...(refetchAllowance - ? { - attempts: refetchAllowance.attempts, - onSendsConsumed: noteTransientSends, - } - : {}), - }, - ); - } finally { - // Refunds only a reservation whose send never happened -- an abort settled before - // the thunk ran. A used or externally settled permit ignores this. - refetchAllowance?.permit?.release(); - } - } finally { - retryRequest.releaseBodyObservation?.(); - } - } catch (err) { - cleanupUpstreamAbort(); - upstream.abort(); - if (options.abortSignal?.aborted) { - return { failed: clientCancelledResponse() }; - } - const msg = describeUpstreamConnectFailure(err, connectMs); - return { failed: formatErrorResponse(502, "upstream_error", msg) }; - } - }; - // Keep recovery kinds in sync with the native Responses `passthroughRecovery:` loop above. - recovery: for (;;) { - if ( - upstreamResponse.status === 401 - && isOAuth401ReplayProvider - && sentOAuthSnapshot - && !oauth401ReplayAttempted - && !sendBudgetExhausted() - ) { - oauth401ReplayAttempted = true; - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - let refreshed: OAuthAccessSnapshot; - try { - refreshed = await refreshResolvedOAuthSelection(sentOAuthSnapshot); - } catch (err) { - cleanupUpstreamAbort(); - return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(err)); - } - if (route.provider.googleMode === "cloud-code-assist" && !refreshed.projectId) { - cleanupUpstreamAbort(); - return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(new Error("Cloud Code Assist project is required"))); - } - sentOAuthSnapshot = refreshed; - replayOAuthCredentialSnapshot = { - accountId: refreshed.accountId, - generation: refreshed.generation, - }; - if (route.providerName === "kiro") { - parsed._kiroAuthContext = { ...(refreshed.kiro ?? {}) }; - } - const refreshedProvider = resolveProviderTransport( - route.providerName, - { - ...route.provider, - apiKey: refreshed.accessToken, - ...(refreshed.projectId ? { project: refreshed.projectId } : {}), - }, - parsed.options.promptCacheKey, - route.providerName === "github-copilot" - ? resolveCopilotApiBaseUrl(refreshed.apiBaseUrl) - : undefined, - ); - route.provider = refreshedProvider; - invalidateSameTargetRequest(); - activeAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, refreshedProvider, inboundWire), - config.cacheRetention, - ); - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: refreshedProvider, - adapterName: activeAdapter.name, - oauthCredentialSnapshot: replayOAuthCredentialSnapshot, - }); - const result = await rebuildAndRefetch("oauth-401"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - continue recovery; - } - - // Static API-key pools can recover a credential-scoped 401 without abandoning the - // provider: one revoked or mistyped key says nothing about its siblings. OAuth providers - // refresh above and never enter here — `hasKeyPoolFailover` rejects oauth/forward modes. - // Runs after the OAuth replay so a refreshable token is never treated as a dead key. - while (upstreamResponse.status === 401 && hasKeyPoolFailover(route.provider)) { - const rotated = rotateProviderTransportOn401(config, route.providerName, route.provider, { - now: Date.now(), - attemptedKey: route.provider.apiKey, - promptCacheKey: parsed.options.promptCacheKey, - }); - if (!rotated) break; - // Release the failed response's socket before retrying; unread bodies otherwise linger - // until runtime cleanup (one per rotated key). - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - route.provider = rotated; - invalidateSameTargetRequest(); - activeAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: route.provider, - adapterName: activeAdapter.name, - }); - const result = await rebuildAndRefetch("key-401"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - } - - // Same-target 429 wait-and-retry (opt-in `retryOn429`, issue #487). Codex never retries - // 429 itself (it retries 5xx only), and single-key pools cannot use the failover below, - // so wait (Retry-After or the fixed interval) and replay the IDENTICAL request on the - // same key first. Pre-stream only: a 429 arrives before any bytes are relayed, so the - // replay is lossless. Runs before key failover so "primary-first" setups keep the same - // key on rate-limit blips; only after the attempts are exhausted does failover run. - while ( - upstreamResponse.status === 429 - && rateLimitPolicy !== null - && rateLimitRetries < rateLimitPolicy.attempts - && !sendBudgetExhausted() - ) { - rateLimitRetries += 1; - // Release unread body + deliberate wait via the shared same-target helper. - const retryAfterHeader = upstreamResponse.headers.get("retry-after"); - try { - for await (const _ of prepareSameTarget429Wait({ - body: upstreamResponse.body, - signal: options.abortSignal, - delayMs: rateLimitRetryDelayMs(rateLimitPolicy, retryAfterHeader, Date.now()), - })) { - // pre-stream: no stall watchdog to feed - } - } catch { - cleanupUpstreamAbort(); - upstream.abort(); - return clientCancelledResponse(); - } - // Client cancellation wins over any stale timer edge: re-check before dispatching the - // replay so an adapter never starts work for a request the client already abandoned. - if (options.abortSignal?.aborted || upstream.signal.aborted) { - cleanupUpstreamAbort(); - upstream.abort(); - return clientCancelledResponse(); - } - const result = await rebuildAndRefetch("rate-limit-429"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - } - - // Multi-key 429 failover: rotate to the next pool key (cooldown-aware) and retry the - // SAME request once per remaining key. OAuth/forward providers and single-key pools - // return null immediately, so this stays a no-op for them (src/providers/key-failover.ts). - while (upstreamResponse.status === 429 && hasKeyPoolFailover(route.provider)) { - const rotated = rotateProviderTransportOn429(config, route.providerName, route.provider, { - retryAfter: upstreamResponse.headers.get("retry-after"), - now: Date.now(), - attemptedKey: route.provider.apiKey, - promptCacheKey: parsed.options.promptCacheKey, - }); - if (!rotated) break; - // Release the failed response's socket before retrying; unread bodies otherwise linger - // until runtime cleanup (one per rotated key under a rate-limit storm). - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - route.provider = rotated; - invalidateSameTargetRequest(); - activeAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: route.provider, - adapterName: activeAdapter.name, - }); - const result = await rebuildAndRefetch("key-429"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - } - - // Opt-in Anthropic OAuth account pool (#294): cool the failed account and retry - // with another eligible OAuth account (bounded per request). Disabled by default. - while ( - upstreamResponse.status === 429 - && anthropicPoolAccountId - && anthropicPoolFailovers < ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST - ) { - const nextAccountId = rotateAnthropicAccountOn429( - config, - anthropicPoolAccountId, - upstreamResponse.headers.get("retry-after"), - anthropicSessionKey, - Date.now(), - upstreamResponse.headers, - ); - if (!nextAccountId) break; - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - try { - const admitted = await commitResolvedOAuthSelection(await getAnthropicPoolAccessSnapshot(nextAccountId)); - if (!admitted) throw new Error("OAuth selection changed during recovery"); - anthropicPoolAccountId = admitted.accountId; - anthropicPoolFailovers += 1; - route.provider = { ...route.provider, apiKey: admitted.accessToken }; - invalidateSameTargetRequest(); - logCtx.provider = formatAnthropicProviderForLog("anthropic", admitted.accountId, config); - activeAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, activeAdapter.name, logCtx.accountLogLabel); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, activeAdapter.name); - const result = await rebuildAndRefetch("anthropic-oauth-429"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - } catch { - break; - } - } - // Generic OAuth account failover (#2568) for providers with no pool of their own. - // Presence is consent since #2568d: rotation is ON by default once two or more eligible - // accounts are stored for the provider, because a second deliberate login is read as the - // operator asking for it. A single-account install is still a strict no-op, and an - // explicit `oauthAccountFailover.enabled: false` (global or per provider) still wins -- - // see isGenericOAuthFailoverEnabled in src/oauth/generic-account-failover.ts. Codex and - // Anthropic are excluded by isGenericFailoverProvider: their pools own quota scopes, - // probe leases and affinity that this must not reimplement. - while ( - upstreamResponse.status === 429 - && genericFailoverAccountId - && genericFailovers < GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST - && isGenericOAuthFailoverEnabled(config, route.providerName) - ) { - // Intersection with the shared request budget. This arm re-sends through - // rebuildAndRefetch, so the roster cap alone would let one request walk the roster on - // an allowance the rest of the request cannot see. A refusal ends the ladder with the - // real 429 already in hand, which is the decided exhaustion contract. - const hop = reserveCredentialHop( - "auth-recovery", - `${route.providerName}|${route.modelId}|adapter-recovery-oauth-429`, - ); - if (!hop.allowed) break; - const nextAccountId = rotateGenericOAuthAccountOn429( - config, - route.providerName, - genericFailoverAccountId, - upstreamResponse.headers.get("retry-after"), - ); - if (!nextAccountId) { - hop.permit?.release(); - break; - } - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - try { - // The FULL snapshot, not just the bearer: Antigravity pairs an account-matched - // projectId with its token and Kiro carries routing metadata, so a token-only swap - // would mix one account's credential with another's routing data. - const snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); - genericFailovers += 1; - if (!await applyFailoverSnapshot(snapshot)) { - hop.permit?.release(); - break; - } - invalidateSameTargetRequest(); - activeAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, activeAdapter.name, logCtx.accountLogLabel); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, activeAdapter.name); - const result = await rebuildAndRefetch("oauth-account-429"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - } catch { - break; - } - } - // Unknown provenance is deliberately fail-soft in pre-flight: after a restart, TTL expiry, - // or LRU eviction, a valid same-backend blob must survive. A decoder's own 4xx identity is - // the missing authoritative signal. Rebuild once through the same sanitation path used by a - // known route switch; invalidating is mandatory because `parsed` mutates in place and the - // same-target cache would otherwise replay the rejected bytes verbatim. - const opaqueBlobRecovery = await attemptOpaqueBlobRecovery({ - response: upstreamResponse, - outboundBody: sameTargetRequest?.body, - adapterName: activeAdapter.name, - parsed, - guard: opaqueBlobRecoveryGuard, - signal: upstream.signal, - }, recovery => { - invalidateSameTargetRequest(); - return rebuildAndRefetch(recovery); - }); - if (opaqueBlobRecovery.kind === "failed") return opaqueBlobRecovery.response; - if (opaqueBlobRecovery.kind === "recovered") { - upstreamResponse = opaqueBlobRecovery.response; - continue recovery; - } - // Anthropic 413 request_too_large: rebuild once with every image one tier lower - // (spiral guard: single attempt). The biased response re-enters the 429 check above. - if (shouldAttemptImageTierRetry({ - status: upstreamResponse.status, - adapterName: activeAdapter.name, - parsed, - alreadyAttempted: imageRetryAttempted, - })) { - imageRetryAttempted = true; - imageTierBias = 1; - invalidateSameTargetRequest(); - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - const result = await rebuildAndRefetch("image-413"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - continue recovery; - } - // Console Go (opencode-zen / opencode-go) intermittently rejects a body it accepts seconds - // later with 400 invalid_request_error / "Invalid upload request." Replay the - // byte-identical request once after the exact gateway rejection. - if (!consoleGoUploadRetryGuard.attempted) { - const uploadRejectionBody = await consoleGoUploadRejectionBody( - upstreamResponse, - consoleGoUploadRetryGuard.attempted, - upstream.signal, - ); - if (uploadRejectionBody !== undefined - && isTransientConsoleGoUploadRejection({ - status: upstreamResponse.status, - errorBody: uploadRejectionBody, - outboundUrl: sameTargetRequest?.url, - })) { - consoleGoUploadRetryGuard.attempted = true; - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - if (!upstream.signal.aborted) { - try { - await sleepWithAbort(CONSOLE_GO_UPLOAD_RETRY_DELAY_MS, upstream.signal); - } catch { cleanupUpstreamAbort(); return clientCancelledResponse(); } - } - if (upstream.signal.aborted) { cleanupUpstreamAbort(); return clientCancelledResponse(); } - const result = await rebuildAndRefetch("console-go-upload-retry"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - continue recovery; - } - } - // Reasoning-effort downgrade, mirroring the passthroughRecovery loop above: learn the - // refused rung, then replay once at the next published one. - if (!reasoningEffortDowngradeGuard.attempted) { - const rejectionText = await reasoningEffortRejectionText( - upstreamResponse, - reasoningEffortDowngradeGuard.attempted, - upstream.signal, - ); - const downgrade = rejectionText === undefined - ? undefined - : planReasoningEffortDowngrade({ - provider: route.provider, - modelId: parsed.modelId, - requested: parsed.options.reasoning, - rejectionText, - }); - if (downgrade) { - reasoningEffortDowngradeGuard.attempted = true; - parsed.options.reasoning = downgrade.effort; - // The same-target cache keys on parsed identity, so a mutated effort needs a token bump. - invalidateSameTargetRequest(); - try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } - const result = await rebuildAndRefetch("reasoning-effort-downgrade"); - if ("failed" in result) return result.failed; - upstreamResponse = result; - continue recovery; - } - } - break; - } - if (!upstreamResponse.ok) { - if (options.comboAttempt) { - // No pre-read guard: `consumeComboFailure` -> `readBoundedResponseBody` reads - // `response.body` itself with the abort signal threaded through, and the combo - // contract is that this body's getter is touched exactly once. A guard here would be - // a second `.body` access for no gain, since the bounded reader owns settlement. - const failure = await consumeComboFailure(upstreamResponse, options.abortSignal) - .finally(cleanupUpstreamAbort); - options.onConsumedComboFailure?.(failure); - return failure.response; - } - let errorText: string; - try { - errorText = await readDisplaySafeErrorText( - upstreamResponse, - upstream.signal, - "unknown error", - ); - } finally { - cleanupUpstreamAbort(); - } - if (upstreamResponse.status === 413) { - return clientRequestedStream - ? streamingContextOverflowResponse(parsed._responseModelId ?? parsed.modelId, translatorBudget) - : jsonContextOverflowResponse(); - } - if (!isFixedCodexAccount(authCtx)) { - recordSubagentQuotaFailureForThreadSpawn( - req.headers, - subagentQuotaFailureModel, - upstreamResponse.status === 429 || upstreamResponse.status === 402 - ? upstreamResponse.status - : `Provider error ${upstreamResponse.status}: ${redactSecretString(errorText.slice(0, 500))}`, - config, - subagentFallbackAccountId, - ); - } - // Upstreams occasionally echo request details in error bodies — scrub token-shaped - // material before it reaches the client-facing error surface. - const upstreamRetryAfter = upstreamResponse.headers.get("retry-after"); - const normalized = normalizeUpstreamErrorText(errorText, "unknown error"); - const message = normalized.cyberPolicy - ? normalized.message - ?? (isCyberPolicyCode(normalized.code) ? CYBER_POLICY_FALLBACK_MESSAGE : normalized.safeText) - : enrichOpenCodeZenUpstreamMessage( - `Provider error ${upstreamResponse.status}: ${normalized.safeText}`, - { - status: upstreamResponse.status, - providerName: route.providerName, - baseUrl: route.provider.baseUrl, - adapter: route.provider.adapter, - authMode: route.provider.authMode, - hasApiKey: Boolean(route.provider.apiKey?.trim()), - upstreamRetryAfter, - // This recovery path is the HTTP Responses wire; custom runTurn transports - // never reach enrichOpenCodeZenUpstreamMessage here. - supportsHttpSameKeyRetry: true, - }, - ); - const retryAfter = normalized.cyberPolicy - ? undefined - : resolveClientRetryAfter({ - status: upstreamResponse.status, - message, - upstreamRetryAfter, - }); - return formatErrorResponse( - upstreamResponse.status, - normalized.cyberPolicy ? (normalized.type ?? CYBER_POLICY_ERROR_CODE) : "upstream_error", - message, - { - ...(normalized.cyberPolicy ? { code: CYBER_POLICY_ERROR_CODE } : {}), - ...(retryAfter !== undefined ? { retryAfter } : {}), - }, - ); - } - } - - cancelBodyOnAbort(upstreamResponse.body, upstream.signal); - - // One bounded internal continuation re-ask for clean end_turn turns that announced an edit - // without emitting a tool call. Anthropic gets this by default; openai-chat providers opt in - // per-provider via `terminalContinuationGuard` (the heuristic was tuned on Anthropic turns, - // so it stays off for the shared openai-chat adapter unless a provider enables it). - const terminalGuardEnabled = (activeAdapter.name === "anthropic" - || (activeAdapter.name === "openai-chat" && route.provider.terminalContinuationGuard === true)) - && !options.comboAttempt && !routedCompaction; - /** - * One bounded internal re-ask for Anthropic end_turn-without-tool-call turns. Replays the - * continuation on a 429 with the same-key retry budget (hoisted per request), then falls - * back to key/account failover; a failure becomes an in-stream adapter error so the client - * never sees a second hidden HTTP response or an unbounded retry loop. - */ - const fetchTerminalGuardContinuation = async function* ( - nextParsed: OcxParsedRequest, - initialRecoveryKind?: AttemptRecoveryKind, - ): AsyncGenerator { - let response: Response | undefined; - // One-shot recovery label for the next top-of-loop continuation send after a failover rotation. - let nextContinuationRecoveryKind: AttemptRecoveryKind | undefined = initialRecoveryKind; - /** - * Build and fetch one terminal-guard continuation. `recoveryKind` tags same-target and - * failover sends (`empty-completion`, `rate-limit-429`, `key-429`, - * `anthropic-oauth-429`, `image-413`); the - * adapter rebuild is deterministic for the same parsed request (tests assert byte-identical - * replays). - */ - const fetchContinuation = async (recoveryKind?: AttemptRecoveryKind): Promise => { - let continuationRequest: AdapterRequest | undefined; - if (sameTargetRequest !== undefined && sameTargetParsed === nextParsed && sameTargetToken === transportToken) { - // Same target (key/adapter/parsed/tier unchanged): replay the exact cached request. - continuationRequest = sameTargetRequest; - } else { - try { - continuationRequest = await activeAdapter.buildRequest(nextParsed, { - headers: selectedForwardHeaders, - translatorBudget, - ...(imageTierBias > 0 ? { imageTierBias } : {}), - }); - recordAdapterReasoning(logCtx, continuationRequest); - recordAdapterTier(logCtx, continuationRequest); - } catch (err) { - // The main body is already streaming, so there is no HTTP error surface: release - // any partial body observation and surface the failure as an in-stream error via - // the outer catch (no upstream.abort() — that would kill the live body stream). - continuationRequest?.releaseBodyObservation?.(); - throw err; - } - sameTargetRequest = continuationRequest; - sameTargetParsed = nextParsed; - sameTargetToken = transportToken; - } - // Both branches assign the request (the build catch rethrows), so capture it in a - // const for the fetch callback and finally below — a `let` read inside a nested - // function keeps its undefined half, which would break the byte-identical replay. - const builtContinuationRequest = continuationRequest; - const continuationEstimate = typeof builtContinuationRequest.usageLog?.inputTokens === "number" - ? builtContinuationRequest.usageLog.inputTokens - : undefined; - if (continuationEstimate !== undefined) logCtx.usageLogInputTokens = continuationEstimate; - // Optional recovery label for same-target / failover continuation sends. - const replayKind: AttemptRecoveryKind | undefined = recoveryKind; - try { - if (activeAdapter.fetchResponse) { - noteAttemptSend(logCtx.activeAttempt, continuationEstimate, replayKind); - await waitForProviderRequestSlot(route.providerName, route.provider, nextParsed.modelId, upstream.signal); - return await activeAdapter.fetchResponse(builtContinuationRequest, { - abortSignal: upstream.signal, - timeoutMs: connectMs, - sendBudget: adapterSendBudget, - onPhysicalSend: send => noteAdapterPhysicalSend(continuationEstimate, send), - stream: nextParsed.stream, - executor: providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(builtContinuationRequest, nextParsed), - providerName: route.providerName, - modelId: nextParsed.modelId, - }), - }); - } - // Same #1851 scope guard as the initial send: transient-5xx retry only for direct - // Google AI Studio; every other adapter keeps reset-only semantics here. - const continuationTransientPolicy = transientRetryPolicyFor(route.provider); - const fetchContinuationWithRetryPolicy = (route.provider.adapter === "google" || continuationTransientPolicy) - ? fetchWithTransientRetry - : fetchWithResetRetry; - return await fetchContinuationWithRetryPolicy( - recovery => { - noteAttemptSend(logCtx.activeAttempt, continuationEstimate, recovery ?? replayKind); - return fetchWithHeaderTimeout( - builtContinuationRequest.url, - applyUpstreamRecoveryInit({ - method: builtContinuationRequest.method, - headers: builtContinuationRequest.headers, - body: builtContinuationRequest.body, - }, recovery), - upstream.signal, - connectMs, - nextParsed.stream, - providerFetch(route.provider, options.codexWsRuntimeIdentity, { - dispatchOverride: oauthDispatch(builtContinuationRequest, nextParsed), - providerName: route.providerName, - modelId: nextParsed.modelId, - }), - ); - }, - { - abortSignal: upstream.signal, - label: safeHostLabel(builtContinuationRequest.url), - // Same request-scoped budget as the initial send and the 429/rotation refetches: - // a terminal-guard continuation is another leg of ONE request, so handing it a - // fresh `attempts` would let one request exceed the configured total-send ceiling. - ...(continuationTransientPolicy - ? { - attempts: remainingTransientSendBudget(continuationTransientPolicy.attempts), - onSendsConsumed: noteTransientSends, - } - : {}), - }, - ); - } finally { - builtContinuationRequest.releaseBodyObservation?.(); - } - }; - while (true) { - try { - const recoveryKind = nextContinuationRecoveryKind; - nextContinuationRecoveryKind = undefined; - response = await fetchContinuation(recoveryKind); - } catch (error) { - if (options.abortSignal?.aborted || upstream.signal.aborted) { - yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; - } else { - yield { type: "error", message: `Provider continuation failed: ${redactSecretString(error instanceof Error ? error.message : String(error))}` }; - } - return; - } - - // Same-target 429 wait-and-retry (opt-in `retryOn429`) before key/account failover: - // a primary-key rate-limit blip replays on the SAME key, matching the main recovery - // loop; only after the attempts are exhausted does the continuation fail over. - while ( - response.status === 429 - && rateLimitPolicy !== null - && rateLimitRetries < rateLimitPolicy.attempts - ) { - rateLimitRetries += 1; - // Release unread body + heartbeat-fed wait via the shared same-target helper. - const retryAfterHeader = response.headers.get("retry-after"); - try { - yield* prepareSameTarget429Wait({ - body: response.body, - // Listen on the upstream signal: once the SSE body is being streamed, a client - // cancel aborts `upstream` through the bridge, and upstream is also linked from - // options.abortSignal — so this covers both cancellation paths. - signal: upstream.signal, - delayMs: rateLimitRetryDelayMs(rateLimitPolicy, retryAfterHeader, Date.now()), - heartbeatIntervalMs: Math.min(10_000, Math.max(250, stallTimeoutMs / 2)), - }); - } catch { - if (options.abortSignal?.aborted || upstream.signal.aborted) { - yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; - } else { - yield { type: "error", message: "Provider continuation failed: retry wait interrupted" }; - } - return; - } - // Client cancellation wins over any stale timer edge: re-check before dispatching the - // replay so the continuation never starts work for a request the client abandoned. - if (options.abortSignal?.aborted || upstream.signal.aborted) { - yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; - return; - } - try { - response = await fetchContinuation("rate-limit-429"); - } catch (error) { - if (options.abortSignal?.aborted || upstream.signal.aborted) { - yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; - } else { - yield { type: "error", message: `Provider continuation failed: ${redactSecretString(error instanceof Error ? error.message : String(error))}` }; - } - return; - } - } - - if (response.status === 429 && hasKeyPoolFailover(route.provider)) { - const rotated = rotateProviderTransportOn429(config, route.providerName, route.provider, { - retryAfter: response.headers.get("retry-after"), - now: Date.now(), - attemptedKey: route.provider.apiKey, - promptCacheKey: nextParsed.options.promptCacheKey, - }); - if (rotated) { - try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } - route.provider = rotated; - invalidateSameTargetRequest(); - activeAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - bindRouteReasoningReplayScope({ - parsed: nextParsed, - providerName: route.providerName, - provider: route.provider, - adapterName: activeAdapter.name, - }); - // Response persistence closes over the outer parsed request; keep its owner binding in - // sync with the terminal-guard clone that builds the rotated continuation request. - bindRouteReasoningReplayScope({ - parsed, - providerName: route.providerName, - provider: route.provider, - adapterName: activeAdapter.name, - }); - nextContinuationRecoveryKind = "key-429"; - continue; - } - } - if ( - response.status === 429 - && anthropicPoolAccountId - && anthropicPoolFailovers < ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST - ) { - const nextAccountId = rotateAnthropicAccountOn429( - config, - anthropicPoolAccountId, - response.headers.get("retry-after"), - anthropicSessionKey, - Date.now(), - response.headers, - ); - if (nextAccountId) { - try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } - try { - const admitted = await commitResolvedOAuthSelection(await getAnthropicPoolAccessSnapshot(nextAccountId)); - if (!admitted) throw new Error("OAuth selection changed during recovery"); - anthropicPoolAccountId = admitted.accountId; - anthropicPoolFailovers += 1; - route.provider = { ...route.provider, apiKey: admitted.accessToken }; - invalidateSameTargetRequest(); - logCtx.provider = formatAnthropicProviderForLog("anthropic", admitted.accountId, config); - activeAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, activeAdapter.name, logCtx.accountLogLabel); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, activeAdapter.name); - nextContinuationRecoveryKind = "anthropic-oauth-429"; - continue; - } catch { - // fall through to emit continuation error below - } - } - } - // Generic OAuth rotation for the continuation loop. The streaming loop grew this arm with - // #2568 and this one did not, so an xAI/Cursor/Kimi/Copilot/Antigravity/Nous continuation - // 429 stayed terminal even with failover fully active -- the same class of divergence the - // two sidecars already produced once. Request-local state is shared with the other arms so - // the per-request bound cannot be silently re-armed by reaching a different loop. - if ( - response.status === 429 - && genericFailoverAccountId - && genericFailovers < GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST - && isGenericOAuthFailoverEnabled(config, route.providerName) - ) { - // Intersection with the shared request budget. The continuation loop re-sends the - // turn, so without this the per-request bound could be re-armed simply by reaching a - // different loop -- which is the divergence the comment above already warns about. - const hop = reserveCredentialHop( - "auth-recovery", - `${route.providerName}|${route.modelId}|continuation-oauth-429`, - ); - const nextAccountId = hop.allowed - ? rotateGenericOAuthAccountOn429( - config, - route.providerName, - genericFailoverAccountId, - response.headers.get("retry-after"), - ) - : null; - if (!nextAccountId) hop.permit?.release(); - if (nextAccountId) { - try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } - try { - // The FULL snapshot through the shared helper, never a bare bearer: Antigravity - // pairs an account-matched projectId with its token and Kiro carries routing - // metadata, so a token-only swap would mix one account's credential with another's - // routing data. - const snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); - genericFailovers += 1; - const applied = await applyFailoverSnapshot(snapshot, nextParsed); - if (!applied) hop.permit?.release(); - if (applied) { - invalidateSameTargetRequest(); - activeAdapter = resolveSelectionAdapter( - resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), - config.cacheRetention, - ); - sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, activeAdapter.name, logCtx.accountLogLabel); - recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, activeAdapter.name); - nextContinuationRecoveryKind = "oauth-account-429"; - continue; - } - } catch { - // fall through to emit continuation error below - } - } - } - if (shouldAttemptImageTierRetry({ - status: response.status, - adapterName: activeAdapter.name, - parsed: nextParsed, - alreadyAttempted: imageTierBias > 0, - })) { - imageTierBias = 1; - invalidateSameTargetRequest(); - try { void response.body?.cancel().catch(() => {}); } catch { /* already closed */ } - nextContinuationRecoveryKind = "image-413"; - continue; - } - break; - } - - if (!response.ok) { - const errorText = await readDisplaySafeErrorText(response, upstream.signal, "unknown error"); - const normalized = normalizeUpstreamErrorText(errorText, "unknown error"); - yield { - type: "error", - status: normalized.cyberPolicy ? 400 : response.status, - message: normalized.cyberPolicy - ? normalized.message - ?? (isCyberPolicyCode(normalized.code) ? CYBER_POLICY_FALLBACK_MESSAGE : normalized.safeText) - : `Provider continuation error ${response.status}: ${normalized.safeText}`, - ...(normalized.cyberPolicy - ? { - errorType: normalized.type ?? CYBER_POLICY_ERROR_CODE, - code: CYBER_POLICY_ERROR_CODE, - retryable: false, - } - : {}), - }; - return; - } - - try { - // Protect the continuation body against a client abort landing between fetch resolution and - // reader attach, exactly as the initial response is guarded above (#390/366e3053). Without - // this, a client cancel during the continuation reopens the Bun fetch-to-reader abort race. - const detachContinuationBodyGuard = cancelBodyOnAbort(response.body, upstream.signal); - try { - if (nextParsed.stream) { - yield* activeAdapter.parseStream(response, translatorBudget, logCtx.activeTierMetadata); - } else if (activeAdapter.parseResponse) { - yield* await activeAdapter.parseResponse(response, translatorBudget, logCtx.activeTierMetadata); - } else { - yield { type: "error", message: "Provider continuation does not support response parsing" }; - } - } finally { - detachContinuationBodyGuard(); - } - } catch (error) { - if (options.abortSignal?.aborted) { - yield { type: "error", message: "client closed request during terminal continuation", status: 499 }; - } else { - yield { type: "error", message: `Provider continuation parse failed: ${redactSecretString(error instanceof Error ? error.message : String(error))}` }; - } - } - }; - - const fetchGuardedEmptyCompletionRetry = (): AsyncIterable => { - const retryEvents = fetchTerminalGuardContinuation(parsed, "empty-completion"); - return terminalGuardEnabled - ? guardTerminalEventStream({ - parsed, - firstEvents: retryEvents, - adapterName: activeAdapter.name, - maxAutoContinuations: 1, - continuation: fetchTerminalGuardContinuation, - }) - : retryEvents; - }; - - if (parsed.stream) { - const initialEventStream = activeAdapter.parseStream( - upstreamResponse, - translatorBudget, - logCtx.activeTierMetadata, - ); - const eventStream = terminalGuardEnabled - ? guardTerminalEventStream({ - parsed, - firstEvents: initialEventStream, - adapterName: activeAdapter.name, - maxAutoContinuations: 1, - continuation: fetchTerminalGuardContinuation, - }) - : initialEventStream; - // The empty-completion guard sits OUTSIDE the terminal guard: a completed - // turn with no text and no tool call is retried with the IDENTICAL request - // (fetchTerminalGuardContinuation(parsed) replays the cached byte-identical - // request — same body, same headers, same signal). - const guardedEventStream = emptyCompletionGuardEnabled - ? guardEmptyCompletionEventStream({ - firstEvents: eventStream, - continuation: fetchGuardedEmptyCompletionRetry, - }) - : eventStream; - const { toolNsMap, declaredToolNames, toolParameterSchemas, freeformToolNames, toolSearchToolNames } = toolBridgeMaps; - const sseStream = bridgeToResponsesSSE( - guardedEventStream, parsed._responseModelId ?? parsed.modelId, toolNsMap, freeformToolNames, toolSearchToolNames, - () => { cancelResponseCompletion(); upstream.abort(); }, 2_000, - { - translatorBudget, - replayCacheScope: parsed._reasoningReplayScope, - ...(options.forceEmptyResponseId ? { responseId: "" } : {}), - stallTimeoutSec: config.stallTimeoutSec, - hideThinkingSummary: parsed.options.hideThinkingSummary, - declaredToolNames, - toolParameterSchemas, - ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), - ...(routedCompaction ? { compaction: true } : {}), - // Same grok-surface split as the runTurn branch above. - ...(logCtx.surface === "grok" ? { heartbeatStyle: "comment" as const } : {}), - onUsage: usage => { - // Raw adapter usage, pre wire-normalization (see the runTurn branch above). - logCtx.usageFromBridge = true; - if (usage) { - logCtx.usage = usage; - if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; - } - }, - onCompletedResponse: (response: Record, providerState?: OcxProviderContinuationState) => { - commitReasoningReplayServingRoute(); - rememberKiroDeliveredFinalAnswer(activeAdapter.name, response); - // Compaction turns must NOT enter the continuation cache: _rawBody still holds the full - // PRE-compaction history, and a later previous_response_id expansion would rehydrate the - // giant stale chain Codex just replaced. - if (!routedCompaction) { - rememberResponseState( - parsed._rawBody, - response, - continuationStateForResponse(providerState), - responseStateOptions(activeAdapter.name === "kiro"), - ); - } - notifyResponseComplete(response); - }, - }, - ); - const bridgeTurnAc = new AbortController(); - const trackedSse = trackStreamLifetime(sseStream, bridgeTurnAc, cleanupUpstreamAbort, options.turnAdmissionLease); - return new Response(trackedSse, { - headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache", "Connection": "keep-alive", "X-Accel-Buffering": "no" }, - }); - } - - if (activeAdapter.parseResponse) { - let events: AdapterEvent[]; - try { - const initialEvents = await activeAdapter.parseResponse( - upstreamResponse, - translatorBudget, - logCtx.activeTierMetadata, - ); - let guardedEvents: AdapterEvent[]; - if (terminalGuardEnabled) { - guardedEvents = []; - for await (const event of guardTerminalEventStream({ - parsed, - firstEvents: (async function* () { yield* initialEvents; })(), - adapterName: activeAdapter.name, - maxAutoContinuations: 1, - continuation: fetchTerminalGuardContinuation, - })) guardedEvents.push(event); - } else { - guardedEvents = initialEvents; - } - if (emptyCompletionGuardEnabled) { - events = []; - for await (const event of guardEmptyCompletionEventStream({ - firstEvents: (async function* () { yield* guardedEvents; })(), - continuation: fetchGuardedEmptyCompletionRetry, - })) events.push(event); - } else { - events = guardedEvents; - } - } finally { - cleanupUpstreamAbort(); - } - const { toolNsMap, declaredToolNames, toolParameterSchemas, freeformToolNames, toolSearchToolNames } = toolBridgeMaps; - let providerState: OcxProviderContinuationState | undefined; - const json = buildResponseJSON(events, parsed._responseModelId ?? parsed.modelId, { - translatorBudget, - replayCacheScope: parsed._reasoningReplayScope, - hideThinkingSummary: parsed.options.hideThinkingSummary, - toolNsMap, - declaredToolNames, - toolParameterSchemas, - freeformToolNames, - toolSearchToolNames, - ...(routedCompaction ? { compaction: true } : {}), - onProviderState: state => { providerState = state; }, - onUsage: usage => { - logCtx.usageFromBridge = true; - if (usage) { - logCtx.usage = usage; - if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; - } - }, - }); - // See the streaming branch: compaction turns skip the continuation cache. - if (!routedCompaction) { - rememberKiroDeliveredFinalAnswer(activeAdapter.name, json); - rememberResponseState( - parsed._rawBody, - json, - continuationStateForResponse(providerState), - responseStateOptions(activeAdapter.name === "kiro"), - ); - } - // #1926 gap 2: same buffered-path durability bound as the primary branch. - await awaitThoughtSignatureDurability(); - if (adapterResponseReachedServingTerminal(events, json)) { - commitReasoningReplayServingRoute(); - } - notifyResponseComplete(json); - return new Response(JSON.stringify(json), { headers: { "Content-Type": "application/json" } }); - } - - return formatErrorResponse(400, "invalid_request_error", "Non-streaming not supported by this adapter"); } finally { - if (pendingHostAdmissionLease) { - releaseUpstreamHostAdmission(pendingHostAdmissionLease); - releaseCodexAuthContextProbeLease(authCtx); - } - } -} - - - -export function linkAbortSignal(upstream: AbortController, signal?: AbortSignal): () => void { - if (!signal) return () => {}; - if (signal.aborted) { - upstream.abort(signal.reason); - return () => {}; - } - const onAbort = () => upstream.abort(signal.reason); - signal.addEventListener("abort", onAbort, { once: true }); - return () => signal.removeEventListener("abort", onAbort); -} + if (admissionState.pendingHostAdmissionLease) { + releaseUpstreamHostAdmission(admissionState.pendingHostAdmissionLease); + releaseCodexAuthContextProbeLease(admissionState.authCtx); + } + } +} + +const requestDispatchers: ResponsesDispatchers = { handleResponses, handleComboResponses }; + +export { adapterNeedsForcedContinuation } from "./core-replay"; +export { sidecarOutcomeRecorder } from "./core-codex-account"; +export { codexLogAccountId } from "./core-codex-account"; +export { shouldAttemptOpaqueBlobRecovery } from "./core-opaque-recovery"; +export { readDisplaySafeErrorText } from "./core-errors"; +export { usesCodexForwardPoolAuth } from "./core-codex-account"; +export { preAuthUpstreamHostCircuitKey } from "./core-codex-account"; +export { upstreamHostCircuitOpenResponse } from "./core-codex-account"; +export { shouldRetryCodexPoolAccountQuota } from "./core-codex-account"; +export { shouldRetryCodexPoolAccountTransient } from "./core-codex-account"; +export { codexAccountGatedCanonicalWireModel } from "./core-codex-account"; +export { codexForwardTerminalOutcomeRecorder } from "./core-codex-account"; +export { decodeRequestErrorResponse } from "./core-errors"; +export { comboUnavailableResponse } from "./core-errors"; +export type { ConsumedComboFailure } from "./core-options"; +export type { HandleResponsesOptions } from "./core-options"; +export { clientCancelledResponse } from "./core-errors"; +export { sanitizedRetryAfter } from "./core-combo-failure"; +export { consumeComboFailure } from "./core-combo-failure"; +export { usageFromComboFailureText } from "./core-combo-failure"; +export { createChildPassthroughCallbackGate } from "./core-combo-failure"; +export { buildComboChildHeaders } from "./core-combo-failure"; +export { UPSTREAM_JSON_BODY_READ_OPTIONS } from "./core-lifetime"; +export { poolCredentialRefreshIncompleteResponse } from "./core-auth"; +export { applyServiceTierGate } from "./core-normalize"; +export { linkAbortSignal } from "./core-lifetime"; +export { DEFAULT_SHADOW_SOURCE_MODELS, isShadowSourceModel, shadowSourceModels } from "../../lib/shadow-call"; diff --git a/src/server/responses/passthrough-delivery.ts b/src/server/responses/passthrough-delivery.ts new file mode 100644 index 0000000000..79e6ca3d46 --- /dev/null +++ b/src/server/responses/passthrough-delivery.ts @@ -0,0 +1,856 @@ +import type { ResponsesRequestContext, ResponsesAdmissionState } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResponsesSidecarAuth } from "./request-sidecar-auth"; +import type { ResponsesEffects } from "./response-effects"; +import type { PassthroughExchange } from "./passthrough-dispatch"; +import { + sanitizePassthroughHeaders, + createSseInspector, + markEagerRelaySseResponse, + markNativePassthroughSseResponse, + consumeForInspection, + consumeForResponseLogMetadata, + relaySseWithFailedTail, + relayWithAbort, +} from "../relay"; +import { isUsageDebugEnabled } from "../../usage/debug"; +import { + codexForwardTerminalOutcomeRecorder, + usesCodexForwardPoolAuth, + codexQuotaOutcomeMeta, + codexDenialOutcomeMeta, + isFixedCodexAccount, + shouldDeferCodexResetDerivedCooldown, +} from "./core-codex-account"; +import type { ResponsesTerminalStatus } from "../../bridge"; +import { isCodexWsQuotaObservedResponse, isCodexWsUpstreamResponse } from "./ws-upstream"; +import { recordSubagentQuotaFailureForThreadSpawn } from "../../codex/subagent-model-fallback"; +import { recordCodexUpstreamOutcome } from "../../codex/routing"; +import { codexProbeLeaseId, codexProbeQuotaScope } from "../../codex/auth-context"; +import { consumeComboFailure } from "./core-combo-failure"; +import { readDisplaySafeErrorText } from "./core-errors"; +import { streamingContextOverflowResponse, jsonContextOverflowResponse } from "./context-overflow"; +import { formatPassthroughUpstreamError } from "./passthrough-error"; +import { + providerModelResponsesTerminalRepair, + providerModelResponsesUpstreamStreaming, +} from "../../providers/registry"; +import { + resolvePassthroughWebSearchBridgeAuth, + planPassthroughWebSearchBridge, + createPassthroughWebSearchBridgeStream, + createPassthroughWebSearchBridgeExecutor, +} from "../../web-search/passthrough-bridge"; +import { fetchWithHeaderTimeout, providerFetch } from "./fetch-helpers"; +import { providerApiKeySelectionIsCurrent } from "../../providers/api-key-selection"; +import { requiresVisionPreprocessing } from "../../vision"; +import { checkOutboundBodySize, describeOutboundBodyRefusal } from "./outbound-body-guard"; +import { relayResponsesSseWithTerminalRepair } from "../responses-terminal-repair"; +import { + hasResponsesSnapshotRepair, + createResponsesSnapshotBlockRewrite, +} from "../responses-snapshot-repair"; +import { createResponsesModelPayloadRewrite, rewriteResponsesModelJson } from "../responses-model-rewrite"; +import { createImageGenCallRestoreRewrite, restoreImageGenCallsInJson } from "../responses-image-gen-repair"; +import { + createSelfNamedToolCallNamespaceScrubRewrite, + scrubSelfNamedToolCallNamespaceInJson, +} from "../responses-self-named-namespace-scrub"; +import { + createMuseToolNameRestoreRewrite, + restoreMuseToolNamesInJson, +} from "../../responses/muse-tool-name-alias"; +import { + createRoutedNamespaceCallRestoreRewrite, + restoreRoutedNamespaceCallsInJson, +} from "../../responses/namespace-tool-compat"; +import { + hasResponsesItemIdRepair, + createResponsesItemIdPayloadRewrite, + repairResponsesJsonItemIds, +} from "../responses-item-id-repair"; +import { + payloadRewriteAsBlockRewrite, + composeSsePayloadRewrites, + composeSseBlockRewrites, + relaySseWithBlockRewrite, +} from "../sse-payload-rewrite"; +import { createRoutedCustomToolRestoreBlockRewrite } from "../responses-custom-tool-repair"; +import { createRoutedToolSearchRestoreBlockRewrite } from "../responses-tool-search-repair"; +import { createGithubCopilotResponsesBlockRewrite } from "../github-copilot-responses-repair"; +import { createGrokResponsesControlFrameBlockRewrite } from "../grok-responses-control-frame"; +import { createGrokResponsesSparseTerminalBlockRewrite } from "../grok-responses-snapshot-repair"; +import { + createPlaintextV2AgentMessageCallRestoreRewrite, + restorePlaintextV2AgentMessageCallsInJsonResult, + PLAINTEXT_V2_AGENT_MESSAGE_RESTORE_OVERFLOW_MESSAGE, +} from "../../responses/plaintext-v2-agent-messages"; +import { createResponsesFieldBackfillBlockRewrite } from "./responses-field-backfill"; +import { createResponsesFunctionToolRepairBlockRewrite } from "../responses-function-tool-repair"; +import { + createUndeclaredToolCallGuardBlockRewrite, + undeclaredToolCallNameInResponse, + undeclaredToolCallMessage, + normalizeDefaultNamespaceInJson, +} from "../responses-undeclared-tool-guard"; +import { isWin32EagerRewrite, selectEagerPath } from "../../lib/bun-stream-caps"; +import { linkAbortSignal, UPSTREAM_JSON_BODY_READ_OPTIONS } from "./core-lifetime"; +import { registerTurn, unregisterTurn, trackStreamLifetime } from "../lifecycle"; +import { relaySseEagerBounded } from "../relay-eager"; +import { readBoundedResponseBody } from "../../lib/bounded-body"; +import { formatErrorResponse } from "../../bridge"; +import { inspectResponseLogJson } from "../request-log"; +import { restoreRoutedCustomCallsInJson } from "../../responses/custom-tool-compat"; +import { restoreRoutedToolSearchCallsInJson } from "../../responses/tool-search-compat"; +import { responsesJsonToSseStream } from "../responses-json-events"; + +/** One responsibility of the Responses request pipeline; state owners are explicit. */ +export async function deliverPassthroughResponse( + requestContext: Pick, + admissionState: ResponsesAdmissionState, + requestState: Pick< + PreparedResponsesRequest, + | "parsed" + | "route" + | "subagentQuotaFailureModel" + | "subagentFallbackAccountId" + | "clientRequestedStream" + | "translatorBudget" + >, + transportState: Pick, + sidecarState: Pick, + responseEffects: Pick< + ResponsesEffects, + | "plaintextV2AgentMessageToolNames" + | "commitReasoningReplayServingRoute" + | "routedMuseToolNameAliases" + | "routedNamespaceToolAliases" + | "plaintextV2AgentMessageAliasedToolNames" + | "recordTerminalOutcomes" + | "responseCompletionCancelled" + >, + nativeExchange: Pick< + PassthroughExchange, + | "upstreamResponse" + | "codexSafetyBufferingOptions" + | "upstream" + | "request" + | "connectMs" + | "imageGenCallAliases" + | "selfNamedNamespaceScrubAuthorization" + | "authorizedBareNamespaceToolAliases" + | "rememberPassthroughResponseChecked" + | "routedCustomToolNames" + | "routedCustomToolRepairNames" + | "declaredWireToolNames" + | "routedToolSearchNames" + | "outboundRequestBody" + | "functionRepairSchemas" + | "undeclaredToolGuardActive" + | "declaredNamelessClientCallTypes" + | "providerExecutedCallTypes" + | "declaredBareWireToolNames" + | "rememberPassthroughResponse" + | "noteInspectedPayload" + | "normalizeFunctionCompletionJson" + >, +): Promise { + const { logCtx, config, options, req } = requestContext; + const { + upstreamResponse, + codexSafetyBufferingOptions, + upstream, + connectMs, + imageGenCallAliases, + selfNamedNamespaceScrubAuthorization, + authorizedBareNamespaceToolAliases, + rememberPassthroughResponseChecked, + routedCustomToolNames, + routedCustomToolRepairNames, + declaredWireToolNames, + routedToolSearchNames, + functionRepairSchemas, + declaredNamelessClientCallTypes, + providerExecutedCallTypes, + declaredBareWireToolNames, + rememberPassthroughResponse, + noteInspectedPayload, + normalizeFunctionCompletionJson, + } = nativeExchange; + const { commitReasoningReplayServingRoute, recordTerminalOutcomes } = responseEffects; + const { parsed, route, subagentQuotaFailureModel, clientRequestedStream, translatorBudget } = requestState; + const { openAiSidecar } = sidecarState; + const { requestBindings } = transportState; + + const headers = sanitizePassthroughHeaders(upstreamResponse.headers, codexSafetyBufferingOptions); + const resolvedModel = headers.get("openai-model")?.trim(); + if (resolvedModel && !logCtx.preserveResolvedModelFromRoute) logCtx.resolvedModel = resolvedModel; + if (isUsageDebugEnabled()) { + const upstreamContentType = upstreamResponse.headers.get("content-type"); + if (upstreamContentType) logCtx.usageDebugContentType = upstreamContentType; + } + // The chatgpt backend may omit Content-Type on SSE responses. Fall back to + // treating a successful body as SSE when the caller requested streaming. + const passthroughCt = headers.get("content-type")?.toLowerCase(); + const isEventStream = passthroughCt?.includes("text/event-stream") + || (responseEffects.plaintextV2AgentMessageToolNames.size === 0 && upstreamResponse.ok && !!upstreamResponse.body && !passthroughCt && parsed.stream); + const recordTerminalOutcome = codexForwardTerminalOutcomeRecorder( + config, + admissionState.authCtx, + route.provider, + route.modelId, + logCtx, + ); + let terminalOutcomeRecorded = false; + const terminalRecorder = recordTerminalOutcome + ? (status: ResponsesTerminalStatus, httpStatusOverride?: number): void => { + if (terminalOutcomeRecorded) return; + terminalOutcomeRecorded = true; + recordTerminalOutcome(status, httpStatusOverride); + } + : undefined; + const terminalBodyWillRecord = !!terminalRecorder && upstreamResponse.ok && isEventStream; + // Capture quota from upstream response for multi-account tracking + if (usesCodexForwardPoolAuth(admissionState.authCtx, route.provider)) { + // primary was the 5h window; it now carries weekly data for GPT plans. + // Prefer primary when present, fall back to secondary for compatibility. + const quotaMeta = { ...codexQuotaOutcomeMeta(upstreamResponse), ...(await codexDenialOutcomeMeta(upstreamResponse)) }; + const { applyAccountQuotaFromUpstreamHeaders } = await import("../../codex/auth-api"); + if (!isCodexWsQuotaObservedResponse(upstreamResponse)) { + applyAccountQuotaFromUpstreamHeaders(admissionState.authCtx.accountId, upstreamResponse.headers, + admissionState.authCtx.writerGeneration, admissionState.authCtx.kind === "main-pool" ? admissionState.authCtx.mainQuotaWriter : undefined, + { modelId: route.modelId, poolWriter: admissionState.authCtx.kind === "pool" ? admissionState.authCtx.poolQuotaWriter : undefined }); + } + if (terminalBodyWillRecord) { + options.setTerminalOutcomeRecorder?.((status, httpStatusOverride) => { + terminalRecorder(status, httpStatusOverride); + if (status === "failed" || status === "incomplete") { + const quotaFailureMessage = [httpStatusOverride, logCtx.terminalHttpStatus] + .find(value => value === 429 || value === 402); + if (!isFixedCodexAccount(admissionState.authCtx) && quotaFailureMessage !== undefined) { + recordSubagentQuotaFailureForThreadSpawn( + req.headers, + subagentQuotaFailureModel, + quotaFailureMessage, + config, + requestState.subagentFallbackAccountId, + ); + } + } + options.onNativePassthroughTerminal?.(status); + }); + } else if (!shouldDeferCodexResetDerivedCooldown( + upstreamResponse, + options.deferCodexResetDerivedCooldown, + )) { + recordCodexUpstreamOutcome(config, admissionState.authCtx.accountId, upstreamResponse.status, { + ...quotaMeta, + threadId: admissionState.authCtx.affinityKey, + fixedAccount: admissionState.authCtx.fixedAccount, + modelId: route.modelId, + probeLeaseId: codexProbeLeaseId(admissionState.authCtx), + probeQuotaScope: codexProbeQuotaScope(admissionState.authCtx), + writerGeneration: admissionState.authCtx.writerGeneration, + // Includes a replay's second 401, which is the case that actually retires the + // account — fence it on the credential the request was holding. + ...(admissionState.authCtx.kind === "pool" ? { credentialGeneration: admissionState.authCtx.generation } : {}), + }); + } + } + + // Non-2xx passthrough failures must never reach Codex as an empty body — + // Codex renders that as the opaque "Unknown error" (#452). Combo attempts + // keep their typed failure envelope. Except for the classified 413 below, + // non-empty bodies are relayed verbatim + // (headers included) so pool-retry Activation B/D and client diagnostics stay intact. + // Manual-redirect policy (#914): a 3xx is relayed as-is (Location preserved + // through sanitizePassthroughHeaders) so a redirect to a dead host can never + // masquerade as a pre-connection failure after the credential was seen. + // The numeric outcome above already classified it neutral — no streak. + if (upstreamResponse.status >= 300 && upstreamResponse.status < 400) { + return new Response(upstreamResponse.body, { + status: upstreamResponse.status, + statusText: upstreamResponse.statusText, + headers: sanitizePassthroughHeaders(upstreamResponse.headers, codexSafetyBufferingOptions), + }); + } + if (!upstreamResponse.ok) { + if (options.comboAttempt) { + // No pre-read guard here: `consumeComboFailure` -> `readBoundedResponseBody` reads + // `response.body` itself and already threads the abort signal through its own read, + // and the combo contract is that this body's getter is touched exactly once (pinned by + // "captures passthrough failed usage from its original bounded body exactly once"). + // Attaching a guard would be a second `.body` access and break that contract for no + // gain, since the bounded reader owns settlement on this path. + const failure = await consumeComboFailure(upstreamResponse, options.abortSignal); + options.onConsumedComboFailure?.(failure); + return failure.response; + } + // The bounded reader owns the original body, deadline, abort settlement, and lock. + // Unsafe partial data falls back to #452's non-empty status-only JSON. + const errorText = await readDisplaySafeErrorText(upstreamResponse, upstream.signal, ""); + if (upstreamResponse.status === 413) { + return clientRequestedStream + ? streamingContextOverflowResponse(parsed._responseModelId ?? parsed.modelId, translatorBudget) + : jsonContextOverflowResponse(); + } + return formatPassthroughUpstreamError(upstreamResponse.status, errorText, { + statusText: upstreamResponse.statusText, + headers, + }); + } + + // Bun#32111 workaround: passthrough SSE uses tee()+native relay to avoid the + // async-pull segfault on Windows. Branch[0] goes directly to the Response (Bun + // native relay, never enters JS Sink.write); branch[1] is consumed in the + // background for terminal-outcome/quota inspection only. + // #314 alternative shape: win32 no-rewrite traffic follows the runtime/config + // gate; darwin no-rewrite traffic joins it only for explicit + // `streamMode: "eager-relay"` opt-in. Darwin `auto` always stays tee. The + // eager shape skips tee and uses one bounded reader with inline inspection + // (src/server/relay-eager.ts; policy: + // devlog/_fin/260731_macos_rss_retention/100_darwin_eager_optin.md). + // The bundled known-bad runtime remains on tee by default on both platforms. + if (isEventStream && upstreamResponse.body) { + // For streamed passthrough, a successful terminal response means non-error upstream status + // before relay starts. Waiting for SSE completion would retain request state across the whole + // stream; a later body failure does not undo that this destination accepted and served the turn. + commitReasoningReplayServingRoute(nativeExchange.request.headers); + const terminalRepairPolicy = providerModelResponsesTerminalRepair( + route.providerName, + route.provider, + route.modelId, + ); + // #3761: opt-in hosted-web-search bridge. Codex always declares the hosted web_search tool, + // and this branch relays that declaration on the assumption the destination executes it. + // A KEY-auth gateway that does not (Ollama Cloud GLM) answers with a function_call named + // web_search that nothing runs, and the undeclared-tool guard below ends the turn. When the + // provider opts in, the bridge intercepts that one call, runs the search, continues the + // conversation upstream, and hands back ordinary Responses SSE — so every rewrite below, + // including the guard itself, still inspects the client-facing stream. Default OFF: without + // the opt-in this is one planner call and the relay is byte-identical to before. + const webSearchBridgeAuth = resolvePassthroughWebSearchBridgeAuth( + route.provider.webSearchBridge?.backend, + config, + openAiSidecar, + ); + const webSearchBridgePlan = planPassthroughWebSearchBridge(parsed, route.provider, { + providerName: route.providerName, + isPassthrough: true, + stream: parsed.stream === true, + auth: webSearchBridgeAuth, + }); + // Capture the binding that actually served the first leg, after its permitted reselection. + const webSearchBridgeBinding = requestBindings.get(nativeExchange.request); + // The bridge wraps the RAW upstream body, so terminal repair below still owns the single + // client-facing terminal — the bridge drops the terminal of every intercepted leg. + const upstreamSseBody = webSearchBridgePlan + ? createPassthroughWebSearchBridgeStream({ + plan: webSearchBridgePlan, + firstLeg: upstreamResponse.body, + requestBody: nativeExchange.request.body, + // Continuation legs replay the same built request with the executed search appended. + // The first leg already passed the recovery ladder, the outbound size ceiling, and the + // host circuit; a KEY-auth destination has no OAuth refresh to replay on a later leg. + send: (continuationBody: string) => fetchWithHeaderTimeout( + nativeExchange.request.url, + { method: nativeExchange.request.method, headers: nativeExchange.request.headers, body: continuationBody }, + upstream.signal, + connectMs, + true, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + // Pacing can outlive a manual selection change. A continuation must retain the + // first leg's key and appended search result, never rebuild from the original turn. + beforeDispatch: () => { + if (webSearchBridgeBinding?.kind !== "api-key" + || !providerApiKeySelectionIsCurrent(config, route.providerName, webSearchBridgeBinding.provider)) { + throw new Error("API key selection changed during a web-search continuation"); + } + }, + providerName: route.providerName, + modelId: route.modelId, + }), + false, + ), + execute: createPassthroughWebSearchBridgeExecutor(webSearchBridgePlan, { + providerApiKey: route.provider.apiKey ?? "", + auth: webSearchBridgeAuth, + hostedTool: parsed._webSearch, + describeImages: requiresVisionPreprocessing(config, route.provider, route.modelId, route.providerName), + sidecar: config.webSearchSidecar, + }), + // Appending a search result can push the continuation past the ceiling the first leg + // was admitted under, so the same limit is re-applied before every later send. + checkOutboundBody: (continuationBody: string) => { + const result = checkOutboundBodySize(continuationBody, config.maxUpstreamBodyBytes); + return result.admitted ? undefined : describeOutboundBodyRefusal(result); + }, + signal: upstream.signal, + }) + : upstreamResponse.body; + const passthroughSseBody = terminalRepairPolicy + ? relayResponsesSseWithTerminalRepair( + upstreamSseBody, + upstream, + terminalRepairPolicy, + translatorBudget, + options.responsesTerminalRepairScheduler, + ) + : upstreamSseBody; + const repairConfig = route.provider.responsesItemIdRepair; + // Grok Build renders deltas live but reconstructs its durable assistant + // turn from the completed response snapshot. Native Responses streams + // may instead carry the complete items in output_item.done, so the + // explicit Grok compatibility marker enables strict client compatibility rewrites. + // The provider's broader snapshot/lifecycle repair remains opt-in. + const grokClientCompatibilityEnabled = logCtx.surface === "grok"; + const snapshotRepairEnabled = hasResponsesSnapshotRepair(route.provider.responsesSnapshotRepair); + const githubCopilotRepairEnabled = route.providerName === "github-copilot"; + const responseModelRewrite = parsed._responseModelId !== undefined + && parsed._responseModelId !== parsed.modelId + ? createResponsesModelPayloadRewrite(parsed._responseModelId) + : undefined; + // Compose opt-in payload rewrites into one parse/stringify pass (image-gen restore first). + const payloadRewrites = [ + createImageGenCallRestoreRewrite(imageGenCallAliases), + // #3217: a call whose namespace repeats its own name is unroutable in codex-rs. + createSelfNamedToolCallNamespaceScrubRewrite(selfNamedNamespaceScrubAuthorization), + responseEffects.routedMuseToolNameAliases.size > 0 + ? createMuseToolNameRestoreRewrite(responseEffects.routedMuseToolNameAliases) + : undefined, + responseEffects.routedNamespaceToolAliases.size > 0 + ? createRoutedNamespaceCallRestoreRewrite(responseEffects.routedNamespaceToolAliases) + : undefined, + authorizedBareNamespaceToolAliases.size > 0 + ? createRoutedNamespaceCallRestoreRewrite(authorizedBareNamespaceToolAliases) + : undefined, + hasResponsesItemIdRepair(repairConfig) + ? createResponsesItemIdPayloadRewrite(repairConfig!, translatorBudget) + : undefined, + responseModelRewrite, + ].filter((rewrite): rewrite is NonNullable => rewrite !== undefined); + // #893: sparse-snapshot gateways get field backfills AND lifecycle event + // injection at the block level, after payload rewrites. Defaults come + // from the finalized OUTBOUND body — the normalized internal tool shapes + // are not the Responses wire shapes the snapshot must mirror. + // Only validated client blocks may publish plaintext continuation state. + // Raw inspection precedes rewriting on eager relays, so it cannot own this write. + const plaintextInspector = responseEffects.plaintextV2AgentMessageToolNames.size > 0 + ? createSseInspector({ onCompletedResponse: rememberPassthroughResponseChecked }) + : undefined; + const plaintextEncoder = plaintextInspector ? new TextEncoder() : undefined; + const rememberPlaintextBlock = plaintextInspector + ? Object.assign((block: string): readonly string[] => { + plaintextInspector.feed(plaintextEncoder!.encode(`${block}\n\n`)); + return [block]; + }, { dispose: () => plaintextInspector.dispose() }) + : undefined; + const blockRewrites = [ + payloadRewrites.length > 0 + ? payloadRewriteAsBlockRewrite(composeSsePayloadRewrites(...payloadRewrites)) + : undefined, + routedCustomToolNames.size > 0 || routedCustomToolRepairNames.size > 0 + ? createRoutedCustomToolRestoreBlockRewrite( + routedCustomToolNames, + translatorBudget, + routedCustomToolRepairNames, + declaredWireToolNames, + ) + : undefined, + routedToolSearchNames.size > 0 + ? createRoutedToolSearchRestoreBlockRewrite(routedToolSearchNames, translatorBudget) + : undefined, + githubCopilotRepairEnabled + ? createGithubCopilotResponsesBlockRewrite(translatorBudget) + : undefined, + grokClientCompatibilityEnabled + ? createGrokResponsesControlFrameBlockRewrite() + : undefined, + grokClientCompatibilityEnabled + ? createGrokResponsesSparseTerminalBlockRewrite(translatorBudget) + : undefined, + snapshotRepairEnabled + ? createResponsesSnapshotBlockRewrite(nativeExchange.outboundRequestBody, translatorBudget) + : undefined, + responseEffects.plaintextV2AgentMessageToolNames.size > 0 + ? payloadRewriteAsBlockRewrite(createPlaintextV2AgentMessageCallRestoreRewrite( + responseEffects.plaintextV2AgentMessageToolNames, responseEffects.plaintextV2AgentMessageAliasedToolNames, + )) + : undefined, + createResponsesFieldBackfillBlockRewrite(), + functionRepairSchemas.size > 0 + ? createResponsesFunctionToolRepairBlockRewrite(functionRepairSchemas, translatorBudget) + : undefined, + // Last: every rewrite above can still rename or reshape a call item, so the guard must + // compare the names the client will actually receive against the declared catalog. + nativeExchange.undeclaredToolGuardActive + ? createUndeclaredToolCallGuardBlockRewrite( + declaredWireToolNames, + declaredNamelessClientCallTypes, + providerExecutedCallTypes, + declaredBareWireToolNames, + ) + : undefined, + rememberPlaintextBlock, + ].filter((rewrite): rewrite is NonNullable => rewrite !== undefined); + const clientBlockRewrite = blockRewrites.length > 0 + ? composeSseBlockRewrites(...blockRewrites) + : undefined; + const needsClientRewrite = clientBlockRewrite !== undefined; + // #864: win32 rewrite traffic must never enter the tee()+JS-pull chain + // (Bun#32111 JS-sink segfault — text frames pass, the terminal block is + // lost). The eager single reader applies the same rewrites inline. + const win32EagerRewrite = isWin32EagerRewrite(process.platform, needsClientRewrite); + const eagerPath = selectEagerPath( + process.platform, + needsClientRewrite, + config.streamMode ?? "auto", + ); + // A successful Codex WS upgrade is a push source. If it entered tee(), + // the inspection branch could drain continuously while the slow client + // branch retained bytes without a bound. Force the existing bounded, + // single-reader relay before tee; HTTP fallback responses stay unmarked. + const forceCodexWsEagerRelay = isCodexWsUpstreamResponse(upstreamResponse); + const inlineEagerRewrite = needsClientRewrite + && (forceCodexWsEagerRelay || win32EagerRewrite || eagerPath?.useEagerRelay === true); + if (forceCodexWsEagerRelay || eagerPath?.useEagerRelay || win32EagerRewrite) { + const turnAc = new AbortController(); + linkAbortSignal(upstream, turnAc.signal); + registerTurn(turnAc, options.turnAdmissionLease); + const reportNativeTerminal = recordTerminalOutcomes + ? (status: ResponsesTerminalStatus, httpStatusOverride?: number) => { + terminalRecorder?.(status, httpStatusOverride); + if (status === "failed" || status === "incomplete") { + const quotaFailureMessage = [httpStatusOverride, logCtx.terminalHttpStatus] + .find(value => value === 429 || value === 402); + if (!isFixedCodexAccount(admissionState.authCtx) && quotaFailureMessage !== undefined) { + recordSubagentQuotaFailureForThreadSpawn( + req.headers, + subagentQuotaFailureModel, + quotaFailureMessage, + config, + requestState.subagentFallbackAccountId, + ); + } + } + options.onNativePassthroughTerminal?.(status); + } + : undefined; + const inspector = createSseInspector({ + onTerminal: reportNativeTerminal, + logCtx, + onCompletedResponse: rememberPassthroughResponse && responseEffects.plaintextV2AgentMessageToolNames.size === 0 ? rememberPassthroughResponseChecked : undefined, + onParsedPayload: noteInspectedPayload, + onFirstOutput: options.onFirstOutput, + pinCompletedResponseIdToFirstSeen: githubCopilotRepairEnabled, + }); + const eagerBody = relaySseEagerBounded(passthroughSseBody, turnAc, { + inspectChunk: chunk => inspector.feed(chunk), + finishInspection: () => inspector.finish(), + disposeInspection: () => inspector.dispose(), + // Stream lifetime follows the protocol terminal even when this request + // has no outcome callback configured (reported() would stay false). + sawTerminal: () => inspector.terminalSeen(), + ...(clientBlockRewrite + ? { rewriteBlocks: clientBlockRewrite } + : {}), + onSynthetic: (kind, reason) => { + if (!reportNativeTerminal) return; + if (kind === "incomplete") { + logCtx.terminalSource = "synthetic"; + reportNativeTerminal("incomplete"); + } else if (reason === "upstream_error") { + logCtx.terminalSource = "synthetic"; + reportNativeTerminal("failed", logCtx.terminalHttpStatus ?? 502); + } else { + logCtx.transportPhase = "mid_stream"; + logCtx.terminalSource = "synthetic"; + if (logCtx.activeAttempt) logCtx.activeAttempt.streamAborted = true; + reportNativeTerminal("failed", 502); + } + }, + onClientCancel: () => { + responseEffects.responseCompletionCancelled = true; + options.onNativePassthroughCancel?.(); + }, + onDone: () => unregisterTurn(turnAc), + }, { + clientGoneSignal: options.abortSignal, + terminalBoundary: codexSafetyBufferingOptions, + ...(inlineEagerRewrite ? { rewriteBudget: translatorBudget } : {}), + ...(logCtx.upstreamError === undefined ? {} : { upstreamError: logCtx.upstreamError }), + }); + // When selected, this relay closes response.completed even if upstream + // keeps the connection alive. Marked Codex WS traffic, Windows + // forced-rewrite traffic, and Darwin explicit eager traffic apply + // client rewrites inline rather than via the tee()+JS-pull chain. + if (!headers.has("content-type")) headers.set("content-type", "text/event-stream"); + return markEagerRelaySseResponse( + markNativePassthroughSseResponse(new Response(eagerBody, { + status: upstreamResponse.status, + headers, + })), + ); + } + const [nativeBody, inspectBody] = passthroughSseBody.tee(); + const turnAc = new AbortController(); + const clientGone = new AbortController(); + linkAbortSignal(upstream, turnAc.signal); + registerTurn(turnAc, options.turnAdmissionLease); + const inspectionConsumerOptions = { + // Request abort can reject the fetch body before the response cancel hook runs. + clientGoneSignal: options.abortSignal + ? AbortSignal.any([clientGone.signal, options.abortSignal]) + : clientGone.signal, + drainBounds: { ms: 15_000, bytes: 32 * 1024 * 1024 }, + upstream, + pinCompletedResponseIdToFirstSeen: githubCopilotRepairEnabled, + onParsedPayload: noteInspectedPayload, + }; + if (recordTerminalOutcomes) { + // A real terminal was parsed from the (teed) inspection stream — record it as the outcome + // even if the client has already disconnected: the turn genuinely reached that terminal, so + // it must log as completed/failed, not be dropped or downgraded to a cancel (#44). A pure + // client-cancel (no terminal seen) is finalized separately via consumeForInspection's onCancel. + const reportNativeTerminal = (status: ResponsesTerminalStatus, httpStatusOverride?: number) => { + terminalRecorder?.(status, httpStatusOverride); + if (status === "failed" || status === "incomplete") { + const quotaFailureMessage = [httpStatusOverride, logCtx.terminalHttpStatus] + .find(value => value === 429 || value === 402); + if (!isFixedCodexAccount(admissionState.authCtx) && quotaFailureMessage !== undefined) { + recordSubagentQuotaFailureForThreadSpawn( + req.headers, + subagentQuotaFailureModel, + quotaFailureMessage, + config, + requestState.subagentFallbackAccountId, + ); + } + } + options.onNativePassthroughTerminal?.(status); + }; + consumeForInspection( + inspectBody, + reportNativeTerminal, + turnAc.signal, + () => unregisterTurn(turnAc), + logCtx, + () => { + responseEffects.responseCompletionCancelled = true; + options.onNativePassthroughCancel?.(); + }, + rememberPassthroughResponse && responseEffects.plaintextV2AgentMessageToolNames.size === 0 ? rememberPassthroughResponseChecked : undefined, + options.onFirstOutput, + inspectionConsumerOptions, + ); + } else { + consumeForResponseLogMetadata( + inspectBody, + logCtx, + turnAc.signal, + () => unregisterTurn(turnAc), + rememberPassthroughResponse && responseEffects.plaintextV2AgentMessageToolNames.size === 0 ? rememberPassthroughResponseChecked : undefined, + options.onFirstOutput, + inspectionConsumerOptions, + ); + } + if (!headers.has("content-type")) headers.set("content-type", "text/event-stream"); + // Windows was handled by the eager terminal-aware branch above. Remaining + // tee traffic can use the JS relay to close on a protocol terminal and to + // convert a mid-stream reset into a clean response.failed event. + const rewrittenBody = clientBlockRewrite !== undefined + ? relaySseWithBlockRewrite(nativeBody, clientBlockRewrite, translatorBudget) + : nativeBody; + const clientBody = relaySseWithFailedTail( + rewrittenBody, + upstream, + reason => { + responseEffects.responseCompletionCancelled = true; + clientGone.abort(reason); + }, + { upstreamError: logCtx.upstreamError, terminalBoundary: codexSafetyBufferingOptions }, + ); + return markNativePassthroughSseResponse(new Response(clientBody, { + status: upstreamResponse.status, + headers, + })); + } + if (headers.get("content-type")?.toLowerCase().includes("application/json")) { + // Bounded whole-body read: a non-streaming upstream JSON body is fully materialized + // here (and again by the request-log finalizer and the WebSocket bridge's reframing), + // so an unbounded .text() would let a hostile or stuck upstream grow proxy memory + // without limit. This path is no longer rare — WebSocket turns for models whose + // streaming terminal event is unreliable are deliberately answered with bounded JSON. + // Oversize and stall deadlines both fail closed; a partial body is never parsed. + const bounded = await readBoundedResponseBody(upstreamResponse, UPSTREAM_JSON_BODY_READ_OPTIONS); + if (bounded.oversized) { + return formatErrorResponse(502, "upstream_error", "upstream JSON response exceeded the safe body limit"); + } + if (bounded.truncated) { + return formatErrorResponse(502, "upstream_error", "upstream JSON response stalled before completing"); + } + const text = bounded.text; + inspectResponseLogJson(logCtx, text); + let plaintextV2RestoreFailed = false; + let clientJson = (() => { + const restoredNamespace = restoreRoutedNamespaceCallsInJson( + scrubSelfNamedToolCallNamespaceInJson( + restoreMuseToolNamesInJson( + restoreImageGenCallsInJson(text, imageGenCallAliases), + responseEffects.routedMuseToolNameAliases, + ), + selfNamedNamespaceScrubAuthorization, + ), + responseEffects.routedNamespaceToolAliases, + ); + const restoredAuthorizedBareNamespace = restoreRoutedNamespaceCallsInJson( + restoredNamespace, + authorizedBareNamespaceToolAliases, + ); + const restored = restoreRoutedCustomCallsInJson( + restoredAuthorizedBareNamespace, + routedCustomToolNames, + routedCustomToolRepairNames, + declaredWireToolNames, + ); + const restoredToolSearch = restoreRoutedToolSearchCallsInJson( + restored, + routedToolSearchNames, + ); + const normalizedJson = normalizeFunctionCompletionJson(restoredToolSearch); + const plaintextRestore = restorePlaintextV2AgentMessageCallsInJsonResult( + normalizedJson, responseEffects.plaintextV2AgentMessageToolNames, responseEffects.plaintextV2AgentMessageAliasedToolNames, + ); + plaintextV2RestoreFailed = plaintextRestore.overflowed; + const repaired = plaintextRestore.value; + const modelRewritten = parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId + ? rewriteResponsesModelJson(repaired, parsed._responseModelId) + : repaired; + return modelRewritten; + })(); + if (plaintextV2RestoreFailed) { + return formatErrorResponse(502, "upstream_error", PLAINTEXT_V2_AGENT_MESSAGE_RESTORE_OVERFLOW_MESSAGE); + } + // #1700: same fail-closed policy as the SSE relay above. Both the plain JSON answer and + // the reframed-SSE branch below are built from this body, so one check covers them. This + // runs BEFORE the continuation cache write below: a refused turn must not become state a + // later `previous_response_id` replay can expand from. + if (nativeExchange.undeclaredToolGuardActive) { + const undeclared = (() => { + try { + return undeclaredToolCallNameInResponse( + JSON.parse(clientJson), + declaredWireToolNames, + declaredNamelessClientCallTypes, + providerExecutedCallTypes, + declaredBareWireToolNames, + ); + } catch { + return undefined; + } + })(); + if (undeclared !== undefined) { + return formatErrorResponse(502, "upstream_error", undeclaredToolCallMessage(undeclared)); + } + clientJson = normalizeDefaultNamespaceInJson( + clientJson, + declaredWireToolNames, + declaredBareWireToolNames, + ); + } + commitReasoningReplayServingRoute(nativeExchange.request.headers); + try { + rememberPassthroughResponseChecked( + JSON.parse(text) as { id?: unknown; output?: unknown; status?: unknown; model?: unknown }, + ); + } catch { /* non-JSON despite content-type; recording is best-effort */ } + // #875: the transport-neutral reliability policy forced a bounded JSON + // upstream for a client that asked for SSE. Reframe the completed JSON + // as the canonical terminal SSE sequence (created → output_item.done → + // terminal → [DONE]) so Codex commits the turn instead of hanging on a + // stream that never closes. Non-streaming clients keep the plain JSON. + if (clientRequestedStream === true + && options.inboundTransport !== "websocket" + && providerModelResponsesUpstreamStreaming(route.providerName, route.provider, route.modelId) === false + && route.provider.adapter === "openai-responses") { + let completed: Record | undefined; + try { + const parsedCompleted = JSON.parse(clientJson) as unknown; + if (!parsedCompleted || typeof parsedCompleted !== "object" || Array.isArray(parsedCompleted)) { + throw new TypeError("bounded Responses JSON is not an object"); + } + let candidate = parsedCompleted as Record; + // The bounded-JSON answer bypasses the SSE relay, so it also bypasses + // the SSE item-id rewrite. Apply the same client-facing normalization + // here or this policy would silently disable id repair for the very + // providers that need it (raw record already happened above). + if (hasResponsesItemIdRepair(route.provider.responsesItemIdRepair)) { + candidate = repairResponsesJsonItemIds(candidate, route.provider.responsesItemIdRepair!, translatorBudget); + } + completed = candidate; + } catch { + // Non-JSON despite content-type: fall through to the plain relay. + } + if (completed) { + let stream: ReadableStream; + try { + stream = responsesJsonToSseStream(completed); + } catch (error) { + if (error instanceof RangeError) { + return formatErrorResponse( + 502, + "upstream_error", + "upstream JSON response exceeded the synthesized SSE item limit", + ); + } + throw error; + } + const sseHeaders = sanitizePassthroughHeaders(headers, codexSafetyBufferingOptions); + sseHeaders.set("content-type", "text/event-stream"); + sseHeaders.set("cache-control", "no-store"); + return new Response(stream, { + status: upstreamResponse.status, + statusText: upstreamResponse.statusText, + headers: sseHeaders, + }); + } + } + // WS turns reframe this JSON into events in the bridge, which is the + // other relay-free path — normalize ids so both bounded-JSON paths agree. + const outboundJson = options.inboundTransport === "websocket" + && providerModelResponsesUpstreamStreaming(route.providerName, route.provider, route.modelId) === false + && hasResponsesItemIdRepair(route.provider.responsesItemIdRepair) + ? (() => { + try { + return JSON.stringify(repairResponsesJsonItemIds( + JSON.parse(clientJson) as Record, + route.provider.responsesItemIdRepair!, + translatorBudget, + )); + } catch { + return clientJson; + } + })() + : clientJson; + return new Response(outboundJson, { + status: upstreamResponse.status, + statusText: upstreamResponse.statusText, + headers, + }); + } + if (responseEffects.plaintextV2AgentMessageToolNames.size > 0) { + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already closed */ } + return formatErrorResponse(502, "upstream_error", "plaintext V2 agent-message response used an unsupported content type"); + } + // An unclassified passthrough body is relayed directly and has no bounded completion observer; + // use the same non-error-status success boundary as SSE instead of retaining per-stream state. + commitReasoningReplayServingRoute(nativeExchange.request.headers); + const body = relayWithAbort(upstreamResponse.body, upstream); + const turnAc = new AbortController(); + const tracked = body ? trackStreamLifetime(body, turnAc, undefined, options.turnAdmissionLease) : null; + return new Response(tracked, { + status: upstreamResponse.status, + headers, + }); +} diff --git a/src/server/responses/passthrough-dispatch.ts b/src/server/responses/passthrough-dispatch.ts new file mode 100644 index 0000000000..f5c5b94694 --- /dev/null +++ b/src/server/responses/passthrough-dispatch.ts @@ -0,0 +1,1476 @@ +import type { + ResponsesRequestContext, + ResponsesAdmissionState, + PassthroughAdmissionState, +} from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResponsesEffects } from "./response-effects"; +import type { ResponsesSendBudget } from "./request-send-budget"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import { codexSafetyBufferingFilterOptions, terminalStatusFromParsed } from "../relay"; +import { imageGenToolCallAliases } from "../responses-image-gen-repair"; +import { rememberResponseState } from "../../responses/state"; +import { + currentTurnWireToolCatalogBody, + hasExplicitWireToolCatalog, + collectDeclaredWireToolNames, + collectDeclaredBareWireToolNames, + collectDeclaredNamelessClientCallTypes, + collectProviderExecutedCallTypes, + undeclaredToolCallName, + undeclaredToolCallNameInResponse, + normalizeDefaultNamespaceInResponse, +} from "../responses-undeclared-tool-guard"; +import { collectSelfNamedNamespaceScrubAuthorization } from "../responses-self-named-namespace-scrub"; +import type { ProviderExecutedCallType } from "../responses-undeclared-tool-guard"; +import { + releaseCodexAuthContextProbeLease, + unwrapUpstreamRetryEvidenceError, + codexProbeLeaseId, + codexProbeQuotaScope, + createCodexReserveDispatchGuard, +} from "../../codex/auth-context"; +import { + NamespaceToolCollisionError, + restoreRoutedNamespaceCalls, +} from "../../responses/namespace-tool-compat"; +import { XaiToolSchemaCompatibilityError } from "../../adapters/xai-tool-schema"; +import { formatErrorResponse } from "../../bridge"; +import { redactSecretString } from "../../lib/redact"; +import { + collectFunctionCallRepairSchemas, + repairFunctionCallsInJson, +} from "../../responses/function-call-compat"; +import type { RoutedNamespaceToolAliases } from "../../responses/namespace-tool-compat"; +import { hasResponsesSnapshotRepair, repairResponsesSnapshotJson } from "../responses-snapshot-repair"; +import { backfillResponsesFieldsJson } from "./responses-field-backfill"; +import type { AdapterRequest } from "../../adapters/base"; +import { isXaiResponsesDestination, resolveProviderTransport } from "../../providers/xai-transport"; +import { CODE_MODE_EXEC_TOOL_NAME } from "../../types"; +import type { ResponsesTerminalStatus } from "../../bridge"; +import { hasPassiveAccountQuota, recordPassiveAccountQuota } from "../../providers/quota"; +import { + isMuseSubscriptionUsagePayload, + parseMuseSubscriptionUsage, +} from "../../providers/muse-subscription-usage"; +import { restoreMuseToolNames } from "../../responses/muse-tool-name-alias"; +import { restoreRoutedCustomCalls } from "../../responses/custom-tool-compat"; +import { restorePlaintextV2AgentMessageCalls } from "../../responses/plaintext-v2-agent-messages"; +import { + recordAdapterReasoning, + recordAdapterTier, + noteAttemptSend, + sealRequestAttemptIdentity, + recordAttemptCredentialSource, +} from "../request-log"; +import { + upstreamHostHealthKey, + normalizeUpstreamHostCircuitThreshold, + disableUpstreamHostCircuitForKey, + acquireUpstreamHostAdmission, + resetUpstreamHostHealth, + releaseUpstreamHostAdmission, + recordUpstreamHostFailure, +} from "../../codex/upstream-host-health"; +import { + safeOriginLabel, + fetchWithHeaderTimeout, + providerFetch, + safeHostLabel, + storedPoolReplayDispatchNotifier, +} from "./fetch-helpers"; +import { clientCancelledResponse } from "./core-errors"; +import { + upstreamHostCircuitOpenResponse, + usesCodexForwardPoolAuth, + codexWsQuotaObserver, + isFixedCodexAccount, + shouldRetryCodexPoolAccountModel400, + shouldRetryCodexPoolAccountQuota, + shouldRetryCodexPoolAccountTransient, + retryCodexPoolOnAlternateAccount, +} from "./core-codex-account"; +import { readCodexWsStage } from "./codex-ws-wire"; +import { linkAbortSignal } from "./core-lifetime"; +import type { CodexAuthContext } from "../../codex/auth-context"; +import { checkOutboundBodySize, describeOutboundBodyRefusal } from "./outbound-body-guard"; +import { streamingContextOverflowResponse } from "./context-overflow"; +import { + SendBudgetExhaustedError, + fetchWithTransientRetry, + applyUpstreamRecoveryInit, + TRANSIENT_RETRY_MAX_ATTEMPTS, + prepareSameTarget429Wait, + sleepWithAbort, +} from "../../lib/upstream-retry"; +import { mapCodexAuthContextErrorToResponse } from "./codex-auth-error"; +import { classifyTransportFailureKind, transportErrorCode } from "../../lib/upstream-reachability"; +import { recordCodexUpstreamOutcome } from "../../codex/routing"; +import { describeUpstreamConnectFailure } from "./upstream-error"; +import type { OpaqueBlobRecoveryGuard } from "./core-opaque-recovery"; +import { rateLimitRetryPolicyFor, rateLimitRetryDelayMs } from "../../providers/key-failover"; +import type { AttemptRecoveryKind } from "../../usage/log"; +import { resolveWireProtocolOverride } from "../adapter-resolve"; +import { refreshPoolForwardAuth, refreshNativeMainForwardAuth, withClaudeNativeSession } from "./core-auth"; +import { bindRouteReasoningReplayScope } from "./core-replay"; +import type { OAuthAccessSnapshot } from "../../oauth"; +import { publicOAuthAuthenticationErrorMessage } from "../../oauth"; +import { resolveCopilotApiBaseUrl } from "../../oauth/github-copilot"; +import { + GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST, + isGenericOAuthFailoverEnabled, + rotateGenericOAuthAccountOn429, + failoverAccountSnapshot, +} from "../../oauth/generic-account-failover"; +import { captureCodexAffinityDiagnostic } from "../../codex/affinity-debug"; +import { ACCOUNT_GATED_NATIVE_OPENAI_MODELS } from "../../codex/catalog/native-models"; +import { + attemptOpaqueBlobRecovery, + outboundResponsesBodyCarriesEncryptedFunctionOutput, + resetStreamedOpaqueBlobLogContext, + consoleGoUploadRejectionBody, + CONSOLE_GO_UPLOAD_RETRY_DELAY_MS, + reasoningEffortRejectionText, +} from "./core-opaque-recovery"; +import type { RequestLogContext } from "../request-log"; +import { preflightComboStreamResponse } from "./combo-stream-preflight"; +import { upstreamErrorMessageFromPayload, ENCRYPTED_FUNCTION_OUTPUT_REJECTION } from "../../lib/errors"; +import { isTransientConsoleGoUploadRejection } from "../../providers/opencode-zen-rate-limit"; +import { planReasoningEffortDowngrade } from "../../providers/reasoning-metadata"; + +/** Prepares and recovers one native Responses exchange before client commitment. */ +export async function preparePassthroughExchange( + requestContext: Pick, + admissionState: ResponsesAdmissionState, + nativeHostState: PassthroughAdmissionState, + requestState: Pick< + PreparedResponsesRequest, + | "route" + | "toolBridgeMaps" + | "parsed" + | "translatorBudget" + | "responseStateOptions" + | "selectedForwardHeaders" + | "clientRequestedStream" + | "inboundWire" + | "substituteMainCredential" + | "callerAuthHeaders" + | "subagentFallbackAccountId" + >, + transportState: Pick< + ResponsesTransport, + | "adapter" + | "genericFailoverAccountId" + | "passiveQuotaWriterGeneration" + | "oauthDispatch" + | "resolveSelectionAdapter" + | "isOAuth401ReplayProvider" + | "sentOAuthSnapshot" + | "refreshResolvedOAuthSelection" + | "replayOAuthCredentialSnapshot" + | "genericFailovers" + | "applyFailoverSnapshot" + >, + responseEffects: Pick< + ResponsesEffects, + | "refreshRequestToolAliases" + | "routedMuseToolNameAliases" + | "plaintextV2AgentMessageToolNames" + | "routedNamespaceToolAliases" + | "plaintextV2AgentMessageAliasedToolNames" + | "notifyResponseComplete" + >, + sendBudgetState: Pick< + ResponsesSendBudget, + | "remainingTransientSendBudget" + | "noteTransientSends" + | "recoverySendAllowance" + | "recoveryClassFor" + | "sendBudgetExhausted" + | "reserveCredentialHop" + | "pendingHopPermit" + | "workflowRootId" + >, +) { + const { config, logCtx, options, req } = requestContext; + const { + route, + toolBridgeMaps, + parsed, + translatorBudget, + responseStateOptions, + clientRequestedStream, + inboundWire, + substituteMainCredential, + callerAuthHeaders, + } = requestState; + const { + passiveQuotaWriterGeneration, + oauthDispatch, + resolveSelectionAdapter, + isOAuth401ReplayProvider, + refreshResolvedOAuthSelection, + applyFailoverSnapshot, + } = transportState; + const { refreshRequestToolAliases, notifyResponseComplete } = responseEffects; + const { + remainingTransientSendBudget, + noteTransientSends, + recoverySendAllowance, + recoveryClassFor, + sendBudgetExhausted, + reserveCredentialHop, + workflowRootId, + } = sendBudgetState; + + const codexSafetyBufferingOptions = isCanonicalOpenAiForwardProvider(route.provider) + ? codexSafetyBufferingFilterOptions(config) + : undefined; + const imageGenCallAliases = route.provider.authMode === "forward" + ? new Map() + : imageGenToolCallAliases(toolBridgeMaps.toolNsMap, parsed._rawBody, translatorBudget); + const routedCustomToolNames = new Set(); + const routedCustomToolRepairNames = new Set(); + const routedToolSearchNames = new Set(); + // Local continuation cache for the ChatGPT passthrough. Codex WS turns chain with + // previous_response_id, ocx converts them to internal HTTP requests, and the ChatGPT Codex + // REST backend rejects the parameter — the adapter strips it in forward mode, so the ONLY + // way a chained turn keeps its earlier context is the local replay expansion. Record + // completed passthrough responses (force bypasses Codex's blanket store:false) so the next + // turn's expansion hits. Never record a body whose own previous_response_id failed to + // expand: its input is a delta, and storing it would replay a truncated conversation. + // Compaction turns are excluded: _rawBody still carries the full pre-compaction history and + // recording it would let a later expansion rehydrate the chain Codex just replaced. + const passthroughRecordEligible = parsed._compactionRequest !== true + && (!parsed.previousResponseId || parsed._previousResponseInputExpanded === true); + const rememberPassthroughResponse = passthroughRecordEligible + ? (response: { id?: unknown; output?: unknown; status?: unknown }) => + rememberResponseState(parsed._rawBody, response, undefined, responseStateOptions(true)) + : undefined; + if (parsed.previousResponseId && !parsed._previousResponseInputExpanded) { + console.warn( + `[responses] previous_response_id ${parsed.previousResponseId} not found in local replay state ` + + `(model ${parsed.modelId}); forwarding without it — earlier turns may be missing from this request`, + ); + } + // Preserve the caller's readable catalog boundary before provider-specific normalization can + // remove an unsupported final entry (for example xAI cached-only web search). + const replayedInputPrefixLength = parsed._replayPrefixLen ?? 0; + const clientToolAuthorizationBody = currentTurnWireToolCatalogBody( + parsed._rawBody, + replayedInputPrefixLength, + ); + const selfNamedNamespaceScrubAuthorization = collectSelfNamedNamespaceScrubAuthorization( + clientToolAuthorizationBody, + toolBridgeMaps.bareCustomToolNames, + toolBridgeMaps.bareFunctionToolNames, + ); + const clientExplicitWireToolCatalog = hasExplicitWireToolCatalog(clientToolAuthorizationBody); + const clientDeclaredWireToolNames = collectDeclaredWireToolNames(clientToolAuthorizationBody); + const clientDeclaredBareWireToolNames = collectDeclaredBareWireToolNames(clientToolAuthorizationBody); + const clientDeclaredNamelessCallTypes = collectDeclaredNamelessClientCallTypes( + clientToolAuthorizationBody, + ); + // Hosted calls the PROVIDER runs itself. Gated on the destination actually being xAI, so a + // declaration alone cannot buy the exemption on some other upstream that never serves it. + // Provider-executed declarations are authorized from the actual outbound body, after the + // adapter has applied destination-specific injection and normalization. Client-executed tool + // authority remains bounded to the caller-owned catalog above. + const providerExecutedCallTypes = new Set(); + let request: Awaited>; + try { + request = await transportState.adapter.buildRequest(parsed, { headers: requestState.selectedForwardHeaders, translatorBudget }); + } catch (error) { + releaseCodexAuthContextProbeLease(admissionState.authCtx); + // A tool catalog this proxy cannot lower onto one wire namespace is a client input error, and + // the rotation-rebuild and bridged paths already answer 400 for the identical throw. Rethrowing + // it here escaped every catch up to the Bun handler, so the same request produced an + // unstructured 500 — and no request log — depending only on whether a rotation ran first. + // Same shape for a tool_choice this proxy cannot honor: the destination rejects a schema the + // catalog had to drop, so the selector naming it is a client input error, not a 500. + if (error instanceof NamespaceToolCollisionError || error instanceof XaiToolSchemaCompatibilityError) { + return formatErrorResponse(400, "invalid_request_error", redactSecretString(error.message)); + } + throw error; + } + const functionRepairSchemas = isCanonicalOpenAiForwardProvider(route.provider) + ? new Map() + : collectFunctionCallRepairSchemas(clientToolAuthorizationBody); + if (!isCanonicalOpenAiForwardProvider(route.provider)) { + for (const name of request.convertedRoutedCustomToolNames ?? []) { + if ( + toolBridgeMaps.freeformToolNames.has(name) + || toolBridgeMaps.toolNsMap.get(name)?.freeform === true + ) routedCustomToolNames.add(name); + } + for (const name of request.routedCustomToolRepairNames ?? []) { + if ( + toolBridgeMaps.freeformToolNames.has(name) + || toolBridgeMaps.toolNsMap.get(name)?.freeform === true + ) routedCustomToolRepairNames.add(name); + } + } + for (const name of request.convertedRoutedToolSearchNames ?? []) { + // The adapter already keeps this set empty when tool_choice forbids the private search. + // Its wire name may be collision-aliased, so comparing it to the caller-facing name here + // would incorrectly disable restoration for the exact ambiguous-name case the alias fixes. + routedToolSearchNames.add(name); + } + refreshRequestToolAliases(request); + // #1700: the bridged paths refuse a call to a tool the request never declared + // (`declaredToolNames`, src/bridge.ts). The passthrough had no equivalent, so a routed + // provider's top-level `apply_patch` — which under Codex code mode exists only as a nested + // `tools.apply_patch(...)` helper inside `exec`, never as a wire tool — reached Codex as a + // call it cannot execute, and the turn showed a bare `aborted` with the file untouched. + // Forward auth is the canonical ChatGPT backend speaking Codex's own protocol rather than a + // routed provider, so it keeps passing through unguarded, as it does for the rewrites above. + // The guard needs a catalog to compare against, so it stands down when the request omits one. + // An explicit empty catalog is still authoritative: it declares that no client tools may be + // called. A passthrough request can legitimately omit `tools` entirely and still receive a call + // the client understands — `tests/providers/github-copilot/github-copilot-stream-contract.test.ts` sends + // `{model, input, stream}` with no tools and Copilot answers with a `custom_tool_call` for + // `apply_patch`. Policing an absent catalog truncates that turn. An unreadable body lands there + // too because the proxy cannot establish the caller's declared authorization boundary. + const parseOutboundRequestBody = (bodyText: string): Record | undefined => { + try { + const body = JSON.parse(bodyText) as unknown; + return body && typeof body === "object" && !Array.isArray(body) + ? body as Record + : undefined; + } catch { + return undefined; + } + }; + let outboundRequestBody: Record | undefined; + const declaredWireToolNames = new Set(); + const declaredBareWireToolNames = new Set(); + const declaredNamelessClientCallTypes = new Set(); + // `buildToolBridgeMaps` creates a bare alias only when the caller selected exactly one + // namespaced tool through a bare tool_choice. Restore that request-bounded identity before + // authorization checks instead of admitting the bare name into the declared set: for `exec`, + // the latter would also authorize the unrelated code-mode helper names. + const authorizedBareNamespaceToolAliases: RoutedNamespaceToolAliases = new Map( + [...toolBridgeMaps.toolNsMap].flatMap(([alias, identity]) => + alias === identity.name + ? [[alias, { + namespace: identity.namespace, + name: identity.name, + kind: identity.freeform ? "custom" as const : "function" as const, + }] as const] + : [] + ), + ); + const restoreAuthorizedBareNamespaceToolCalls = (value: unknown): unknown => + restoreRoutedNamespaceCalls(value, authorizedBareNamespaceToolAliases).value; + const normalizeFunctionCompletionJson = (text: string): string => { + const snapshot = hasResponsesSnapshotRepair(route.provider.responsesSnapshotRepair) + ? repairResponsesSnapshotJson(text, outboundRequestBody) + : text; + // Sparse gateways need completion status inferred before schema repair can + // distinguish completed arguments from in-progress placeholders. + return repairFunctionCallsInJson(backfillResponsesFieldsJson(snapshot), functionRepairSchemas); + }; + let undeclaredToolGuardActive = false; + const refreshUndeclaredToolGuard = (builtRequest: AdapterRequest): void => { + outboundRequestBody = parseOutboundRequestBody(builtRequest.body); + providerExecutedCallTypes.clear(); + if (isXaiResponsesDestination(route.provider)) { + // Preserve the caller-declared authorization recognized by the original classifier, then + // add adapter-injected declarations from the actual current-turn outbound catalog. + for (const callType of collectProviderExecutedCallTypes(clientToolAuthorizationBody)) { + providerExecutedCallTypes.add(callType); + } + const currentOutboundCatalog = currentTurnWireToolCatalogBody( + outboundRequestBody, + replayedInputPrefixLength, + ); + for (const callType of collectProviderExecutedCallTypes(currentOutboundCatalog)) { + providerExecutedCallTypes.add(callType); + } + } + declaredWireToolNames.clear(); + // With no replay prefix the full outbound body belongs to this turn and its normalized + // aliases are authoritative. A continuation's outbound body still contains historical + // catalogs (and may promote historical tool-search definitions), so it can never widen the + // current caller snapshot captured above. + declaredBareWireToolNames.clear(); + if (replayedInputPrefixLength === 0) { + for (const name of collectDeclaredWireToolNames(outboundRequestBody)) { + declaredWireToolNames.add(name); + } + for (const name of collectDeclaredBareWireToolNames(outboundRequestBody)) { + declaredBareWireToolNames.add(name); + } + } + for (const name of clientDeclaredWireToolNames) declaredWireToolNames.add(name); + for (const name of clientDeclaredBareWireToolNames) declaredBareWireToolNames.add(name); + declaredNamelessClientCallTypes.clear(); + if (replayedInputPrefixLength === 0) { + for (const callType of collectDeclaredNamelessClientCallTypes(outboundRequestBody)) { + declaredNamelessClientCallTypes.add(callType); + } + } + for (const callType of clientDeclaredNamelessCallTypes) { + declaredNamelessClientCallTypes.add(callType); + } + // On an ordinary request these maps capture caller-catalog identities that normalization may + // replace on the outbound wire (for example a client image tool becoming hosted). On replay, + // however, the parsed maps also contain historical catalog entries, so only the bounded + // current-turn wire snapshot above may authorize a call. + if (replayedInputPrefixLength === 0) { + for (const name of toolBridgeMaps.declaredToolNames) { + // `buildToolBridgeMaps` also aliases a namespaced tool under its bare name when the + // caller's `tool_choice` selected it unambiguously, which the bridge needs to route the + // call back. For `exec` alone that alias would also switch on nested-helper + // normalization and re-authorize `exec_command`/`shell_command`/`apply_patch`/`view_image`, so it is + // admitted here only when the caller's own catalog declared a bare `exec`. Selecting an + // MCP `exec` is not a declaration of the code-mode shell tool. + if ( + name === CODE_MODE_EXEC_TOOL_NAME + && !clientDeclaredWireToolNames.has(CODE_MODE_EXEC_TOOL_NAME) + ) continue; + declaredWireToolNames.add(name); + } + } + undeclaredToolGuardActive = ( + declaredWireToolNames.size > 0 + || clientDeclaredNamelessCallTypes.size > 0 + || clientExplicitWireToolCatalog + ) && route.provider.authMode !== "forward"; + }; + refreshUndeclaredToolGuard(request); + // A refused turn must not seed `previous_response_id` replay. The inspection branch reads the + // untouched upstream stream, so it can still observe a `response.completed` the client never + // received; checking the payload itself rather than a flag shared with the client relay keeps + // this free of tee ordering races. + // + // Checking only the terminal snapshot is not enough. An upstream can announce the undeclared + // call in `response.output_item.added`, which trips the client guard, and then close with a + // `response.completed` whose `output` is empty. The client gets `response.failed`, the terminal + // check sees nothing undeclared, and the refused turn enters continuation state anyway. So the + // rejection is sticky for the whole turn, set from every parsed payload on the inspection side. + let inspectionSawUndeclaredTool = false; + let inspectedTerminal: ResponsesTerminalStatus | null = null; + let inspectedCompletionSeen = false; + let firstTerminalAllowsRecall = false; + const passiveQuotaObserved = hasPassiveAccountQuota(route.providerName) + && route.provider.authMode === "oauth"; + const noteInspectedPayload = (payload: unknown) => { + // First terminal stays authoritative even in metadata-only inspection, which + // intentionally continues parsing after a failed/incomplete terminal. + const terminal = terminalStatusFromParsed(payload); + if (inspectedTerminal === null && terminal !== null) { + inspectedTerminal = terminal; + // The client boundary accepts a terminal by event type, even without a + // response object. Such a terminal must permanently decline recall. + if (terminal === "completed" && payload && typeof payload === "object" + && "response" in payload && payload.response && typeof payload.response === "object" + && !Array.isArray(payload.response) && "model" in payload.response) { + firstTerminalAllowsRecall = typeof payload.response.model === "string" + && payload.response.model.trim().length > 0; + } + } + // Meta reports subscription usage ONLY as an in-stream event; there is no endpoint + // to poll (003 §E probed 17 paths, all 404). Observed here rather than behind a + // dedicated inspector handler because onParsedPayload already reaches every + // passthrough shape -- eager relay and both tee consumers -- through this one + // function. + // + // Placed BEFORE the undeclared-tool early return below, which is load-bearing: that + // guard latches for the rest of the turn once it fires, and a turn that tripped it + // still legitimately reports usage. + if (passiveQuotaObserved && isMuseSubscriptionUsagePayload(payload)) { + const quota = parseMuseSubscriptionUsage(payload); + // Read at EVENT time, not at handler construction: failover rebinds this, and the + // quota belongs to the account that actually served the turn. + const servingAccountId = transportState.genericFailoverAccountId; + if (quota && servingAccountId) { + recordPassiveAccountQuota(route.providerName, servingAccountId, quota, passiveQuotaWriterGeneration); + } + } + // Gated on the same flag as the guard itself: with no readable catalog (or a forward-auth + // provider) every name looks undeclared, and flipping this would stop recording continuation + // state for exactly the passthrough traffic the guard deliberately stands down for. + if (undeclaredToolGuardActive && !inspectionSawUndeclaredTool && undeclaredToolCallName( + restoreAuthorizedBareNamespaceToolCalls( + restoreMuseToolNames(payload, responseEffects.routedMuseToolNameAliases).value, + ), + declaredWireToolNames, + declaredNamelessClientCallTypes, + providerExecutedCallTypes, + declaredBareWireToolNames, + ) !== undefined) { + inspectionSawUndeclaredTool = true; + } + // The snapshot callback opts the inspector into output reconstruction. Compaction + // has no continuation cache, so use the parsed terminal here without adding retention. + if (responseEffects.plaintextV2AgentMessageToolNames.size === 0 && !rememberPassthroughResponse && payload && typeof payload === "object" + && "type" in payload && payload.type === "response.completed" + && "response" in payload && payload.response && typeof payload.response === "object" + && !Array.isArray(payload.response)) { + rememberPassthroughResponseChecked(payload.response as Record); + } + }; + const rememberPassthroughResponseChecked = ( + response: { id?: unknown; output?: unknown; status?: unknown; model?: unknown }, + ) => { + if (inspectionSawUndeclaredTool) return; + const restored = restoreRoutedCustomCalls( + restoreAuthorizedBareNamespaceToolCalls( + restoreRoutedNamespaceCalls( + restoreMuseToolNames(response, responseEffects.routedMuseToolNameAliases).value, + responseEffects.routedNamespaceToolAliases, + ).value, + ), + routedCustomToolNames, + routedCustomToolRepairNames, + declaredWireToolNames, + ).value; + const normalizedResponse = (functionRepairSchemas.size > 0 + ? JSON.parse(normalizeFunctionCompletionJson(JSON.stringify(restored))) + : restored) as { id?: unknown; output?: unknown; status?: unknown }; + const plaintextRestore = restorePlaintextV2AgentMessageCalls( + normalizedResponse, responseEffects.plaintextV2AgentMessageToolNames, responseEffects.plaintextV2AgentMessageAliasedToolNames, + ); + if (plaintextRestore.overflowed) return; + const restoredResponse = plaintextRestore.value as typeof normalizedResponse; + // Replay overlap compares the items the client echoes, including visible reasoning shape. + const replayResponse = restoredResponse; + if ( + undeclaredToolGuardActive + && undeclaredToolCallNameInResponse( + restoredResponse, + declaredWireToolNames, + declaredNamelessClientCallTypes, + providerExecutedCallTypes, + declaredBareWireToolNames, + ) !== undefined + ) { + return; + } + const normalizedReplayResponse = (undeclaredToolGuardActive + ? normalizeDefaultNamespaceInResponse( + replayResponse, + declaredWireToolNames, + declaredBareWireToolNames, + ).value + : replayResponse) as typeof replayResponse; + rememberPassthroughResponse?.(normalizedReplayResponse); + const firstCompletion = !inspectedCompletionSeen; + inspectedCompletionSeen = true; + if (firstCompletion && (inspectedTerminal === null || firstTerminalAllowsRecall)) { + // A model-less first completion permanently declines recall; later terminal + // frames are hidden by the client boundary and cannot supply its identity. + // Native inspection sees the pre-rewrite model. Only an actual terminal + // model can seed recall; an absent model never falls back to the pick. + if (typeof response.model === "string" && response.model.trim()) { + notifyResponseComplete({ + status: response.status, + model: parsed._responseModelId !== undefined && parsed._responseModelId !== parsed.modelId + ? parsed._responseModelId : response.model, + }); + } + } + }; + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + const actualHostKey = upstreamHostHealthKey( + route.providerName, + safeOriginLabel(request.url), + ); + const hostKey = route.provider.authMode === "forward" + ? actualHostKey + : null; + const hostCircuitEnabled = hostKey !== null + && normalizeUpstreamHostCircuitThreshold(config.upstreamHostCircuitThreshold) > 0; + if (hostKey !== null && !hostCircuitEnabled) { + disableUpstreamHostCircuitForKey(actualHostKey); + } + if (nativeHostState.lease && nativeHostState.lease.key !== hostKey) { + return formatErrorResponse(502, "upstream_error", "Provider host changed after circuit admission"); + } + if (options.abortSignal?.aborted) { + releaseCodexAuthContextProbeLease(admissionState.authCtx); + return clientCancelledResponse(); + } + if (!nativeHostState.lease && hostCircuitEnabled) { + const admission = acquireUpstreamHostAdmission( + hostKey!, + config.upstreamHostCircuitThreshold, + ); + if (admission.kind === "blocked") { + releaseCodexAuthContextProbeLease(admissionState.authCtx); + return upstreamHostCircuitOpenResponse(admission.retryAfterSeconds); + } + nativeHostState.lease = admission.lease; + } + const settleObservedHostResponse = (): void => { + if (hostCircuitEnabled) { + resetUpstreamHostHealth(actualHostKey, nativeHostState.lease); + } else { + resetUpstreamHostHealth(actualHostKey); + } + nativeHostState.lease = null; + }; + /** + * #4191: a Codex WS exchange pins its content-free stage record on the + * Response it resolves (markCodexWsStage). Adopting the record here, at + * the single funnel every physical upstream response passes through, + * binds it to the attempt that actually served it — including the 502/504 + * pre-response JSON settles that never reach the SSE relay. + */ + const adoptCodexWsStage = (response: Response): void => { + const stage = readCodexWsStage(response); + if (stage && logCtx.activeAttempt) logCtx.activeAttempt.codexWsStage = stage; + }; + const adoptObservedResponse = (response: T): T => { + settleObservedHostResponse(); + adoptCodexWsStage(response); + return response; + }; + let passthroughEstimate = typeof request.usageLog?.inputTokens === "number" + ? request.usageLog.inputTokens + : undefined; + if (passthroughEstimate !== undefined) { + logCtx.usageLogInputTokens = passthroughEstimate; + } + // Abort the upstream if the client disconnects. A directly-relayed body does not propagate the + // consumer's cancel to a signalled fetch, so we pass the signal and relay through relayWithAbort, + // whose cancel() aborts the upstream — preventing leaked connections (RC2, passthrough path). + const upstream = new AbortController(); + linkAbortSignal(upstream, options.abortSignal); + const connectMs = config.connectTimeoutMs ?? 200_000; + let upstreamResponse: Response; + /** + * Refuse a built body that exceeds the operator's configured ceiling, before it is sent. + * + * Unconfigured this measures nothing and returns undefined, so an unset proxy behaves + * exactly as it does today. Runs at every point a body is built or rebuilt, because a + * rebuild can produce a payload the initial check never saw. + */ + const refuseOversizedOutboundBody = ( + builtRequest: AdapterRequest, + refusalAuthCtx: CodexAuthContext = admissionState.authCtx, + ): Response | undefined => { + const result = checkOutboundBodySize(builtRequest.body, config.maxUpstreamBodyBytes); + if (result.admitted) return undefined; + + // This returns before the surrounding fetch/finally owns the observation, so release + // it here or one refused body holds translator budget for the process lifetime. + builtRequest.releaseBodyObservation?.(); + upstream.abort(); + releaseUpstreamHostAdmission(nativeHostState.lease); + nativeHostState.lease = null; + releaseCodexAuthContextProbeLease(refusalAuthCtx); + logCtx.errorCode = "outbound_body_too_large"; + console.warn( + `[responses] refused an oversized outbound body: bytes=${result.bytes} limit=${result.limit} ` + + `input_images=${result.imageCount} image_bytes=${result.imageBytes} ` + + `model=${JSON.stringify(parsed.modelId)}`, + ); + // A streaming client treats HTTP 413 as a retryable transport error and resends the same + // oversized body — the reconnect loop #3177 exists to stop. Terminal overflow is the + // honest shape, and it is what the upstream-413 path already returns. + if (clientRequestedStream) { + return streamingContextOverflowResponse( + parsed._responseModelId ?? parsed.modelId, + translatorBudget, + ); + } + return formatErrorResponse( + 413, + "outbound_body_too_large", + describeOutboundBodyRefusal(result), + ); + }; + const transportFailureResponse = (err: unknown): Response => { + upstream.abort(); + if (options.abortSignal?.aborted) { + releaseUpstreamHostAdmission(nativeHostState.lease); + nativeHostState.lease = null; + releaseCodexAuthContextProbeLease(admissionState.authCtx); + return clientCancelledResponse(); + } + // A budget refusal is a proxy decision, not an upstream fault. Reporting it as + // 502 upstream_error would blame the provider for a limit this process applied, and + // would record a fake reachability failure against the account's health. + if (err instanceof SendBudgetExhaustedError) { + releaseUpstreamHostAdmission(nativeHostState.lease); + nativeHostState.lease = null; + releaseCodexAuthContextProbeLease(admissionState.authCtx); + return formatErrorResponse(429, "request_send_budget_exhausted", err.message); + } + const localRefusal = mapCodexAuthContextErrorToResponse(unwrapUpstreamRetryEvidenceError(err), { + now: Date.now(), accountSelector: route.codexAccountNamespace, + }); + if (localRefusal) { + releaseUpstreamHostAdmission(nativeHostState.lease); + nativeHostState.lease = null; + releaseCodexAuthContextProbeLease(admissionState.authCtx); + return localRefusal; + } + const outcome = classifyTransportFailureKind(err); + // Host-level evidence stands regardless of pool membership: a direct + // forward send has no pool accounting, but the reachability failure is + // still host-wide, not account evidence (#914 review). + if (outcome === "connect_neutral") { + if (hostCircuitEnabled) { + recordUpstreamHostFailure(actualHostKey, { + code: transportErrorCode(err), + threshold: config.upstreamHostCircuitThreshold, + lease: nativeHostState.lease, + }); + } else { + recordUpstreamHostFailure(actualHostKey, { code: transportErrorCode(err) }); + } + nativeHostState.lease = null; + } else { + releaseUpstreamHostAdmission(nativeHostState.lease); + nativeHostState.lease = null; + } + if (usesCodexForwardPoolAuth(admissionState.authCtx, route.provider)) { + recordCodexUpstreamOutcome(config, admissionState.authCtx.accountId, outcome, { + threadId: admissionState.authCtx.affinityKey, + fixedAccount: admissionState.authCtx.fixedAccount, + modelId: route.modelId, + probeLeaseId: codexProbeLeaseId(admissionState.authCtx), + probeQuotaScope: codexProbeQuotaScope(admissionState.authCtx), + writerGeneration: admissionState.authCtx.writerGeneration, + }); + } + const msg = outcome === "timeout" + ? `Provider connect timeout after ${connectMs}ms` + : describeUpstreamConnectFailure(err, connectMs); + return formatErrorResponse(502, "upstream_error", msg); + }; + const initialBodyRefusal = refuseOversizedOutboundBody(request); + if (initialBodyRefusal) return initialBodyRefusal; + try { + // Transient-5xx pre-stream retry (devlog/_plan/260716_claudecode_hardening/010): + // the ChatGPT backend emits transient 502/520s that an immediate retry absorbs. + // Body is a replayable string; nothing has streamed to the client yet. + upstreamResponse = await fetchWithTransientRetry( + recovery => { + noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, recovery); + return fetchWithHeaderTimeout(request.url, applyUpstreamRecoveryInit({ + method: request.method, + headers: request.headers, + body: request.body, + }, recovery), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(request), + providerName: route.providerName, + modelId: route.modelId, + onCodexWsQuota: codexWsQuotaObserver(admissionState.authCtx, route.provider, route.modelId), + beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) + ? createCodexReserveDispatchGuard(admissionState.authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, + }), + route.provider.authMode === "forward") + // Every real attempt response — including an intermediate 5xx the + // retry wrapper replaces — proves the host was reached (#914 review). + .then(adoptObservedResponse); + }, + { abortSignal: upstream.signal, label: safeHostLabel(request.url), attempts: remainingTransientSendBudget(TRANSIENT_RETRY_MAX_ATTEMPTS), onSendsConsumed: noteTransientSends }, + ); + } catch (err) { + return transportFailureResponse(err); + } finally { + request.releaseBodyObservation?.(); + } + + const opaqueBlobRecoveryGuard: OpaqueBlobRecoveryGuard = { attempted: false }; + // At most one reasoning-effort downgrade per request. + const reasoningEffortDowngradeGuard: { attempted: boolean } = { attempted: false }; + let oauth401ReplayAttempted = false; + let codex401ReplayKind: "main" | "stored" | null = null; + // Console Go answers a transient 400 "Invalid upload request." for bodies it accepts + // moments later; at most one byte-identical replay is allowed per request. + const consoleGoUploadRetryGuard: { attempted: boolean } = { attempted: false }; + const rateLimitPolicy = rateLimitRetryPolicyFor(route.provider); + let rateLimitRetries = 0; + const rebuildAndRefetch = async ( + recovery: AttemptRecoveryKind, + ): Promise => { + const retryAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + if (!("passthrough" in retryAdapter) || !retryAdapter.passthrough) { + upstream.abort(); + return { failed: formatErrorResponse(502, "upstream_error", "Recovery changed the provider wire unexpectedly") }; + } + try { + if (recovery !== "console-go-upload-retry") { + request = await retryAdapter.buildRequest(parsed, { + headers: requestState.selectedForwardHeaders, + translatorBudget, + }); + } + refreshRequestToolAliases(request); + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + } catch (err) { + upstream.abort(); + if (options.abortSignal?.aborted) return { failed: clientCancelledResponse() }; + const msg = err instanceof Error ? err.message : String(err); + return { failed: formatErrorResponse(400, "invalid_request_error", redactSecretString(msg)) }; + } + passthroughEstimate = typeof request.usageLog?.inputTokens === "number" + ? request.usageLog.inputTokens + : undefined; + if (passthroughEstimate !== undefined) logCtx.usageLogInputTokens = passthroughEstimate; + refreshUndeclaredToolGuard(request); + logCtx.providerAdapter = retryAdapter.name; + sealRequestAttemptIdentity( + logCtx.activeAttempt, + logCtx.provider, + retryAdapter.name, + logCtx.accountLogLabel, + ); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, retryAdapter.name); + const rebuiltBodyRefusal = refuseOversizedOutboundBody(request); + if (rebuiltBodyRefusal) return { failed: rebuiltBodyRefusal }; + // The base allowance is spent first; once it is gone this leg may still draw the one + // shared final-recovery reserve, which is what keeps a validated sanitized rebuild + // after a 5xx streak alive at four total sends instead of dying at three. Reserved + // outside the try so the finally can hand it back if the leg never reached its send. + const allowance = recoverySendAllowance( + TRANSIENT_RETRY_MAX_ATTEMPTS, + recoveryClassFor(recovery), + `${route.providerName}|${route.modelId}|${recovery}`, + ); + try { + return await fetchWithTransientRetry( + innerRecovery => { + // Gated on the return, not fire-and-forget: a consumed permit means this leg + // already sent once, and letting the second call through would be a free send. + if (allowance.permit && !allowance.permit.use()) { + throw new SendBudgetExhaustedError(safeHostLabel(request.url)); + } + noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, innerRecovery ?? recovery); + return fetchWithHeaderTimeout(request.url, applyUpstreamRecoveryInit({ + method: request.method, + headers: request.headers, + body: request.body, + }, innerRecovery), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(request), + providerName: route.providerName, + modelId: route.modelId, + onCodexWsQuota: codexWsQuotaObserver(admissionState.authCtx, route.provider, route.modelId), + beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) + ? createCodexReserveDispatchGuard(admissionState.authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, + }), + route.provider.authMode === "forward") + .then(adoptObservedResponse); + }, + { abortSignal: upstream.signal, label: safeHostLabel(request.url), attempts: allowance.attempts, onSendsConsumed: noteTransientSends }, + ); + } catch (err) { + return { failed: transportFailureResponse(err) }; + } finally { + // A no-op once the permit was used or once onSendsConsumed settled it; it only refunds + // a reservation whose send never happened. + allowance.permit?.release(); + request.releaseBodyObservation?.(); + } + }; + + // Keep recovery kinds in sync with the generic `recovery:` loop below. + passthroughRecovery: for (;;) { + + if ( + upstreamResponse.status === 401 + && (admissionState.authCtx.kind === "main-pool" || admissionState.authCtx.kind === "pool") + && usesCodexForwardPoolAuth(admissionState.authCtx, route.provider) + && codex401ReplayKind === null + ) { + codex401ReplayKind = admissionState.authCtx.kind === "pool" ? "stored" : "main"; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed */ } + const poolAuthCtx = admissionState.authCtx.kind === "pool" ? admissionState.authCtx : undefined; + const poolReplay = poolAuthCtx + ? await refreshPoolForwardAuth({ req, config, route, authCtx: poolAuthCtx, substituteMainCredential, options, logCtx }) + : undefined; + const replay = poolReplay + ?? await refreshNativeMainForwardAuth({ req, config, route, authCtx: admissionState.authCtx, substituteMainCredential, options }); + if (!replay.ok) { + // Compact already records this; core historically returned without recording, + // so a dead grant stayed selectable and every request repeated the same doomed + // refresh. Fenced by the generation the 401 belongs to (#2887). + if (poolAuthCtx && poolReplay && !poolReplay.ok && poolReplay.quarantine) { + recordCodexUpstreamOutcome(config, poolAuthCtx.accountId, 401, { + threadId: poolAuthCtx.affinityKey, + fixedAccount: poolAuthCtx.fixedAccount, + modelId: route.modelId, + writerGeneration: poolAuthCtx.writerGeneration, + credentialGeneration: poolReplay.quarantineGeneration ?? poolAuthCtx.generation, + }); + } + upstream.abort(); + releaseCodexAuthContextProbeLease(admissionState.authCtx); + return replay.response; + } + admissionState.authCtx = replay.authCtx; + route.provider = replay.provider; + requestState.selectedForwardHeaders = withClaudeNativeSession(replay.headers, replay.provider, options.claudeNativeSessionId); + const replayAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, replay.provider, inboundWire), + config.cacheRetention, + ); + if (!("passthrough" in replayAdapter) || !replayAdapter.passthrough) { + upstream.abort(); + return formatErrorResponse(502, "upstream_error", "Native main refresh changed the provider wire unexpectedly"); + } + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: replay.provider, + adapterName: replayAdapter.name, + codexAuthContext: admissionState.authCtx, + forwardHeaders: requestState.selectedForwardHeaders, + }); + logCtx.providerAdapter = replayAdapter.name; + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, replayAdapter.name, logCtx.accountLogLabel); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, replayAdapter.name); + try { + request = await replayAdapter.buildRequest(parsed, { + headers: requestState.selectedForwardHeaders, + translatorBudget, + }); + refreshRequestToolAliases(request); + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + refreshUndeclaredToolGuard(request); + // The 401 replay rebuilds the body before sending, so it needs the same ceiling as + // every other build site; a replay is exactly when a grown payload reappears. + const replayBodyRefusal = refuseOversizedOutboundBody(request); + if (replayBodyRefusal) return replayBodyRefusal; + noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, "oauth-401"); + upstreamResponse = await fetchWithHeaderTimeout( + request.url, + { method: request.method, headers: request.headers, body: request.body }, + upstream.signal, + connectMs, + parsed.stream, + // The replay-dispatched signal is what bounds the rest of this logical request, so it + // has to describe a send that actually happened. fetchWithHeaderTimeout awaits pacing + // admission BEFORE calling the executor, so signalling at the call site would spend the + // budget even when a rejected pacing wait means nothing reaches the network. Wrapping + // the executor moves the signal to the last moment before the send, where a throw from + // here on is a genuine transport attempt. + storedPoolReplayDispatchNotifier( + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(request), + providerName: route.providerName, + modelId: route.modelId, + onCodexWsQuota: codexWsQuotaObserver(admissionState.authCtx, route.provider, route.modelId), + beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) + ? createCodexReserveDispatchGuard(admissionState.authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, + }), + codex401ReplayKind === "stored" ? options.onStoredPool401ReplayDispatched : undefined, + ), + route.provider.authMode === "forward", + ).then(adoptObservedResponse); + } catch (err) { + return transportFailureResponse(err); + } finally { + request.releaseBodyObservation?.(); + } + continue passthroughRecovery; + } + + if (codex401ReplayKind !== null && upstreamResponse.status === 401) break; + + // Native Responses providers return before the generic adapter recovery loop below. Keep + // their OAuth contract identical: one pre-stream 401 forces a credential refresh and one + // rebuilt replay. xAI's current subscription models use this branch now that their official + // Grok CLI catalog declares the Responses backend. + if ( + upstreamResponse.status === 401 + && isOAuth401ReplayProvider + && transportState.sentOAuthSnapshot + && !oauth401ReplayAttempted + // Refused here, before the 401 body is cancelled: once it is gone the request can only + // answer with a synthetic 502, which would report a proxy budget decision as an upstream + // fault and throw away the credential evidence the client needs. + && !sendBudgetExhausted() + ) { + oauth401ReplayAttempted = true; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + let refreshed: OAuthAccessSnapshot; + try { + refreshed = await refreshResolvedOAuthSelection(transportState.sentOAuthSnapshot); + } catch (err) { + upstream.abort(); + releaseCodexAuthContextProbeLease(admissionState.authCtx); + return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(err)); + } + if (route.provider.googleMode === "cloud-code-assist" && !refreshed.projectId) { + upstream.abort(); + releaseCodexAuthContextProbeLease(admissionState.authCtx); + return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(new Error("Cloud Code Assist project is required"))); + } + transportState.sentOAuthSnapshot = refreshed; + transportState.replayOAuthCredentialSnapshot = { + accountId: refreshed.accountId, + generation: refreshed.generation, + }; + if (route.providerName === "kiro") { + parsed._kiroAuthContext = { ...(refreshed.kiro ?? {}) }; + } + const refreshedProvider = resolveProviderTransport( + route.providerName, + { + ...route.provider, + apiKey: refreshed.accessToken, + ...(refreshed.projectId ? { project: refreshed.projectId } : {}), + }, + parsed.options.promptCacheKey, + route.providerName === "github-copilot" + ? resolveCopilotApiBaseUrl(refreshed.apiBaseUrl) + : undefined, + ); + route.provider = refreshedProvider; + const refreshedAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, refreshedProvider, inboundWire), + config.cacheRetention, + ); + if (!("passthrough" in refreshedAdapter) || !refreshedAdapter.passthrough) { + upstream.abort(); + return formatErrorResponse(502, "upstream_error", "OAuth refresh changed the provider wire unexpectedly"); + } + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: refreshedProvider, + adapterName: refreshedAdapter.name, + oauthCredentialSnapshot: transportState.replayOAuthCredentialSnapshot, + }); + logCtx.providerAdapter = refreshedAdapter.name; + sealRequestAttemptIdentity( + logCtx.activeAttempt, + logCtx.provider, + refreshedAdapter.name, + logCtx.accountLogLabel, + ); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, refreshedAdapter.name); + try { + request = await refreshedAdapter.buildRequest(parsed, { + headers: requestState.selectedForwardHeaders, + translatorBudget, + }); + refreshRequestToolAliases(request); + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + } catch (err) { + upstream.abort(); + if (options.abortSignal?.aborted) return clientCancelledResponse(); + const msg = err instanceof Error ? err.message : String(err); + return formatErrorResponse(400, "invalid_request_error", redactSecretString(msg)); + } + refreshUndeclaredToolGuard(request); + const refreshedBodyRefusal = refuseOversizedOutboundBody(request); + if (refreshedBodyRefusal) return refreshedBodyRefusal; + try { + upstreamResponse = await fetchWithTransientRetry( + recovery => { + noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, recovery ?? "oauth-401"); + return fetchWithHeaderTimeout(request.url, applyUpstreamRecoveryInit({ + method: request.method, + headers: request.headers, + body: request.body, + }, recovery), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(request), + providerName: route.providerName, + modelId: route.modelId, + onCodexWsQuota: codexWsQuotaObserver(admissionState.authCtx, route.provider, route.modelId), + beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) + ? createCodexReserveDispatchGuard(admissionState.authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, + }), + route.provider.authMode === "forward") + .then(adoptObservedResponse); + }, + { abortSignal: upstream.signal, label: safeHostLabel(request.url), attempts: remainingTransientSendBudget(TRANSIENT_RETRY_MAX_ATTEMPTS), onSendsConsumed: noteTransientSends }, + ); + } catch (err) { + return transportFailureResponse(err); + } finally { + request.releaseBodyObservation?.(); + } + } + + // Native Responses returns before the generic adapter's OAuth rotation loop. Keep + // the same quorum, cooldown and request budget here, before any client bytes flow. + if ( + upstreamResponse.status === 429 + && transportState.genericFailoverAccountId + && transportState.genericFailovers < GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST + && isGenericOAuthFailoverEnabled(config, route.providerName) + ) { + // The roster cap above is one half of the bound; the request's shared budget is the + // other. A refused hop leaves the real 429 -- body, Retry-After and any quota evidence + // -- exactly as upstream sent it. + const hop = reserveCredentialHop( + "auth-recovery", + `${route.providerName}|${route.modelId}|oauth-account-429`, + true, + ); + if (hop.allowed) { + const nextAccountId = rotateGenericOAuthAccountOn429( + config, route.providerName, transportState.genericFailoverAccountId, + upstreamResponse.headers.get("retry-after"), + ); + let snapshot: OAuthAccessSnapshot | undefined; + if (nextAccountId) { + try { snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); } + catch { /* Keep the original 429 body readable when the next credential is unavailable. */ } + } + if (snapshot && await applyFailoverSnapshot(snapshot)) { + transportState.genericFailovers += 1; + route.provider = resolveProviderTransport( + route.providerName, route.provider, parsed.options.promptCacheKey, transportState.sentOAuthSnapshot?.apiBaseUrl, + ); + bindRouteReasoningReplayScope({ + parsed, providerName: route.providerName, provider: route.provider, + adapterName: "openai-responses", oauthCredentialSnapshot: transportState.replayOAuthCredentialSnapshot, + }); + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already closed */ } + // The replay IS this hop's send, so the rebuild spends the reservation instead of + // asking for one of its own. + sendBudgetState.pendingHopPermit = hop.permit; + const result = await rebuildAndRefetch("oauth-account-429"); + sendBudgetState.pendingHopPermit = undefined; + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue passthroughRecovery; + } + // No credential moved, so the reservation costs nothing. + hop.permit?.release(); + } + } + + // Same-target 429 wait-and-retry (opt-in `retryOn429`) for key-auth providers on the + // passthrough wire. This branch returns before the recovery loop below, so Responses-shaped + // key-auth gateways (e.g. the built-in DeepSeek preset) would otherwise surface 429 + // immediately with no same-key replay. Pre-stream only — nothing has been relayed yet, so + // the replay is lossless (same invariant as the recovery loop). Forward/OAuth providers + // keep their pool logic below (rateLimitRetryPolicyFor returns null for them). + while ( + upstreamResponse.status === 429 + && rateLimitPolicy !== null + && rateLimitRetries < rateLimitPolicy.attempts + // Checked here rather than inside the helper: prepareSameTarget429Wait releases the 429 + // body, so a refusal discovered after the wait can no longer return the real rate-limit + // answer and would surface a synthetic 502 instead. + && !sendBudgetExhausted() + ) { + rateLimitRetries += 1; + // Release unread body + deliberate wait via the shared same-target helper. + const retryAfterHeader = upstreamResponse.headers.get("retry-after"); + try { + for await (const _ of prepareSameTarget429Wait({ + body: upstreamResponse.body, + signal: options.abortSignal, + delayMs: rateLimitRetryDelayMs(rateLimitPolicy, retryAfterHeader, Date.now()), + })) { + // pre-stream: no stall watchdog to feed + } + } catch { + upstream.abort(); + return clientCancelledResponse(); + } + // Client cancellation wins over any stale timer edge: re-check before dispatching the + // replay so the wire never starts work for a request the client already abandoned. + if (options.abortSignal?.aborted || upstream.signal.aborted) { + upstream.abort(); + return clientCancelledResponse(); + } + try { + upstreamResponse = await fetchWithTransientRetry( + recovery => { + // The first send of every replay is itself a rate-limit retry; inner transient-5xx + // recoveries keep their own label (recovery is provided for those). + noteAttemptSend(logCtx.activeAttempt, passthroughEstimate, recovery ?? "rate-limit-429"); + return fetchWithHeaderTimeout(request.url, applyUpstreamRecoveryInit({ + method: request.method, + headers: request.headers, + body: request.body, + }, recovery), upstream.signal, connectMs, parsed.stream, + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(request), + providerName: route.providerName, + modelId: route.modelId, + onCodexWsQuota: codexWsQuotaObserver(admissionState.authCtx, route.provider, route.modelId), + beforeDispatch: isCanonicalOpenAiForwardProvider(route.provider) + ? createCodexReserveDispatchGuard(admissionState.authCtx, options.codexAuthPolicy ?? config, route.modelId, options.admission, options.visionDescribeTerminal === true) : undefined, + }), + route.provider.authMode === "forward") + .then(adoptObservedResponse); + }, + { abortSignal: upstream.signal, label: safeHostLabel(request.url), attempts: remainingTransientSendBudget(TRANSIENT_RETRY_MAX_ATTEMPTS), onSendsConsumed: noteTransientSends }, + ); + } catch (err) { + return transportFailureResponse(err); + } + } + + const captureAffinityResponse = ( + response: Response, + captureAuthCtx: CodexAuthContext = admissionState.authCtx, + captureRequest: Awaited> = request, + credentialSubstituted = substituteMainCredential + || captureAuthCtx.kind === "pool" + || captureAuthCtx.kind === "main-pool", + ): void => { + if (!isCanonicalOpenAiForwardProvider(route.provider)) return; + captureCodexAffinityDiagnostic({ + inboundHeaders: req.headers, + outboundHeaders: captureRequest.headers, + authKind: captureAuthCtx.kind, + accountMode: route.codexAccountMode, + fixedAccount: isFixedCodexAccount(captureAuthCtx), + credentialSubstituted, + accountGatedModel: ACCOUNT_GATED_NATIVE_OPENAI_MODELS.has(route.modelId), + wireModelNormalized: parsed.modelId !== route.modelId, + status: response.status, + }); + }; + captureAffinityResponse(upstreamResponse); + + if (usesCodexForwardPoolAuth(admissionState.authCtx, route.provider)) { + let poolRetryOutcome: number | undefined; + if (await shouldRetryCodexPoolAccountModel400( + upstreamResponse, + route.modelId, + options.abortSignal, + )) { + poolRetryOutcome = 400; + } else if (!admissionState.authCtx.fixedAccount && await shouldRetryCodexPoolAccountQuota( + upstreamResponse, + options.abortSignal, + )) { + // Pre-stream only: once SSE has begun, mid-stream quota stays terminal. + // ChatGPT sometimes wraps quota exhaustion in a generic 5xx. Normalize only + // body-confirmed cases to quota evidence so cooldown and rotation both apply. + poolRetryOutcome = upstreamResponse.status >= 500 ? 429 : upstreamResponse.status; + } else if (!admissionState.authCtx.fixedAccount && shouldRetryCodexPoolAccountTransient(upstreamResponse)) { + // A plain transient 5xx the same-account retry layer could not absorb. Keep the real + // status so it records as transient rather than quota. + poolRetryOutcome = upstreamResponse.status; + } + + if (poolRetryOutcome !== undefined) { + // A stored Pool 401 spent this request's account budget on its own refresh and replay, so + // nothing afterwards may be paid for out of a DIFFERENT account. One flag carries that, + // rather than a status check here as well: a quota failure has no same-account move, so + // `sameAccountOnly` makes it terminal by refusing the alternate; the gated-model 400 + // ladder does have one — retrying the account the refreshed roster still grants — and + // keeps it. An earlier revision also broke here on a non-400 outcome, which no test could + // justify because this flag already produced the identical result. + const storedReplaySpent = codex401ReplayKind === "stored"; + const retry = await retryCodexPoolOnAlternateAccount({ + callerAuthHeaders, + config, + route, + parsed, + logCtx, + options: { ...options, workflowRootId }, + firstAuthCtx: admissionState.authCtx, + firstResponse: upstreamResponse, + outcomeStatus: poolRetryOutcome, + sameAccountOnly: storedReplaySpent, + upstream, + connectMs, + passthroughEstimate, + stream: parsed.stream, + onResponse: (response, retryAuthCtx, retryRequest) => { + adoptCodexWsStage(response); + captureAffinityResponse( + response, + retryAuthCtx, + retryRequest, + retryAuthCtx.kind !== "main", + ); + }, + }); + if (retry.kind === "transport") { + admissionState.authCtx = retry.authCtx; + return transportFailureResponse(retry.error); + } + if (retry.kind === "retried") { + admissionState.authCtx = retry.authCtx; + request = retry.request; + refreshRequestToolAliases(request); + refreshUndeclaredToolGuard(request); + upstreamResponse = retry.upstreamResponse; + requestState.selectedForwardHeaders = retry.selectedForwardHeaders; + // Keep subagent quota-failure health keyed to the account that actually served. + requestState.subagentFallbackAccountId = retry.authCtx.accountId; + } + } + } + // The deterministic route record cannot classify history it never observed (restart, expiry, + // eviction, or an older transcript). Inspect only a bounded clone of a 4xx whose exact outbound + // Responses body still carries opaque state, then rebuild once through the ordinary adapter + // sanitation path. A second rejection falls through unchanged because the guard stays armed. + const opaqueBlobRecovery = await attemptOpaqueBlobRecovery({ + response: upstreamResponse, + outboundBody: request.body, + adapterName: transportState.adapter.name, + parsed, + guard: opaqueBlobRecoveryGuard, + signal: upstream.signal, + }, rebuildAndRefetch); + if (opaqueBlobRecovery.kind === "failed") return opaqueBlobRecovery.response; + if (opaqueBlobRecovery.kind === "recovered") { + upstreamResponse = opaqueBlobRecovery.response; + continue passthroughRecovery; + } + + const recoveryContentType = upstreamResponse.headers.get("content-type")?.toLowerCase() ?? ""; + const streamedFunctionOutputCandidate = upstreamResponse.ok + && !!upstreamResponse.body + && (recoveryContentType.includes("text/event-stream") || (!recoveryContentType && parsed.stream)) + && !opaqueBlobRecoveryGuard.attempted + && outboundResponsesBodyCarriesEncryptedFunctionOutput(request.body); + if (streamedFunctionOutputCandidate) { + const preflightLog: RequestLogContext = { model: logCtx.model, provider: logCtx.provider }; + const preflight = await preflightComboStreamResponse(upstreamResponse, preflightLog, + payload => { + if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; + const type = (payload as { type?: unknown }).type; + return (type === "error" || type === "response.failed" || type === "response.incomplete") + && upstreamErrorMessageFromPayload(payload) === ENCRYPTED_FUNCTION_OUTPUT_REJECTION; + }, { + allowMissingContentType: !recoveryContentType && parsed.stream, + replayReadErrors: true, + }); + if (options.abortSignal?.aborted) return transportFailureResponse(options.abortSignal.reason); + upstreamResponse = preflight.response; + if (preflight.kind === "failed") { + const streamedOpaqueRecovery = await attemptOpaqueBlobRecovery({ + response: upstreamResponse, + outboundBody: request.body, + adapterName: transportState.adapter.name, + parsed, + guard: opaqueBlobRecoveryGuard, + signal: upstream.signal, + }, rebuildAndRefetch); + if (streamedOpaqueRecovery.kind === "failed") return streamedOpaqueRecovery.response; + if (streamedOpaqueRecovery.kind === "recovered") { + resetStreamedOpaqueBlobLogContext(logCtx); + upstreamResponse = streamedOpaqueRecovery.response; + continue passthroughRecovery; + } + logCtx.upstreamError = preflightLog.upstreamError; + logCtx.terminalHttpStatus = preflightLog.terminalHttpStatus; + logCtx.terminalErrorCode = preflightLog.terminalErrorCode; + logCtx.terminalIncompleteReason = preflightLog.terminalIncompleteReason; + } + } + // Console Go (opencode-zen / opencode-go) intermittently rejects a body it accepts seconds + // later with 400 invalid_request_error / "Invalid upload request." Replay the byte-identical + // request once after the exact gateway rejection. Single-shot guard. + // This recovery reuses the captured request; other recovery kinds still rebuild. + if (!consoleGoUploadRetryGuard.attempted) { + const uploadRejectionBody = await consoleGoUploadRejectionBody( + upstreamResponse, + consoleGoUploadRetryGuard.attempted, + upstream.signal, + ); + if (uploadRejectionBody !== undefined + && isTransientConsoleGoUploadRejection({ + status: upstreamResponse.status, + errorBody: uploadRejectionBody, + outboundUrl: request.url, + })) { + consoleGoUploadRetryGuard.attempted = true; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + if (!upstream.signal.aborted) { + try { + await sleepWithAbort(CONSOLE_GO_UPLOAD_RETRY_DELAY_MS, upstream.signal); + } catch { return clientCancelledResponse(); } + } + if (upstream.signal.aborted) return clientCancelledResponse(); + const result = await rebuildAndRefetch("console-go-upload-retry"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue passthroughRecovery; + } + } + // Reasoning-effort downgrade: a rung the catalog still advertises can be refused upstream -- + // the metadata records the model's ladder, not this account's entitlement (a Muse Code + // subscription gates max on muse-spark-1.3-contributor, for example). Learn the refusal so + // later turns clamp before dispatch, then replay once at the next lower published rung + // instead of failing the turn; requestedEffort/effectiveEffort keep both values in usage. + if (!reasoningEffortDowngradeGuard.attempted) { + const rejectionText = await reasoningEffortRejectionText( + upstreamResponse, + reasoningEffortDowngradeGuard.attempted, + upstream.signal, + ); + const downgrade = rejectionText === undefined + ? undefined + : planReasoningEffortDowngrade({ + provider: route.provider, + modelId: parsed.modelId, + requested: parsed.options.reasoning, + rejectionText, + }); + if (downgrade) { + reasoningEffortDowngradeGuard.attempted = true; + parsed.options.reasoning = downgrade.effort; + try { void upstreamResponse.body?.cancel().catch(() => {}); } catch { /* already consumed/closed */ } + const result = await rebuildAndRefetch("reasoning-effort-downgrade"); + if ("failed" in result) return result.failed; + upstreamResponse = result; + continue passthroughRecovery; + } + } + break; + } + + return { + codexSafetyBufferingOptions, + imageGenCallAliases, + routedCustomToolNames, + routedCustomToolRepairNames, + routedToolSearchNames, + rememberPassthroughResponse, + selfNamedNamespaceScrubAuthorization, + providerExecutedCallTypes, + get request(): Awaited> { + return request; + }, + set request(value: Awaited>) { + request = value; + }, + functionRepairSchemas, + get outboundRequestBody(): Record | undefined { + return outboundRequestBody; + }, + set outboundRequestBody(value: Record | undefined) { + outboundRequestBody = value; + }, + declaredWireToolNames, + declaredBareWireToolNames, + declaredNamelessClientCallTypes, + authorizedBareNamespaceToolAliases, + normalizeFunctionCompletionJson, + get undeclaredToolGuardActive(): typeof undeclaredToolGuardActive { + return undeclaredToolGuardActive; + }, + set undeclaredToolGuardActive(value: typeof undeclaredToolGuardActive) { + undeclaredToolGuardActive = value; + }, + noteInspectedPayload, + rememberPassthroughResponseChecked, + upstream, + connectMs, + upstreamResponse, + }; +} + +export type PassthroughExchange = Exclude>, Response>; diff --git a/src/server/responses/passthrough-execution.ts b/src/server/responses/passthrough-execution.ts new file mode 100644 index 0000000000..4fcdf84eeb --- /dev/null +++ b/src/server/responses/passthrough-execution.ts @@ -0,0 +1,54 @@ +import type { + ResponsesRequestContext, + ResponsesAdmissionState, + PassthroughAdmissionState, +} from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResponsesSidecarAuth } from "./request-sidecar-auth"; +import type { ResponsesEffects } from "./response-effects"; +import type { ResponsesSendBudget } from "./request-send-budget"; +import { preparePassthroughExchange } from "./passthrough-dispatch"; +import { deliverPassthroughResponse } from "./passthrough-delivery"; +import { releaseUpstreamHostAdmission } from "../../codex/upstream-host-health"; +import { releaseCodexAuthContextProbeLease } from "../../codex/auth-context"; + +/** Owns the native host lease across dispatch, recovery, and response construction. */ +export async function executePassthroughResponse( + requestContext: ResponsesRequestContext, + admissionState: ResponsesAdmissionState, + requestState: PreparedResponsesRequest, + transportState: ResponsesTransport, + sidecarState: ResponsesSidecarAuth, + responseEffects: ResponsesEffects, + sendBudgetState: ResponsesSendBudget, +): Promise { + const nativeHostState: PassthroughAdmissionState = { lease: admissionState.pendingHostAdmissionLease }; + admissionState.pendingHostAdmissionLease = null; + try { + const nativeExchange = await preparePassthroughExchange( + requestContext, + admissionState, + nativeHostState, + requestState, + transportState, + responseEffects, + sendBudgetState, + ); + if (nativeExchange instanceof Response) return nativeExchange; + return await deliverPassthroughResponse( + requestContext, + admissionState, + requestState, + transportState, + sidecarState, + responseEffects, + nativeExchange, + ); + } finally { + if (nativeHostState.lease) { + releaseUpstreamHostAdmission(nativeHostState.lease); + releaseCodexAuthContextProbeLease(admissionState.authCtx); + } + } +} diff --git a/src/server/responses/request-prepare.ts b/src/server/responses/request-prepare.ts new file mode 100644 index 0000000000..4d69c1ec89 --- /dev/null +++ b/src/server/responses/request-prepare.ts @@ -0,0 +1,958 @@ +import type { ResponsesRequestContext, ResponsesAdmissionState, ResponsesDispatchers } from "./core-options"; +import { + agentTaskRecoveryConfig, + restoreCachedEncryptedAgentTasks, + recoverEncryptedAgentTaskWithResult, +} from "./agent-task-recovery"; +import { readJsonRequestBody, resolveInboundBodyLimitBytes } from "../request-decompress"; +import { + clientCancelledResponse, + decodeRequestErrorResponse, + comboUnavailable, + unreadableEncryptedAgentTaskResponse, +} from "./core-errors"; +import { parseSyntheticRowId } from "../fast-row"; +import { resolveComboId, comboIdFromRawBody, NoAvailableComboTargetsError } from "../../combos"; +import { recallComboForLane } from "./combo-session-recall"; +import { + sessionLaneIdFromRequest, + conversationIdFromResponsesRequest, + sessionIdHeaderFromRequest, + reasoningReplayConversationIdFromResponsesRequest, +} from "../request-log-conversation"; +import { + isShadowSourceModel, + shadowSourceModelPrefix, + shouldInterceptShadowCall, +} from "../../lib/shadow-call"; +import { sanitizeLogMetadataString } from "../../lib/redact"; +import { + hasUnreadableEncryptedAgentTask, + sanitizeEncryptedContentInPlace, + stripAgentMessageCiphertextInPlace, +} from "./encrypted-payload"; +import { + codexPoolAffinityKey, + previewCodexPoolLineage, + applyCodexAuthContextToProvider, +} from "../../codex/auth-context"; +import { + copyPreviousResponseReplayProvenance, + expandPreviousResponseInput, + previousResponseScopeMismatch, + previousResponseReplayFailure, + markBodyNonPersistable, + previousResponseProviderState, +} from "../../responses/state"; +import { formatErrorResponse } from "../../bridge"; +import type { OcxParsedRequest } from "../../types"; +import { buildToolBridgeMaps } from "./collaboration"; +import { parseRequest } from "../../responses/parser"; +import { anthropicSessionKeyFromParts } from "../../oauth/anthropic-routing"; +import { isTranslatorBudgetExceededError } from "../../lib/translator-budget"; +import { bindTurnTerminationScope, rememberDeliveredFinalAnswer } from "../../responses/turn-termination"; +import { requestLogSpeedLabel, readConfiguredCodexServiceTier } from "../request-log"; +import type { RouteResult } from "../../router"; +import { + routeConcreteModel, + routeCompactionModel, + routeModel, + NoEligiblePolicyCandidateError, +} from "../../router"; +import { evidenceFromBody } from "../../routing/request-evidence"; +import { OPENAI_CODEX_PROVIDER_ID, isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import { isThreadSpawnRequest } from "../effort-policy"; +import { + resolveSubagentFallbackChain, + maybePrimeSubagentQuota, + applySubagentModelFallback, +} from "../../codex/subagent-model-fallback"; +import { codexAccountSelectionForTurn } from "../lifecycle"; +import { isNativeMainTrafficBlocked } from "../../codex/native-profile-startup"; +import type { + SubagentPoolAccountPreview, + SubagentModelEligibleAccountIds, +} from "../../codex/subagent-model-fallback"; +import { + codexRouteCredentialDomainHeaders, + codexRouteCredentialOwnership, + resolveResponsesCodexAuth, + withClaudeNativeSession, +} from "./core-auth"; +import { + resolveSubagentFallbackModelEligibility, + canPassThroughEncryptedV2AgentTask, + applyFinalRouteRequestNormalization, +} from "./core-normalize"; +import { resolveCodexModelEntitlements } from "../../codex/model-entitlements"; +import { + previewCodexAccountForRequest, + codexQuotaScopeForModel, + formatCodexProviderForLog, +} from "../../codex/routing"; +import { isInjectionDebugEnabled } from "../../lib/debug-settings"; +import { injectionDebugLog } from "../../lib/injection-debug-log"; +import { slugsEquivalent } from "../../providers/slug-codec"; +import type { AgentTaskRecoveryFailureReason } from "./agent-task-recovery"; +import { resolveWireProtocolOverride } from "../adapter-resolve"; +import { hasUnmappedRoutedCustomToolOutput } from "../../responses/custom-tool-compat"; +import { + isCodexReserveHelperUnsupported, + CODEX_RESERVE_HELPER_UNSUPPORTED_MESSAGE, +} from "../../codex/loopback-target"; +import { checkInputAdmission } from "./input-admission"; +import { nativeContextLimits } from "../../codex/catalog"; +import { streamingContextOverflowResponse } from "./context-overflow"; +import { + preAuthUpstreamHostCircuitKey, + upstreamHostCircuitOpenResponse, + applyCodexAccountGatedWireNormalization, + codexLogAccountId, +} from "./core-codex-account"; +import { acquireUpstreamHostAdmission } from "../../codex/upstream-host-health"; +import { codexAuthContextLogLabel } from "../../codex/account-label"; +import { + conversationStateBindingFromAuth, + applyAccountChangeConversationStateScrub, +} from "./account-change-state"; + +/** Parses, selects, and admits one request without changing the dispatch policy. */ +export async function prepareResponsesRequest( + requestContext: Pick, + admissionState: ResponsesAdmissionState, + requestDispatchers: ResponsesDispatchers, +) { + const { options, config, req, logCtx } = requestContext; + + // The Chat and Anthropic surfaces replay through here with a Responses-shaped body, + // so an omitted value means a genuine Responses inbound. + const inboundWire = options.inboundWire ?? "responses"; + const translatorBudget = options.translatorBudget; + const agentTaskRecovery = agentTaskRecoveryConfig(config); + let body: unknown; + try { + body = await readJsonRequestBody(req, translatorBudget, resolveInboundBodyLimitBytes(config.maxInboundBodyBytes)); + } catch (err) { + if (options.abortSignal?.aborted || req.signal.aborted) { + return clientCancelledResponse(); + } + return decodeRequestErrorResponse(err, "responses"); + } + // An effort row naming a table-less combo (`combo/x--high`) must reach the combo dispatcher + // as its base id, so the selector is normalized here, before comboIdFromRawBody reads model. + const comboRows = !options.comboAttempt && body && typeof body === "object" && !Array.isArray(body) + && typeof (body as { model?: unknown }).model === "string" + // One parse for both grammars, from the selector as the client sent it. Parsing them + // separately made the outcome depend on which ran first. + ? parseSyntheticRowId((body as { model: string }).model, config) + : { fastRow: null, effortRow: null }; + const comboEffortRow = comboRows.effortRow; + if (comboRows.fastRow) { + // Same reason as the effort row above: the combo dispatcher reads `model` next, so the + // selector has to be normalized before it, or a combo child is built from a synthetic id. + const raw = body as Record; + raw.model = comboRows.fastRow.baseId; + // A caller INTENT, not a decision. decideTier still rules on eligibility downstream, so + // fastMode:false and an ineligible route both still suppress it. + raw.service_tier = "priority"; + } + if (comboEffortRow) { + const raw = body as Record; + raw.model = comboEffortRow.baseId; + const rawReasoning = raw.reasoning; + raw.reasoning = { + ...(rawReasoning && typeof rawReasoning === "object" && !Array.isArray(rawReasoning) + ? rawReasoning as Record + : {}), + effort: comboEffortRow.effort, + }; + } + // Compaction may send the last client-visible bare model after a combo switch. + // Configured selectors take precedence; otherwise recall before combo dispatch (#3891). + if (!options.comboAttempt && body && typeof body === "object" && !Array.isArray(body)) { + const rawModel = (body as { model?: unknown }).model; + const rawInput = (body as { input?: unknown }).input; + const isCompactionTrigger = Array.isArray(rawInput) + && rawInput.some((item: unknown) => + typeof item === "object" && item !== null && (item as { type?: string }).type === "compaction_trigger"); + if (typeof rawModel === "string" && !rawModel.includes("/") && isCompactionTrigger + && !comboRows.fastRow && !comboEffortRow + && !resolveComboId(config, rawModel)) { + const recalledComboId = recallComboForLane(config, sessionLaneIdFromRequest(req.headers), rawModel); + if (recalledComboId) { + (body as Record).model = `combo/${recalledComboId}`; + } + } + } + // A shadow-call replacement that names a COMBO is routing policy, not the identity of any + // one pick. The late intercept site below resolves it through routeModel/tryPickComboModel, + // which collapses the table to a single target while still tagging `routeKind: "combo"`, so + // the combo gate on the next line never fires, handleComboResponses never runs, and 429/5xx + // hops — which only exist inside that loop — are unreachable (#4129). Rewrite the selector + // here instead, before comboIdFromRawBody reads `model`, and identify the combo by CONFIG + // LOOKUP so the check can never observe a one-candidate collapse. + if (!options.comboAttempt && body && typeof body === "object" && !Array.isArray(body)) { + const shadowIntercept = config.shadowCallIntercept; + const rawShadowModel = (body as { model?: unknown }).model; + if (shadowIntercept?.enabled && shadowIntercept.model && typeof rawShadowModel === "string" + && isShadowSourceModel(rawShadowModel, shadowIntercept.sourceModels)) { + const shadowComboId = resolveComboId(config, shadowIntercept.model); + if (shadowComboId && Object.hasOwn(config.combos ?? {}, shadowComboId)) { + (body as Record).model = shadowIntercept.model; + // Same rule as the late intercept site: record the operator-configured prefix that + // matched, never the caller's raw model string. Matching is by prefix, so the raw + // value is caller-controlled and reaches usage.jsonl and /api/logs. + logCtx.shadowCallRewrittenFrom = sanitizeLogMetadataString( + shadowSourceModelPrefix(rawShadowModel, shadowIntercept.sourceModels), + ); + } + } + } + const comboId = !options.comboAttempt ? comboIdFromRawBody(body, config) : null; + if (comboId && Object.hasOwn(config.combos ?? {}, comboId)) { + options.onRequestBodyRead?.(); + return requestDispatchers.handleComboResponses(req, body, comboId, config, logCtx, { + ...options, + // The original request body was accepted above. Combo children are synthetic + // replays and must not repeat the caller-owned timeout transition. + onRequestBodyRead: undefined, + }); + } + let unreadableEncryptedAgentTask = hasUnreadableEncryptedAgentTask( + (body as { input?: unknown } | undefined)?.input, + ); + const inboundClientThreadId = req.headers.get("x-codex-parent-thread-id")?.trim() || undefined; + const cursorClientThreadId = codexPoolAffinityKey(req.headers); + const originalBody = body; + if (options.comboReplaySnapshot) { + copyPreviousResponseReplayProvenance(options.comboReplaySnapshot.sourceBody, body); + } else { + body = expandPreviousResponseInput(body, inboundClientThreadId); + if (previousResponseScopeMismatch(body)) { + console.warn("[opencodex] dropped a previous_response_id with a mismatched client task scope; continuing fresh"); + } + if (previousResponseReplayFailure(body)) { + return formatErrorResponse( + 400, + "previous_response_not_found", + "Continuation state is unavailable or corrupt; resend the full conversation without previous_response_id.", + ); + } + } + const previousResponseInputExpanded = options.comboReplaySnapshot?.previousResponseInputExpanded + ?? (body !== originalBody + && typeof (body as { previous_response_id?: unknown }).previous_response_id === "string"); + + // Spawn-message compatibility (both directions): agent_message task payloads ride in + // encrypted_content slots as plaintext. Rewrite them to input_text on the RAW body BEFORE + // parsing so every consumer sees the payload: parseRequest (routed/translated providers read + // the parsed messages) and the native passthrough (_rawBody is this same object, serialized + // verbatim). Genuine backend ciphertext is left byte-identical (looksLikeBackendCiphertext). + { + const rewritten = sanitizeEncryptedContentInPlace( + (body as { input?: unknown } | undefined)?.input, + ); + if (rewritten > 0) + console.warn( + `[opencodex] rewrote ${rewritten} plaintext encrypted_content part(s) to input_text (spawn-message compatibility)`, + ); + } + + let parsed: OcxParsedRequest; + let toolBridgeMaps: ReturnType; + try { + parsed = parseRequest(body); + parsed._promptCacheKeyIsSharedCohort = options.promptCacheKeyIsSharedCohort; + // Captured before any parser mutates it, so both grammars see the client's id. + const { fastRow, effortRow } = parseSyntheticRowId(parsed.modelId, config); + if (fastRow) { + parsed.modelId = fastRow.baseId; + parsed.options.serviceTier = "priority"; + const raw = parsed._rawBody as Record; + raw.model = fastRow.baseId; + raw.service_tier = "priority"; + } + if (effortRow) { + parsed.modelId = effortRow.baseId; + parsed.options.reasoning = effortRow.effort; + const raw = parsed._rawBody as Record; + const rawReasoning = raw.reasoning; + raw.model = effortRow.baseId; + raw.reasoning = { + ...(rawReasoning && typeof rawReasoning === "object" && !Array.isArray(rawReasoning) + ? rawReasoning as Record + : {}), + effort: effortRow.effort, + }; + } + if (options.comboReplaySnapshot?.recoveredPlaintext) { + markBodyNonPersistable(parsed._rawBody); + } + toolBridgeMaps = buildToolBridgeMaps(parsed, translatorBudget); + if (previousResponseInputExpanded) parsed._previousResponseInputExpanded = true; + const providerContinuationCandidate = options.comboReplaySnapshot + ? options.comboReplaySnapshot.providerContinuation + : previousResponseProviderState(parsed.previousResponseId); + if (providerContinuationCandidate) parsed._providerContinuationCandidate = providerContinuationCandidate; + if (inboundClientThreadId) { + parsed._clientThreadId = inboundClientThreadId; + } else if ( + options.inboundWire === "anthropic" + && options.promptCacheKeyIsSharedCohort !== true + && typeof parsed.options.promptCacheKey === "string" + && parsed.options.promptCacheKey.trim().length > 0 + ) { + // Claude Code has no Codex parent-thread header, but its metadata.user_id is + // translated into a stable per-session prompt_cache_key. Use it as the replay + // thread identity so Gemini thought signatures are remembered by call_id for + // Anthropic Messages clients too (#1735/#1926). Keep `_clientThreadId` unset so + // existing provider session-id derivation (first-user-text fallback) is unchanged. + // Normalize through anthropicSessionKeyFromParts so overlong keys are hashed and + // trimming matches the affinity/session-key path exactly (no raw >128-char ids). + const normalizedCacheKey = anthropicSessionKeyFromParts({ + promptCacheKey: parsed.options.promptCacheKey, + // The enclosing branch already proves this is not the shared cohort. + promptCacheKeyIsSharedCohort: false, + }); + if (normalizedCacheKey) { + parsed._reasoningReplayScope = { clientThreadId: normalizedCacheKey }; + } + } + if (cursorClientThreadId) parsed._cursorClientThreadId = cursorClientThreadId; + } catch (err) { + if (isTranslatorBudgetExceededError(err)) { + return formatErrorResponse(413, "request_too_large", "request translation buffer exceeded the safe limit", { + code: "translation_buffer_limit", + }); + } + return formatErrorResponse(400, "invalid_request_error", err instanceof Error ? err.message : String(err)); + } + options.onRequestBodyRead?.(); + const responseStateOptions = (force = false): { force?: boolean; clientThreadId?: string } => ({ + ...(force ? { force: true } : {}), + ...(parsed._clientThreadId ? { clientThreadId: parsed._clientThreadId } : {}), + }); + const resolvedConversationId = conversationIdFromResponsesRequest({ + clientThreadId: parsed._clientThreadId, + sessionIdHeader: sessionIdHeaderFromRequest(req.headers), + threadIdHeader: req.headers.get("thread-id"), + cursorConversationId: parsed._cursorConversationId, + }); + bindTurnTerminationScope(parsed, resolvedConversationId); + const rememberKiroDeliveredFinalAnswer = (adapterName: string, response: unknown): void => { + if (adapterName === "kiro") rememberDeliveredFinalAnswer(parsed, response); + }; + // _clientThreadId remains the routing/continuation identity supplied by Codex. Replay state uses + // a dedicated raw conversation namespace so mixed headers that carry the same identity still + // match, and a shared/synthetic session_id cannot coalesce distinct thread/Cursor conversations. + // Keep an Anthropic prompt_cache_key scope already bound above (#1735/#1926). + if (!parsed._reasoningReplayScope) { + const reasoningReplayConversationId = reasoningReplayConversationIdFromResponsesRequest({ + clientThreadId: parsed._clientThreadId, + threadIdHeader: req.headers.get("thread-id"), + cursorConversationId: parsed._cursorConversationId, + sessionIdHeader: sessionIdHeaderFromRequest(req.headers), + }); + if (reasoningReplayConversationId) { + parsed._reasoningReplayScope = { clientThreadId: reasoningReplayConversationId }; + } + } + // Prefer a pre-populated id (routed Claude) over Responses headers that may be + // absent or synthetically injected (session_id from prompt_cache_key). + if (!logCtx.conversationId) { + logCtx.conversationId = resolvedConversationId; + } + logCtx.requestedModel = parsed.modelId; + logCtx.requestedEffort = parsed.options.reasoning; + logCtx.callerServiceTier = sanitizeLogMetadataString(parsed.options.serviceTier); + logCtx.requestedServiceTier = parsed.options.serviceTier; + logCtx.requestedSpeedLabel = requestLogSpeedLabel(parsed.options.serviceTier); + logCtx.configuredServiceTier = readConfiguredCodexServiceTier(); + logCtx.configuredSpeedLabel = requestLogSpeedLabel(logCtx.configuredServiceTier); + + let route: RouteResult; + let credentialDomainWasRewritten = false; + try { + // A `compaction_trigger` turn may name a bare native model the operator has + // no canonical OpenAI route for (#2901). Only the initial compaction route + // may fall back to the configured default provider; combo attempts and the + // later fallback/recovery re-routes keep the ordinary reservation. + const resolveRoute = (modelId: string) => options.comboAttempt + ? routeConcreteModel(config, modelId) + : parsed._compactionRequest === true + ? routeCompactionModel(config, modelId, evidenceFromBody(parsed._rawBody)) + : routeModel(config, modelId, evidenceFromBody(parsed._rawBody)); + const _sci = config.shadowCallIntercept; + let shadowRoute: RouteResult | undefined; + if (_sci?.enabled && _sci.model && isShadowSourceModel(parsed.modelId, _sci.sourceModels)) { + const sourcePrefix = shadowSourceModelPrefix(parsed.modelId, _sci.sourceModels)!; + let sourceIdentity = { providerName: OPENAI_CODEX_PROVIDER_ID, modelId: sourcePrefix }; + try { + const resolvedSource = routeConcreteModel(config, parsed.modelId); + sourceIdentity = { providerName: resolvedSource.providerName, modelId: sourcePrefix }; + } catch { /* Native Codex helper calls remain OpenAI-owned without an enabled OpenAI route. */ } + const targetRoute = resolveRoute(_sci.model); + if (shouldInterceptShadowCall(parsed.modelId, _sci.sourceModels, sourceIdentity, targetRoute)) { + credentialDomainWasRewritten = true; + const _sciOriginal = parsed.modelId; + parsed.modelId = _sci.model; + if (parsed._rawBody && typeof parsed._rawBody === "object") { + (parsed._rawBody as { model?: string }).model = _sci.model; + } + // Record the operator-configured prefix that matched, NOT the caller's raw model string. + // Matching is by prefix, so a caller can append arbitrary text and still intercept; that + // raw value would then land in usage.jsonl and /api/logs behind a pattern-based redactor + // that does not recognize every credential family. The prefix is a value the operator + // configured, so no caller-controlled string is persisted. + logCtx.shadowCallRewrittenFrom = sanitizeLogMetadataString( + shadowSourceModelPrefix(_sciOriginal, _sci.sourceModels), + ); + // Helpers must not resume/append into the parent thread's Cursor conversation. + parsed._cursorIsolateConversation = true; + shadowRoute = targetRoute; + } + } + if (parsed._compactionRequest === true) parsed._cursorIsolateConversation = true; + route = shadowRoute ?? resolveRoute(parsed.modelId); + logCtx.routeDecision = route.routeDecision; + } catch (err) { + if (err instanceof NoAvailableComboTargetsError) { + return comboUnavailable(err.comboId); + } + if (err instanceof NoEligiblePolicyCandidateError) { + // Persist the evaluation trace (per-candidate exclusions + the + // no-eligible reason) so failed policy requests stay auditable. + logCtx.routeDecision = err.trace; + } + return formatErrorResponse(404, "invalid_request_error", err instanceof Error ? err.message : String(err)); + } + + const hasUnexpandedPreviousResponse = !!parsed.previousResponseId + && parsed._previousResponseInputExpanded !== true; + // Exact account selectors are isolated from Pool-wide quota work. A canonical replay miss must + // also fail closed without polling quota upstream. Cached fallback state can still select a + // provider with native continuation support below. + const threadSpawn = isThreadSpawnRequest(req.headers); + const initialSubagentFallbackChain = threadSpawn && !options.comboAttempt + ? resolveSubagentFallbackChain(parsed, config) + : null; + const previewSelectionAdmission = threadSpawn + && !options.comboAttempt + && (route.codexAccountId === undefined || initialSubagentFallbackChain !== null) + ? codexAccountSelectionForTurn(options.turnAdmissionLease)?.() + : undefined; + const nativeMainRecoveryBlocked = isNativeMainTrafficBlocked(); + const nativeMainReadsForbidden = nativeMainRecoveryBlocked + || previewSelectionAdmission?.mainProfileDraining === true; + const previewSelectionOptions = { + nativeMainSelectionOnly: !nativeMainRecoveryBlocked + && previewSelectionAdmission?.mainProfileDraining === true, + }; + let selectedForwardHeaders = req.headers; + let subagentFallbackAccountId = config.activeCodexAccountId ?? null; + let subagentFallbackAccountPreview: SubagentPoolAccountPreview | undefined; + let subagentFallbackModelEligibleAccountIdsForModel: SubagentModelEligibleAccountIds | undefined; + let subagentQuotaFailureModel = parsed.modelId; + const parentThreadId = req.headers.get("x-codex-parent-thread-id")?.trim() ?? null; + const poolAffinityKey = codexPoolAffinityKey(req.headers) ?? null; + // Preview has to see the same lineage resolve does. Without it, a child's first turn is + // previewed as a cold pick and resolved onto the family account, and the subagent fallback + // then decides model eligibility against an account the request will never use. + // + // "The same" means both halves of the question the final resolution asks. The Authorization + // it will be given, because the lineage scope is an HMAC of exactly that header; and its own + // Pool-state predicate, because a fixed account selector and a request-owned credential + // deliberately create no affinity at all -- previewing a family binding for one of those would + // hand model fallback an account this request can never authenticate as. Read-only: the record + // is written by the resolution that binds, never by a preview that may own no Pool state. + const previewAuthHeaders = codexRouteCredentialDomainHeaders( + req, + route, + options, + credentialDomainWasRewritten, + ); + const poolLineage = previewCodexPoolLineage(previewAuthHeaders, options.codexAuthPolicy ?? config, { + accountId: route.codexAccountId, + modelId: route.modelId, + admission: options.admission, + requestScopedMainCredential: codexRouteCredentialOwnership( + previewAuthHeaders, + config, + route, + options, + ).requestScopedMainCredential, + }); + + try { + if ( + threadSpawn + && route.codexAccountId === undefined + && !(hasUnexpandedPreviousResponse && isCanonicalOpenAiForwardProvider(route.provider)) + ) { + await maybePrimeSubagentQuota(config, Date.now(), { nativeMainReadsForbidden }); + } + + // Subagent fallback must settle the final model/provider BEFORE route-dependent + // normalization (virtual models, effort caps, service tier, wire protocol). + // Preview the preferred Codex account without acquiring a probe lease or refreshing + // tokens — auth is resolved only after the final route is selected. + if ( + threadSpawn + && !options.comboAttempt + && (route.codexAccountId === undefined || initialSubagentFallbackChain !== null) + ) { + // The final resolveCodexAuthContext binds under codexQuotaScopeForModel(route.modelId), + // so the preview must read the same scope slot — an undefined scope would map to the + // "legacy" affinity bucket and never find a binding made under "shared" or a native + // model scope, making the preview diverge from the account that actually authenticates. + const fallbackChain = initialSubagentFallbackChain; + subagentFallbackModelEligibleAccountIdsForModel = await resolveSubagentFallbackModelEligibility({ + config, + fallbackChain, + nativeMainReadsForbidden, + resolver: options.resolveCodexModelEntitlements ?? resolveCodexModelEntitlements, + }); + const fallbackNow = Date.now(); + subagentFallbackAccountPreview = (modelId, previewNow, modelEligibleAccountIds) => previewCodexAccountForRequest( + poolAffinityKey, + config, + previewNow, + codexQuotaScopeForModel(modelId), + { ...previewSelectionOptions, modelEligibleAccountIds }, + modelId, + poolLineage, + ); + const previewAccountId = route.codexAccountId ?? subagentFallbackAccountPreview( + route.modelId, + fallbackNow, + subagentFallbackModelEligibleAccountIdsForModel?.(route.modelId), + ); + subagentFallbackAccountId = previewAccountId ?? config.activeCodexAccountId ?? null; + const fallback = applySubagentModelFallback( + parsed, + req.headers, + config, + previewAccountId, + fallbackNow, + unreadableEncryptedAgentTask, + previewSelectionOptions, + subagentFallbackAccountPreview, + subagentFallbackModelEligibleAccountIdsForModel, + fallbackChain, + candidateRoute => canPassThroughEncryptedV2AgentTask(candidateRoute, inboundWire), + ); + if (fallback) { + (logCtx as unknown as Record).subagentModelFallbackFrom = fallback.from; + (logCtx as unknown as Record).subagentModelFallbackTo = fallback.to; + if (isInjectionDebugEnabled()) { + injectionDebugLog(`[opencodex] subagent model fallback ${fallback.from} -> ${fallback.to}`); + } + } + subagentQuotaFailureModel = fallback?.to ?? parsed.modelId; + + if (fallback?.to && !slugsEquivalent(fallback.to, route.modelId)) { + try { + route = routeModel(config, fallback.to, evidenceFromBody(parsed._rawBody)); + credentialDomainWasRewritten = true; + logCtx.routeDecision = route.routeDecision; + } catch (err) { + if (err instanceof NoAvailableComboTargetsError) { + return comboUnavailable(err.comboId); + } + if (err instanceof NoEligiblePolicyCandidateError) { + logCtx.routeDecision = err.trace; + } + return formatErrorResponse(404, "invalid_request_error", err instanceof Error ? err.message : String(err)); + } + } + } + } finally { + previewSelectionAdmission?.release(); + } + + let recoveryFailureReason: AgentTaskRecoveryFailureReason | undefined; + // Native fallback and explicitly trusted direct Responses routes can consume ciphertext, + // so recover only after final route selection. + // + // Deliberately NOT gated on `threadSpawn` (#4089). Switching a live thread from a native + // ChatGPT model to a routed provider replays a backend-minted encrypted agent message on every + // later turn, and a model switch is not a spawn, so the spawn requirement failed the thread + // closed permanently without ever attempting recovery. The trust boundary is + // `recoveryAdmission()` in ./agent-task-recovery -- Codex originator, live native ChatGPT + // bearer, matching chatgpt-account-id, no inbound API key, no proxy-admission secret -- which + // admits only the owner of the session that would be spent. `threadSpawn` narrowed which of + // that owner's own requests could use their own session; it kept nobody else out. The combo + // gate above keeps its spawn requirement: that path has its own native-target filtering and + // per-attempt failover, and the reported defect is on this path. + if ( + inboundWire === "responses" + && agentTaskRecovery + && !isCanonicalOpenAiForwardProvider(route.provider) + && !options.comboAttempt + && !canPassThroughEncryptedV2AgentTask(route, inboundWire) + ) { + let recovered = restoreCachedEncryptedAgentTasks( + req, (body as { input?: unknown } | undefined)?.input, config, { parentThreadId }, + ) > 0; + unreadableEncryptedAgentTask = hasUnreadableEncryptedAgentTask( + (body as { input?: unknown } | undefined)?.input, + ); + if (unreadableEncryptedAgentTask) try { + const result = await recoverEncryptedAgentTaskWithResult( + req, + (body as { input?: unknown } | undefined)?.input, + agentTaskRecovery, + config, + { parentThreadId, abortSignal: options.abortSignal }, + ); + recovered = result.recovered; + recoveryFailureReason = result.recovered ? undefined : result.reason; + } catch { + recovered = false; + recoveryFailureReason = undefined; + } + if (recovered) { + unreadableEncryptedAgentTask = hasUnreadableEncryptedAgentTask( + (body as { input?: unknown } | undefined)?.input, + ); + if (!unreadableEncryptedAgentTask) { + try { + const reparsed = parseRequest(body); + const kept: Array = [ + "_previousResponseInputExpanded", + "_providerContinuation", + "_providerContinuationCandidate", + "_providerContinuationOwner", + "_cursorConversationId", + "_clientThreadId", + "_promptCacheKeyIsSharedCohort", + "_cursorClientThreadId", + "_reasoningReplayScope", + "_cursorIsolateConversation", + ]; + for (const key of kept) { + if (parsed[key] !== undefined) { + (reparsed as unknown as Record)[key] = parsed[key]; + } + } + bindTurnTerminationScope(reparsed, resolvedConversationId); + parsed = reparsed; + // The recovery mutated `body.input` in place, so `_rawBody` now carries decrypted task + // text. Bar it from the continuation cache before any recording path can reach it — + // that cache is persisted to disk, which would defeat the recovery cache's TTL. + markBodyNonPersistable(parsed._rawBody); + + // The ciphertext-only pass intentionally excludes routed candidates. Once recovery + // makes the assignment readable, run selection again with the full configured chain + // and keep the route in sync with any newly selected fallback. + const recoverySelectionAdmission = codexAccountSelectionForTurn(options.turnAdmissionLease)?.(); + const fallback = (() => { + try { + const recoveryNativeMainBlocked = isNativeMainTrafficBlocked(); + const recoverySelectionOptions = { + nativeMainSelectionOnly: !recoveryNativeMainBlocked + && recoverySelectionAdmission?.mainProfileDraining === true, + }; + const recoveryNow = Date.now(); + // Carry the entitlement filter through recovery too (#2509/#2623). The scope was + // already re-previewed per candidate here; the ELIGIBLE-ACCOUNT set was not, so a + // recovered assignment could select an account that is not entitled to the model + // and then fail closed at final auth — the same class of stale-selection bug as + // the quota scope, one layer over. + subagentFallbackAccountPreview = (modelId, previewNow, modelEligibleAccountIds) => previewCodexAccountForRequest( + poolAffinityKey, + config, + previewNow, + codexQuotaScopeForModel(modelId), + { ...recoverySelectionOptions, modelEligibleAccountIds }, + modelId, + poolLineage, + ); + const recoveryPreviewAccountId = subagentFallbackAccountPreview( + parsed.modelId, + recoveryNow, + subagentFallbackModelEligibleAccountIdsForModel?.(parsed.modelId), + ); + return applySubagentModelFallback( + parsed, + req.headers, + config, + recoveryPreviewAccountId, + recoveryNow, + false, + recoverySelectionOptions, + subagentFallbackAccountPreview, + subagentFallbackModelEligibleAccountIdsForModel, + ); + } finally { + recoverySelectionAdmission?.release(); + } + })(); + if (fallback) { + (logCtx as unknown as Record).subagentModelFallbackFrom = fallback.from; + (logCtx as unknown as Record).subagentModelFallbackTo = fallback.to; + if (isInjectionDebugEnabled()) { + injectionDebugLog(`[opencodex] subagent model fallback ${fallback.from} -> ${fallback.to}`); + } + } + subagentQuotaFailureModel = fallback?.to ?? parsed.modelId; + + if (fallback?.to && !slugsEquivalent(fallback.to, route.modelId)) { + try { + route = routeModel(config, fallback.to, evidenceFromBody(parsed._rawBody)); + credentialDomainWasRewritten = true; + logCtx.routeDecision = route.routeDecision; + } catch (err) { + if (err instanceof NoAvailableComboTargetsError) { + return comboUnavailable(err.comboId); + } + if (err instanceof NoEligiblePolicyCandidateError) { + logCtx.routeDecision = err.trace; + } + return formatErrorResponse( + 404, + "invalid_request_error", + err instanceof Error ? err.message : String(err), + ); + } + } + } catch { + unreadableEncryptedAgentTask = true; + } + } + } + } + + if (options.abortSignal?.aborted) return clientCancelledResponse(); + + // Encrypted child tasks may reach the canonical native backend or an explicitly trusted + // direct Responses route. This runs against the FINAL route so native-only fallback can + // rescue an incompatible primary without weakening combo behavior. + const finalRouteCanPassThroughEncryptedTask = !options.comboAttempt + && canPassThroughEncryptedV2AgentTask(route, inboundWire); + if ( + (route.combo !== undefined || !isCanonicalOpenAiForwardProvider(route.provider)) + && !finalRouteCanPassThroughEncryptedTask + && unreadableEncryptedAgentTask + ) { + return unreadableEncryptedAgentTaskResponse(recoveryFailureReason); + } + + // The guard above asks whether the CURRENT worker task is readable, and it only inspects the + // tail item. An `agent_message` that mixes readable text with backend ciphertext answers + // "readable" to that question at every position, so it passed -- and then + // `normalizeRoutedAgentMessages` refused to lower it, because lowering requires every part to + // be representable. The raw Responses passthrough serialized the private item as it stood, so + // backend ciphertext and an item type only the Codex backend declares reached a third-party + // provider, which answered `422 unknown item type "agent_message"` (#4454). + // + // The opaque-blob path already knows the repair: replace the undecryptable part with an + // omission marker, which leaves the item lowerable. It applied that repair only AFTER an + // upstream rejection. For a destination that cannot accept the private item under any + // circumstances, that round trip was never going to succeed and sent the ciphertext to find + // out, so do the repair here instead. Recovery above has already had its chance to turn the + // same bytes into real plaintext; only what it could not rescue reaches this. + if (inboundWire === "responses" && !finalRouteCanPassThroughEncryptedTask) { + // Only the raw Responses passthrough puts input items on the wire verbatim, so that is the + // only wire this has to repair: translated wires rebuild the body from parsed messages, where + // `inputContentParts` drops an encrypted part instead of forwarding it. The exemption is the + // canonical Codex backend alone, because it is the one destination that minted these bytes and + // can read them. `authMode: "forward"` is NOT that test -- a noncanonical forward gateway is + // somebody else's server that happens to be configured for passthrough, and it receives the + // ciphertext like any other third party. + // + // Combo children run this too. Each child carries its own `structuredClone` of the body + // (`concreteComboRequestBody`) and its own concrete route, so a sibling's repair is invisible + // here and a target that resolves to a routed Responses wire would otherwise send the + // ciphertext that the parent's own dispatch no longer does. + const wireProvider = resolveWireProtocolOverride( + route.providerName, + route.modelId, + route.provider, + inboundWire, + ); + if (wireProvider.adapter === "openai-responses" && !isCanonicalOpenAiForwardProvider(wireProvider)) { + const repaired = stripAgentMessageCiphertextInPlace((body as { input?: unknown } | undefined)?.input); + if (repaired > 0) { + console.warn( + `[opencodex] replaced ciphertext in ${repaired} replayed agent message(s) with an omission marker; the selected provider cannot read native ChatGPT ciphertext`, + ); + } + } + } + + // The canonical ChatGPT backend rejects previous_response_id, so a local replay miss leaves no + // safe way to recover the omitted history. Fail before auth, adapter construction, or upstream + // I/O instead of stripping the id and silently forwarding a context-free delta (#702). + // Codex recognizes previous_response_not_found on WebSocket errors and reconnects with its + // full input. A generic invalid_request_error instead terminates the task after cache expiry. + if ( + hasUnexpandedPreviousResponse + && isCanonicalOpenAiForwardProvider(route.provider) + ) { + return formatErrorResponse( + 400, + "previous_response_not_found", + "OpenAI forward continuation state is unavailable or expired; resend the full conversation without previous_response_id.", + ); + } + + if (hasUnexpandedPreviousResponse) { + const continuationProvider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire); + // Stateless destinations cannot resolve the omitted prefix. Stateful destinations may, + // but a lowered custom result still needs its call to recover the original wire type. + // Native function/custom continuations without lowering keep their upstream-owned state. + if (continuationProvider.adapter === "openai-responses" + && (continuationProvider.statelessResponses === true + || hasUnmappedRoutedCustomToolOutput(parsed._rawBody, continuationProvider.supportsResponsesCustomTools))) { + return formatErrorResponse( + 400, + "previous_response_not_found", + "Routed continuation requires unavailable local history; resend the full conversation without previous_response_id.", + ); + } + } + + // Captured before normalization: whether the CLIENT asked for SSE. The + // transport-neutral upstream-streaming policy below may force a bounded JSON + // upstream for reliability (#875); the answer must then be reframed to SSE + // for streaming clients. + const clientRequestedStream = parsed.stream; + await applyFinalRouteRequestNormalization({ + parsed, + route, + config, + req, + logCtx, + inboundWire, + inboundTransport: options.inboundTransport, + claudeGoAffinity: options.claudeGoAffinity, + }); + // Attribute local auth/cooldown failures to the public selector too; exact auth may fail before + // the normal post-resolution provider label is assigned. + if (route.codexAccountNamespace) { + logCtx.provider = `${route.providerName}-${route.codexAccountNamespace}`; + } + + if (options.abortSignal?.aborted) return clientCancelledResponse(); + // Resolve aliases/combo children before refusing helpers; do not spend main auth or host budget. + if (isCanonicalOpenAiForwardProvider(route.provider) + && isCodexReserveHelperUnsupported(options.codexAuthPolicy ?? config, route.modelId, + options.admission, options.visionDescribeTerminal === true)) { + return formatErrorResponse(400, "invalid_request_error", CODEX_RESERVE_HELPER_UNSUPPORTED_MESSAGE); + } + // Refuse an input that cannot plausibly fit the model context window before spending auth, + // circuit budget, or upstream bandwidth on a turn the provider will reject anyway (#1412). + // + // Compaction turns are exempt: Codex sends compaction_trigger BECAUSE context is full, so + // refusing the turn that shrinks the context would deadlock the client against the very + // limit this gate reports — it would be told to compact and then denied the compaction. + if (parsed._compactionRequest !== true) { + const inputAdmission = checkInputAdmission(parsed, route.provider, route.providerName, parsed.modelId, nativeContextLimits(config)); + if (!inputAdmission.admitted) { + // #1524: this is a LOCAL preflight refusal, not an upstream verdict. A policy or combo + // fallback must be able to skip this candidate and try one whose context window fits, + // instead of treating the first incompatible candidate as the end of the chain. The + // distinct code is what lets the fallback layer tell the two apart -- an upstream + // `context_length_exceeded` still stops, because retrying it elsewhere is guesswork. + if (clientRequestedStream && !options.comboAttempt) { + return streamingContextOverflowResponse( + parsed._responseModelId ?? parsed.modelId, + translatorBudget, + ); + } + return formatErrorResponse( + 413, + "input_admission_refused", + `Estimated input (~${inputAdmission.estimatedTokens} tokens) is far past the context window ` + + `of ${parsed.modelId} (${inputAdmission.ceiling} tokens). Start a new session or choose a ` + + `model with a larger context window.`, + ); + } + } + const preAuthHostKey = preAuthUpstreamHostCircuitKey(route, config); + if (preAuthHostKey) { + const admission = acquireUpstreamHostAdmission( + preAuthHostKey, + config.upstreamHostCircuitThreshold, + ); + if (admission.kind === "blocked") { + return upstreamHostCircuitOpenResponse(admission.retryAfterSeconds); + } + admissionState.pendingHostAdmissionLease = admission.lease; + } + + let substituteMainCredential = false; + let callerAuthHeaders: Headers; + { + const finalAuth = await resolveResponsesCodexAuth(req, config, route, options, credentialDomainWasRewritten); + if (!finalAuth.ok) return finalAuth.response; + admissionState.authCtx = finalAuth.authCtx; + selectedForwardHeaders = withClaudeNativeSession(finalAuth.headers, route.provider, options.claudeNativeSessionId); + callerAuthHeaders = withClaudeNativeSession(finalAuth.callerAuthHeaders, route.provider, options.claudeNativeSessionId); + substituteMainCredential = finalAuth.substituteMainCredential; + } + + route.provider = applyCodexAuthContextToProvider(route.provider, admissionState.authCtx, route.codexAccountMode); + applyCodexAccountGatedWireNormalization(parsed, route, logCtx); + logCtx.provider = route.codexAccountNamespace + ? `${route.providerName}-${route.codexAccountNamespace}` + : formatCodexProviderForLog(route.providerName, codexLogAccountId(admissionState.authCtx), config); + logCtx.accountLogLabel = codexAuthContextLogLabel(admissionState.authCtx, config); + // A move is the expensive event: it discards the prefix warmed on the previous account. Record + // it as an event with its cause, so the operator reads it off one line instead of inferring it + // from account labels across many (#4546). + if (admissionState.authCtx.kind === "pool" && admissionState.authCtx.affinityDecision) { + logCtx.affinity = admissionState.authCtx.affinityDecision.move; + logCtx.affinityReason = admissionState.authCtx.affinityDecision.reason; + } + { + const binding = conversationStateBindingFromAuth(admissionState.authCtx, poolAffinityKey); + if (binding) { + applyAccountChangeConversationStateScrub({ + body: parsed._rawBody, + parsed, + bindingKey: binding.bindingKey, + servingAccountId: binding.accountId, + logCtx, + }); + } + } + // Seed an account-derived scope before final adapter binding. Cursor never treats it as + // authoritative: bindRouteReasoningReplayScope replaces it with the exact route owner or a + // per-request fail-closed sentinel after the final provider and credential are known. + const identityScope = codexLogAccountId(admissionState.authCtx); + if (identityScope) parsed._cursorIdentityScope = identityScope; + subagentFallbackAccountId = admissionState.authCtx.kind === "pool" || admissionState.authCtx.kind === "main-pool" + ? admissionState.authCtx.accountId + : config.activeCodexAccountId ?? null; + + return { + inboundWire, + translatorBudget, + parsed, + toolBridgeMaps, + responseStateOptions, + rememberKiroDeliveredFinalAnswer, + route, + get selectedForwardHeaders(): typeof selectedForwardHeaders { + return selectedForwardHeaders; + }, + set selectedForwardHeaders(value: typeof selectedForwardHeaders) { + selectedForwardHeaders = value; + }, + get subagentFallbackAccountId(): typeof subagentFallbackAccountId { + return subagentFallbackAccountId; + }, + set subagentFallbackAccountId(value: typeof subagentFallbackAccountId) { + subagentFallbackAccountId = value; + }, + subagentQuotaFailureModel, + poolAffinityKey, + clientRequestedStream, + substituteMainCredential, + callerAuthHeaders, + }; +} + +export type PreparedResponsesRequest = Exclude>, Response>; diff --git a/src/server/responses/request-send-budget.ts b/src/server/responses/request-send-budget.ts new file mode 100644 index 0000000000..c5879e106f --- /dev/null +++ b/src/server/responses/request-send-budget.ts @@ -0,0 +1,164 @@ +import type { ResponsesRequestContext } from "./core-options"; +import { createRequestExecutionBudget, isRequestExecutionBudget } from "../../lib/request-execution-budget"; +import { chargeWorkflowSends, workflowSendCeilingReached } from "../../lib/workflow-budget"; +import { workflowRefusalResponse } from "../workflow-refusal"; +import type { AttemptRecoveryKind } from "../../usage/log"; +import { noteAttemptSend } from "../request-log"; +import { TRANSIENT_RETRY_MAX_ATTEMPTS } from "../../lib/upstream-retry"; +import type { SingleUseDispatchPermit, SendClass } from "../../lib/request-execution-budget"; + +/** Owns the shared request send counter and recovery permits. */ +export function createResponsesSendBudget( + requestContext: Pick, +) { + const { options, req, logCtx } = requestContext; + + + // One transient-retry budget for the whole LOGICAL request, read ABOVE the passthrough branch + // so that branch shares it too. It used to be a local declared below, which put it in the + // temporal dead zone for the passthrough sends and left each recovery leg taking the helper's + // fresh default of 3. It is now a holder carried on options, so a combo child inherits the + // parent's spend instead of starting over per target -- both halves of the measured + // amplification in #4546. + const sendBudget = options.sendBudget ?? createRequestExecutionBudget(); + // The root workflow is the user-visible task. A per-request cap cannot bound a fan-out that + // sends once per child seven hundred times, so every send charged to the request is charged + // to the root as well (#4546). + const workflowRootId = req.headers.get("x-codex-parent-thread-id")?.trim() || undefined; + const noteTransientSends = (used: number): void => { + const charged = Math.max(0, used); + sendBudget.used += charged; + chargeWorkflowSends(workflowRootId, charged); + }; + // Refused before any dispatch, and deliberately not by evicting the root's ledger entry: + // dropping the record to make room would hand the fan-out a fresh allowance, which is the + // laundering this ceiling exists to stop. The client is told the task needs a new grant + // rather than being given a synthetic upstream error. + if (workflowSendCeilingReached(workflowRootId)) { + // A log context exists here, unlike at HTTP admission, so the row this request writes is + // marked synthetic rather than reading as a request that vanished with zero sends. + return workflowRefusalResponse("workflow-sends-exhausted", logCtx, undefined, workflowRootId); + } + // No floor. Math.max(1, ...) meant an exhausted request still funded one send on every + // recovery leg, so a bounded per-leg allowance never became a bounded per-request one. + const remainingTransientSendBudget = (budget: number): number => + isRequestExecutionBudget(sendBudget) + ? sendBudget.remainingBaseSends(budget) + : Math.max(0, budget - sendBudget.used); + // The adapter contract needs the full budget, not just the counter. options.sendBudget is + // typed as the narrow holder so a caller that predates this can still pass one, so narrow it + // once here rather than asserting at each adapter call site. + const adapterSendBudget = isRequestExecutionBudget(sendBudget) ? sendBudget : undefined; + /** + * Records an adapter's OWN inner retries against this attempt. + * + * Ordinal 1 is the send each call site already recorded through `noteAttemptSend`, so only + * the extra physical sends are added here and an adapter that does not retry internally + * leaves its log byte-for-byte as it was. Kiro reaches roughly eighteen sends per call and + * Cursor re-sends a whole turn, and both reported one; a count that cannot be observed + * cannot be pinned by a regression, which is why the instrumentation precedes the cap. + */ + const noteAdapterPhysicalSend = ( + inputTokens: number | undefined, + send: { ordinal: number; recovery?: AttemptRecoveryKind }, + ): void => { + if (send.ordinal <= 1) return; + noteAttemptSend(logCtx.activeAttempt, inputTokens, send.recovery); + }; + const sendBudgetExhausted = (): boolean => + remainingTransientSendBudget(TRANSIENT_RETRY_MAX_ATTEMPTS) === 0; + /** + * A credential hop reserves the send its own replay will make, and that replay is a recovery + * leg. The leg must SPEND the hop's reservation instead of taking a second one: the + * final-recovery reserve is single, so a rebuild that reserved on top of a hop would be + * refused and the request would answer with a synthetic 502 in place of the real 429 the hop + * was recovering from. + */ + let pendingHopPermit: SingleUseDispatchPermit | undefined; + /** + * How many sends a recovery leg may make, and the permit that authorises the last one. + * + * The base allowance is spent first. Once it is gone a recovery class may still draw the + * single shared final-recovery reserve -- which is what keeps the validated sanitized rebuild + * after a 5xx streak alive at four total sends -- but an account move and a rebuild cannot + * each take one. `countedExternally` is set because these legs run through the retry helper, + * which reports the same send again through `onSendsConsumed`. + */ + const recoverySendAllowance = ( + cap: number, + sendClass: SendClass, + targetKey: string, + ): { attempts: number; permit?: SingleUseDispatchPermit } => { + const base = remainingTransientSendBudget(cap); + if (base > 0) return { attempts: base }; + if (pendingHopPermit) { + const hopPermit = pendingHopPermit; + pendingHopPermit = undefined; + return { attempts: 1, permit: hopPermit }; + } + if (!isRequestExecutionBudget(sendBudget)) return { attempts: 0 }; + const decision = sendBudget.reserveDispatch({ sendClass, targetKey, countedExternally: true }); + return decision.allowed ? { attempts: 1, permit: decision.permit } : { attempts: 0 }; + }; + /** + * One credential hop of this logical request, admitted by the INTERSECTION of two bounds. + * + * `GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST` and `ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST` + * stay exactly as they are: they bound rotation within one credential roster. What neither + * can see is everything else this request already sent, so three hops layered on a spent + * budget still reached upstream three more times. A hop now happens only when its own layer + * cap AND the shared budget both permit it, and the smaller of the two wins. + * + * `countedExternally` is for the hops whose replay goes out through the retry helper, which + * reports the same physical send through `onSendsConsumed`; the others are charged here and + * nowhere else. A refusal is not an error: the caller keeps the real upstream response -- + * status, `Retry-After`, quota body -- because return-the-last-answer is the exhaustion + * contract this unit settled on. + */ + /** + * A credential rotation inside ONE provider's roster is "auth-recovery", not + * "account-failover". The distinction is load-bearing: "account-failover" sets + * `isAlternateTarget` unconditionally, so under `maxAlternateTargetSends: 1` the first + * rotation would refuse every later one AND consume the single slot a genuine cross-pool + * move needs -- a roster whose first two accounts are both 429'd would return the 429 + * while a free third account sat unused. The roster cap bounds how far rotation walks; + * the shared total bounds how many sends the request makes. Reserve "account-failover" + * for a real move between pools. + */ + const reserveCredentialHop = ( + sendClass: SendClass, + targetKey: string, + countedExternally = false, + ): { allowed: boolean; permit?: SingleUseDispatchPermit } => { + if (!isRequestExecutionBudget(sendBudget)) return { allowed: true }; + const decision = sendBudget.reserveDispatch({ sendClass, targetKey, countedExternally }); + return decision.allowed ? { allowed: true, permit: decision.permit } : { allowed: false }; + }; + /** + * Both classes share the one reserve, so this only changes what the decision is called -- + * but a recovery event that says "repair" when a credential refresh drove it is the kind of + * mislabelled evidence #4592 existed to stop. + */ + const recoveryClassFor = (recovery: AttemptRecoveryKind): SendClass => + /401|429|oauth|rate-limit|key/.test(recovery) ? "auth-recovery" : "repair"; + + return { + workflowRootId, + noteTransientSends, + remainingTransientSendBudget, + adapterSendBudget, + noteAdapterPhysicalSend, + sendBudgetExhausted, + get pendingHopPermit(): SingleUseDispatchPermit | undefined { + return pendingHopPermit; + }, + set pendingHopPermit(value: SingleUseDispatchPermit | undefined) { + pendingHopPermit = value; + }, + recoverySendAllowance, + reserveCredentialHop, + recoveryClassFor, + }; +} + +export type ResponsesSendBudget = Exclude, Response>; diff --git a/src/server/responses/request-sidecar-auth.ts b/src/server/responses/request-sidecar-auth.ts new file mode 100644 index 0000000000..84989d46bb --- /dev/null +++ b/src/server/responses/request-sidecar-auth.ts @@ -0,0 +1,149 @@ +import type { ResponsesRequestContext } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResolvedOpenAiForwardSidecar } from "../../providers/openai-sidecar"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import { + shouldResolveOpenAiVisionSidecar, + resolveOpenAiVisionModel, + planVisionSidecar, + describeImagesInPlace, + requiresVisionPreprocessing, + stripImagesInPlace, +} from "../../vision"; +import { shouldResolveOpenAiWebSearchSidecar } from "../../web-search"; +import { shouldResolveOpenAiPassthroughWebSearchBridge } from "../../web-search/passthrough-bridge"; +import { + listOpenAiForwardSidecarCandidates, + captureExplicitOpenAiCallerAuth, + resolveFirstUsableOpenAiSidecar, +} from "../../providers/openai-sidecar"; +import { + tryClaimNativeMainProfileForTurn as tryClaimStoredSidecarMainProfile, +} from "../../codex/native-main-admission"; +import { codexAccountSelectionForTurn } from "../lifecycle"; +import { + CodexPoolAuthenticationError, + CodexAuthContextError, + CodexAccountCooldownError, + CodexThreadAffinityExpiredError, + CodexMainProfileDrainingError, +} from "../../codex/auth-context"; + +/** One responsibility of the Responses request pipeline; state owners are explicit. */ +export async function prepareResponsesSidecarAuth( + requestContext: Pick, + requestState: Pick< + PreparedResponsesRequest, + | "parsed" + | "route" + | "selectedForwardHeaders" + | "translatorBudget" + >, + transportState: Pick, +) { + const { options, config, req } = requestContext; + const { parsed, route, translatorBudget } = requestState; + const { isPassthrough } = transportState; + + + let openAiSidecar: ResolvedOpenAiForwardSidecar | undefined; + const visionDescribeTerminal = options.visionDescribeTerminal === true; + const routedCompaction = parsed._compactionRequest === true + && !isCanonicalOpenAiForwardProvider(route.provider); + const needsOpenAiVision = !visionDescribeTerminal + && shouldResolveOpenAiVisionSidecar(config, route.provider, route.modelId, parsed, route.providerName); + const needsOpenAiSearch = !routedCompaction && !transportState.adapter.runTurn + && (shouldResolveOpenAiWebSearchSidecar(config, parsed, isPassthrough) + || shouldResolveOpenAiPassthroughWebSearchBridge(route.provider, parsed, isPassthrough)); + if (needsOpenAiVision || needsOpenAiSearch) { + try { + const candidates = listOpenAiForwardSidecarCandidates(config); + let sidecarAuth = options.openAiSidecarAuth; + if (!sidecarAuth && options.allowStoredOpenAiSidecarAuth === true + && route.codexAccountId === undefined + && candidates.some(candidate => candidate.accountMode === "direct") + && tryClaimStoredSidecarMainProfile(options.turnAdmissionLease)) { + // Request-local helper authority only: never promote this pair to caller, primary, + // or retry credentials. Claim before reading so profile switches remain fenced. + try { + const { getMainAccountToken } = await import("../../codex/main-account"); + const token = getMainAccountToken(); + if (token) sidecarAuth = captureExplicitOpenAiCallerAuth(new Headers({ + authorization: `Bearer ${token.accessToken}`, "chatgpt-account-id": token.chatgptAccountId, + }), config); + } catch { /* stored enrichment is optional */ } + } + // Preserve explicit OpenAI helper auth across route changes without returning it to + // primary-provider headers or alternate-main retry. The resolver revalidates scope. + const sidecarHeaders = new Headers(req.headers); + sidecarHeaders.delete("authorization"); + sidecarHeaders.delete("chatgpt-account-id"); + if (sidecarAuth) { + sidecarHeaders.set("authorization", sidecarAuth.authorization); + sidecarHeaders.set("chatgpt-account-id", sidecarAuth.chatgptAccountId); + } + openAiSidecar = await resolveFirstUsableOpenAiSidecar( + candidates, + sidecarHeaders, + config, + { + admission: options.admission, + codexAuthPolicy: options.codexAuthPolicy, + // Account-qualified native routes are passthrough, so their in-turn helper is vision. + // Scope its cooldown and outcome to the helper model, not the routed text model. + ...(route.codexAccountId !== undefined + ? { exactAccount: { accountId: route.codexAccountId, modelId: resolveOpenAiVisionModel(config) } } + : {}), + beginCodexAccountSelection: codexAccountSelectionForTurn(options.turnAdmissionLease), + }, + ); + } catch (err) { + // Sidecars are optional helpers for an otherwise independent routed turn. + // An unavailable/cooling/expired Multi credential disables the helper; it + // must not turn a valid routed-provider request into a Codex-auth failure. + if ( + !(err instanceof CodexPoolAuthenticationError) + && !(err instanceof CodexAuthContextError) + && !(err instanceof CodexAccountCooldownError) + && !(err instanceof CodexThreadAffinityExpiredError) + && !(err instanceof CodexMainProfileDrainingError) + ) throw err; + } + } + + // Vision sidecar: the routed model can't see images (provider.noVisionModels). Describe each + // attached image through the selected sidecar backend and replace it with text BEFORE the main + // call, so the text-only model can reason about it. + // Terminal describe fence (roadmap 180): the sidecar's OWN loopback describe + // call must never plan another describe. The flag arrives from the Chat + // surface (whose bridge rebuilds headers) or as the raw header for native + // Responses callers. Marked + text-only routed model → strip, depth cap 1. + const visionPlan = visionDescribeTerminal + ? undefined + : planVisionSidecar(config, route.provider, route.modelId, parsed, openAiSidecar, { + admission: options.admission, codexAuthPolicy: options.codexAuthPolicy, providerName: route.providerName, + }); + const recordSidecarOutcome = openAiSidecar?.recordOutcome; + if (visionPlan) { + await describeImagesInPlace( + parsed, + visionPlan, + openAiSidecar?.headers ?? requestState.selectedForwardHeaders, + options.abortSignal, + recordSidecarOutcome, + translatorBudget, + ); + } else if (requiresVisionPreprocessing(config, route.provider, route.modelId, route.providerName)) { + // Image capability is not positively proven but no sidecar plan is dispatchable: fail closed. + // Never forward raw image bytes to an unverified upstream. + stripImagesInPlace(parsed, translatorBudget); + } + + return { + openAiSidecar, + routedCompaction, + }; +} + +export type ResponsesSidecarAuth = Exclude>, Response>; diff --git a/src/server/responses/request-transport.ts b/src/server/responses/request-transport.ts new file mode 100644 index 0000000000..67f863858f --- /dev/null +++ b/src/server/responses/request-transport.ts @@ -0,0 +1,752 @@ +import type { ResponsesRequestContext, ResponsesAdmissionState } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { OAuthAccessSnapshot } from "../../oauth"; +import { + captureOAuthAccountSelection, + commitOAuthAccountSelection, + getAccountCredentialWithStatus, + credentialGeneration, +} from "../../oauth/store"; +import type { ProviderAdapter, AdapterRequest } from "../../adapters/base"; +import type { OcxParsedRequest, OcxProviderConfig } from "../../types"; +import type { AnthropicAccountSelectionReason } from "../../oauth/anthropic-routing"; +import { + isAnthropicAccountPoolEnabled, + getAnthropicPoolAccessSnapshot, + commitAnthropicSelectionRouting, + formatAnthropicProviderForLog, + anthropicSessionKeyFromParts, + resolveAnthropicAccountForSession, + getAnthropicPoolRetryAfterSeconds, + hasAnthropicFailoverQuorum, +} from "../../oauth/anthropic-routing"; +import { + getValidAccessSnapshotForAccount, + forceRefreshOAuthAccessSnapshot, + getValidAccessTokenSnapshot, + publicOAuthAuthenticationErrorMessage, + UnsupportedOAuthProviderError, +} from "../../oauth"; +import { + forgetGenericFailoverRoster, + isGenericFailoverProvider, + preferredInitialAccount, + noteGenericPoolSelection, +} from "../../oauth/generic-account-failover"; +import { stampOAuthAccountLabel } from "../../providers/label"; +import { resolveProviderTransport } from "../../providers/xai-transport"; +import { resolveCopilotApiBaseUrl } from "../../oauth/github-copilot"; +import { + providerApiKeySelectionIsCurrent, + resolveCurrentProviderApiKeyTransport, +} from "../../providers/api-key-selection"; +import { resolveAdapter, resolveWireProtocolOverride } from "../adapter-resolve"; +import { providerFetch } from "./fetch-helpers"; +import type { ProviderFetchOptions } from "./fetch-helpers"; +import { captureConfigGeneration } from "../../lib/state-store-sweeper"; +import { recordAnthropicAccountQuotaFromHeaders, hasPassiveAccountQuota } from "../../providers/quota"; +import { checkOutboundBodySize, describeOutboundBodyRefusal } from "./outbound-body-guard"; +import { formatErrorResponse } from "../../bridge"; +import { bindRouteReasoningReplayScope } from "./core-replay"; +import { sessionIdHeaderFromRequest, normalizeLogConversationId } from "../request-log-conversation"; +import { redactSecretString } from "../../lib/redact"; +import { selectProactiveApiKeyTransport } from "../../providers/key-failover"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import { providerConsumesCallerAuthorization } from "../../providers/caller-authorization"; +import { releaseCodexAuthContextProbeLease, stripCodexRuntimeProviderFields } from "../../codex/auth-context"; +import { + beginRequestAttempt, + sealRequestAttemptIdentity, + recordAttemptCredentialSource, + recordAdapterTierMetadata, +} from "../request-log"; +import { resolvePassiveRouteSubjectId } from "../passive-route-linker"; + +/** Owns live credential selection and adapter bindings for one request. */ +export async function prepareResponsesTransport( + requestContext: Pick, + admissionState: ResponsesAdmissionState, + requestState: Pick< + PreparedResponsesRequest, + | "route" + | "parsed" + | "inboundWire" + | "selectedForwardHeaders" + | "translatorBudget" + >, +) { + const { config, logCtx, options, req } = requestContext; + const { route, parsed, inboundWire, translatorBudget } = requestState; + + + // OAuth providers: swap in a fresh access token (auto-refreshed) as the Bearer key, so the + // existing openai-chat / anthropic adapters authenticate with no change. + const isOAuth401ReplayProvider = ( + route.providerName === "xai" + || route.providerName === "github-copilot" + || route.providerName === "kiro" + || route.providerName === "google-antigravity" + || route.providerName === "orcarouter-oauth" + ) && route.provider.authMode === "oauth"; + let sentOAuthSnapshot: OAuthAccessSnapshot | undefined; + let replayOAuthCredentialSnapshot: Pick | undefined; + let anthropicPoolAccountId: string | null = null; + let anthropicPoolFailovers = 0; + // Generic OAuth rotation (#2568) for providers with no pool of their own. Bound to the account + // the request actually used, so a concurrent rotation cannot cool an innocent replacement. + let genericFailoverAccountId: string | null = null; + let genericFailovers = 0; + let oauthSelection = route.provider.authMode === "oauth" + ? captureOAuthAccountSelection(route.providerName) : null; + let servingOAuthSnapshot: OAuthAccessSnapshot | undefined; + // These owners also serve early passthrough and sidecar sends. A dispatch-time + // rebuild must update every later builder, without entering a later block's TDZ. + let adapter: ProviderAdapter; + let activeAdapter: ProviderAdapter; + let runTurnAdapter: ProviderAdapter; + let sameTargetRequest: AdapterRequest | undefined; + let sameTargetParsed: OcxParsedRequest | undefined; + let sameTargetToken = 0; + let transportToken = 0; + let imageTierBias = 0; + const invalidateSameTargetRequest = (): void => { transportToken += 1; }; + type DispatchBinding = + | { kind: "oauth"; selection: NonNullable; snapshot: OAuthAccessSnapshot } + | { kind: "api-key"; provider: OcxProviderConfig }; + const requestBindings = new WeakMap(); + const adapterBindings = new WeakMap(); + const rawRunTurns = new WeakMap>(); + const commitResolvedOAuthSelection = async ( + candidate: OAuthAccessSnapshot, + proactive = false, + anthropicReason?: AnthropicAccountSelectionReason, + ): Promise => { + const maxSelectionAttempts = 3; + for (let attempt = 0; attempt < maxSelectionAttempts; attempt++) { + if (!oauthSelection) return null; + const proactiveEnabled = route.providerName === "anthropic" + ? isAnthropicAccountPoolEnabled(config) + : (config.providers[route.providerName]?.oauthAccountFailover?.enabled + ?? config.oauthAccountFailover?.enabled) === true; + if (proactive && candidate.accountId !== oauthSelection.accountId && !proactiveEnabled) { + oauthSelection = captureOAuthAccountSelection(route.providerName); + if (!oauthSelection) return null; + candidate = route.providerName === "anthropic" + ? await getAnthropicPoolAccessSnapshot(oauthSelection.accountId) + : await getValidAccessSnapshotForAccount(route.providerName, oauthSelection.accountId, { requireUsableAccount: true }); + } + const committed = await commitOAuthAccountSelection(route.providerName, candidate.accountId, { + expectedSelection: oauthSelection, + expectedCredentialGeneration: candidate.generation, + requireUsableAccount: true, + }); + if (committed) { + if (route.providerName === "anthropic" && !commitAnthropicSelectionRouting( + candidate.accountId, oauthSelection, committed, + { config, sessionKey: anthropicSessionKey, reason: anthropicReason, expectedCredentialGeneration: candidate.generation }, + )) return null; + oauthSelection = committed; + servingOAuthSnapshot = candidate; + forgetGenericFailoverRoster(route.providerName); + return candidate; + } + // A newer manual choice wins over this request's old proposal, including A→B→A. + // Resolve that choice, not the rejected candidate, before trying admission again. + oauthSelection = captureOAuthAccountSelection(route.providerName); + if (!oauthSelection) return null; + candidate = route.providerName === "anthropic" + ? await getAnthropicPoolAccessSnapshot(oauthSelection.accountId) + : await getValidAccessSnapshotForAccount(route.providerName, oauthSelection.accountId, { requireUsableAccount: true }); + if (route.provider.googleMode === "cloud-code-assist" && !candidate.projectId) return null; + } + return null; + }; + const refreshResolvedOAuthSelection = async (sent: OAuthAccessSnapshot): Promise => { + const current = captureOAuthAccountSelection(route.providerName); + const unchanged = current?.accountId === oauthSelection?.accountId + && current?.revision === oauthSelection?.revision; + const candidate = unchanged ? await forceRefreshOAuthAccessSnapshot(sent) : sent; + const admitted = await commitResolvedOAuthSelection(candidate); + if (!admitted) throw new Error("OAuth selection changed during credential recovery"); + genericFailoverAccountId = admitted.accountId; + stampOAuthAccountLabel(logCtx, route.providerName, route.provider, admitted.accountId); + return admitted; + }; + /** + * Config generation captured where the serving credential is RESOLVED, not where the + * quota is written. A streaming turn is a long await, so a generation captured at write + * time cannot see a config or account change that happened earlier in the same turn — + * the case the fence exists for. Stays 0 for every provider without a passive quota. + */ + let passiveQuotaWriterGeneration = 0; + /** + * Apply a rotated account's FULL credential snapshot to the live route (#2568d). + * + * One helper for all three rotation sites on purpose. Each site used to inline the same four + * lines, and the divergence that produced was the bug: `apiKey` was swapped while the routing + * metadata paired with it stayed behind. + * + * Returns false when the snapshot cannot be used safely, and the caller must then abandon the + * rotation rather than send a half-applied identity: + * + * - Copilot pins its bearer to an account-scoped regional origin, so transport is re-resolved + * with the new account's `apiBaseUrl` instead of inheriting the previous account's host. The + * snapshot value is RESOLVED first: `rotatedProvider` is a clone of the FAILED account's + * provider, so passing a bare `undefined` origin let the transport resolver fall through its + * own `?? validateCopilotApiBaseUrl(provider.baseUrl)` step to the previous account's host — + * pairing B's bearer with A's accepted origin. Login and refresh always persist a resolved + * origin, so this fallback protects malformed or manually seeded credentials. + * - A Cloud Code Assist provider needs an account-matched project. Antigravity's refresh path + * tolerates project discovery failing, so a stored account can legitimately have no project; + * sending that account's bearer with the FAILED account's project is worse than not rotating. + */ + const applyFailoverSnapshot = async ( + snapshot: OAuthAccessSnapshot, + retryParsed: OcxParsedRequest = parsed, + ): Promise => { + if (route.provider.googleMode === "cloud-code-assist" && !snapshot.projectId) return false; + const committed = await commitResolvedOAuthSelection(snapshot); + if (!committed) return false; + snapshot = committed; + let rotatedProvider: OcxProviderConfig = { ...route.provider, apiKey: snapshot.accessToken }; + if (route.providerName === "github-copilot") { + rotatedProvider = resolveProviderTransport( + route.providerName, + rotatedProvider, + parsed.options.promptCacheKey, + resolveCopilotApiBaseUrl(snapshot.apiBaseUrl), + ) as OcxProviderConfig; + } + if (snapshot.projectId) rotatedProvider = { ...rotatedProvider, project: snapshot.projectId }; + route.provider = rotatedProvider; + if (route.providerName === "kiro") { + const kiroContext = { ...(snapshot.kiro ?? {}) }; + // Terminal-guard continuations are rebuilt from a shallow clone. Updating only the + // outer request pairs the new bearer with the failed account's region/profile on + // the retry. Keep both owners synchronized; for ordinary paths they are identical. + parsed._kiroAuthContext = kiroContext; + if (retryParsed !== parsed) retryParsed._kiroAuthContext = { ...kiroContext }; + } + // Re-stamp: a request that rotated accounts must be attributed to the account that actually + // served it. All three rotation sites funnel through here, so this is the only re-stamp + // needed -- and putting it anywhere else would let one of the three drift. + stampOAuthAccountLabel(logCtx, route.providerName, route.provider, snapshot.accountId); + if (route.providerName === "anthropic") { + anthropicPoolAccountId = snapshot.accountId; + logCtx.provider = formatAnthropicProviderForLog("anthropic", snapshot.accountId, config); + } else { + genericFailoverAccountId = snapshot.accountId; + } + sentOAuthSnapshot = snapshot; + replayOAuthCredentialSnapshot = { accountId: snapshot.accountId, generation: snapshot.generation }; + return true; + }; + const selectionIsCurrent = (binding: DispatchBinding | undefined): boolean => { + if (route.provider.authMode === "forward") return true; + if (!binding) return false; + if (binding.kind === "api-key") return providerApiKeySelectionIsCurrent(config, route.providerName, binding.provider); + const selected = captureOAuthAccountSelection(route.providerName); + const row = getAccountCredentialWithStatus(route.providerName, binding.snapshot.accountId); + return selected?.accountId === binding.selection.accountId && selected?.revision === binding.selection.revision + && !!row && !row.needsReauth && row.credential.expires > Date.now() + && credentialGeneration(row.credential) === binding.snapshot.generation; + }; + const resolveSelectionAdapter = (provider: OcxProviderConfig, retention = config.cacheRetention): ProviderAdapter => { + const resolved = resolveAdapter(provider, retention, route.providerName); + if (route.provider.authMode === "forward") return resolved; + const binding: DispatchBinding | undefined = route.provider.authMode === "oauth" + ? oauthSelection && servingOAuthSnapshot + ? { kind: "oauth", selection: { ...oauthSelection }, snapshot: servingOAuthSnapshot } + : undefined + : { kind: "api-key", provider: { ...route.provider } }; + if (binding) adapterBindings.set(resolved, binding); + const build = resolved.buildRequest.bind(resolved); + resolved.buildRequest = async (requestParsed, incoming) => { + const request = await build(requestParsed, incoming); + // Capture at adapter creation, never from mutable serving state after an await. + if (binding) requestBindings.set(request, binding); + return request; + }; + if (resolved.runTurn) { + rawRunTurns.set(resolved, resolved.runTurn.bind(resolved)); + resolved.runTurn = (requestParsed, incoming, emit) => runSelectedTurn(resolved, requestParsed, incoming, emit); + } + return resolved; + }; + const refreshDispatchAdapter = async (requestParsed: OcxParsedRequest): Promise => { + if (route.provider.authMode === "oauth") { + if (!servingOAuthSnapshot || !await applyFailoverSnapshot(servingOAuthSnapshot, requestParsed)) { + throw new Error("OAuth account selection changed before dispatch"); + } + } else { + const current = resolveCurrentProviderApiKeyTransport(config, route.providerName, route.provider); + if (!current) throw new Error("API key selection is unavailable before dispatch"); + route.provider = current; + } + adapter = activeAdapter = runTurnAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + ); + invalidateSameTargetRequest(); + return adapter; + }; + const refreshRunTurnAdapter = async (requestParsed: OcxParsedRequest): Promise => { + requestParsed._cursorIdentityScope = undefined; + requestParsed._cursorConversationId = undefined; + if (requestParsed._providerContinuation?.cursor) { + const { cursor: _oldCursor, ...rest } = requestParsed._providerContinuation; + requestParsed._providerContinuation = rest; + } + return refreshDispatchAdapter(requestParsed); + }; + const runSelectedTurn = async ( + selectedAdapter: ProviderAdapter, + ...[requestParsed, incoming, emit]: Parameters> + ): Promise => { + for (let attempt = 0; attempt < 3; attempt++) { + if (!selectionIsCurrent(adapterBindings.get(selectedAdapter))) selectedAdapter = await refreshRunTurnAdapter(requestParsed); + const binding = adapterBindings.get(selectedAdapter); + const run = rawRunTurns.get(selectedAdapter); + if (!run) throw new Error("Selected provider no longer supports this turn transport"); + let sent = false; + let refused = false; + // Both main and image-loop callers already acquired the initial pacing slot. + // Subsequent physical messages retain this adapter/credential and are paced normally. + const fetch = providerFetch(route.provider, options.codexWsRuntimeIdentity, { + providerName: route.providerName, modelId: route.modelId, pacingSlotAcquired: true, + beforeDispatch: () => { + if (sent) return; + if (!selectionIsCurrent(binding)) { + refused = true; + throw new Error("Account selection changed before the first turn dispatch"); + } + sent = true; + }, + }); + try { + await run(requestParsed, { ...incoming, providerFetch: fetch }, event => { if (!refused) emit(event); }); + } catch (error) { + if (!refused) throw error; + } + if (!refused) return; + // The adapter may map the guard's exception to an error event. Neither that + // event nor a refused send may escape before retrying the newly selected account. + selectedAdapter = await refreshRunTurnAdapter(requestParsed); + } + throw new Error("Account selection changed repeatedly before turn dispatch"); + }; + const oauthDispatch = (wireRequest: AdapterRequest, requestParsed = parsed): ProviderFetchOptions["dispatchOverride"] => { + if (route.provider.authMode === "forward") return undefined; + return async (input, init, execute) => { + let destination = input; + let dispatchInit = init; + for (let attempt = 0; attempt < 3; attempt++) { + if (selectionIsCurrent(requestBindings.get(wireRequest))) { + const fetchImpl = (route.provider as OcxProviderConfig & { fetch?: typeof globalThis.fetch }).fetch ?? execute; + const binding = requestBindings.get(wireRequest); + const snapshot = route.providerName === "anthropic" && anthropicPoolAccountId && binding?.kind === "oauth" + ? binding.snapshot : undefined; + const writerGeneration = snapshot ? captureConfigGeneration() : 0; + const sentHeaders = snapshot ? new Headers(dispatchInit.headers) : undefined; + const ownsBearer = snapshot !== undefined + && sentHeaders?.get("authorization") === `Bearer ${snapshot.accessToken}` + && !sentHeaders?.has("x-api-key"); + // Reselection can choose a provider override instead of the supplied executor. + const response = await fetchImpl(destination, { ...dispatchInit, redirect: "manual" }); + // Observe each physical response before retries replace it. The binding belongs to + // this dispatch, so a manual switch cannot file A's headers against B. Header + // overrides and credential replacement make ownership unprovable: skip those writes. + if (ownsBearer && snapshot) { + try { + const current = getAccountCredentialWithStatus("anthropic", snapshot.accountId); + if (current && !current.needsReauth && credentialGeneration(current.credential) === snapshot.generation) { + recordAnthropicAccountQuotaFromHeaders(snapshot.accountId, response.headers, writerGeneration); + } + } catch { /* best-effort observation cannot fail the response */ } + } + return response; + } + const nextAdapter = await refreshDispatchAdapter(requestParsed); + const rebuilt = await nextAdapter.buildRequest(requestParsed, { + headers: requestState.selectedForwardHeaders, translatorBudget, + ...(imageTierBias > 0 ? { imageTierBias } : {}), + }); + const bodySize = checkOutboundBodySize(rebuilt.body, config.maxUpstreamBodyBytes); + if (!bodySize.admitted) { + rebuilt.releaseBodyObservation?.(); + return formatErrorResponse(413, "outbound_body_too_large", describeOutboundBodyRefusal(bodySize)); + } + const headers = new Headers(dispatchInit.headers); + for (const name of Object.keys(wireRequest.headers)) headers.delete(name); + for (const [name, value] of Object.entries(rebuilt.headers)) headers.set(name, value); + wireRequest.releaseBodyObservation?.(); + Object.assign(wireRequest, rebuilt); + const binding = requestBindings.get(rebuilt); + if (binding) requestBindings.set(wireRequest, binding); + else requestBindings.delete(wireRequest); + sameTargetRequest = wireRequest; + sameTargetParsed = requestParsed; + sameTargetToken = transportToken; + destination = rebuilt.url; + dispatchInit = { ...dispatchInit, method: rebuilt.method, headers, body: rebuilt.body }; + bindRouteReasoningReplayScope({ parsed: requestParsed, providerName: route.providerName, provider: route.provider, + adapterName: nextAdapter.name, oauthCredentialSnapshot: replayOAuthCredentialSnapshot }); + // The next iteration validates synchronously and calls fetch in that same turn. + } + throw new Error("OAuth account selection changed repeatedly before dispatch"); + }; + }; + const anthropicSessionKey = route.providerName === "anthropic" && route.provider.authMode === "oauth" + ? anthropicSessionKeyFromParts({ + sessionIdHeader: sessionIdHeaderFromRequest(req.headers), + threadIdHeader: req.headers.get("thread-id"), + promptCacheKey: typeof parsed.options.promptCacheKey === "string" ? parsed.options.promptCacheKey : null, + clientThreadId: typeof parsed._clientThreadId === "string" ? parsed._clientThreadId : null, + promptCacheKeyIsSharedCohort: options.promptCacheKeyIsSharedCohort === true, + }) + : null; + if (route.provider.authMode === "oauth") { + try { + if (route.providerName === "anthropic" && isAnthropicAccountPoolEnabled(config)) { + const selection = resolveAnthropicAccountForSession(anthropicSessionKey, config); + if (!selection.accountId) { + if (selection.reason === "all-cooled") { + const retryAfterSec = getAnthropicPoolRetryAfterSeconds(); + return formatErrorResponse( + 429, + "rate_limit_error", + "All Anthropic OAuth accounts are temporarily rate-limited", + retryAfterSec !== null ? { retryAfter: String(retryAfterSec) } : undefined, + ); + } + return formatErrorResponse(401, "authentication_error", "No eligible Anthropic OAuth account available"); + } + const admitted = await commitResolvedOAuthSelection(await getAnthropicPoolAccessSnapshot(selection.accountId), true, selection.reason); + if (!admitted) return formatErrorResponse(409, "conflict_error", "OAuth account selection changed; retry the request"); + anthropicPoolAccountId = admitted.accountId; + route.provider = { ...route.provider, apiKey: admitted.accessToken }; + logCtx.provider = formatAnthropicProviderForLog("anthropic", admitted.accountId, config); + } else { + // Prefer the account with known headroom BEFORE the first attempt. Rotation alone + // only reacts to a 429, so a turn could open on an account a previous probe already + // measured as spent. A null answer means "use the active account", so every provider + // without quota evidence keeps the resolution it has today. + const preferredAccountId = isGenericFailoverProvider(route.providerName, route.provider) + ? preferredInitialAccount(config, route.providerName) + : null; + // Resolved account-scoped, NOT through failoverAccountSnapshot: that helper marks a + // rotation site, and rotation sites must apply their credential through + // applyFailoverSnapshot's pairing rules. This is initial resolution — the code below + // already pairs the snapshot's Kiro metadata, Copilot origin and Antigravity project + // with this same bearer, exactly as it does for the active account. + let usedPreferredAccount = preferredAccountId !== null; + let resolved: OAuthAccessSnapshot; + if (preferredAccountId) { + try { + // `requireUsableAccount` makes a removed OR reauth-flagged account throw from + // inside the resolver's own store read. Without it a revoked account resolves + // successfully — its credential is still readable — and the request would + // dispatch on an account already known to need a fresh login. + resolved = await getValidAccessSnapshotForAccount( + route.providerName, + preferredAccountId, + { requireUsableAccount: true }, + ); + } catch { + // The roster is read behind a short TTL, so a preferred account can be removed + // or flagged for reauth in the window after it was cached. Resolving it then + // throws, and a PREFERENCE that turns a healthy request into a 401 is worse + // than no preference at all — the active account is still perfectly usable. + // Drop the stale roster so the next request re-reads it, and carry on. + forgetGenericFailoverRoster(route.providerName); + usedPreferredAccount = false; + resolved = await getValidAccessTokenSnapshot(route.providerName); + } + } else { + resolved = await getValidAccessTokenSnapshot(route.providerName); + } + // A Cloud Code Assist account needs its own project. Antigravity's refresh path + // tolerates project discovery failing, so a stored account can legitimately have + // none — and a PREFERENCE must never turn a working request into an error. Fall + // back to the ordinary active-account resolution instead, which is exactly what + // would have happened had the preference never existed. + if (usedPreferredAccount && route.provider.googleMode === "cloud-code-assist" && !resolved.projectId) { + resolved = await getValidAccessTokenSnapshot(route.providerName); + usedPreferredAccount = false; + } + const admitted = await commitResolvedOAuthSelection(resolved, true); + if (!admitted) return formatErrorResponse(409, "conflict_error", "OAuth account selection changed; retry the request"); + if (admitted.accountId !== resolved.accountId) usedPreferredAccount = true; + resolved = admitted; + replayOAuthCredentialSnapshot = { + accountId: resolved.accountId, + generation: resolved.generation, + }; + if (isOAuth401ReplayProvider) sentOAuthSnapshot = resolved; + route.provider = { ...route.provider, apiKey: resolved.accessToken }; + // Attribution is independent of failover (#2699): stamped from the resolved snapshot + // itself, not from inside the `isGenericFailoverProvider` branch below, so a future + // narrowing of that predicate cannot silently switch attribution off. + stampOAuthAccountLabel(logCtx, route.providerName, route.provider, resolved.accountId); + // Remember which account actually served this request so a 429 cools THAT one, not + // whichever account is active by the time the response comes back (#2568). + if (isGenericFailoverProvider(route.providerName, route.provider)) { + genericFailoverAccountId = resolved.accountId; + // Advance the pool cursor only now that this account is actually admitted. The + // helper returns immediately unless the kernel is on AND the strategy is + // round-robin, so quota and fill-first pools reach it without being touched. + noteGenericPoolSelection(config, route.providerName, resolved.accountId); + } + // Anthropic is excluded from isGenericFailoverProvider -- its own pool owns affinity and + // a fail-closed local-cli credential rule -- so without this stamp its identity is + // dropped whenever the pool flag is off, and a later 429 has no account to cool. Reactive + // failover needs only the id: no affinity bind, no promotion, no quota-ranked pick. Those + // are proactive and stay behind anthropicAccountPool.enabled. + if (route.providerName === "anthropic" && hasAnthropicFailoverQuorum()) { + anthropicPoolAccountId = resolved.accountId; + } + // Captured beside the account it fences, so the two can never disagree. + if (hasPassiveAccountQuota(route.providerName)) { + passiveQuotaWriterGeneration = captureConfigGeneration(); + } + if (route.providerName === "kiro") { + // `{}` is intentional: this is an account-scoped request with no stored routing metadata. + // Only genuinely accountless adapter calls leave the context undefined and use local/env fallback. + parsed._kiroAuthContext = { ...(resolved.kiro ?? {}) }; + } + // Project identity belongs to the admitted account on EVERY request, including + // the request after a pool transition made that account the persisted active one. + if (route.provider.googleMode === "cloud-code-assist") { + if (!resolved.projectId) return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(new Error("Cloud Code Assist account project is unavailable"))); + route.provider = { ...route.provider, project: resolved.projectId }; + } + } + } catch (err) { + if (err instanceof UnsupportedOAuthProviderError) { + const safeProviderName = redactSecretString(route.providerName); + return formatErrorResponse( + 400, + "invalid_request_error", + `${redactSecretString(err.message)}. Remove or reconfigure provider '${safeProviderName}' in the OpenCodex configuration.`, + ); + } + return formatErrorResponse(401, "authentication_error", publicOAuthAuthenticationErrorMessage(err)); + } + } + // Key-auth twin of the OAuth preference above: pick a warm key BEFORE the first attempt when + // the committed one is already cooling, instead of spending the request earning a 429 the + // runtime could already predict. The picker refuses to override a healthy committed key and + // returns null without a configured strategy, so an ordinary install evaluates one predicate. + // + // It RETURNS a rebuilt route rather than mutating one, and the assignment has to land here -- + // ahead of the transport pin below, the adapterProvider copy that follows it, and the request + // the HTTP path bakes later. The image bridge and web search read route.provider directly and + // have no stale-selection re-read to save them, so ordering is the whole correctness argument. + // + // The Transport variant, not the bare picker: the picker answers with the PERSISTED row, and + // a built-in provider stored in its valid minimal form would lose the adapter id, base URL + // and static headers registry backfill supplies, throwing `Unknown adapter: undefined`. + const proactiveKeyProvider = selectProactiveApiKeyTransport( + config, + route.providerName, + route.provider, + parsed.options.promptCacheKey, + ); + if (proactiveKeyProvider) route.provider = proactiveKeyProvider; + route.provider = resolveProviderTransport( + route.providerName, + route.provider, + parsed.options.promptCacheKey, + route.providerName === "github-copilot" && route.provider.authMode === "oauth" + ? resolveCopilotApiBaseUrl(sentOAuthSnapshot?.apiBaseUrl) + : undefined, + ); + let adapterProvider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire); + const stripClaudeMainAuth = options.stripClaudeMainAuthForNoncanonicalForward === true + && !isCanonicalOpenAiForwardProvider(adapterProvider) + && ((adapterProvider.adapter === "openai-responses" && adapterProvider.authMode === "forward") + || providerConsumesCallerAuthorization(adapterProvider)); + if (stripClaudeMainAuth) { + releaseCodexAuthContextProbeLease(admissionState.authCtx); + admissionState.authCtx = { kind: "main", accountId: null }; + route.provider = stripCodexRuntimeProviderFields(route.provider); + adapterProvider = stripCodexRuntimeProviderFields(adapterProvider); + requestState.selectedForwardHeaders = new Headers(requestState.selectedForwardHeaders); + requestState.selectedForwardHeaders.delete("authorization"); + requestState.selectedForwardHeaders.delete("chatgpt-account-id"); + delete route.codexAccountMode; + delete route.codexAccountId; + delete route.codexAccountNamespace; + logCtx.provider = route.providerName; + delete logCtx.accountLogLabel; + } + adapter = resolveSelectionAdapter(adapterProvider, config.cacheRetention); + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: adapterProvider, + adapterName: adapter.name, + oauthCredentialSnapshot: replayOAuthCredentialSnapshot, + codexAuthContext: admissionState.authCtx, + forwardHeaders: requestState.selectedForwardHeaders, + }); + if (!logCtx.conversationId && parsed._cursorConversationId) { + logCtx.conversationId = normalizeLogConversationId(parsed._cursorConversationId); + } + logCtx.providerAdapter = adapter.name; + // Ordinary requests receive one durable attempt only after their final initial + // adapter is resolved. Combo children own their attempt and retries keep it. + if (!options.comboAttempt && !logCtx.activeAttempt) { + const attempt = beginRequestAttempt( + (logCtx.attempts?.length ?? 0) + 1, + logCtx.provider, + route.modelId, + adapter.name, + ); + logCtx.activeAttempt = attempt; + logCtx.activeAttemptStartedAt = Date.now(); + (logCtx.attempts ??= []).push(attempt); + } + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, adapter.name, logCtx.accountLogLabel); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, adapterProvider, adapter.name); + runTurnAdapter = adapter; + if (adapter.runTurn) { + recordAdapterTierMetadata(logCtx, adapter.tierLogForRunTurn?.(parsed)); + } + // Optional route-identity linkage for attempt correlation (CL-09 consumes it). The slot + // resolves to null unless an opt-in subsystem registered a linker, so an install without + // routing profiles does no work here and loads no additional module. The non-throwing + // guarantee lives in the slot helper. + if (logCtx.activeAttempt && !logCtx.activeAttempt.labRouteSubjectId) { + const passiveSubjectId = resolvePassiveRouteSubjectId( + config, + route.providerName, + route.modelId, + route.provider, + inboundWire, + ); + if (passiveSubjectId) logCtx.activeAttempt.labRouteSubjectId = passiveSubjectId; + } + const isPassthrough = "passthrough" in adapter && !!adapter.passthrough; + + const rawInput = (parsed._rawBody as { input?: unknown }).input; + if (!isPassthrough && Array.isArray(rawInput) && rawInput.some( + item => item !== null && typeof item === "object" && item.type === "computer_call_output", + )) { + return formatErrorResponse( + 400, + "invalid_request_error", + "computer_call_output requires a Responses passthrough route; send screenshots as user input_image content on translated routes.", + ); + } + + if (adapter.name === "kiro" && parsed.previousResponseId && !parsed._previousResponseInputExpanded) { + return formatErrorResponse( + 400, + "invalid_request_error", + "Kiro continuation state is missing; start a new session instead of reusing this previous_response_id.", + ); + } + + return { + isOAuth401ReplayProvider, + get sentOAuthSnapshot(): OAuthAccessSnapshot | undefined { + return sentOAuthSnapshot; + }, + set sentOAuthSnapshot(value: OAuthAccessSnapshot | undefined) { + sentOAuthSnapshot = value; + }, + get replayOAuthCredentialSnapshot(): Pick | undefined { + return replayOAuthCredentialSnapshot; + }, + set replayOAuthCredentialSnapshot(value: Pick | undefined) { + replayOAuthCredentialSnapshot = value; + }, + get anthropicPoolAccountId(): string | null { + return anthropicPoolAccountId; + }, + set anthropicPoolAccountId(value: string | null) { + anthropicPoolAccountId = value; + }, + get anthropicPoolFailovers(): typeof anthropicPoolFailovers { + return anthropicPoolFailovers; + }, + set anthropicPoolFailovers(value: typeof anthropicPoolFailovers) { + anthropicPoolFailovers = value; + }, + get genericFailoverAccountId(): string | null { + return genericFailoverAccountId; + }, + set genericFailoverAccountId(value: string | null) { + genericFailoverAccountId = value; + }, + get genericFailovers(): typeof genericFailovers { + return genericFailovers; + }, + set genericFailovers(value: typeof genericFailovers) { + genericFailovers = value; + }, + get adapter(): ProviderAdapter { + return adapter; + }, + set adapter(value: ProviderAdapter) { + adapter = value; + }, + get activeAdapter(): ProviderAdapter { + return activeAdapter; + }, + set activeAdapter(value: ProviderAdapter) { + activeAdapter = value; + }, + get runTurnAdapter(): ProviderAdapter { + return runTurnAdapter; + }, + set runTurnAdapter(value: ProviderAdapter) { + runTurnAdapter = value; + }, + get sameTargetRequest(): AdapterRequest | undefined { + return sameTargetRequest; + }, + set sameTargetRequest(value: AdapterRequest | undefined) { + sameTargetRequest = value; + }, + get sameTargetParsed(): OcxParsedRequest | undefined { + return sameTargetParsed; + }, + set sameTargetParsed(value: OcxParsedRequest | undefined) { + sameTargetParsed = value; + }, + get sameTargetToken(): typeof sameTargetToken { + return sameTargetToken; + }, + set sameTargetToken(value: typeof sameTargetToken) { + sameTargetToken = value; + }, + get transportToken(): typeof transportToken { + return transportToken; + }, + set transportToken(value: typeof transportToken) { + transportToken = value; + }, + get imageTierBias(): typeof imageTierBias { + return imageTierBias; + }, + set imageTierBias(value: typeof imageTierBias) { + imageTierBias = value; + }, + invalidateSameTargetRequest, + requestBindings, + adapterBindings, + commitResolvedOAuthSelection, + refreshResolvedOAuthSelection, + passiveQuotaWriterGeneration, + applyFailoverSnapshot, + selectionIsCurrent, + resolveSelectionAdapter, + refreshRunTurnAdapter, + oauthDispatch, + anthropicSessionKey, + isPassthrough, + }; +} + +export type ResponsesTransport = Exclude>, Response>; diff --git a/src/server/responses/response-effects.ts b/src/server/responses/response-effects.ts new file mode 100644 index 0000000000..efbe1d7026 --- /dev/null +++ b/src/server/responses/response-effects.ts @@ -0,0 +1,157 @@ +import type { ResponsesRequestContext, ResponsesAdmissionState } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesSidecarAuth } from "./request-sidecar-auth"; +import type { OcxProviderContinuationState } from "../../types"; +import { providerContinuationPayload } from "./core-replay"; +import { mergeProviderContinuationPayload } from "../../responses/provider-continuation"; +import { commitReasoningReplayServingIdentity } from "../../responses/reasoning-replay-cache"; +import { rememberServingConversationStateIssuer } from "./account-change-state"; +import { isCanonicalOpenAiForwardProvider } from "../../providers/openai-tiers"; +import { contextRelayActivated } from "../../codex/context-compat"; +import { recordContextSessionOwner } from "../../codex/context-owner"; +import { resolveContextPrincipal } from "../auth-cors"; +import { COMPACT_PROMPT } from "../../responses/compaction"; +import type { RoutedNamespaceToolAliases } from "../../responses/namespace-tool-compat"; +import type { MuseToolNameAliases } from "../../responses/muse-tool-name-alias"; +import type { AdapterRequest } from "../../adapters/base"; + +/** Owns completion callbacks, replay publication, and live tool aliases. */ +export function createResponsesEffects( + requestContext: Pick, + admissionState: ResponsesAdmissionState, + requestState: Pick< + PreparedResponsesRequest, + | "parsed" + | "poolAffinityKey" + | "route" + | "substituteMainCredential" + >, + sidecarState: Pick, +) { + const { options, req, config } = requestContext; + const { parsed, poolAffinityKey, route, substituteMainCredential } = requestState; + const { routedCompaction } = sidecarState; + + + const recordTerminalOutcomes = options.recordTerminalOutcomes !== false; + let responseCompletionNotified = false; + let responseCompletionCancelled = false; + const cancelResponseCompletion = (): void => { responseCompletionCancelled = true; }; + const notifyResponseComplete = (response: { status?: unknown; model?: unknown }): void => { + if (responseCompletionNotified || responseCompletionCancelled + || options.abortSignal?.aborted || req.signal.aborted + || response.status !== "completed" + || typeof response.model !== "string" || !response.model.trim()) return; + responseCompletionNotified = true; + options.onResponseComplete?.(response.model); + }; + + const continuationStateForResponse = ( + emitted?: OcxProviderContinuationState, + ): OcxProviderContinuationState | undefined => { + const cursorConversationId = parsed._cursorConversationId; + const inherited = providerContinuationPayload(parsed._providerContinuation); + const emittedPayload = providerContinuationPayload(emitted); + if (!emittedPayload && !inherited && !cursorConversationId) return undefined; + const merged = mergeProviderContinuationPayload( + inherited ?? {}, + emittedPayload ?? {}, + ) as OcxProviderContinuationState; + if (cursorConversationId) { + merged.cursor = { ...(merged.cursor ?? {}), conversationId: cursorConversationId }; + } + return parsed._providerContinuationOwner + ? { ...merged, __ocxOwner: { ...parsed._providerContinuationOwner } } + : merged; + }; + + // Remote compaction v2 on a ROUTED model: Codex sent `compaction_trigger` and requires exactly + // one `{type:"compaction"}` output item (codex-rs compact_remote_v2.rs). Passthrough handles it + // natively upstream; here we run the routed model as a plain summarizer — no tools, no web-search + // sidecar — and the bridge appends the synthetic compaction item (src/responses/compaction.ts). + // A Responses-shaped wire does not imply support for Codex's private + // `compaction_trigger` item — only the canonical ChatGPT backend speaks that + // contract. An API-key gateway would receive the trigger, answer with an ordinary + // message, and leave Codex fataling on a missing compaction item (#422). + const commitReasoningReplayServingRoute = (outboundHeaders?: HeadersInit): void => { + commitReasoningReplayServingIdentity(parsed._reasoningReplayScope); + rememberServingConversationStateIssuer(admissionState.authCtx, poolAffinityKey); + // History has no model namespace. Record the account that actually accepted this + // final attempt, after refresh/failover, rather than guessing from mutable affinity. + // Recording is relay state. With the feature off there is no relay, so building an owner + // registry for it is out of scope for this request. + if (outboundHeaders && isCanonicalOpenAiForwardProvider(route.provider) && contextRelayActivated()) { + recordContextSessionOwner(resolveContextPrincipal(req, config, options.admission), req.headers, + route.provider.baseUrl, admissionState.authCtx, new Headers(outboundHeaders), substituteMainCredential); + } + }; + if (routedCompaction) { + delete parsed.context.tools; + delete parsed._webSearch; + delete parsed.options.toolChoice; + delete parsed.options.parallelToolCalls; + // The compaction turn is a plain prose summary; a surviving structured-output format + // would force schema-constrained JSON into the synthetic compaction item. The flag and + // the raw `text` control go too: the key-mode openai-responses adapter builds from + // _rawBody, so a surviving format there would still reach the upstream. (The Kiro + // guard no longer reads _rawBody.text; it refuses structured output only.) + delete parsed.options.textFormat; + delete parsed._structuredOutput; + if (parsed._rawBody && typeof parsed._rawBody === "object") { + delete (parsed._rawBody as Record).text; + } + parsed.context.messages.push({ role: "user", content: COMPACT_PROMPT, timestamp: Date.now() }); + } + + let routedNamespaceToolAliases: RoutedNamespaceToolAliases = new Map(); + let plaintextV2AgentMessageToolNames: ReadonlySet = new Set(); + let plaintextV2AgentMessageAliasedToolNames: ReadonlySet = new Set(); + let routedMuseToolNameAliases: MuseToolNameAliases = new Map(); + const refreshRequestToolAliases = (builtRequest: AdapterRequest): void => { + routedNamespaceToolAliases = builtRequest.convertedRoutedNamespaceToolAliases ?? new Map(); + plaintextV2AgentMessageToolNames = builtRequest.plaintextV2AgentMessageToolNames ?? new Set(); + plaintextV2AgentMessageAliasedToolNames = builtRequest.plaintextV2AgentMessageAliasedToolNames ?? new Set(); + routedMuseToolNameAliases = builtRequest.convertedMuseToolNameAliases ?? new Map(); + }; + + return { + recordTerminalOutcomes, + get responseCompletionCancelled(): typeof responseCompletionCancelled { + return responseCompletionCancelled; + }, + set responseCompletionCancelled(value: typeof responseCompletionCancelled) { + responseCompletionCancelled = value; + }, + cancelResponseCompletion, + notifyResponseComplete, + continuationStateForResponse, + commitReasoningReplayServingRoute, + get routedNamespaceToolAliases(): RoutedNamespaceToolAliases { + return routedNamespaceToolAliases; + }, + set routedNamespaceToolAliases(value: RoutedNamespaceToolAliases) { + routedNamespaceToolAliases = value; + }, + get plaintextV2AgentMessageToolNames(): ReadonlySet { + return plaintextV2AgentMessageToolNames; + }, + set plaintextV2AgentMessageToolNames(value: ReadonlySet) { + plaintextV2AgentMessageToolNames = value; + }, + get plaintextV2AgentMessageAliasedToolNames(): ReadonlySet { + return plaintextV2AgentMessageAliasedToolNames; + }, + set plaintextV2AgentMessageAliasedToolNames(value: ReadonlySet) { + plaintextV2AgentMessageAliasedToolNames = value; + }, + get routedMuseToolNameAliases(): MuseToolNameAliases { + return routedMuseToolNameAliases; + }, + set routedMuseToolNameAliases(value: MuseToolNameAliases) { + routedMuseToolNameAliases = value; + }, + refreshRequestToolAliases, + }; +} + +export type ResponsesEffects = Exclude, Response>; diff --git a/src/server/responses/run-turn-execution.ts b/src/server/responses/run-turn-execution.ts new file mode 100644 index 0000000000..24e96802fb --- /dev/null +++ b/src/server/responses/run-turn-execution.ts @@ -0,0 +1,448 @@ +import type { ResponsesRequestContext, ResponsesAdmissionState } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResponsesSidecarAuth } from "./request-sidecar-auth"; +import type { ResponsesEffects } from "./response-effects"; +import type { ResponsesSendBudget } from "./request-send-budget"; +import type { ResponsesCompletionPolicy } from "./completion-policy"; +import { linkAbortSignal, runTurnAdapterSseResponses } from "./core-lifetime"; +import { createAdapterEventQueue, preflightAdapterEvents } from "../../adapters/run-turn-queue"; +import { + bindRouteReasoningReplayScope, + adapterNeedsForcedContinuation, + adapterResponseReachedServingTerminal, +} from "./core-replay"; +import { sealRequestAttemptIdentity, noteAttemptSend, recordAttemptCredentialSource } from "../request-log"; +import { waitForProviderRequestSlot, RequestPacingQueueOverloadError } from "../../providers/request-pacing"; +import type { AdapterEventQueue } from "../../adapters/run-turn-queue"; +import type { AttemptRecoveryKind } from "../../usage/log"; +import { providerFetch } from "./fetch-helpers"; +import { normalizeLogConversationId } from "../request-log-conversation"; +import type { AdapterEvent, OcxProviderContinuationState } from "../../types"; +import { adapterFailureFromMessage } from "../../lib/errors"; +import { + GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST, + isGenericOAuthFailoverEnabled, + rotateGenericOAuthAccountOn429, + failoverAccountSnapshot, +} from "../../oauth/generic-account-failover"; +import { resolveWireProtocolOverride } from "../adapter-resolve"; +import { formatErrorResponse, bridgeToResponsesSSE, buildResponseJSON } from "../../bridge"; +import { redactSecretString } from "../../lib/redact"; +import { + guardEmptyCompletionEventStream, + observeEmptyCompletion, + emptyCompletionNotice, +} from "./empty-completion-guard"; +import { rememberResponseState } from "../../responses/state"; +import { trackStreamLifetime } from "../lifecycle"; +import { awaitThoughtSignatureDurability } from "../../responses/thought-signature-replay"; + +/** One responsibility of the Responses request pipeline; state owners are explicit. */ +export async function executeResponsesRunTurn( + requestContext: Pick, + admissionState: ResponsesAdmissionState, + requestState: Pick< + PreparedResponsesRequest, + | "parsed" + | "route" + | "selectedForwardHeaders" + | "translatorBudget" + | "inboundWire" + | "toolBridgeMaps" + | "rememberKiroDeliveredFinalAnswer" + | "responseStateOptions" + >, + transportState: Pick< + ResponsesTransport, + | "selectionIsCurrent" + | "adapterBindings" + | "runTurnAdapter" + | "refreshRunTurnAdapter" + | "replayOAuthCredentialSnapshot" + | "genericFailoverAccountId" + | "genericFailovers" + | "applyFailoverSnapshot" + | "resolveSelectionAdapter" + | "adapter" + >, + sidecarState: Pick, + responseEffects: Pick< + ResponsesEffects, + | "cancelResponseCompletion" + | "commitReasoningReplayServingRoute" + | "continuationStateForResponse" + | "notifyResponseComplete" + >, + sendBudgetState: Pick, + completionPolicy: Pick, +): Promise { + const { options, logCtx, config } = requestContext; + const { + selectionIsCurrent, + adapterBindings, + refreshRunTurnAdapter, + applyFailoverSnapshot, + resolveSelectionAdapter, + } = transportState; + const { + parsed, + route, + translatorBudget, + inboundWire, + toolBridgeMaps, + rememberKiroDeliveredFinalAnswer, + responseStateOptions, + } = requestState; + const { adapterSendBudget, reserveCredentialHop } = sendBudgetState; + const { emptyCompletionGuardEnabled } = completionPolicy; + const { + cancelResponseCompletion, + commitReasoningReplayServingRoute, + continuationStateForResponse, + notifyResponseComplete, + } = responseEffects; + const { routedCompaction } = sidecarState; + + const runTurnAbort = new AbortController(); + const cleanupRunTurnAbort = linkAbortSignal(runTurnAbort, options.abortSignal); + const queue = createAdapterEventQueue({ + onBacklogExceeded: () => runTurnAbort.abort(), + }); + const refreshRunTurnSelection = async (): Promise => { + if (selectionIsCurrent(adapterBindings.get(transportState.runTurnAdapter))) return; + await refreshRunTurnAdapter(parsed); + bindRouteReasoningReplayScope({ parsed, providerName: route.providerName, provider: route.provider, + adapterName: transportState.runTurnAdapter.name, oauthCredentialSnapshot: transportState.replayOAuthCredentialSnapshot }); + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, transportState.runTurnAdapter.name, logCtx.accountLogLabel); + }; + // Initial admission must settle before the streaming Response commits HTTP 200. + // Let the outer Responses facade preserve the local retryable-429 contract. + try { + await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, runTurnAbort.signal); + } catch (error) { + cleanupRunTurnAbort(); + queue.close(); + throw error; + } + // One attempt of the runTurn transport, against an explicit queue. The + // empty-completion guard re-invokes the IDENTICAL turn (same parsed request, + // same forwarded headers, same abort signal) through a fresh queue, so the + // attempt body must not capture the first queue. Each attempt consumes its + // own provider pacing slot (#1584): retries are paced like first attempts. + const runTurnAttempt = async ( + targetQueue: AdapterEventQueue, + recovery?: AttemptRecoveryKind, + pacingSlotAcquired = false, + ): Promise => { + try { + if (!pacingSlotAcquired) { + await waitForProviderRequestSlot(route.providerName, route.provider, route.modelId, runTurnAbort.signal); + } + await refreshRunTurnSelection(); + noteAttemptSend(logCtx.activeAttempt, logCtx.usageLogInputTokens, recovery); + const runTurnProviderFetch = providerFetch( + route.provider, + options.codexWsRuntimeIdentity, + { + providerName: route.providerName, + modelId: route.modelId, + // runTurnAttempt acquired this logical turn's first physical-request slot above. + // Cursor HTTP/1.1 consumes it for RunSSE; every BidiAppend and redial then waits on + // the same provider queue through this stateful wrapper. + pacingSlotAcquired: true, + }, + ); + await transportState.runTurnAdapter.runTurn?.( + parsed, + { + headers: requestState.selectedForwardHeaders, + abortSignal: runTurnAbort.signal, + translatorBudget, + providerFetch: runTurnProviderFetch, + // The only way the request budget reaches a transport the adapter owns. Without it + // a Cursor turn's inner ladder was three physical sends the cap read as one. + ...(adapterSendBudget ? { sendBudget: adapterSendBudget } : {}), + }, + targetQueue.push, + ); + } catch (err) { + targetQueue.push(err instanceof RequestPacingQueueOverloadError + ? { + type: "error", + status: 429, + errorType: "rate_limit_error", + retryable: true, + message: err.message, + } + : { + type: "error", + message: err instanceof Error ? err.message : String(err), + }); + } finally { + // Cursor assigns a stable conversation id inside runTurn on the first headerless + // turn; backfill so Logs can filter/total that opening request (#330 / #522). + if (!logCtx.conversationId && parsed._cursorConversationId) { + logCtx.conversationId = normalizeLogConversationId(parsed._cursorConversationId); + } + targetQueue.close(); + } + }; + const runTurn = async (): Promise => runTurnAttempt(queue, undefined, true); + const rotateRunTurnAdapterOnPreflight429 = async ( + error: Extract, + ): Promise => { + const status = error.status ?? adapterFailureFromMessage(error.message).httpStatus; + if ( + status !== 429 + || !transportState.genericFailoverAccountId + || transportState.genericFailovers >= GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST + || !isGenericOAuthFailoverEnabled(config, route.providerName) + ) return false; + // Intersection with the request's shared budget: the roster bound above answers "may this + // credential set rotate again", this answers "may this request send again at all". The + // replayed turn is dispatched by runTurnAttempt and never reaches `onSendsConsumed`, so + // this reservation is the charge. Refusing returns false, which leaves the preflight 429 + // to reach the client exactly as the adapter produced it. + const hop = reserveCredentialHop( + "auth-recovery", + `${route.providerName}|${route.modelId}|runturn-oauth-429`, + ); + if (!hop.allowed) return false; + const nextAccountId = rotateGenericOAuthAccountOn429( + config, + route.providerName, + transportState.genericFailoverAccountId, + null, + ); + if (!nextAccountId) { + hop.permit?.release(); + return false; + } + try { + const snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); + transportState.genericFailovers += 1; + if (!await applyFailoverSnapshot(snapshot)) { + hop.permit?.release(); + return false; + } + // A Cursor conversation/checkpoint is credential-scoped. The failed attempt emitted no + // client-visible bytes, so replay is safe, but carrying its account identity into the next + // account would not be. Let the rotated adapter derive a fresh identity and conversation. + parsed._cursorIdentityScope = undefined; + parsed._cursorConversationId = undefined; + if (parsed._providerContinuation?.cursor) { + const { cursor: _discardedCursor, ...otherProviderState } = parsed._providerContinuation; + parsed._providerContinuation = otherProviderState; + } + const rotatedProvider = resolveWireProtocolOverride( + route.providerName, + route.modelId, + route.provider, + inboundWire, + ); + const rotatedAdapter = resolveSelectionAdapter(rotatedProvider, config.cacheRetention); + if (!rotatedAdapter.runTurn) { + hop.permit?.release(); + return false; + } + transportState.runTurnAdapter = rotatedAdapter; + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: rotatedProvider, + adapterName: rotatedAdapter.name, + oauthCredentialSnapshot: { accountId: snapshot.accountId, generation: snapshot.generation }, + codexAuthContext: admissionState.authCtx, + forwardHeaders: requestState.selectedForwardHeaders, + }); + sealRequestAttemptIdentity(logCtx.activeAttempt, logCtx.provider, rotatedAdapter.name, logCtx.accountLogLabel); + recordAttemptCredentialSource(logCtx.activeAttempt, route.providerName, route.provider, rotatedAdapter.name); + // The caller replays the turn on this rotation, so the reservation is now confirmed. + hop.permit?.use(); + return true; + } catch { + hop.permit?.release(); + return false; + } + }; + const preflightRunTurnFailover = async ( + firstSource: AsyncIterable, + ): Promise> => { + let source = firstSource; + while (true) { + const preflight = await preflightAdapterEvents(source); + if (!preflight.error || !(await rotateRunTurnAdapterOnPreflight429(preflight.error))) { + return preflight.stream; + } + const retryQueue = createAdapterEventQueue({ + onBacklogExceeded: () => runTurnAbort.abort(), + }); + void runTurnAttempt(retryQueue, "oauth-account-429"); + source = retryQueue.stream(); + } + }; + // The empty-completion retry re-runs the turn against a fresh queue: the + // first queue is closed once its attempt settles, and pushing into it after + // close is a silent no-op. + const runTurnRetrySource = (): AsyncIterable => { + const retryQueue = createAdapterEventQueue({ + onBacklogExceeded: () => runTurnAbort.abort(), + }); + void runTurnAttempt(retryQueue, "empty-completion"); + return retryQueue.stream(); + }; + + const { toolNsMap, declaredToolNames, toolParameterSchemas, freeformToolNames, toolSearchToolNames } = toolBridgeMaps; + if (parsed.stream) { + void runTurn(); + let eventSource: AsyncIterable = queue.stream(); + if (route.provider.authMode === "oauth" || (transportState.genericFailoverAccountId && isGenericOAuthFailoverEnabled(config, route.providerName))) { + // Preflight holds only heartbeats and the first meaningful event. A first-event 429 can be + // replayed transparently; after any output reaches the bridge, a later error stays terminal. + eventSource = await preflightRunTurnFailover(eventSource); + } + if (options.comboAttempt) { + const preflight = await preflightAdapterEvents(eventSource); + if (preflight.error || preflight.empty) { + runTurnAbort.abort(); + queue.close(); + const message = preflight.error?.message ?? "Adapter ended before producing a response"; + return formatErrorResponse(502, "upstream_error", redactSecretString(message)); + } + eventSource = preflight.stream; + } + const guardedSource = emptyCompletionGuardEnabled + ? guardEmptyCompletionEventStream({ + firstEvents: eventSource, + // Identical-turn retry: same parsed request, same headers, same + // signal — run the adapter transport again against a fresh queue. + continuation: runTurnRetrySource, + }) + // Guard off (the default): leave the stream alone, but record that the turn ended + // empty so the user has something to correlate instead of an unexplained blank + // result (#2472). Retrying by default would re-send a turn that may already have had + // billable side effects, so the honest default is observability, not recovery. + : observeEmptyCompletion(eventSource, () => { + console.warn(emptyCompletionNotice(route.providerName, route.modelId)); + }); + const sseStream = bridgeToResponsesSSE( + guardedSource, parsed._responseModelId ?? parsed.modelId, toolNsMap, freeformToolNames, toolSearchToolNames, + () => { + cancelResponseCompletion(); + runTurnAbort.abort(); + queue.close(); + }, 2_000, + { + translatorBudget, + replayCacheScope: parsed._reasoningReplayScope, + ...(options.forceEmptyResponseId ? { responseId: "" } : {}), + stallTimeoutSec: config.stallTimeoutSec, + hideThinkingSummary: parsed.options.hideThinkingSummary, + declaredToolNames, + toolParameterSchemas, + ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), + ...(routedCompaction ? { compaction: true } : {}), + // grok-build's strict decoder dies on the typed response.heartbeat frame; its + // eventsource layer tolerates comment keep-alives. Codex needs the opposite. + ...(logCtx.surface === "grok" ? { heartbeatStyle: "comment" as const } : {}), + onUsage: usage => { + // Raw adapter usage, pre wire-normalization: the bridged SSE now always carries + // zero-default detail objects, so provenance must come from here (cache_detail_missing). + logCtx.usageFromBridge = true; + if (usage) { + logCtx.usage = usage; + if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; + } + }, + onCompletedResponse: (response: Record, providerState?: OcxProviderContinuationState) => { + commitReasoningReplayServingRoute(); + rememberKiroDeliveredFinalAnswer(transportState.adapter.name, response); + if (!routedCompaction) { + rememberResponseState( + parsed._rawBody, + response, + continuationStateForResponse(providerState), + responseStateOptions(adapterNeedsForcedContinuation(transportState.adapter.name)), + ); + } + notifyResponseComplete(response); + }, + }, + ); + const bridgeTurnAc = new AbortController(); + const trackedSse = trackStreamLifetime(sseStream, bridgeTurnAc, undefined, options.turnAdmissionLease); + const response = new Response(trackedSse, { + headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache", "Connection": "keep-alive", "X-Accel-Buffering": "no" }, + }); + runTurnAdapterSseResponses.add(response); + return response; + } + + await runTurn(); + const firstAttemptEvents = await queue.collect(); + let runTurnEvents: AdapterEvent[] = firstAttemptEvents; + if (route.provider.authMode === "oauth" || (transportState.genericFailoverAccountId && isGenericOAuthFailoverEnabled(config, route.providerName))) { + runTurnEvents = []; + for await (const event of await preflightRunTurnFailover( + (async function* () { yield* firstAttemptEvents; })(), + )) runTurnEvents.push(event); + } + let events: AdapterEvent[]; + if (emptyCompletionGuardEnabled) { + events = []; + for await (const event of guardEmptyCompletionEventStream({ + firstEvents: (async function* () { yield* runTurnEvents; })(), + continuation: runTurnRetrySource, + })) events.push(event); + } else { + events = runTurnEvents; + } + if (options.comboAttempt) { + const firstMeaningful = events.find(event => event.type !== "heartbeat"); + if (!firstMeaningful || firstMeaningful.type === "error") { + const message = firstMeaningful?.type === "error" + ? firstMeaningful.message + : "Adapter ended before producing a response"; + return formatErrorResponse(502, "upstream_error", redactSecretString(message)); + } + } + let providerState: OcxProviderContinuationState | undefined; + const json = buildResponseJSON(events, parsed._responseModelId ?? parsed.modelId, { + translatorBudget, + replayCacheScope: parsed._reasoningReplayScope, + hideThinkingSummary: parsed.options.hideThinkingSummary, + toolNsMap, + declaredToolNames, + toolParameterSchemas, + freeformToolNames, + toolSearchToolNames, + ...(routedCompaction ? { compaction: true } : {}), + onProviderState: state => { providerState = state; }, + onUsage: usage => { + logCtx.usageFromBridge = true; + if (usage) { + logCtx.usage = usage; + if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; + } + }, + }); + if (!routedCompaction) { + rememberKiroDeliveredFinalAnswer(transportState.adapter.name, json); + rememberResponseState( + parsed._rawBody, + json, + continuationStateForResponse(providerState), + responseStateOptions(adapterNeedsForcedContinuation(transportState.adapter.name)), + ); + } + // #1926 gap 2: the buffered path queued its signature persists inside + // buildResponseJSON; bound the durability window before the JSON becomes + // externally visible. + await awaitThoughtSignatureDurability(); + if (adapterResponseReachedServingTerminal(events, json)) { + commitReasoningReplayServingRoute(); + } + notifyResponseComplete(json); + return new Response(JSON.stringify(json), { headers: { "Content-Type": "application/json" } }); +} diff --git a/src/server/responses/sidecar-execution.ts b/src/server/responses/sidecar-execution.ts new file mode 100644 index 0000000000..7987d5ca93 --- /dev/null +++ b/src/server/responses/sidecar-execution.ts @@ -0,0 +1,469 @@ +import type { ResponsesRequestContext } from "./core-options"; +import type { PreparedResponsesRequest } from "./request-prepare"; +import type { ResponsesTransport } from "./request-transport"; +import type { ResponsesSidecarAuth } from "./request-sidecar-auth"; +import type { ResponsesEffects } from "./response-effects"; +import type { ResponsesSendBudget } from "./request-send-budget"; +import { formatErrorResponse } from "../../bridge"; +import { planWebSearch, buildWebSearchTool, runWithWebSearch } from "../../web-search"; +import { + planImageBridge, + planVideoBridge, + IMAGE_GEN_TOOL_NAME, + buildImageTool, + VIDEO_GEN_TOOL_NAME, + buildVideoTool, + runWithImageBridge, + clampImageMaxRounds, +} from "../../images"; +import type { ProviderAdapter } from "../../adapters/base"; +import { rotateProviderTransportOn429, rateLimitRetryPolicyFor } from "../../providers/key-failover"; +import { + GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST, + isGenericOAuthFailoverEnabled, + rotateGenericOAuthAccountOn429, + failoverAccountSnapshot, +} from "../../oauth/generic-account-failover"; +import { + ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST, + rotateAnthropicAccountOn429, + getAnthropicPoolAccessSnapshot, + formatAnthropicProviderForLog, +} from "../../oauth/anthropic-routing"; +import { resolveWireProtocolOverride } from "../adapter-resolve"; +import { bindRouteReasoningReplayScope, adapterNeedsForcedContinuation } from "./core-replay"; +import { namespacedToolName } from "../../types"; +import { providerFetch } from "./fetch-helpers"; +import type { AttemptRecoveryKind } from "../../usage/log"; +import { noteAttemptSend, recordAdapterReasoning, recordAdapterTier } from "../request-log"; +import { normalizeLogConversationId } from "../request-log-conversation"; +import { rememberResponseState } from "../../responses/state"; +import { trackStreamLifetime } from "../lifecycle"; + +/** One responsibility of the Responses request pipeline; state owners are explicit. */ +export async function executeResponsesSidecars( + requestContext: Pick, + requestState: Pick< + PreparedResponsesRequest, + | "parsed" + | "route" + | "inboundWire" + | "selectedForwardHeaders" + | "translatorBudget" + | "rememberKiroDeliveredFinalAnswer" + | "responseStateOptions" + >, + transportState: Pick< + ResponsesTransport, + | "adapter" + | "genericFailoverAccountId" + | "genericFailovers" + | "applyFailoverSnapshot" + | "anthropicPoolAccountId" + | "anthropicPoolFailovers" + | "anthropicSessionKey" + | "commitResolvedOAuthSelection" + | "resolveSelectionAdapter" + | "oauthDispatch" + >, + sidecarState: Pick, + responseEffects: Pick< + ResponsesEffects, + | "commitReasoningReplayServingRoute" + | "continuationStateForResponse" + | "notifyResponseComplete" + | "cancelResponseCompletion" + >, + sendBudgetState: Pick, +) { + const { config, options, logCtx } = requestContext; + const { + applyFailoverSnapshot, + anthropicSessionKey, + commitResolvedOAuthSelection, + resolveSelectionAdapter, + oauthDispatch, + } = transportState; + const { + parsed, + route, + inboundWire, + translatorBudget, + rememberKiroDeliveredFinalAnswer, + responseStateOptions, + } = requestState; + const { routedCompaction, openAiSidecar } = sidecarState; + const { reserveCredentialHop } = sendBudgetState; + const { + commitReasoningReplayServingRoute, + continuationStateForResponse, + notifyResponseComplete, + cancelResponseCompletion, + } = responseEffects; + + + // Tool results are PAIRED by call_id. parseRequest writes it into OcxToolResultMessage.toolCallId + // (parser.ts:738/752) without validating it, because inputItemSchema's permissive catch-all + // (schema.ts:106) accepts a tool item whose strict schema failed only for a missing call_id. A + // translating adapter then consumes `toolCallId: string` holding undefined: kiro-wire.ts:32 + // TypeErrors, ollama-native.ts:334 throws, and anthropic.ts:775 sends + // "[tool_result without adjacent tool_use: undefined]" upstream (issue #3259). + // + // This CANNOT move into the schema. parseRequest (:2812) runs before the passthrough branch + // (:3719), so a parse-time rejection would also kill forward/key passthrough and routed + // compaction — paths that never read context.messages, build from _rawBody, and already + // degrade an unpaired output to "[tool output for unknown call]" on their own. + // + // Keyed on the adapter, not on position: routedCompaction skips the passthrough branch above + // yet still builds from _rawBody (see the :3703 comment). + if (!("passthrough" in transportState.adapter && transportState.adapter.passthrough)) { + const unpaired = parsed.context.messages.find( + message => message.role === "toolResult" + && (typeof (message as { toolCallId?: unknown }).toolCallId !== "string" + || (message as { toolCallId: string }).toolCallId.length === 0), + ); + if (unpaired) { + // Never interpolate the tool output: this message reaches the client and the logs. + return formatErrorResponse( + 400, + "invalid_request_error", + "tool result requires a non-empty string call_id", + ); + } + } + + // Image / web-search sidecars: plan once, then dispatch with runTurn-aware priority. + // Routed-compaction turns must NOT hit the image bridge: compaction clears tools/_webSearch but + // leaves _imageGeneration, so planImageBridge would activate and return a normal Responses + // completion instead of the synthetic compaction item Codex expects (#424). + // + // Web-search's loop only supports buildRequest/fetch/parseStream — NOT adapter.runTurn. Sending + // Cursor/runTurn requests into runWithWebSearch produces empty HTTP failures. So: + // - non-runTurn: web-search wins over image when both eligible (documented priority) + // - runTurn: image bridge may run (it supports runTurn); web-search is skipped so runTurn + // can proceed for web-search-only turns + const wsPlan = !routedCompaction + ? planWebSearch(config, parsed, false, route.provider, route.modelId, openAiSidecar, { + admission: options.admission, codexAuthPolicy: options.codexAuthPolicy, providerName: route.providerName, + }) + : undefined; + const imgPlan = !routedCompaction ? await planImageBridge(config, parsed, route.provider) : undefined; + const vidPlan = !routedCompaction ? await planVideoBridge(config, parsed, route.provider) : undefined; + const canRunWebSearch = !!wsPlan && !transportState.adapter.runTurn; + const rotateSidecarProviderOn429 = async ( + retryAfter: string | null, + responseHeaders?: Headers, + ): Promise => { + const rotated = rotateProviderTransportOn429(config, route.providerName, route.provider, { + retryAfter, + now: Date.now(), + attemptedKey: route.provider.apiKey, + promptCacheKey: parsed.options.promptCacheKey, + }); + if (rotated) { + route.provider = rotated; + } else if ( + // A POSITIVE gate, not an early return. An early `return null` here made every later arm + // unreachable: Anthropic never has a genericFailoverAccountId (isGenericFailoverProvider + // excludes it), so its sidecar 429s died on this guard before the Anthropic arm below + // could ever be considered. + transportState.genericFailoverAccountId + && transportState.genericFailovers < GENERIC_OAUTH_MAX_FAILOVERS_PER_REQUEST + && isGenericOAuthFailoverEnabled(config, route.providerName) + ) { + // Intersection with the request's shared budget. The sidecar replay is dispatched by the + // web-search/image loop and never reaches `onSendsConsumed`, so this reservation is the + // charge; a refusal returns null and the caller keeps the real 429 it already has. + const hop = reserveCredentialHop( + "auth-recovery", + `${route.providerName}|${route.modelId}|sidecar-oauth-429`, + ); + if (!hop.allowed) return null; + const nextAccountId = rotateGenericOAuthAccountOn429( + config, + route.providerName, + transportState.genericFailoverAccountId, + retryAfter, + ); + if (!nextAccountId) { + hop.permit?.release(); + return null; + } + try { + const snapshot = await failoverAccountSnapshot(route.providerName, nextAccountId); + transportState.genericFailovers += 1; + if (!await applyFailoverSnapshot(snapshot)) { + hop.permit?.release(); + return null; + } + } catch { + hop.permit?.release(); + return null; + } + hop.permit?.use(); + } else if ( + // Anthropic's pool is excluded from generic failover, so without this arm a 429 inside a + // web-search or image-bridge turn was terminal even with the pool fully enabled -- while + // the very same 429 on the main response path rotated. + transportState.anthropicPoolAccountId + && transportState.anthropicPoolFailovers < ANTHROPIC_POOL_MAX_FAILOVERS_PER_REQUEST + ) { + // Same intersection for the Anthropic roster: its own per-request bound still applies, + // and the shared budget decides whether this request may spend another send at all. + const hop = reserveCredentialHop( + "auth-recovery", + `${route.providerName}|${route.modelId}|sidecar-anthropic-429`, + ); + if (!hop.allowed) return null; + const nextAccountId = rotateAnthropicAccountOn429( + config, + transportState.anthropicPoolAccountId, + retryAfter, + anthropicSessionKey, + Date.now(), + responseHeaders, + ); + if (!nextAccountId) { + hop.permit?.release(); + return null; + } + try { + // Deliberately NOT applyFailoverSnapshot: that helper exists to pair per-account routing + // metadata (Copilot origin, Antigravity project, Kiro context) with its bearer. Anthropic + // carries none, and getAnthropicPoolAccessToken is what enforces its fail-closed + // local-cli credential rule. Both existing Anthropic rotation sites apply the token the + // same way. + const admitted = await commitResolvedOAuthSelection(await getAnthropicPoolAccessSnapshot(nextAccountId)); + if (!admitted) throw new Error("OAuth selection changed during recovery"); + transportState.anthropicPoolAccountId = admitted.accountId; + transportState.anthropicPoolFailovers += 1; + route.provider = { ...route.provider, apiKey: admitted.accessToken }; + logCtx.provider = formatAnthropicProviderForLog("anthropic", admitted.accountId, config); + } catch { + hop.permit?.release(); + return null; + } + hop.permit?.use(); + } else { + // No key pool, no generic OAuth roster, no Anthropic pool could produce a replacement + // credential. The 429 is terminal for this sidecar turn. + return null; + } + const rotatedAdapter = resolveSelectionAdapter( + resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, inboundWire), + config.cacheRetention, + ); + bindRouteReasoningReplayScope({ + parsed, + providerName: route.providerName, + provider: route.provider, + adapterName: rotatedAdapter.name, + }); + return rotatedAdapter; + }; + if ((imgPlan || vidPlan) && canRunWebSearch) { + // Web search takes priority when both are active — the media bridge cannot run + // alongside runWithWebSearch. Surface a runtime signal so the user knows their + // configured video/image bridge was skipped for this turn, rather than silently + // dropping a paid capability. + if (vidPlan) console.warn("[videos] video bridge skipped: web search is active for this turn"); + if (imgPlan) console.warn("[images] image bridge skipped: web search is active for this turn"); + } + if ((imgPlan || vidPlan) && (!wsPlan || transportState.adapter.runTurn)) { + // The image bridge detects a hosted image_generation tool and requires streaming. + // The video bridge activates from config and injects a tool — it also needs streaming + // (the loop returns SSE). For video-only (no imgPlan) on a non-streaming request, skip + // the bridge entirely so enabling the feature doesn't break ordinary non-streaming traffic. + if (!parsed.stream) { + if (imgPlan) { + return formatErrorResponse(400, "invalid_request_error", "image bridge requires stream=true"); + } + // Video-only: skip bridge for non-streaming requests + } else { + // Replace any pre-existing image_gen/video_gen aliases instead of appending duplicate wire names. + const priorTools = parsed.context.tools ?? []; + const bridgeTools = [...priorTools.filter(t => { + if (t.imageGeneration) return false; + if (t.videoGeneration) return false; + if (imgPlan && imgPlan.toolNames.has(t.name)) return false; + if (imgPlan && t.namespace && imgPlan.toolNames.has(namespacedToolName(t.namespace, t.name))) return false; + // Only strip unnamespaced video_gen aliases — a namespaced MCP video_gen is left alone. + if (vidPlan && !t.namespace && vidPlan.toolNames.has(t.name)) return false; + return true; + })]; + const existingNames = new Set(bridgeTools.map(t => t.name)); + if (imgPlan && !existingNames.has(IMAGE_GEN_TOOL_NAME)) bridgeTools.push(buildImageTool()); + if (vidPlan && !existingNames.has(VIDEO_GEN_TOOL_NAME)) bridgeTools.push(buildVideoTool()); + parsed.context.tools = bridgeTools; + // Hosted image_generation tool_choice / allowed_tools must target the synthetic function name. + // Gate on imgPlan — in a video-only turn buildImageTool() was never injected, so rewriting + // image_generation/image_gen aliases would add an undeclared tool that strict upstreams reject. + const tc = parsed.options.toolChoice; + if (imgPlan && tc && typeof tc === "object" && "allowedTools" in tc && Array.isArray(tc.allowedTools)) { + const mapped = tc.allowedTools.map(name => + name === "image_generation" || name === "image_gen" || (imgPlan.toolNames.has(name) ?? false) + ? IMAGE_GEN_TOOL_NAME + : name, + ); + parsed.options.toolChoice = { ...tc, allowedTools: [...new Set(mapped)] }; + } else if (imgPlan && tc && typeof tc === "object" && "name" in tc && typeof tc.name === "string" + && (tc.name === "image_generation" || imgPlan.toolNames.has(tc.name))) { + parsed.options.toolChoice = { ...tc, name: IMAGE_GEN_TOOL_NAME }; + } + const imageProviderFetch = providerFetch( + route.provider, + options.codexWsRuntimeIdentity, + { providerName: route.providerName, modelId: route.modelId }, + ); + const imgResponse = await runWithImageBridge({ + parsed, adapter: transportState.adapter, + incomingMeta: { headers: requestState.selectedForwardHeaders, abortSignal: options.abortSignal, translatorBudget }, + ...(imgPlan ? { plan: imgPlan } : {}), + ...(vidPlan ? { videoPlan: vidPlan } : {}), + forwardHeaders: requestState.selectedForwardHeaders, + onAttemptSend: (recovery?: AttemptRecoveryKind) => + noteAttemptSend(logCtx.activeAttempt, logCtx.usageLogInputTokens, recovery), + abortSignal: options.abortSignal, + maxRounds: imgPlan && vidPlan + ? clampImageMaxRounds(Math.min(config.images?.maxRounds ?? 3, config.images?.videoMaxRounds ?? 2)) + : imgPlan + ? clampImageMaxRounds(config.images?.maxRounds) + : clampImageMaxRounds(config.images?.videoMaxRounds ?? 2), + connectTimeoutMs: config.connectTimeoutMs ?? 200_000, + stallTimeoutSec: config.stallTimeoutSec, + waitForRequestSlot: imageProviderFetch.waitForPacing, + fetchImpl: imageProviderFetch.unpacedFetch ?? imageProviderFetch, + fetchForRequest: (request, iterParsed) => { + const fetch = providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(request, iterParsed), + providerName: route.providerName, modelId: route.modelId, + }); + return fetch.unpacedFetch ?? fetch; + }, + onRequestBuilt: request => { + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + }, + ...(vidPlan?.timeoutMs ? { videoTimeoutMs: vidPlan.timeoutMs } : {}), + onUsage: usage => { + // Cursor may assign _cursorConversationId inside the image loop's first runTurn; + // backfill so Logs can filter/total that opening request (parity with the normal + // runTurn branch). + if (!logCtx.conversationId && parsed._cursorConversationId) { + logCtx.conversationId = normalizeLogConversationId(parsed._cursorConversationId); + } + logCtx.usageFromBridge = true; + if (usage) { + logCtx.usage = usage; + if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; + } + }, + on429: rotateSidecarProviderOn429, + retryOn429Policy: rateLimitRetryPolicyFor(route.provider), + ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), + ...(options.forceEmptyResponseId ? { forceEmptyResponseId: true } : {}), + onCompletedResponse: (response, providerState) => { + commitReasoningReplayServingRoute(); + rememberKiroDeliveredFinalAnswer(transportState.adapter.name, response); + rememberResponseState( + parsed._rawBody, + response, + continuationStateForResponse(providerState), + responseStateOptions(adapterNeedsForcedContinuation(transportState.adapter.name)), + ); + notifyResponseComplete(response); + }, + }); + if (imgResponse.body) { + const imgTurnAc = new AbortController(); + imgTurnAc.signal.addEventListener("abort", cancelResponseCompletion, { once: true }); + return new Response(trackStreamLifetime(imgResponse.body, imgTurnAc, undefined, options.turnAdmissionLease), { + status: imgResponse.status, + headers: imgResponse.headers, + }); + } + return imgResponse; + } // end else (streaming bridge) + } + + // Web-search sidecar: Codex enabled web_search but this is a routed (non-OpenAI) model that can't + // run it server-side. Expose web_search as a function tool and run searches via the gpt-mini sidecar + // through the ChatGPT passthrough, looping until the model answers. Otherwise take the normal path. + // Placed BEFORE the runTurn early-return for non-runTurn adapters so dual-tool turns dispatch + // through web-search instead of being swallowed. runTurn adapters never enter this branch. + if (canRunWebSearch && wsPlan) { + parsed.context.tools = [...(parsed.context.tools ?? []), buildWebSearchTool()]; + // Resolve the mutable route at send time: a 429 rotation replaces route.provider, so retaining + // one pre-rotation providerFetch would keep the old credential and transport pin. + const routedProviderFetch = ((input: Parameters[0], init?: RequestInit) => + providerFetch(route.provider, options.codexWsRuntimeIdentity, { + providerName: route.providerName, + modelId: route.modelId, + })(input, init)) as typeof globalThis.fetch; + const wsResponse = await runWithWebSearch({ + parsed, adapter: transportState.adapter, + fetchForRequest: (request, iterParsed) => providerFetch(route.provider, options.codexWsRuntimeIdentity, { + dispatchOverride: oauthDispatch(request, iterParsed), + providerName: route.providerName, modelId: route.modelId, + }), + incomingMeta: { + headers: requestState.selectedForwardHeaders, + abortSignal: options.abortSignal, + translatorBudget, + providerFetch: routedProviderFetch, + }, + backend: wsPlan.backend, + forwardProvider: wsPlan.forwardSidecar?.provider, + anthropicSidecar: wsPlan.anthropicSidecar, + xaiSidecar: wsPlan.xaiSidecar, + geminiSidecar: wsPlan.geminiSidecar, + xaiSearchOptions: wsPlan.xaiSearchOptions, + // The exa key never rides the plan: read it from config at unpack time (L9). + ...(wsPlan.exaConfigured ? { exaApiKey: config.webSearchSidecar?.exaApiKey } : {}), + hostedTool: wsPlan.hostedTool, + selectedForwardHeaders: wsPlan.forwardSidecar?.headers ?? requestState.selectedForwardHeaders, + settings: wsPlan.settings, + maxSearches: wsPlan.maxSearches, + forceEmptyResponseId: true, + abortSignal: options.abortSignal, + ...(options.onFirstOutput ? { onFirstOutput: options.onFirstOutput } : {}), + onRequestBuilt: request => { + recordAdapterReasoning(logCtx, request); + recordAdapterTier(logCtx, request); + }, + onAttemptSend: (recovery?: AttemptRecoveryKind) => + noteAttemptSend(logCtx.activeAttempt, logCtx.usageLogInputTokens, recovery), + onUsage: usage => { + logCtx.usageFromBridge = true; + if (usage) { + logCtx.usage = usage; + if (logCtx.activeAttempt) logCtx.activeAttempt.usage = usage; + } + }, + recordSidecarOutcome: wsPlan.forwardSidecar?.recordOutcome, + connectTimeoutMs: config.connectTimeoutMs ?? 200_000, + routedModelStallTimeoutMs: wsPlan.routedModelStallTimeoutMs, + stallTimeoutSec: wsPlan.stallTimeoutSec, + streamRoutedModelOutput: wsPlan.streamRoutedModelOutput, + on429: rotateSidecarProviderOn429, + retryOn429Policy: rateLimitRetryPolicyFor(route.provider), + onCompletedResponse: response => { + commitReasoningReplayServingRoute(); + notifyResponseComplete(response); + }, + }); + // Register the sidecar stream as an active turn so drainAndShutdown waits for (or aborts) + // in-flight web-search turns instead of skipping them during graceful shutdown. + if (wsResponse.body) { + const wsTurnAc = new AbortController(); + wsTurnAc.signal.addEventListener("abort", cancelResponseCompletion, { once: true }); + return new Response(trackStreamLifetime(wsResponse.body, wsTurnAc, undefined, options.turnAdmissionLease), { + status: wsResponse.status, + headers: wsResponse.headers, + }); + } + return wsResponse; + } + + return undefined; +} diff --git a/structure/adapters/registry.md b/structure/adapters/registry.md index 908633f265..3efbaa0058 100644 --- a/structure/adapters/registry.md +++ b/structure/adapters/registry.md @@ -1,5 +1,8 @@ # Adapter Registry Authority +Request-local adapter bindings are separate from registry authority in the Responses +[core module ownership](../transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/catalog.md b/structure/catalog.md index e3976b185d..cfaf9549cf 100644 --- a/structure/catalog.md +++ b/structure/catalog.md @@ -1,5 +1,8 @@ # Model Catalog +Catalog discovery remains separate from the Responses final-route +[core module ownership](transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/clients/claude-desktop.md b/structure/clients/claude-desktop.md index f64a278757..7b93b95ffd 100644 --- a/structure/clients/claude-desktop.md +++ b/structure/clients/claude-desktop.md @@ -1,5 +1,8 @@ # Claude Desktop Integration +Desktop callers retain their existing ingress through the Responses +[core module ownership](../transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/data-planes/images.md b/structure/data-planes/images.md index fdbdee6b32..7d490788cf 100644 --- a/structure/data-planes/images.md +++ b/structure/data-planes/images.md @@ -1,5 +1,8 @@ # Images Data Plane +Vision preprocessing and image/video/search execution use the Responses +[core module ownership](../transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/data-planes/inbound-compat.md b/structure/data-planes/inbound-compat.md index aa9aa15f52..d683935dd2 100644 --- a/structure/data-planes/inbound-compat.md +++ b/structure/data-planes/inbound-compat.md @@ -1,5 +1,8 @@ # Inbound Compatibility Surfaces +Compatibility callers retain the public Responses ingress described by the +[core module ownership](../transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/gui-and-management-api.md b/structure/gui-and-management-api.md index 9eb9f1fa74..873d0eb516 100644 --- a/structure/gui-and-management-api.md +++ b/structure/gui-and-management-api.md @@ -1,5 +1,8 @@ # GUI And Management API +The shared server request path follows the Responses +[core module ownership](transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/ops/service-and-sidecars.md b/structure/ops/service-and-sidecars.md index e9d8fb7c00..ec6cc683c8 100644 --- a/structure/ops/service-and-sidecars.md +++ b/structure/ops/service-and-sidecars.md @@ -1,5 +1,8 @@ # Background Service And Sidecars +Service endpoints are unchanged by the Responses +[core module ownership](../transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/providers/xai-grok.md b/structure/providers/xai-grok.md index ba5333ea58..7b407da354 100644 --- a/structure/providers/xai-grok.md +++ b/structure/providers/xai-grok.md @@ -1,5 +1,8 @@ # xAI Grok Provider +xAI uses the same shared credential and delivery policies through the Responses +[core module ownership](../transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/runtime.md b/structure/runtime.md index 9d97326b34..dc592ec8bd 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -1,5 +1,8 @@ # Runtime +Responses admission and finalization are composed through the +[core module ownership](transports/responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/subagents.md b/structure/subagents.md index 70c93a9ac5..32dfee4e60 100644 --- a/structure/subagents.md +++ b/structure/subagents.md @@ -1,5 +1,8 @@ # Subagents And Multi-Agent Surface +Encrypted-task and fallback request handling follow the Responses +[core module ownership](transports/responses.md#core-module-ownership). This surface retains its existing behavior. + ## Plaintext V2 agent messages `src/responses/plaintext-v2-agent-messages.ts` owns the experimental, configuration-only diff --git a/structure/transports/byte-accounting.md b/structure/transports/byte-accounting.md index e758afeaf2..fb2cdde470 100644 --- a/structure/transports/byte-accounting.md +++ b/structure/transports/byte-accounting.md @@ -1,5 +1,8 @@ # Byte Accounting +Responses body-reader limits and lifetime handling follow the +[core module ownership](responses.md#core-module-ownership). This surface retains its existing behavior. + How opencodex measures request and stream bytes without allocating copies solely to count them. These contracts are shared by request parsing, SSE rewriting, the provider adapters and the translator budget, which is why so many documents link here rather than restating them. diff --git a/structure/transports/inventory.md b/structure/transports/inventory.md index 8c04f7b633..3f4bfbf689 100644 --- a/structure/transports/inventory.md +++ b/structure/transports/inventory.md @@ -1,5 +1,8 @@ # Transport Inventory +The existing Responses transport is divided by responsibility in the +[core module ownership](responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/structure/transports/responses.md b/structure/transports/responses.md index 6a5ec27013..96ddb693a2 100644 --- a/structure/transports/responses.md +++ b/structure/transports/responses.md @@ -609,3 +609,54 @@ Translated Chat request construction uses the [inline-image budget](streaming-he The [explicit model-capability contract](../config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior. Provider-scoped approval reviewer settings are projected by the [catalog owner](../catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior. Translated audio/file admission follows the [final-adapter input contract](../adapters/registry.md#untranslated-input-media); native raw passthrough remains separate. + +## Core module ownership + +`src/server/responses/core.ts` is the public ingress and compatibility-export surface. +The parent `src/server/responses.ts` facade retains its existing imports. Per-request execution +is composed from the following owners in `src/server/responses/`; none is a generated artifact. + +| Owner | Responsibility | +| --- | --- | +| `request-prepare.ts` | Body parsing, combo handoff, final route, encrypted-task recovery and initial admission. | +| `request-transport.ts` | Live credential selection, dispatch bindings, adapter replacement and same-target request identity. | +| `request-sidecar-auth.ts` | Sidecar credential resolution and vision preprocessing. | +| `response-effects.ts` | Completion notification, replay publication and live request-tool aliases. | +| `request-send-budget.ts` | Request-wide send accounting, remaining allowance and the pending recovery permit. | +| `passthrough-execution.ts` | Native host-lease transfer and the enclosing dispatch/delivery `finally`. | +| `passthrough-dispatch.ts` | Native request preparation, upstream sends and pre-commit recovery. | +| `passthrough-delivery.ts` | Native HTTP/SSE/JSON delivery, rewrite/inspection and terminal accounting. | +| `sidecar-execution.ts` | Image/video versus web-search execution and their shared rotation hook. | +| `completion-policy.ts`, `run-turn-execution.ts` | Empty-completion eligibility and adapter-owned event turns. | +| `adapter-dispatch.ts` | Translated initial dispatch, bounded recovery and the shared continuation retry counter. | +| `adapter-continuation.ts`, `adapter-delivery.ts` | Continuation event sources and final streaming/buffered bridging. | + +Reusable helpers live in `core-auth.ts`, `core-codex-account.ts`, `core-combo.ts`, +`core-combo-failure.ts`, `core-errors.ts`, `core-lifetime.ts`, `core-normalize.ts`, +`core-opaque-recovery.ts` and `core-replay.ts`. `core-options.ts` owns the public option types +and small composition contracts. Existing public helper names are re-exported by `core.ts`. +Adapter construction remains with the existing registry; `fetch-helpers.ts` remains a leaf. + +Mutable values are not copied across phases. A phase exposes only the values consumed by later +phases, with getters/setters over the original local bindings where a retry or callback can +change them. Consumers receive typed `Pick` views. In particular, adapter replacement, credential +snapshots, request-tool aliases, cancellation, pending permits and continuation retry counts +remain live. Owner names are distinct from local decision variables: `admissionState` retains the +lease while a block-local `admission` holds only the acquisition result. + +`handleResponses` creates or inherits the same logical-request send holder. The budget owner +reads that holder rather than minting a per-phase allowance. Combo recursion is injected through +`ResponsesDispatchers`: a child re-enters the public handler without a reverse runtime import +from the combo implementation into `core.ts`. `core-lifetime.ts` owns the shared run-turn response +marker and translator-budget finalization, so the combo and delivery paths observe one identity. + +The outer admission `finally` remains in `core.ts`. Native execution explicitly transfers its +pending lease to `passthrough-execution.ts`; both owners await response construction before +cleanup. Stream body ownership, cancellation and post-commit behavior stay in the delivery owners. +This decomposition changes ownership boundaries, not credential-selection or retry policy. + +`tests/responses/responses-core-modules.test.ts` covers the owner inventory, the 1,999-line ceiling, +acyclic dependencies, recursive dispatch, lease-transfer wiring, capture-name hygiene and live +send-holder/permit behavior. Cross-owner source assertions read the actual implementations via +`tests/helpers/responses-core-source.ts`; focused passthrough and subagent assertions read their +specific delivery/preparation owner. Existing runtime Lab-boundary tests still start at `core.ts`. diff --git a/structure/transports/streaming-health.md b/structure/transports/streaming-health.md index 0b54c97058..47f9ecbc77 100644 --- a/structure/transports/streaming-health.md +++ b/structure/transports/streaming-health.md @@ -1,5 +1,8 @@ # Streaming Health And WebSocket +Native and translated delivery now have separate owners in the +[core module ownership](responses.md#core-module-ownership). This surface retains its existing behavior. + The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. diff --git a/tests/fixtures/file-size-baseline.json b/tests/fixtures/file-size-baseline.json index 27d6e9073b..56835250c9 100644 --- a/tests/fixtures/file-size-baseline.json +++ b/tests/fixtures/file-size-baseline.json @@ -25,7 +25,7 @@ "src/config.ts": 460, "src/providers/registry.ts": 232, "src/server/index.ts": 893, - "src/server/responses/core.ts": 9386, + "src/server/responses/core.ts": 210, "tests/ci-workflows/ci-workflows.test.ts": 5628, "tests/cli/cli-account.test.ts": 2313, "tests/codex-integration/codex-auth-api.test.ts": 6549, diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 7e50fb16b5..d2eb5d244b 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -1,4 +1,5 @@ { + "responses-core-modules.test.ts": "responses", "chat-responses-control-integration.test.ts": "responses", "coding-agent-tool-result-images.test.ts": "adapters", "hub-usage.test.ts": "server", diff --git a/tests/helpers/responses-core-source.ts b/tests/helpers/responses-core-source.ts new file mode 100644 index 0000000000..35fea3be44 --- /dev/null +++ b/tests/helpers/responses-core-source.ts @@ -0,0 +1,45 @@ +import { readFileSync } from "node:fs"; +import { repoPath } from "./repo-root"; + +/** + * Source-only inventory for cross-owner wiring assertions. Files are read, not + * executed. responses-core-modules.test.ts compares the inventory to the source import graph. + */ +export const RESPONSES_CORE_MODULES = [ + "core.ts", + "core-options.ts", + "core-lifetime.ts", + "core-replay.ts", + "core-errors.ts", + "core-opaque-recovery.ts", + "core-codex-account.ts", + "core-combo-failure.ts", + "core-auth.ts", + "core-normalize.ts", + "core-combo.ts", + "request-prepare.ts", + "request-transport.ts", + "request-sidecar-auth.ts", + "response-effects.ts", + "request-send-budget.ts", + "passthrough-execution.ts", + "passthrough-dispatch.ts", + "passthrough-delivery.ts", + "sidecar-execution.ts", + "completion-policy.ts", + "run-turn-execution.ts", + "adapter-dispatch.ts", + "adapter-continuation.ts", + "adapter-delivery.ts", +] as const; + +export type ResponsesCoreModule = typeof RESPONSES_CORE_MODULES[number]; + +export function readResponsesCoreModule(name: ResponsesCoreModule): string { + return readFileSync(repoPath("src", "server", "responses", name), "utf8"); +} + +/** Preserve cross-site source assertions without reading only the thin facade. */ +export function readResponsesCoreSource(): string { + return RESPONSES_CORE_MODULES.map(readResponsesCoreModule).join("\n"); +} diff --git a/tests/lab/lab-passive-production-evidence.test.ts b/tests/lab/lab-passive-production-evidence.test.ts index fd8a18b7d1..471aba9505 100644 --- a/tests/lab/lab-passive-production-evidence.test.ts +++ b/tests/lab/lab-passive-production-evidence.test.ts @@ -1,3 +1,4 @@ +import { readResponsesCoreSource } from "../helpers/responses-core-source"; import { mkdtempSync, readFileSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; @@ -271,7 +272,7 @@ describe("CL-09 no-feedback architecture guards", () => { }); test("production request path only links the exact subject and never reads passive history", () => { - const source = readFileSync("src/server/responses/core.ts", "utf8"); + const source = readResponsesCoreSource(); // Inverted by devlog/_fin/260814_lab_core_decoupling: subject construction moved OUT of // the per-request path into a core-owned slot, so an install with no routing profile // executes no Lab code. Core must now name only the slot, never Lab. diff --git a/tests/lib/reasoning-replay-scope-source.test.ts b/tests/lib/reasoning-replay-scope-source.test.ts index 17cfdb7607..a76b62daa3 100644 --- a/tests/lib/reasoning-replay-scope-source.test.ts +++ b/tests/lib/reasoning-replay-scope-source.test.ts @@ -1,3 +1,4 @@ +import { readResponsesCoreSource } from "../helpers/responses-core-source"; import { describe, expect, test } from "bun:test"; import { readFileSync } from "node:fs"; import { join } from "node:path"; @@ -8,7 +9,7 @@ const source = (relative: string): string => describe("reasoning replay scope propagation", () => { test("every production bridge call passes the provider-bound scope holder", () => { - const core = source("server/responses/core.ts"); + const core = readResponsesCoreSource(); const images = source("images/loop.ts"); const webSearch = source("web-search/loop.ts"); expect(core.match(/replayCacheScope: parsed\._reasoningReplayScope,/g)).toHaveLength(4); diff --git a/tests/lib/transient-budget-scope-source.test.ts b/tests/lib/transient-budget-scope-source.test.ts index 4a192a43a3..8bc9c8d0ea 100644 --- a/tests/lib/transient-budget-scope-source.test.ts +++ b/tests/lib/transient-budget-scope-source.test.ts @@ -1,5 +1,6 @@ +import { readResponsesCoreSource } from "../helpers/responses-core-source"; test("the gated-model 400 ladder is charged, and keeps its own bound", () => { - const core = source("server/responses/core.ts"); + const core = readResponsesCoreSource(); // Every rung reserves and charges, so the ladder is visible to later legs instead of // spending the request's allowance invisibly -- that part was the real defect. expect(core).toContain("targetKey: ladderTargetKey,"); @@ -39,7 +40,7 @@ const source = (relative: string): string => */ describe("transient send budget stays request-scoped", () => { test("every transient-retry call site draws from the shared counter", () => { - const core = source("server/responses/core.ts"); + const core = readResponsesCoreSource(); // One holder per LOGICAL request, read before any leg can send and inherited by combo // children through the options spread rather than recreated per child turn. @@ -144,7 +145,7 @@ describe("every dispatch path reports into the shared budget", () => { }); test("credential hops keep their roster cap AND reserve from the shared budget", () => { - const core = source("server/responses/core.ts"); + const core = readResponsesCoreSource(); // Six hop sites: the native passthrough 429, the shared sidecar hook's generic and // Anthropic arms, the runTurn preflight 429, the adapter recovery loop, and the // continuation loop. The last two were the arms that actually iterate the roster, so @@ -163,7 +164,7 @@ describe("every dispatch path reports into the shared budget", () => { }); test("the gated-model 400 ladder is charged, and keeps its own bound", () => { - const core = source("server/responses/core.ts"); + const core = readResponsesCoreSource(); // Every rung reserves and charges, so the ladder is visible to later legs instead of // spending the request's allowance invisibly -- that was the real defect. expect(core).toContain("targetKey: ladderTargetKey,"); diff --git a/tests/oauth/generic-oauth-failover.test.ts b/tests/oauth/generic-oauth-failover.test.ts index fbf47c776a..ea8eff11bb 100644 --- a/tests/oauth/generic-oauth-failover.test.ts +++ b/tests/oauth/generic-oauth-failover.test.ts @@ -1,3 +1,4 @@ +import { readResponsesCoreSource } from "../helpers/responses-core-source"; import { afterEach, beforeEach, describe, expect, test } from "bun:test"; import { mkdtempSync, readFileSync} from "node:fs"; import { tmpdir } from "node:os"; @@ -279,10 +280,7 @@ describe("#2568 generic OAuth account failover", () => { * first place: the main response path grew generic rotation and the two sidecars did not. */ describe("sidecar on429 wiring", () => { - const coreSource = readFileSync( - repoPath("src", "server", "responses", "core.ts"), - "utf8", - ); + const coreSource = readResponsesCoreSource(); test("both sidecar loops receive the SAME hook, so neither can drift key-pool-only", () => { const hooks = coreSource.match(/^\s*on429: (\w+),$/gm)?.map(line => line.trim()) ?? []; diff --git a/tests/responses/passthrough-abort.test.ts b/tests/responses/passthrough-abort.test.ts index 913da3eadd..0ce5b112ce 100644 --- a/tests/responses/passthrough-abort.test.ts +++ b/tests/responses/passthrough-abort.test.ts @@ -43,7 +43,7 @@ async function readAll(stream: ReadableStream): Promise { describe("passthrough relayWithAbort (RC2, passthrough path)", () => { test("native passthrough SSE keeps the real platform gate and pure native relay invariants", async () => { - const coreSource = await readSource("src/server/responses/core.ts"); + const coreSource = await readSource("src/server/responses/passthrough-delivery.ts"); const relaySource = await readSource("src/server/relay.ts"); const capsSource = await readSource("src/lib/bun-stream-caps.ts"); const sseBranch = coreSource.slice( @@ -79,11 +79,11 @@ describe("passthrough relayWithAbort (RC2, passthrough path)", () => { expect(sseBranch).toContain("rewriteBlocks: clientBlockRewrite"); // Elsewhere the failed-tail relay converts mid-stream resets into a clean response.failed. expect(sseBranch).toMatch( - /relaySseWithFailedTail\(\s*rewrittenBody,\s*upstream,\s*reason\s*=>\s*\{\s*responseCompletionCancelled\s*=\s*true;\s*clientGone\.abort\(reason\);\s*\},\s*\{\s*upstreamError:\s*logCtx\.upstreamError,\s*terminalBoundary:\s*codexSafetyBufferingOptions\s*\},\s*\)/, + /relaySseWithFailedTail\(\s*rewrittenBody,\s*upstream,\s*reason\s*=>\s*\{\s*responseEffects\.responseCompletionCancelled\s*=\s*true;\s*clientGone\.abort\(reason\);\s*\},\s*\{\s*upstreamError:\s*logCtx\.upstreamError,\s*terminalBoundary:\s*codexSafetyBufferingOptions\s*\},\s*\)/, ); expect(sseBranch).toContain("new Response(clientBody"); expect(sseBranch).toContain("markNativePassthroughSseResponse"); - // #314/phase 100 two-platform contract: the real core gate delegates to the + // #314/phase 100 two-platform contract: the delivery owner delegates to the // selector, whose darwin branch admits only explicit config-eager decisions. expect(sseBranch).toContain("const eagerPath = selectEagerPath("); expect(sseBranch).toContain("config.streamMode ?? \"auto\","); diff --git a/tests/responses/responses-core-modules.test.ts b/tests/responses/responses-core-modules.test.ts new file mode 100644 index 0000000000..2d94f02f98 --- /dev/null +++ b/tests/responses/responses-core-modules.test.ts @@ -0,0 +1,177 @@ +import { describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { repoPath } from "../helpers/repo-root"; +import { + RESPONSES_CORE_MODULES, + readResponsesCoreModule, +} from "../helpers/responses-core-source"; +import { createResponsesSendBudget } from "../../src/server/responses/request-send-budget"; +import { createRequestExecutionBudget } from "../../src/lib/request-execution-budget"; +import { createTranslatorBudget } from "../../src/lib/translator-budget"; +import type { TransientSendBudget } from "../../src/lib/upstream-retry"; + +// Existing, separately owned siblings at the extraction boundary. A new owner +// cannot silently disappear from source-oracle coverage by being absent from the inventory. +const EXISTING_BOUNDARIES = new Set([ + "account-change-state.ts", "agent-task-recovery.ts", "codex-auth-error.ts", + "codex-ws-metadata.ts", "codex-ws-wire.ts", "collaboration.ts", + "combo-session-recall.ts", "combo-stream-preflight.ts", "context-overflow.ts", + "empty-completion-guard.ts", "encrypted-payload.ts", "fetch-helpers.ts", + "input-admission.ts", "outbound-body-guard.ts", "passthrough-error.ts", + "responses-field-backfill.ts", "terminal-guard.ts", "upstream-error.ts", "ws-upstream.ts", +]); + +function siblingImports(source: string): string[] { + return Array.from(source.matchAll(/\bfrom\s+["']\.\/([^"']+)["']/g), match => + match[1]!.endsWith(".ts") ? match[1]! : `${match[1]}.ts`); +} + +function ownerGraph(): Map { + const graph = new Map(); + const pending = ["core.ts"]; + while (pending.length > 0) { + const name = pending.pop()!; + if (graph.has(name) || EXISTING_BOUNDARIES.has(name)) continue; + const source = readFileSync(repoPath("src", "server", "responses", name), "utf8"); + const children = siblingImports(source).filter(child => !EXISTING_BOUNDARIES.has(child)); + graph.set(name, children); + pending.push(...children); + } + return graph; +} + +describe("Responses core module boundaries", () => { + test("every extracted owner is covered and remains below 2000 physical lines", () => { + const graph = ownerGraph(); + expect([...graph.keys()].sort()).toEqual([...RESPONSES_CORE_MODULES].sort()); + for (const name of RESPONSES_CORE_MODULES) { + const text = readResponsesCoreModule(name); + const lines = text.split("\n").length - (text.endsWith("\n") ? 1 : 0); + expect({ name, belowLimit: lines < 2000 }).toEqual({ name, belowLimit: true }); + } + }); + + test("owner dependencies are acyclic, including type-only state contracts", () => { + const graph = ownerGraph(); + const complete = new Set(); + const active = new Set(); + const visit = (name: string): void => { + expect({ name, cycle: active.has(name) }).toEqual({ name, cycle: false }); + if (complete.has(name)) return; + active.add(name); + for (const child of graph.get(name) ?? []) visit(child); + active.delete(name); + complete.add(name); + }; + visit("core.ts"); + }); + + test("recursive combo dispatch enters the public ingress without a reverse core import", () => { + const combo = readResponsesCoreModule("core-combo.ts"); + const prepare = readResponsesCoreModule("request-prepare.ts"); + expect(combo).toContain("requestDispatchers.handleResponses("); + expect(prepare).toContain("requestDispatchers.handleComboResponses("); + for (const name of RESPONSES_CORE_MODULES) { + if (name !== "core.ts") expect(siblingImports(readResponsesCoreModule(name))).not.toContain("core.ts"); + } + expect(readResponsesCoreModule("core.ts")) + .toContain("const requestDispatchers: ResponsesDispatchers = { handleResponses, handleComboResponses };"); + }); + + test("lease transfer retains both finally owners until response construction settles", () => { + const ingress = readResponsesCoreModule("core.ts"); + const native = readResponsesCoreModule("passthrough-execution.ts"); + expect(ingress).toContain("return await executePassthroughResponse("); + expect(native).toContain("return await deliverPassthroughResponse("); + expect(native.indexOf("admissionState.pendingHostAdmissionLease = null;")) + .toBeLessThan(native.indexOf("await preparePassthroughExchange(")); + expect(native).toMatch(/finally\s*\{\s*if \(nativeHostState\.lease\)\s*\{\s*releaseUpstreamHostAdmission\(nativeHostState\.lease\);\s*releaseCodexAuthContextProbeLease\(admissionState\.authCtx\);/); + expect(ingress).toMatch(/finally\s*\{\s*if \(admissionState\.pendingHostAdmissionLease\)/); + }); + + test("local admission decisions cannot shadow the outer lease owner", () => { + const prepare = readResponsesCoreModule("request-prepare.ts"); + expect(prepare).toContain("const admission = acquireUpstreamHostAdmission("); + expect(prepare).toContain("admissionState.pendingHostAdmissionLease = admission.lease;"); + expect(prepare).not.toContain("admission.pendingHostAdmissionLease = admission.lease;"); + }); + + test("live adapter, alias and continuation counters are not copied into snapshots", () => { + const transport = readResponsesCoreModule("request-transport.ts"); + const effects = readResponsesCoreModule("response-effects.ts"); + const exchange = readResponsesCoreModule("adapter-dispatch.ts"); + const continuation = readResponsesCoreModule("adapter-continuation.ts"); + for (const name of ["activeAdapter", "runTurnAdapter", "sameTargetRequest", "transportToken", "genericFailovers"]) { + expect(transport).toContain(`get ${name}()`); + expect(transport).toContain(`set ${name}(value:`); + } + expect(effects).toContain("set responseCompletionCancelled(value:"); + expect(exchange).toContain("set rateLimitRetries(value:"); + expect(continuation).toContain("adapterExchange.rateLimitRetries"); + expect(continuation).toContain("transportState.activeAdapter"); + }); +}); + +function budgetOwner(sendBudget: TransientSendBudget) { + const translatorBudget = createTranslatorBudget(); + const result = createResponsesSendBudget({ + req: new Request("http://localhost/v1/responses"), + logCtx: { model: "test", provider: "test" }, + options: { translatorBudget, sendBudget }, + }); + if (result instanceof Response) { + translatorBudget.dispose(); + throw new Error("Unexpected workflow refusal without a workflow root"); + } + return { owner: result, dispose: () => translatorBudget.dispose() }; +} + +describe("Responses request-owned send budget after extraction", () => { + test("legacy holders retain identity and an exhausted remainder stays zero", () => { + const holder = { used: 2 }; + const { owner, dispose } = budgetOwner(holder); + try { + expect(owner.remainingTransientSendBudget(3)).toBe(1); + owner.noteTransientSends(1); + expect(holder.used).toBe(3); + expect(owner.remainingTransientSendBudget(3)).toBe(0); + expect(owner.adapterSendBudget).toBeUndefined(); + } finally { dispose(); } + }); + + test("two call frames inheriting one holder consume the same allowance", () => { + const holder = createRequestExecutionBudget(); + const a = budgetOwner(holder); + const b = budgetOwner(holder); + try { + expect(a.owner.adapterSendBudget).toBe(holder); + expect(b.owner.adapterSendBudget).toBe(holder); + a.owner.noteTransientSends(1); + b.owner.noteTransientSends(1); + expect(holder.used).toBe(2); + expect(a.owner.remainingTransientSendBudget(3)).toBe(1); + expect(b.owner.remainingTransientSendBudget(3)).toBe(1); + } finally { a.dispose(); b.dispose(); } + }); + + test("a transferred recovery permit is the exact closure-owned single-use permit", () => { + const holder = createRequestExecutionBudget(); + const { owner, dispose } = budgetOwner(holder); + try { + owner.noteTransientSends(3); + const hop = owner.reserveCredentialHop("auth-recovery", "test|model", true); + expect(hop.allowed).toBe(true); + if (!hop.permit) throw new Error("Expected a recovery permit"); + owner.pendingHopPermit = hop.permit; + const allowance = owner.recoverySendAllowance(3, "auth-recovery", "test|model"); + expect(allowance.attempts).toBe(1); + expect(allowance.permit).toBe(hop.permit); + expect(owner.pendingHopPermit).toBeUndefined(); + expect(hop.permit.use()).toBe(true); + expect(hop.permit.use()).toBe(false); + owner.noteTransientSends(1); + expect(holder.used).toBe(4); + expect(owner.remainingTransientSendBudget(3)).toBe(0); + } finally { dispose(); } + }); +}); diff --git a/tests/routing/subagent-fallback-handle-responses.test.ts b/tests/routing/subagent-fallback-handle-responses.test.ts index a9fdcf6348..be6ec1d6cb 100644 --- a/tests/routing/subagent-fallback-handle-responses.test.ts +++ b/tests/routing/subagent-fallback-handle-responses.test.ts @@ -1583,7 +1583,7 @@ describe("native fallback account preview", () => { */ test("both fallback preview sites pass the model-eligible account set (#2509)", async () => { const source = await Bun.file( - fileURLToPath(new URL("../../src/server/responses/core.ts", import.meta.url)), + fileURLToPath(new URL("../../src/server/responses/request-prepare.ts", import.meta.url)), ).text(); const previews = source.match(/subagentFallbackAccountPreview = \([^)]*\)/g) ?? []; diff --git a/tests/server/cancel-body-on-abort.test.ts b/tests/server/cancel-body-on-abort.test.ts index 23dbb3e394..decf0ebcd2 100644 --- a/tests/server/cancel-body-on-abort.test.ts +++ b/tests/server/cancel-body-on-abort.test.ts @@ -1,3 +1,4 @@ +import { readResponsesCoreSource } from "../helpers/responses-core-source"; import { describe, expect, test } from "bun:test"; import { cancelBodyOnAbort } from "../../src/lib/abort"; import { readBodyCapped } from "../../src/server/live"; @@ -83,7 +84,7 @@ describe("readBodyCapped settles the stream when a read throws", () => { }); test("the bounded reader exclusively owns all non-combo Responses error bodies", async () => { - const source = await Bun.file(new URL("../../src/server/responses/core.ts", import.meta.url)).text(); + const source = readResponsesCoreSource(); expect(source.match(/\breadDisplaySafeErrorText\(/g)).toHaveLength(4); expect(source).not.toContain("detachPassthroughErrorGuard"); @@ -101,7 +102,7 @@ describe("readBodyCapped settles the stream when a read throws", () => { // tests/server/server-combo-failover-e2e.test.ts). An earlier revision guarded them anyway and // broke that test by adding a second `.body` read. test("the combo failure branches do not add a second body read", async () => { - const source = await Bun.file(new URL("../../src/server/responses/core.ts", import.meta.url)).text(); + const source = readResponsesCoreSource(); for (const marker of ["const failure = await consumeComboFailure("]) { let from = 0; diff --git a/tests/server/passive-route-linker.test.ts b/tests/server/passive-route-linker.test.ts index a8c6dec203..f235448d0f 100644 --- a/tests/server/passive-route-linker.test.ts +++ b/tests/server/passive-route-linker.test.ts @@ -1,3 +1,4 @@ +import { readResponsesCoreSource } from "../helpers/responses-core-source"; import { describe, expect, test, beforeEach } from "bun:test"; import { setPassiveRouteLinker, @@ -64,8 +65,8 @@ describe("passive route linker slot", () => { describe("core request path boundary", () => { // Guard 1 for this phase: the per-request module must not name Lab or the // compatibility layer at all. Driven red by restoring the old import. - test("responses/core.ts does not import lab or routing/compatibility", async () => { - const source = await Bun.file(new URL("../../src/server/responses/core.ts", import.meta.url)).text(); + test("Responses owners do not import lab or routing/compatibility", async () => { + const source = readResponsesCoreSource(); expect(source).not.toContain("routing/compatibility"); expect(source).not.toContain('from "../../lab/'); expect(source).not.toContain("resolveProductionRouteSubject");