diff --git a/devlog/_plan/260907_track2_protocol/000_plan.md b/devlog/_plan/260907_track2_protocol/000_plan.md new file mode 100644 index 0000000000..42ded4b4ff --- /dev/null +++ b/devlog/_plan/260907_track2_protocol/000_plan.md @@ -0,0 +1,28 @@ +# Track 2 protocol delivery + +- Archetype: satisfy-spec repair with an evidence-backed defer outcome. +- Trigger: maintainer assigns track 2 and authorizes ordinary PR chains, no-verify pushes, final remote CI first, and admin integration. +- Goal: preserve Chat JSON/SSE semantics (#3770/#3779), refusal (#3767), supported custom efforts (#3775), hosted-search execution (#3761), and opt-in Claude compatibility (#3730), or document a concrete unresolved blocker. +- Non-goals: native GitHub stacks, local tests/typechecks/builds/install, other tracks, service/config changes outside repository, releases/deployments. Do not weaken CI definitions or treat skipped tests as passing. +- Baseline: dev 7d8523eed75a67f7a4a15b533744fcd0e6059aa8, including #3771. +- Verifier: existing workflow_dispatch ci.yml lane=all at the final integration head; lower diagnostic CI only if final fails. Commands are NOT RUN locally by explicit instruction. Read workflow definitions to establish target coverage. Independent source review precedes remote execution. +- Stop: feasible reviewed changes land through dev PRs with verification evidence; other items receive explicit evidence-backed dispositions. No completion claims for deferred issues. +- Artifact: this numbered unit; private security analysis and raw tool results only in /tmp/cf54-*. +- Expected outcomes: landed, already implemented, deferred with concrete blocker, or blocked by external CI/service state. +- Escalation: parent reclaims failed worker slices; never widen auth/routing trust or retry provider work to make a test pass. No user budget was set. + +## Roadmap + +1. Docs-only roadmap and independent audit. +2. 010: JSON Responses to streaming Chat semantics; carry #3779 with attribution. +3. 020: refusal across live SSE, final snapshots, JSON, collection, and JSON-to-SSE. +4. 030: custom effort provenance/capability repair after independent Codex source check. +5. 040: hosted-search path feasibility, then scoped execution/continuation repair or defer. +6. 050: opt-in Claude compatibility gate after independent official-contract/security audit or defer. +7. 060: final source audit, remote CI, ordinary PR-chain integration and exact dev ancestry proof. + +The semantic stack is JSON fallback -> refusal. Catalog and Claude slices have disjoint implementation owners and join the integration tip. Source refs are ordinary branches, not registered native stacks. Do not run a separate lower-level CI before the final integration failure. + +## Process availability + +Installed cxc skills resolved to 0.2.20 because the named 0.2.19 directory is absent. No SessionStart binding was injected into this task; SESSION-IDENTITY-01 forbids borrowing a prior/transcript id. Therefore no FSM activation is claimed. Durable P/A/B/C/D artifacts and the native active goal still track authorized work; tests remain pending until remote evidence exists. diff --git a/devlog/_plan/260907_track2_protocol/010_chat_json_sse.md b/devlog/_plan/260907_track2_protocol/010_chat_json_sse.md new file mode 100644 index 0000000000..437133fce5 --- /dev/null +++ b/devlog/_plan/260907_track2_protocol/010_chat_json_sse.md @@ -0,0 +1,22 @@ +# JSON Responses to streaming Chat + +Depends on roadmap only. Class C3; PR #3779 is the public implementation source. + +## File delta + +- MODIFY src/server/chat-completions.ts:455: keep responsesJsonToChatCompletion as semantic authority. Replace text-only extraction and constant stop with converted choice.message content/reasoning_content/refusal; project tool_calls with stable array-order indices; preserve converted finish_reason. One role event, at most one combined delta, one terminal and one DONE. No extra upstream inference. +- NEW tests/responses/chat-json-sse-fallback.test.ts from the source PR: actual loopback Responses upstream -> handleChatCompletions. Include one/two tools, reasoning+text, incomplete length, empty completion, cancellation and translator-budget release. +- MODIFY scripts/test-layout/layout.json and tests/fixtures/test-layout-expected.json: register new test under responses. +- MODIFY docs-site/src/content/docs/reference/proxy-formats.md and structure/04_transports-and-sidecars.md: buffered fallback delivery, semantic parity and no additional request. + +## Activation and oracle + +Streaming Chat request + JSON Responses upstream is the trigger. Native Chat and real SSE bypass this path. Hardcoded official Chat fixtures require indexed function calls, nullable finish for intermediate chunks and original terminal finish. First-choice scope follows the existing Responses single-result contract. Official openai-node ChatCompletionChunk/ChatCompletionMessage are the independent shape oracle; source PR tests are evidence candidates, not a passing result. + +## Check and delivery + +No local execution. Final remote CI must execute tests/responses/chat-json-sse-fallback.test.ts and existing chat-completions-endpoint coverage, typecheck and test-layout guards. Preserve upstream author credit in carried commit and final PR body. Lower refs are published with --no-verify; no native stack registration. + +## Build checkpoint + +Carried #3779 and applied independent-audit corrections: shared native serializer, indexed tools, typed unknown incompletes, correct length/content_filter precedence, and explicit converted/serialized byte ownership. New tests preserve the source PR cases and add official-contract boundary/accounting cases. Local suites/typecheck/build NOT RUN by instruction; git diff --check is a whitespace check only. Remote verification remains pending. diff --git a/devlog/_plan/260907_track2_protocol/011_chat_audit_amendment.md b/devlog/_plan/260907_track2_protocol/011_chat_audit_amendment.md new file mode 100644 index 0000000000..4634664830 --- /dev/null +++ b/devlog/_plan/260907_track2_protocol/011_chat_audit_amendment.md @@ -0,0 +1,15 @@ +# Chat plan audit resolution + +Independent reviewer Fermat returned GO-WITH-FIXES, five blockers. No local command was executed. + +1. Accepted: source oracle differs from current converter. Incomplete max_output_tokens -> length and content_filter -> content_filter take precedence over tools. Incomplete missing/max_messages/steered/other reason becomes typed upstream truncation rather than fabricated token exhaustion, matching existing live-SSE handling of unknown incompletes. Both JSON and SSE public handlers map typed error, without success DONE. +2. Accepted with scope: use existing budget owner; add optional budget to pure converter so runtime caller charges retained copied content/reasoning/tools. Reuse existing native jsonCompletionSse owner for both JSON-to-SSE routes after it gains optional budget-aware serialization and proper final tool indexes. Charge serialized strings and output buffer while simultaneously live; release temporaries on transfer, retain response bytes until consumption/cancel finalization. No general translator-budget refactor. Positive charge and small configured-budget overflow tests required; no local runs. +3. Accepted: refusal parts indexed by original output_index/content_index; item.id/item_id are optional correlation constraints and a present mismatch fails. Preserve original array positions. Buffer refusal parts until terminal and emit in output/content order, avoiding interleaved-part reordering. Deltas append; equal snapshots deduplicate; extending snapshots fill suffix; shorter-prefix/empty snapshot preserves known data (sparse compatible provider); absent field is no new evidence; explicit non-string/contradictory non-prefix snapshot fails. Budget text plus per-entry metadata/key bytes, release on terminal/cancel/fail. Zero-length parts cannot bypass map accounting. Final JSON and collector use nullable refusal field. +4. Accepted: native JSON-to-SSE uses same shared helper and preserves refusal. Matrix covers native/translated upstream JSON/SSE with client JSON/SSE. Native streaming passthrough remains opaque. +5. Accepted: collectChatCompletion catch cancels reader before releasing lock, invoking upstream translator cancel; tests prove cancellation under processing overflow and no successful partial JSON. Existing outer budget finalizer remains final response owner. + +These replace conflicting portions of 010/020. Re-audit before source edits. Official SDK source field definitions retained in /tmp/cf54-openai-responses-types.ts and /tmp/cf54-openai-chat-types.ts. Aside page-open reached its host deadline without content; no browser-source proof claimed. + +## Re-audit resolution + +Fermat re-audit accepted the five resolutions and found one remaining terminal-order blocker. Accepted: stage all final role/tool/refusal/finish/DONE frames as one bounded terminal batch; serialize and reserve every frame before enqueueing any success frame. On reservation failure, release the staged reservations/refusal state and emit only the bounded typed overflow error (no success finish or DONE). Commit terminated success only after batch admission. Merely moving the terminated assignment is insufficient. Add a small-budget fixture that fails at final batch admission and asserts absence of success finish/DONE plus typed error and cancellation. diff --git a/devlog/_plan/260907_track2_protocol/020_refusal.md b/devlog/_plan/260907_track2_protocol/020_refusal.md new file mode 100644 index 0000000000..8bacb10a5d --- /dev/null +++ b/devlog/_plan/260907_track2_protocol/020_refusal.md @@ -0,0 +1,17 @@ +# Preserve refusal across Chat projections + +Depends on 010 for JSON-to-SSE field delivery. Class C3 public wire contract. + +## File delta + +- MODIFY src/chat/outbound.ts: responsesJsonToChatCompletion accumulates content parts with type refusal and their refusal string into message.refusal, alongside existing content/reasoning/tool fields. +- MODIFY same file live translator: map response.refusal.delta to delta.refusal. Track each output/content part separately with the existing translator budget. Final response.refusal.done, content_part.done, output_item.done and completed/incomplete snapshots may add only an unseen matching suffix. Repeated final representations must not duplicate text. Conflicting snapshots cannot be represented as append-only deltas and must terminate as a typed translation failure, not false success. Map storage and release follow existing turn-budget lifecycle. +- MODIFY collectChatCompletion: collect delta.refusal with retained_collectors budget and serialize message.refusal. Never coerce refusal into ordinary assistant answer text. +- NEW tests/responses/chat-refusal.test.ts (register both test-layout inventories): cover direct JSON, split live deltas plus all snapshot representations, done-only, terminal-only, multiple parts, mixed text/refusal, stream collection, contradictory snapshot failure, cancellation/overflow, and JSON-to-SSE handler path inherited from 010. +- MODIFY proxy-formats.md and structure/04_transports-and-sidecars.md: document refusal field and parity across delivery shapes without claiming a new policy decision. + +## Independent oracle and acceptance + +OpenAI Responses docs define refusal.delta.delta and refusal.done.refusal, indexed by output_index/content_index; official Chat SDK defines delta.refusal and message.refusal. Local official Codex corpus is read only for consumer behavior; it is not automatically the sole Chat API schema authority. + +Trigger known refusal events/parts, expect exactly one concatenated refusal, unchanged normal content/tool semantics, one terminal+DONE on valid completion, typed failure without success DONE on invalid final snapshot or overflow. Tests use inert fixture messages rather than provoking live model refusals. No local suites run; remote final CI owns executable proof. diff --git a/devlog/_plan/260907_track2_protocol/030_custom_efforts.md b/devlog/_plan/260907_track2_protocol/030_custom_efforts.md new file mode 100644 index 0000000000..0a8856a0bb --- /dev/null +++ b/devlog/_plan/260907_track2_protocol/030_custom_efforts.md @@ -0,0 +1,18 @@ +# Proven custom native capability projection + +Depends on roadmap; independent from Chat semantics. Class C3. Issue #3775 remains partial because an arbitrary gateway model name does not prove native capability, and current official source is not a binary proof for Desktop 0.153.4. + +## File delta + +- MODIFY src/codex/catalog/provider-fetch.ts: in current custom-row producer, retain existing canonical openai forward destination and capability-backed model ID proof. AFTER custom/inherited metadata merge, intersect explicit reasoningEfforts with nativeReasoningEfforts for that proved model. If explicit [] preserve [] and remove default; if nonempty declared list has no supported entry, use proved native default as singleton. Otherwise choose declared default only if present, then proved default if present, then first surviving effort. Do not clamp arbitrary provider/model names, destination overrides, or ordinary routed custom models. +- MODIFY src/codex/catalog/sync.ts: in retained sync merge, current invocation's live custom rows must not have max re-added. Use current config/producer provenance rather than a disk marker. Keep ordinary provider/combo/Reserve rules. +- MODIFY existing catalog-custom-models, sync-hardening, convergence and Claude model-discovery tests after reading actual filenames: canonical Astra with none/minimal + valid ladder; explicit []; all-invalid nonempty; valid default preserved; same-name noncanonical gateway unchanged; destination override unchanged; second sync no max resurrection; both gather entry points and /models client-version projection. +- MODIFY relevant English catalog/reasoning reference plus structure/03_catalog-and-subagents.md to state the narrow capability proof and explicit empty-list behavior. Translations must not claim broader gateway/client repair. + +## Official evidence and deferrals + +Local official source corpus 121_openai-codex: protocol/src/openai_models.rs allows nonempty custom effort strings; models-manager/src/manager.rs qualified lookup is consumer lookup, not gateway provenance; multi_agents_common.rs validates chosen-row membership. Codex source supports ultra and translates it on wire. API model docs lacking ultra do not justify deleting Codex ultra. + +Field chain is existing custom config -> fetch custom row -> deriveEntry/retained merge -> catalog file/direct /models consumers. No new schema field or request-time effort override. Existing threads and version-specific stale runtime state are outside this repair. + +Source/code inspection is completed; proposed regression commands are NOT RUN locally. Final remote CI must exercise modified tests and typecheck. #3775 must remain open for gateway destination and exact Desktop/version evidence; this fix addresses only proven canonical rows. diff --git a/devlog/_plan/260907_track2_protocol/040_hosted_search_disposition.md b/devlog/_plan/260907_track2_protocol/040_hosted_search_disposition.md new file mode 100644 index 0000000000..44e3c7714e --- /dev/null +++ b/devlog/_plan/260907_track2_protocol/040_hosted_search_disposition.md @@ -0,0 +1,13 @@ +# Hosted search passthrough disposition + +Outcome: DEFER #3761; no production diff in this track. + +## Source findings + +src/web-search/loop.ts mutates normalized messages, while src/adapters/openai-responses.ts serializes passthrough _rawBody. Existing loop parsing is compaction-oriented and does not preserve the native tool/reasoning conversation needed for search-result continuation. src/server/sse-payload-rewrite.ts is synchronous rewriting rather than an asynchronous execution loop. Mixed ordinary tools/search and replay need a separate raw conversation contract; continuation storage alone does not supply it. + +Official Ollama local middleware supports hosted Responses search, including cloud model execution, but that does not establish the direct ollama.com/v1 endpoint contract. References opened during investigation: https://github.com/ollama/ollama/pull/17686 and https://docs.ollama.com/integrations/codex . Local official client source: corpus 121_openai-codex. No direct-cloud authenticated run was performed. + +## Resume criteria + +Define destination/backend execution policy; preserve raw Responses tools/reasoning when inserting search results; exercise mixed tools, cancellation, bounded iteration, SSE/JSON/WS, compact and replay remotely with a confirmed destination contract. A guard-only change is rejected because it cannot deliver these results. The issue remains open and will receive this disposition; no claimed fix, workflow or client change. diff --git a/devlog/_plan/260907_track2_protocol/050_claude_compatibility.md b/devlog/_plan/260907_track2_protocol/050_claude_compatibility.md new file mode 100644 index 0000000000..7cf73f0b0b --- /dev/null +++ b/devlog/_plan/260907_track2_protocol/050_claude_compatibility.md @@ -0,0 +1,23 @@ +# Claude compatibility carry + +Depends on roadmap and an independent source/security audit. Class C3 with C4 review of admission and persisted diagnostics. Public source is PR #3730 at 18d64748ade8001e05327726b2ae4b22e8393418. + +## Published scope and file map + +- NEW src/claude/compatibility.ts: opt-in analysis for translated Messages; no Lab imports, source envelope, credential access or adapter execution. +- MODIFY src/server/claude-messages.ts: gate translated requests after real native passthrough returns and before inference. Preserve existing auth/origin and logging ownership. +- MODIFY src/types/config.ts: compatibility mode property. +- MODIFY src/server/request-log.ts and src/usage/log.ts: bounded optional metadata, persisted-row normalization and hydration. +- NEW tests/claude-integration/claude-compatibility.test.ts and MODIFY existing endpoint/usage tests; register new filename in both test-layout inventories. +- MODIFY server configuration reference and Claude guide only where current scope needs clarification. + +The source PR needs substantial classifier corrections before adoption. Detailed security-sensitive findings, official feature matrix, exact corrections and review evidence are held in task scratch space. The parent accepts a uniform conservative translated-path contract; no per-adapter exemption based on preliminary routing. Existing unset/native behavior remains unchanged. Shadow is observational and enforce is endpoint compatibility admission, not a global security boundary. + +## Field chain and acceptance + +Mode: typed config -> persisted JSON -> existing loader -> translated Messages gate. Present invalid mode must produce a fixed visible configuration failure rather than silently disable checking. No global config fallback change. +Evidence: gate -> request context -> ring -> usage row -> normalized disk row -> hydration. Only closed protocol codes may be stored; no body/header/credential/signature material. Old rows remain valid. + +Test unset/shadow/enforce/invalid modes, real native bypass versus translated Anthropic, zero-inference rejected requests, normal tools versus hosted feature declarations, nested supported content positions, large headers, persistence and reload. Exact behavior follows the private reviewed feature matrix. No local test/typecheck/build/install; final remote CI and explicit independent security review required. + +Co-authored-by: SB Yoon <44089734+yansigit@users.noreply.github.com> diff --git a/devlog/_plan/260907_track2_protocol/060_remote_delivery.md b/devlog/_plan/260907_track2_protocol/060_remote_delivery.md new file mode 100644 index 0000000000..612f9342fd --- /dev/null +++ b/devlog/_plan/260907_track2_protocol/060_remote_delivery.md @@ -0,0 +1,15 @@ +# Remote validation and delivery + +Depends on all accepted implementation slices. Class C3 integration; any admission changes require independent security review. + +## Delta + +- MODIFY only this unit's outcome/evidence record after source review. +- Publish ordinary branches using git push --no-verify. Native stack registration and repo-wide workflow edits are excluded. +- Final integration branch contains all accepted layers and current dev. Dispatch existing .github/workflows/ci.yml lane=all at its exact SHA; inspect every expected Linux/macOS/Windows shard and supporting gate. No local tests, typecheck, builds or install. +- If final CI fails, inspect log/artifact and then use lower-head or bounded remote cases to isolate it. Never claim skipped/cancelled checks passed or silently weaken assertions. Existing automatic PR jobs may run; no fabricated status. +- Open template-complete ordinary parent/child PRs; preserve contributor trailers. Record local NOT RUN, final integration proof, independently verified issue scope and explicit maintainer integration. +- Merge accepted work through dev PRs with admin authority. Refresh actual base/head and maintainer objections before each write, retain parent refs while children target them, and prove each final merge is an ancestor of fetched dev. If dev changes in another track, review integration delta and refresh final proof where needed. +- Close only fully resolved issues and superseded source PRs with attribution and replacement links; partial/deferred issues remain open with precise status. + +No source test is executed merely to verify that its command exists. CI workflow and package scripts are the source-inspection evidence of coverage. Report what each remote job actually did. diff --git a/docs-site/src/content/docs/reference/proxy-formats.md b/docs-site/src/content/docs/reference/proxy-formats.md index bba98afef1..d2167bbfa4 100644 --- a/docs-site/src/content/docs/reference/proxy-formats.md +++ b/docs-site/src/content/docs/reference/proxy-formats.md @@ -251,6 +251,15 @@ Non-streaming output has `object: "chat.completion"`. Streaming output uses SSE `data: [DONE]`. Tool-call and usage information are translated back where the source events carry them. +If a streaming Chat request receives a complete JSON Responses result upstream, the proxy +synthesizes SSE from the converted completion. It preserves answer and reasoning content, +function tool calls (with a separate stream `index` for each call), usage, and the converted +`finish_reason`, including `tool_calls` and `length`. This fallback delivers the completed result +in chunks; it cannot provide token-by-token delivery before the upstream JSON response arrives. +It does not issue an additional inference request. An incomplete response caused by the output +token limit or content filtering retains `length` or `content_filter`, even if it includes tool +output. Other incomplete boundaries return an upstream error instead of claiming a normal finish. + Because the internal execution path is Responses-based, a provider adapter can impose a narrower feature set. For example, a request feature that cannot be represented by the selected adapter is returned as an error instead of silently changing its meaning. diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 0fbe7cf746..2e2b806dee 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -270,6 +270,7 @@ "catalog-verbosity-default.test.ts": "codex-integration", "catalog-vision-sidecar-modalities.test.ts": "codex-integration", "chat-completions-endpoint.test.ts": "responses", + "chat-json-sse-fallback.test.ts": "responses", "chatgpt-device-auth.test.ts": "oauth", "chatgpt-oauth.test.ts": "oauth", "chatgpt-token-expiry.test.ts": "oauth", diff --git a/src/chat/outbound.ts b/src/chat/outbound.ts index 03e133bb90..812f1b553f 100644 --- a/src/chat/outbound.ts +++ b/src/chat/outbound.ts @@ -560,51 +560,87 @@ export function responsesSseToChatCompletionsSse( } /** Non-streaming: /v1/responses JSON -> Chat Completions message JSON. */ -export function responsesJsonToChatCompletion(json: unknown, model: string): Rec { +export function responsesJsonToChatCompletion(json: unknown, model: string, translatorBudget?: TranslatorBudget): Rec { const body = isRec(json) ? json : {}; + const incomplete = isRec(body.incomplete_details) ? body.incomplete_details : {}; + let incompleteFinish: "length" | "content_filter" | undefined; + if (body.status === "incomplete") { + if (incomplete.reason === "max_output_tokens") incompleteFinish = "length"; + else if (incomplete.reason === "content_filter") incompleteFinish = "content_filter"; + else throw new ChatCompletionsStreamError("upstream response ended without a supported completion boundary", { + code: "upstream_incomplete", type: "upstream_error", + }); + } const output = Array.isArray(body.output) ? body.output : []; let content = ""; let reasoning = ""; + let contentBytes = 0; + let reasoningBytes = 0; const toolCalls: Rec[] = []; + const append = (previous: string, previousBytes: number, fragment: string): { text: string; bytes: number } => { + if (!fragment) return { text: previous, bytes: previousBytes }; + const scope = { kind: "retained_collectors" as const }; + const nextBytes = appendedUtf8Bytes(previous, previousBytes, fragment); + const reservation = translatorBudget?.reserveTransient(nextBytes, scope); + try { + const next = previous + fragment; + reservation?.commitRetained(); + translatorBudget?.releaseRetained(previousBytes, scope); + return { text: next, bytes: nextBytes }; + } catch (error) { + reservation?.release(); + throw error; + } + }; for (const raw of output) { if (!isRec(raw)) continue; if (raw.type === "message" && Array.isArray(raw.content)) { for (const part of raw.content) { if (isRec(part) && part.type === "output_text" && typeof part.text === "string") { - content += part.text; + ({ text: content, bytes: contentBytes } = append(content, contentBytes, part.text)); } } } else if (raw.type === "reasoning") { if (Array.isArray(raw.summary)) { for (const part of raw.summary) { if (isRec(part) && part.type === "summary_text" && typeof part.text === "string") { - reasoning += part.text; + ({ text: reasoning, bytes: reasoningBytes } = append(reasoning, reasoningBytes, part.text)); } } } if (Array.isArray(raw.content)) { for (const part of raw.content) { if (isRec(part) && part.type === "reasoning_text" && typeof part.text === "string") { - reasoning += part.text; + ({ text: reasoning, bytes: reasoningBytes } = append(reasoning, reasoningBytes, part.text)); } } } } else if (raw.type === "function_call") { - toolCalls.push({ + const call = { id: typeof raw.call_id === "string" ? raw.call_id : `call_${uuid().slice(0, 16)}`, type: "function", function: { name: typeof raw.name === "string" ? raw.name : "", arguments: typeof raw.arguments === "string" ? raw.arguments : "{}", }, + }; + // A complete buffered call still obeys the same per-call cap as live deltas. + // Reserve before serializing, then transfer ownership to the complete call. + // The internal scope stays nonempty even when an upstream call_id is empty. + const argumentsReservation = translatorBudget?.reserveTransient(Buffer.byteLength(call.function.arguments), { + kind: "tool_args", callId: `chat_json_${toolCalls.length}`, }); + try { + translatorBudget?.chargeRetained(Buffer.byteLength(JSON.stringify(call)), { kind: "retained_collectors" }); + toolCalls.push(call); + } finally { + argumentsReservation?.release(); + } } } - const finishReason = toolCalls.length > 0 ? "tool_calls" - : body.status === "incomplete" ? "length" - : "stop"; + const finishReason = incompleteFinish ?? (toolCalls.length > 0 ? "tool_calls" : "stop"); const message: Rec = { role: "assistant", diff --git a/src/server/chat-completions.ts b/src/server/chat-completions.ts index d66a0df0b6..e7fd04f42d 100644 --- a/src/server/chat-completions.ts +++ b/src/server/chat-completions.ts @@ -46,6 +46,7 @@ import { type TranslatorBudget, } from "../lib/translator-budget"; import { handleNativeChatCompletions, isNativeChatRouteEligible } from "./chat-native"; +import { jsonCompletionSse } from "./chat-native-sse"; import { parseRequestEffortRowId } from "./effort-row"; import { parseSyntheticRowId } from "./fast-row"; import { isCanonicalOpenAiForwardProvider } from "../providers/openai-tiers"; @@ -79,6 +80,14 @@ export async function handleChatCompletions( ); } catch (error) { translatorBudget.dispose(); + if (isTranslatorBudgetExceededError(error)) { + if (logIds) addFinalRequestLog(logIds.requestId, logIds.start, logCtx, 502, { closeReason: "non_stream" }); + return chatCompletionsErrorResponse(502, "upstream translation buffer exceeded the safe limit", "upstream_error", "translation_buffer_limit"); + } + if (isChatCompletionsStreamError(error)) { + if (logIds) addFinalRequestLog(logIds.requestId, logIds.start, logCtx, error.status, { closeReason: "non_stream" }); + return chatCompletionsErrorResponse(error.status, error.message, error.type, error.code); + } throw error; } } @@ -277,7 +286,7 @@ async function handleChatCompletionsWithBudget( }); let nativeLogged = false; - const finalizeNativeLog = (status: number, meta: { terminalStatus?: RequestLogEntry["terminalStatus"]; closeReason: "terminal" | "client_cancel" }) => { + const finalizeNativeLog = (status: number, meta: { terminalStatus?: RequestLogEntry["terminalStatus"]; closeReason: "terminal" | "client_cancel" | "non_stream" }) => { if (!logIds || nativeLogged) return; nativeLogged = true; addFinalRequestLog(logIds.requestId, logIds.start, logCtx, status, meta); @@ -378,11 +387,14 @@ async function handleChatCompletionsWithBudget( : rewritten; } - const response = logIds + const contentType = upstream.headers.get("content-type") ?? ""; + // JSON is not complete for the client until its Chat projection succeeds. + // Logging the upstream JSON body here would persist 200 before a later + // conversion/serialization error, double-counting both the request and usage. + const response = logIds && contentType.includes("text/event-stream") ? responseWithDeferredRequestLog(upstream, logIds.requestId, logIds.start, logCtx) : upstream; - const contentType = response.headers.get("content-type") ?? ""; if (contentType.includes("text/event-stream") && response.body) { const chatSse = responsesSseToChatCompletionsSse(response.body, requestedModel, { translatorBudget }); if (stream) { @@ -416,11 +428,15 @@ async function handleChatCompletionsWithBudget( } // Defensive: JSON despite stream:true. + const finishJson = (result: Response): Response => { + finalizeNativeLog(result.status, { closeReason: "non_stream" }); + return result; + }; let json: unknown; try { json = await response.json(); } catch { - return chatCompletionsErrorResponse(502, "internal replay returned a non-JSON response", "server_error"); + return finishJson(chatCompletionsErrorResponse(502, "internal replay returned a non-JSON response", "server_error")); } const status = (json as Rec)?.status; if (status === "failed") { @@ -438,44 +454,24 @@ async function handleChatCompletionsWithBudget( classified.code = "model_not_found"; classified.type = "invalid_request_error"; } - return chatCompletionsErrorResponse( + return finishJson(chatCompletionsErrorResponse( classified.code === "translation_buffer_limit" ? 502 : isCyberPolicyCode(classified.code) ? 400 : 502, message, classified.type, classified.code, - ); + )); } - const completion = responsesJsonToChatCompletion(json, requestedModel); - if (!stream) { - return new Response(JSON.stringify(completion), { - status: 200, - headers: { "Content-Type": "application/json" }, - }); - } - - // Streaming client + JSON upstream: synthesize a minimal Chat Completions stream. - const encoder = new TextEncoder(); - const id = typeof completion.id === "string" ? completion.id : `chatcmpl-${Date.now()}`; - const created = typeof completion.created === "number" ? completion.created : Math.floor(Date.now() / 1000); - const message = isRec((completion.choices as Rec[] | undefined)?.[0]) - ? ((completion.choices as Rec[])[0] as Rec).message as Rec | undefined - : undefined; - const content = message && typeof message.content === "string" ? message.content : ""; - const frames = [ - `data: ${JSON.stringify({ id, object: "chat.completion.chunk", created, model: requestedModel, choices: [{ index: 0, delta: { role: "assistant", content: "" }, finish_reason: null }] })}\n\n`, - ...(content - ? [`data: ${JSON.stringify({ id, object: "chat.completion.chunk", created, model: requestedModel, choices: [{ index: 0, delta: { content }, finish_reason: null }] })}\n\n`] - : []), - `data: ${JSON.stringify({ id, object: "chat.completion.chunk", created, model: requestedModel, choices: [{ index: 0, delta: {}, finish_reason: "stop" }], usage: completion.usage })}\n\n`, - "data: [DONE]\n\n", - ]; - return new Response(encoder.encode(frames.join("")), { + const completion = responsesJsonToChatCompletion(json, requestedModel, translatorBudget); + const body = stream + ? jsonCompletionSse(completion, requestedModel, translatorBudget) + : JSON.stringify(completion); + if (!stream) translatorBudget.chargeRetained(Buffer.byteLength(body) * 2, { kind: "live_transient" }); + return finishJson(new Response(body, { status: 200, - headers: { - "Content-Type": "text/event-stream; charset=utf-8", - "Cache-Control": "no-cache", - }, - }); + headers: stream + ? { "Content-Type": "text/event-stream; charset=utf-8", "Cache-Control": "no-cache", Connection: "keep-alive" } + : { "Content-Type": "application/json" }, + })); } diff --git a/src/server/chat-native-sse.ts b/src/server/chat-native-sse.ts index fa9255369c..6570e62a4b 100644 --- a/src/server/chat-native-sse.ts +++ b/src/server/chat-native-sse.ts @@ -61,7 +61,7 @@ function normalizedChunk(value: Rec, requestedModel: string): Rec { }; } -export function jsonCompletionSse(value: Rec, requestedModel: string): string { +export function jsonCompletionSse(value: Rec, requestedModel: string, budget?: TranslatorBudget): string { const id = typeof value.id === "string" ? value.id : `chatcmpl-${crypto.randomUUID().replace(/-/g, "").slice(0, 24)}`; const created = typeof value.created === "number" ? value.created : Math.floor(Date.now() / 1000); const model = requestedModel; @@ -81,7 +81,7 @@ export function jsonCompletionSse(value: Rec, requestedModel: string): string { delta.reasoning_content = message.reasoning_content; } if (Array.isArray(message.tool_calls) && message.tool_calls.length > 0) { - delta.tool_calls = message.tool_calls.map((tool, index) => isRec(tool) ? { index, ...tool } : tool); + delta.tool_calls = message.tool_calls.filter(isRec).map((tool, index) => ({ ...tool, index })); } if (Object.keys(delta).length > 0) { frames.push({ id, object: "chat.completion.chunk", created, model, choices: [{ index: 0, delta, finish_reason: null }] }); @@ -94,7 +94,26 @@ export function jsonCompletionSse(value: Rec, requestedModel: string): string { choices: [{ index: 0, delta: {}, finish_reason: typeof choice.finish_reason === "string" ? choice.finish_reason : "stop" }], ...(value.usage !== undefined ? { usage: value.usage } : {}), }); - return `${frames.map(frame => `data: ${JSON.stringify(frame)}\n\n`).join("")}data: [DONE]\n\n`; + // Keep the frame strings charged while the joined body is allocated. The final + // string and Response's UTF-8 body coexist until response ownership ends. + const scope = { kind: "live_transient" as const }; + let frameBytes = 0; + const serialized: string[] = []; + try { + for (const frame of frames) { + const text = `data: ${JSON.stringify(frame)}\n\n`; + const bytes = Buffer.byteLength(text); + budget?.chargeRetained(bytes, scope); + frameBytes += bytes; + serialized.push(text); + } + const done = "data: [DONE]\n\n"; + const outputBytes = frameBytes + Buffer.byteLength(done); + budget?.chargeRetained(outputBytes * 2, scope); + return serialized.join("") + done; + } finally { + budget?.releaseRetained(frameBytes, scope); + } } interface NativeChatSseOptions { diff --git a/src/server/chat-native.ts b/src/server/chat-native.ts index 30ff39ff44..49e4beb61a 100644 --- a/src/server/chat-native.ts +++ b/src/server/chat-native.ts @@ -502,15 +502,22 @@ export async function handleNativeChatCompletions(options: HandleNativeChatOptio attempt.usage = usage; } if (logIds) recordFirstOutput(logCtx, logIds.start); - finishLog(200); - if (requestedStream) { - return new Response(jsonCompletionSse(completion, requestedModel), { + try { + const serialized = requestedStream + ? jsonCompletionSse(completion, requestedModel, translatorBudget) + : JSON.stringify(completion); + if (!requestedStream) translatorBudget.chargeRetained(Buffer.byteLength(serialized) * 2, { kind: "live_transient" }); + finishLog(200); + return new Response(serialized, { status: 200, - headers: { "Content-Type": "text/event-stream; charset=utf-8", "Cache-Control": "no-cache" }, + headers: requestedStream + ? { "Content-Type": "text/event-stream; charset=utf-8", "Cache-Control": "no-cache" } + : { "Content-Type": "application/json" }, }); + } catch (error) { + if (isTranslatorBudgetExceededError(error)) { + return fail(502, "upstream translation buffer exceeded the safe limit", "upstream_error", "translation_buffer_limit"); + } + throw error; } - return new Response(JSON.stringify(completion), { - status: 200, - headers: { "Content-Type": "application/json" }, - }); } diff --git a/structure/04_transports-and-sidecars.md b/structure/04_transports-and-sidecars.md index 4716c447bb..0c87f3f50d 100644 --- a/structure/04_transports-and-sidecars.md +++ b/structure/04_transports-and-sidecars.md @@ -1302,6 +1302,27 @@ messages are redacted before either JSON or SSE reaches the client. The native p request-attempt logging, reset retry, same-key 429 replay, key rotation, usage extraction, and request-signal cancellation contracts as routed Responses transport. +## Chat streaming client with a JSON upstream result + +The translated inbound path in `src/server/chat-completions.ts` may receive a complete JSON +Responses result even when the Chat client requested SSE. Its synthetic stream reuses +`responsesJsonToChatCompletion` as the semantic authority: converted text, reasoning, available +refusal content, tool calls, finish reason, and usage must survive this final delivery conversion. +Tool calls gain their array-order stream `index`; the stream retains one assistant-role frame, +one terminal choice, and one `[DONE]`. Both native and translated JSON fallbacks share +`jsonCompletionSse`; its temporary frame strings and final body ownership are charged to the +existing translator budget. Known incomplete limits take precedence over tool finish reasons; +unmapped incomplete boundaries remain errors. The existing response-body lifecycle owns translation-budget +release on consumption or cancellation. Actual upstream SSE and native Chat bypass this fallback. + +[Decision Log] +- 목적과 의도: Keep tool execution and incomplete-response detection working when a streaming client receives a JSON upstream result. +- 기존 구현 및 제약 조건: The existing fallback copied only text and forced `stop`, despite the JSON converter already retaining tool calls, reasoning, and incomplete status. +- 검토한 주요 대안: Duplicate Responses parsing in the emitter; perform another inference request; preserve the already-converted Chat completion. +- 선택한 방식: Copy supported converted message fields into one delta, assign tool-call stream indexes, and retain the converted finish reason. +- 다른 대안 대신 이 방식을 선택한 이유: One conversion authority prevents the streaming fallback from drifting from non-streaming semantics without changing routing or retry behavior. +- 장점, 단점 및 영향: No additional upstream request or dependency; this remains buffered delivery, not token-by-token upstream streaming. Handler regressions cover tools, reasoning, length, ordinary and empty completions, and budget release. + ## Parallel tool calls (default-on for chat providers) The openai-chat adapter buffers ALL streamed `tool_calls` deltas (keyed by `index`, falling back to diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index db2583b00b..a1f0198e44 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -105,6 +105,7 @@ "catalog-verbosity-default.test.ts": "codex-integration", "catalog-vision-sidecar-modalities.test.ts": "codex-integration", "chat-completions-endpoint.test.ts": "responses", + "chat-json-sse-fallback.test.ts": "responses", "chatgpt-device-auth.test.ts": "oauth", "chatgpt-oauth.test.ts": "oauth", "chatgpt-token-expiry.test.ts": "oauth", diff --git a/tests/responses/chat-json-sse-fallback.test.ts b/tests/responses/chat-json-sse-fallback.test.ts new file mode 100644 index 0000000000..684917a05b --- /dev/null +++ b/tests/responses/chat-json-sse-fallback.test.ts @@ -0,0 +1,254 @@ +import { afterEach, expect, test } from "bun:test"; +import { handleChatCompletions } from "../../src/server/chat-completions"; +import { createTranslatorBudget, isTranslatorBudgetExceededError, translatorObservedBufferSnapshot } from "../../src/lib/translator-budget"; +import type { OcxConfig } from "../../src/types"; +import { responsesJsonToChatCompletion, isChatCompletionsStreamError } from "../../src/chat/outbound"; +import { jsonCompletionSse } from "../../src/server/chat-native-sse"; +import { getRequestLogEntries } from "../../src/server/request-log"; +import { readUsageEntries } from "../../src/usage/log"; + +let upstream: ReturnType | undefined; +afterEach(async () => { await upstream?.stop(true); upstream = undefined; }); + +interface Chunk { + choices: Array<{ index: number; delta: { + role?: string; content?: string; reasoning_content?: string; + tool_calls?: Array<{ index: number; id: string; type: string; function: { name: string; arguments: string } }>; + }; finish_reason: string | null }>; + usage?: { prompt_tokens: number; completion_tokens: number }; +} + +async function streamFixture(output: unknown[], status = "completed", cancel = false, reason = "max_output_tokens", delivery: { jsonFinish?: string; error?: boolean; errorCode?: string } = {}): Promise { + const budgetBefore = translatorObservedBufferSnapshot().currentBytes; + const requestId = `chat-json-fixture-${crypto.randomUUID()}`; + let requests = 0; + upstream = Bun.serve({ hostname: "127.0.0.1", port: 0, async fetch(req) { + expect(new URL(req.url).pathname).toBe("/v1/responses"); + expect((await req.json() as { stream: boolean }).stream).toBe(true); + requests++; + return Response.json({ id: "resp_fixture", status, output, + ...(status === "incomplete" ? { incomplete_details: { reason } } : {}), + usage: { input_tokens: 11, output_tokens: 7 } }); + } }); + const config: OcxConfig = { port: 0, defaultProvider: "fixture", providers: { fixture: { + adapter: "openai-responses", baseUrl: `http://127.0.0.1:${upstream.port}/v1`, + authMode: "key", apiKey: "fixture-key", allowPrivateNetwork: true, models: ["model"], + } } }; + const response = await handleChatCompletions(new Request("http://localhost/v1/chat/completions", { + method: "POST", headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "fixture/model", stream: !delivery.jsonFinish, messages: [{ role: "user", content: "fixture" }], + tools: [{ type: "function", function: { name: "lookup", parameters: { type: "object" } } }] }), + }), config, { model: "", provider: "" }, { requestId, start: Date.now() }); + const assertSingleFinal = () => { + const rows = getRequestLogEntries().filter(entry => entry.requestId === requestId); + expect(rows).toHaveLength(1); + expect(rows[0]?.status).toBe(delivery.error ? 502 : 200); + const persisted = readUsageEntries().filter(entry => entry.requestId === requestId); + expect(persisted).toHaveLength(1); + expect(persisted[0]?.status).toBe(delivery.error ? 502 : 200); + }; + if (delivery.error) { + expect(response.status).toBe(502); + expect(await response.json()).toMatchObject({ error: { type: "upstream_error", code: delivery.errorCode ?? "upstream_incomplete" } }); + assertSingleFinal(); + expect(requests).toBe(1); + expect(translatorObservedBufferSnapshot().currentBytes).toBe(budgetBefore); + return []; + } + expect(response.status).toBe(200); + if (delivery.jsonFinish) { + expect(await response.json()).toMatchObject({ choices: [{ finish_reason: delivery.jsonFinish }] }); + assertSingleFinal(); + expect(requests).toBe(1); + expect(translatorObservedBufferSnapshot().currentBytes).toBe(budgetBefore); + return []; + } + expect(response.headers.get("content-type")).toContain("text/event-stream"); + if (cancel) { + await response.body!.cancel("fixture cancellation"); + assertSingleFinal(); + expect(requests).toBe(1); + expect(translatorObservedBufferSnapshot().currentBytes).toBe(budgetBefore); + return []; + } + const text = await response.text(); + assertSingleFinal(); + expect(translatorObservedBufferSnapshot().currentBytes).toBe(budgetBefore); + expect(requests).toBe(1); + const payloads = text.split(/\r?\n/).filter(line => line.startsWith("data: ")).map(line => line.slice(6)); + expect(payloads.filter(value => value === "[DONE]")).toHaveLength(1); + expect(payloads.at(-1)).toBe("[DONE]"); + const chunks = payloads.filter(value => value !== "[DONE]").map(value => JSON.parse(value) as Chunk); + expect(chunks.flatMap(chunk => chunk.choices).filter(choice => choice.finish_reason !== null)).toHaveLength(1); + expect(chunks.at(-1)?.usage).toMatchObject({ prompt_tokens: 11, completion_tokens: 7 }); + return chunks; +} + +test.each([1, 2])("JSON-to-SSE keeps %s indexed tool calls and tool_calls finish", async count => { + const calls = Array.from({ length: count }, (_, index) => ({ type: "function_call", + call_id: `call_fixture_${index}`, name: "lookup", arguments: JSON.stringify({ index }) })); + const chunks = await streamFixture(calls); + expect(chunks.flatMap(chunk => chunk.choices.flatMap(choice => choice.delta.tool_calls ?? []))) + .toEqual(calls.map((call, index) => ({ index, id: call.call_id, type: "function", + function: { name: call.name, arguments: call.arguments } }))); + expect(chunks.at(-1)?.choices[0]?.finish_reason).toBe("tool_calls"); +}); + +test("JSON-to-SSE keeps reasoning alongside answer text", async () => { + const chunks = await streamFixture([ + { type: "reasoning", summary: [{ type: "summary_text", text: "Fixture reasoning." }] }, + { type: "message", role: "assistant", content: [{ type: "output_text", text: "Answer." }] }, + ]); + expect(chunks.flatMap(chunk => chunk.choices).map(choice => choice.delta.reasoning_content ?? "").join("")) + .toBe("Fixture reasoning."); + expect(chunks.flatMap(chunk => chunk.choices).map(choice => choice.delta.content ?? "").join("")) + .toBe("Answer."); + expect(chunks.at(-1)?.choices[0]?.finish_reason).toBe("stop"); +}); + +test("JSON-to-SSE preserves length instead of claiming a normal stop", async () => { + const chunks = await streamFixture([ + { type: "message", role: "assistant", content: [{ type: "output_text", text: "Partial answer." }] }, + ], "incomplete"); + expect(chunks.at(-1)?.choices[0]?.finish_reason).toBe("length"); +}); + +test("JSON-to-SSE preserves ordinary text and a single empty completion terminal", async () => { + const chunks = await streamFixture([ + { type: "message", role: "assistant", content: [{ type: "output_text", text: "Ordinary text." }] }, + ]); + expect(chunks.flatMap(chunk => chunk.choices).map(choice => choice.delta.content ?? "").join("")) + .toBe("Ordinary text."); + expect(chunks.at(-1)?.choices[0]?.finish_reason).toBe("stop"); +}); + +test("JSON-to-SSE empty completion still terminates once", async () => { + const chunks = await streamFixture([]); + expect(chunks).toHaveLength(2); + expect(chunks.at(-1)?.choices[0]?.finish_reason).toBe("stop"); +}); + +test("JSON-to-SSE cancellation releases the existing translation budget", async () => { + await streamFixture([{ type: "function_call", call_id: "call_cancel", name: "lookup", arguments: "{}" }], "completed", true); +}); + +// Expected finish values come from the official Chat contract, not the converter. +test.each([ + ["max_output_tokens", "length"], + ["content_filter", "content_filter"], +])("JSON-to-SSE incomplete %s takes precedence over a partial tool call", async (reason, finish) => { + const chunks = await streamFixture([ + { type: "function_call", call_id: "call_partial", name: "lookup", arguments: '{"unfinished":' }, + ], "incomplete", false, reason); + expect(chunks.at(-1)?.choices[0]?.finish_reason).toBe(finish); +}); + +test.each(["max_output_tokens", "content_filter"])("JSON projection preserves incomplete %s with tools", reason => { + const completion = responsesJsonToChatCompletion({ status: "incomplete", incomplete_details: { reason }, + output: [{ type: "function_call", call_id: "call_partial", name: "lookup", arguments: "{}" }], + }, "fixture/model"); + expect(completion.choices).toMatchObject([{ finish_reason: reason === "max_output_tokens" ? "length" : "content_filter" }]); +}); + +test.each([undefined, "max_messages", "steered", "adapter_eof"])("JSON projection does not invent length for %s", reason => { + try { + responsesJsonToChatCompletion({ status: "incomplete", incomplete_details: { reason }, output: [] }, "fixture/model"); + throw new Error("expected typed truncation"); + } catch (error) { + expect(isChatCompletionsStreamError(error)).toBe(true); + expect(error).toMatchObject({ status: 502, type: "upstream_error", code: "upstream_incomplete" }); + } +}); + +test("shared JSON-to-SSE serializer assigns tool indices and charges positive retained output", () => { + const budget = createTranslatorBudget({ maxTurnBytes: 8192 }); + try { + const converted = responsesJsonToChatCompletion({ status: "completed", output: [ + { type: "message", content: [{ type: "output_text", text: "Fixture answer" }] }, + { type: "function_call", call_id: "call_one", name: "lookup", arguments: "{}" }, + ] }, "model", budget); + expect(budget.snapshot().currentBytes).toBeGreaterThan(0); + const text = jsonCompletionSse(converted, "model", budget); + const chunks = text.split("\n").filter(x => x.startsWith("data: {")).map(x => JSON.parse(x.slice(6)) as Chunk); + const calls = chunks.flatMap(c => c.choices.flatMap(x => x.delta.tool_calls ?? [])); + expect(calls[0]?.index).toBe(0); + expect(budget.snapshot().currentBytes).toBeGreaterThanOrEqual(Buffer.byteLength(text) * 2); + expect(budget.snapshot().highWaterBytes).toBeLessThanOrEqual(8192); + } finally { budget.dispose(); } + expect(budget.snapshot().currentBytes).toBe(0); +}); + +test("shared JSON-to-SSE serializer rejects an oversized terminal batch before returning success", () => { + const budget = createTranslatorBudget({ maxTurnBytes: 128 }); + try { + expect(() => jsonCompletionSse({ choices: [{ message: { content: "fixture" }, finish_reason: "stop" }] }, "model", budget)) + .toThrow(); + expect(budget.snapshot().overflows).toBe(1); + expect(budget.snapshot().currentBytes).toBe(0); + } finally { budget.dispose(); } +}); + +test("JSON projection rejects retained output overflow with the existing typed budget error", () => { + const budget = createTranslatorBudget({ maxTurnBytes: 16 }); + try { + let failure: unknown; + try { responsesJsonToChatCompletion({ output: [{ type: "message", content: [{ type: "output_text", text: "x".repeat(32) }] }] }, "model", budget); } + catch (error) { failure = error; } + expect(isTranslatorBudgetExceededError(failure)).toBe(true); + expect(budget.snapshot().currentBytes).toBe(0); + } finally { budget.dispose(); } +}); + + +test.each(["max_messages", "steered", "adapter_eof"])("handler reports unsupported incomplete %s as a typed error", async reason => { + await streamFixture([], "incomplete", false, reason, { error: true }); +}); + +test.each([["max_output_tokens", "length"], ["content_filter", "content_filter"]])( + "JSON client receives %s boundary rather than tool_calls", async (reason, finish) => { + await streamFixture([{ type: "function_call", call_id: "call_partial", name: "lookup", arguments: "{}" }], + "incomplete", false, reason, { jsonFinish: finish }); + }, +); + + +test("JSON projection accounts split Unicode and ignores empty fragments", () => { + const budget = createTranslatorBudget({ maxTurnBytes: 8192 }); + try { + const completion = responsesJsonToChatCompletion({ output: [ + { type: "message", content: [ + { type: "output_text", text: "\ud83d" }, + ...Array.from({ length: 100 }, () => ({ type: "output_text", text: "" })), + { type: "output_text", text: "\ude00" }, + ] }, + { type: "reasoning", summary: [{ type: "summary_text", text: "\ud83d" }, { type: "summary_text", text: "\ude00" }] }, + ] }, "model", budget); + expect(completion.choices).toMatchObject([{ message: { content: "😀", reasoning_content: "😀" } }]); + expect(budget.snapshot().currentBytes).toBe(8); + } finally { budget.dispose(); } +}); + + +test("buffered calls enforce their per-call cap, including an empty upstream ID", () => { + for (const call_id of ["fixture-call", ""]) { + const budget = createTranslatorBudget({ maxCallArgumentBytes: 4, maxTurnBytes: 8192 }); + try { + let failure: unknown; + try { + responsesJsonToChatCompletion({ output: [{ type: "function_call", call_id, name: "lookup", arguments: "12345" }] }, "model", budget); + } catch (error) { failure = error; } + expect(isTranslatorBudgetExceededError(failure)).toBe(true); + expect(failure).toMatchObject({ code: "translation_buffer_limit", kind: "tool_args", limitBytes: 4 }); + expect(budget.snapshot().currentBytes).toBe(0); + expect(budget.snapshot().activeCalls).toBe(0); + const result = responsesJsonToChatCompletion({ output: [{ type: "function_call", call_id, name: "lookup", arguments: "1234" }] }, "model", budget); + expect(result.choices).toMatchObject([{ message: { tool_calls: [{ function: { arguments: "1234" } }] } }]); + expect(budget.snapshot().activeCalls).toBe(0); + } finally { budget.dispose(); } + } +}); + +test("JSON-to-SSE rejects a call above 2 MiB without success output or duplicate usage", async () => { + await streamFixture([{ type: "function_call", call_id: "large-call", name: "lookup", arguments: JSON.stringify({ text: "x".repeat(2 * 1024 * 1024) }) }], + "completed", false, "max_output_tokens", { error: true, errorCode: "translation_buffer_limit" }); +});