diff --git a/docs-site/src/content/docs/fr/guides/combos.md b/docs-site/src/content/docs/fr/guides/combos.md index dc18f68971..3e2ab4a6b6 100644 --- a/docs-site/src/content/docs/fr/guides/combos.md +++ b/docs-site/src/content/docs/fr/guides/combos.md @@ -365,3 +365,9 @@ message de validation. L’erreur était terminale plutôt que spécifique à la cible. Corriger une entrée invalide, réduire un contexte surdimensionné, gérer un refus de politique ou corriger l’origine de la demande rejetée. Les combos ne sautent pas dans ces cas-là. + +## Compatibilité des paramètres facultatifs + +Exception aux erreurs 400 terminales : un rejet structuré de `user`, une valeur non prise en charge pour `reasoning.effort`/`reasoning_effort`, ou un rejet d’entrée d’image propre à un modèle (`param: input`) peut faire passer le combo à la cible admissible suivante, sans délai de refroidissement. Le refus de sécurité, l’annulation et une sortie déjà commencée restent non rejouables. + +[Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/guides/combos.md b/docs-site/src/content/docs/guides/combos.md index 979a84ffcd..5ec04c842b 100644 --- a/docs-site/src/content/docs/guides/combos.md +++ b/docs-site/src/content/docs/guides/combos.md @@ -213,7 +213,8 @@ Combo failures are divided into **hop** failures and **terminal** failures. | HTTP 401, 403, 404, 408, 429, or any 5xx | Cool the target and hop to the next eligible target. | | HTTP 410 with an explicit model end-of-life, retired, deprecated, sunset, decommissioned, or no-longer-available signal | Cool that target and hop. Unrelated 410 responses remain terminal. | | Classified authentication, subscription, quota, rate-limit, overload, or upstream-server error | Cool the target and hop, even when the status alone is not sufficient. | -| Client cancellation (499), `origin_rejected`, cyber-policy refusal, context overflow, or invalid request | Stop and return the error; another target would not make the request valid. | +| Client cancellation (499), `origin_rejected`, cyber-policy refusal, context overflow, or other invalid request | Stop and return the error; another target would not make the request valid. | +| Structured HTTP 400 rejecting optional `user`, an unsupported reasoning effort, or model-scoped image input | Hop before output commitment without cooling; see optional-control compatibility below. | | Any other unclassified error | Stop and return the error. | When `cooldownMs` is unset, a hopped target uses an upstream fallback: 5 seconds for request-rate @@ -227,8 +228,8 @@ fallback for upstream rate-limit codes `1302`/`1305` → the 60-second default. `Retry-After: 0` remains an immediate upstream directive rather than being replaced by a configured cooldown. -The current request never retries the same attempted target. Later requests skip it until its -cooldown expires. A `Retry-After` HTTP-date that is already in the past is also preserved as an +The current request never retries the same attempted target. Later requests skip a cooled target until its +cooldown expires; request-local compatibility rejections do not cool the target. A `Retry-After` HTTP-date that is already in the past is also preserved as an immediate upstream directive, just like `Retry-After: 0`. Set `waitForCooldownMs` to allow a later request to wait for the earliest eligible target cooldown, up to that cap on each selection attempt, and then make one fresh selection. A request may therefore wait up to `hops × waitForCooldownMs` @@ -254,6 +255,12 @@ another provider, which prevents duplicate text and tool execution. If the pre-o its safety cap without a terminal or output boundary, OpenCodex also commits the current target instead of growing memory without a bound. +## Request-local target compatibility + +When routing Claude Code to the canonical ChatGPT Codex backend, OpenCodex removes the unsupported top-level `user` metadata field without changing the session/cache key, input messages, tool schemas, or safety identifiers. Public Responses API and noncanonical forward gateways keep that field. + +A combo can also advance after an intact HTTP 400 `invalid_request_error` that specifically rejects `user`, reports `unsupported_value` for `reasoning.effort`/`reasoning_effort`, or reports `param: input` with an exact model-scoped `does not support image inputs` rejection. This is a mismatch for that request, not evidence that the target is unhealthy, so it records no cooldown. This compatibility recovery does not silently change `none` into a different effort or broaden this exception to arbitrary invalid requests. Policy refusals, cancellation and already-committed output remain non-replayable. A single-target request still returns an unresolved upstream rejection. + ## Default reasoning effort `defaultEffort` fills an absent `reasoning.effort` when the combo has a non-null default and the selected target has a known, nonempty supported ladder. If the target supports the configured value, it is retained; otherwise the highest supported rung at or below it is used, or the lowest supported rung when none is lower. Unknown or empty ladders omit the default. diff --git a/docs-site/src/content/docs/guides/sidecars.md b/docs-site/src/content/docs/guides/sidecars.md index d87c5b9513..a99042b94f 100644 --- a/docs-site/src/content/docs/guides/sidecars.md +++ b/docs-site/src/content/docs/guides/sidecars.md @@ -123,11 +123,9 @@ failures after response headers have started are delivered as `response.failed` ## Vision sidecar -When the routed model is listed in its provider's `noVisionModels` — or declared text-only for -that model via `modelInputModalities` — and a request carries an image, opencodex describes each -image **before** the main call and replaces it with text, provided a vision sidecar plan is -available. Without an available plan the raw image is stripped rather than forwarded to a -text-only backend. The model catalog advertises image input for every sidecar-covered model. +Image routing is capability-aware. Before an image-bearing upstream send, opencodex resolves the selected model's effective input modalities from runtime provider evidence, explicit operator declarations, backend/registry metadata, and generated vendor metadata. A target positively known to be text-only goes through the Vision Sidecar first; the image is described **before** the main call and replaced inline with text. A target positively known to support images receives the image directly. Unknown custom models keep the existing compatibility behavior rather than being guessed text-only. + +For the canonical ChatGPT Codex route, opencodex uses the `openai-codex` metadata bundle rather than public OpenAI API metadata, so backend-specific modality differences are respected. The native Chat fast path uses the same gate and cannot bypass a known text-only verdict. Without an available sidecar plan, raw images are stripped before a proven text-only backend. Combos advertise image input only when every member accepts images, either natively or through a sidecar, and the combo's `imageInput` setting is not disabled, so clients such as the Codex app allow attachments instead of blocking them before the sidecar runs. When @@ -164,10 +162,12 @@ sidecar-backed by default; Zen routes are unchanged and were not probed in this keys (Anthropic keys omit it, since that field is ignored there); mutable `https:` images are not cached. -The management API and Dashboard picker now list models that can actually accept image input. -When the matching backend is available, `gpt-5.6-luna` (OpenAI) and `claude-haiku-4-5` (Anthropic) -are always offered as baseline options. `PUT /api/sidecar-settings` rejects a model known to be -text-only, but still accepts an unknown id so custom or ahead-of-catalog names keep working. +The management API and Dashboard picker list models that can accept image input. When the matching +backend is available, `gpt-5.6-luna` (OpenAI) and `claude-haiku-4-5` (Anthropic) are always offered +as baseline options. `PUT /api/sidecar-settings` may retain an unknown custom/ahead-of-catalog id. +An explicitly configured routed Vision Sidecar is therefore usable unless capability evidence proves +that model cannot accept images; this preserves operator-selected custom sidecars without allowing a +known text-only sidecar to receive image bytes. ```json { diff --git a/docs-site/src/content/docs/ja/guides/combos.md b/docs-site/src/content/docs/ja/guides/combos.md index 7b2c62a9ec..4682d4c6d2 100644 --- a/docs-site/src/content/docs/ja/guides/combos.md +++ b/docs-site/src/content/docs/ja/guides/combos.md @@ -260,3 +260,9 @@ ocx combo remove --yes ### 最初のエラーの後にフェイルオーバーが停止したのはなぜですか? このエラーはターゲット固有のものではなく、最終的なものでした。無効な入力を修正し、大きすぎるコンテキストを削減し、ポリシーの拒否を処理し、拒否されたリクエストの送信元を修正します。コンボはそのような場合には機能しません。 + +## 任意パラメーターの互換性 + +通常の 400 エラーは終了扱いですが、`user` の明示的な拒否、`reasoning.effort`/`reasoning_effort` の非対応値、またはモデル固有の画像入力拒否（`param: input`）を示す構造化エラーでは、出力開始前に次の適格なターゲットへ進めます。この不一致ではクールダウンを記録しません。安全ポリシーによる拒否、キャンセル、出力開始後の再実行は禁止のままです。 + +[Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/ko/guides/combos.md b/docs-site/src/content/docs/ko/guides/combos.md index e507889c6f..1b99029570 100644 --- a/docs-site/src/content/docs/ko/guides/combos.md +++ b/docs-site/src/content/docs/ko/guides/combos.md @@ -272,3 +272,9 @@ opencodex 인스턴스에 기록했는지 확인하세요. 대상별 오류가 아니라 종결 오류였기 때문입니다. 잘못된 입력을 수정하고, 너무 큰 context를 줄이고, 정책 거부를 처리하거나, 거부된 요청 origin을 바로잡으세요. combo는 이런 경우 다음 대상으로 넘어가지 않습니다. + +## 선택적 매개변수 호환성 + +일반적인 400 오류는 종료되지만, `user`를 명시적으로 거부하거나 `reasoning.effort`/`reasoning_effort`의 지원되지 않는 값 또는 모델별 이미지 입력 거부(`param: input`)를 나타내는 구조화된 오류는 출력 시작 전에 다음 적격 대상으로 넘어갈 수 있습니다. 이 경우 쿨다운을 기록하지 않습니다. 보안 정책 거부, 취소 및 이미 시작된 출력은 재실행하지 않습니다. + +[Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/ru/guides/combos.md b/docs-site/src/content/docs/ru/guides/combos.md index ddd16f6175..bc11183712 100644 --- a/docs-site/src/content/docs/ru/guides/combos.md +++ b/docs-site/src/content/docs/ru/guides/combos.md @@ -319,3 +319,9 @@ CLI и дашборд показывают точное сообщение ва Ошибка была terminal, а не специфичной для конкретной цели. Исправьте некорректный ввод, сократите слишком большой контекст, обработайте отказ политики или исправьте отклонённое происхождение запроса. В таких случаях combo не выполняют hop. + +## Совместимость необязательных параметров + +Исключение для завершающих ошибок 400: структурированный отказ от `user`, неподдерживаемое значение `reasoning.effort`/`reasoning_effort` или специфичный для модели отказ входного изображения (`param: input`) позволяет до начала вывода перейти к следующей допустимой цели без периода охлаждения. Отказ политики безопасности, отмена и уже начавшийся вывод по-прежнему запрещают повторное выполнение. + +[Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/tr/guides/combos.md b/docs-site/src/content/docs/tr/guides/combos.md index 4ac9e0febf..8931c96915 100644 --- a/docs-site/src/content/docs/tr/guides/combos.md +++ b/docs-site/src/content/docs/tr/guides/combos.md @@ -404,3 +404,9 @@ Hata hedefe özgü olmaktan ziyade uç (terminal) bir hataydı. Geçersiz girdiy düzeltin, aşırı büyük bir bağlamı azaltın, bir politika reddini işleyin veya reddedilen istek kaynağını düzeltin. Kombolar bu durumlar için atlama yapmaz. + +## İsteğe bağlı parametre uyumluluğu + +Sonlandırıcı 400 hatalarının dar bir istisnası vardır: `user` alanını açıkça reddeden, `reasoning.effort`/`reasoning_effort` için desteklenmeyen değer bildiren veya modele özgü görüntü girdisini reddeden (`param: input`) yapılandırılmış hata, çıktı başlamadan sonraki uygun hedefe geçebilir. Bu uyumsuzluk için bekleme süresi kaydedilmez. Güvenlik politikası reddi, iptal ve başlamış çıktı yeniden yürütülmez. + +[Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/zh-cn/guides/combos.md b/docs-site/src/content/docs/zh-cn/guides/combos.md index 7c8c8efd63..951b752d90 100644 --- a/docs-site/src/content/docs/zh-cn/guides/combos.md +++ b/docs-site/src/content/docs/zh-cn/guides/combos.md @@ -290,3 +290,9 @@ combo id 不存在。响应是 HTTP 404，类型为 `invalid_request_error`。 ### 为什么故障切换在第一次错误后就停止了？ 该错误是终止性的，而不是针对目标的。修复无效输入、缩小过大的上下文、处理策略拒绝，或者纠正被拒绝的请求来源。对于这些情况，combo 不会继续跳转。 + +## 可选参数兼容性 + +一般 400 错误仍会终止请求，但明确拒绝 `user`、对 `reasoning.effort`/`reasoning_effort` 返回不支持值，或返回模型特定图像输入拒绝（`param: input`）的结构化错误，可让 combo 在输出开始前尝试下一个符合条件的目标，而不记录冷却时间。安全策略拒绝、取消以及已经开始的输出仍不可重放。 + +[Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/zh-tw/guides/combos.md b/docs-site/src/content/docs/zh-tw/guides/combos.md index be566c77cd..7ebd4a6512 100644 --- a/docs-site/src/content/docs/zh-tw/guides/combos.md +++ b/docs-site/src/content/docs/zh-tw/guides/combos.md @@ -290,3 +290,9 @@ Combo id 未知。回應為 HTTP 404 並帶 type `invalid_request_error`。執 ### 為什麼 failover 在第一個錯誤後就停止了？ 該錯誤是終端的而非目標特定的。修正無效輸入、縮減過大的上下文、處理策略拒絕，或更正被拒的請求來源。Combo 對那些情況不會跳轉。 + +## 選用參數相容性 + +一般 400 錯誤仍會終止請求，但明確拒絕 `user`、對 `reasoning.effort`/`reasoning_effort` 回傳不支援值，或回傳模型特定影像輸入拒絕（`param: input`）的結構化錯誤，可讓 combo 在輸出開始前嘗試下一個符合條件的目標，而不記錄冷卻時間。安全政策拒絕、取消及已開始的輸出仍不可重播。 + +[Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/scripts/generate-model-metadata.ts b/scripts/generate-model-metadata.ts index 2441e32bc9..fe3b72b960 100644 --- a/scripts/generate-model-metadata.ts +++ b/scripts/generate-model-metadata.ts @@ -44,8 +44,13 @@ const COST_VENDOR_BUNDLES = [ "zai", "mistral", "cerebras", "azure-openai", "amazon-bedrock", "xiaomi", ]; +// Capability-only bundles may describe a transport/backend that is not a public provider preset. +// The canonical ChatGPT Codex forward route has modality differences from the public OpenAI API +// (for example retired Spark is text-only there), so keep that evidence distinct. +const CAPABILITY_ONLY_BUNDLES = ["openai-codex"]; + const allowedProviders = Array.from( - new Set([...Object.values(PROVIDER_ALIASES), ...COST_VENDOR_BUNDLES]), + new Set([...Object.values(PROVIDER_ALIASES), ...COST_VENDOR_BUNDLES, ...CAPABILITY_ONLY_BUNDLES]), ).sort(); // Models the upstream catalogue lists but the provider rejects at request time. Issue #82: diff --git a/src/adapters/openai-responses.ts b/src/adapters/openai-responses.ts index e2c7cf2a14..bc5bf51209 100644 --- a/src/adapters/openai-responses.ts +++ b/src/adapters/openai-responses.ts @@ -1285,6 +1285,15 @@ function stripUnsupportedForwardParams(body: unknown): unknown { return rest; } +/** Drop client identity metadata only at the canonical Codex destination, not public gateways. + * Claude translation still owns local session/cache identity; never mutate its replay body. + */ +function stripCanonicalForwardUser(body: unknown): unknown { + if (!isPlainObject(body) || !Object.hasOwn(body, "user")) return body; + const { user: _user, ...rest } = body; + return rest; +} + /** Return the lossless text represented by one system message, or null when it is multimodal. */ function canonicalForwardSystemText(item: Record): string | null { const content = item.content; @@ -2256,6 +2265,7 @@ export function createResponsesPassthroughAdapter(provider: OcxProviderConfig): if (isCanonicalOpenAiForwardProvider(provider)) { outBody = stripDeprecatedPromptCacheRetention(outBody, parsed.modelId); outBody = stripCanonicalForwardPromptCacheOptions(outBody); + outBody = stripCanonicalForwardUser(outBody); outBody = normalizeCanonicalForwardPromptEnvelope(outBody); outBody = normalizeCanonicalForwardContinuationEnvelope(outBody); } diff --git a/src/combos/failover.ts b/src/combos/failover.ts index e592298d88..e60fe83dee 100644 --- a/src/combos/failover.ts +++ b/src/combos/failover.ts @@ -351,6 +351,49 @@ const PROVIDER_SCOPED_FAILURE_CODES = new Set([ "insufficient_balance", ]); +/** + * Precise target-local request incompatibilities are request-local, not terminal for a combo. + * Require a bounded, intact provider envelope; never infer compatibility from echoed prompt text. + * Only OpenCodex's exact error wrapper may be unwrapped, with a fixed depth budget. Unknown or + * conflicting codes fail closed. No fields are removed here and no same-target replay is added. + * Image rejection requires `param: input` and an exact model-scoped prefix. + */ +function isRequestLocalTargetIncompatibility(status: number, message: string, code?: string | null): boolean { + if (status !== 400 || message.length > 16_384) return false; + const genericCodes = new Set(["", "invalid_request_error", "unsupported_parameter", "unsupported_value"]); + if (!genericCodes.has(normalizedFailureCode(code))) return false; + let text = message.trim(); + for (let depth = 0; depth < 3; depth += 1) { + if (text.startsWith("Provider error 400: ")) text = text.slice("Provider error 400: ".length); + let payload: unknown; + try { payload = JSON.parse(text); } catch { return false; } + if (!payload || typeof payload !== "object" || Array.isArray(payload)) return false; + const error = (payload as Record).error; + if (!error || typeof error !== "object" || Array.isArray(error)) return false; + const e = error as Record; + if (e.code !== undefined && e.code !== null && typeof e.code !== "string") return false; + const errorCode = normalizedFailureCode(typeof e.code === "string" ? e.code : undefined); + if (!genericCodes.has(errorCode) || typeof e.message !== "string") return false; + if (e.type !== "invalid_request_error" && e.type !== "upstream_error") return false; + if (e.message.startsWith("Provider error 400: ") && e.param === undefined + && (errorCode === "" || errorCode === "invalid_request_error")) { + text = e.message; + continue; + } + if (e.type !== "invalid_request_error") return false; + if (e.message === "Unsupported parameter: user") { + return (e.param === undefined || e.param === "user") && errorCode !== "unsupported_value"; + } + if (errorCode === "unsupported_value" + && (e.param === "reasoning.effort" || e.param === "reasoning_effort") + && e.message.startsWith("Unsupported value:") && e.message.includes("not supported")) return true; + return e.param === "input" + && (errorCode === "" || errorCode === "invalid_request_error") + && /^Model '[^']{1,256}' does not support image inputs\./.test(e.message); + } + return false; +} + export function comboFailureCooldownScope( status: number, message: string, @@ -363,6 +406,7 @@ export function comboFailureCooldownScope( || REQUEST_SHAPE_FAILURE_CODES.has(code) || isRequestLocalFreePromptCap(status, message, options?.code) || isProviderTargetContextOverflow(status, message, options?.code) + || isRequestLocalTargetIncompatibility(status, message, options?.code) ) return "none"; if (isProviderScopedQuotaCap(status, message, options?.code)) return "provider"; // A rejected or unpaid credential is provider-wide evidence: every target that routes @@ -465,6 +509,7 @@ export function comboFailureDecision( // `free_rate_limited` no longer routes through `isProviderScopedQuotaCap` (it is a // per-request cap, not provider-wide evidence), so keep its hop verdict explicit here. if (failureCode === "free_rate_limited") return "hop"; + if (isRequestLocalTargetIncompatibility(status, message, options?.code)) return "hop"; if (["origin_rejected", "context_length_exceeded", "invalid_request_error"].includes(error.code ?? "")) { return "stop"; } diff --git a/src/generated/model-metadata.ts b/src/generated/model-metadata.ts index 7220aa7509..42f92b2d59 100644 --- a/src/generated/model-metadata.ts +++ b/src/generated/model-metadata.ts @@ -48,6 +48,7 @@ const DATA: Record = { "mistral": [["codestral-latest",256000,4096,"text",0,null,0.3,0.9,0,0],["devstral-2512",262144,262144,"text",0,null,0.4,2,0,0],["devstral-latest",262144,262144,"text",0,null,0.4,2,0,0],["devstral-medium-2507",128000,128000,"text",0,null,0.4,2,0,0],["devstral-medium-latest",262144,262144,"text",0,null,0.4,2,0,0],["devstral-small-2505",128000,128000,"text",0,null,0.1,0.3,0,0],["devstral-small-2507",128000,128000,"text",0,null,0.1,0.3,0,0],["labs-devstral-small-2512",256000,256000,"text,image",0,null,0,0,0,0],["magistral-medium-latest",128000,16384,"text",1,null,2,5,0,0],["magistral-small",128000,128000,"text",1,null,0.5,1.5,0,0],["ministral-3b-latest",128000,128000,"text",0,null,0.04,0.04,0,0],["ministral-8b-latest",128000,128000,"text",0,null,0.1,0.1,0,0],["mistral-large-2411",131072,16384,"text",0,null,2,6,0,0],["mistral-large-2512",262144,262144,"text,image",0,null,0.5,1.5,0,0],["mistral-large-latest",262144,262144,"text,image",0,null,0.5,1.5,0,0],["mistral-medium-2505",131072,131072,"text,image",0,null,0.4,2,0,0],["mistral-medium-2508",262144,262144,"text,image",0,null,0.4,2,0,0],["mistral-medium-2604",262144,262144,"text,image",1,null,1.5,7.5,0,0],["mistral-medium-latest",262144,262144,"text,image",1,null,1.5,7.5,0,0],["mistral-nemo",128000,128000,"text",0,null,0.15,0.15,0,0],["mistral-small-2506",128000,16384,"text,image",0,null,0.1,0.3,0,0],["mistral-small-2603",256000,256000,"text,image",1,null,0.15,0.6,0,0],["mistral-small-latest",256000,256000,"text,image",1,null,0.15,0.6,0,0],["open-mistral-7b",8000,8000,"text",0,null,0.25,0.25,0,0],["open-mistral-nemo",128000,128000,"text",0,null,0.15,0.15,0,0],["open-mixtral-8x22b",64000,64000,"text",0,null,2,6,0,0],["open-mixtral-8x7b",32000,32000,"text",0,null,0.7,0.7,0,0],["pixtral-12b",128000,128000,"text,image",0,null,0.15,0.15,0,0],["pixtral-large-latest",128000,128000,"text,image",0,null,2,6,0,0]], "moonshot": [["kimi-k2.5",262144,65536,"text,image",1,null,0,0,0,0]], "openai": [["codex-mini-latest",200000,100000,"text",1,null,1.5,6,0.375,0],["gpt-4",8192,8192,"text",0,null,30,60,0,0],["gpt-4-turbo",128000,4096,"text,image",0,null,10,30,0,0],["gpt-4.1",1047576,32768,"text,image",0,null,2,8,0.5,0],["gpt-4.1-mini",1047576,32768,"text,image",0,null,0.4,1.6,0.1,0],["gpt-4.1-nano",1047576,32768,"text,image",0,null,0.1,0.4,0.025,0],["gpt-4o",128000,16384,"text,image",0,null,2.5,10,1.25,0],["gpt-4o-2024-05-13",128000,4096,"text,image",0,null,5,15,0,0],["gpt-4o-2024-08-06",128000,16384,"text,image",0,null,2.5,10,1.25,0],["gpt-4o-2024-11-20",128000,16384,"text,image",0,null,2.5,10,1.25,0],["gpt-4o-mini",128000,16384,"text,image",0,null,0.15,0.6,0.075,0],["gpt-5",400000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5-chat-latest",128000,16384,"text,image",0,null,1.25,10,0.125,0],["gpt-5-codex",272000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5-mini",400000,128000,"text,image",1,null,0.25,2,0.025,0],["gpt-5-nano",400000,128000,"text,image",1,null,0.05,0.4,0.005,0],["gpt-5-pro",400000,272000,"text,image",1,null,15,120,0,0],["gpt-5.1",400000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5.1-chat-latest",128000,16384,"text,image",1,null,1.25,10,0.125,0],["gpt-5.1-codex",272000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5.1-codex-max",272000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5.1-codex-mini",272000,128000,"text,image",1,null,0.25,2,0.025,0],["gpt-5.2",400000,128000,"text,image",1,null,1.75,14,0.175,0],["gpt-5.2-chat-latest",128000,16384,"text,image",1,null,1.75,14,0.175,0],["gpt-5.2-codex",272000,128000,"text,image",1,null,1.75,14,0.175,0],["gpt-5.2-pro",400000,128000,"text,image",1,null,21,168,0,0],["gpt-5.3-chat-latest",128000,16384,"text,image",0,null,1.75,14,0.175,0],["gpt-5.3-codex",272000,128000,"text,image",1,null,1.75,14,0.175,0],["gpt-5.3-codex-spark",128000,32000,"text,image",1,null,1.75,14,0.175,0],["gpt-5.4",1050000,128000,"text,image",1,null,2.5,15,0.25,0],["gpt-5.4-mini",400000,128000,"text,image",1,null,0.75,4.5,0.075,0],["gpt-5.4-nano",400000,128000,"text,image",1,null,0.2,1.25,0.02,0],["gpt-5.4-pro",1050000,128000,"text,image",1,null,30,180,0,0],["gpt-5.5",1050000,128000,"text,image",1,null,5,30,0.5,0],["gpt-5.5-pro",1050000,128000,"text,image",1,null,30,180,0,0],["gpt-5.6",373000,128000,"text,image",1,null,5,30,0.5,6.25],["gpt-5.6-luna",373000,128000,"text,image",1,null,0.2,1.2,0.02,0.25],["gpt-5.6-sol",373000,128000,"text,image",1,null,5,30,0.5,6.25],["gpt-5.6-terra",373000,128000,"text,image",1,null,2,12,0.2,2.5],["gpt-realtime-2.1",128000,32000,"text,image",1,null,4,24,0.4,0],["o1",200000,100000,"text,image",1,null,15,60,7.5,0],["o1-pro",200000,100000,"text,image",1,null,150,600,0,0],["o3",200000,100000,"text,image",1,null,2,8,0.5,0],["o3-deep-research",200000,100000,"text,image",1,null,10,40,2.5,0],["o3-mini",200000,100000,"text",1,null,1.1,4.4,0.55,0],["o3-pro",200000,100000,"text,image",1,null,20,80,0,0],["o4-mini",200000,100000,"text,image",1,null,1.1,4.4,0.275,0],["o4-mini-deep-research",200000,100000,"text,image",1,null,2,8,0.5,0]], + "openai-codex": [["codex-auto-review",1000000,128000,"text,image",1,null,0,0,0,0],["gpt-5",400000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5-codex",272000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5-codex-mini",272000,128000,"text,image",1,null,0,0,0,0],["gpt-5.1",400000,128000,"text,image",1,null,1.25,10,0.13,0],["gpt-5.1-codex",272000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5.1-codex-max",272000,128000,"text,image",1,null,1.25,10,0.125,0],["gpt-5.1-codex-mini",272000,128000,"text,image",1,null,0.25,2,0.025,0],["gpt-5.2",272000,128000,"text,image",1,null,1.75,14,0.175,0],["gpt-5.2-codex",272000,128000,"text,image",1,null,1.75,14,0.175,0],["gpt-5.3-codex",272000,128000,"text,image",1,null,1.75,14,0.175,0],["gpt-5.3-codex-spark",128000,128000,"text",1,null,1.75,14,0.175,0],["gpt-5.4",1000000,128000,"text,image",1,null,2.5,15,0.25,0],["gpt-5.4-mini",272000,128000,"text,image",1,null,0.75,4.5,0.075,0],["gpt-5.4-nano",272000,128000,"text,image",1,null,0.2,1.25,0.02,0],["gpt-5.5",272000,128000,"text,image",1,null,5,30,0.5,0],["gpt-5.6-luna",373000,128000,"text,image",1,null,0.2,1.2,0.02,0.25],["gpt-5.6-sol",373000,128000,"text,image",1,null,5,30,0.5,6.25],["gpt-5.6-terra",373000,128000,"text,image",1,null,2,12,0.2,2.5]], "opencode-go": [["deepseek-v4-flash",1000000,384000,"text",1,null,0.14,0.28,0.0028,0],["deepseek-v4-pro",1000000,384000,"text",1,null,1.74,3.48,0.0145,0],["glm-5",204800,131072,"text",1,null,1,3.2,0.2,0],["glm-5.1",200000,131072,"text",1,null,1.4,4.4,0.26,0],["glm-5.2",1000000,131072,"text",1,null,1.4,4.4,0.26,0],["glm-5.3",1000000,131072,"text",1,null,1.4,4.4,0.26,0],["grok-4.5",500000,500000,"text,image",1,null,2,6,0.5,0],["grok-4.6",500000,500000,"text,image",1,null,2,6,0.5,0],["hy3",256000,64000,"text",1,null,0.14,0.58,0.035,0],["kimi-k2.5",262144,262144,"text,image",1,null,0.3,1.9,0,0],["kimi-k2.6",262144,262144,"text,image",1,null,0.95,4,0.2,0],["kimi-k2.7-code",262144,262144,"text,image",1,null,0.95,4,0.19,0],["kimi-k3",1048576,131072,"text,image",1,null,3,15,0.3,0],["mimo-v2-omni",262144,131072,"text,image",1,null,0.4,2,0.08,0],["mimo-v2-pro",1048576,131072,"text",1,null,1,3,0.2,0],["mimo-v2.5",1048576,131072,"text,image",1,null,0.14,0.28,0.0028,0],["mimo-v2.5-pro",1048576,131072,"text",1,null,1.74,3.48,0.0145,0],["minimax-m2.5",204800,131072,"text",1,null,0.3,1.2,0.06,0.375],["minimax-m2.7",204800,131072,"text",1,null,0.3,1.2,0.06,0.375],["minimax-m3",512000,128000,"text,image",1,null,0.3,1.2,0.06,0],["qwen3.5-plus",1000000,65536,"text,image",1,null,0.4,2.4,0,0],["qwen3.6-plus",1000000,65536,"text,image",1,null,2,6,0.2,2.5],["qwen3.7-max",1000000,65536,"text",1,null,2.5,7.5,0.5,3.125],["qwen3.7-plus",1000000,64000,"text,image",1,null,1.2,4.8,0.12,1.5]], "openrouter": [["~anthropic/claude-fable-latest",1000000,128000,"text,image",1,null,10,50,1,12.5],["~anthropic/claude-haiku-latest",200000,64000,"text,image",1,null,1,5,0.09999999999999999,1.25],["~anthropic/claude-opus-latest",1000000,128000,"text,image",1,null,5,25,0.5,6.25],["~anthropic/claude-sonnet-latest",1000000,128000,"text,image",1,null,2,10,0.19999999999999998,2.5],["~google/gemini-flash-latest",1048576,65536,"text,image",1,null,1.5,7.5,0.15,0.08333333333333334],["~google/gemini-pro-latest",1048576,65536,"text,image",1,null,2,12,0.19999999999999998,0.375],["~moonshotai/kimi-latest",1048576,8888,"text,image",1,null,3,15,0.3,0],["~openai/gpt-latest",1050000,128000,"text,image",1,null,5,30,0.5,6.25],["~openai/gpt-mini-latest",400000,128000,"text,image",1,null,0.75,4.5,0.075,0],["~x-ai/grok-latest",500000,8888,"text,image",1,null,2,6,0.3,0],["ai21/jamba-large-1.7",256000,4096,"text",0,null,2,8,0,0],["aion-labs/aion-2.0",131072,32768,"text",1,null,0.7999999999999999,1.5999999999999999,0.19999999999999998,0],["aion-labs/aion-3.0",131072,32768,"text",1,null,3,6,0.75,0],["aion-labs/aion-3.0-mini",131072,32768,"text",1,null,0.7,1.4,0.18,0],["alibaba/tongyi-deepresearch-30b-a3b",131072,131072,"text",1,null,0.09,0.44999999999999996,0.09,0],["allenai/olmo-3.1-32b-instruct",65536,16384,"text",0,null,0.19999999999999998,0.6,0,0],["amazon/nova-2-lite-v1",1000000,65535,"text,image",1,null,0.3,2.5,0,0],["amazon/nova-lite-v1",300000,5120,"text,image",0,null,0.06,0.24,0,0],["amazon/nova-micro-v1",128000,5120,"text",0,null,0.035,0.14,0,0],["amazon/nova-premier-v1",1000000,32000,"text,image",0,null,2.5,12.5,0.625,0],["amazon/nova-pro-v1",300000,5120,"text,image",0,null,0.7999999999999999,3.1999999999999997,0,0],["anthropic/claude-3-haiku",200000,4096,"text,image",0,null,0.25,1.25,0.03,0.3],["anthropic/claude-3.5-haiku",200000,8192,"text,image",0,null,0.7999999999999999,4,0.08,1],["anthropic/claude-3.5-sonnet",200000,8192,"text,image",0,null,6,30,0.6,7.5],["anthropic/claude-3.7-sonnet",200000,128000,"text,image",1,null,3,15,0.3,3.75],["anthropic/claude-3.7-sonnet:thinking",200000,64000,"text,image",1,null,3,15,0.3,3.75],["anthropic/claude-fable-5",1000000,128000,"text,image",1,null,10,50,1,12.5],["anthropic/claude-haiku-4.5",200000,64000,"text,image",0,null,1,5,0.09999999999999999,1.25],["anthropic/claude-opus-4",200000,32000,"text,image",1,null,15,75,1.5,18.75],["anthropic/claude-opus-4.1",200000,32000,"text,image",1,null,15,75,1.5,18.75],["anthropic/claude-opus-4.5",200000,64000,"text,image",1,null,5,25,0.5,6.25],["anthropic/claude-opus-4.6",1000000,128000,"text,image",1,null,5,25,0.5,6.25],["anthropic/claude-opus-4.6-fast",1000000,128000,"text,image",1,null,30,150,3,37.5],["anthropic/claude-opus-4.7",1000000,128000,"text,image",1,null,5,25,0.5,6.25],["anthropic/claude-opus-4.7-fast",1000000,128000,"text,image",1,null,30,150,3,37.5],["anthropic/claude-opus-4.8",1000000,128000,"text,image",1,null,5,25,0.5,6.25],["anthropic/claude-opus-4.8-fast",1000000,128000,"text,image",1,null,10,50,1,12.5],["anthropic/claude-opus-5",1000000,128000,"text,image",1,null,5,25,0.5,6.25],["anthropic/claude-opus-5-fast",1000000,128000,"text,image",1,null,10,50,1,12.5],["anthropic/claude-sonnet-4",1000000,64000,"text,image",1,null,3,15,0.3,3.75],["anthropic/claude-sonnet-4.5",1000000,64000,"text,image",1,null,3,15,0.3,3.75],["anthropic/claude-sonnet-4.6",1000000,128000,"text,image",1,null,3,15,0.3,3.75],["anthropic/claude-sonnet-5",1000000,128000,"text,image",1,null,2,10,0.19999999999999998,2.5],["arcee-ai/trinity-large-preview",131000,8888,"text",0,null,0.15,0.44999999999999996,0,0],["arcee-ai/trinity-large-preview:free",131000,8888,"text",0,null,0,0,0,0],["arcee-ai/trinity-large-thinking",262144,262144,"text",1,null,0.22,0.85,0.06,0],["arcee-ai/trinity-large-thinking:free",262144,80000,"text",1,null,0,0,0,0],["arcee-ai/trinity-mini",131072,131072,"text",1,null,0.045,0.15,0,0],["arcee-ai/trinity-mini:free",131072,8888,"text",1,null,0,0,0,0],["arcee-ai/virtuoso-large",131072,64000,"text",0,null,0.75,1.2,0,0],["auto",2000000,30000,"text,image",1,null,0,0,0,0],["baidu/cobuddy:free",131072,65536,"text",1,null,0,0,0,0],["baidu/ernie-4.5-21b-a3b",131072,8000,"text",0,null,0.07,0.28,0,0],["baidu/ernie-4.5-vl-28b-a3b",131072,8000,"text,image",1,null,0.14,0.56,0,0],["bytedance-seed/seed-1.6",262144,32768,"text,image",1,null,0.25,2,0,0],["bytedance-seed/seed-1.6-flash",262144,32768,"text,image",1,null,0.075,0.3,0,0],["bytedance-seed/seed-2.0-lite",262144,131072,"text,image",1,null,0.25,2,0,0],["bytedance-seed/seed-2.0-mini",262144,131072,"text,image",1,null,0.09999999999999999,0.39999999999999997,0,0],["cohere/command-r-08-2024",128000,4000,"text",0,null,0.15,0.6,0,0],["cohere/command-r-plus-08-2024",128000,4000,"text",0,null,2.5,10,0,0],["cohere/north-mini-code:free",256000,64000,"text",1,null,0,0,0,0],["deepseek/deepseek-chat",163840,16000,"text",0,null,0.20020000000000002,0.8000999999999999,0.15,0],["deepseek/deepseek-chat-v3-0324",163840,65536,"text",1,null,0.27,1.12,0.135,0],["deepseek/deepseek-chat-v3.1",163840,32768,"text",1,null,0.25,0.95,0.13,0],["deepseek/deepseek-r1",163840,16000,"text",1,null,0.7,2.5,0,0],["deepseek/deepseek-r1-0528",163840,32768,"text",1,null,0.5,2.1500000000000004,0.35,0],["deepseek/deepseek-v3.1-terminus",163840,32768,"text",1,null,0.27,1,0.135,0],["deepseek/deepseek-v3.1-terminus:exacto",163840,8888,"text",1,null,0.21,0.7899999999999999,0.16799999999999998,0],["deepseek/deepseek-v3.2",163840,65536,"text",1,null,0.26899999999999996,0.39999999999999997,0.13449999999999998,0],["deepseek/deepseek-v3.2-exp",163840,65536,"text",1,null,0.27,0.41,0,0],["deepseek/deepseek-v4-flash",1048576,384000,"text",1,null,0.09380000000000001,0.18760000000000002,0.01876,0],["deepseek/deepseek-v4-flash:free",1048576,384000,"text",1,null,0,0,0,0],["deepseek/deepseek-v4-pro",1048576,384000,"text",1,null,0.435,0.87,0.003625,0],["essentialai/rnj-1-instruct",32768,8888,"text",0,null,0.15,0.15,0,0],["google/gemini-2.0-flash-001",1048576,8192,"text,image",0,null,0.09999999999999999,0.39999999999999997,0.024999999999999998,0.08333333333333334],["google/gemini-2.0-flash-lite-001",1048576,8192,"text,image",0,null,0.075,0.3,0,0],["google/gemini-2.5-flash",1048576,65535,"text,image",1,null,0.3,2.5,0.03,0.08333333333333334],["google/gemini-2.5-flash-lite",1048576,65535,"text,image",0,null,0.09999999999999999,0.39999999999999997,0.01,0.08333333333333334],["google/gemini-2.5-flash-lite-preview-09-2025",1048576,65535,"text,image",1,null,0.09999999999999999,0.39999999999999997,0.01,0.08333333333333334],["google/gemini-2.5-flash-preview-09-2025",1048576,65536,"text,image",1,null,0.3,2.5,0.03,0.08333333333333334],["google/gemini-2.5-pro",1048576,65536,"text,image",1,null,1.25,10,0.125,0.375],["google/gemini-2.5-pro-preview",1048576,65536,"text,image",1,null,1.25,10,0.125,0.375],["google/gemini-2.5-pro-preview-05-06",1048576,65535,"text,image",1,null,1.25,10,0.125,0.375],["google/gemini-3-flash-preview",1048576,65535,"text,image",1,null,0.5,3,0.049999999999999996,0.08333333333333334],["google/gemini-3-pro-image",131072,32768,"text,image",1,null,2,12,0.19999999999999998,0.375],["google/gemini-3-pro-preview",1048000,64000,"text,image",1,null,2,12,0.19999999999999998,0.375],["google/gemini-3.1-flash-lite",1048576,65536,"text,image",1,null,0.25,1.5,0.024999999999999998,0.08333333333333334],["google/gemini-3.1-flash-lite-preview",1048576,65536,"text,image",0,null,0.25,1.5,0.024999999999999998,0.08333333333333334],["google/gemini-3.1-pro-preview",1048576,65536,"text,image",1,null,2,12,0.19999999999999998,0.375],["google/gemini-3.1-pro-preview-customtools",1048576,65536,"text,image",1,null,2,12,0.19999999999999998,0.375],["google/gemini-3.5-flash",1048576,65536,"text,image",1,null,1.5,9,0.15,0.08333333333333334],["google/gemini-3.5-flash-lite",1048576,65536,"text,image",1,null,0.3,2.5,0.03,0.08333333333333334],["google/gemini-3.6-flash",1048576,65536,"text,image",1,null,1.5,7.5,0.15,0.08333333333333334],["google/gemma-3-12b-it",131072,16384,"text,image",0,null,0.049999999999999996,0.15,0,0],["google/gemma-3-27b-it",262144,131072,"text,image",1,null,0.08,0.44999999999999996,0.04,0],["google/gemma-3-27b-it:free",131072,8192,"text,image",0,null,0,0,0,0],["google/gemma-4-26b-a4b-it",262144,262144,"text,image",1,null,0.12,0.35,0.049999999999999996,0],["google/gemma-4-26b-a4b-it:free",262144,32768,"text,image",1,null,0,0,0,0],["google/gemma-4-31b-it",262144,262144,"text,image",1,null,0.14,0.39999999999999997,0.09,0],["google/gemma-4-31b-it:free",262144,32768,"text,image",1,null,0,0,0,0],["ibm-granite/granite-4.1-8b",131072,131072,"text",0,null,0.049999999999999996,0.09999999999999999,0.049999999999999996,0],["inception/mercury",128000,32000,"text",0,null,0.25,0.75,0.024999999999999998,0],["inception/mercury-2",128000,50000,"text",1,null,0.25,0.75,0.024999999999999998,0],["inception/mercury-coder",128000,32000,"text",0,null,0.25,0.75,0.024999999999999998,0],["inclusionai/ling-2.6-1t",262144,32768,"text",0,null,0.075,0.625,0.015,0],["inclusionai/ling-2.6-1t:free",262144,32768,"text",0,null,0,0,0,0],["inclusionai/ling-2.6-flash",262144,32768,"text",0,null,0.01,0.03,0.002,0],["inclusionai/ling-2.6-flash:free",262144,32768,"text",0,null,0,0,0,0],["inclusionai/ling-3.0-flash:free",262144,32768,"text",1,null,0,0,0,0],["inclusionai/ring-2.6-1t",262144,65536,"text",1,null,0.075,0.625,0.015,0],["inclusionai/ring-2.6-1t:free",262144,65536,"text",1,null,0,0,0,0],["kwaipilot/kat-coder-air-v2.5",256000,80000,"text",0,null,0.15,0.6,0.03,0],["kwaipilot/kat-coder-pro",256000,128000,"text",0,null,0.207,0.828,0.0414,0],["kwaipilot/kat-coder-pro-v2",262144,80000,"text",0,null,0.3,1.2,0.06,0],["kwaipilot/kat-coder-pro-v2.5",256000,80000,"text",0,null,0.74,2.96,0.15,0],["liquid/lfm-2.5-1.2b-thinking:free",32768,8888,"text",1,null,0,0,0,0],["meituan/longcat-2.0",1048756,262144,"text",1,null,0.3,1.2,0.006,0],["meituan/longcat-flash-chat",131072,131072,"text",0,null,0.19999999999999998,0.7999999999999999,0.19999999999999998,0],["meta-llama/llama-3-8b-instruct",8192,16384,"text",0,null,0.03,0.04,0,0],["meta-llama/llama-3.1-405b-instruct",131000,8888,"text",0,null,4,4,0,0],["meta-llama/llama-3.1-70b-instruct",131072,16384,"text",0,null,0.39999999999999997,0.39999999999999997,0,0],["meta-llama/llama-3.1-8b-instruct",131072,131072,"text",0,null,0.049999999999999996,0.08,0.024999999999999998,0],["meta-llama/llama-3.3-70b-instruct",131072,128000,"text",0,null,0.13,0.39999999999999997,0,0],["meta-llama/llama-3.3-70b-instruct:free",131072,8888,"text",0,null,0,0,0,0],["meta-llama/llama-4-maverick",1048576,16384,"text,image",0,null,0.19999999999999998,0.7999999999999999,0,0],["meta-llama/llama-4-scout",1310720,16384,"text,image",0,null,0.09999999999999999,0.3,0,0],["meta/muse-spark-1.1",1048576,8888,"text,image",1,null,1.25,4.25,0.15,0],["minimax/minimax-m1",1000000,40000,"text",1,null,0.55,2.2,0,0],["minimax/minimax-m2",204800,131072,"text",1,null,0.255,1.02,0.03,0],["minimax/minimax-m2.1",204800,131072,"text",1,null,0.3,1.2,0.03,0],["minimax/minimax-m2.5",204800,196608,"text",1,null,0.15,0.8999999999999999,0.049999999999999996,0],["minimax/minimax-m2.5:free",262144,8192,"text",1,null,0,0,0,0],["minimax/minimax-m2.7",204800,131072,"text",1,null,0.25,1,0.049999999999999996,0],["minimax/minimax-m3",1048576,512000,"text,image",1,null,0.3,1.2,0.06,0],["mistralai/codestral-2508",256000,8888,"text",0,null,0.3,0.8999999999999999,0.03,0],["mistralai/devstral-2512",262144,8888,"text",0,null,0.39999999999999997,2,0.04,0],["mistralai/devstral-medium",131072,8888,"text",0,null,0.39999999999999997,2,0.04,0],["mistralai/devstral-small",131072,8888,"text",0,null,0.09999999999999999,0.3,0.01,0],["mistralai/ministral-14b-2512",262144,8888,"text,image",0,null,0.19999999999999998,0.19999999999999998,0.02,0],["mistralai/ministral-3b-2512",131072,8888,"text,image",0,null,0.09999999999999999,0.09999999999999999,0.01,0],["mistralai/ministral-8b-2512",262144,8888,"text,image",0,null,0.15,0.15,0.015,0],["mistralai/mistral-large",128000,8888,"text",0,null,2,6,0.19999999999999998,0],["mistralai/mistral-large-2407",131072,8888,"text",0,null,2,6,0.19999999999999998,0],["mistralai/mistral-large-2411",131072,8888,"text",0,null,2,6,0.19999999999999998,0],["mistralai/mistral-large-2512",262144,8888,"text,image",0,null,0.5,1.5,0.049999999999999996,0],["mistralai/mistral-medium-3",131072,8888,"text,image",0,null,0.39999999999999997,2,0.04,0],["mistralai/mistral-medium-3-5",262144,8888,"text,image",1,null,1.5,7.5,0,0],["mistralai/mistral-medium-3.1",131072,8888,"text,image",0,null,0.39999999999999997,2,0.04,0],["mistralai/mistral-nemo",131072,16384,"text",0,null,0.019000000000000003,0.03,0,0],["mistralai/mistral-saba",32768,8888,"text",0,null,0.19999999999999998,0.6,0.02,0],["mistralai/mistral-small-24b-instruct-2501",32768,16384,"text",0,null,0.049999999999999996,0.08,0,0],["mistralai/mistral-small-2603",262144,8888,"text,image",1,null,0.15,0.6,0.015,0],["mistralai/mistral-small-3.1-24b-instruct",131072,131072,"text,image",0,null,0.03,0.11,0.015,0],["mistralai/mistral-small-3.1-24b-instruct:free",128000,8888,"text,image",0,null,0,0,0,0],["mistralai/mistral-small-3.2-24b-instruct",256000,8888,"text,image",0,null,0.09999999999999999,0.3,0.01,0],["mistralai/mistral-small-creative",32768,8888,"text",0,null,0.09999999999999999,0.3,0.01,0],["mistralai/mixtral-8x22b-instruct",65536,13108,"text",0,null,2,6,0.19999999999999998,0],["mistralai/mixtral-8x7b-instruct",32768,16384,"text",0,null,0.54,0.54,0,0],["mistralai/pixtral-large-2411",131072,8888,"text,image",0,null,2,6,0.19999999999999998,0],["mistralai/voxtral-small-24b-2507",32000,8888,"text",0,null,0.09999999999999999,0.3,0.01,0],["moonshotai/kimi-k2",131072,100352,"text",0,null,0.5700000000000001,2.3,0,0],["moonshotai/kimi-k2-0905",262144,100352,"text",0,null,0.6,2.5,0.15,0],["moonshotai/kimi-k2-0905:exacto",262144,8888,"text",0,null,0.6,2.5,0,0],["moonshotai/kimi-k2-thinking",262144,100352,"text",1,null,0.6,2.5,0.15,0],["moonshotai/kimi-k2.5",262144,262144,"text,image",1,null,0.5700000000000001,2.8499999999999996,0.095,0],["moonshotai/kimi-k2.6",262144,262144,"text,image",1,null,0.646,2.7199999999999998,0.1088,0],["moonshotai/kimi-k2.6:free",262144,8888,"text,image",1,null,0,0,0,0],["moonshotai/kimi-k2.7-code",262144,262144,"text,image",1,null,0.78,3.5,0.15,0],["moonshotai/kimi-k3",1048576,131072,"text,image",1,null,3,15,0.3,0],["nex-agi/deepseek-v3.1-nex-n1",131072,163840,"text",0,null,0.135,0.5,0,0],["nex-agi/nex-n2-mini",262144,262144,"text,image",1,null,0.024999999999999998,0.09999999999999999,0.0025,0],["nex-agi/nex-n2-pro",262144,262144,"text,image",1,null,0.25,1,0.024999999999999998,0],["nex-agi/nex-n2-pro:free",262144,262144,"text,image",1,null,0,0,0,0],["nousresearch/deephermes-3-mistral-24b-preview",32768,32768,"text",1,null,0.02,0.09999999999999999,0.01,0],["nousresearch/hermes-4-70b",131072,131072,"text",1,null,0.11,0.38,0.055,0],["nvidia/llama-3.1-nemotron-70b-instruct",131072,16384,"text",0,null,1.2,1.2,0,0],["nvidia/llama-3.3-nemotron-super-49b-v1.5",131072,16384,"text",1,null,0.39999999999999997,0.39999999999999997,0,0],["nvidia/nemotron-3-nano-30b-a3b",262144,228000,"text",1,null,0.049999999999999996,0.19999999999999998,0,0],["nvidia/nemotron-3-nano-30b-a3b:free",256000,8888,"text",1,null,0,0,0,0],["nvidia/nemotron-3-nano-omni-30b-a3b-reasoning:free",256000,65536,"text,image",1,null,0,0,0,0],["nvidia/nemotron-3-super-120b-a12b",1000000,16384,"text",1,null,0.08499999999999999,0.39999999999999997,0.09999999999999999,0],["nvidia/nemotron-3-super-120b-a12b:free",262144,262144,"text",1,null,0,0,0,0],["nvidia/nemotron-3-ultra-550b-a55b",512288,65536,"text",1,null,0.6,3.5999999999999996,0.19999999999999998,0],["nvidia/nemotron-3-ultra-550b-a55b:free",1000000,65536,"text",1,null,0,0,0,0],["nvidia/nemotron-nano-12b-v2-vl:free",128000,128000,"text,image",1,null,0,0,0,0],["nvidia/nemotron-nano-9b-v2",131072,16384,"text",1,null,0.04,0.16,0,0],["nvidia/nemotron-nano-9b-v2:free",128000,8888,"text",1,null,0,0,0,0],["openai/gpt-3.5-turbo",16385,4096,"text",0,null,0.5,1.5,0,0],["openai/gpt-3.5-turbo-0613",4095,4096,"text",0,null,1,2,0,0],["openai/gpt-3.5-turbo-16k",16385,4096,"text",0,null,3,4,0,0],["openai/gpt-4",8191,8192,"text",0,null,30,60,0,0],["openai/gpt-4-0314",8191,4096,"text",0,null,30,60,0,0],["openai/gpt-4-1106-preview",128000,4096,"text",0,null,10,30,0,0],["openai/gpt-4-turbo",128000,4096,"text,image",0,null,10,30,0,0],["openai/gpt-4-turbo-preview",128000,4096,"text",0,null,10,30,0,0],["openai/gpt-4.1",1047576,32768,"text,image",0,null,2,8,0.5,0],["openai/gpt-4.1-mini",1047576,32768,"text,image",0,null,0.39999999999999997,1.5999999999999999,0.09999999999999999,0],["openai/gpt-4.1-nano",1047576,32768,"text,image",0,null,0.09999999999999999,0.39999999999999997,0.024999999999999998,0],["openai/gpt-4o",128000,16384,"text,image",0,null,2.5,10,1.25,0],["openai/gpt-4o-2024-05-13",128000,4096,"text,image",0,null,5,15,0,0],["openai/gpt-4o-2024-08-06",128000,16384,"text,image",0,null,2.5,10,1.25,0],["openai/gpt-4o-2024-11-20",128000,16384,"text,image",0,null,2.5,10,1.25,0],["openai/gpt-4o-audio-preview",128000,16384,"text",0,null,2.5,10,0,0],["openai/gpt-4o-mini",128000,16384,"text,image",0,null,0.15,0.6,0.075,0],["openai/gpt-4o-mini-2024-07-18",128000,16384,"text,image",0,null,0.15,0.6,0.075,0],["openai/gpt-4o:extended",128000,64000,"text,image",0,null,6,18,0,0],["openai/gpt-5",400000,128000,"text,image",1,null,1.25,10,0.125,0],["openai/gpt-5-codex",272000,128000,"text,image",1,null,1.25,10,0.125,0],["openai/gpt-5-image",400000,128000,"text,image",1,null,10,10,1.25,0],["openai/gpt-5-image-mini",400000,128000,"text,image",1,null,2.5,2,0.25,0],["openai/gpt-5-mini",400000,128000,"text,image",1,null,0.25,2,0.024999999999999998,0],["openai/gpt-5-nano",400000,128000,"text,image",1,null,0.049999999999999996,0.39999999999999997,0.005,0],["openai/gpt-5-pro",400000,128000,"text,image",1,null,15,120,0,0],["openai/gpt-5.1",400000,128000,"text,image",1,null,1.25,10,0.125,0],["openai/gpt-5.1-chat",128000,16384,"text,image",0,null,1.25,10,0.125,0],["openai/gpt-5.1-codex",272000,128000,"text,image",1,null,1.25,10,0.125,0],["openai/gpt-5.1-codex-max",272000,128000,"text,image",1,null,1.25,10,0.125,0],["openai/gpt-5.1-codex-mini",272000,100000,"text,image",1,null,0.25,2,0.024999999999999998,0],["openai/gpt-5.2",400000,128000,"text,image",1,null,1.75,14,0.175,0],["openai/gpt-5.2-chat",128000,16384,"text,image",0,null,1.75,14,0.175,0],["openai/gpt-5.2-codex",272000,128000,"text,image",1,null,1.75,14,0.175,0],["openai/gpt-5.2-pro",400000,128000,"text,image",1,null,21,168,0,0],["openai/gpt-5.3-chat",128000,16384,"text",0,null,1.75,14,0.175,0],["openai/gpt-5.3-codex",272000,128000,"text,image",1,null,1.75,14,0.175,0],["openai/gpt-5.4",1050000,128000,"text,image",1,null,2.5,15,0.25,0],["openai/gpt-5.4-mini",400000,128000,"text",0,null,0.75,4.5,0.075,0],["openai/gpt-5.4-nano",400000,128000,"text",0,null,0.19999999999999998,1.25,0.02,0],["openai/gpt-5.4-pro",1050000,128000,"text,image",1,null,30,180,0,0],["openai/gpt-5.5",1050000,128000,"text,image",1,null,5,30,0.5,0],["openai/gpt-5.5-pro",1050000,128000,"text,image",1,null,30,180,0,0],["openai/gpt-5.6-luna",373000,128000,"text,image",1,null,1,6,0.09999999999999999,1.25],["openai/gpt-5.6-luna-pro",373000,128000,"text,image",1,null,1,6,0.09999999999999999,1.25],["openai/gpt-5.6-sol",373000,128000,"text,image",1,null,5,30,0.5,6.25],["openai/gpt-5.6-sol-pro",373000,128000,"text,image",1,null,5,30,0.5,6.25],["openai/gpt-5.6-terra",373000,128000,"text,image",1,null,2.5,15,0.25,3.125],["openai/gpt-5.6-terra-pro",373000,128000,"text,image",1,null,2.5,15,0.25,3.125],["openai/gpt-audio",128000,16384,"text",0,null,2.5,10,0,0],["openai/gpt-audio-mini",128000,16384,"text",0,null,0.6,2.4,0,0],["openai/gpt-chat-latest",400000,128000,"text,image",0,null,5,30,0.5,0],["openai/gpt-oss-120b",131072,131072,"text",1,null,0.037,0.16999999999999998,0,0],["openai/gpt-oss-120b:exacto",131072,8888,"text",1,null,0.039,0.19,0,0],["openai/gpt-oss-120b:free",131072,131072,"text",1,null,0,0,0,0],["openai/gpt-oss-20b",131072,131072,"text",1,null,0.03,0.13,0.03,0],["openai/gpt-oss-20b:free",131072,32768,"text",1,null,0,0,0,0],["openai/gpt-oss-safeguard-20b",131072,65536,"text",1,null,0.075,0.3,0.0375,0],["openai/o1",200000,100000,"text,image",1,null,15,60,7.5,0],["openai/o3",200000,100000,"text,image",1,null,2,8,0.5,0],["openai/o3-deep-research",200000,100000,"text,image",1,null,10,40,2.5,0],["openai/o3-mini",200000,100000,"text",1,null,1.1,4.4,0.55,0],["openai/o3-mini-high",200000,100000,"text",1,null,1.1,4.4,0.55,0],["openai/o3-pro",200000,100000,"text,image",1,null,20,80,0,0],["openai/o4-mini",200000,100000,"text,image",1,null,1.1,4.4,0.275,0],["openai/o4-mini-deep-research",200000,100000,"text,image",1,null,2,8,0.5,0],["openai/o4-mini-high",200000,100000,"text,image",1,null,1.1,4.4,0.275,0],["openrouter/aurora-alpha",128000,50000,"text",1,null,0,0,0,0],["openrouter/auto",2000000,8888,"text,image",1,null,-1000000,-1000000,0,0],["openrouter/auto-beta",2000000,8888,"text,image",1,null,-1000000,-1000000,0,0],["openrouter/elephant-alpha",262144,32768,"text",0,null,0,0,0,0],["openrouter/free",200000,8888,"text,image",1,null,0,0,0,0],["openrouter/healer-alpha",262144,32000,"text,image",1,null,0,0,0,0],["openrouter/hunter-alpha",1048576,32000,"text",1,null,0,0,0,0],["openrouter/owl-alpha",1048756,262144,"text",0,null,0,0,0,0],["poolside/laguna-m.1",262144,32768,"text",1,null,0.19999999999999998,0.39999999999999997,0.09999999999999999,0],["poolside/laguna-m.1:free",262144,32768,"text",1,null,0,0,0,0],["poolside/laguna-s-2.1",1048576,131072,"text",1,null,0.09999999999999999,0.19999999999999998,0.01,0],["poolside/laguna-s-2.1:free",262144,32768,"text",1,null,0,0,0,0],["poolside/laguna-xs-2.1",262144,32768,"text",1,null,0.06,0.12,0.03,0],["poolside/laguna-xs-2.1:free",262144,32768,"text",1,null,0,0,0,0],["poolside/laguna-xs.2",262144,32768,"text",1,null,0.09999999999999999,0.19999999999999998,0.049999999999999996,0],["poolside/laguna-xs.2:free",262144,32768,"text",1,null,0,0,0,0],["prime-intellect/intellect-3",131072,131072,"text",1,null,0.19999999999999998,1.1,0,0],["qwen/qwen-2.5-72b-instruct",32768,16384,"text",0,null,0.36,0.39999999999999997,0,0],["qwen/qwen-2.5-7b-instruct",32768,32768,"text",0,null,0.04,0.09999999999999999,0,0],["qwen/qwen-max",32768,8192,"text",0,null,1.04,4.16,0.20800000000000002,0],["qwen/qwen-plus",1000000,32768,"text",0,null,0.26,0.78,0.052000000000000005,0.325],["qwen/qwen-plus-2025-07-28",1000000,32768,"text",0,null,0.26,0.78,0,0.325],["qwen/qwen-plus-2025-07-28:thinking",1000000,32768,"text",1,null,0.26,0.78,0,0.325],["qwen/qwen-turbo",131072,8192,"text",0,null,0.0325,0.13,0.006500000000000001,0],["qwen/qwen-vl-max",131072,32768,"text,image",0,null,0.52,2.08,0,0],["qwen/qwen3-14b",131072,8192,"text",1,null,0.22749999999999998,0.9099999999999999,0,0],["qwen/qwen3-235b-a22b",131072,8192,"text",1,null,0.45499999999999996,1.8199999999999998,0,0],["qwen/qwen3-235b-a22b-2507",262144,16384,"text",1,null,0.09,0.55,0,0],["qwen/qwen3-235b-a22b-thinking-2507",262144,32768,"text",1,null,0.3,3,0.09999999999999999,0],["qwen/qwen3-30b-a3b",131072,8192,"text",1,null,0.13,0.52,0,0],["qwen/qwen3-30b-a3b-instruct-2507",262144,32000,"text",0,null,0.04815,0.19305,0,0],["qwen/qwen3-30b-a3b-thinking-2507",81920,32768,"text",1,null,0.13,1.56,0.08,0],["qwen/qwen3-32b",131072,16384,"text",1,null,0.08,0.28,0.04,0],["qwen/qwen3-4b",131072,8192,"text",1,null,0.0715,0.273,0,0],["qwen/qwen3-4b:free",40960,8888,"text",1,null,0,0,0,0],["qwen/qwen3-8b",131072,8192,"text",1,null,0.117,0.45499999999999996,0.049999999999999996,0],["qwen/qwen3-coder",262144,65536,"text",0,null,0.3,1,0.09999999999999999,0],["qwen/qwen3-coder-30b-a3b-instruct",262144,32768,"text",0,null,0.07,0.27,0,0],["qwen/qwen3-coder-flash",1000000,65536,"text",0,null,0.195,0.975,0.039,0.24375],["qwen/qwen3-coder-next",262144,262144,"text",0,null,0.11,0.7999999999999999,0.07,0],["qwen/qwen3-coder-plus",1000000,65536,"text",0,null,0.65,3.25,0.13,0.8125],["qwen/qwen3-coder:exacto",262144,65536,"text",0,null,0.22,1.7999999999999998,0.022,0],["qwen/qwen3-coder:free",1048576,262000,"text",0,null,0,0,0,0],["qwen/qwen3-max",262144,32768,"text",1,null,0.78,3.9,0.156,0.975],["qwen/qwen3-max-thinking",262144,32768,"text",1,null,0.78,3.9,0,0],["qwen/qwen3-next-80b-a3b-instruct",262144,262144,"text",0,null,0.09999999999999999,1.1,0.07,0],["qwen/qwen3-next-80b-a3b-instruct:free",262144,8888,"text",0,null,0,0,0,0],["qwen/qwen3-next-80b-a3b-thinking",262144,32768,"text",1,null,0.0975,0.78,0,0],["qwen/qwen3-vl-235b-a22b-instruct",262144,32768,"text,image",0,null,0.21,1.9,0.09999999999999999,0],["qwen/qwen3-vl-235b-a22b-thinking",131072,32768,"text,image",1,null,0.26,2.6,0,0],["qwen/qwen3-vl-30b-a3b-instruct",262144,16384,"text,image",0,null,0.15,0.6,0,0],["qwen/qwen3-vl-30b-a3b-thinking",262144,32768,"text,image",1,null,0.13,1.56,0,0],["qwen/qwen3-vl-32b-instruct",131072,32768,"text,image",0,null,0.10400000000000001,0.41600000000000004,0,0],["qwen/qwen3-vl-8b-instruct",262144,32768,"text,image",0,null,0.117,0.45499999999999996,0,0],["qwen/qwen3-vl-8b-thinking",131072,32768,"text,image",1,null,0.117,1.365,0,0],["qwen/qwen3.5-122b-a10b",262144,65536,"text,image",1,null,0.26,2.08,0,0],["qwen/qwen3.5-27b",262144,65536,"text,image",1,null,0.195,1.56,0,0],["qwen/qwen3.5-35b-a3b",262144,262144,"text,image",1,null,0.14,1,0.049999999999999996,0],["qwen/qwen3.5-397b-a17b",262144,65536,"text,image",1,null,0.39,2.34,0.111,0],["qwen/qwen3.5-9b",262144,262144,"text,image",1,null,0.09999999999999999,0.15,0,0],["qwen/qwen3.5-flash-02-23",1000000,65536,"text,image",1,null,0.065,0.26,0,0.08125],["qwen/qwen3.5-plus-02-15",1000000,65536,"text,image",1,null,0.26,1.56,0,0.325],["qwen/qwen3.5-plus-20260420",1000000,65536,"text,image",1,null,0.3,1.7999999999999998,0,0.375],["qwen/qwen3.6-27b",262144,131072,"text,image",1,null,0.28900000000000003,2.4,0.15,0],["qwen/qwen3.6-35b-a3b",262144,262144,"text,image",1,null,0.14,1,0.049999999999999996,0],["qwen/qwen3.6-flash",1000000,65536,"text,image",1,null,0.1875,1.125,0,0.234375],["qwen/qwen3.6-max-preview",262144,65536,"text",1,null,1.04,6.24,0,1.3],["qwen/qwen3.6-plus",1000000,65536,"text",1,null,0.325,1.95,0,0.40625],["qwen/qwen3.6-plus-preview:free",1000000,32000,"text",1,null,0,0,0,0],["qwen/qwen3.6-plus:free",1000000,65536,"text,image",1,null,0,0,0,0],["qwen/qwen3.7-max",1000000,65536,"text",1,null,1.475,4.425,0.295,1.84375],["qwen/qwen3.7-plus",1000000,65536,"text,image",1,null,0.32,1.28,0.064,0.39999999999999997],["qwen/qwq-32b",131072,131072,"text",1,null,0.15,0.58,0,0],["reka/reka-edge",16384,16384,"text,image",0,null,0.09999999999999999,0.09999999999999999,0,0],["rekaai/reka-edge",16384,16384,"text,image",0,null,0.09999999999999999,0.09999999999999999,0,0],["relace/relace-search",256000,128000,"text",0,null,1,3,0,0],["sakana/fugu-ultra",1000000,128000,"text,image",1,null,5,30,0.5,0],["sao10k/l3-euryale-70b",8192,8192,"text",0,null,1.48,1.48,0,0],["sao10k/l3.1-euryale-70b",131072,16384,"text",0,null,0.85,0.85,0,0],["stepfun/step-3.5-flash",262144,65536,"text",0,null,0.09999999999999999,0.3,0.02,0],["stepfun/step-3.5-flash:free",256000,256000,"text",1,null,0,0,0,0],["stepfun/step-3.7-flash",262144,256000,"text,image",1,null,0.19999999999999998,1.15,0.04,0],["tencent/hy3",262144,128000,"text",1,null,0.13199999999999998,0.5279999999999999,0.032999999999999995,0],["tencent/hy3-preview",262144,64000,"text",1,null,0.063,0.21,0.020999999999999998,0],["tencent/hy3-preview:free",262144,262144,"text",1,null,0,0,0,0],["tencent/hy3:free",262144,262144,"text",1,null,0,0,0,0],["thedrummer/rocinante-12b",32768,32768,"text",0,null,0.16999999999999998,0.43,0,0],["thedrummer/unslopnemo-12b",32768,32768,"text",0,null,0.39999999999999997,0.39999999999999997,0,0],["thinkingmachines/inkling",1048576,8888,"text,image",1,null,1,4.05,0.16999999999999998,0],["tngtech/deepseek-r1t2-chimera",163840,163840,"text",1,null,0.3,1.1,0.15,0],["tngtech/tng-r1t-chimera",163840,65536,"text",1,null,0.25,0.85,0.125,0],["upstage/solar-pro-3",128000,8888,"text",1,null,0.15,0.6,0.015,0],["upstage/solar-pro-3:free",128000,8888,"text",1,null,0,0,0,0],["x-ai/grok-3",131072,8888,"text",0,null,3,15,0.75,0],["x-ai/grok-3-beta",131072,8888,"text",0,null,3,15,0.75,0],["x-ai/grok-3-mini",131072,8888,"text",1,null,0.3,0.5,0.075,0],["x-ai/grok-3-mini-beta",131072,8888,"text",1,null,0.3,0.5,0.075,0],["x-ai/grok-4",256000,64000,"text,image",1,null,3,15,0.75,0],["x-ai/grok-4-fast",2000000,30000,"text,image",1,null,0.19999999999999998,0.5,0.049999999999999996,0],["x-ai/grok-4.1-fast",2000000,30000,"text,image",1,null,0.19999999999999998,0.5,0.049999999999999996,0],["x-ai/grok-4.20",2000000,8888,"text,image",1,null,1.25,2.5,0.19999999999999998,0],["x-ai/grok-4.20-beta",2000000,8888,"text,image",1,null,2,6,0.19999999999999998,0],["x-ai/grok-4.3",1000000,1000000,"text,image",1,null,1.25,2.5,0.19999999999999998,0],["x-ai/grok-4.5",500000,500000,"text,image",1,null,2,6,0.3,0],["x-ai/grok-4.6",500000,500000,"text,image",1,null,2,6,0.3,0],["x-ai/grok-build-0.1",256000,256000,"text,image",1,null,1,2,0.19999999999999998,0],["x-ai/grok-code-fast-1",256000,10000,"text",1,null,0.19999999999999998,1.5,0.02,0],["xiaomi/mimo-v2-flash",262144,65536,"text",1,null,0.09999999999999999,0.3,0.01,0],["xiaomi/mimo-v2-omni",262144,65536,"text,image",1,null,0.39999999999999997,2,0.08,0],["xiaomi/mimo-v2-pro",1048576,131072,"text",1,null,1,3,0.19999999999999998,0],["xiaomi/mimo-v2.5",1050000,131072,"text,image",1,null,0.14,0.28,0.0028,0],["xiaomi/mimo-v2.5-pro",1050000,131072,"text",1,null,0.435,0.87,0.0036,0],["z-ai/glm-4-32b",128000,8888,"text",0,null,0.09999999999999999,0.09999999999999999,0,0],["z-ai/glm-4.5",131072,98304,"text",1,null,0.6,2.2,0.11,0],["z-ai/glm-4.5-air",131072,98304,"text",1,null,0.13,0.85,0.024999999999999998,0],["z-ai/glm-4.5-air:free",131072,96000,"text",1,null,0,0,0,0],["z-ai/glm-4.5v",65536,16384,"text,image",1,null,0.6,1.7999999999999998,0.11,0],["z-ai/glm-4.6",204800,131072,"text",1,null,0.5,2,0.09999999999999999,0],["z-ai/glm-4.6:exacto",204800,131072,"text",1,null,0.44,1.76,0.11,0],["z-ai/glm-4.6v",131072,32768,"text,image",1,null,0.3,0.8999999999999999,0.055,0],["z-ai/glm-4.7",204800,131072,"text",1,null,0.39999999999999997,1.75,0.08,0],["z-ai/glm-4.7-flash",202752,16384,"text",1,null,0.06,0.39999999999999997,0.01,0],["z-ai/glm-5",204800,131072,"text",1,null,0.95,2.5500000000000003,0.19999999999999998,0],["z-ai/glm-5-turbo",202752,131072,"text",1,null,1.2,4,0.24,0],["z-ai/glm-5.1",204800,128000,"text",1,null,0.966,3.036,0.1794,0],["z-ai/glm-5.2",1048576,131072,"text",1,null,0.707,2.222,0.1313,0],["z-ai/glm-5.3",1048576,131072,"text",1,null,0.707,2.222,0.1313,0],["z-ai/glm-5v-turbo",202752,131072,"text,image",1,null,1.2,4,0.24,0]], "xai": [["grok-2",131072,8192,"text",0,null,2,10,2,0],["grok-2-1212",131072,8192,"text",0,null,2,10,2,0],["grok-2-latest",131072,8192,"text",0,null,2,10,2,0],["grok-2-vision",8192,4096,"text,image",0,null,2,10,2,0],["grok-2-vision-1212",8192,4096,"text,image",0,null,2,10,2,0],["grok-2-vision-latest",8192,4096,"text,image",0,null,2,10,2,0],["grok-3",131072,8192,"text",0,null,3,15,0.75,0],["grok-3-fast",131072,8192,"text",0,null,5,25,1.25,0],["grok-3-fast-latest",131072,8192,"text",0,null,5,25,1.25,0],["grok-3-latest",131072,8192,"text",0,null,3,15,0.75,0],["grok-3-mini",131072,8192,"text",1,null,0.3,0.5,0.075,0],["grok-3-mini-fast",131072,8192,"text",1,null,0.6,4,0.15,0],["grok-3-mini-fast-latest",131072,8192,"text",1,null,0.6,4,0.15,0],["grok-3-mini-latest",131072,8192,"text",1,null,0.3,0.5,0.075,0],["grok-4",256000,64000,"text",1,null,3,15,0.75,0],["grok-4-1-fast",2000000,30000,"text,image",1,null,0.2,0.5,0.05,0],["grok-4-1-fast-non-reasoning",2000000,30000,"text,image",0,null,0.2,0.5,0.05,0],["grok-4-fast",2000000,30000,"text,image",1,null,0.2,0.5,0.05,0],["grok-4-fast-non-reasoning",2000000,30000,"text,image",0,null,0.2,0.5,0.05,0],["grok-4.20-0309-non-reasoning",1000000,30000,"text,image",0,null,1.25,2.5,0.2,0],["grok-4.20-0309-reasoning",1000000,30000,"text,image",1,null,1.25,2.5,0.2,0],["grok-4.20-beta-latest-non-reasoning",2000000,30000,"text,image",0,null,2,6,0.2,0],["grok-4.20-beta-latest-reasoning",2000000,30000,"text,image",1,null,2,6,0.2,0],["grok-4.20-multi-agent-0309",1000000,30000,"text,image",1,null,1.25,2.5,0.2,0],["grok-4.20-multi-agent-beta-latest",2000000,30000,"text,image",1,null,2,6,0.2,0],["grok-4.3",1000000,30000,"text,image",1,null,1.25,2.5,0.2,0],["grok-4.5",500000,500000,"text,image",1,null,2,6,0.3,0],["grok-4.6",500000,500000,"text,image",1,null,2,6,0.3,0],["grok-beta",131072,4096,"text",0,null,5,15,5,0],["grok-build-0.1",256000,256000,"text,image",1,null,1,2,0.2,0],["grok-code-fast-1",256000,10000,"text",1,null,0.2,1.5,0.02,0],["grok-composer-2.5-fast",200000,64000,"text",1,null,0,0,0,0],["grok-vision-beta",8192,4096,"text,image",0,null,5,15,5,0]], diff --git a/src/providers/registry.ts b/src/providers/registry.ts index e724b93533..1d69788ffc 100644 --- a/src/providers/registry.ts +++ b/src/providers/registry.ts @@ -722,10 +722,9 @@ const OPENCODE_FREE_DEEPSEEK_MODELS = ["deepseek-v4-flash-free"]; * `[404] No endpoints found that support image input` and `big-pickle` with the * exact deserialize error quoted in #1043. * - * `mimo-v2.5-free` and `longcat-2.0-free` ACCEPT images and are deliberately - * absent. Adding them would silently replace a working image with a caption, - * which is worse than the loud 400 this list exists to prevent — see the negative - * assertion in tests/providers/provider-registry-parity.test.ts. + * `mimo-v2.5-free` and `longcat-2.0-free` ACCEPT images. They remain absent + * from the blind list and are recorded separately as positive input-modality evidence, + * so capability-positive dispatch can forward images without relying on blacklist absence. * * Zen's roster is discovered live while this list is static, so it is a dated * exception list, not a capability model. Re-probe before extending it. @@ -739,6 +738,7 @@ const OPENCODE_ZEN_TEXT_ONLY_MODELS = [ "laguna-s-2.1-free", "deepseek-v4-flash-free", ]; +const OPENCODE_ZEN_IMAGE_MODELS = ["mimo-v2.5-free", "longcat-2.0-free"] as const; /* * DeepSeek's Codex ladder is low/high/max. With the V4 Pro GA release * (DeepSeek-V4-Pro-0813) the official thinking-mode table is IDENTICAL for both @@ -3209,6 +3209,7 @@ export const PROVIDER_REGISTRY: readonly ProviderRegistryEntry[] = [ }, modelInputModalities: { [DEEPSEEK_VISION_PREVIEW_MODEL]: ["text", "image"], + ...Object.fromEntries(OPENCODE_ZEN_IMAGE_MODELS.map(id => [id, ["text", "image"] as string[]])), }, noVisionModels: [...OPENCODE_ZEN_TEXT_ONLY_MODELS, ...DEEPSEEK_GATEWAY_THINKING_MODELS], // Same DeepSeek routes as the Go preset above, behind the same vendor, so they carry @@ -3250,6 +3251,7 @@ export const PROVIDER_REGISTRY: readonly ProviderRegistryEntry[] = [ }, modelInputModalities: { [DEEPSEEK_VISION_PREVIEW_MODEL]: ["text", "image"], + ...Object.fromEntries(OPENCODE_ZEN_IMAGE_MODELS.map(id => [id, ["text", "image"] as string[]])), }, // Same Zen roster behind the same base URL, so it carries the same measured // text-only list rather than only its DeepSeek member (#1043). diff --git a/src/server/chat-completions.ts b/src/server/chat-completions.ts index b50af22cb5..8326118ff8 100644 --- a/src/server/chat-completions.ts +++ b/src/server/chat-completions.ts @@ -165,7 +165,7 @@ async function handleChatCompletionsWithBudget( if (chatBody.tools !== undefined) parts.push(JSON.stringify(chatBody.tools)); logCtx.usageLogInputTokens = Math.max(1, estimateTokens(parts.join("\n"), requestedModel)); } - if (!effortRow && isNativeChatRouteEligible(route, chatBody)) chatNativeRoute = route; + if (!effortRow && isNativeChatRouteEligible(route, chatBody, config)) chatNativeRoute = route; } catch (err) { if (err instanceof UnknownRoutingPolicyError) { logCtx.requestedModel = requestedModel; diff --git a/src/server/chat-native.ts b/src/server/chat-native.ts index 8cc633665c..b644b922b1 100644 --- a/src/server/chat-native.ts +++ b/src/server/chat-native.ts @@ -19,7 +19,7 @@ import type { AdmissionLease } from "../lib/admission"; import { readBoundedResponseBody } from "../lib/bounded-body"; import { redactSecretString } from "../lib/redact"; import { resolveClientRetryAfter } from "../lib/retry-after"; -import { isModelTextOnly } from "../vision"; +import { isModelTextOnly, requiresVisionPreprocessing } from "../vision"; import { applyUpstreamRecoveryInit, fetchWithResetRetry, @@ -138,7 +138,7 @@ function isRec(value: unknown): value is Rec { return value !== null && typeof value === "object" && !Array.isArray(value); } -export function isNativeChatRouteEligible(route: RouteResult, rawBody: Rec): boolean { +export function isNativeChatRouteEligible(route: RouteResult, rawBody: Rec, config?: OcxConfig): boolean { const provider = route.provider; if (provider.adapter !== "openai-chat") return false; if (provider.authMode !== undefined && provider.authMode !== "key" && provider.authMode !== "local") return false; @@ -152,7 +152,12 @@ export function isNativeChatRouteEligible(route: RouteResult, rawBody: Rec): boo // site describes or strips the image. The native fast path has no vision // handling, so letting it keep such a request forwards raw pixels to a // model the operator declared blind. - if (isModelTextOnly(provider, route.modelId) && chatBodyCarriesImage(rawBody)) return false; + if (chatBodyCarriesImage(rawBody)) { + const needsVision = config + ? requiresVisionPreprocessing(config, provider, route.modelId, route.providerName) + : isModelTextOnly(provider, route.modelId); + if (needsVision) return false; + } if (Array.isArray(rawBody.tools)) { for (const tool of rawBody.tools) { if (!isRec(tool)) continue; @@ -326,7 +331,7 @@ export async function handleNativeChatCompletions(options: HandleNativeChatOptio dispatchOverride: async (_input, init, execute) => { if (!providerApiKeySelectionIsCurrent(config, route.providerName, activeProvider)) { const current = resolveCurrentProviderApiKeyTransport(config, route.providerName, activeProvider); - if (!current || !isNativeChatRouteEligible({ ...route, provider: current }, options.chatBody)) { + if (!current || !isNativeChatRouteEligible({ ...route, provider: current }, options.chatBody, config)) { throw new Error("Provider key selection is no longer available for native Chat"); } activeProvider = current; diff --git a/src/server/responses/core.ts b/src/server/responses/core.ts index 3fc8917c5d..568665f045 100644 --- a/src/server/responses/core.ts +++ b/src/server/responses/core.ts @@ -165,7 +165,7 @@ import { shouldResolveOpenAiPassthroughWebSearchBridge, } from "../../web-search/passthrough-bridge"; import { buildImageTool, buildVideoTool, planImageBridge, planVideoBridge, runWithImageBridge, clampImageMaxRounds, IMAGE_GEN_TOOL_NAME, VIDEO_GEN_TOOL_NAME } from "../../images"; -import { describeImagesInPlace, isModelTextOnly, planVisionSidecar, resolveOpenAiVisionModel, shouldResolveOpenAiVisionSidecar, stripImagesInPlace } from "../../vision"; +import { describeImagesInPlace, planVisionSidecar, requiresVisionPreprocessing, resolveOpenAiVisionModel, shouldResolveOpenAiVisionSidecar, stripImagesInPlace } from "../../vision"; import { createAdapterEventQueue, preflightAdapterEvents, type AdapterEventQueue } from "../../adapters/run-turn-queue"; import { applyCodexAuthContextToProvider, @@ -4769,7 +4769,7 @@ async function handleResponsesInner( const routedCompaction = parsed._compactionRequest === true && !isCanonicalOpenAiForwardProvider(route.provider); const needsOpenAiVision = !visionDescribeTerminal - && shouldResolveOpenAiVisionSidecar(config, route.provider, route.modelId, parsed); + && shouldResolveOpenAiVisionSidecar(config, route.provider, route.modelId, parsed, route.providerName); const needsOpenAiSearch = !routedCompaction && !adapter.runTurn && (shouldResolveOpenAiWebSearchSidecar(config, parsed, isPassthrough) || shouldResolveOpenAiPassthroughWebSearchBridge(route.provider, parsed, isPassthrough)); @@ -4839,7 +4839,7 @@ async function handleResponsesInner( const visionPlan = visionDescribeTerminal ? undefined : planVisionSidecar(config, route.provider, route.modelId, parsed, openAiSidecar, { - admission: options.admission, codexAuthPolicy: options.codexAuthPolicy, + admission: options.admission, codexAuthPolicy: options.codexAuthPolicy, providerName: route.providerName, }); const recordSidecarOutcome = openAiSidecar?.recordOutcome; if (visionPlan) { @@ -4851,9 +4851,9 @@ async function handleResponsesInner( recordSidecarOutcome, translatorBudget, ); - } else if (isModelTextOnly(route.provider, route.modelId)) { - // Sidecar-covered model but NO plan (no forward provider / missing forwarded auth / sidecar - // disabled): fail closed — never forward raw images to a text-only upstream. + } else if (requiresVisionPreprocessing(config, route.provider, route.modelId, route.providerName)) { + // Image capability is not positively proven but no sidecar plan is dispatchable: fail closed. + // Never forward raw image bytes to an unverified upstream. stripImagesInPlace(parsed, translatorBudget); } @@ -6291,7 +6291,7 @@ async function handleResponsesInner( providerApiKey: route.provider.apiKey ?? "", auth: webSearchBridgeAuth, hostedTool: parsed._webSearch, - describeImages: isModelTextOnly(route.provider, route.modelId), + describeImages: requiresVisionPreprocessing(config, route.provider, route.modelId, route.providerName), sidecar: config.webSearchSidecar, }), // Appending a search result can push the continuation past the ceiling the first leg @@ -6817,7 +6817,7 @@ async function handleResponsesInner( // can proceed for web-search-only turns const wsPlan = !routedCompaction ? planWebSearch(config, parsed, false, route.provider, route.modelId, openAiSidecar, { - admission: options.admission, codexAuthPolicy: options.codexAuthPolicy, + admission: options.admission, codexAuthPolicy: options.codexAuthPolicy, providerName: route.providerName, }) : undefined; const imgPlan = !routedCompaction ? await planImageBridge(config, parsed, route.provider) : undefined; diff --git a/src/vision/eligibility.ts b/src/vision/eligibility.ts index bec455fc7d..5bdd14024f 100644 --- a/src/vision/eligibility.ts +++ b/src/vision/eligibility.ts @@ -26,6 +26,7 @@ import { getModelMetadataCaseInsensitive, resolveMetadataProvider } from "../gen import { nativeInputModalities } from "../codex/catalog/metadata"; import { SUPPORTED_NATIVE_OPENAI_SLUGS } from "../codex/catalog/native-models"; import { enrichProviderFromRegistry } from "../providers/derive"; +import { isCanonicalOpenAiForwardProvider } from "../providers/openai-tiers-destination"; /** * The wire protocols `planVisionSidecar` can dispatch to (#2188 roadmap 170 @@ -113,7 +114,28 @@ function enrichedProviderForVision( if (cached) return cached; const configured = config.providers?.[providerName]; if (!configured) return undefined; - const enriched = structuredClone(configured); + // Runtime providers may carry non-cloneable hooks (for example a provider-scoped fetch). + // Enrichment mutates top-level fields but does not mutate nested provider values in place, so + // a shallow copy plus private copies of the vision-capability containers is sufficient and + // avoids both config mutation and structuredClone(DataCloneError) on runtime functions. + const enriched: OcxProviderConfig = { + ...configured, + ...(configured.noVisionModels ? { noVisionModels: [...configured.noVisionModels] } : {}), + ...(configured.modelInputModalities ? { + modelInputModalities: Object.fromEntries( + Object.entries(configured.modelInputModalities).map(([id, modalities]) => [id, [...modalities]]), + ), + } : {}), + ...(configured.modelCapabilities ? { + modelCapabilities: Object.fromEntries(Object.entries(configured.modelCapabilities).map(([id, capability]) => [ + id, + { + ...capability, + ...(capability.inputModalities ? { inputModalities: [...capability.inputModalities] } : {}), + }, + ])), + } : {}), + }; enrichProviderFromRegistry(providerName, enriched); cache.set(providerName, enriched); return enriched; @@ -166,6 +188,16 @@ function modelAcceptsImageInputWithCache( const declared = Object.hasOwn(provider?.modelCapabilities ?? {}, candidate.id) ? provider?.modelCapabilities?.[candidate.id]?.inputModalities : undefined; if (declared !== undefined) return declared.includes("image"); + const configuredModalities = provider ? modelRecordValue(provider.modelInputModalities, candidate.id) : undefined; + const fromConfiguredModalities = advertisesImageInput(configuredModalities); + if (fromConfiguredModalities !== undefined) return fromConfiguredModalities; + const canonicalCodex = candidate.provider === "openai" + && provider !== undefined + && isCanonicalOpenAiForwardProvider(provider); + if (canonicalCodex) { + const fromCodexBackend = metadataImageInput("openai-codex", candidate.id); + if (fromCodexBackend !== undefined) return fromCodexBackend; + } const fromRow = advertisesImageInput(candidate.inputModalities); if (fromRow !== undefined) return fromRow; return metadataImageInput(candidate.provider, candidate.id); diff --git a/src/vision/index.ts b/src/vision/index.ts index 633e2217c3..f955245ccc 100644 --- a/src/vision/index.ts +++ b/src/vision/index.ts @@ -35,6 +35,7 @@ export { resolveEffectiveVisionModel, shouldResolveOpenAiVisionSidecar, planVisionSidecar, + requiresVisionPreprocessing, } from "./plan"; export type { AnthropicVisionProvider, VisionPlan } from "./plan"; export { stripImagesInPlace } from "./image-rewrite"; diff --git a/src/vision/plan.ts b/src/vision/plan.ts index fb2aae6d49..8c77735daf 100644 --- a/src/vision/plan.ts +++ b/src/vision/plan.ts @@ -1,5 +1,5 @@ import type { OcxConfig, OcxContentPart, OcxParsedRequest, OcxProviderConfig } from "../types"; -import type { VisionReasoningEffort } from "../reasoning-effort"; +import { modelRecordValue, type VisionReasoningEffort } from "../reasoning-effort"; import type { VisionSettings } from "./describe"; import type { ResolvedOpenAiForwardSidecar } from "../providers/openai-sidecar"; import type { CodexAuthPolicyConfig } from "../codex/auth-context"; @@ -92,6 +92,32 @@ function messagesHaveImage(parsed: OcxParsedRequest): boolean { carriesImages(m.role) && Array.isArray(m.content) && (m.content as OcxContentPart[]).some(p => p.type === "image")); } +/** + * Fail-safe direct-image admission for a routed target. Raw image bytes may reach the target + * only when capability evidence positively proves image support. Explicit text-only config, + * registry/vendor metadata, and unknown capability all require the vision preprocessor. + * The provider-only fallback keeps legacy unit callers stable; production dispatch always + * supplies providerName so the complete capability chain is consulted. + */ +export function requiresVisionPreprocessing( + config: Pick, + provider: Pick, + modelId: string, + providerName?: string, +): boolean { + if (isModelTextOnly(provider, modelId)) return true; + const runtimeDeclared = Object.hasOwn(provider.modelCapabilities ?? {}, modelId) + ? provider.modelCapabilities?.[modelId]?.inputModalities + : undefined; + if (runtimeDeclared !== undefined) return !runtimeDeclared.includes("image"); + const runtimeModalities = modelRecordValue(provider.modelInputModalities, modelId); + if (Array.isArray(runtimeModalities) && runtimeModalities.length > 0) { + return !runtimeModalities.includes("image"); + } + if (!providerName) return false; + return modelAcceptsImageInput(config, { provider: providerName, id: modelId }) === false; +} + /** Shared by auth admission and planning so a routed describer never borrows OpenAI auth. */ function usableRoutedVisionModel(config: OcxConfig): string | undefined { const cfg = config.visionSidecar; @@ -109,10 +135,11 @@ function usableRoutedVisionModel(config: OcxConfig): string | undefined { export function shouldResolveOpenAiVisionSidecar( config: OcxConfig, provider: OcxProviderConfig, - modelId: string, - parsed: OcxParsedRequest, + modelId: string, + parsed: OcxParsedRequest, + providerName?: string, ): boolean { - if (!isModelTextOnly(provider, modelId) || !messagesHaveImage(parsed)) return false; + if (!requiresVisionPreprocessing(config, provider, modelId, providerName) || !messagesHaveImage(parsed)) return false; const cfg = config.visionSidecar ?? {}; if (cfg.enabled === false) return false; if (usableRoutedVisionModel(config)) return false; @@ -132,10 +159,12 @@ export interface VisionPlan { } /** - * Decide whether the vision sidecar should pre-describe images for this request, returning the plan - * if so. Active when: the routed model is in `provider.noVisionModels`, the request actually carries - * an image, the sidecar isn't disabled, and the selected backend has usable auth. Returns undefined - * otherwise (the caller strips images before sending to a text-only model). + * Decide whether the vision sidecar should pre-describe images for this request. Raw image + * delivery is capability-driven: targets proven text-only are preprocessed, targets proven + * image-capable bypass this planner, and genuinely unknown custom targets retain legacy behavior. + * The request must + * carry an image, the sidecar must be enabled, and the selected backend must be dispatchable. + * Returns undefined otherwise; the caller strips images before any unverified upstream send. */ export function planVisionSidecar( config: OcxConfig, @@ -143,9 +172,13 @@ export function planVisionSidecar( modelId: string, parsed: OcxParsedRequest, openAiSidecar?: ResolvedOpenAiForwardSidecar, - options: { admission?: Pick; codexAuthPolicy?: CodexAuthPolicyConfig } = {}, + options: { + admission?: Pick; + codexAuthPolicy?: CodexAuthPolicyConfig; + providerName?: string; + } = {}, ): VisionPlan | undefined { - if (!isModelTextOnly(provider, modelId)) return undefined; + if (!requiresVisionPreprocessing(config, provider, modelId, options.providerName)) return undefined; if (!messagesHaveImage(parsed)) return undefined; const cfg = config.visionSidecar ?? {}; if (cfg.enabled === false) return undefined; diff --git a/src/web-search/index.ts b/src/web-search/index.ts index e700e48a15..e85adc94dd 100644 --- a/src/web-search/index.ts +++ b/src/web-search/index.ts @@ -1,6 +1,6 @@ import type { OcxConfig, OcxParsedRequest, OcxProviderConfig } from "../types"; import { modelInList, toolChoiceToolPredicate } from "../types"; -import { isModelTextOnly } from "../vision"; +import { requiresVisionPreprocessing } from "../vision"; import type { SidecarSettings } from "./executor"; import type { CodexAuthPolicyConfig } from "../codex/auth-context"; import { isCodexReserveRequestEligible } from "../codex/loopback-target"; @@ -166,7 +166,11 @@ export function planWebSearch( provider: OcxProviderConfig, modelId: string, openAiSidecar?: ResolvedOpenAiForwardSidecar, - options: { admission?: Pick; codexAuthPolicy?: CodexAuthPolicyConfig } = {}, + options: { + admission?: Pick; + codexAuthPolicy?: CodexAuthPolicyConfig; + providerName?: string; + } = {}, ): SidecarPlan | undefined { if (!parsed._webSearch || isPassthrough) return undefined; if (!toolChoiceToolPredicate(parsed.options.toolChoice)(buildWebSearchTool())) return undefined; @@ -190,8 +194,9 @@ export function planWebSearch( routedModelStallTimeoutMs, timeoutMs, ); - // The routed model being text-only means the search model must verbalize image results (either backend). - const describeImages = isModelTextOnly(provider, modelId); + // Search-result images are never passed through to a target unless image capability is + // positively proven. Unknown or text-only targets receive verbalized image results instead. + const describeImages = requiresVisionPreprocessing(config, provider, modelId, options.providerName); const reasoning = cfg.reasoning ?? DEFAULT_SIDECAR_REASONING; const streamRoutedModelOutput = cfg.streamRoutedModelOutput === true; diff --git a/structure/adapters/registry.md b/structure/adapters/registry.md index 1952e3615f..2c29a1670c 100644 --- a/structure/adapters/registry.md +++ b/structure/adapters/registry.md @@ -141,3 +141,5 @@ medium/high/max UID before accepting a suffix already present in the model id. The merged `devin` provider uses this resolver for every account, whichever login path minted the credential. Omitted effort preserves an explicit variant; unrelated model families retain their existing suffix precedence. + +Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged. diff --git a/structure/data-planes/inbound-compat.md b/structure/data-planes/inbound-compat.md index 8b27dd5688..00d4050a04 100644 --- a/structure/data-planes/inbound-compat.md +++ b/structure/data-planes/inbound-compat.md @@ -247,3 +247,5 @@ Translated Chat request construction uses the [inline-image budget](../transport The [explicit model-capability contract](../config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior. Provider-scoped approval reviewer settings are projected by the [catalog owner](../catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior. + +Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged. diff --git a/structure/providers/chat-compat.md b/structure/providers/chat-compat.md index 2362e46bcd..493a8d411e 100644 --- a/structure/providers/chat-compat.md +++ b/structure/providers/chat-compat.md @@ -292,3 +292,5 @@ byte-limit boundaries. Canonical Spark Lite metadata follows the final serialized model and surviving nonempty Lite tool catalog; see [Responses transport](../transports/responses.md). Translated Chat request construction uses the [inline-image budget](../transports/streaming-health.md#translated-chat-inline-image-budget); the shared normalizer counts retained bytes even when a wire-specific drop callback keeps the image attached. + +Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged. diff --git a/structure/providers/cursor.md b/structure/providers/cursor.md index 55f017685b..356c6af64e 100644 --- a/structure/providers/cursor.md +++ b/structure/providers/cursor.md @@ -106,3 +106,5 @@ Combo child requests normalize effort and thinking controls against the selected `src/adapters/cursor.ts` surfaces the first bare context overflow before attempting conversation remint on later eligible requests. `cursorClientThreadOwner` recognizes both client thread aliases; `src/adapters/cursor/thread-continuity.ts` limits recovery to three remints per retained identity-scoped owner, with a one-hour idle TTL and 2,048-entry bound. Conversation-only requests have no stable owner and do not automatically remint. Quota/rate errors, tool-result resumes, partial output, local side effects, isolated helper/shadow requests and compaction remain fail-closed. Isolated requests neither consume the parent allowance nor invalidate its checkpoint. Eligible overflow checks refresh existing retention timestamps and LRU position even after the cap is exhausted, without allocating absent scopes. Retention expiry, eviction or process restart resets the in-memory allowance; this is not a persistent lifetime cap or semantic-progress policy. Translated Chat request construction uses the [inline-image budget](../transports/streaming-health.md#translated-chat-inline-image-budget); the shared normalizer counts retained bytes even when a wire-specific drop callback keeps the image attached. + +Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged. diff --git a/structure/runtime.md b/structure/runtime.md index a09d007d7b..90b890c56b 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -343,10 +343,28 @@ Translated Chat request construction uses the [inline-image budget](transports/s OpenCode catalog discovery in `src/cli/opencode.ts` uses the local admin credential and a validated numeric-loopback management origin. It dials through `src/server/direct-local-http.ts`, rejects redirects and preserves the request/body deadline. Hub ingress selection stays separate from exported inference settings. -The [explicit model-capability contract](config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior. +The [explicit model-capability contract](config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture. Vision dispatch consumes those declarations together with registry/vendor metadata before any image-bearing upstream send. -Exact [model input declarations](config.md#explicit-per-model-capability-declarations) now feed text-only eligibility and catalog hints; existing image-description/omission handling consumes them before the main upstream send. +## Capability-aware image admission + +`src/vision/plan.ts` prevents raw image bytes from reaching any target whose effective capability is positively known to exclude image input. Evidence from the resolved runtime provider and explicit operator declarations takes precedence, followed by backend-specific/registry/vendor metadata. A proven text-only target is preprocessed through the configured Vision Sidecar; a positively image-capable target receives the image directly. Genuinely unknown custom models retain the existing compatibility path rather than being guessed text-only. + +Canonical ChatGPT Codex forwarding uses the generated `openai-codex` capability bundle rather than the public `openai` bundle. This matters when the two backends differ: for example, the vendored metadata records `gpt-5.3-codex-spark` as text-only on `openai-codex` while the public OpenAI row lists image input. The native Chat fast path and web-search image verbalization consume the same effective-capability decision. + +An explicitly configured routed `visionSidecar.model` is dispatchable unless capability evidence positively proves it cannot accept images; an unknown custom sidecar is not guessed blind. If a proven text-only main target has no usable sidecar plan, image parts are stripped before the upstream request rather than forwarded raw. `modelInputModalities` is symmetric evidence: `["text","image"]` proves image support while `["text"]` triggers preprocessing. Runtime provider hooks such as injected `fetch` functions are preserved without mutation during capability enrichment. + +Regression coverage: `tests/vision/vision-cache.test.ts`, `tests/vision/vision-eligibility.test.ts`, `tests/vision/vision-routed.test.ts`, and `tests/adapters/openai/openai-chat-native-policy.test.ts`. Provider-scoped approval reviewer settings are projected by the [catalog owner](catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior. Renamed fixed-key providers receive [missing reasoning metadata](catalog.md#renamed-destination-reasoning-metadata) during derivation; explicit per-model entries and provider defaults retain precedence. + +## Request-local target compatibility + +`src/adapters/openai-responses.ts` omits only top-level `user` at the canonical ChatGPT Codex forward destination. Claude translation retains its original identity and prompt-cache key; public API and noncanonical gateways retain their `user` field. Input roles, tool-schema properties, safety identifiers and original replay bodies are not changed. + +`src/combos/failover.ts` treats three intact HTTP 400 invalid-request envelopes as request-local incompatibilities: exactly `Unsupported parameter: user`; `unsupported_value` naming `reasoning.effort` or `reasoning_effort` with an explicit unsupported-value message; and `param: input` with a bounded model-scoped `does not support image inputs` message. A null provider code is accepted only for that observed image envelope. Only the exact proxy wrapper is unwrapped, within three envelopes and 16,384 characters; conflicting codes, malformed/truncated envelopes and reflected JSON do not gain hop permission. + +The combo may advance to its next eligible unattempted target before output commitment. It records no target/provider cooldown for these request-local mismatches and does not silently drop reasoning controls or raise `none` to a supported rung. Cancellation, origin/cyber-policy rejection, non-replayable post-send errors and the existing streaming commit boundary stay authoritative. Other invalid requests remain terminal. + +Regression coverage: `tests/responses/responses-forward-prompt-envelope.test.ts`, `tests/routing/router-combo-failover-classification.test.ts`, and `tests/server/server-combo-failover-e2e.test.ts`. diff --git a/structure/transports/byte-accounting.md b/structure/transports/byte-accounting.md index 830d1af7a9..275fb016a7 100644 --- a/structure/transports/byte-accounting.md +++ b/structure/transports/byte-accounting.md @@ -33,3 +33,5 @@ plus exact separators, without joining a second full JSON array. `src/lib/admiss truncates diagnostic text at UTF-8 code-point boundaries without allocating arrays per character; byte sizing retains TextEncoder's coercion behavior for legacy non-string runtime callers. These optimizations do not add request queues, retry policies, or RSS-based admission gates. + +Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged. diff --git a/structure/transports/inventory.md b/structure/transports/inventory.md index 10b5be6a6b..ebd182ec67 100644 --- a/structure/transports/inventory.md +++ b/structure/transports/inventory.md @@ -142,3 +142,5 @@ The [explicit model-capability contract](../config.md#explicit-per-model-capabil Provider-scoped approval reviewer settings are projected by the [catalog owner](../catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior. Renamed fixed-key providers receive [missing reasoning metadata](../catalog.md#renamed-destination-reasoning-metadata) during derivation; explicit per-model entries and provider defaults retain precedence. + +Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged. diff --git a/structure/transports/responses.md b/structure/transports/responses.md index 1f1bd3cd2f..c561171062 100644 --- a/structure/transports/responses.md +++ b/structure/transports/responses.md @@ -1,8 +1,6 @@ # Responses Transport - The configuration-only [plaintext V2 contract](../subagents.md#plaintext-v2-agent-messages) is scoped to canonical ChatGPT Responses forwarding; other source-area behavior described here is unchanged. - Plaintext collaboration restoration treats a null namespace as absent, rejects non-string namespace types, and restores the native namespace/name pair before HTTP/WS delivery and continuation publication. ## Responses HTTP/SSE @@ -536,7 +534,7 @@ combo whose remaining eligible targets use other providers. capability ladders remove effort and thinking controls in every combo mode; adaptive mode also removes those controls for unknown ladders and preserves `reasoning.summary`. Known non-empty ladders retain the existing per-target effort resolution. This request normalization does not -change target order, attempt accounting, or the existing provider-400 failover classification. +change target order or attempt accounting; provider-400 decisions follow the request-local target compatibility contract below. The shared Responses path follows the [bounded multipart recovery contract](../subagents.md#multipart-encrypted-task-recovery); credential admission and retry policy remain unchanged. @@ -598,3 +596,5 @@ Translated Chat request construction uses the [inline-image budget](streaming-he The [explicit model-capability contract](../config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior. Provider-scoped approval reviewer settings are projected by the [catalog owner](../catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior. + +Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged. diff --git a/tests/adapters/openai/openai-chat-native-policy.test.ts b/tests/adapters/openai/openai-chat-native-policy.test.ts index ac6fb80b12..9357cf2215 100644 --- a/tests/adapters/openai/openai-chat-native-policy.test.ts +++ b/tests/adapters/openai/openai-chat-native-policy.test.ts @@ -398,7 +398,7 @@ describe("main and native Chat tier authorization parity", () => { } }); - test("the native lane keeps caller image bytes instead of normalizing them", async () => { + test("the native lane keeps caller image bytes only for positively vision-capable models", async () => { // Scope boundary for the openai-chat inline image budget (see // tests/adapters/openai/openai-chat-image-normalization.test.ts). That budget lives in // the adapter's buildRequest, but an eligible Chat-inbound request is dispatched down @@ -417,7 +417,9 @@ describe("main and native Chat tier authorization parity", () => { }); }) as typeof fetch; - const target = provider(); + const target = provider({ + modelCapabilities: { [MODEL_ID]: { inputModalities: ["text", "image"] } }, + }); const response = await handleChatCompletions( new Request("http://localhost/v1/chat/completions", { method: "POST", @@ -455,6 +457,17 @@ test("explicit text-only capabilities divert image-bearing native Chat requests" const { routeModel } = await import("../../../src/router"); const config = { port: 10100, defaultProvider: "custom", providers: { custom: provider({ modelCapabilities: { model: { inputModalities: ["text"] } } }) } } as OcxConfig; const route = routeModel(config, "custom/model"); - expect(isNativeChatRouteEligible(route, { messages: [{ role: "user", content: [{ type: "image_url", image_url: { url: "data:image/png;base64,YQ==" } }] }] })).toBe(false); - expect(isNativeChatRouteEligible(route, { messages: [{ role: "user", content: "hello" }] })).toBe(true); + expect(isNativeChatRouteEligible(route, { messages: [{ role: "user", content: [{ type: "image_url", image_url: { url: "data:image/png;base64,YQ==" } }] }] }, config)).toBe(false); + expect(isNativeChatRouteEligible(route, { messages: [{ role: "user", content: "hello" }] }, config)).toBe(true); +}); + +test("unknown image capability retains native Chat compatibility until capability is known", async () => { + const { isNativeChatRouteEligible } = await import("../../../src/server/chat-native"); + const { routeModel } = await import("../../../src/router"); + const config = { port: 10100, defaultProvider: "custom", providers: { custom: provider() } } as OcxConfig; + const route = routeModel(config, "custom/model"); + const imageBody = { messages: [{ role: "user", content: [{ + type: "image_url", image_url: { url: "data:image/png;base64,YQ==" }, + }] }] }; + expect(isNativeChatRouteEligible(route, imageBody, config)).toBe(true); }); diff --git a/tests/codex-integration/bearer-admission-routed-provider.test.ts b/tests/codex-integration/bearer-admission-routed-provider.test.ts index 277f790bd5..f8aafdfec7 100644 --- a/tests/codex-integration/bearer-admission-routed-provider.test.ts +++ b/tests/codex-integration/bearer-admission-routed-provider.test.ts @@ -461,7 +461,11 @@ describe("bearer admission is not reused as a Cursor upstream credential", () => if (scenario === "default-sidecars") delete config.visionSidecar; if (scenario === "disabled-vision") config.visionSidecar.enabled = false; if (scenario === "routed-vision") { - config.providers.gateway = { ...mixedConfig().providers.gateway!, models: ["vision-model"] }; + config.providers.gateway = { + ...mixedConfig().providers.gateway!, + models: ["vision-model"], + modelInputModalities: { "vision-model": ["text", "image"] }, + }; config.visionSidecar = { enabled: true, backend: "routed", model: "gateway/vision-model" }; } if (scenario === "anthropic-vision") config.visionSidecar.backend = "anthropic"; diff --git a/tests/responses/responses-compaction-routing.test.ts b/tests/responses/responses-compaction-routing.test.ts index bef18aec4f..c612aed502 100644 --- a/tests/responses/responses-compaction-routing.test.ts +++ b/tests/responses/responses-compaction-routing.test.ts @@ -2282,7 +2282,10 @@ describe("computer screenshot output translation boundary", () => { test("rejects before an otherwise active vision description", async () => { const config = keyProviderConfig({ adapter: "openai-chat", noVisionModels: ["model"] }); config.visionSidecar = { enabled: true, backend: "routed", model: "vision/seeing" }; - config.providers.vision = { adapter: "openai-chat", baseUrl: "https://vision.example/v1", apiKey: "test-key" }; + config.providers.vision = { + adapter: "openai-chat", baseUrl: "https://vision.example/v1", apiKey: "test-key", + modelInputModalities: { seeing: ["text", "image"] }, + }; // Routed vision needs no live OpenAI account for this controlled description dependency. const resolveAuth = spyOn(visionModule, "shouldResolveOpenAiVisionSidecar").mockReturnValue(false); const describe = spyOn(visionModule, "describeImagesInPlace").mockImplementation(async () => {}); diff --git a/tests/responses/responses-forward-prompt-envelope.test.ts b/tests/responses/responses-forward-prompt-envelope.test.ts index 6f0b078818..9ca3578971 100644 --- a/tests/responses/responses-forward-prompt-envelope.test.ts +++ b/tests/responses/responses-forward-prompt-envelope.test.ts @@ -1,4 +1,5 @@ import { describe, expect, test } from "bun:test"; +import { anthropicToResponsesBody } from "../../src/claude/inbound"; import { createResponsesPassthroughAdapter as createProductionAdapter } from "../../src/adapters/openai-responses"; import type { OcxProviderConfig } from "../../src/types"; import { withTestTranslatorBudget } from "../helpers/translator-budget"; @@ -140,3 +141,56 @@ describe("canonical ChatGPT forward prompt envelope", () => { expect(body.input).toEqual(input); }); }); + + +describe("canonical forward user metadata boundary", () => { + test.each(["gpt-5.3-codex-spark", "gpt-5.6-luna"])("strips only top-level user for %s", model => { + const raw = { + model, user: "synthetic-client", prompt_cache_key: "synthetic-cache", + safety_identifier: "synthetic-safety", stream: true, + input: [{ type: "message", role: "user", content: "hello" }], + tools: [{ type: "function", name: "lookup", parameters: { + type: "object", properties: { user: { type: "string" } }, required: ["user"], + } }], + }; + const snapshot = structuredClone(raw); + const body = outboundBody(canonicalForward, raw); + expect(Object.hasOwn(body, "user")).toBe(false); + expect(body.prompt_cache_key).toBe(raw.prompt_cache_key); + expect(body.safety_identifier).toBe(raw.safety_identifier); + expect(body.input).toEqual(raw.input); + expect(body.tools).toEqual(raw.tools); + expect(raw).toEqual(snapshot); + }); + test("Claude translation retains session metadata locally but omits user on the native wire", () => { + const raw = anthropicToResponsesBody({ + model: "gpt-5.3-codex-spark", max_tokens: 32, + metadata: { user_id: "synthetic-claude-session" }, + messages: [{ role: "user", content: "hello" }], + }); + expect(raw.user).toBe("synthetic-claude-session"); + const snapshot = structuredClone(raw); + const body = outboundBody(canonicalForward, raw); + expect(Object.hasOwn(body, "user")).toBe(false); + expect(body.prompt_cache_key).toBe(raw.prompt_cache_key); + expect(String(body.prompt_cache_key)).toMatch(/^[a-f0-9]{32}$/); + expect(raw).toEqual(snapshot); + }); + test.each([ + { ...canonicalForward, baseUrl: "https://api.openai.com/v1", authMode: "key" as const }, + { ...canonicalForward, baseUrl: "https://gateway.example/v1" }, + { ...canonicalForward, baseUrl: "https://chatgpt.com.example/backend-api/codex" }, + { ...canonicalForward, authMode: "key" as const }, + ])("does not strip user from a different destination: %j", provider => { + const body = outboundBody(provider, { model: "model", user: "synthetic-client", input: "hello" }); + expect(body.user).toBe("synthetic-client"); + }); + test("normalizes the canonical trailing slash without changing field-absent requests", () => { + const provider = { ...canonicalForward, baseUrl: canonicalForward.baseUrl + "/" }; + const raw = { model: "gpt-5.3-codex-spark", user: "synthetic-client", input: "hello" }; + expect(Object.hasOwn(outboundBody(provider, raw), "user")).toBe(false); + const without = { model: raw.model, input: raw.input }; + expect(Object.hasOwn(outboundBody(provider, without), "user")).toBe(false); + expect(without).toEqual({ model: raw.model, input: raw.input }); + }); +}); diff --git a/tests/routing/router-combo-failover-classification.test.ts b/tests/routing/router-combo-failover-classification.test.ts index 93b58170a6..cb5d765459 100644 --- a/tests/routing/router-combo-failover-classification.test.ts +++ b/tests/routing/router-combo-failover-classification.test.ts @@ -203,3 +203,98 @@ describe("malformed upstream bytes are a provider failure", () => { }); }); }); + +const unsupportedUser = { type: "invalid_request_error", message: "Unsupported parameter: user" }; +const unsupportedEffort = { + type: "invalid_request_error", code: "unsupported_value", param: "reasoning.effort", + message: "Unsupported value: 'none' is not supported with the 'gpt-5.3-codex-spark' model. Supported values are: 'low', 'medium', 'high', and 'xhigh'.", +}; + +const unsupportedImage = { + type: "invalid_request_error", code: null, param: "input", + message: "Model 'gpt-5.3-codex-spark' does not support image inputs. Try again with a vision model.", +}; + +describe("request-local optional control incompatibility", () => { + test.each([unsupportedUser, unsupportedEffort, unsupportedImage])("hops a structured target-local request rejection without cooling: %j", error => { + const body = JSON.stringify({ error }); + for (const message of [body, `Provider error 400: ${body}`]) { + expect(comboFailureDecision(400, message, { code: "invalid_request_error" })).toBe("hop"); + expect(comboFailureCooldownScope(400, message, { code: "invalid_request_error" })).toBe("none"); + } + }); + test.each([ + { type: "invalid_request_error", message: "Unsupported parameter: tools" }, + { type: "invalid_request_error", message: "Unsupported parameter: safety_identifier" }, + { type: "invalid_request_error", message: "Unsupported parameter: user.name" }, + { ...unsupportedUser, param: "input" }, + { ...unsupportedUser, code: "origin_rejected" }, + { ...unsupportedUser, code: "context_length_exceeded" }, + { ...unsupportedUser, code: "unknown_terminal_code" }, + { ...unsupportedEffort, param: "input" }, + { ...unsupportedEffort, code: "unknown_terminal_code" }, + { ...unsupportedEffort, code: "cyber_policy" }, + ])("does not relax an unrelated or conflicting refusal: %j", error => { + expect(comboFailureDecision(400, JSON.stringify({ error }))).toBe("stop"); + }); + test("reflected text, truncated envelopes and oversized diagnostics stay terminal", () => { + const body = JSON.stringify({ error: unsupportedUser }); + for (const text of [ + `invalid input contains ${body}`, + JSON.stringify({ error: { type: "invalid_request_error", message: body } }), + body.slice(0, -1), + JSON.stringify({ error: unsupportedUser, padding: "x".repeat(16_384) }), + ]) expect(comboFailureDecision(400, text)).toBe("stop"); + }); + test("hard refusal and non-replayable codes take precedence over a compatible message", () => { + const message = JSON.stringify({ error: unsupportedUser }); + for (const code of ["origin_rejected", "context_length_exceeded", "upstream_no_response", "upstream_closed_before_response"]) { + expect(comboFailureDecision(400, message, { code })).toBe("stop"); + } + expect(comboFailureDecision(499, message)).toBe("stop"); + expect(comboFailureDecision(413, message)).toBe("stop"); + }); +}); + +describe("bounded optional-control error envelopes", () => { + const wrapped = (message: string, code = "invalid_request_error") => JSON.stringify({ + error: { type: "invalid_request_error", code, message: `Provider error 400: ${message}` }, + }); + test("accepts the proxy wrapper but not an unrelated message containing JSON", () => { + const raw = JSON.stringify({ error: unsupportedUser }); + expect(comboFailureDecision(400, wrapped(raw))).toBe("hop"); + expect(comboFailureDecision(400, wrapped(wrapped(raw)))).toBe("hop"); + expect(comboFailureDecision(400, wrapped(wrapped(wrapped(raw))))).toBe("stop"); + expect(comboFailureDecision(400, wrapped(raw, "cyber_policy"))).toBe("stop"); + expect(comboFailureDecision(400, wrapped(raw, "context_length_exceeded"))).toBe("stop"); + }); + test("malformed envelope fields do not throw or acquire hop permission", () => { + for (const value of [null, [], "user", { error: null }, { error: [] }, + { error: { ...unsupportedUser, code: {} } }, + { error: { ...unsupportedUser, param: null } }, + { error: { ...unsupportedUser, type: "custom_failure" } }, + ]) expect(comboFailureDecision(400, JSON.stringify(value))).toBe("stop"); + }); + test("the precise reasoning code works without a proxy-generated generic code", () => { + const message = JSON.stringify({ error: unsupportedEffort }); + expect(comboFailureDecision(400, message, { code: "unsupported_value" })).toBe("hop"); + expect(comboFailureCooldownScope(400, message, { code: "unsupported_value" })).toBe("none"); + }); +}); + +describe("image rejection classifier bounds", () => { + test.each([ + { ...unsupportedImage, param: "tools" }, + { ...unsupportedImage, code: "origin_rejected" }, + { ...unsupportedImage, code: "unknown_terminal_code" }, + { ...unsupportedImage, message: "This model does not support image inputs." }, + { ...unsupportedImage, message: "Model 'x' does not support image inputs" }, + ])("does not hop on a lookalike or conflicting image refusal: %j", error => { + expect(comboFailureDecision(400, JSON.stringify({ error }))).toBe("stop"); + }); + test("accepts the observed null-code envelope with an outer generic code", () => { + const message = JSON.stringify({ error: unsupportedImage }); + expect(comboFailureDecision(400, message, { code: "invalid_request_error" })).toBe("hop"); + expect(comboFailureCooldownScope(400, message, { code: "invalid_request_error" })).toBe("none"); + }); +}); diff --git a/tests/server/server-combo-failover-e2e.test.ts b/tests/server/server-combo-failover-e2e.test.ts index 59400ff921..041ed847c0 100644 --- a/tests/server/server-combo-failover-e2e.test.ts +++ b/tests/server/server-combo-failover-e2e.test.ts @@ -3027,6 +3027,7 @@ describe("server combo failover 030 activation matrix", () => { }), b: provider("openai-chat", baseUrl(b), "key-b", { reasoningEfforts: ["low", "high"], + modelInputModalities: { m2: ["text", "image"] }, }), }, undefined, { defaultEffort: "high" }); const response = await post(config, { @@ -4083,3 +4084,83 @@ describe("thinking-summary defaults follow the serving combo route", () => { }); } }); + +describe("optional-control rejection failover regression", () => { + for (const stream of [false, true]) { + test.each(["user", "reasoning.effort"])(`429 -> optional %s 400 -> 200, stream=${stream}`, async parameter => { + const hits: string[] = []; + const first = serve(() => { + hits.push("quota"); + return Response.json({ error: { type: "rate_limit_error", message: "Rate limit exceeded" } }, { status: 429 }); + }); + const incompatible = serve(async request => { + hits.push("incompatible"); + const raw = await request.json() as Record; + expect(raw.user).toBe("synthetic-client"); + const error = parameter === "user" + ? { type: "invalid_request_error", message: "Unsupported parameter: user" } + : { type: "invalid_request_error", code: "unsupported_value", param: "reasoning.effort", + message: "Unsupported value: 'none' is not supported with this model. Supported values are: 'low', 'medium', 'high', and 'xhigh'." }; + return Response.json({ error }, { status: 400 }); + }); + const backup = serve(() => { + hits.push("backup"); + return stream ? chatStream("recovered optional control") : chatSuccess("recovered optional control", "m3"); + }); + const config = comboConfig({ + a: provider("openai-chat", baseUrl(first), "key-a"), + b: provider("openai-responses", baseUrl(incompatible), "key-b"), + c: provider("openai-chat", baseUrl(backup), "key-c"), + }); + const response = await post(config, { + stream, user: "synthetic-client", prompt_cache_key: "synthetic-cache", + reasoning: { effort: "none" }, + }); + const text = await response.text(); + expect(response.status).toBe(200); + expect(text).toContain("recovered optional control"); + expect(text).not.toContain("Unsupported parameter"); + expect(text).not.toContain("Unsupported value"); + expect(hits).toEqual(["quota", "incompatible", "backup"]); + expect(isComboTargetInCooldown("free", { provider: "b", model: "m2" })).toBe(false); + }); + } +}); + +describe("image-capability rejection failover regression", () => { + test("429 -> model-scoped image 400 -> healthy target does not abort the turn", async () => { + const hits: string[] = []; + const quota = serve(() => { + hits.push("quota"); + return Response.json({ error: { type: "rate_limit_error", message: "Rate limit exceeded" } }, { status: 429 }); + }); + const textOnly = serve(() => { + hits.push("text-only"); + return Response.json({ error: { + type: "invalid_request_error", code: null, param: "input", + message: "Model 'gpt-5.3-codex-spark' does not support image inputs. Try again with a vision model.", + } }, { status: 400 }); + }); + const vision = serve(() => { + hits.push("vision"); + return chatSuccess("vision fallback recovered", "m3"); + }); + const config = comboConfig({ + a: provider("openai-chat", baseUrl(quota), "key-a"), + b: provider("openai-responses", baseUrl(textOnly), "key-b"), + c: provider("openai-chat", baseUrl(vision), "key-c"), + }); + const response = await post(config, { + input: [{ type: "message", role: "user", content: [ + { type: "input_text", text: "inspect this" }, + { type: "input_image", image_url: "data:image/png;base64,AA==" }, + ] }], + }); + const text = await response.text(); + expect(response.status).toBe(200); + expect(text).toContain("vision fallback recovered"); + expect(text).not.toContain("does not support image inputs"); + expect(hits).toEqual(["quota", "text-only", "vision"]); + expect(isComboTargetInCooldown("free", { provider: "b", model: "m2" })).toBe(false); + }); +}); diff --git a/tests/vision/vision-cache.test.ts b/tests/vision/vision-cache.test.ts index 6e482d4255..d62610115c 100644 --- a/tests/vision/vision-cache.test.ts +++ b/tests/vision/vision-cache.test.ts @@ -18,6 +18,7 @@ import { setVisionDescriptionCacheLimitsForTests, shouldResolveOpenAiVisionSidecar, planVisionSidecar, + requiresVisionPreprocessing, visionDescriptionRetainedStoreSnapshot, type VisionPlan, } from "../../src/vision"; @@ -43,6 +44,50 @@ const textOnlyProvider: OcxProviderConfig = { noVisionModels: ["text-model"], }; +test("direct image admission preprocesses proven-negative capability without guessing unknown custom models", () => { + const config = { + port: 10100, defaultProvider: "custom", providers: { + custom: { adapter: "openai-chat", baseUrl: "https://custom.test/v1" }, + declared: { + adapter: "openai-chat", baseUrl: "https://declared.test/v1", + modelInputModalities: { vision: ["text", "image"] }, + }, + openrouter: { adapter: "openai-chat", baseUrl: "https://openrouter.ai/api/v1" }, + }, + } as OcxConfig; + expect(requiresVisionPreprocessing(config, config.providers.custom!, "unknown-model", "custom")).toBe(false); + expect(requiresVisionPreprocessing(config, config.providers.declared!, "vision", "declared")).toBe(false); + const runtimePositive = { + ...config.providers.custom!, + modelCapabilities: { runtimeVision: { inputModalities: ["text", "image"] } }, + }; + expect(requiresVisionPreprocessing(config, runtimePositive, "runtimeVision", "custom")).toBe(false); + expect(requiresVisionPreprocessing( + config, config.providers.openrouter!, "openai/gpt-5.4-mini", "openrouter", + )).toBe(true); +}); + +test("routed vision sidecar rejects proven-blind models without guessing unknown configured models", () => { + const main: OcxProviderConfig = { + adapter: "openai-chat", baseUrl: "https://main.test/v1", noVisionModels: ["blind"], + }; + const helper: OcxProviderConfig = { adapter: "openai-chat", baseUrl: "https://helper.test/v1" }; + const request = parseRequest({ + model: "main/blind", + input: [{ type: "message", role: "user", content: [{ type: "input_image", image_url: DATA_A }] }], + }); + const config = { + port: 10100, defaultProvider: "main", providers: { main, helper }, + visionSidecar: { enabled: true, backend: "routed", model: "helper/unknown" }, + } as OcxConfig; + expect(planVisionSidecar(config, main, "blind", request, undefined, { providerName: "main" })?.backend).toBe("routed"); + config.providers.helper!.modelInputModalities = { blind: ["text"], vision: ["text", "image"] }; + config.visionSidecar!.model = "helper/blind"; + expect(planVisionSidecar(config, main, "blind", request, undefined, { providerName: "main" })).toBeUndefined(); + config.visionSidecar!.model = "helper/vision"; + expect(planVisionSidecar(config, main, "blind", request, undefined, { providerName: "main" })?.backend).toBe("routed"); +}); + function plan(overrides: Partial = {}): VisionPlan { return { backend: "openai", @@ -78,7 +123,10 @@ test("vision sidecar auth stays lazy for no-image and disabled branches", () => test("vision auth and planning agree on a routed describer and its legacy fallback", () => { const cfg: OcxConfig = { port: 10100, defaultProvider: "routed", providers: { - routed: textOnlyProvider, sighted: { adapter: "openai-chat", baseUrl: "https://vision.test/v1", apiKey: "vision-key" }, + routed: textOnlyProvider, sighted: { + adapter: "openai-chat", baseUrl: "https://vision.test/v1", apiKey: "vision-key", + modelInputModalities: { "vision-model": ["text", "image"] }, + }, }, visionSidecar: { enabled: true, backend: "routed", model: "sighted/vision-model" } }; const request = parseRequest({ model: "routed/text-model", input: [{ type: "message", role: "user", content: [{ type: "input_image", image_url: DATA_A }] }], diff --git a/tests/vision/vision-eligibility.test.ts b/tests/vision/vision-eligibility.test.ts index fc43a0d2f3..cf2514e900 100644 --- a/tests/vision/vision-eligibility.test.ts +++ b/tests/vision/vision-eligibility.test.ts @@ -158,6 +158,37 @@ describe("vision eligibility core", () => { })).toBe(false); }); + test("11b. canonical Codex backend metadata overrides a generic image-capable Spark row", () => { + const config = configWithProviders({ + openai: { + adapter: "openai-responses", + authMode: "forward", + baseUrl: "https://chatgpt.com/backend-api/codex", + }, + }); + const candidate: VisionCandidateModel = { + provider: "openai", + id: "gpt-5.3-codex-spark", + inputModalities: ["text", "image"], + }; + // No model-specific blacklist is required: backend metadata owns the verdict. + expect(isVisionSidecarConsumer(config, "openai", candidate.id)).toBe(false); + expect(modelAcceptsImageInput(config, candidate)).toBe(false); + expect(isVisionEligibleModel(config, candidate)).toBe(false); + }); + + test("11c. runtime provider hooks do not make capability enrichment uncloneable", () => { + const provider = { + adapter: "openai-chat", + baseUrl: "https://example.test/v1", + fetch: (() => Promise.reject(new Error("not called"))) as typeof fetch, + modelInputModalities: { vision: ["text", "image"] }, + } as OcxProviderConfig & { fetch: typeof fetch }; + const config = configWithProviders({ runtime: provider }); + expect(() => modelAcceptsImageInput(config, { provider: "runtime", id: "vision" })).not.toThrow(); + expect(modelAcceptsImageInput(config, { provider: "runtime", id: "vision" })).toBe(true); + }); + test("12. only the selected Anthropic OAuth provider contributes Anthropic options", () => { const config = configWithProviders({ anthropic: { diff --git a/tests/vision/vision-routed.test.ts b/tests/vision/vision-routed.test.ts index c1182a0953..3673793789 100644 --- a/tests/vision/vision-routed.test.ts +++ b/tests/vision/vision-routed.test.ts @@ -147,7 +147,10 @@ describe("describeImageRouted unit", () => { apiKey: "routed-key", noVisionModels: ["text-model"], }; - const vlm: OcxProviderConfig = { adapter: "openai-chat", baseUrl: "https://vlm.test/v1", apiKey: "k" }; + const vlm: OcxProviderConfig = { + adapter: "openai-chat", baseUrl: "https://vlm.test/v1", apiKey: "k", + modelInputModalities: { "qwen-vl": ["text", "image"] }, + }; const request = parseRequest({ model: "routed/text-model", input: [{ @@ -292,7 +295,7 @@ describe("chat-surface recursion fence (full path)", () => { } }); - test("routed describer end-to-end: image described via loopback before the text-only main call", async () => { + test("routed describer end-to-end: declared text-only main target receives only the caption", async () => { const mainBodies: string[] = []; const describerBodies: string[] = []; upstream = Bun.serve({ @@ -332,7 +335,7 @@ describe("chat-surface recursion fence (full path)", () => { baseUrl: `http://127.0.0.1:${upstream.port}/v1`, allowPrivateNetwork: true, apiKey: "k", - noVisionModels: ["text-only"], + modelInputModalities: { "text-only": ["text"] }, }, vision: { adapter: "openai-chat", diff --git a/tests/vision/vision-sidecar-e2e.test.ts b/tests/vision/vision-sidecar-e2e.test.ts index 29c06c4fe9..0e8b78dc71 100644 --- a/tests/vision/vision-sidecar-e2e.test.ts +++ b/tests/vision/vision-sidecar-e2e.test.ts @@ -377,7 +377,7 @@ describe("vision sidecar fallback (issue #88, end-to-end)", () => { } }); - test("models outside noVisionModels keep their image untouched (no sidecar call)", async () => { + test("positively vision-capable models keep their image untouched (no sidecar call)", async () => { let upstreamBody = ""; let sidecarHits = 0; upstream = serveUpstream(b => { upstreamBody = b; }); @@ -392,6 +392,7 @@ describe("vision sidecar fallback (issue #88, end-to-end)", () => { allowPrivateNetwork: true, apiKey: "key-alpha-000111222333", noVisionModels: ["blind-model"], + modelInputModalities: { "vision-model": ["text", "image"] }, }, openai: { adapter: "openai-responses", authMode: "forward", baseUrl: "https://chatgpt.com/backend-api/codex" }, }, @@ -527,6 +528,7 @@ describe("vision sidecar fallback (issue #88, end-to-end)", () => { allowPrivateNetwork: true, apiKey: "key-alpha-000111222333", noVisionModels: zen?.noVisionModels, + modelInputModalities: zen?.modelInputModalities, }, openai: { adapter: "openai-responses", authMode: "forward", baseUrl: "https://chatgpt.com/backend-api/codex" }, },