diff --git a/docs-site/src/content/docs/ja/reference/configuration/providers.md b/docs-site/src/content/docs/ja/reference/configuration/providers.md index f1210892a5..7d897f705f 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ja/reference/configuration/providers.md @@ -68,6 +68,7 @@ account を削除しても mapping は保持され、同じ id を再追加す | `models?` | `string[]` |シード/フォールバック モデルのリスト。 `liveModels: false` では、発見されたモデルはこれらのみです。 | | `liveModels?` | `boolean` |開始/同期時にライブ カタログをフェッチします (デフォルトは `true`)。カスタムプロバイダーは `${baseUrl}/models` を使用します。組み込みはレジストリ URL とフィルターを使用する場合があります。 | | `selectedModels?` | `string[]` |検出後のカタログ許可リスト。空でない場合は、それらの ID のみが公開されます。空または省略すると、検出されたすべてのモデルが公開されます。 | +| `retainModels?` | `string[]` | ライブディスカバリで省略された場合でもカタログに保持するモデルIDの許可リスト(未プロビジョニングのアカウントや未掲載モデルなど)。ここに記載されていない設定済みIDは、ディスカバリで省略された場合に引き続き除外(prune)されます。 | | `contextWindow?` | `number` | アップストリームのメタデータが無い場合に使うプロバイダー全体のコンテキスト値。メタデータがある場合は上限として働き、より小さいライブ値をそのまま残します。Models ダッシュボードでは `providerContextCaps` とは別に設定します。 | | `modelContextWindows?` | `Record` | モデルごとのコンテキスト値および上限。`contextWindow` より優先され、ウィンドウが不明なら設定値を使い、より小さいライブメタデータがあればそちらが優先されます。 | | `modelInputModalities?` | `Record` | `["text"]` や `["text", "image"]` などのモデルごとの入力ヒント。 | diff --git a/docs-site/src/content/docs/ko/reference/configuration/providers.md b/docs-site/src/content/docs/ko/reference/configuration/providers.md index ccacb0a94f..d393a0db24 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ko/reference/configuration/providers.md @@ -68,6 +68,7 @@ managed map을 활성화하면 privacy-safe selector를 만들고, 이후 계정 | `models?` | `string[]` | 시드/폴백 모델 목록입니다. `liveModels: false`이면 이 목록만 발견된 모델로 취급합니다. | | `liveModels?` | `boolean` | 시작 또는 동기화 시 라이브 카탈로그를 가져옵니다. 기본값은 `true`입니다. 사용자 지정 공급자는 `${baseUrl}/models`를 사용하고, 내장은 레지스트리 URL을 사용한 뒤 필터링할 수 있습니다. | | `selectedModels?` | `string[]` | 발견 후 카탈로그 허용 목록입니다. 값이 비어 있지 않으면 그 id만 노출하고, 비어 있거나 생략하면 발견된 모델을 모두 노출합니다. | +| `retainModels?` | `string[]` | 라이브 발견에서 누락되더라도 카탈로그에 유지할 모델 ID 허용 목록입니다 (예: 미프로비저닝 계정 또는 미등록 모델). 여기에 나열되지 않은 구성 ID는 발견에서 누락될 때 계속 제외(prune)됩니다. | | `contextWindow?` | `number` | 업스트림 메타데이터가 없을 때 쓰이는 공급자 전반의 컨텍스트 값입니다. 메타데이터가 있으면 상한으로 동작해 더 작은 라이브 값을 그대로 둡니다. Models 대시보드에서 `providerContextCaps`와 별도로 설정합니다. | | `modelContextWindows?` | `Record` | 모델별 컨텍스트 값이자 상한입니다. `contextWindow`보다 우선하며, 창 크기를 알 수 없으면 설정값을 쓰고 더 작은 라이브 메타데이터가 있으면 그쪽을 따릅니다. | | `modelInputModalities?` | `Record` | `["text"]` 또는 `["text", "image"]` 같은 모델별 입력 힌트입니다. | diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 782effab52..463c3e9062 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -82,6 +82,7 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids. | `models?` | `string[]` | Seed/fallback model list. With `liveModels: false`, these are the only discovered models. | | `liveModels?` | `boolean` | Fetch the live catalog on start/sync (default `true`). Custom providers use `${baseUrl}/models`; built-ins may use a registry URL and filter. | | `selectedModels?` | `string[]` | Catalog allowlist after discovery. Non-empty exposes only those ids; empty or omitted exposes all discovered models. | +| `retainModels?` | `string[]` | Provider-level allowlist of model IDs to retain when live discovery omits them (for example, unprovisioned accounts or unlisted models). Configured IDs not listed here continue to be pruned when omitted by discovery. | | `contextWindow?` | `number` | Provider-wide context fallback when upstream metadata is absent; otherwise a cap that retains smaller live metadata. The Models dashboard exposes this separately from `providerContextCaps`. | | `modelContextWindows?` | `Record` | Per-model context fallbacks/caps. These override `contextWindow`: an unknown window uses the configured value, while smaller live metadata remains authoritative. | | `modelInputModalities?` | `Record` | Per-model input hints such as `["text"]` or `["text", "image"]`. | diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md index 3630a9ba6c..105008a9d6 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md @@ -68,6 +68,7 @@ selector,而不是分配一个新名称。 | `models?` | `string[]` | 种子/回退模型列表。配合 `liveModels: false` 时,这些就是唯一发现到的模型。 | | `liveModels?` | `boolean` | 启动/同步时获取实时目录(默认 `true`)。自定义提供者使用 `${baseUrl}/models`;内置项可能使用注册表 URL 并进行过滤。 | | `selectedModels?` | `string[]` | 发现之后的目录允许列表。非空时只暴露这些 id;为空或省略时则暴露全部发现到的模型。 | +| `retainModels?` | `string[]` | 提供者级模型 ID 保留允许列表,当实时发现遗漏它们时仍予以保留(例如未开通账号或未列出模型)。未在此列出的配置模型在发现遗漏时继续被剪除(prune)。 | | `contextWindow?` | `number` | 上游缺少元数据时使用的提供者级上下文数值;有元数据时作为上限,保留更小的实时数值。Models 面板中与 `providerContextCaps` 分开设置。 | | `modelContextWindows?` | `Record` | 按模型设置的上下文数值与上限。优先于 `contextWindow`:窗口未知时采用所配置的数值,而更小的实时元数据仍然优先。 | | `modelInputModalities?` | `Record` | 按模型设置的输入提示,例如 `["text"]` 或 `["text", "image"]`。 | diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md index b0a46f49ec..991dc084bd 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md @@ -50,6 +50,7 @@ description: 供應商項目、認證、端點、模型目錄、配額、context | `models?` | `string[]` | 播種/後備模型清單。在 `liveModels: false` 時,這些是唯一探索的模型。 | | `liveModels?` | `boolean` | 在啟動/同步時擷取即時目錄(預設 `true`)。自訂供應商使用 `${baseUrl}/models`;內建可能使用 registry URL 並過濾。 | | `selectedModels?` | `string[]` | 探索後的目錄允許清單。非空時僅暴露那些 id;空或省略時暴露所有探索的模型。 | +| `retainModels?` | `string[]` | 當即時探索遺漏模型時仍保留在目錄中的允許清單(例如未開通帳號或未列出模型)。未在此列出的配置模型在探索遺漏時維持剪除(prune)。 | | `contextWindow?` | `number` | 供應商範圍的 Codex 可見 context 上限。較小的即時中繼資料被保留。 | | `modelContextWindows?` | `Record` | Per-model context 上限。這些覆寫 `contextWindow` 且永不提高較小的即時中繼資料。 | | `modelInputModalities?` | `Record` | Per-model 輸入提示,如 `["text"]` 或 `["text", "image"]`。 | diff --git a/src/codex/catalog/parsing.ts b/src/codex/catalog/parsing.ts index 94241e8a75..be3168e46c 100644 --- a/src/codex/catalog/parsing.ts +++ b/src/codex/catalog/parsing.ts @@ -131,6 +131,8 @@ export interface CatalogModel { /** Optional provider-specific copy for the advertised Fast tier. */ fastTierDescription?: string; supportsReasoningSummaries?: boolean; + /** Whether this model was retained in the catalog via retainModels without live discovery confirmation. */ + retainedWithoutDiscovery?: boolean; /** * Codex tool calling mode for this routed model. * "code_mode_only" (default) sets entry.tool_mode = "code_mode_only". diff --git a/src/codex/catalog/provider-fetch.ts b/src/codex/catalog/provider-fetch.ts index 5ef9dffa2b..f218cf14d4 100644 --- a/src/codex/catalog/provider-fetch.ts +++ b/src/codex/catalog/provider-fetch.ts @@ -946,6 +946,8 @@ export function reconcileProviderFetchWarnings(generation: number): number { if (generation <= lastWarningReconciledGeneration) return 0; const removed = lastDropWarnSignature.size; lastDropWarnSignature.clear(); + retainedWithoutDiscoveryRefs.clear(); + warnedRetained404Refs.clear(); lastWarningReconciledGeneration = generation; return removed; } @@ -978,6 +980,42 @@ export function warnDroppedConfiguredIdsOnce(name: string, droppedConfiguredIds: ); } +/** + * Model ids on each provider that `retainModels` kept in the catalog even though live + * discovery did not report them. Used by dispatch error paths to explain a later + * upstream 404 (model_not_found) instead of letting the operator blame the proxy. + */ +const retainedWithoutDiscoveryRefs = new Map>(); +const warnedRetained404Refs = new Set(); + +/** + * Emit a one-shot warning when a model retained via `retainModels` is rejected by the + * upstream (HTTP 404 / model_not_found). Only fires for models that were actually + * retained without live-discovery confirmation, and only once per provider/model. + */ +export function warnRetainedModel404Once(providerName: string, modelId: string): void { + const refs = retainedWithoutDiscoveryRefs.get(providerName); + if (!refs || !refs.has(modelId)) return; + const signature = `${providerName}/${modelId}`; + if (warnedRetained404Refs.has(signature)) return; + warnedRetained404Refs.add(signature); + console.warn( + `[opencodex] Model "${modelId}" on provider "${providerName}" is retained via retainModels but upstream returned 404/model_not_found; the account or project may not be provisioned for it. Remove it from retainModels if it should not be callable.`, + ); +} + +/** Test-only helper: check whether a provider model is currently tracked as retained without discovery. */ +export function isRetainedModelWithoutDiscoveryForTests(providerName: string, modelId: string): boolean { + return retainedWithoutDiscoveryRefs.get(providerName)?.has(modelId) === true; +} + +/** Test-only helper: reset retained model warning states and reconciled generation. */ +export function resetRetainedModelWarningsForTests(): void { + retainedWithoutDiscoveryRefs.clear(); + warnedRetained404Refs.clear(); + lastWarningReconciledGeneration = 0; +} + /** * Z.AI and Neuralwatt advertise GLM reasoning as a bare boolean, which would otherwise * collapse to the four-tier default ladder that omits `max`. These two helpers name the @@ -1201,20 +1239,48 @@ async function fetchProviderModelsWithAuth( resolveAuth: ModelsAuthResolver, ): Promise { const { name, provider: prov, discovery, request } = captured; - const observed = ( - models: CatalogModel[], - state: CatalogGatherProviderModelOutcome["state"], - ): ProviderModelsResult => ({ models, outcome: { provider: name, state } }); // Capture before any credential refresh or outbound await. OAuth account changes clear this // generation, so a request started with the former account cannot later publish its result. const cacheGeneration = captureModelCacheGeneration(name); const isCurrentCacheGeneration = () => isModelCacheGenerationCurrent(name, cacheGeneration); + function syncRetainedModelDiagnostics( + models: readonly CatalogModel[], + state: CatalogGatherProviderModelOutcome["state"], + ): void { + if (state !== "authoritative") return; + if (prov.liveModels === false || !Array.isArray(prov.retainModels) || prov.retainModels.length === 0) { + retainedWithoutDiscoveryRefs.delete(name); + return; + } + const retainSet = new Set(prov.retainModels); + const retainedIds = models + .filter(m => m.retainedWithoutDiscovery === true && retainSet.has(m.id)) + .map(m => m.id); + if (retainedIds.length > 0) { + retainedWithoutDiscoveryRefs.set(name, new Set(retainedIds)); + } else { + retainedWithoutDiscoveryRefs.delete(name); + } + } + const observed = ( + models: CatalogModel[], + state: CatalogGatherProviderModelOutcome["state"], + ): ProviderModelsResult => { + if (isCurrentCacheGeneration()) { + syncRetainedModelDiagnostics(models, state); + } + return { models, outcome: { provider: name, state } }; + }; if (prov.authMode === "forward") return observed([], "authoritative"); // ChatGPT backend has no /models const seedVertexDefault = prov.adapter === "google" && prov.googleMode === "vertex" && (prov.models?.length ?? 0) === 0 && Boolean(prov.defaultModel); - const configuredIds = seedVertexDefault && prov.defaultModel ? [prov.defaultModel] : (prov.models ?? []); + const configuredIds = Array.from(new Set([ + ...(seedVertexDefault && prov.defaultModel ? [prov.defaultModel] : []), + ...(prov.models ?? []), + ...(prov.retainModels ?? []), + ])); const configured: CatalogModel[] = configuredIds.map(id => ({ id, provider: name, @@ -1321,7 +1387,7 @@ async function fetchProviderModelsWithAuth( return observed(withConfiguredRetention(configured), "degraded"); } markProviderDiscoveryOk(name, liveResult.models.length); - return observed(withConfiguredRetention(forCache, { warnDrops: true }), "authoritative"); + return observed(withConfiguredRetention(result, { warnDrops: true }), "authoritative"); } if (isCurrentCacheGeneration()) { markModelsFetchFailure(name); @@ -1468,10 +1534,10 @@ async function fetchProviderModelsWithAuth( provider: name, // CCA only exposes a numeric thinking budget. Until the adapter owns an exact Codex // effort-to-wire mapping for a newly discovered model, do not advertise a false ladder. - reasoningEfforts: [], - ...(model.contextWindow ? { contextWindow: model.contextWindow } : {}), - ...(model.inputModalities ? { inputModalities: model.inputModalities } : {}), - }, contextCap)); + reasoningEfforts: [], + ...(model.contextWindow ? { contextWindow: model.contextWindow } : {}), + ...(model.inputModalities ? { inputModalities: model.inputModalities } : {}), + }, contextCap)); const forCache = withConfiguredRetention(live, { retainComboTargets: false }); if (!setCached(name, forCache, Date.now(), cacheGeneration)) { return observed(withConfiguredRetention(configured), "degraded"); @@ -1481,7 +1547,7 @@ async function fetchProviderModelsWithAuth( cacheGeneration, }); markProviderDiscoveryOk(name, live.length); - return observed(withConfiguredRetention(forCache, { warnDrops: true }), "authoritative"); + return observed(withConfiguredRetention(live, { warnDrops: true }), "authoritative"); } const extracted = extractProviderModelItems(bounded.value, discovery); if (!extracted.ok) { @@ -1512,15 +1578,15 @@ async function fetchProviderModelsWithAuth( .filter(m => shouldExposeProviderModel(name, m.id)); // Capture the count BEFORE the alias/configured augmentation below pushes extra rows into // `live`; otherwise configured entries would be reported as discovered ones. - const liveModelCount = live.length; - // Dated-release aliases + configured retention (compat allow-list, combo targets, - // Vertex default). Cache without combo retention so a later gather re-applies the - // current capture's retain set on read (warm-cache OCX-111 / #1308). + const liveModelCount = live.length; + // Dated-release aliases + configured retention (compat allow-list, combo targets, + // Vertex default). Cache without combo retention so a later gather re-applies the + // current capture's retain set on read (warm-cache OCX-111 / #1308). const forCache = withConfiguredRetention(live, { retainComboTargets: false }); - const returned = withConfiguredRetention(forCache, { warnDrops: true }); - const droppedConfiguredIds = configured - .map(model => model.id) - .filter(id => !returned.some(model => model.id === id)); + const returned = withConfiguredRetention(live, { warnDrops: true }); + const droppedConfiguredIds = configured + .map(model => model.id) + .filter(id => !returned.some(model => model.id === id)); if (returned.length === 0 && name !== OPENAI_API_PROVIDER_ID) { console.warn( `[opencodex] Provider model discovery for "${name}" returned an authoritative empty catalog; ${droppedConfiguredIds.length > 0 ? `dropping configured model ids: ${droppedConfiguredIds.join(", ")}` : "no models will be exposed"}.`, @@ -1596,7 +1662,7 @@ export function mergeConfiguredModelsIntoLiveCatalog(opts: { contextCap?: number; seedVertexDefault?: boolean; retainComboTargets?: boolean; -}): { models: CatalogModel[]; droppedConfiguredIds: string[] } { +}): { models: CatalogModel[]; droppedConfiguredIds: string[]; retainedConfiguredIds: string[] } { const { name, provider: prov, @@ -1609,6 +1675,8 @@ export function mergeConfiguredModelsIntoLiveCatalog(opts: { const out = [...opts.models]; const present = new Set(out.map(model => model.id)); const droppedConfiguredIds: string[] = []; + const retainedConfiguredIds: string[] = []; + const providerRetainModels = Array.isArray(prov.retainModels) ? new Set(prov.retainModels) : undefined; for (const candidate of configured) { if (present.has(candidate.id)) continue; const dated = out.find(live => isDatedVariantId(live.id, candidate.id)); @@ -1621,14 +1689,17 @@ export function mergeConfiguredModelsIntoLiveCatalog(opts: { seedVertexDefault === true || shouldRetainConfiguredProviderModel(name, candidate.id) || (retainComboTargets && retainConfiguredModelIds?.has(candidate.id) === true) + || (providerRetainModels?.has(candidate.id) === true) ) { - out.push(candidate); + const isRetainedFromConfig = providerRetainModels?.has(candidate.id) === true; + out.push(isRetainedFromConfig ? { ...candidate, retainedWithoutDiscovery: true } : candidate); present.add(candidate.id); + if (isRetainedFromConfig) retainedConfiguredIds.push(candidate.id); continue; } droppedConfiguredIds.push(candidate.id); } - return { models: out, droppedConfiguredIds }; + return { models: out, droppedConfiguredIds, retainedConfiguredIds }; } export function filterCatalogVisibleModels( diff --git a/src/config.ts b/src/config.ts index 1308b3a64b..5a4f650739 100644 --- a/src/config.ts +++ b/src/config.ts @@ -513,6 +513,9 @@ const providerConfigSchema = z.object({ .nullish() .transform(value => value ?? undefined), directGeminiWireRenames: z.boolean().optional(), + retainModels: z.array(z.string().min(1)) + .transform(normalizeNonBlankStringArray) + .optional(), noStructuredOutputModels: z.array(z.string().min(1)) .transform(normalizeNonBlankStringArray) .optional(), @@ -1213,6 +1216,17 @@ const configSchema = z.object({ message: structuredOutputOptOutError, }); } + const retainModelsError = nonBlankStringArrayConfigError( + (provider as { retainModels?: unknown }).retainModels, + "retainModels", + ); + if (retainModelsError) { + ctx.addIssue({ + code: "custom", + path: ["providers", redactSecretString(name), "retainModels"], + message: retainModelsError, + }); + } if (Object.hasOwn(provider, "codexAccountMode") && provider.codexAccountMode !== undefined) { // Persisted account mode is valid ONLY on the canonical built-in `openai` forward provider. // Old openai-multi rows stay parseable (they never carry a mode) so startup can migrate them. diff --git a/src/providers/model-rename-migration.ts b/src/providers/model-rename-migration.ts index 3895386b58..4d2f04172f 100644 --- a/src/providers/model-rename-migration.ts +++ b/src/providers/model-rename-migration.ts @@ -99,6 +99,7 @@ const MODEL_ID_LISTS = [ // their catalog instead of being renamed. OAuth reconciliation does not cover this // field, so the rename has to. "selectedModels", + "retainModels", "noVisionModels", "noReasoningModels", "noTemperatureModels", diff --git a/src/server/chat-completions.ts b/src/server/chat-completions.ts index db084df490..287bcc2a95 100644 --- a/src/server/chat-completions.ts +++ b/src/server/chat-completions.ts @@ -46,6 +46,7 @@ import { type TranslatorBudget, } from "../lib/translator-budget"; import { handleNativeChatCompletions, isNativeChatRouteEligible } from "./chat-native"; +import { warnRetainedModel404Once } from "../codex/catalog/provider-fetch"; type Rec = Record; @@ -310,6 +311,9 @@ async function handleChatCompletionsWithBudget( : "invalid_request_error"), message, ); + if (settledRoute && (upstream.status === 404 || upstreamCode === "model_not_found")) { + warnRetainedModel404Once(settledRoute.providerName, settledRoute.modelId); + } if (isCyberPolicyCode(upstreamCode) || classified.code === CYBER_POLICY_ERROR_CODE) { classified.code = CYBER_POLICY_ERROR_CODE; classified.type = cyberPolicyErrorType(upstreamType); @@ -406,6 +410,7 @@ async function handleChatCompletionsWithBudget( // Same deliberate preserve as the non-OK path: structured code beats generic classify. classified.code = "model_not_found"; classified.type = "invalid_request_error"; + if (settledRoute) warnRetainedModel404Once(settledRoute.providerName, settledRoute.modelId); } return chatCompletionsErrorResponse( classified.code === "translation_buffer_limit" diff --git a/src/server/chat-native.ts b/src/server/chat-native.ts index 2e84dc68d3..e485db1b41 100644 --- a/src/server/chat-native.ts +++ b/src/server/chat-native.ts @@ -48,6 +48,7 @@ import { type RequestLogContext, } from "./request-log"; import { jsonCompletionSse, nativeChatSse, structuredError, usageFromChat } from "./chat-native-sse"; +import { warnRetainedModel404Once } from "../codex/catalog/provider-fetch"; import { registerTurn, unregisterTurn } from "./lifecycle"; type Rec = Record; @@ -312,6 +313,9 @@ export async function handleNativeChatCompletions(options: HandleNativeChatOptio : response.status >= 500 ? "server_error" : "invalid_request_error"), message, ); + if (response.status === 404 || upstreamCode === "model_not_found") { + warnRetainedModel404Once(route.providerName, route.modelId); + } if (isCyberPolicyCode(upstreamCode) || classified.code === CYBER_POLICY_ERROR_CODE) { classified.code = CYBER_POLICY_ERROR_CODE; classified.type = cyberPolicyErrorType(upstreamType); diff --git a/src/server/claude-messages.ts b/src/server/claude-messages.ts index 8425395b18..5a0e5aff2d 100644 --- a/src/server/claude-messages.ts +++ b/src/server/claude-messages.ts @@ -50,6 +50,7 @@ import { isTranslatorBudgetExceededError, type TranslatorBudget, } from "../lib/translator-budget"; +import { warnRetainedModel404Once } from "../codex/catalog/provider-fetch"; type Rec = Record; @@ -679,8 +680,10 @@ async function handleClaudeMessagesWithBudget( // bodies: it 400s on sampling params ("Unsupported parameter: max_output_tokens", // verified live 2026-07-11). Strip them for that route; routed providers keep them. let nativeRoute = false; + let settledRoute: ReturnType | null = null; try { const route = routeModel(config, internalBody.model as string, evidenceFromBody(internalBody)); + settledRoute = route; // Settle the wire once so the sampling decision below reads the effective // adapter rather than the provider-wide default (#404). route.provider = resolveWireProtocolOverride(route.providerName, route.modelId, route.provider, "anthropic"); @@ -786,6 +789,8 @@ async function handleClaudeMessagesWithBudget( const response = logIds ? responseWithDeferredRequestLog(upstream, logIds.requestId, logIds.start, logCtx) : upstream; if (!response.ok) { + // Retained-but-unprovisioned models surface here as upstream 404; explain once. + if (response.status === 404 && settledRoute) warnRetainedModel404Once(settledRoute.providerName, settledRoute.modelId); // Re-shape the OpenAI-style error envelope into the Anthropic one, preserving status. let message = `upstream error (${response.status})`; try { diff --git a/src/server/responses/core.ts b/src/server/responses/core.ts index 691d75ada4..6972768daa 100644 --- a/src/server/responses/core.ts +++ b/src/server/responses/core.ts @@ -8,6 +8,7 @@ import { } from "./responses-field-backfill"; import { checkInputAdmission } from "./input-admission"; import { nativeContextLimits } from "../../codex/catalog"; +import { warnRetainedModel404Once } from "../../codex/catalog/provider-fetch"; import { describeUpstreamConnectFailure } from "./upstream-error"; import { multiAgentGuidanceEnabled, @@ -4157,6 +4158,8 @@ async function handleResponsesInner( }); } if (!upstreamResponse.ok) { + // Retained-but-unprovisioned models surface here as upstream 404; explain once. + if (upstreamResponse.status === 404) warnRetainedModel404Once(route.providerName, route.modelId); if (options.comboAttempt) { // No pre-read guard here: `consumeComboFailure` -> `readBoundedResponseBody` reads // `response.body` itself and already threads the abort signal through its own read, @@ -5646,9 +5649,10 @@ async function handleResponsesInner( } break; } - if (!upstreamResponse.ok) { - if (options.comboAttempt) { - // No pre-read guard: `consumeComboFailure` -> `readBoundedResponseBody` reads + if (!upstreamResponse.ok) { + if (upstreamResponse.status === 404) warnRetainedModel404Once(route.providerName, route.modelId); + if (options.comboAttempt) { + // No pre-read guard: `consumeComboFailure` -> `readBoundedResponseBody` reads // `response.body` itself with the abort signal threaded through, and the combo // contract is that this body's getter is touched exactly once. A guard here would be // a second `.body` access for no gain, since the bounded reader owns settlement. diff --git a/src/types/provider.ts b/src/types/provider.ts index b7ba042506..b4a4da2568 100644 --- a/src/types/provider.ts +++ b/src/types/provider.ts @@ -308,6 +308,14 @@ export interface OcxProviderConfig { */ fallback?: "preset-empty"; }; + /** + * Model IDs to retain in the authoritative live catalog even when upstream discovery omits them. + * IDs listed here are independently seeded as configured entries and kept through + * `mergeConfiguredModelsIntoLiveCatalog`; they do not need to also appear in `models`. + * Useful when an upstream account has unlisted or upcoming models (e.g. unprovisioned + * Antigravity models or regional previews). + */ + retainModels?: string[]; /** Provider-wide fallback when context metadata is absent; otherwise caps the reported window. */ contextWindow?: number; /** Per-model fallback when context metadata is absent; otherwise caps the reported window. */ diff --git a/tests/provider-retain-models.test.ts b/tests/provider-retain-models.test.ts new file mode 100644 index 0000000000..fbf130e784 --- /dev/null +++ b/tests/provider-retain-models.test.ts @@ -0,0 +1,582 @@ +import { describe, expect, test } from "bun:test"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { installIsolatedCodexHome } from "./helpers/isolated-codex-home"; +import { + filterCatalogVisibleModels, + isRetainedModelWithoutDiscoveryForTests, + mergeConfiguredModelsIntoLiveCatalog, + reconcileProviderFetchWarnings, + resetRetainedModelWarningsForTests, + warnRetainedModel404Once, +} from "../src/codex/catalog/provider-fetch"; +import { clearModelCache } from "../src/codex/model-cache"; +import { nonBlankStringArrayConfigError } from "../src/config"; +import { saveConfig } from "../src/config"; +import { startServer } from "../src/server"; +import type { CatalogModel } from "../src/codex/catalog/parsing"; +import type { OcxConfig, OcxProviderConfig } from "../src/types"; + +function model(id: string, provider = "test-prov"): CatalogModel { + return { id, provider }; +} + +describe("#1690 retainModels provider configuration", () => { + test("retains configured models listed in retainModels when live discovery omits them", () => { + const prov: OcxProviderConfig = { + adapter: "google", + baseUrl: "https://daily-cloudcode-pa.googleapis.com", + retainModels: ["gemini-3.7-flash"], + }; + const live = [model("gemini-3.5-flash")]; + const configured = [model("gemini-3.7-flash"), model("unrelated-model")]; + + const { models, droppedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({ + name: "google-antigravity", + provider: prov, + models: live, + configured, + }); + + expect(models.map(m => m.id)).toEqual(["gemini-3.5-flash", "gemini-3.7-flash"]); + expect(droppedConfiguredIds).toEqual(["unrelated-model"]); + expect(models.find(m => m.id === "gemini-3.7-flash")?.retainedWithoutDiscovery).toBe(true); + expect(models.find(m => m.id === "gemini-3.5-flash")?.retainedWithoutDiscovery).toBeUndefined(); + }); + + test("drops unlisted models when retainModels is empty", () => { + const prov: OcxProviderConfig = { + adapter: "openai-chat", + baseUrl: "https://api.example.com/v1", + retainModels: [], + }; + const live = [model("live-model-1")]; + const configured = [model("configured-model-1")]; + + const { models, droppedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({ + name: "custom-prov", + provider: prov, + models: live, + configured, + }); + + expect(models.map(m => m.id)).toEqual(["live-model-1"]); + expect(droppedConfiguredIds).toEqual(["configured-model-1"]); + }); + + test("drops unlisted models when retainModels is undefined", () => { + const prov: OcxProviderConfig = { + adapter: "openai-chat", + baseUrl: "https://api.example.com/v1", + }; + const live = [model("live-model-1")]; + const configured = [model("configured-model-1")]; + + const { models, droppedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({ + name: "custom-prov", + provider: prov, + models: live, + configured, + }); + + expect(models.map(m => m.id)).toEqual(["live-model-1"]); + expect(droppedConfiguredIds).toEqual(["configured-model-1"]); + }); + + test("preserves discovered models that match retainModels without duplication", () => { + const prov: OcxProviderConfig = { + adapter: "google", + baseUrl: "https://daily-cloudcode-pa.googleapis.com", + retainModels: ["gemini-3.7-flash"], + }; + const live = [model("gemini-3.7-flash")]; + const configured = [model("gemini-3.7-flash")]; + + const { models, droppedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({ + name: "google-antigravity", + provider: prov, + models: live, + configured, + }); + + expect(models.map(m => m.id)).toEqual(["gemini-3.7-flash"]); + expect(droppedConfiguredIds).toEqual([]); + expect(models[0]!.retainedWithoutDiscovery).toBeUndefined(); + }); + + test("retains multiple specified models across an empty live discovery", () => { + const prov: OcxProviderConfig = { + adapter: "google", + baseUrl: "https://daily-cloudcode-pa.googleapis.com", + retainModels: ["gemini-3.7-flash", "claude-sonnet-4-6"], + }; + const live: CatalogModel[] = []; + const configured = [ + model("gemini-3.7-flash"), + model("claude-sonnet-4-6"), + model("dropped-model"), + ]; + + const { models, droppedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({ + name: "google-antigravity", + provider: prov, + models: live, + configured, + }); + + expect(models.map(m => m.id)).toEqual(["gemini-3.7-flash", "claude-sonnet-4-6"]); + expect(droppedConfiguredIds).toEqual(["dropped-model"]); + }); + + test("reports retainedConfiguredIds and stamps retainedWithoutDiscovery", () => { + const prov: OcxProviderConfig = { + adapter: "google", + baseUrl: "https://daily-cloudcode-pa.googleapis.com", + retainModels: ["gemini-3.7-flash", "claude-sonnet-4-6"], + }; + const live = [model("gemini-3.5-flash")]; + const configured = [ + model("gemini-3.7-flash"), + model("claude-sonnet-4-6"), + model("unrelated-model"), + ]; + + const { models, droppedConfiguredIds, retainedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({ + name: "google-antigravity", + provider: prov, + models: live, + configured, + }); + + expect(models.map(m => m.id)).toEqual(["gemini-3.5-flash", "gemini-3.7-flash", "claude-sonnet-4-6"]); + expect(retainedConfiguredIds.sort()).toEqual(["claude-sonnet-4-6", "gemini-3.7-flash"]); + expect(droppedConfiguredIds).toEqual(["unrelated-model"]); + }); + + test("does not report live-discovered models as retainedConfiguredIds", () => { + const prov: OcxProviderConfig = { + adapter: "google", + baseUrl: "https://daily-cloudcode-pa.googleapis.com", + retainModels: ["gemini-3.7-flash"], + }; + const live = [model("gemini-3.7-flash")]; + const configured = [model("gemini-3.7-flash")]; + + const { models, retainedConfiguredIds } = mergeConfiguredModelsIntoLiveCatalog({ + name: "google-antigravity", + provider: prov, + models: live, + configured, + }); + + expect(models.map(m => m.id)).toEqual(["gemini-3.7-flash"]); + expect(retainedConfiguredIds).toEqual([]); + }); + + test("rejects whitespace-only retainModels entries via nonBlankStringArrayConfigError", () => { + const error = nonBlankStringArrayConfigError([" "], "retainModels"); + expect(error).not.toBeNull(); + expect(error).toContain("nonblank"); + expect(nonBlankStringArrayConfigError(["gemini-3.7-flash", " gemini-3.5-flash "], "retainModels")).toBeNull(); + }); + + test("respects selectedModels and disabledModels filtering after retaining models", () => { + const prov: OcxProviderConfig = { + adapter: "google", + baseUrl: "https://daily-cloudcode-pa.googleapis.com", + retainModels: ["gemini-3.7-flash", "retained-unselected"], + selectedModels: ["gemini-3.7-flash"], + }; + const live = [model("gemini-3.5-flash", "google-antigravity")]; + const configured = [model("gemini-3.7-flash", "google-antigravity"), model("retained-unselected", "google-antigravity")]; + + const { models } = mergeConfiguredModelsIntoLiveCatalog({ + name: "google-antigravity", + provider: prov, + models: live, + configured, + }); + + expect(models.map(m => m.id)).toEqual(["gemini-3.5-flash", "gemini-3.7-flash", "retained-unselected"]); + + const visible = filterCatalogVisibleModels(models, { + providers: { "google-antigravity": prov }, + }); + expect(visible.map(m => m.id)).toEqual(["gemini-3.7-flash"]); + }); + + test("warnRetainedModel404Once stays silent for a model that was never retained", () => { + resetRetainedModelWarningsForTests(); + reconcileProviderFetchWarnings(1); + + const warnCalls: string[] = []; + const originalWarn = console.warn; + console.warn = (...args: any[]) => { + const msg = args.join(" "); + if (msg.includes("retained via retainModels")) warnCalls.push(msg); + }; + + try { + // Model not in retained refs does not warn + warnRetainedModel404Once("test-prov", "unknown-model"); + expect(warnCalls.length).toBe(0); + } finally { + console.warn = originalWarn; + } + }); +}); + +describe("catalog lifecycle and server 404 diagnostics for retained models", () => { + test("warm-cache gather restores provenance after generation reconcile and permits 404 warning", async () => { + const isolated = installIsolatedCodexHome("ocx-retain-warm-"); + const testDir = mkdtempSync(join(tmpdir(), "ocx-retain-warm-")); + const prevHome = process.env.OPENCODEX_HOME; + process.env.OPENCODEX_HOME = testDir; + + resetRetainedModelWarningsForTests(); + const warnCalls: string[] = []; + const originalWarn = console.warn; + console.warn = (...args: any[]) => { + const msg = args.join(" "); + if (msg.includes("retained via retainModels")) warnCalls.push(msg); + }; + + let upstreamCalls = 0; + const upstream = Bun.serve({ + port: 0, + fetch(req) { + const url = new URL(req.url); + if (url.pathname.includes("/models")) { + upstreamCalls += 1; + return Response.json({ + data: [{ id: "claude-live-1", owned_by: "anthropic" }], + }); + } + return Response.json({ + type: "error", + error: { type: "not_found_error", message: "model: claude-retained" }, + }, { status: 404 }); + }, + }); + + saveConfig({ + port: 0, + defaultProvider: "test-warm-prov", + providers: { + "test-warm-prov": { + adapter: "anthropic", + baseUrl: `http://127.0.0.1:${upstream.port}`, + apiKey: "test-key", + allowPrivateNetwork: true, + retainModels: ["claude-retained"], + }, + }, + } as OcxConfig); + const server = startServer(0); + + try { + // 1. Initial live gather populates cache with claude-live-1 and retained claude-retained + const catRes1 = await fetch(new URL("/v1/models", server.url), { + headers: { authorization: "Bearer test-caller-token" }, + }); + expect(catRes1.status).toBe(200); + const cat1 = (await catRes1.json()) as { data: Array<{ id: string }> }; + expect(cat1.data.map(m => m.id)).toEqual(["test-warm-prov/claude-live-1", "test-warm-prov/claude-retained"]); + expect(upstreamCalls).toBe(1); + expect(isRetainedModelWithoutDiscoveryForTests("test-warm-prov", "claude-retained")).toBe(true); + + // 2. Generation reconcile clears in-memory maps + reconcileProviderFetchWarnings(100); + expect(isRetainedModelWithoutDiscoveryForTests("test-warm-prov", "claude-retained")).toBe(false); + + // 3. Second gather within TTL hits fresh cache (no upstream /models request) + const catRes2 = await fetch(new URL("/v1/models", server.url), { + headers: { authorization: "Bearer test-caller-token" }, + }); + expect(catRes2.status).toBe(200); + expect(upstreamCalls).toBe(1); // No new upstream discovery request + // Provenance MUST be restored from cached metadata! + expect(isRetainedModelWithoutDiscoveryForTests("test-warm-prov", "claude-retained")).toBe(true); + + // 4. Request for retained model fails with 404 upstream -> emits exactly one warning + const resp404 = await fetch(new URL("/v1/responses", server.url), { + method: "POST", + headers: { + "content-type": "application/json", + authorization: "Bearer test-caller-token", + }, + body: JSON.stringify({ + model: "claude-retained", + input: [{ type: "message", role: "user", content: "hello" }], + }), + }); + expect(resp404.status).toBe(404); + expect(warnCalls.length).toBe(1); + expect(warnCalls[0]).toContain('Model "claude-retained" on provider "test-warm-prov" is retained via retainModels'); + + // 5. Second 404 is suppressed + await fetch(new URL("/v1/responses", server.url), { + method: "POST", + headers: { + "content-type": "application/json", + authorization: "Bearer test-caller-token", + }, + body: JSON.stringify({ + model: "claude-retained", + input: [{ type: "message", role: "user", content: "hello" }], + }), + }); + expect(warnCalls.length).toBe(1); + } finally { + await server.stop(true); + upstream.stop(true); + console.warn = originalWarn; + if (prevHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = prevHome; + isolated.restore(); + rmSync(testDir, { recursive: true, force: true }); + } + }); + + test("/v1/chat/completions triggers warnRetainedModel404Once on plain 404 and model_not_found", async () => { + const isolated = installIsolatedCodexHome("ocx-retain-chat-"); + const testDir = mkdtempSync(join(tmpdir(), "ocx-retain-chat-")); + const prevHome = process.env.OPENCODEX_HOME; + process.env.OPENCODEX_HOME = testDir; + + resetRetainedModelWarningsForTests(); + const warnCalls: string[] = []; + const originalWarn = console.warn; + console.warn = (...args: any[]) => { + warnCalls.push(args.join(" ")); + }; + const retainedWarnings = () => warnCalls.filter(line => line.includes("is retained via retainModels")); + + const upstream = Bun.serve({ + port: 0, + fetch(req) { + const url = new URL(req.url); + if (url.pathname.includes("/models")) { + return Response.json({ data: [] }); + } + return Response.json({ + error: { message: "Model does not exist", type: "invalid_request_error" }, + }, { status: 404 }); + }, + }); + + saveConfig({ + port: 0, + defaultProvider: "test-chat-prov", + providers: { + "test-chat-prov": { + adapter: "openai-chat", + baseUrl: `http://127.0.0.1:${upstream.port}/v1`, + apiKey: "test-key", + allowPrivateNetwork: true, + retainModels: ["retained-chat-model"], + }, + }, + } as OcxConfig); + const server = startServer(0); + + try { + // Populate models + await fetch(new URL("/v1/models", server.url), { + headers: { authorization: "Bearer test-caller-token" }, + }); + + const response = await fetch(new URL("/v1/chat/completions", server.url), { + method: "POST", + headers: { + "content-type": "application/json", + authorization: "Bearer test-caller-token", + }, + body: JSON.stringify({ + model: "retained-chat-model", + stream: false, + messages: [{ role: "user", content: "hello" }], + }), + }); + expect(response.status).toBe(404); + expect(retainedWarnings()).toHaveLength(1); + expect(retainedWarnings()[0]).toContain('Model "retained-chat-model" on provider "test-chat-prov"'); + } finally { + await server.stop(true); + upstream.stop(true); + console.warn = originalWarn; + if (prevHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = prevHome; + isolated.restore(); + rmSync(testDir, { recursive: true, force: true }); + } + }); + + test("delayed stale writer from prior generation cannot install stale diagnostic state", async () => { + const { fetchProviderModels } = await import("../src/codex/catalog/provider-fetch"); + resetRetainedModelWarningsForTests(); + clearModelCache(); + + let resolveDelayedDiscovery: (res: Response) => void; + const delayedPromise = new Promise(resolve => { + resolveDelayedDiscovery = resolve; + }); + + const provGen1: OcxProviderConfig = { + adapter: "openai-chat", + baseUrl: "https://api.example.com/v1", + apiKey: "test-key", + retainModels: ["retained-gen1"], + fetch: (async (url: RequestInfo | URL) => { + if (String(url).includes("/models")) { + return delayedPromise; + } + return new Response(JSON.stringify({ data: [] }), { status: 200, headers: { "content-type": "application/json" } }); + }) as typeof fetch, + }; + + // 1. Start live gather for Gen 1 (hangs on delayedPromise) + const inFlightGather = fetchProviderModels("test-stale-prov", provGen1, 60_000); + + // 2. Cache is cleared / generation bumped (e.g. config changed to Gen 2 without retainModels) + clearModelCache("test-stale-prov"); + reconcileProviderFetchWarnings(200); + + // 3. Resolve delayed discovery from Gen 1 + resolveDelayedDiscovery!(new Response(JSON.stringify({ data: [{ id: "live-gen1" }] }), { + status: 200, + headers: { "content-type": "application/json" }, + })); + await inFlightGather; + + // Stale writer was rejected by setCached, so retainedWithoutDiscoveryRefs must not have retained-gen1! + expect(isRetainedModelWithoutDiscoveryForTests("test-stale-prov", "retained-gen1")).toBe(false); + }); + + test("degraded catalog gather preserves prior retained model provenance", async () => { + const { fetchProviderModels } = await import("../src/codex/catalog/provider-fetch"); + resetRetainedModelWarningsForTests(); + clearModelCache(); + + let failDiscovery = false; + const prov: OcxProviderConfig = { + adapter: "openai-chat", + baseUrl: "https://api.example.com/v1", + apiKey: "test-key", + retainModels: ["retained-model"], + fetch: (async (url: RequestInfo | URL) => { + if (String(url).includes("/models")) { + if (failDiscovery) { + return new Response("Internal Server Error", { status: 500 }); + } + return new Response(JSON.stringify({ data: [{ id: "live-model" }] }), { + status: 200, + headers: { "content-type": "application/json" }, + }); + } + return new Response("{}", { status: 200 }); + }) as typeof fetch, + }; + + // 1. Authoritative gather registers retained-model provenance + const firstModels = await fetchProviderModels("test-degrade-prov", prov, 10); + expect(firstModels.map(m => m.id)).toEqual(["live-model", "retained-model"]); + expect(isRetainedModelWithoutDiscoveryForTests("test-degrade-prov", "retained-model")).toBe(true); + + // 2. Degraded gather (upstream 500) must NOT erase retained model provenance + failDiscovery = true; + await new Promise(resolve => setTimeout(resolve, 20)); + const secondModels = await fetchProviderModels("test-degrade-prov", prov, 10); + expect(secondModels.map(m => m.id)).toEqual(["live-model", "retained-model"]); + expect(isRetainedModelWithoutDiscoveryForTests("test-degrade-prov", "retained-model")).toBe(true); + }); + + test("Cursor and Antigravity retain configured combo targets in returned results but exclude from forCache", async () => { + const { gatherRoutedModels, clearGatherRoutedModelsInflight } = await import("../src/codex/catalog/provider-fetch"); + const { getStaleCached } = await import("../src/codex/model-cache"); + const { setFetchCursorUsableModelsForTests } = await import("../src/adapters/cursor/live-models"); + const { withStubbedProviderFetch } = await import("./helpers/catalog-provider-fetch"); + const { resetCatalogRuntimeStateForTests } = await import("../src/codex/catalog/sync"); + resetRetainedModelWarningsForTests(); + resetCatalogRuntimeStateForTests(); + clearGatherRoutedModelsInflight(); + + const warnCalls: string[] = []; + const originalWarn = console.warn; + console.warn = (...args: any[]) => { + warnCalls.push(args.join(" ")); + }; + + const prevFetch = globalThis.fetch; + setFetchCursorUsableModelsForTests(async () => ({ + ok: true, + models: ["gpt-5.5"], + })); + globalThis.fetch = (async (url: RequestInfo | URL) => { + if (String(url).includes("fetchAvailableModels")) { + return new Response(JSON.stringify({ + models: { "gemini-3.7-flash": { maxTokens: 1_048_576 } }, + agentModelSorts: [{ groups: [{ modelIds: ["gemini-3.7-flash"] }] }], + }), { + status: 200, + headers: { "content-type": "application/json" }, + }); + } + return new Response("{}", { status: 200 }); + }) as typeof fetch; + + try { + const config = { + providers: { + "cursor-test": { + adapter: "cursor" as const, + baseUrl: "https://api.cursor.test", + apiKey: "test-key", + models: ["gpt-5.5", "retained-combo-cursor", "configured-ghost-cursor"], + }, + "ag-test": { + adapter: "google" as const, + baseUrl: "https://daily-cloudcode-pa.googleapis.com", + apiKey: "test-key", + project: "test-project", + googleMode: "cloud-code-assist" as const, + models: ["gemini-3.7-flash", "retained-combo-ag", "configured-ghost-ag"], + }, + }, + combos: { + "combo-cursor": { + strategy: "failover" as const, + targets: [{ provider: "cursor-test", model: "retained-combo-cursor" }], + }, + "combo-ag": { + strategy: "failover" as const, + targets: [{ provider: "ag-test", model: "retained-combo-ag" }], + }, + }, + }; + + const gathered = await gatherRoutedModels(withStubbedProviderFetch(config) as any); + const gatheredIds = gathered.map(m => m.id); + expect(gatheredIds).toContain("retained-combo-cursor"); + expect(gatheredIds).toContain("retained-combo-ag"); + expect(gatheredIds).toContain("gpt-5.5"); + expect(gatheredIds).toContain("gemini-3.7-flash"); + expect(gatheredIds).not.toContain("configured-ghost-cursor"); + expect(gatheredIds).not.toContain("configured-ghost-ag"); + + // Cache excludes combo targets + expect(getStaleCached("cursor-test")?.map(m => m.id) ?? []).toEqual(["gpt-5.5"]); + expect(getStaleCached("ag-test")?.map(m => m.id) ?? []).toEqual(["gemini-3.7-flash"]); + + // Drop warnings emitted for configured ghosts + const droppedWarnings = warnCalls.filter(msg => msg.includes("omitted configured model ids")); + expect(droppedWarnings.some(msg => msg.includes("configured-ghost-cursor"))).toBe(true); + expect(droppedWarnings.some(msg => msg.includes("configured-ghost-ag"))).toBe(true); + } finally { + setFetchCursorUsableModelsForTests(null); + globalThis.fetch = prevFetch; + console.warn = originalWarn; + } + }); +});