Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs-site/src/content/docs/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,10 @@ The request then follows normal combo selection and failover.

Explicit provider/combo selectors and configured combo aliases take precedence over this recall.
Failed, incomplete, or cancelled responses do not replace the last successful selection. Recall is
process-local and bounded to 256 lanes for 30 minutes; it does not store account credentials.
process-local and bounded to 256 lanes for 30 minutes; expired records are also swept periodically.
Model names are limited to 1 KiB each and 64 KiB total UTF-8 payload; the oldest records are evicted
when the count or byte budget is exceeded. A newer accepted completion with an oversized model
clears that lane's old record instead of retaining an outdated selection. No account credentials are stored.
Without usable conversation identity or valid remembered state, normal compaction routing applies.
A restart clears the remembered state.

Expand Down
2 changes: 1 addition & 1 deletion docs-site/src/content/docs/ko/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ alias는 클라이언트가 요청하는 공개 이름만 바꿉니다. 콤보

클라이언트가 콤보를 바꾼 뒤 공급자 접두사 없는 모델 이름으로 압축을 요청하면, opencodex는 같은 대화에서 가장 최근에 응답을 성공적으로 마친 콤보를 기억해 사용할 수 있습니다. 모델 이름이 완료된 응답과 일치하고, 현재 설정에 해당 콤보와 대상이 남아 있어야 합니다. 압축 요청도 일반 콤보 선택과 페일오버를 따릅니다.

명시한 공급자·콤보 선택자와 설정된 콤보 별칭이 기억한 값보다 우선합니다. 실패·미완료·취소된 응답은 마지막 성공 기록을 덮어쓰지 않습니다. 기록은 프로세스 안에서 최대 256개 대화, 30분 동안 유지하며 계정 자격증명을 저장하지 않습니다. 유효한 대화 식별자나 기록이 없으면 일반 압축 라우팅을 사용합니다. 재시작하면 기록은 사라집니다.
명시한 공급자·콤보 선택자와 설정된 콤보 별칭이 기억한 값보다 우선합니다. 실패·미완료·취소된 응답은 마지막 성공 기록을 덮어쓰지 않습니다. 기록은 프로세스 안에서 최대 256개 대화, 30분 동안 유지하며 만료된 기록을 주기적으로 정리합니다. 모델명은 UTF-8 기준 개별 1 KiB, 합계 64 KiB로 제한하고 개수나 용량 한도를 넘으면 가장 오래된 기록부터 제거합니다. 새로 승인된 완료 응답의 모델명이 너무 크면 해당 대화의 이전 기록도 지워 오래된 선택을 남기지 않습니다. 계정 자격증명은 저장하지 않습니다. 유효한 대화 식별자나 기록이 없으면 일반 압축 라우팅을 사용합니다. 재시작하면 기록은 사라집니다.

## 전략 선택

Expand Down
4 changes: 2 additions & 2 deletions src/lib/state-store-registrations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import {
} from "../combos/failover";
import { reconcileComboWarningMemos } from "../combos/request";
import { reconcileComboRotationState } from "../combos/resolve";
import { reconcileComboRecall } from "../server/responses/combo-session-recall";
import { reconcileComboRecall, sweepExpiredComboRecall } from "../server/responses/combo-session-recall";
import { listLiveComboTargetKeys } from "../combos/types";
import {
listLiveConfigOwnershipRoots,
Expand Down Expand Up @@ -112,7 +112,7 @@ export const STATE_STORE_REGISTRATIONS = [
{ name: "model-cache-history", reconcileGeneration: reconcileModelCacheGeneration },
{ name: "pool-rotation", reconcileGeneration: reconcilePoolRotationState },
{ name: "combo-rotation", reconcileGeneration: reconcileComboRotationState },
{ name: "combo-session-recall", reconcileGeneration: reconcileComboRecall },
{ name: "combo-session-recall", sweepExpired: sweepExpiredComboRecall, reconcileGeneration: reconcileComboRecall },
{ name: "guardian-backoff", reconcileGeneration: reconcileGuardianBackoff },
{ name: "codex-reauth", reconcileGeneration: reconcileCodexReauthState },
{ name: "oauth-reauth", reconcileGeneration: reconcileOAuthReauthState },
Expand Down
54 changes: 47 additions & 7 deletions src/server/responses/combo-session-recall.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,16 @@ interface ComboRecallEntry {
comboId: string;
target: Pick<OcxComboTarget, "provider" | "model">;
responseModel: string;
responseModelBytes: number;
at: number;
}

const RECALL_CAPACITY = 256;
const RECALL_TTL_MS = 30 * 60 * 1000;
const RECALL_MODEL_MAX_BYTES = 1024;
const RECALL_MODEL_TOTAL_BYTES = 64 * 1024;
const recall = new Map<string, ComboRecallEntry>();
let retainedModelBytes = 0;
let lastReconciledGeneration = 0;
let liveOwners: Pick<GenerationContext, "comboIds" | "comboTargets" | "providerNames"> | undefined;

Expand All @@ -22,24 +26,49 @@ function ownsEntry(context: Pick<GenerationContext, "comboIds" | "comboTargets"
&& context.comboTargets.has(`${entry.comboId}::${targetKey(entry.target)}`);
}

function deleteEntry(lane: string): boolean {
const entry = recall.get(lane);
if (!entry) return false;
retainedModelBytes -= entry.responseModelBytes;
return recall.delete(lane);
}

function boundedModelBytes(model: string): number | undefined {
// Check code units before trimming or encoding to avoid another unbounded copy.
if (model.length > RECALL_MODEL_MAX_BYTES) return undefined;
const bytes = new TextEncoder().encode(model).byteLength;
return bytes <= RECALL_MODEL_MAX_BYTES ? bytes : undefined;
}

export function rememberComboForLane(
lane: string | undefined,
comboId: string,
target: Pick<OcxComboTarget, "provider" | "model">,
responseModel: string,
writerGeneration: number,
): void {
if (!lane || !comboId || !responseModel.trim()) return;
if (!lane || !comboId) return;
// Reject even a same-named recreated owner: its previous in-flight turn is obsolete.
if (writerGeneration < Math.max(lastReconciledGeneration, captureConfigGeneration())) return;
const entry = { comboId, target: { provider: target.provider, model: target.model }, responseModel, at: Date.now() };
const entry = { comboId, target: { provider: target.provider, model: target.model }, responseModel, responseModelBytes: 0, at: Date.now() };
if (liveOwners && !ownsEntry(liveOwners, entry)) return;
recall.delete(lane);
// Whitespace-only completions were always a no-op; test before the size branch so an
// oversized blank cannot fall into the clear path, and without allocating a trimmed copy.
if (!/\S/u.test(responseModel)) return;
const modelBytes = boundedModelBytes(responseModel);
if (modelBytes === undefined) {
// This accepted completion supersedes the lane even when its model cannot be retained.
deleteEntry(lane);
return;
}
entry.responseModelBytes = modelBytes;
deleteEntry(lane);
recall.set(lane, entry);
while (recall.size > RECALL_CAPACITY) {
retainedModelBytes += modelBytes;
while (recall.size > RECALL_CAPACITY || retainedModelBytes > RECALL_MODEL_TOTAL_BYTES) {
const oldest = recall.keys().next().value;
if (oldest === undefined) break;
recall.delete(oldest);
deleteEntry(oldest);
}
}

Expand All @@ -57,7 +86,7 @@ export function recallComboForLane(
|| !Object.hasOwn(config.providers, entry.target.provider)
|| !provider || provider.disabled === true
|| !combo?.targets.some(target => targetKey(target) === targetKey(entry.target))) {
recall.delete(lane);
deleteEntry(lane);
return undefined;
}
return entry.responseModel === model ? entry.comboId : undefined;
Expand All @@ -74,16 +103,27 @@ export function reconcileComboRecall(context: GenerationContext): number {
let removed = 0;
for (const [lane, entry] of recall) {
if (!ownsEntry(context, entry) || Date.now() - entry.at >= RECALL_TTL_MS) {
recall.delete(lane);
deleteEntry(lane);
removed += 1;
}
}
return removed;
}

export function sweepExpiredComboRecall(now: number): number {
let removed = 0;
for (const [lane, entry] of recall) {
if (now - entry.at < RECALL_TTL_MS) continue;
deleteEntry(lane);
removed += 1;
}
return removed;
}

/** Test-only reset, alongside the combo rotation/cooldown resets. */
export function clearComboRecallForTests(): void {
recall.clear();
retainedModelBytes = 0;
lastReconciledGeneration = 0;
liveOwners = undefined;
}
2 changes: 2 additions & 0 deletions structure/adapters/registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,3 +183,5 @@ implement legacy call/result pairing. Modern tool-image carriers are unchanged.
raw passthrough; `tests/responses/chat-media-translation.test.ts` reaches the real HTTP
translation boundary and verifies that rejection sends no upstream request.
Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged.

Combo recall follows the [Responses retention contract](../transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -390,3 +390,5 @@ Exact [model input declarations](config.md#explicit-per-model-capability-declara
## Renamed destination reasoning metadata

`src/providers/derive.ts` fills missing reasoning tables for renamed providers accepted by the existing fixed-key destination matcher. Model entries are cloned and explicit user entries (including empty arrays) win. Provider-wide effort defaults fill only when undefined; Command Code unknown models therefore keep the registry's empty picker policy unless overridden. Identity, transport and other capability axes are unchanged. The gathered row drives client exports; this metadata contract does not prove arbitrary gateway routing.

Combo recall follows the [Responses retention contract](transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/clients/claude-desktop.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,3 +157,5 @@ The [explicit model-capability contract](../config.md#explicit-per-model-capabil
Exact [model input declarations](../config.md#explicit-per-model-capability-declarations) now feed text-only eligibility and catalog hints; existing image-description/omission handling consumes them before the main upstream send.

Provider-scoped approval reviewer settings are projected by the [catalog owner](../catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior.

Combo recall follows the [Responses retention contract](../transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/clients/integrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,3 +195,5 @@ existing explicit confirmation. The journal endpoint evaluates Undo against the
Recovery reads commit history and ownership through strict store methods. Unreadable or malformed
metadata is uncertainty, never evidence that a transaction did not commit. Pending records validate
complete ownership, exact Cline paths and result fingerprints before either native file is replaced.

Combo recall follows the [Responses retention contract](../transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/data-planes/images.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,3 +121,5 @@ Live sideband admission and its bounded upstream handshake follow the [runtime c
The [explicit model-capability contract](../config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior.

Provider-scoped approval reviewer settings are projected by the [catalog owner](../catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior.

Combo recall follows the [Responses retention contract](../transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/data-planes/inbound-compat.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,3 +331,5 @@ Modern `tool` images continue through the existing following-user carrier. These
an OpenCodex conversion limit, not a provider capability claim. Final Responses-to-adapter
admission follows the [registry contract](../adapters/registry.md#untranslated-input-media).
Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged.

Combo recall follows the [Responses retention contract](../transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 1 addition & 1 deletion structure/gui-and-management-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -631,4 +631,4 @@ The [explicit model-capability contract](config.md#explicit-per-model-capability

Exact [model input declarations](config.md#explicit-per-model-capability-declarations) now feed text-only eligibility and catalog hints; existing image-description/omission handling consumes them before the main upstream send.

The raw provider editor round-trips `autoReviewModel` and `autoReviewModelOverrides` through editor-owned DTO fields. POST/PATCH/PUT share validation; PUT copies schema-normalized values into the persisted and live candidate before adoption. Canonical `openai` rejects these fields, including clear forms. Field-masked writes (PATCH, editor PUT, reload) pin every registry-seed key and ignore operator overlays the seed never defines, most commonly `selectedModels`; POST keeps the exact-key comparison. Canonical `openai` still rejects `allowPrivateNetwork`, which must not short-circuit destination DNS checks on the ChatGPT forward row. Existing authentication, origin checks and stale-baseline protection still govern the writes. See [reviewer projection](catalog.md#provider-scoped-approval-reviewer).
The raw provider editor round-trips `autoReviewModel` and `autoReviewModelOverrides` through editor-owned DTO fields. POST/PATCH/PUT share validation; PUT copies schema-normalized values into the persisted and live candidate before adoption. Canonical `openai` rejects these fields, including clear forms. Field-masked writes (PATCH, editor PUT, reload) pin every registry-seed key and ignore operator overlays the seed never defines, most commonly `selectedModels`; POST keeps the exact-key comparison. Canonical `openai` still rejects `allowPrivateNetwork`, which must not short-circuit destination DNS checks on the ChatGPT forward row. Existing authentication, origin checks and stale-baseline protection still govern the writes. See [reviewer projection](catalog.md#provider-scoped-approval-reviewer). Combo recall follows the [Responses retention contract](transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/ops/docs-and-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -384,3 +384,5 @@ Exact [model input declarations](../config.md#explicit-per-model-capability-decl
Provider-scoped approval reviewer settings are projected by the [catalog owner](../catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior.

Renamed fixed-key providers receive [missing reasoning metadata](../catalog.md#renamed-destination-reasoning-metadata) during derivation; explicit per-model entries and provider defaults retain precedence.

Combo recall follows the [Responses retention contract](../transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/ops/service-and-sidecars.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,3 +182,5 @@ Live sideband admission and its bounded upstream handshake follow the [runtime c
The [explicit model-capability contract](../config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior.

Provider-scoped approval reviewer settings are projected by the [catalog owner](../catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior.

Combo recall follows the [Responses retention contract](../transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,3 +147,5 @@ Translated Chat request construction uses the [inline-image budget](transports/s
The [explicit model-capability contract](config.md#explicit-per-model-capability-declarations) preserves operator declarations through provider storage and catalog capture; it does not infer upstream capability or change this surface's routing behavior.

Provider-scoped approval reviewer settings are projected by the [catalog owner](catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior.

Combo recall follows the [Responses retention contract](transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/providers/xai-grok.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,3 +141,5 @@ Account quota surfaces use [safe probe diagnostics](../transports/inventory.md#a
Live sideband admission and its bounded upstream handshake follow the [runtime contract](../runtime.md#live-sideband-handshake); the ordinary Responses WebSocket exchange remains separate.

Translated audio/file admission follows the [final-adapter input contract](../adapters/registry.md#untranslated-input-media); native raw passthrough remains separate.

Combo recall follows the [Responses retention contract](../transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ their own files.

## Lifecycle

Combo recall follows the [Responses retention contract](transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.

Startup catalog sync and native restore apply the [retired-native policy](catalog.md#shared-catalog).
Codex quota processing has shared and Reserve scopes; retired model evidence is suppressed as
described in [OpenAI quota ownership](providers/openai-tiers.md#public-provider-contract).
Expand Down
2 changes: 2 additions & 0 deletions structure/subagents.md
Original file line number Diff line number Diff line change
Expand Up @@ -374,3 +374,5 @@ Exact [model input declarations](config.md#explicit-per-model-capability-declara
Provider-scoped approval reviewer settings are projected by the [catalog owner](catalog.md#provider-scoped-approval-reviewer); this surface retains its existing routing, transport and account-selection behavior.

Renamed fixed-key providers receive [missing reasoning metadata](catalog.md#renamed-destination-reasoning-metadata) during derivation; explicit per-model entries and provider defaults retain precedence.

Combo recall follows the [Responses retention contract](transports/responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/transports/byte-accounting.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,5 @@ These optimizations do not add request queues, retry policies, or RSS-based admi

Translated audio/file admission follows the [final-adapter input contract](../adapters/registry.md#untranslated-input-media); native raw passthrough remains separate.
Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged.

Combo recall follows the [Responses retention contract](responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
2 changes: 2 additions & 0 deletions structure/transports/inventory.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,3 +148,5 @@ Renamed fixed-key providers receive [missing reasoning metadata](../catalog.md#r

Translated audio/file admission follows the [final-adapter input contract](../adapters/registry.md#untranslated-input-media); native raw passthrough remains separate.
Canonical Responses identity sanitation and narrowly scoped pre-output combo recovery follow [request-local target compatibility](../runtime.md#request-local-target-compatibility); other adapter contracts remain unchanged.

Combo recall follows the [Responses retention contract](responses.md#responses-httpsse), including model-byte limits, accepted-writer invalidation and shared TTL cleanup.
Loading
Loading