Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/fr/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,8 @@ indique la réinitialisation de fenêtre à venir la plus proche (cinq heures, h
Le fournisseur dont le quota se renouvelle en premier est ainsi sollicité. Les cibles dépourvues de données de quota
récentes et les égalités conservent l’ordre de configuration. `weight` et `stickyLimit` n’affectent pas cette stratégie.

Ce classement et l’exclusion des fournisseurs avant l’envoi exigent des limites récentes d’inférence de modèles applicables dans leur ensemble à l’unique clé API actuelle. Les résumés OAuth ou du compte courant, les routes transmettant les identifiants de l’appelant, les configurations à plusieurs clés et les instantanés dont les identifiants ou la destination ont changé servent uniquement à l’affichage pour cette décision préalable. Il en va de même lorsque les en-têtes `Authorization`, `x-api-key` ou `x-goog-api-key` remplacent les identifiants ; les fenêtres réservées à la recherche ou à MCP sont exclues. Si aucune cible admissible n’a de réinitialisation applicable, l’ordre de configuration prévaut. La sélection des comptes et les nouvelles tentatives appliquent toujours leurs limites habituelles.

## Que se passe-t-il lorsqu'une cible échoue

Les échecs d’un combo se répartissent entre ceux qui entraînent un **basculement** et les échecs **terminaux**.
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,8 @@ shows the soonest upcoming window reset (five-hour, weekly, monthly, or custom).
provider that refreshes first. Targets without fresh quota data, and ties, keep configuration
order. Weights and `stickyLimit` do not affect this strategy.

This ranking and provider exclusion before dispatch require fresh model-inference limits that apply to the current single API key as a whole. OAuth/current-account summaries, caller-forward routes, multiple keys, and snapshots with changed credentials or destinations are display-only for this early decision. The same applies when `Authorization`, `x-api-key`, or `x-goog-api-key` headers override credentials; search-only and MCP-only windows are excluded. If no eligible target has an applicable reset, configuration order wins. Account selection and retries still enforce their normal limits.

## What happens when a target fails

Combo failures are divided into **hop** failures and **terminal** failures.
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ja/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ ocx combo set balanced \

`reset-window` は、キャッシュされたプロバイダーのクォータスナップショットで、次回のウィンドウリセット(5 時間、週次、月次、またはカスタム)が最も早い適格なターゲットへ、各リクエストをルーティングします。これにより、最初にクォータが補充されるプロバイダーを先に使用します。新しいクォータデータがないターゲットと、リセット時刻が同じターゲットでは、構成順序が維持されます。`weight` と `stickyLimit` はこの戦略に影響しません。

この順位付けと送信前のプロバイダー除外には、現在の単一 API キー全体に適用される最新のモデル推論制限が必要です。OAuth/現在のアカウントの概要、呼び出し元の認証情報を転送するルート、複数キー、認証情報や送信先が変わったスナップショットは、この事前判断では表示専用です。`Authorization`、`x-api-key`、`x-goog-api-key` ヘッダーで認証情報を上書きする場合も同様で、検索専用および MCP 専用ウィンドウは対象外です。適用可能なリセット情報を持つ適格な対象がなければ、設定順序を使用します。アカウント選択と再試行には引き続き通常の制限が適用されます。

## ターゲットが失敗すると何が起こるか

コンボ障害は、**ホップ** 障害と **ターミナル** 障害に分類されます。
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ko/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,8 @@ ocx combo set balanced \

`reset-window`는 캐시된 공급자 할당량 스냅샷에서 가장 가까운 다음 기간 재설정(5시간, 주간, 월간 또는 사용자 지정)이 표시되는 적합한 대상으로 각 요청을 라우팅합니다. 이렇게 하면 가장 먼저 새로 충전되는 공급자를 사용합니다. 최신 할당량 데이터가 없는 대상과 동률인 대상은 설정 순서를 유지합니다. `weight`와 `stickyLimit`은 이 전략에 영향을 주지 않습니다.

이 순위 결정과 전송 전 공급자 제외에는 현재 단일 API 키의 전체 모델 추론에 적용되는 최신 한도 정보가 필요합니다. OAuth·현재 계정 요약, 호출자 인증을 전달하는 경로, 여러 키, 인증 정보나 목적지가 달라진 스냅샷은 이 사전 판단에서 표시 용도로만 사용합니다. `Authorization`, `x-api-key`, `x-goog-api-key` 헤더로 인증을 덮어쓰는 경우도 같으며, 검색 전용·MCP 전용 기간은 제외합니다. 적격 대상 중 적용 가능한 초기화 정보가 없으면 설정 순서를 따릅니다. 실제 계정 선택과 재시도에는 기존 제한이 계속 적용됩니다.

## 대상 실패 시 동작

콤보 실패는 **홉** 실패와 **종결** 실패로 나뉩니다.
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/ru/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,8 @@ ocx combo set balanced \
данных о квоте, а также цели с одинаковым временем сброса сохраняют порядок конфигурации. Значения
`weight` и `stickyLimit` не влияют на эту стратегию.

Для этого ранжирования и исключения провайдеров до отправки нужны свежие лимиты инференса моделей, применимые к единственному текущему API-ключу в целом. Сводки OAuth и текущего аккаунта, маршруты с передачей учётных данных вызывающей стороны, несколько ключей и снимки с изменившимися учётными данными или адресом назначения служат только для отображения при этом предварительном решении. То же относится к переопределению учётных данных заголовками `Authorization`, `x-api-key` или `x-goog-api-key`; окна только для поиска или MCP исключаются. Если ни у одной допустимой цели нет подходящего времени сброса, используется порядок конфигурации. При выборе аккаунта и повторных попытках по-прежнему действуют обычные ограничения.

## Что происходит, когда цель сбоит

Сбои в combo делятся на **hop**-сбои и **terminal**-сбои.
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/tr/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,8 @@ kullanılır. Güncel kota verisi bulunmayan hedeflerde ve eşitliklerde
yapılandırma sırası korunur. `weight` değerleri ve `stickyLimit` bu stratejiyi
etkilemez.

Bu sıralama ve gönderim öncesi sağlayıcı elemesi, mevcut tek API anahtarının model çıkarımı kullanımının tamamına uygulanan güncel sınırlara dayanır. OAuth veya geçerli hesap özetleri, çağıranın kimlik bilgilerini ileten rotalar, birden fazla anahtar ve kimlik bilgileri ya da hedefi değişmiş anlık görüntüler, bu ön kararda yalnızca görüntüleme amaçlıdır. `Authorization`, `x-api-key` veya `x-goog-api-key` başlıkları kimlik bilgilerini geçersiz kıldığında da aynı kural uygulanır; yalnızca arama veya MCP için olan pencereler hariç tutulur. Uygun hedeflerin hiçbirinde geçerli sıfırlama bilgisi yoksa yapılandırma sırası kullanılır. Hesap seçimi ve yeniden denemelerde normal sınırlar uygulanmaya devam eder.

## Bir hedef başarısız olduğunda ne olur?

Kombo hataları **atlama (hop)** hataları ve **uç (terminal)** hatalar olarak
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/zh-cn/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,8 @@ ocx combo set balanced \

`reset-window` 会将每个请求路由到合格目标中,其缓存的提供商额度快照显示下一个窗口最早重置者(五小时、每周、每月或自定义窗口)。这样会优先消耗最先刷新额度的提供商。没有最新额度数据的目标以及并列目标会保持配置顺序。`weight` 和 `stickyLimit` 不影响此策略。

此排序和发送前的提供商排除,需要适用于当前单个 API 密钥全部模型推理的最新限额信息。OAuth/当前账户摘要、转发调用方凭据的路由、多密钥以及凭据或目标地址已改变的快照,在这项提前判断中仅供显示。通过 `Authorization`、`x-api-key` 或 `x-goog-api-key` 请求头覆盖凭据时也适用相同规则;仅用于搜索或 MCP 的窗口不参与判断。如果所有符合条件的目标都没有适用的重置时间,则按配置顺序选择。实际账户选择和重试仍执行正常限制。

## 目标失败时会发生什么

combo 失败分为 **跳转** 失败和 **终止** 失败。
Expand Down
2 changes: 2 additions & 0 deletions docs-site/src/content/docs/zh-tw/guides/combos.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,6 +154,8 @@ ocx combo set balanced \

`reset-window` 將每個請求路由至快取供應商配額快照顯示下一個時段最早重設的合格目標(五小時、每週、每月或自訂)。這會優先使用最早重新取得額度的供應商。沒有最新配額資料的目標,以及發生平手時,皆維持設定順序。`weight` 與 `stickyLimit` 不影響此策略。

此排序與傳送前的供應商排除,需要適用於目前單一 API 金鑰全部模型推論的最新限額資訊。OAuth/目前帳戶摘要、轉送呼叫者憑證的路由、多金鑰,以及憑證或目的地位址已變更的快照,在這項預先判斷中僅供顯示。透過 `Authorization`、`x-api-key` 或 `x-goog-api-key` 標頭覆寫憑證時也適用相同規則;僅供搜尋或 MCP 使用的時段不參與判斷。若所有符合條件的目標都沒有適用的重設時間,則依設定順序選擇。實際帳戶選擇與重試仍套用一般限制。

## 目標失敗時會發生什麼

Combo 失敗分為**跳轉**失敗與**終端**失敗。
Expand Down
14 changes: 7 additions & 7 deletions src/combos/resolve.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import type { OcxComboTarget, OcxConfig } from "../types";
import { getCachedProviderQuota } from "../providers/quota-routing-cache";
import { getCachedProviderRoutingQuota } from "../providers/quota-routing-cache";
import type { ProviderQuota } from "../providers/quota-types";
import { isCanonicalOpenAiForwardProvider } from "../providers/openai-tiers";
import { sleepWithAbort } from "../lib/upstream-retry";
import {
coolComboTarget,
Expand Down Expand Up @@ -65,9 +64,7 @@ function targetProviderIsUsable(config: OcxConfig, target: OcxComboTarget, now:
if (!Object.hasOwn(config.providers, target.provider)) return false;
const provider = config.providers[target.provider];
if (!provider || provider.disabled === true) return false;
// Native account selection owns model-scoped quota; a provider summary cannot veto it.
return isCanonicalOpenAiForwardProvider(provider)
|| !cachedProviderQuotaIsExhausted(getCachedProviderQuota(target.provider, now), now);
return !cachedProviderQuotaIsExhausted(getCachedProviderRoutingQuota(target.provider, provider, now), now);
}

function quotaWindowExhausted(percent: number | undefined, resetAt: number | undefined, now: number): boolean {
Expand Down Expand Up @@ -129,6 +126,7 @@ function smoothWeightedIndex(
* unknown (Infinity).
*/
function resetWindowIndex(
config: OcxConfig,
targets: Required<OcxComboTarget>[],
eligible: (target: Required<OcxComboTarget>) => boolean,
now = Date.now(),
Expand All @@ -138,7 +136,9 @@ function resetWindowIndex(
for (let index = 0; index < targets.length; index++) {
const target = targets[index]!;
if (!eligible(target)) continue;
const remaining = quotaResetRemainingMs(getCachedProviderQuota(target.provider, now), now);
const remaining = quotaResetRemainingMs(
getCachedProviderRoutingQuota(target.provider, config.providers[target.provider], now), now,
);
// Strict comparison deliberately retains configured order for ties,
// including the no-snapshot fallback where every value is Infinity.
if (selected < 0 || remaining < smallestRemaining) {
Expand Down Expand Up @@ -224,7 +224,7 @@ export function pickComboTarget(
}
}
} else if (combo.strategy === "reset-window") {
targetIndex = resetWindowIndex(combo.targets, eligible, now);
targetIndex = resetWindowIndex(config, combo.targets, eligible, now);
} else {
targetIndex = combo.targets.findIndex(eligible);
}
Expand Down
67 changes: 62 additions & 5 deletions src/providers/quota-routing-cache.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,53 @@
import { createHash } from "node:crypto";
import type { OcxProviderConfig } from "../types";
import type { ProviderQuota, ProviderQuotaReport } from "./quota";
import { providerUsesKeyAuthOverride, resolveProviderApiKey } from "./key-store";
import { getProviderRegistryEntry } from "./registry";

const quotaCache = new Map<string, ProviderQuota>();
export interface ProviderQuotaRoutingEvidence {
quota: ProviderQuota;
binding: string;
}

type CachedQuota = {
quota: ProviderQuota;
routing?: ProviderQuotaRoutingEvidence | { quota: ProviderQuota; testOnly: true };
};

const quotaCache = new Map<string, CachedQuota>();

/** Private cache identity; neither key material nor this digest enters management reports. */
export function providerQuotaRoutingBinding(
name: string,
provider: OcxProviderConfig,
credential = resolveProviderApiKey(provider.apiKey)?.trim(),
): string | null {
if ((provider.authMode ?? "key") !== "key" || !credential) return null;
// Registry-owned OAuth/forward rows normalize saved authMode before dispatch.
// A key probe must not constrain that later account selection.
const entry = getProviderRegistryEntry(name);
if (entry && (entry.authKind === "oauth" || entry.authKind === "forward")
&& !providerUsesKeyAuthOverride(entry, provider, credential)) return null;
// Static auth headers can replace or combine with the probed API-key header.
// Its semantics belong to the adapter, so it is not provider-wide quota evidence.
if (Object.keys(provider.headers ?? {}).some(header =>
["authorization", "x-api-key", "x-goog-api-key"].includes(header.toLowerCase()))) return null;
return createHash("sha256").update(JSON.stringify([
name, provider.adapter, provider.baseUrl, credential,
])).digest("hex");
}

export function clearCachedProviderQuotas(): void {
quotaCache.clear();
}

export function replaceCachedProviderQuotas(reports: ProviderQuotaReport[]): void {
export function replaceCachedProviderQuotas(
reports: ProviderQuotaReport[],
routingEvidence?: WeakMap<ProviderQuotaReport, ProviderQuotaRoutingEvidence>,
): void {
quotaCache.clear();
for (const report of reports) {
quotaCache.set(report.provider, report.quota);
quotaCache.set(report.provider, { quota: report.quota, routing: routingEvidence?.get(report) });
}
}

Expand All @@ -18,15 +56,34 @@ export function getCachedProviderQuota(
now: number,
maxAgeMs = 30 * 60_000,
): ProviderQuota | null {
const quota = quotaCache.get(provider);
const quota = quotaCache.get(provider)?.quota;
if (!quota) return null;
if (now - quota.updatedAt > maxAgeMs) return null;
return quota;
}

/** Only inference-wide evidence for this sole credential may rank or veto a whole provider. */
export function getCachedProviderRoutingQuota(
name: string,
provider: OcxProviderConfig | undefined,
now: number,
maxAgeMs = 30 * 60_000,
): ProviderQuota | null {
if (!provider || provider.disabled === true || (provider.authMode ?? "key") !== "key") return null;
// An active-key report cannot speak for the other keys the dispatcher may select.
if ((provider.apiKeyPool?.length ?? 0) > 1) return null;
const routing = quotaCache.get(name)?.routing;
if (!routing || now - routing.quota.updatedAt > maxAgeMs) return null;
const binding = providerQuotaRoutingBinding(name, provider);
if (!binding || (!("testOnly" in routing) && routing.binding !== binding)) return null;
return routing.quota;
}

export function setCachedProviderQuotaForTests(
provider: string,
quota: ProviderQuota,
): void {
quotaCache.set(provider, quota);
// Unit tests deliberately assert the supplied quota's scope. Production publication
// requires the producer's private, credential-bound evidence map above.
quotaCache.set(provider, { quota, routing: { quota, testOnly: true } });
}
Loading
Loading