Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs-site/src/content/docs/reference/configuration/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ predictions. Explicit provider/model price overrides still take precedence.
| --- | --- | --- |
| `adapter` | `string` | One of `openai-chat`, `openai-responses`, `anthropic`, `google`, `kiro`, `cursor`, `ollama-native`, `azure-openai` (or alias `azure`). |
| `baseUrl` | `string` | Upstream API base URL. Most built-in fixed endpoints ignore a mismatch; collision-safe key presets preserve an older same-named custom destination. |
| `proxy?` | `string` | Optional provider-specific HTTP(S) proxy URL. When omitted, the provider inherits the server-level `proxy`, `noProxy`, and global `proxy: "auto"` behavior. Only `http://` and `https://` URLs are accepted; provider-level `auto`, `direct`, `null`, empty/whitespace values, SOCKS, and malformed URLs are rejected. |
| `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | Optional client-side outbound request-start pacing, separate from upstream usage, billing, and rate-limit indicators. RPM is converted to an even interval; `minIntervalMs` may impose a longer interval. Provider limits apply across all models, while `models` entries use exact upstream model IDs (for example `nvidia/llama-3.1-nemotron-ultra-253b-v1`) and can only add delay. Queue waits do not consume the upstream response-header timeout. HTTP, Responses WebSocket, and explicit adapter `fetchResponse`/`runTurn` dispatches are covered. |
| `upstreamHttpVersion?` | `"auto" \| "http1.1" \| "h1" \| "http2" \| "h2"` | Pin the HTTP version used for upstream requests to this provider. Defaults to `auto`, which lets Bun negotiate. An explicit pin requires an HTTPS target and fails locally when it cannot be honored. Set `http1.1` when a provider's HTTP/2 SSE stream stalls instead of delivering events — the symptom is a long-running streaming request that produces nothing and eventually times out. For Cursor, `http1.1`/`h1` selects its `RunSSE` + `BidiAppend` compatibility transport for inference and also pins live model discovery. Management `POST`/`PATCH` accept `null` to clear it back to `auto`. |
| `responsesPath?` | `string` | Relative resource path for key-auth `openai-responses` requests. It must start with `/` and contain no scheme, query, or fragment. |
Expand Down Expand Up @@ -209,6 +210,27 @@ to the native default as a single choice. Defaults must belong to the final list
the catalog projection, not stored configuration or arbitrary gateway models sharing a GPT name.
See [custom native catalog examples](/guides/codex-app-models/).

### Per-provider HTTP(S) proxy

Set `providers.<name>.proxy` when one provider should use a different HTTP(S) proxy from
the server-level route:

```jsonc
{
"providers": {
"deepseek": {
"proxy": "http://127.0.0.1:7897"
}
}
}
```

An explicit provider proxy is applied to the provider's core HTTP/SSE and provider outbound
requests. On the verified HTTP/HTTPS fetch paths, it takes precedence over the inherited
global `NO_PROXY` decision. Provider-level direct routing, automatic proxy discovery, and
`null` are not supported; omit the field to inherit the server-level behavior. Proxy credentials
may be included in the URL and are redacted from dashboard responses.

### Discovered model display names

Use `modelDisplayNames` when a provider returns machine friendly ids but the Codex model picker
Expand Down
3 changes: 3 additions & 0 deletions scripts/test-layout/layout.json
Original file line number Diff line number Diff line change
Expand Up @@ -962,6 +962,9 @@
"provider-discovery-log-suppression.test.ts": "providers",
"provider-id-rewrite.test.ts": "providers",
"provider-key-store.test.ts": "providers",
"provider-egress-fetch.test.ts": "responses",
"provider-egress-outbound.test.ts": "providers",
"provider-egress.test.ts": "lib",
"provider-live-models.test.ts": "providers",
"provider-model-aliases.test.ts": "providers",
"provider-model-discovery-contract.test.ts": "providers",
Expand Down
11 changes: 11 additions & 0 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
positiveIntegerRecordConfigError,
providerBaseUrlConfigError,
providerHeadersConfigError,
providerProxyConfigError,
reasoningSummaryDeliveryRecordConfigError,
upstreamHttpVersionConfigError,
} from "./config/provider-validation";
Expand Down Expand Up @@ -548,6 +549,8 @@ const providerConfigSchema = z.object({
decodesNativeCompactionBlobs: z.boolean().optional(),
allowEncryptedV2AgentTasks: z.boolean().optional(),
allowPrivateNetwork: z.boolean().optional(),
// Optional provider HTTP(S) proxy override; absent inherits global routing.
proxy: z.string().min(1).optional(),
// The management API accepts `null` as "clear this", so a config written before the POST
// canonicalization below can hold one on disk. Rejecting it here would send the operator
// through invalid-config recovery for a value the API told them was fine.
Expand Down Expand Up @@ -1343,6 +1346,14 @@ const configSchema = z.object({
message: responsesPathError,
});
}
const proxyError = providerProxyConfigError((provider as { proxy?: unknown }).proxy);
if (proxyError) {
ctx.addIssue({
code: "custom",
path: ["providers", redactSecretString(name), "proxy"],
message: proxyError,
});
}
const headersError = providerHeadersConfigError((provider as { headers?: unknown }).headers);
if (headersError) {
ctx.addIssue({
Expand Down
19 changes: 19 additions & 0 deletions src/config/provider-validation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,25 @@ export function upstreamHttpVersionConfigError(value: unknown): string | null {
return null;
}

/** Validates an optional provider HTTP(S) proxy URL. */
export function providerProxyConfigError(value: unknown): string | null {
if (value === undefined) return null;
if (typeof value !== "string") return "proxy must be a string URL or omitted";
const trimmed = value.trim();
if (!trimmed) return "proxy must not be empty; omit the field to inherit the global proxy behavior";
const lowered = trimmed.toLowerCase();
if (lowered === "direct") return "proxy direct is not supported in Phase 1A; use global noProxy for direct routes";
if (lowered === "auto") return "proxy auto is not supported in Phase 1A; omit the field to inherit the global proxy behavior";
let protocol: string;
try {
protocol = new URL(trimmed).protocol;
} catch {
return "proxy must be a valid http(s) proxy URL";
}
if (protocol !== "http:" && protocol !== "https:") return "proxy must be an http(s) proxy URL";
return null;
}

export function positiveIntegerRecordConfigError(value: unknown, field: string): string | null {
if (value === undefined) return null;
if (!value || typeof value !== "object" || Array.isArray(value)) return `${field} must be a plain object`;
Expand Down
116 changes: 116 additions & 0 deletions src/lib/provider-egress.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
// Resolves provider-scoped HTTP(S) proxy overrides.
import type { OcxProviderConfig } from "../types";

export class InvalidProviderEgressError extends Error {
override readonly name = "InvalidProviderEgressError";
}

export interface ProviderEgressContext {
providerName: string;
modelId?: string;
provider: Pick<OcxProviderConfig, "proxy">;
url: string | URL;
purpose?: string;
}

export type ProviderEgress =
| { kind: "inherit" }
| { kind: "proxy"; proxyUrl: string; routeKey: string };

function egressFailure(providerName: string, reason: string, purpose?: string): never {
const scope = purpose ? " (" + purpose + ")" : "";
throw new InvalidProviderEgressError(
"providers." + providerName + ".proxy is invalid" + scope + ": " + reason + ". " +
"Phase 1 supports only an explicit http(s) proxy URL; omit the field to inherit global behavior."
);
}

function fnv1aHex(input: string): string {
let hash = 0x811c9dc5;
for (let i = 0; i < input.length; i++) {
hash ^= input.charCodeAt(i);
hash = Math.imul(hash, 0x01000193);
}
const unsigned = hash >>> 0;
return unsigned.toString(16).padStart(8, "0");
}

// Builds a credential-free connection reuse key.
export function providerEgressRouteKey(proxyUrl: string): string {
const parsed = new URL(proxyUrl);
const port = parsed.port || (parsed.protocol === "https:" ? "443" : "80");
return "proxy|" + parsed.protocol + "//" + parsed.hostname.toLowerCase() + "|" + port + "|" + fnv1aHex(proxyUrl);
}

// Returns a credential-free proxy origin for logs.
export function sanitizeProxyUrlForLog(proxyUrl: string): string {
try {
return new URL(proxyUrl).origin;
} catch {
return "<unparseable-proxy-url>";
}
}

export function describeProviderEgressForLog(egress: ProviderEgress): string {
if (egress.kind === "inherit") return "inherit";
return "proxy(" + sanitizeProxyUrlForLog(egress.proxyUrl) + " route=" + egress.routeKey + ")";
}

export function egressRequestUrl(input: string | URL | Request): URL | null {
try {
if (typeof input === "string") return new URL(input);
if (input instanceof URL) return new URL(input.toString());
return new URL(input.url);
} catch {
return null;
}
}

function parseExplicitProxyUrl(providerName: string, trimmed: string, purpose?: string): URL {
let parsed: URL;
try {
parsed = new URL(trimmed);
} catch {
return egressFailure(providerName, "proxy is not a parseable absolute URL", purpose);
}
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
return egressFailure(
providerName,
"unsupported proxy scheme " + sanitizeProxyUrlForLog(trimmed) + "; Phase 1 supports only http(s)",
purpose
);
}
return parsed;
}

export function resolveProviderEgress(context: ProviderEgressContext): ProviderEgress {
const providerName = context.providerName;
const provider = context.provider;
const purpose = context.purpose;
const raw = provider.proxy;
if (raw === undefined) return { kind: "inherit" };
if (raw === null) {
return egressFailure(providerName, "proxy null is not accepted; omit the field to inherit or use an explicit http(s) proxy URL", purpose);
}
if (typeof raw !== "string") {
return egressFailure(providerName, "proxy must be a string URL or omitted", purpose);
}
const trimmed = raw.trim();
if (trimmed.length === 0) {
return egressFailure(providerName, "empty proxy value is not DIRECT; omit the field to inherit", purpose);
}
const lowered = trimmed.toLowerCase();
if (lowered === "direct") {
return egressFailure(providerName, "direct has no safe request-scoped transport on this runtime; use global noProxy", purpose);
}
if (lowered === "auto") {
return egressFailure(providerName, "provider auto proxy is deferred in Phase 1A; omit the field to inherit the global proxy behavior", purpose);
}
const parsed = parseExplicitProxyUrl(providerName, trimmed, purpose);
const target = egressRequestUrl(context.url);
if (target === null) {
return egressFailure(providerName, "target URL is not parseable", purpose);
}
const proxyUrl = parsed.toString();
return { kind: "proxy", proxyUrl: proxyUrl, routeKey: providerEgressRouteKey(proxyUrl) };
}
27 changes: 20 additions & 7 deletions src/lib/provider-outbound.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,12 @@ import {
} from "./destination-policy";
import { pinnedHttpGet, pinnedHttpPost } from "./pinned-http";
import { effectiveProxyFor, noProxyMatches, normalizeProxyHostname, outboundProxyConfigured } from "./proxy-env";
import { InvalidProviderEgressError, resolveProviderEgress } from "./provider-egress";
import { publicProviderBaseUrl } from "./provider-url";

type ProviderGetInit = Omit<RequestInit, "body" | "method" | "redirect">;
type ProviderPostInit = ProviderGetInit & { body: string };
type ProviderOutboundConfig = Pick<OcxProviderConfig, "baseUrl" | "allowPrivateNetwork"> & {
type ProviderOutboundConfig = Pick<OcxProviderConfig, "baseUrl" | "allowPrivateNetwork" | "proxy"> & {
fetch?: typeof globalThis.fetch;
};
export interface ProviderOutboundDependencies {
Expand Down Expand Up @@ -116,7 +117,15 @@ async function providerOutboundRequest(
if (postUrl?.protocol !== undefined && postUrl.protocol !== "https:") {
throw new ProviderOutboundPolicyError("provider POST URL must use HTTPS");
}
// Resolve provider egress before DNS and transport policy.
const egress = resolveProviderEgress({ providerName: name, provider, url, purpose: "providerOutbound" });
if (provider.fetch) {
// Explicit routes require the built-in executor.
if (egress.kind === "proxy") {
throw new InvalidProviderEgressError(
"providers." + name + ".proxy cannot be honored through a caller-owned fetch executor."
);
}
// A caller-owned executor cannot be peer-pinned here. This branch keeps literal/config
// checks and redirect blocking, but does not provide the resolved-address guarantees of
// the built-in transport. Main-request migration must define that executor contract first.
Expand All @@ -138,13 +147,16 @@ async function providerOutboundRequest(
return provider.fetch(url, { ...init, method, redirect: "manual" });
}
const parsed = postUrl ?? new URL(url);
const proxyConfigured = outboundProxyConfigured();
// Explicit routes override inherited NO_PROXY selection.
const explicitProxy = egress.kind === "proxy" ? egress.proxyUrl : null;
const proxyConfigured = explicitProxy !== null || outboundProxyConfigured();
// Snapshot the scheme-matched proxy once, before the DNS await, so admission and transport
// below reason about the same value. `null` here means "no proxy fetch would actually use",
// even if some other proxy variable is set.
const effectiveProxy = effectiveProxyFor(parsed);
const effectiveProxy = explicitProxy ?? effectiveProxyFor(parsed);
const noProxyBypass = explicitProxy !== null ? false : noProxyMatches(parsed);
const isCanonicalUrl = dependencies.isCanonicalUrl ?? (() => false);
const allowMihomoIpv6FakeIp = (effectiveProxy !== null && !noProxyMatches(parsed))
const allowMihomoIpv6FakeIp = (effectiveProxy !== null && !noProxyBypass)
|| transparentFakeIpException(url, parsed, isCanonicalUrl, name);
const resolveAddresses = dependencies.resolveAddresses ?? resolvePublicAddresses;
const pinnedGet = dependencies.pinnedGet ?? pinnedHttpGet;
Expand All @@ -168,7 +180,7 @@ async function providerOutboundRequest(
// proof is on the final request URL — not the provider name — because an
// OAuth/forward name matches any baseUrl by design while the bearer is
// pinned to the registry destination independently.
allowBenchmarkAddresses: (proxyConfigured && !noProxyMatches(parsed))
allowBenchmarkAddresses: (proxyConfigured && !noProxyBypass)
|| transparentFakeIpException(url, parsed, isCanonicalUrl, name),
// Mihomo IPv6 fake-IP (fdfe:dcba:9876::/48) answers are admitted either when bound
// to a scheme-matched proxy (#3462) or under the TUN transparency exception for a
Expand All @@ -184,7 +196,8 @@ async function providerOutboundRequest(
if (!proxyConfigured) throw error;
warnProxyBoundaryOnce();
warnProxyDnsDegradationOnce();
return globalThis.fetch(url, { ...init, method, redirect: "manual" });
// Preserve an explicit provider route on the fallback request.
return globalThis.fetch(url, { ...init, method, redirect: "manual", ...(explicitProxy ? { proxy: explicitProxy } : {}) });
}
// A canonical TUN exception with no scheme-matched proxy must retain the
// validated address, even when an unrelated HTTP_PROXY/ALL_PROXY is present.
Expand All @@ -195,7 +208,7 @@ async function providerOutboundRequest(
const proxy = (allowMihomoIpv6FakeIp && effectiveProxy) ? effectiveProxy : undefined;
return globalThis.fetch(url, { ...init, method, redirect: "manual", ...(proxy ? { proxy } : {}) });
}
if (proxyConfigured && resolved.privateNetwork && !noProxyMatches(parsed)) {
if (proxyConfigured && resolved.privateNetwork && !noProxyBypass) {
const hostname = normalizeProxyHostname(parsed.hostname);
throw new Error(
`provider URL resolves to a private-network destination; add ${hostname} to NO_PROXY before using allowPrivateNetwork with an outbound proxy`,
Expand Down
8 changes: 8 additions & 0 deletions src/server/auth-cors.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import {
positiveIntegerRecordConfigError,
providerBaseUrlConfigError,
providerHeadersConfigError,
providerProxyConfigError,
reasoningSummaryDeliveryRecordConfigError,
upstreamHttpVersionConfigError,
} from "../config/provider-validation";
Expand Down Expand Up @@ -634,6 +635,11 @@ export function providerManagementConfigError(name: unknown, provider: unknown):
if (destinationError) return `provider ${name} ${destinationError}`;
const headersError = providerHeadersConfigError(typed.headers);
if (headersError) return `provider ${name} ${headersError}`;
// Reject provider proxy values the request-scoped transport cannot honor.
const proxyError = providerProxyConfigError(typed.proxy);
if (proxyError) {
return `provider ${JSON.stringify(redactSecretString(name))} ${proxyError}`;
}
const retryOn429Error = retryOn429PolicyConfigError(raw.retryOn429);
if (retryOn429Error) {
// The provider name is caller-controlled and can be token-shaped; redact and JSON-escape
Expand Down Expand Up @@ -789,6 +795,8 @@ const PROVIDER_CONFIG_FIELD_POLICY = {
allowEncryptedV2AgentTasks: "editor",
allowPrivateNetwork: "editor",
upstreamHttpVersion: "editor",
// Keep proxy credentials out of dashboard responses.
proxy: "redacted",
upstreamWebsocket: "editor",
directGeminiWireRenames: "editor",
disabled: "editor",
Expand Down
Loading
Loading