[Provider compatibility] Proxy-originated model discovery sends no User-Agent; WAF-fronted gateways reject it with HTTP 403

[mdwsk88]

Client or integration

Codex App

Provider or upstream service

AI2API - an internal corporate gateway speaking the OpenAI Responses API (provider config: adapter: "openai-responses", authMode: "key"). The endpoint is internal-only; there is no public hostname or public specification for the gateway. Its WAF rejects any request that carries no User-Agent header with HTTP 403 and an empty body.

OpenCodex version

2.59.0 (npm @bitkyc08/opencodex). Source-checked against the installed package: src/oauth/index.ts buildModelsRequest() sets only Authorization: Bearer ... for this adapter shape (plus registry static headers where applicable), and src/lib/provider-outbound.ts executes it through pinnedHttpGet() in src/lib/pinned-http.ts, a Node-style node:https request that adds no default User-Agent.

Endpoint or capability

Model discovery (GET {baseUrl}/models).

Current behaviour

After adding the provider, no models ever appear: the Models page shows no rows for it and the Codex catalog gains no entries. On 2.59.0 a newly registered key-auth provider starts with initialModelSelection.status: "pending", and a pending provider has all of its model rows hidden, so the visible symptom is "provider added but invisible" with no pointer to the discovery failure in the UI.

service.log repeats on every discovery pass:

[opencodex] Provider model discovery for "AI2API" failed with HTTP 403 [urlClass=provider-models, fallback=configured].

Root cause as observed: the gateway's WAF requires a User-Agent on every request. The discovery fetch is proxy-originated, so there is no client request to inherit a UA from, and the pinned Node-style transport sends none. Controlled comparison from the same machine, same credential, and same resolved address:

• curl (sends User-Agent: curl/x) - HTTP 200 with the full model list
• Bun fetch (sends User-Agent: Bun/x) - HTTP 200
• Node-style https.request with only Authorization: Bearer ... (what discovery sends today) - HTTP 403, empty body
• The same Node-style request plus any User-Agent value - HTTP 200

Inference through the proxy to the same provider works, consistent with #4699/#4702: applyCallerUserAgentFallback preserves the caller's User-Agent on upstream Responses requests. The gap is specific to the proxy-originated /models discovery fetch, which that fix cannot cover by design (no caller to inherit from).

Expected behaviour

Model discovery should succeed against gateways that require a User-Agent, or at minimum the failure should name the likely cause. Since discovery requests are proxy-originated rather than client-proxied, sending a default User-Agent on this path does not conflict with the wire-compatibility rationale in #1751, which declined a global product UA for client-fingerprinted traffic.

Minimal redacted request or reproduction

Provider config (redacted):

{
  "AI2API": {
    "adapter": "openai-responses",
    "baseUrl": "https://gateway.internal.example/v1",
    "authMode": "key",
    "allowPrivateNetwork": true
  }
}

Transport difference (redacted; same host, same credential, same resolved address):

# 1) curl - default UA present - HTTP 200
curl -H "Authorization: Bearer <redacted>" https://gateway.internal.example/v1/models

# 2) Node-style request with Authorization only - what discovery sends today
#    headers = { "Authorization": "Bearer <redacted>" }  ->  HTTP 403, empty body

# 3) Same Node-style request plus any User-Agent - HTTP 200
#    headers = { "Authorization": "Bearer <redacted>", "User-Agent": "opencodex" }

Workaround verified on 2.59.0 - a static provider header is merged into both discovery and inference requests:

{
  "headers": { "User-Agent": "opencodex" }
}

After this change discovery succeeded (54 models), initialModelSelection converged from pending to all-off (54 ids is above the 20-model threshold, so they all default to off), and the enabled models appeared in the live /v1/models and the Codex catalog.

Actual response or error

[opencodex] Provider model discovery for "AI2API" failed with HTTP 403 [urlClass=provider-models, fallback=configured].

The gateway's 403 body is empty (WAF-style rejection). No other error is surfaced anywhere; the provider simply has no models.

Upstream documentation

No public specification exists for the gateway (internal corporate deployment). The requirement was established empirically by the controlled client comparison above: requests without a User-Agent header are rejected with 403; requests with any User-Agent value are accepted. UA-based filtering is a common WAF/CDN posture, so this is not specific to one vendor.

Suggested mapping or implementation notes

• The fix(responses): preserve caller User-Agent #4702 caller-UA fallback cannot cover discovery: there is no caller to inherit a UA from on a proxy-originated fetch.
• [Feature] Add an OpenCodex-specific User-Agent header #1751 declined a global opencodex/x.y.z UA for client-proxied traffic to preserve client fingerprints. Discovery is not client-proxied traffic, so a default UA on this path (e.g. opencodex, matching the existing convention in src/oauth/github-copilot.ts, or opencodex/<version>) would not touch any client-fingerprinted path. A provider-configured headers value should still win, as it does today.
• Lower-cost alternatives if a default UA is still unwanted:
  1. When discovery fails with 403, mention the UA-requiring-gateway case. POST /api/providers/test currently reports only upstream model discovery returned 403, which gives no hint.
  2. Document/surface the provider headers workaround for WAF-fronted gateways in the provider form or docs.

Additional context and attachments

• Same gateway family as [Provider compatibility] Key-auth Responses gateway (Kimi K3) echoes hosted web_search as client function_call; webSearchBridge executor is Ollama-only #4429 (AI2API, internal corporate deployment), but a different layer: that report was about hosted-tool protocol expression on the inference path; this one is about the model-discovery transport.
• The initial-model-selection gating amplified the confusion: a pending provider hides all rows, so the symptom looks like "the provider has no models" rather than "discovery is being rejected".

Checks

• I searched existing provider and compatibility issues.
• The request and response were redacted.
• The expected behaviour is based on an upstream specification or a concrete client requirement.

[lidge-jun]

Closed by #5198, squash-merged into dev as 98b9b344e1. Carried from #5186 with the author credited.

Proxy-originated outbound now fills a default User-Agent, so a WAF-fronted gateway no longer sees a header-less request from model discovery and answers 403. An operator-set header still wins.

Evidence is exact-head hosted CI at 6861bc3fd087f4...: test 1/4 through 4/4, gates, structure gate, macos 1/2, plus the keyring and npm-global matrices, all successful. No local suite was run in this lane.