[Bug] Web Search/Vision Sidecar Backend Fixed to OpenAI·Anthropic - Routing Turns Fail with 499/502 When Limits are Exhausted

[jhste102lab]

Client or integration

Codex CLI (opencode-go 세션, glm-5.2 max 라우팅)

Area

Proxy and routing (Provider adapter 영역과 직결)

Summary

웹 검색 사이드카와 비전 사이드카의 backend가 openai / anthropic 두 값으로만 고정되어 있습니다. 그래서 메인 라우팅 모델이 OpenAI·Anthropic 외 provider(예: z.ai GLM-5.2)일 때, 검색/비전이 필요한 턴은 반드시 OpenAI(ChatGPT forward) 또는 Anthropic(stored OAuth) 사이드카로 실행되어야 합니다.

이때 ChatGPT/Claude 계정 한도가 소진되면 사이드카 호출이 약 200초간 매달리다 턴 전체가 499 / 502(드물게 504)로 실패합니다. 타 provider 모델은 사이드카 백엔드로 지정할 수 없고, 사이드카 백엔드가 불가용일 때 "해당 턴만 검색/비전 결과 없이 계속 진행" 같은 graceful degradation도 없어 작업이 아예 멈춥니다.

관측된 대시보드 로그(아래 Logs)의 201,434ms는 웹 검색 사이드카 기본 타임아웃 DEFAULT_TIMEOUT_MS = 200,000과 거의 일치하여, 사이드카 백엔드(ChatGPT forward) 과부하/한도 소진이 실패의 직접 원인일 확률이 높습니다. (정확한 최종 귀속은 해당 턴이 웹 검색/비전을 실제로 호출했는지 사이드카/포워드 계정 로그로 확인해야 합니다.)

Reproduction

1. 타 provider를 라우팅 모델로 설정 (예: z.ai GLM, adapter: "anthropic", defaultModel: "glm-5.2"). GLM은 비전 미지원이므로 noVisionModels: ["glm-5.2"]로 비전 사이드카 강제.
2. 사이드카 backend는 기본값(openai) → ChatGPT forward 계정 + 로그인에 의존.
3. ChatGPT(및/또는 Anthropic OAuth) 계정 한도를 소진해 사이드카 백엔드가 불가용/불건전 상태로 만듦.
4. glm-5.2 max에서 웹 검색을 트리거하는 opencode-go 턴(또는 이미지를 붙여넣어 비전 사이드카를 트리거) 실행.
5. 관찰: 턴이 약 130–201초 매달린 뒤 499(클라이언트 취소) 또는 502(upstream error) 반환. graceful skip 없음.

Version

2.7.35 (main @ 2.7.39 소스에서 동일 제약 확인)

Operating system

Ubuntu 24.04 (WSL2)

Provider and model

z.ai / glm-5.2 (라우팅 모델). 사이드카 백엔드: OpenAI(ChatGPT forward) 기본.

Logs or error output

오후 5:31:38  미보고  glm-5.2 max  opencode-go  ocx-mryoljdv-sc  499   130,206 ms
오후 5:28:17  미보고  glm-5.2 max  opencode-go  ocx-mryoh7rt-sb  502   201,434 ms

• 201,434 ms ≈ 웹 검색 사이드카 기본 타임아웃 DEFAULT_TIMEOUT_MS = 200,000(src/web-search/index.ts).

Root cause (소스 근거)

• 사이드카 backend 타입이 openai | anthropic으로만 고정:
  • src/types.ts:413 webSearchSidecar?: { backend?: "openai" | "anthropic"; model?: string }
  • src/types.ts:415 visionSidecar?: { backend?: "openai" | "anthropic"; model?: string }
  • src/types.ts:667(OcxVisionSidecarConfig) / src/types.ts:685(OcxWebSearchSidecarConfig) 동일.
• 대시보드 관리 API(PUT /api/sidecar-settings)가 위 값 외 백엔드를 명시적으로 400 거부:
  • src/server/management/config-routes.ts — "webSearch.backend must be openai, anthropic, or null", "vision.backend must be openai, anthropic, or null".
• 백엔드 분해도 위 두 값만:
  • src/web-search/index.ts resolveSidecarBackend() → "anthropic"(명시) 아니면 "openai".
  • src/vision/index.ts resolveVisionBackend() → 명시값 / anthropic OAuth 가용 여부 / else "openai".
  • OpenAI 백엔드는 ChatGPT forward provider + 로그인 필요, anthropic 백엔드는 stored anthropic OAuth 필요. GLM 등 타 provider로 사이드카를 라우팅할 provider: 선택자는 없음. (model 필드 주석: "must be a native ChatGPT model".)
• 사이드카/라우팅-모델 실패가 턴 전체 HTTP 상태로 전파:
  • src/web-search/loop.ts:319/341 → LoopError(499, "client closed request during web-search")
  • src/web-search/loop.ts:343 → LoopError(502, "Provider unreachable: …")
  • src/web-search/loop.ts:378 → RoutedModelInactivityError → LoopError(504, …)
  • src/web-search/loop.ts:379/380/389 → LoopError(502, "Provider stream error…" / terminal error)
• 이 한정의 배경: 사이드카 executor는 provider-native 서버사이드 기능(OpenAI 호스팅 web_search, Anthropic web_search_20250305 / 비전)을 직접 구현. devlog 260630_native-sidecar-parity/00_research.md도 "웹 검색 사이드카는 ChatGPT forward /responses 백엔드에서 실제 hosted web_search를 실행"한다고 기록.

Expected behaviour

1. Graceful degradation: 사이드카 백엔드가 불가용/한도 소진일 때 턴 전체를 실패시키지 않고, 해당 도구(검색/비전)만 실패-클로즈하고 턴은 계속 진행. 기존 비전 stripImagesInPlace 마커([image omitted: … vision sidecar is unavailable …])처럼 명시적 표식과 함께.
2. 임의 provider 지정(또는 문서화): 사이드카 백엔드를 설정된 provider(GLM/Gemini/Grok/DeepSeek 등)로 지정할 수 있거나, 라우팅 모델 자신의 provider가 검색/비전 역량을 지원하면 그 provider로 사이드카를 위임. 불가하다면 "사이드카는 OpenAI/Claude 계정 한도와 커플링된다"는 한도를 README/사이드카 문서에 명시.

Redacted configuration

{
  "providers": {
    "glm-zai": {
      "adapter": "anthropic",
      "defaultModel": "glm-5.2",
      "noVisionModels": ["glm-5.2"]
    }
  },
  "webSearchSidecar": { "backend": "openai", "model": "gpt-5.6-luna" },
  "visionSidecar": { "backend": "openai", "model": "gpt-5.4-mini" }
}

Screenshots and supporting files

대시보드 캡처(본문 Logs표 참조). 추가 첨부 필요 시 제공 가능.

Additional context

• 관련 기존 이슈 검색 결과: [Bug]: Vision sidecar is unusable from Codex App - noVisionModels catalog should advertise image input so images reach the proxy #349(Codex App 사전검사/noVisionModels 카탈로그 메타데이터 — 사이드카 백엔드 한정과는 별개), Claude Code WebSearch fails for routed GPT models with empty domain filters #381(라우팅 GPT의 WebSearch 도메인 필터 — 별개). 본 이슈는 "백엔드가 openai·anthropic으로 고정 + 한도 소진 시 전체 턴 499/502"라는 별개 결함/제약을 다룹니다.
• 영향: OpenAI/Claude 한도를 아끼려 GLM/DeepSeek 등 타 provider로 라우팅하는 정확히 그 사용자가, 한도 소진 순간 검색/비전 턴에서 막힘. 비전은 이미지 클라이언트 사전검사와 겹쳐 더 취약([Bug]: Vision sidecar is unusable from Codex App - noVisionModels catalog should advertise image input so images reach the proxy #349 참조).

Checks

• 기존 이슈와 문서를 검색했습니다. ([Bug]: Vision sidecar is unusable from Codex App - noVisionModels catalog should advertise image input so images reach the proxy #349, Claude Code WebSearch fails for routed GPT models with empty domain filters #381 확인 — 별개 결함.)
• 시크릿, 토큰, 계정 정보, 요청 자격증명, 개인정보를 제거했습니다.

[github-actions]

English translation (original: ko)

Client or integration

Codex CLI (opencode-go session, glm-5.2 max routing)

Area

Proxy and routing (directly related to Provider adapter area)

Summary

The backend of the web search sidecar and vision sidecar is fixed to only two values: openai / anthropic. Therefore, when the main routing model is a provider other than OpenAI·Anthropic (e.g., z.ai GLM-5.2), the turns requiring search/vision must necessarily be executed through the OpenAI (ChatGPT forward) or Anthropic (stored OAuth) sidecar.

At this time, if the ChatGPT/Claude account limit is exhausted, the sidecar call hangs for about 200 seconds, and the entire turn fails with 499 / 502 (rarely 504). Other provider models cannot be specified as the sidecar backend, and there is no graceful degradation like "proceeding with the turn without the search/vision results only for that turn" when the sidecar backend is unavailable, causing the work to stop completely.

The observed dashboard log (201,434ms below) is almost identical to the web search sidecar's default timeout DEFAULT_TIMEOUT_MS = 200,000, suggesting that overload/exhaustion of the sidecar backend (ChatGPT forward) is likely a direct cause of the failure. (The exact final attribution needs to be verified whether that turn actually called web search/vision via the sidecar/forward account logs.)

Reproduction

1. Set a different provider as the routing model (e.g., z.ai GLM, adapter: "anthropic", defaultModel: "glm-5.2"). Since GLM does not support vision, force the vision sidecar with noVisionModels: ["glm-5.2"].
2. Sidecar backend defaults to openai → relies on ChatGPT forward account + login.
3. Exhaust the limits of the ChatGPT (and/or Anthropic OAuth) account to make the sidecar backend unavailable/unhealthy.
4. Execute an opencode-go turn that triggers web search with glm-5.2 max (or paste an image to trigger the vision sidecar).
5. Observation: After about 130–201 seconds hanging, it returns 499 (client cancel) or 502 (upstream error). No graceful skip.

Version

2.7.35 (same constraints confirmed from main (at) 2.7.39 source)

Operating system

Ubuntu 24.04 (WSL2)

Provider and model

z.ai / glm-5.2 (routing model). Sidecar backend: OpenAI (ChatGPT forward) by default.

Logs or error output

5:31:38 PM  unreported  glm-5.2 max  opencode-go  ocx-mryoljdv-sc  499   130,206 ms
5:28:17 PM  unreported  glm-5.2 max  opencode-go  ocx-mryoh7rt-sb  502   201,434 ms

• 201,434 ms ≈ web search sidecar default timeout DEFAULT_TIMEOUT_MS = 200,000 (src/web-search/index.ts).

Root cause (source evidence)

• The sidecar backend type is fixed to openai | anthropic only:
  • src/types.ts:413 webSearchSidecar?: { backend?: "openai" | "anthropic"; model?: string }
  • src/types.ts:415 visionSidecar?: { backend?: "openai" | "anthropic"; model?: string }
  • src/types.ts:667(OcxVisionSidecarConfig) / src/types.ts:685(OcxWebSearchSidecarConfig) are the same.
• The dashboard management API (PUT /api/sidecar-settings) explicitly rejects any backends other than the above values with a 400:
  • src/server/management/config-routes.ts — "webSearch.backend must be openai, anthropic, or null", "vision.backend must be openai, anthropic, or null".
• The backend composition only includes these two values:
  • src/web-search/index.ts resolveSidecarBackend() → either explicitly "anthropic" or "openai".
  • src/vision/index.ts resolveVisionBackend() → explicit value / availability of anthropic OAuth / else "openai".
  • The OpenAI backend requires ChatGPT forward provider + login, and the anthropic backend requires stored anthropic OAuth. There is no selector to route the sidecar to other providers like GLM. (model field comment: "must be a native ChatGPT model.")
• Sidecar/routing-model failures propagate as the entire turn’s HTTP status:
  • src/web-search/loop.ts:319/341 → LoopError(499, "client closed request during web-search")
  • src/web-search/loop.ts:343 → LoopError(502, "Provider unreachable: …")
  • src/web-search/loop.ts:378 → RoutedModelInactivityError → LoopError(504, …)
  • src/web-search/loop.ts:379/380/389 → LoopError(502, "Provider stream error…" / terminal error)
• The background of this limitation: sidecar executor implements the provider-native server-side functionality (OpenAI-hosted web_search, Anthropic web_search_20250305 / vision) directly. The devlog 260630_native-sidecar-parity/00_research.md also records, "The web search sidecar runs the actual hosted web_search from the ChatGPT forward /responses backend."

Expected behaviour

1. Graceful degradation: When the sidecar backend is unavailable/exhausted, instead of failing the entire turn, only the tools (search/vision) should fail-close, and the turn should continue to progress with an explicit indication similar to the existing vision stripImagesInPlace marker ("[image omitted: … vision sidecar is unavailable …]").
2. Arbitrary provider specifying (or documentation): The sidecar backend can be specified to an established provider (GLM/Gemini/Grok/DeepSeek, etc.) or, if the routing model’s own provider supports search/vision capabilities, delegate the sidecar to that provider. If not possible, state in the README/sidecar document that "the sidecar is coupled with OpenAI/Claude account limits."

Redacted configuration

{
  "providers": {
    "glm-zai": {
      "adapter": "anthropic",
      "defaultModel": "glm-5.2",
      "noVisionModels": ["glm-5.2"]
    }
  },
  "webSearchSidecar": { "backend": "openai", "model": "gpt-5.6-luna" },
  "visionSidecar": { "backend": "openai", "model": "gpt-5.4-mini" }
}

Screenshots and supporting files

Dashboard capture (refer to the body Logs table). Additional attachments can be provided if necessary.

Additional context

• Related existing issue search results: [Bug]: Vision sidecar is unusable from Codex App - noVisionModels catalog should advertise image input so images reach the proxy #349 (Codex App pre-checks / noVisionModels catalog metadata — separate from sidecar backend limitation), Claude Code WebSearch fails for routed GPT models with empty domain filters #381 (Routing GPT's WebSearch domain filter — separate). This issue addresses the separate defect/limitation of "backend fixed to openai·anthropic + entire turn 499/502 upon limit exhaustion."
• Impact: Users who route to other providers like GLM/DeepSeek to conserve OpenAI/Claude limits are exactly the ones getting blocked at the moment of limit exhaustion during search/vision turns. Vision is further vulnerable due to overlap with the image client pre-check ([Bug]: Vision sidecar is unusable from Codex App - noVisionModels catalog should advertise image input so images reach the proxy #349 reference).

Checks

• I searched existing issues and documents. (Checked [Bug]: Vision sidecar is unusable from Codex App - noVisionModels catalog should advertise image input so images reach the proxy #349, Claude Code WebSearch fails for routed GPT models with empty domain filters #381 — separate defects.)
• I removed secrets, tokens, account information, request credentials, and personal information.

[github-actions]

Possibly related issues:

• [Bug]: Vision sidecar is unusable from Codex App - noVisionModels catalog should advertise image input so images reach the proxy #349
• [Bug]: Claude settings - Search/Vision Sidecar dropdowns clipped by container boundary, bottom options unselectable #340
• The call speed is super slow, but calling the GPT model directly without OpenCodex is very fast #338

Reason: The new issue discusses problems related to the Vision Sidecar feature, which aligns closely with existing issues regarding the usability and configuration of Vision Sidecar in the Codex App.

Detected automatically via GitHub Models.

[lidge-jun]

Thanks for the precise report — the 201,434ms observation was the key.

Root cause

The web-search sidecar's per-search deadline defaulted to 200s (DEFAULT_TIMEOUT_MS = 200_000 in src/web-search/index.ts). The executors already degrade an HTTP error or timeout into a failed tool result (they are "never throws"), but when the backend simply hangs on an exhausted account, one search burned the full ~200s. The client then cancels first (the turn surfaces as 499), or the forced answer-without-search routed iteration fails afterward (502). So the 499/502 was a symptom of the 200s wait, not a direct sidecar emission.

Fix (dev)

• Lowered the default sidecar search deadline 200s → 60s (hosted-search p90 is ~43s, so 60s bounds hangs while leaving tail margin). webSearchSidecar.timeoutMs still overrides it. The routed-model body-inactivity budget (routedModelStallTimeoutMs) is unchanged.
• Added a defensive guard around the sidecar dispatch so an unexpected throw degrades to a failed tool result, while a genuine client cancel still maps to 499.
• Redacted the Anthropic executor's echoed HTTP error body (it could contain header/token material).
• Docs synced (config reference EN + zh-CN, structure/04). Regression tests cover the 60s default and thrown-sidecar degradation (with a secret-non-leak assertion).

With this, a limit-exhausted/unavailable search backend degrades to a normal no-search answer within ~1 min instead of hanging the whole turn.

Follow-ups (the other half of your report)

You also asked for search backends beyond the fixed openai/anthropic pair — split into two tracked issues:

• feat(sidecar): add Exa and other search providers as web-search sidecar backends #414 — add Exa and other dedicated search providers as sidecar backends.
• feat(sidecar): investigate search-API-capable providers (e.g. Gemini) as web-search sidecar backends #415 — investigate/enable provider-native search APIs (Gemini grounding first) as sidecar backends.

If you can still reproduce a >60s hang after updating, please attach the dashboard per-request timing and whether the search or vision sidecar was active on that turn.

[lidge-jun]

Fixed by #416 (bound sidecar search deadline to 60s + graceful degradation). Merged to dev — will ship in the next release.