499 client_closed_request errors during web-search tool usage

[fede-oss]

Description

When using the web-search tool (or when the model invokes web search during a conversation), the request fails with a 499 status code and client_closed_request error. The upstream error message is "client closed request during web-search".

Reproduction

1. Use a model that triggers web search (e.g. qwen3.8-max-preview via alibaba-token-plan-intl provider)
2. Send a request that requires web search
3. The request fails with status 499

Log entry

{
  "requestId": "ocx-ms2c6sko-10n",
  "timestamp": 1785102920232,
  "model": "qwen3.8-max-preview",
  "provider": "alibaba-token-plan-intl",
  "requestedModel": "alibaba-token-plan-intl/qwen3.8-max-preview",
  "requestedEffort": "high",
  "configuredServiceTier": "default",
  "resolvedModel": "qwen3.8-max-preview",
  "status": 499,
  "durationMs": 304120,
  "errorCode": "client_closed_request",
  "terminalStatus": "failed",
  "closeReason": "client_cancel",
  "upstreamError": "client closed request during web-search",
  "usageStatus": "unreported",
  "displayMetrics": {
    "tokPerSecond": { "kind": "unavailable", "reason": "usage_missing" },
    "cost": { "kind": "unavailable", "reason": "usage_missing" }
  }
}

Key details

• Status: 499 (client closed request)
• Error code: client_closed_request
• Close reason: client_cancel
• Upstream error: client closed request during web-search
• Duration: ~304s before failure
• Provider: alibaba-token-plan-intl
• Model: qwen3.8-max-preview

The request appears to be cancelled by the client while the upstream provider is still processing the web search, resulting in a 499 rather than a successful response.

[github-actions]

[Wibias]

Thanks for the detailed logs — the 499 / client_closed_request / client closed request during web-search shape matches how we intentionally classify this path today (see #158), so this does not look like a simple mislabel.

We checked current dev: this is not already fixed there, and #416's ~60s sidecar deadline would not explain a ~304s turn death. That duration lines up more with a ~300s wall-clock cancel (client abort or similar) than with the sidecar timeout or a routed 200s 504.

Before we dig into a deeper fix, could you share:

1. ocx version / git SHA
2. Client (Codex CLI / App / Cursor / other?)
3. Whether the web-search UI appeared, and roughly how many searches ran
4. The last SSE events before failure (especially response.incomplete vs response.failed)
5. Any custom stallTimeoutSec / webSearchSidecar config, plus ChatGPT forward health at the time

Happy to keep investigating once we have those details — leaving this open.

[fede-oss]

Thanks — I checked the local OCX data and the Codex App logs for the failing turn.

1. OCX version: 2.7.41 (installed package and running GUI). This npm install does not expose a gitHead.

2. Client: Codex App / Codex Desktop (originator: Codex Desktop, Codex App build reported 0.146.0-alpha.3.1 in the local session metadata).

3. Web-search UI: it did not appear. I therefore cannot give a UI-visible search count.

4. Last events / actual sequence: I do not see either response.incomplete or response.failed arriving from the SSE stream. Each request first got 200 OK with content-type: text/event-stream, then produced no further SSE activity until the Codex client logged:
   stream disconnected before completion: idle timeout waiting for SSE

   The same turn was retried automatically. I found four consecutive OCX request records with the same failure shape:

   • ocx-ms2bn77h-xs: 499 after 303,985 ms
   • ocx-ms2btpxv-yc: 499 after 303,914 ms
   • ocx-ms2c08ru-yy: 499 after 304,721 ms
   • ocx-ms2c6sko-10n: 499 after 304,120 ms

   Corresponding Codex log timestamps show retry 1 at 21:45:10, retry 2 at 21:50:14, retry 3 at 21:55:19, and retry 4 at 22:00:24, each specifically because of the SSE idle timeout.

5. Configuration / health: no custom stallTimeoutSec is present. The web-search sidecar is openai/gpt-5.6-luna (backend openai). During the incident window, the usage log contains many successful OpenAI requests, including multiple gpt-5.6-luna requests returning 200 in roughly 1.5–12.5s, so the ChatGPT forwarding path was generally healthy at that time.

This looks less like a single ~300s App wall-clock cancellation and more like the App's SSE idle timeout firing repeatedly while the routed web-search request never emits another event.

[Wibias]

Thanks — that reply is exactly what we needed, and it usefully revises the earlier ~300s wall-clock cancel read.

Updated read

The four back-to-back ~304s failures with Codex’s stream disconnected before completion: idle timeout waiting for SSE, no response.incomplete / response.failed, and no web-search UI appearing points less at a generic App cancel and more at:

Codex Desktop’s SSE idle timer firing while the turn emitted no further events after the initial 200 OK.

That is the failure mode OpenCodex’s response.heartbeat keep-alives are meant to prevent on the Responses bridge. So the open questions are now:

1. whether heartbeats are actually leaving the proxy on this web-search turn, and/or
2. whether Codex Desktop 0.146.0-alpha.3.1 re-arms its idle timer on response.heartbeat the same way the CLI path does.

The OCX 499 / client_closed_request / client closed request during web-search classification still looks correct as the logged shape once the client drops the stream (#158) — it does not by itself explain why the stream went silent for ~5 minutes.

One more artifact, if you can

For a single failing turn, can you confirm whether any of these frames appeared after 200 OK?

• response.created
• response.heartbeat
• response.output_item.* / text deltas
• anything else before the idle disconnect

A short Desktop/SSE log snippet or a note that zero events arrived after headers is enough. That splits “proxy never heartbeat’d / never started the bridge loop” from “Desktop ignored heartbeats.”

Leaving this open under needs-info until we have that.

[fede-oss]

I checked the raw Codex Desktop log DB and the rollout record for the specific failing request ocx-ms2c6sko-10n.

What is observable:

• The request started at 21:55:20.
• Desktop logged the HTTP response at 21:55:24 as 200 OK, content-type: text/event-stream.
• From that point until 22:00:24, there are no decoded response frames/items recorded for that request: no response.created, no response.heartbeat, no response.output_item.*, no text delta, and no terminal event.
• The next request-scoped record is the idle-timeout/retry exactly five minutes later:
  stream disconnected before completion: idle timeout waiting for SSE
• The rollout JSONL likewise contains no response item in that interval; its next entry is only the token_count emitted at the timeout/retry boundary.

One important limitation: these are Desktop's decoded-event/application logs, not a raw SSE byte capture. The retained OCX diagnostics for this old failed request contain only the final usage/error record, not an outbound-frame trace. So I can confirm zero events were surfaced/decoded by Desktop after the headers, but I cannot prove from the retained artifacts whether OCX sent no response.heartbeat frames or Desktop received them and did not recognize/log/re-arm on them.

A fresh reproduction with OCX outbound SSE-frame logging or a loopback packet capture would be needed to split those two cases definitively.

[Wibias]

Thanks — that is enough to move this off pure guessing.

What we can take as established

For ocx-ms2c6sko-10n, Desktop saw 200 OK / text/event-stream, then no decoded frames for ~5 minutes (response.created, response.heartbeat, deltas, terminal events all absent) until the SSE idle timeout. Your caveat is noted: this is decoded/application evidence, not a raw byte capture, so it still cannot prove “OCX sent nothing” versus “Desktop received frames and did not log/re-arm.”

Given that response.created is also missing, the stronger working hypothesis is now:

the Responses bridge never started writing frames on this web-search turn (or stalled before the first event), rather than “Desktop ignores heartbeats.” Heartbeat-ignore would more often still show response.created first.

The logged OCX 499 / client_closed_request / client closed request during web-search shape remains the correct downstream classification once Desktop drops the idle stream (#158); it is not the root cause of the five-minute silence.

Next step on our side

We will treat this as actionable and investigate the web-search / Alibaba + sidecar path for “SSE headers, then zero outbound frames until client idle cancel.”

A fresh repro with temporary OCX outbound SSE-frame logging (or a loopback capture) would still be useful if you hit it again — that is the only way to close the last send-vs-decode split — but it is no longer required to keep the issue blocked.

Clearing the needs-info wait on our side and leaving this open for investigation.

[Wibias]

Investigation landed a fix in #555: wire response.heartbeat keepalives no longer get suppressed by invisible adapter heartbeats during web-search buffering / raw-byte progress. That should stop Codex Desktop's ~5 min SSE idle timeout from firing while upstream is still alive.

[lidge-jun]

Fixed by PR #555, merged into dev at df57bd9. The bridge now separates upstream liveness from client-visible wire activity, so invisible adapter heartbeats during web-search buffering no longer suppress parser-ignored response.heartbeat frames. Verified on rebased head d2bef13 with local bridge test/typecheck/privacy/full pre-push gate, independent Sol audit, CodeRabbit, and hosted cross-platform CI; the initial macOS timeout was rerun and passed.