Skip to content

Throttled upgrades retry in a synchronised herd: the 403 path sends no retry-after header #174

Description

@bman654

Found while tracing the retry chain for #172 (connection pacing). This is a pre-existing
production issue on the real upgrade-403 path, not something #172 introduced, and it is likely a
larger real-world win than the pacer for the 403 case specifically — it improves recovery from
throttles that have already happened, rather than only reducing how often they happen.

What happens

When OpenAI's edge throttles a Responses WebSocket upgrade, failContext emits a synthetic error
frame carrying retry_after_seconds in the JSON body, and the Response itself is created with
only a content-type header.

Two things then combine:

  1. The SDK strips the field. @ai-sdk/openai's openaiErrorDataSchema is a strict z.object
    keeping only message, type, param and code. retry_after_seconds is dropped before
    anything sees it.
  2. The SDK's backoff reads headers, not the body. getRetryDelayInMs in ai looks at
    error.responseHeaders['retry-after'] / ['retry-after-ms'], and falls back to plain
    exponential backoff when neither is present.

Net effect: on a genuine throttle the SDK retries on the fixed 2s/4s/8s ladder with no jitter, so
every request in a fan-out that was 403'd retries at exactly the same instant, gets throttled
together, and repeats. That is the synchronised-herd problem, live today, on the path this whole area
is about.

(clodex's own sdkUpstreamErrorDetails does recover the hint — from the prose retry after Ns,
which is why that text is load-bearing — but that value goes downstream to Claude Code. It never
reaches the SDK's in-process backoff.)

The fix is small, and the value is already in hand

The 403 site already parses OpenAI's own header via
numericRetryAfterHeader(response.headers['retry-after']) and clamps it into retryAfterSeconds. It
simply never reaches the response headers where the SDK looks. Setting a real retry-after header on
the response converts the synchronised ladder into one spaced by each request's own hint.

#172 does exactly this for its synthetic refusals and it works — see pacedRefusalResponse in
src/oauth/responses-websocket.ts, where dropping or corrupting the header turns a test red. The 403
path should get the same treatment.

Why it was not done in #172

It changes behaviour on a pre-existing path that #172's review panel examined only as context, and it
needs its own test and mutation proof. Bolting it on at the end of that review would have required
re-reviewing everything.

Verification notes for whoever picks this up

  • Confirm the schema strip at node_modules/@ai-sdk/openai/dist/index.js (openaiErrorDataSchema)
    and the header lookup in node_modules/ai/dist/index.js (getRetryDelayInMs) — both were read
    directly rather than inferred.
  • A test should assert the header on the response, not just the frame body, since the body field is
    provably invisible to the SDK.
  • Worth checking whether the connection-limit branch and the other retryAfterSeconds sites in
    handleSocketMessage have the same gap.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions