feat(proxy): add responses websocket fallback support

[ding113]

Summary

• Add /v1/responses WebSocket runtime support via the standalone Node upgrade wrapper while preserving ordinary HTTP /v1/responses behavior.
• Add Codex-only upstream Responses WebSocket transport with HTTP/SSE/non-stream fallback, FIFO per-socket queueing, store=false socket-scoped cache boundaries, and decision-chain metadata.
• Add global enableOpenAIResponsesWebSocket setting with admin UI/i18n and optional provider diagnostics metadata.

Problem

Codex CLI and other OpenAI-compatible clients are transitioning to WebSocket transport for the /v1/responses endpoint for real-time streaming. The proxy previously only supported HTTP/SSE transport for the Responses API, requiring all clients to use HTTP regardless of transport capability.

Related Issues:

• Resolves OpenaiWebSocket快发布吧 #1095 - Users requesting OpenAI WebSocket feature release

Related PRs:

• Supersedes feat(ws): add OpenAI Responses WebSocket support #887 - Previous WS implementation attempt (closed); this PR is a complete architectural rewrite with new src/server/ pipeline, FIFO queueing, and decision-chain observability
• Related to feat: add OpenAI Responses WebSocket transport for /v1/responses #882 - First WS implementation attempt (closed), superseded by feat(ws): add OpenAI Responses WebSocket support #887 and now this PR
• Related to Prevent Codex provider tests from misclassifying streamed success #1093 - Prior fix for Codex provider test SSE parsing; this PR extends provider testing with WebSocket-specific probing
• Related to fix: align non-chat endpoint fallback with raw policy #1091 - Prior work on non-chat endpoint fallback patterns in the same proxy area

Solution

Architecture

The implementation uses a standalone Node.js HTTP upgrade wrapper (scripts/responses-websocket-standalone-server.ts) that intercepts WebSocket upgrade requests on /v1/responses while forwarding all other requests to the Next.js server. The pipeline consists of:

1. Runtime (responses-websocket-runtime.ts) - Manages WebSocket server lifecycle, connection acceptance, and graceful shutdown
2. Protocol (responses-websocket-protocol.ts) - Frame validation, message parsing, and WebSocket protocol handling
3. Session State (responses-websocket-session-state.ts) - Per-socket FIFO queue with socket-scoped cache boundaries (store=false)
4. Proxy Executor (responses-websocket-proxy-executor.ts) - Bridges WS frames to the existing HTTP guard pipeline via loopback
5. Upstream Adapter (responses-websocket-upstream-adapter.ts) - Codex-only upstream WS connection with configurable timeouts
6. Fallback Bridge (responses-websocket-fallback-bridge.ts) - Graceful HTTP/SSE/non-stream fallback when upstream WS is unavailable

Key Design Decisions

• Upstream WebSocket fallback is treated as transport capability metadata and does NOT count as provider/endpoint/vendor breaker failure
• Decision chain records clientTransport, upstream WS attempt/connection, downgrade status/reason, queue wait, and cache hit metadata
• HTTP/SSE/non-stream fallback stays on existing Responses path and keeps client WebSocket open when safe

Changes

Core Changes (6 new files, ~3,032 lines)

• src/server/responses-websocket-upstream-adapter.ts (+884) - Outbound WebSocket adapter with timeout management and event collection
• src/server/responses-websocket-proxy-executor.ts (+673) - Proxy execution bridge between WS ingress and HTTP guard pipeline
• src/server/responses-websocket-protocol.ts (+589) - Frame validation and WebSocket protocol handling
• src/server/responses-websocket-runtime.ts (+414) - WebSocket server runtime and connection management
• src/server/responses-websocket-session-state.ts (+291) - Per-socket FIFO queue and session state
• src/server/responses-websocket-fallback-bridge.ts (+171) - HTTP/SSE/non-stream fallback bridging

Database / Schema

• drizzle/0099_violet_richard_fisk.sql - Migration: ALTER TABLE "system_settings" ADD COLUMN "enable_openai_responses_websocket" boolean DEFAULT true NOT NULL
• src/drizzle/schema.ts - Schema definition for new column

Settings UI & i18n

• src/app/[locale]/settings/config/_components/system-settings-form.tsx - Toggle in System Settings form
• messages/{en,ja,ru,zh-CN,zh-TW}/settings/config.json - i18n strings for all 5 languages

Provider Testing

• src/lib/provider-testing/responses-websocket-probe.ts (+130) - WebSocket capability probing with transport/handshake metadata
• src/lib/provider-testing/test-service.ts - Extended with WS test support
• src/lib/provider-testing/types.ts - New WS-specific types

Supporting Changes

• scripts/responses-websocket-standalone-server.ts - Standalone server entry point for WS upgrade
• scripts/smoke-responses-websocket-runtime.ts - Smoke test script
• scripts/copy-version-to-standalone.cjs - Extended to copy WS server assets
• src/app/v1/_lib/proxy/session.ts - Session context for WS transport
• src/repository/system-config.ts - Config persistence for new setting
• src/types/system-config.ts, src/types/message.ts - Type definitions
• src/lib/config/system-settings-cache.ts - Cache support for new setting

Breaking Changes

None. This is a fully backwards-compatible feature:

• Toggle defaults to true (enabled) but existing HTTP /v1/responses behavior is preserved as fallback
• Only Codex-style providers attempt WS first with automatic HTTP/SSE/non-stream fallback
• No changes to existing API contracts or response formats

Verification

• timeout 420s bun run build -> exit 0; standalone WebSocket wrapper installed.
• timeout 240s bun run lint -> exit 0; 8 existing warnings / 3 infos remain.
• timeout 240s bun run lint:fix -> exit 0; no safe fixes applied.
• timeout 240s bun run typecheck -> exit 0.
• timeout 420s bun run test -> exit 0; 558 files passed, 2 skipped; 5130 tests passed, 13 skipped.
• Final verification wave: F1 APPROVE, F2 APPROVE, F3 APPROVE, F4 APPROVE.

Testing

Automated Tests (13 new test files, ~3,852 lines)

• tests/unit/runtime/responses-websocket-runtime.test.ts (666 lines) - Runtime lifecycle
• tests/unit/runtime/responses-websocket-upstream-adapter.test.ts (795 lines) - Upstream WS adapter
• tests/unit/runtime/responses-websocket-proxy-executor.test.ts (364 lines) - Proxy executor
• tests/unit/runtime/responses-websocket-session-queue-cache.test.ts (324 lines) - Session queue/cache
• tests/unit/runtime/responses-websocket-codex-session-continuity.test.ts (361 lines) - Multi-turn continuity
• tests/unit/runtime/responses-websocket-contract.test.ts (238 lines) - Protocol contract
• tests/unit/runtime/responses-websocket-decision-chain-observability.test.ts (196 lines) - Observability
• tests/unit/runtime/responses-websocket-fallback-bridge.test.ts (148 lines) - Fallback bridging
• tests/unit/runtime/responses-websocket-inbound-handler.test.ts (133 lines) - Inbound frame handling
• tests/unit/runtime/responses-websocket-inbound-errors.test.ts (74 lines) - Error scenarios
• tests/unit/actions/system-config-openai-responses-websocket-setting.test.ts (200 lines) - Config persistence
• tests/unit/provider-testing-test-service.test.ts (164 lines) - Provider testing WS support
• tests/unit/settings/system-settings-form-openai-responses-websocket.test.tsx (188 lines) - UI toggle

Manual Testing

1. Toggle enableOpenAIResponsesWebSocket in Settings UI, verify persistence
2. Provider test with Codex provider shows WS transport metadata in result card
3. Existing HTTP /v1/responses POST route unaffected when toggle is on or off

Decision-chain / fallback notes

• WebSocket decisions record clientTransport, upstream WS attempt/connection, downgrade status/reason, queue wait, cache hit, and unsupported-cache hits.
• Upstream WebSocket unsupported/fallback is treated as transport capability metadata and does not count as provider/endpoint/vendor breaker failure.
• HTTP/SSE/non-stream fallback stays on the existing Responses path and keeps the client WebSocket open when fallback is safe.

UI evidence

• Settings toggle screenshot: https://i.111666.best/image/UF1FAJHTMVwitU0WSBVejf.png

Notes

• Branch was developed in isolated worktree claude-code-hub-responses-websocket-support.
• .sisyphus/evidence/* was used for local verification evidence and is intentionally not included in this PR.

---

Description enhanced by Claude AI

Greptile Summary

This PR adds WebSocket transport support for /v1/responses via a standalone Node.js upgrade-interceptor wrapper, a six-file pipeline (runtime → protocol → session state → proxy executor → upstream adapter → fallback bridge), a new enableOpenAIResponsesWebSocket system setting, and provider-testing WS capability probing. The implementation is architecturally complete with comprehensive test coverage and backward-compatible HTTP fallback.

• P1 — Probe timeout ineffective after handshake: NodeWebSocketConnection.readFrame() has no abort-signal listener; once a WS handshake succeeds the AbortController used for the probe deadline fires but never breaks out of the frame-reading loop, so a silent upstream causes runDefaultResponsesWebSocketProbe to block indefinitely.
• P2 — Provider chain deduplication bypassed for WS: isWebSocketDecision || … short-circuits the existing id + reason + attemptNumber deduplication guard; every inbound frame appends a new chain entry, growing providerChain at O(N) requests per session before serialisation to the database.

Confidence Score: 4/5

Safe to merge with one functional issue to address: the probe timeout is a no-op after WS handshake succeeds, which can cause provider tests to hang indefinitely.

One P1 (probe abort signal not wired into readFrame) caps the score at 4. The P2 (provider chain deduplication bypass) is a quality concern with no immediate data integrity impact. The core proxy path is well-tested and architecturally sound. Existing HTTP behaviour is fully preserved.

src/server/responses-websocket-upstream-adapter.ts (readFrame abort support), src/lib/provider-testing/responses-websocket-probe.ts (timeout propagation), src/app/v1/_lib/proxy/session.ts (deduplication bypass)

Important Files Changed

Filename	Overview
src/server/responses-websocket-upstream-adapter.ts	Outbound WS adapter with complete raw-socket implementation; probe abort signal not wired into post-handshake readFrame loop, causing probe timeout to be ineffective after connection succeeds.
src/lib/provider-testing/responses-websocket-probe.ts	Provider WS capability probe; AbortController timeout does not propagate into the frame-reading loop, so the probe can block indefinitely after a successful handshake.
src/server/responses-websocket-runtime.ts	WS server runtime with custom frame decoder; handles masking, fragmentation, control frames, and proper RFC 6455 close codes; no new issues found beyond those flagged in previous review threads.
src/server/responses-websocket-protocol.ts	Protocol parsing, FIFO queue, and inbound handler; collectResponsesWebSocketEvents fully buffers the AsyncIterable (previously flagged); no additional new issues.
src/server/responses-websocket-proxy-executor.ts	Bridges WS ingress to existing HTTP guard pipeline; type cast on enableOpenAIResponsesWebSocket was previously flagged; logic is otherwise sound with proper guard and fallback handling.
src/server/responses-websocket-fallback-bridge.ts	HTTP/SSE fallback bridge; response.text() full-buffering was previously flagged; no additional new issues.
src/server/responses-websocket-session-state.ts	Per-socket FIFO session state and store=false cache; hash-based validation and TTL/size limits look correct.
src/app/v1/_lib/proxy/session.ts	Adds WS decision metadata fields to provider chain; deduplication guard is completely bypassed for WS decisions, causing O(N) chain growth for multi-turn sessions.
src/repository/system-config.ts	Adds multi-tier fallback read/write for new enableOpenAIResponsesWebSocket column, consistent with existing pattern for column-missing graceful degradation.
scripts/responses-websocket-standalone-server.ts	Standalone Node.js wrapper that patches http.createServer to intercept WS upgrades; eval() workaround was flagged in a previous thread.
drizzle/0099_violet_richard_fisk.sql	Adds enable_openai_responses_websocket boolean column with DEFAULT true NOT NULL; migration is correct.

Sequence Diagram

sequenceDiagram
    participant C as Codex CLI (WS)
    participant RT as responses-websocket-runtime
    participant PH as ResponsesWebSocketInboundHandler
    participant PE as responses-websocket-proxy-executor
    participant UA as responses-websocket-upstream-adapter
    participant US as Upstream Provider (WS)
    participant FB as responses-websocket-fallback-bridge
    participant HTTP as Upstream Provider (HTTP/SSE)

    C->>RT: GET /v1/responses (WS upgrade)
    RT->>RT: RFC 6455 handshake + accept
    C->>RT: text frame (response.create JSON)
    RT->>PH: handleFrame(frame)
    PH->>PH: FIFO queue enqueue
    PH->>PE: executor(input)
    PE->>PE: GuardPipeline (auth/rate-limit)
    PE->>UA: createResponsesWebSocketUpstreamEventStream
    alt Codex provider + WS enabled + not cached-unsupported
        UA->>US: TCP/TLS connect + WS handshake
        US-->>UA: 101 Switching Protocols
        UA-->>PE: events AsyncIterable
        PE-->>PH: AsyncIterable (collectResponsesWebSocketEvents buffers all)
        PH-->>RT: ResponsesWebSocketJsonEvent[]
        RT-->>C: text frames (burst after full collection)
    else non-Codex / disabled / cached-unsupported
        UA-->>PE: skipped
        PE->>FB: httpFallback()
        FB->>HTTP: POST /v1/responses
        HTTP-->>FB: Response (response.text() buffers all)
        FB-->>PE: events (burst after full buffer)
        PE-->>PH: events
        PH-->>RT: frames
        RT-->>C: text frames
    end

Loading

Prompt To Fix All With AI

This is a comment left during a code review.
Path: src/lib/provider-testing/responses-websocket-probe.ts
Line: 28-56

Comment:
**Probe timeout doesn't interrupt frame reading after handshake**

`controller.abort()` is only wired into `createConnectedSocket` and `readHandshakeResponse` inside `openNodeWebSocketConnection`. Once both complete and a `NodeWebSocketConnection` is returned, the abort signal is no longer monitored — `NodeWebSocketConnection.readFrame()` only resolves on socket `data`, `close`, or `error` events, with no abort listener. If an upstream accepts the WS handshake but goes silent, `readFirstEvent` blocks indefinitely past the configured deadline, making the probe's timeout a no-op after a successful handshake.

The fix requires propagating the `signal` into the frame-reading loop inside `NodeWebSocketConnection`.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/server/responses-websocket-upstream-adapter.ts
Line: 659-671

Comment:
**`readFrame` has no abort signal support**

The `NodeWebSocketConnection.readFrame()` loop only wakes on socket events (`data`, `close`, `error`) — there is no abort signal listener. Once a connection is established, calling `signal.abort()` has no effect on a blocking `readFrame()` call. This is the root cause of the probe-timeout issue noted above. The signal needs to be plumbed into the waiter mechanism so an abort resolves the pending promise with an error.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/app/v1/_lib/proxy/session.ts
Line: 629-636

Comment:
**WebSocket decisions unconditionally bypass deduplication**

`isWebSocketDecision || …` short-circuits the entire deduplication guard, so every `persistResponsesWebSocketDecision` call — one per inbound frame — appends a new entry to `providerChain` regardless of whether the provider, reason, or attempt number changed. Over a long multi-turn WebSocket session the chain grows at O(N) in the number of requests, and all entries are later serialised with `updateMessageRequestDetails`. Consider applying the same `id + reason + attemptNumber` check for WS decisions, or tracking WS-specific metadata in a separate field rather than disabling deduplication entirely.

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (7): Last reviewed commit: "test(proxy): unify responses websocket f..." | Re-trigger Greptile}

Greptile also left 2 inline comments on this PR.