diff --git a/devlog/_plan/260912_audio_apis_stack/000_plan.md b/devlog/_plan/260912_audio_apis_stack/000_plan.md new file mode 100644 index 0000000000..e237ea9722 --- /dev/null +++ b/devlog/_plan/260912_audio_apis_stack/000_plan.md @@ -0,0 +1,43 @@ +# Standalone audio APIs + +Expose file transcription, streaming dictation and GPT-Live to external clients with OpenCodex credentials. Connections gains separate Dictation and Live Voice sections. Existing OpenAI account selection and transport lifecycle remain the integration points. + +## Loop contract + +Owner steering during wp1 C: no local Bun suites, product tests, typecheck, build or dependency installation. Push with --no-verify and use exact-head remote CI for remaining executable verification. This supersedes every local command example in the layer plans. Already completed checks are historical evidence only; interrupted/crashed checks are not passing evidence. All active task-owned local suites were stopped. Functional layer closure uses the completed source review and pre-restriction focused evidence; PR readiness and final completion retain the remote CI gate under wp3 publication. + +- Archetype: satisfy-spec, C4 API/auth and C3 dashboard integration. +- Trigger: owner requested both audio capabilities, inherited subagent verification and a published dependent PR stack. +- Goal: three independently reviewable ordinary PRs with protocol tests, documentation and usable client controls. +- Non-goals: merge, release, deployment, account configuration, microphone capture during agent QA, paid upstream probes, Responses protocol conversion, native GitHub stack registration. +- Verification: focused Bun transport tests observe mock upstream requests and real loopback sockets; typecheck observes tsconfig source includes; GUI build/lint/i18n and browser smoke observe rendered controls. Final review readiness also requires repository test runner and exact-head remote CI. +- Stop: all three PRs published with final-head evidence and no unresolved actionable independent findings. +- Artifacts: this unit for functional design; ignored .tmp/audio-security for trust-boundary working notes; session-bound goalplan and test receipts for orchestration. +- Outcomes: DONE means all criteria met; external dependency failures remain unmet with evidence. No invented budget exhaustion or scope reduction. +- Escalation: main resolves technical review findings; no additional authority for destructive operations or external messages. Main reclaims a failed delegated packet after two distinct agents; implementation delegation requires an explicit plan amendment. +- Resources: existing shell/Git/GitHub and inherited model tools; task worktree only; no new paid services, billable voice calls or personal recordings. User set no token, cost or wall-clock cap and allowed unrestricted parallel subagents. + +## Current ownership + +Source root: task-owned linked worktree on origin/dev ec065aa0c6fb46b376a2f01873bd677327b99150. Native session state stays in the original checkout. Existing uncommitted user work stays there. + +| Cycle | Plan | Branch / PR base | Output | +| --- | --- | --- | --- | +| wp0 | all documents in this unit | documentation checkpoint on first branch | audited complete roadmap | +| wp1 | 010_transcription.md | codex/audio-transcription -> dev | bounded file transcription and audio upstream contract | +| wp2 | 020_streaming_voice.md | codex/audio-streaming -> codex/audio-transcription | dictation stream and externally owned live sessions | +| wp3 | 030_connections.md | codex/audio-connections -> codex/audio-streaming | endpoint metadata, audio controls, examples and publication | + +All layers include their own regression coverage and source-of-truth sync. The branch base is the dependency edge; no branch is merged in this task. + +## Evidence and decisions + +Local Codex reference: 095da4b7e8b70b01afb5c6131ef926dcb8c0d85d, realtime default gpt-live-1-codex. Installed desktop package 26.908.40834 has separate file and dictation-stream transports. File API interoperability reference: Soju06/codex-lb 82567556f9f75ea13986667fc5282f035b7ca8d2, app/modules/proxy/api.py and app/core/clients/proxy.py. Its gpt-4o-transcribe subscription model is a compatibility identifier, not proof of the backend model. + +No native architect field is exposed by the host spawn schema. Main owns the plan; an inherited generic read-only design verifier supplies the consultation evidence. This is recorded as a transport deviation, not native architect completion. Independent A/code reviewers inherit the requested model/context; model-family independence is not claimed. + +Configuration alone cannot create the missing audio endpoints. Reuse the existing sidecar selection, admission, response envelope, stream limits and API workspace instead of a separate service or provider registry. + +## Progress + +wp0: roadmap locked after independent PASS and resolved design reflection. Documentation checkpoint 5e4ade8432; staged whitespace check passed. Production code unchanged. Next cycle executes 010_transcription.md; protocol and auth decisions remain the dependency foundation. diff --git a/devlog/_plan/260912_audio_apis_stack/002_review_disposition.md b/devlog/_plan/260912_audio_apis_stack/002_review_disposition.md new file mode 100644 index 0000000000..af90b33f1b --- /dev/null +++ b/devlog/_plan/260912_audio_apis_stack/002_review_disposition.md @@ -0,0 +1,7 @@ +# Roadmap review disposition + +Inherited design verification accepted the narrow sidecar reuse, bounded transport and call-lifetime decisions. Reflection found an ambiguous Location contract; 020 now requires a proxy-relative join path for external callers and independently trusted upstream destinations. Follow-up reflection: ALIGNED. + +Independent A round 1 returned FAIL on three concrete contract omissions. All accepted: 010 now specifies explicit-key resolution before loopback shortcut and stored-main Direct materialization under a lease; 020 specifies tagged key/native ownership on both listeners; HTTP-only AUTH_MATRIX gets protocol-correct multipart fixtures in api-key-attribution.test.ts, while WebSocket auth is separately advertised and tested with upgrades. + +Baseline command: bun test tests/server/server-live.test.ts tests/server/api-access-endpoints.test.ts. Result: 46 pass, 0 fail, 351 assertions at ec065aa0c6fb46b376a2f01873bd677327b99150 before production changes. This is baseline evidence, not feature verification. diff --git a/devlog/_plan/260912_audio_apis_stack/010_transcription.md b/devlog/_plan/260912_audio_apis_stack/010_transcription.md new file mode 100644 index 0000000000..ba5b6551db --- /dev/null +++ b/devlog/_plan/260912_audio_apis_stack/010_transcription.md @@ -0,0 +1,52 @@ +# File transcription + +Depends on wp0. This layer owns the unary audio transport and the shared upstream boundary used by wp2. + +## File changes + +| Operation | Path | Contract change | +| --- | --- | --- | +| NEW | src/server/audio-upstream.ts | audio model constants, proxy-owned upstream resolution reusing existing OpenAI sidecar auth; no provider registry mutation | +| NEW | src/server/audio-transcriptions.ts | parse bounded multipart, select transcription model, relay to subscription or keyed upstream, map response/errors and cancel | +| MODIFY | src/server/live.ts | retain public live exports; share only reusable upstream resolution as needed, preserving native call-create behavior | +| MODIFY | src/server/index.ts | register exact POST /v1/audio/transcriptions before unknown-v1 guard; perform admission/origin/drain checks before body consumption; record request outcome | +| MODIFY | src/server/auth-cors.ts | append matching transcription row to AUTH_MATRIX; preserve existing header precedence | +| MODIFY | tests/server/api-key-attribution.test.ts | send valid multipart for transcription matrix rows, retaining per-header denial and attribution assertions | +| NEW | tests/server/audio-transcriptions.test.ts | isolated home and synthetic file fixtures, mock canonical upstream fetch, actual ingress requests | +| MODIFY | scripts/test-layout/layout.json | register the new server-domain test | +| MODIFY | tests/fixtures/test-layout-expected.json | add matching expected test path | +| MODIFY | structure/data-planes/inbound-compat.md | describe the audio data-plane contract and owned source/test paths | +| MODIFY | docs-site/src/content/docs/reference/proxy-formats.md | document multipart audio usage and supported response fields | + +## Before / after contracts + +Before: unknown POST /v1/audio/transcriptions falls through to JSON 404. After: authentication and origin rejection run first; an admitted request reaches handleAudioTranscriptions(req, config, logCtx, lease). + +New handler accepts exactly one nonempty file, required model, optional prompt/language/response_format. gpt-4o-transcribe is the subscription compatibility model. Only explicitly supported JSON/text output is accepted; unsupported streaming/timestamp/temperature extensions fail clearly rather than being silently claimed. File limit 25,000,000 bytes; whole multipart limit 32 MiB; text field limits and duplicate checks bound parsing. No file is written to disk. The entire body is capped while reading before Web-platform multipart parsing; multipart overhead therefore remains bounded. + +Subscription destination is canonical https://chatgpt.com/backend-api/transcribe (remove only the known /codex suffix). Rebuild FormData so the boundary matches; send file and supported context fields, omit compatibility model and response_format. Keyed OpenAI destination is /v1/audio/transcriptions and receives the validated model/options. Use registered OpenAI account helpers; never forward the client OpenCodex secret. Existing native explicit credential flow remains available only under its existing admission contract. No generic caller-controlled destination. + +The resolver takes the already-resolved DataPlaneAdmission and uses resolveFirstUsableOpenAiSidecar directly, rather than invoking the complete Live handler. Retain account context/provider identity/outcome callback for wp2. For a proxy-owned Authorization value, remove only that admission credential before constructing upstream headers and use admitted stored-account resolution. Direct-mode behavior is explicitly tested; no automatic paid-provider fallback after a selected ChatGPT account error. Missing stored Direct credentials report unavailable rather than silently reclassifying an API key as a native token. + +New audio ingress resolves explicitly supplied credentials with resolveDataPlaneAdmissionSecret even on loopback; precedence is dedicated header, Bearer, x-api-key. Invalid explicit credentials cannot fall through to loopback admission. File transcription and dictation streaming require a valid proxy key on either listener. This rule is audio-owned and does not modify global resolveApiAuth behavior. + +For Direct with a valid proxy key, call resolveCodexAuthContext with mode=direct, substituteMainCredentialForDirect=true and beginCodexAccountSelection=codexAccountSelectionForTurn(lease); then materializeCodexUpstreamAuth with substituteMainCredential=true. This path claims stored-main ownership before reading it. A missing lease, draining main, missing/expired stored credential fails before I/O. Test successful stored-main substitution with isolated fake home, missing credentials and draining state. Native explicit caller auth keeps the existing sidecar path. + +Response is bounded before JSON parsing. JSON requires a string text field. text format returns text/plain. Forward sanitized machine-readable upstream status/errors, not arbitrary exception bodies. Client abort, upstream timeout and redirect rejection follow existing relay conventions; outcome callback and admission/sidecar cleanup must settle once. + +New transport types are internal: creation in audio-transcriptions, in-memory consumers only, no persistence or reviver. Public multipart is parsed once at ingress. AUTH_MATRIX is serialized by existing management response and validated by GUI isApiAuthMatrix; its consumer remains ApiKeysEndpointsPanel. + +## Acceptance and checks + +1. POST synthetic WAV with a configured proxy Bearer key: upstream receives the selected stored account/API credential and correct multipart bytes; client gets text. +2. Missing/wrong key and hostile Origin: 401/403 before upload read or upstream call. Main and optional companion listener policies are tested separately. +3. Missing/duplicate file/model, unsupported model or format, malformed multipart: explicit 400; no upstream call. +4. Declared and streamed body oversize, file oversize, text oversize: 413 at configured boundary; body cancellation observed. +5. JSON and text response modes, upstream malformed JSON and excessive response, redirect, timeout and abort: exact response and cleanup assertions. +6. Existing live and auth matrix tests retain their behavior. + +Commands: bun test tests/server/audio-transcriptions.test.ts tests/server/server-live.test.ts tests/server/api-keys-routes.test.ts; bun run typecheck; bun run structure:check; bun run privacy:scan. Before new test exists, its execution is NOT RUN; existing commands are declared in package.json and focused file arguments directly observe this layer. Full repository suite runs before review-ready publication, not as a documentation check. + +Security control activation cases and residual assumptions are recorded in ignored scratch, reviewed independently, and never copied into the public planning record. + +Implementation review amendment: readBodyCapped gains an optional AbortSignal to cancel its locked reader; the transcription operation registers a controller with its turn lease and has an overall deadline plus upload deadline. src/providers/openai-sidecar.ts releases probe ownership if header materialization or post-resolution validation fails before returning a context. Explicit validated native Direct caller auth uses the existing sidecar path; stored-main substitution applies only to proxy-key-only Direct. These narrow changes are prerequisites for safe reuse and preserve existing callers by default. diff --git a/devlog/_plan/260912_audio_apis_stack/011_transcription_checks.md b/devlog/_plan/260912_audio_apis_stack/011_transcription_checks.md new file mode 100644 index 0000000000..738be4ba17 --- /dev/null +++ b/devlog/_plan/260912_audio_apis_stack/011_transcription_checks.md @@ -0,0 +1,15 @@ +# Transcription implementation checks + +File transcription is implemented with explicit client-key admission, canonical OpenAI upstreams, bounded multipart and response handling, leased Direct substitution, explicit native Direct passthrough, and cancellation linked from upload through shutdown. Responses/chat routing is unchanged. + +Fresh targeted evidence: 24 audio regression tests pass; 17 test-layout tests pass; typecheck and structure checks pass. Import-connected check: bun run test:changed selected 326 of 1202 files and completed 8138 pass, 2 skip, 0 fail. The final two probe-release regressions were added afterwards and passed in the focused audio run. No paid upstream call or personal recording was used. + +Independent review found upload lifetime, shutdown cancellation, explicit Direct handling, probe cleanup and final-outcome defects; these were fixed and rechecked. The final bounded Noether review returned VERDICT: PASS. Additional probe-release tests cover both pre-return helper failure paths. + +The original ec065aa0c6 layout JSON incorrectly seeded cline-client.test.ts and cline-writer.test.ts as providers despite explicit clients ownership. This was reproduced using the original JSON and unchanged resolver. The two seed families now agree; no assertion or coverage was removed. + +Synthetic curl QA completed against the built handler with a mocked canonical upstream: success 200, invalid key 401, unsupported model 400; teardown confirmed no listener. Documentation build completed 425 pages. A later expanded curl case correctly returned 413 after HTTP 100 Continue; the QA script misclassified that interim status, so the expanded run is not a passing receipt. + +The default full suite crashed inside Bun 1.4.2 with SIGSEGV on a separate immutable verification checkout. Its failed-file count includes aborted work and is not an assertion-failure count; baseline causation remains unproven. The serial diagnostic was interrupted by the owner's explicit no-local-suite instruction and is NOT PASS. All future product checks move to exact-head remote CI; first PR 4391 stays draft pending that gate. Local product tests/typecheck/build/install are NOT RUN after this steering, and pushes use --no-verify. + +wp1 functional implementation is complete based on the pre-restriction focused/affected checks and independent source review; remote review-readiness remains tracked by the publication criterion. Next cycle consumes 020_streaming_voice.md and the completed audio upstream boundary. Real OpenAI/ChatGPT account entitlement and server behavior remain outside synthetic verification and are not claimed. diff --git a/devlog/_plan/260912_audio_apis_stack/020_streaming_voice.md b/devlog/_plan/260912_audio_apis_stack/020_streaming_voice.md new file mode 100644 index 0000000000..68d6fa8392 --- /dev/null +++ b/devlog/_plan/260912_audio_apis_stack/020_streaming_voice.md @@ -0,0 +1,51 @@ +# Streaming dictation and external live voice + +Depends on wp1 audio upstream resolution. Preserve existing Codex transport exports and native integrations. + +## File changes + +| Operation | Path | Contract change | +| --- | --- | --- | +| NEW | src/server/audio-dictation.ts | exact streaming route, canonical upstream URL/protocol construction and bounded session policy | +| NEW | src/server/live-call-bindings.ts | per-server bounded expiring call ownership, keyed by opaque call id and admission owner | +| MODIFY | src/server/live.ts | bind successful call-create to its resolved upstream; join via binding; proxy Bearer credential support; default V3 model/negotiation only on standalone /live when absent | +| MODIFY | src/server/index.ts | route dictation upgrades through existing bounded WebSocket bridge; thread lifecycle metadata and close cleanup | +| MODIFY | src/server/ws-bridge.ts | add only necessary session protocol/expiry fields to WsData | +| MODIFY | src/server/auth-cors.ts | advertise only HTTP call-create rows; WebSocket admission is separate audio metadata | +| MODIFY | tests/server/api-key-attribution.test.ts | valid SDP multipart fixtures for HTTP call-create matrix rows; WebSocket auth is tested by real upgrades in audio/server-live tests | +| NEW | tests/server/audio-dictation.test.ts | mock WebSocket upstream with real JSON audio events and close/cancel checks | +| MODIFY | tests/server/server-live.test.ts | external Bearer key and call-owner/session lifecycle regressions | +| MODIFY | scripts/test-layout/layout.json | register dictation test in server domain | +| MODIFY | tests/fixtures/test-layout-expected.json | matching expected test path | +| MODIFY | structure/data-planes/inbound-compat.md | document streams, ownership and source/test contract | +| MODIFY | docs-site/src/content/docs/reference/proxy-formats.md | document distinct dictation and GPT-Live wire examples and limitations | + +## Before / after contracts + +Before: existing native HTTP and WebSocket voice relay has no exported dictation transport. After: WS /v1/audio/transcriptions/stream resolves ChatGPT auth on the server and connects to wss://chatgpt.com/backend-api/dictation/stream with the observed chatgpt-dictation, openai-bearer token, codex-desktop subprotocols. Upstream token remains confined to server WebSocket construction, never the downstream selected protocol or errors. The route is explicitly an OpenCodex extension, not public Realtime API compatibility. + +Client uses observed session.start/config, audio append and session.close shapes; server events include session.started, transcript.segment/final and session.updated closed. Preserve text frames without UTF-8 reframing. Reuse existing pending-frame and frame-byte limits, add finite session lifetime consistent with five-minute desktop contract, and close both directions on error/abort/shutdown. Unsupported keyed-only dictation reports an actionable unavailable response. No automatic replay after audio has been accepted. + +Observed dictation audio is JSON {type: "audio.append", audio: "BASE64_PCM16"}; mono PCM16 uses the actual sample_rate_hz supplied by the client. transcript.segment/final revisions replace prior text for the same utterance_id. Closing acknowledgment is session.updated with session.status=closed. The requested limits are client policy, not proven upstream maxima. For browser clients the downstream protocol pair is opencodex-audio plus opencodex-key.; only opencodex-audio can be selected back. Explicit HTTP admission headers retain precedence. Never accept this carrier on ordinary Responses routes. + +Live call-create keeps SDP/multipart conversion and Location response. A successful call stores a bounded per-server binding to the resolved account/provider and caller admission identity for follow-up joins. Never retain raw client API keys in persisted state; no persistence is needed. Follow-up requests authenticate again, reject mismatched/expired owners, and cannot change the selected account. Existing native clients using the same local/session identity retain their workflow. Validate invalid Location before reporting usable creation. + +Bindings survive sideband disconnect for bounded reconnect; server shutdown clears them. Resolve the recorded exact account before join, preserving physical account identity across credential refresh. Tagged ownership distinguishes configured key ID, environment admission and legacy loopback native session. Unknown calls are rejected for proxy-key clients; any native externally-created-call compatibility must remain limited to explicit caller-auth and documented separately. Reuse the existing socket bridge within the composition root for this layer; extracting all legacy socket machinery is optional and requires its own regression evidence. + +Live resolves a presented proxy key before the global loopback shortcut, so a key-owned call remains configured-key owned on both listeners. Invalid explicit proxy credentials fail closed. Legacy native callers with no proxy key retain loopback/session identity only under existing local policy and cannot join configured-key-owned calls. The registry's tagged owner prevents any equality between native loopback and configured key IDs. Test same and different key joins on both primary and companion listeners. Do not add WS paths to the existing HTTP-only AUTH_MATRIX; advertise those through wp3 audio metadata and prove header/protocol admission with actual WebSocket upgrades. + +For external proxy-key call-create, return a proxy-relative Location /v1/live/ (or the matching realtime/calls form). Never return an absolute upstream Location to a proxy-key client. The upstream WebSocket destination is independently selected from trusted config, not from Location. Native legacy response compatibility remains scoped to its existing explicit caller-auth contract. A regression follows the returned relative Location with the creator key and checks the recorded account after pool rotation. + +Standalone WS /v1/live accepts an explicit model or defaults to gpt-live-1-codex, with gpt-live-1 as documented alias if implemented. Missing V3 negotiation is added only to this Frameless path. /v1/realtime preserves its current adapter semantics. A raw live client receives delegation events; proxy does not execute tools or fabricate delegation results. + +WsData fields are created at server.upgrade, serialized only by Bun in process, read at open/message/close, and disposed at relay closure; no disk reviver. Call binding types are in-memory only. Public query/model and protocol inputs are validated at ingress; no secret-bearing URL query authentication is added. + +## Acceptance and checks + +1. Synthetic streamed PCM JSON reaches the mock upstream under server-owned auth; transcript events reach the client byte-for-byte. +2. Wrong key/origin, malformed start/audio, large frames/pending data, missing ChatGPT capability: fail before further forwarding. +3. Session close, disconnect before upstream opens, expired session and upstream refusal: both sockets, timers and leases settle. +4. Two configured keys and two upstream accounts: creator can join even after pool selection changes; other key cannot join; unknown/expired binding fails. +5. Existing native sideband forms and standalone V1/V3 remain covered; HTTP SDP forwarding retains status/content-type and returns the documented proxy-relative Location for external clients. + +Commands: bun test tests/server/audio-dictation.test.ts tests/server/server-live.test.ts; bun run typecheck; bun run structure:check; bun run privacy:scan. New paths are NOT RUN until implemented. Real socket fixtures observe the actual branch and cleanup, not only helper return values. diff --git a/devlog/_plan/260912_audio_apis_stack/030_connections.md b/devlog/_plan/260912_audio_apis_stack/030_connections.md new file mode 100644 index 0000000000..ccb9fa24ce --- /dev/null +++ b/devlog/_plan/260912_audio_apis_stack/030_connections.md @@ -0,0 +1,45 @@ +# Connections audio controls and PR publication + +Depends on wp2 completed audio routes. The first two layers remain independently usable through external client examples. + +## File changes + +| Operation | Path | Contract change | +| --- | --- | --- | +| MODIFY | src/server/management/api-access.ts | extend ApiAccessEndpoints with transcription, dictationStream, live and realtimeCalls URLs plus truthful capability metadata | +| MODIFY | src/server/management-api.ts | include audio metadata using existing management authentication | +| MODIFY | tests/server/api-access-endpoints.test.ts | URL host/protocol and capability projection tests | +| MODIFY | gui/src/pages/api-keys-utils.ts | extend endpoint type/default/derive chain for new endpoints | +| MODIFY | gui/src/pages/ApiKeys.tsx | consume serialized endpoint metadata through KeysResponse, CachedKeysShape, cache validation and fetchKeys | +| MODIFY | gui/src/components/apikeys-workspace/ApiKeysWorkspace.tsx | place two unframed audio sections in existing Connections/API layout | +| NEW | gui/src/components/apikeys-workspace/AudioApiPanel.tsx | accessible Dictation and Live Voice controls, endpoint/model display, sample copying, transient key/file controls and result/error states | +| NEW | gui/src/audio-api-client.ts | bounded cancelable upload and socket client protocol helpers; no saved secrets | +| NEW | gui/tests/audio-api-client.test.ts | request generation, cancellation and transcript assembly tests | +| MODIFY | gui/src/i18n/en.ts and every locale module | complete localized label/status/action keys | +| MODIFY | gui/src/styles-apikeys-workspace.css | restrained aligned responsive audio sections using existing tokens | +| MODIFY | structure/gui-and-management-api.md | metadata and control ownership/current contract | +| MODIFY | docs-site/src/content/docs/reference/proxy-formats.md and localized counterparts | final client examples and explicit protocol support | + +## Before / after contracts + +Before: ApiAccessEndpoints contains Responses/chat/messages/models only. After: backend generates audio endpoints from the same resolved base, converts HTTP->WS and HTTPS->WSS with URL APIs, and reports configured availability separately from runtime-proven connectivity. GUI consumes these fields, with a conservative unavailable/unknown fallback for older servers. + +Metadata fields complete chain: creation buildApiAccessEndpoints -> JSON management response -> API page validation/mapping -> ApiEndpointInfo/AudioApiPanel. Defaults cannot claim configured availability. No new provider is registered and no audio model enters a text completion test. + +Dictation section contains model and endpoint copy actions, file input, transient API key input, transcribe/cancel, text result/copy and clear error states. Stream example names extension protocol and gives start/audio/close events. Live Voice section contains actual GPT-Live model and both WS/WebRTC connection endpoints, transient client key, a connect/disconnect test with status and observed event output. Browser WebSocket auth must use a short-lived local session mechanism or supported client protocol carrier; never expose ChatGPT credentials or persist raw keys. Do not create a fake success check or billable background probe. All test actions require a deliberate user click. + +UI is unframed and follows existing workspace colors/type/spacing. Icons reuse gui icons, all visible text is localized. At desktop and mobile widths long endpoint text wraps or scrolls within its own element without overlapping controls. Buttons have stable dimensions and stateful controls are keyboard reachable. + +Pass the existing active flag through ApiKeysWorkspace. Integrations hides panels without unmounting; requests, sockets and timers must stop on deactivation as well as unmount. Existing key rows contain only prefixes: controls use a newly generated key or an explicitly entered transient key, never pretend a key ID can authenticate. Browser voice connection uses an OpenCodex-only WebSocket protocol credential carrier accepted solely by the audio routes; exact supported carrier and precedence are documented/tested in wp2. No persistent key or query authentication. + +## Acceptance and publication + +1. API metadata correctly derives HTTPS/WSS, wildcard, IPv6 and companion-listener addresses and shows missing upstream as unavailable. +2. Mocked browser flow uploads a fixture, receives text, copies it, cancels a pending call and displays a server error. No real audio/provider requests during agent QA. +3. Mocked voice flow connects, observes a protocol event, disconnects and releases callbacks/timers; API keys never enter storage, screenshots or URL queries. +4. Desktop and mobile browser screenshots are read back and corrected. Screenshot attached to UI PR with synthetic data only. +5. Run GUI focused tests, lint:i18n, lint, build and repository typecheck/full suite before review ready; per-layer CI uses exact PR head. The final PRs fill Summary, Verification and Checklist plus ordinary stack map. + +Commands are defined by root/gui package.json. Source paths and existing stylesheet/fetch owner are revalidated at this cycle P before implementation; any renamed path is amended with exact ownership evidence. No disconnected metadata fields or fake audio model tests are acceptable. + +The three PRs use codex/audio-transcription -> dev, codex/audio-streaming -> codex/audio-transcription, codex/audio-connections -> codex/audio-streaming. Leave all open. Record test results and head/base SHAs without claiming human approval, merge or real provider availability. diff --git a/docs-site/src/content/docs/reference/proxy-formats.md b/docs-site/src/content/docs/reference/proxy-formats.md index 7cec1af766..c5fa271bf1 100644 --- a/docs-site/src/content/docs/reference/proxy-formats.md +++ b/docs-site/src/content/docs/reference/proxy-formats.md @@ -33,9 +33,45 @@ Credential-bearing model, image, video, and search requests do not automatically | Anthropic Messages | `POST /v1/messages` | Anthropic `message` JSON | Anthropic Messages SSE | | Anthropic token count | `POST /v1/messages/count_tokens` | `{ "input_tokens": number }` | Not applicable | | Model discovery | `GET /v1/models` | Catalog or explicit Desktop snapshot | Not applicable | +| File transcription | `POST /v1/audio/transcriptions` | `{ "text": string }` or plain text | Not supported on this file endpoint | | Voice and Realtime | `POST /v1/live`, `POST /v1/realtime/calls` | Relayed call-creation response | A separate sideband WebSocket relays frames in both directions | | Responses compaction | `POST /v1/responses/compact` | Replacement-history JSON | Not applicable | +## File transcription + +`POST /v1/audio/transcriptions` accepts an OpenCodex data-plane key in +`Authorization: Bearer`, `x-opencodex-api-key`, or `x-api-key`, including on a local +listener. An explicitly supplied invalid key is rejected. Upload one audio file +as multipart `file` and provide `model=gpt-4o-transcribe` for a connected ChatGPT +account. OpenCodex resolves the upstream credential; never supply a ChatGPT token +as the client API key. + +```bash +curl "$OPENCODEX_BASE_URL/audio/transcriptions" \ + -H "Authorization: Bearer $OPENCODEX_API_KEY" \ + -F 'model=gpt-4o-transcribe' \ + -F 'file=@recording.wav' \ + -F 'language=ko' +``` + +Set `OPENCODEX_BASE_URL` to your proxy URL ending in `/v1`. Optional fields are +`prompt`, `language`, and `response_format` (`json`, the default, or `text`). The +JSON result contains `text` only. Files must be nonempty and no larger than +25,000,000 bytes; multipart bodies are limited to 32 MiB and text fields to +16 KiB. The configured listener body limit can impose a smaller ceiling. +Duplicate or unsupported fields, including `stream`, are rejected. This endpoint +does not promise timestamps, diarization, subtitles, or token-usage metadata. + +The ChatGPT subscription path uses `gpt-4o-transcribe` as a compatibility identifier +and does not send a model name to the private transcription endpoint. It is not +evidence of the backend's internal model. An enabled OpenAI API-key provider also +supports `gpt-4o-mini-transcribe` and `whisper-1`; when a ChatGPT provider is selected, +an authentication failure does not silently switch to that paid provider. +Direct mode uses the stored main account under the existing profile admission +rules; Pool mode uses the selected stored account. Missing, expired or draining +credentials return an error. Cancellation stops the outbound request and audio +content is not written to request history. + ## `POST /v1/responses` This is the native opencodex data-plane shape. The request body must be a JSON object with a diff --git a/gui/tests/locale-parity.test.ts b/gui/tests/locale-parity.test.ts index 3e05bb632f..745440fcc8 100644 --- a/gui/tests/locale-parity.test.ts +++ b/gui/tests/locale-parity.test.ts @@ -99,6 +99,9 @@ const ZH_TW_KEEP_ENGLISH: ReadonlySet = new Set([ "nav.grok", // Other short identifiers, commands, and product names kept in English "api.clientConfig.clientOpencode", + // Cline CLI is a product name, not untranslated interface copy. + "integrations.tab.cline", + "api.clientConfig.clientCline", "api.clientConfig.clientPi", "api.clientConfig.clientOmp", "api.clientConfig.clientHermes", diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 4f4ec44705..c4363a7305 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -246,6 +246,7 @@ "aside-profiles-routes.test.ts": "server", "aside-profiles.test.ts": "clients", "assert-mergeable-review.test.ts": "ci-workflows", + "audio-transcriptions.test.ts": "server", "auto-compact-budget.test.ts": "providers", "autostart-health.test.ts": "service", "azure-adapter.test.ts": "providers", @@ -543,6 +544,7 @@ "context-compat.test.ts": "codex-integration", "context-history-ownership.test.ts": "server", "context-history.test.ts": "server", + "context-window-seed-repair.test.ts": "providers", "continuation-dedup.test.ts": "responses", "core-lab-boundary.test.ts": "lab", "cost-cap-unknown-evidence.test.ts": "usage", @@ -632,6 +634,8 @@ "desktop-remote-store.test.ts": "clients", "destination-policy-resolved.test.ts": "routing", "devin-adapter.test.ts": "providers", + "devin-cli-authmode-migration.test.ts": "providers", + "devin-cli-login.test.ts": "providers", "devin-hardening.test.ts": "providers", "digitalocean-scaleway-provider.test.ts": "providers", "docs-429-failover-claims.test.ts": "ci-workflows", @@ -679,7 +683,6 @@ "gemini-inline.test.ts": "images", "gemini-web-search.test.ts": "adapters/google", "generic-oauth-failover.test.ts": "oauth", - "pool-kernel-generic-sweep.test.ts": "oauth", "github-copilot-account-origin.test.ts": "providers/github-copilot", "github-copilot-oauth.test.ts": "providers/github-copilot", "github-copilot-sse-rewrite.test.ts": "providers/github-copilot", @@ -867,7 +870,6 @@ "model-pinned-effort.test.ts": "codex-integration", "model-presets.test.ts": "providers", "model-rename-migration.test.ts": "providers", - "context-window-seed-repair.test.ts": "providers", "model-selection-guidance.test.ts": "cli", "model-visibility-management-api.test.ts": "codex-integration", "models-feedback-callback.test.ts": "gui", @@ -954,9 +956,9 @@ "openai-chat-hardening.test.ts": "adapters/openai", "openai-chat-invalid-tool-call-diagnostics.test.ts": "adapters/openai", "openai-chat-model-suffix.test.ts": "adapters/openai", - "openai-chat-path-override.test.ts": "adapters/openai", "openai-chat-native-policy.test.ts": "adapters/openai", "openai-chat-parallel-stream.test.ts": "adapters/openai", + "openai-chat-path-override.test.ts": "adapters/openai", "openai-chat-system-order.test.ts": "adapters/openai", "openai-chat-tool-result-images.test.ts": "adapters/openai", "openai-chat-url.test.ts": "adapters/openai", @@ -995,6 +997,7 @@ "plan-video.test.ts": "videos", "plan.test.ts": "images", "policy-execution.test.ts": "routing", + "pool-kernel-generic-sweep.test.ts": "oauth", "port-reclaim.test.ts": "server", "ports.test.ts": "server", "prime-client.test.ts": "clients", @@ -1205,7 +1208,6 @@ "sidecar-settings-web-search-stream.test.ts": "vision", "sidecar-tracker.test.ts": "vision", "skill-ocx.test.ts": "ci-workflows", - "structure-ssot.test.ts": "ci-workflows", "slug-codec.test.ts": "codex-integration", "sponsor-presets.test.ts": "providers", "sse-client-frame-bounds.test.ts": "responses", @@ -1237,6 +1239,7 @@ "storage-worker-teardown-isolate.test.ts": "storage", "stream-aborted-marker.test.ts": "server", "strict-semver.test.ts": "lib", + "structure-ssot.test.ts": "ci-workflows", "subagent-context-staleness.test.ts": "routing", "subagent-defaults.test.ts": "routing", "subagent-fallback-handle-responses.test.ts": "routing", @@ -1370,9 +1373,7 @@ "zhipu-bigmodel-responses-quota.test.ts": "providers", "zz-ci-api-usage-isolation.test.ts": "ci-workflows", "zz-ci-storage-policy-isolation.test.ts": "ci-workflows", - "zz-pr-coderabbit-readiness-revalidation.test.ts": "ci-workflows", - "devin-cli-login.test.ts": "providers", - "devin-cli-authmode-migration.test.ts": "providers" + "zz-pr-coderabbit-readiness-revalidation.test.ts": "ci-workflows" }, "migrated": [ "adapters", diff --git a/src/providers/openai-sidecar.ts b/src/providers/openai-sidecar.ts index 3b98a2b521..2b0b41d26d 100644 --- a/src/providers/openai-sidecar.ts +++ b/src/providers/openai-sidecar.ts @@ -5,6 +5,7 @@ import { hasCallerCodexBearer, isCodexAuthContextUsable, resolveCodexAuthContext, + releaseCodexAuthContextProbeLease, type CodexAccountSelectionAdmission, type CodexAuthContext, type CodexAuthPolicyConfig, @@ -152,11 +153,18 @@ export async function resolveFirstUsableOpenAiSidecar( admission: options.admission, beginCodexAccountSelection: options.beginCodexAccountSelection, }); - const selectedHeaders = headersForCodexAuthContext(incomingHeaders, authContext, policy, exactAccount.modelId, options.admission); + let selectedHeaders: Headers; + try { + selectedHeaders = headersForCodexAuthContext(incomingHeaders, authContext, policy, exactAccount.modelId, options.admission); + } catch (error) { + releaseCodexAuthContextProbeLease(authContext); + throw error; + } if ((authContext.kind !== "pool" && authContext.kind !== "main-pool") || !isCodexAuthContextUsable(authContext, config)) { // Exact selection is fail-closed. A generation/runtime-state race must not fall through // to the caller-bearer error or let a later candidate select another account. + releaseCodexAuthContextProbeLease(authContext); throw new CodexPoolAuthenticationError("Selected Codex account is unavailable"); } return { @@ -195,8 +203,17 @@ export async function resolveFirstUsableOpenAiSidecar( admission: options.admission, beginCodexAccountSelection: options.beginCodexAccountSelection, }); - const selectedHeaders = headersForCodexAuthContext(incomingHeaders, authContext, policy, undefined, options.admission); - if (!isCodexAuthContextUsable(authContext, config)) continue; + let selectedHeaders: Headers; + try { + selectedHeaders = headersForCodexAuthContext(incomingHeaders, authContext, policy, undefined, options.admission); + } catch (error) { + releaseCodexAuthContextProbeLease(authContext); + throw error; + } + if (!isCodexAuthContextUsable(authContext, config)) { + releaseCodexAuthContextProbeLease(authContext); + continue; + } return { ...candidate, authContext, diff --git a/src/server/audio-transcriptions.ts b/src/server/audio-transcriptions.ts new file mode 100644 index 0000000000..4de98b0b86 --- /dev/null +++ b/src/server/audio-transcriptions.ts @@ -0,0 +1,183 @@ +import { formatErrorResponse } from "../bridge"; +import { cancelBodyOnAbort, clearableDeadline, signalWithTimeout } from "../lib/abort"; +import type { AdmissionLease } from "../lib/admission"; +import { sidecarEnter } from "../lib/sidecar-tracker"; +import type { OcxConfig } from "../types"; +import type { DataPlaneAdmission } from "./auth-cors"; +import { resolveAudioUpstream, TRANSCRIPTION_MODEL } from "./audio-upstream"; +import { readBodyCapped } from "./live"; +import type { RequestLogContext } from "./request-log"; +import { registerTurn, unregisterTurn } from "./lifecycle"; + +export const AUDIO_FILE_MAX_BYTES = 25_000_000; +export const AUDIO_BODY_MAX_BYTES = 32 * 1024 * 1024; +const AUDIO_FIELD_MAX_BYTES = 16 * 1024; +const AUDIO_RESPONSE_MAX_BYTES = 2 * 1024 * 1024; +const AUDIO_TIMEOUT_MS = 120_000; +const FIELDS = new Set(["file", "model", "prompt", "language", "response_format"]); +const MODELS = new Set([TRANSCRIPTION_MODEL, "gpt-4o-mini-transcribe", "whisper-1"]); + +interface TranscriptionInput { + file: File; + model: string; + prompt?: string; + language?: string; + format: "json" | "text"; +} + +function invalid(message: string, status = 400): Response { + return formatErrorResponse(status, "invalid_request_error", message); +} + +async function parseTranscription(req: Request, operationSignal: AbortSignal): Promise { + if (!/^multipart\/form-data\s*;/i.test(req.headers.get("content-type") ?? "")) { + return invalid("Expected multipart/form-data with file and model"); + } + const encoding = req.headers.get("content-encoding")?.trim().toLowerCase(); + if (encoding && encoding !== "identity") return invalid("Compressed audio request bodies are not supported"); + const length = req.headers.get("content-length"); + if (length && /^\d+$/.test(length) && Number(length) > AUDIO_BODY_MAX_BYTES) { + await req.body?.cancel().catch(() => {}); + return invalid("Audio request exceeds 32 MiB", 413); + } + let form: FormData; + const uploadDeadline = clearableDeadline(30_000, operationSignal); + try { + const body = await readBodyCapped(req.body, AUDIO_BODY_MAX_BYTES, () => "Audio request too large", uploadDeadline.signal); + if (body instanceof Response) return invalid("Audio request exceeds 32 MiB", 413); + if (operationSignal.aborted) return formatErrorResponse(499, "client_closed_request", "Audio upload canceled"); + form = await new Response(body, { headers: { "content-type": req.headers.get("content-type")! } }).formData(); + } catch { + if (uploadDeadline.didExpire()) return formatErrorResponse(408, "request_timeout", "Audio upload timed out"); + return operationSignal.aborted + ? formatErrorResponse(499, "client_closed_request", "Audio upload canceled") + : invalid("Malformed audio multipart body"); + } finally { + uploadDeadline.clear(); + } + const seen = new Set(); + for (const [name, value] of form) { + if (!FIELDS.has(name)) return invalid(`Unsupported transcription field: ${name.slice(0, 64)}`); + if (seen.has(name)) return invalid(`Duplicate transcription field: ${name}`); + seen.add(name); + if (name !== "file" && (typeof value !== "string" || Buffer.byteLength(value) > AUDIO_FIELD_MAX_BYTES)) { + return invalid("Audio text fields must be strings no larger than 16 KiB", 413); + } + } + const file = form.get("file"); + if (!(file instanceof File) || file.size === 0) return invalid("A nonempty audio file is required"); + if (file.size > AUDIO_FILE_MAX_BYTES) return invalid("Audio file exceeds 25,000,000 bytes", 413); + const model = form.get("model"); + if (typeof model !== "string" || !MODELS.has(model)) return invalid("Unsupported transcription model"); + const format = form.get("response_format") ?? "json"; + if (format !== "json" && format !== "text") return invalid("response_format must be json or text"); + const prompt = form.get("prompt") as string | null; + const language = form.get("language") as string | null; + if (language !== null && !/^[a-z]{2,3}(?:-[A-Za-z0-9]{2,8})?$/.test(language)) return invalid("Invalid transcription language"); + return { file, model, format, ...(prompt !== null ? { prompt } : {}), ...(language !== null ? { language } : {}) }; +} + +async function transcribeAdmitted( + req: Request, + config: OcxConfig, + log: RequestLogContext, + admission: DataPlaneAdmission, + operation: { lease?: AdmissionLease; signal: AbortSignal; didExpire: () => boolean }, +): Promise { + const input = await parseTranscription(req, operation.signal); + if (input instanceof Response) return input; + if (operation.signal.aborted) return formatErrorResponse(499, "client_closed_request", "Audio request canceled"); + const relay = await resolveAudioUpstream(req.headers, config, log, { admission, model: input.model, lease: operation.lease, signal: operation.signal }); + if (relay instanceof Response) return relay; + const signal = signalWithTimeout(AUDIO_TIMEOUT_MS, operation.signal); + const exit = sidecarEnter("audio-transcription"); + let outcome: number | "timeout" | "connect_error" | undefined; + try { + if (!relay.keyed && input.model !== TRANSCRIPTION_MODEL) return invalid(`ChatGPT transcription supports only ${TRANSCRIPTION_MODEL}`); + const url = relay.keyed + ? `${relay.providerBaseUrl}/audio/transcriptions` + : "https://chatgpt.com/backend-api/transcribe"; + const headers = new Headers(); + for (const name of ["authorization", "chatgpt-account-id", "user-agent", "originator", "version"]) { + const value = new Headers(relay.headers).get(name); + if (value) headers.set(name, value); + } + if (!relay.keyed) { + const nativeAgent = req.headers.get("user-agent"); + headers.set("user-agent", nativeAgent && /^codex(?:[_ /-]|$)/i.test(nativeAgent) + ? nativeAgent.slice(0, 2048) : "codex_cli_rs"); + if (!headers.has("originator")) headers.set("originator", "codex_cli_rs"); + } + const form = new FormData(); + form.append("file", input.file, "audio" + (/\.[a-z0-9]{1,8}$/i.exec(input.file.name)?.[0] ?? ".webm")); + if (input.prompt !== undefined) form.append("prompt", input.prompt); + if (input.language !== undefined) form.append("language", input.language); + if (relay.keyed) { + form.append("model", input.model); + form.append("response_format", "json"); + } + const upstream = await fetch(url, { method: "POST", headers, body: form, signal: signal.signal, redirect: "manual" }); + outcome = upstream.ok ? 502 : upstream.status; + const detach = cancelBodyOnAbort(upstream.body, signal.signal); + let body: ArrayBuffer | Response; + try { + body = await readBodyCapped(upstream.body, AUDIO_RESPONSE_MAX_BYTES, () => "Audio upstream response too large", signal.signal); + } finally { + detach(); + } + if (body instanceof Response) return body; + if (!upstream.ok) { + const status = upstream.status >= 400 && upstream.status <= 599 ? upstream.status : 502; + return formatErrorResponse(status, "upstream_error", `Audio upstream returned HTTP ${upstream.status}`); + } + let payload: unknown; + try { payload = JSON.parse(new TextDecoder().decode(body)); } catch { + return formatErrorResponse(502, "upstream_error", "Audio upstream returned invalid JSON"); + } + if (!payload || typeof payload !== "object" || !("text" in payload) || typeof payload.text !== "string") { + return formatErrorResponse(502, "upstream_error", "Audio upstream response is missing text"); + } + outcome = upstream.status; + return input.format === "text" + ? new Response(payload.text, { headers: { "content-type": "text/plain; charset=utf-8" } }) + : Response.json({ text: payload.text }); + } catch { + if (operation.signal.aborted) { + outcome = operation.didExpire() ? "timeout" : undefined; + return formatErrorResponse(499, "client_closed_request", "Audio request canceled"); + } + const timedOut = signal.signal.aborted; + outcome = timedOut ? "timeout" : "connect_error"; + return formatErrorResponse(timedOut ? 504 : 502, "upstream_error", timedOut ? "Audio upstream timed out" : "Audio upstream connection failed"); + } finally { + try { + if (outcome !== undefined) relay.recordOutcome?.(outcome); + } finally { + relay.release(); + signal.cleanup(); + exit(); + } + } +} + +export async function handleAudioTranscriptions( + req: Request, + config: OcxConfig, + log: RequestLogContext, + admission: DataPlaneAdmission, + lease?: AdmissionLease, +): Promise { + const operation = new AbortController(); + if (lease) registerTurn(operation, lease); + const deadline = clearableDeadline(AUDIO_TIMEOUT_MS, AbortSignal.any([req.signal, operation.signal])); + try { + const response = await transcribeAdmitted(req, config, log, admission, { lease, signal: deadline.signal, didExpire: deadline.didExpire }); + if (req.signal.aborted) return formatErrorResponse(499, "client_closed_request", "Audio request canceled"); + if (operation.signal.aborted) return formatErrorResponse(503, "server_draining", "Audio request stopped during shutdown"); + if (deadline.didExpire()) return formatErrorResponse(504, "upstream_error", "Audio request timed out"); + return response; + } finally { + deadline.clear(); + if (lease) unregisterTurn(operation); + } +} diff --git a/src/server/audio-upstream.ts b/src/server/audio-upstream.ts new file mode 100644 index 0000000000..a59b7aef95 --- /dev/null +++ b/src/server/audio-upstream.ts @@ -0,0 +1,146 @@ +import { formatErrorResponse } from "../bridge"; +import { + CodexAccountCooldownError, + CodexMainProfileDrainingError, + codexMainProfileDrainingResponse, + cooldownErrorResponse, + isCodexAuthContextUsable, + materializeCodexUpstreamAuth, + releaseCodexAuthContextProbeLease, + resolveCodexAuthContext, + type CodexAuthContext, +} from "../codex/auth-context"; +import { formatCodexProviderForLog } from "../codex/routing"; +import type { AdmissionLease } from "../lib/admission"; +import { captureExplicitOpenAiCallerAuth, resolveFirstUsableOpenAiSidecar, selectOpenAiImagesProvider } from "../providers/openai-sidecar"; +import type { OcxConfig } from "../types"; +import { + isProxyAdmissionSecret, + resolveDataPlaneAdmissionSecret, + validateForwardAdmissionCredential, + type DataPlaneAdmission, +} from "./auth-cors"; +import { codexAccountSelectionForTurn } from "./lifecycle"; +import type { RequestLogContext } from "./request-log"; + +export const TRANSCRIPTION_MODEL = "gpt-4o-transcribe"; +export const LIVE_AUDIO_MODEL = "gpt-live-1-codex"; + +/** Audio keys remain identifiable even on the otherwise unauthenticated local listener. */ +export function resolveAudioAdmission(headers: Headers, config: OcxConfig): DataPlaneAdmission | null { + const dedicated = headers.get("x-opencodex-api-key")?.trim(); + if (dedicated) return resolveDataPlaneAdmissionSecret(dedicated, config, "dedicated"); + const authorization = headers.get("authorization")?.trim(); + if (authorization) { + const token = /^Bearer\s+([^\s,]+)$/i.exec(authorization)?.[1]; + return token ? resolveDataPlaneAdmissionSecret(token, config, "bearer") : null; + } + const key = headers.get("x-api-key")?.trim(); + return key ? resolveDataPlaneAdmissionSecret(key, config, "x-api-key") : null; +} + +export interface AudioUpstream { + providerName: string; + providerBaseUrl: string; + headers: Record; + keyed: boolean; + authContext?: CodexAuthContext; + recordOutcome?: (status: number | "timeout" | "connect_error") => void; + release: () => void; +} + +export interface AudioUpstreamOptions { + admission: DataPlaneAdmission; + model: string; + lease?: AdmissionLease; + exactAccountId?: string; + signal?: AbortSignal; +} + +/** Keep client admission, account selection, and outbound credentials as separate values. */ +export async function resolveAudioUpstream( + incoming: Headers, + config: OcxConfig, + log: RequestLogContext, + options: AudioUpstreamOptions, +): Promise { + const candidates = selectOpenAiImagesProvider(config); + const headers = new Headers(incoming); + const bearer = headers.get("authorization")?.replace(/^Bearer\s+/i, "").trim(); + if (bearer && isProxyAdmissionSecret(bearer, config)) { + headers.delete("authorization"); + headers.delete("chatgpt-account-id"); + } + headers.delete("x-opencodex-api-key"); + headers.delete("x-api-key"); + headers.delete("sec-websocket-protocol"); + let context: CodexAuthContext | undefined; + try { + options.signal?.throwIfAborted(); + const candidate = candidates.forwardCandidates[0]; + if (candidate) { + let selected: Headers; + let recordOutcome: AudioUpstream["recordOutcome"]; + const beginCodexAccountSelection = codexAccountSelectionForTurn(options.lease); + if (candidate.accountMode === "direct" && options.exactAccountId === undefined + && !captureExplicitOpenAiCallerAuth(incoming, config)) { + context = await resolveCodexAuthContext(headers, config, "direct", { + admission: options.admission, + substituteMainCredentialForDirect: true, + beginCodexAccountSelection, + signal: options.signal, + }); + selected = materializeCodexUpstreamAuth(headers, context, { + config, + admission: options.admission, + substituteMainCredential: true, + }); + } else { + const resolved = await resolveFirstUsableOpenAiSidecar(candidates.forwardCandidates, headers, config, { + admission: options.admission, + beginCodexAccountSelection, + ...(options.exactAccountId ? { exactAccount: { accountId: options.exactAccountId, modelId: options.model } } : {}), + }); + if (!resolved) return formatErrorResponse(401, "authentication_error", "Connect a ChatGPT account to use audio"); + context = resolved.authContext; + selected = resolved.headers; + recordOutcome = resolved.recordOutcome; + } + options.signal?.throwIfAborted(); + if (!isCodexAuthContextUsable(context, config)) { + releaseCodexAuthContextProbeLease(context); + return formatErrorResponse(401, "authentication_error", "Selected audio account is unavailable"); + } + validateForwardAdmissionCredential(selected, config); + log.provider = formatCodexProviderForLog(candidate.providerName, context.accountId, config); + log.model = options.model; + return { + providerName: candidate.providerName, + providerBaseUrl: candidate.provider.baseUrl, + headers: Object.fromEntries(selected), + keyed: false, + authContext: context, + recordOutcome, + release: () => releaseCodexAuthContextProbeLease(context), + }; + } + if (candidates.keyed && options.exactAccountId === undefined) { + const { providerName, provider, apiKey } = candidates.keyed; + const selected = new Headers(provider.headers); + selected.set("authorization", `Bearer ${apiKey}`); + validateForwardAdmissionCredential(selected, config); + log.provider = providerName; + log.model = options.model; + return { + providerName, providerBaseUrl: provider.baseUrl, headers: Object.fromEntries(selected), + keyed: true, release: () => {}, + }; + } + return formatErrorResponse(400, "invalid_request_error", "Audio requires a connected ChatGPT account or OpenAI API provider"); + } catch (error) { + releaseCodexAuthContextProbeLease(context); + if (error instanceof CodexAccountCooldownError) return cooldownErrorResponse(error); + if (error instanceof CodexMainProfileDrainingError) return codexMainProfileDrainingResponse(); + return formatErrorResponse(401, "authentication_error", "Audio account authentication unavailable; reconnect the selected OpenAI account"); + } +} diff --git a/src/server/auth-cors.ts b/src/server/auth-cors.ts index 4bf99b4f0f..37ce4b1127 100644 --- a/src/server/auth-cors.ts +++ b/src/server/auth-cors.ts @@ -482,6 +482,7 @@ export interface ApiAuthMatrixRow { * against every cell rather than reading the table back to itself. */ export const AUTH_MATRIX: readonly ApiAuthMatrixRow[] = [ + { endpoint: "/v1/audio/transcriptions", bearer: "accepted", dedicated: "accepted", xApiKey: "accepted" }, // #1686: a bearer that is one of OUR admission secrets is now accepted here. It is safe // because materializeCodexUpstreamAuth substitutes the stored main credential rather than // forwarding it; a bearer that is NOT our secret stays unadmitted and remains Codex Direct diff --git a/src/server/index.ts b/src/server/index.ts index 3e458a3fea..5460f443e9 100644 --- a/src/server/index.ts +++ b/src/server/index.ts @@ -204,6 +204,8 @@ import { } from "../codex/native-profile-startup"; import { handleImages } from "./images"; import { handleLive, logLiveSidebandFrame, parseLiveSidebandTarget, resolveLiveSidebandUpgrade } from "./live"; +import { handleAudioTranscriptions } from "./audio-transcriptions"; +import { resolveAudioAdmission, TRANSCRIPTION_MODEL } from "./audio-upstream"; import { handleSearch } from "./search"; import { handleContextHistory } from "./context-history"; import { codexCompatibleUrl, contextEndpoint, contextRelayActivated } from "../codex/context-compat"; @@ -846,6 +848,7 @@ export function startServer(port?: number, deps: StartServerDeps = {}): Server { + const response = await handleAudioTranscriptions(req, config, logCtx, admission, lease); + addFinalRequestLog(requestId, start, logCtx, response.status); + return withCors(response, req, policy); + }); + } + // ChatGPT / Codex App voice (GPT‑Live / Frameless Bidi) + OpenAI Realtime call-create. // Clients hit either /v1/live (Frameless App) or /v1/realtime/calls (codex RealtimeCallClient / // public Realtime API). Sideband WS joins are handled just below. diff --git a/src/server/live.ts b/src/server/live.ts index eab37d6bda..7d6b5c7357 100644 --- a/src/server/live.ts +++ b/src/server/live.ts @@ -424,14 +424,20 @@ export async function readBodyCapped( stream: ReadableStream | null, maxBytes: number, tooLargeMessage: (total: number) => string, + signal?: AbortSignal, ): Promise { if (!stream) return new ArrayBuffer(0); const reader = stream.getReader(); + const abortRead = () => { void reader.cancel(signal?.reason).catch(() => {}); }; + signal?.addEventListener("abort", abortRead, { once: true }); const chunks: Uint8Array[] = []; let total = 0; try { + if (signal?.aborted) abortRead(); + signal?.throwIfAborted(); for (;;) { const { done, value } = await reader.read(); + signal?.throwIfAborted(); if (done) break; if (!value || value.byteLength === 0) continue; total += value.byteLength; @@ -450,6 +456,7 @@ export async function readBodyCapped( await reader.cancel(err).catch(() => {}); throw err; } finally { + signal?.removeEventListener("abort", abortRead); try { // Always release: `reader.cancel()` does NOT drop the lock, and holding it would leave // the stream permanently locked for any later consumer (audit R-WP5-2). diff --git a/structure/data-planes/inbound-compat.md b/structure/data-planes/inbound-compat.md index 3e940b27a4..01b3db72a8 100644 --- a/structure/data-planes/inbound-compat.md +++ b/structure/data-planes/inbound-compat.md @@ -1,5 +1,22 @@ # Inbound Compatibility Surfaces +## Standalone file transcription + +`src/server/audio-transcriptions.ts` owns `POST /v1/audio/transcriptions`, independently of +Responses and Chat conversion. `src/server/audio-upstream.ts` resolves explicit data-plane keys +on both listeners and substitutes stored OpenAI credentials. Direct stored-main access claims +the enclosing admission lease; Pool uses the existing sidecar account resolver. A selected +ChatGPT authentication failure never falls through to the paid OpenAI provider. + +The bounded multipart input accepts one nonempty file up to 25,000,000 bytes within a 32 MiB +body, required model, and optional prompt, language and JSON/text response format. Unknown or +duplicate fields fail. Subscription requests use the compatibility identifier gpt-4o-transcribe +and send no model upstream; the keyed API also accepts gpt-4o-mini-transcribe and whisper-1. +Responses retain only text. Manual redirects, capped response reads and linked cancellation +keep credentials and audio content out of redirects, request logs and durable storage. +`tests/server/audio-transcriptions.test.ts` exercises the real ingress and synthetic upstream; +`tests/server/api-key-attribution.test.ts` uses multipart fixtures for the HTTP auth matrix. + ## Chat Completions inbound native path `POST /v1/chat/completions` sends eligible `openai-chat` routes directly to the provider's Chat diff --git a/structure/providers/openai-tiers.md b/structure/providers/openai-tiers.md index 50c7d28ce3..8080c8bf04 100644 --- a/structure/providers/openai-tiers.md +++ b/structure/providers/openai-tiers.md @@ -404,6 +404,13 @@ HTTP/SSE, Responses WebSocket, compact, images, search, and vision resolve the s There is one mode-aware `openai` forward sidecar candidate; `openai-apikey` is not a ChatGPT-forward sidecar candidate and cannot hide a failed Codex credential with separately billed API usage. +`src/server/audio-upstream.ts` uses the same selection for standalone transcription. Explicit +native Direct auth remains caller-owned; proxy-key-only Direct claims stored main before +materialization. `src/providers/openai-sidecar.ts` releases quota-probe ownership on every +materialization or usability failure before transferring a resolved context to its caller. +Audio reports one terminal upstream outcome after validating the response body; redirects remain +neutral and client/shutdown cancellation does not manufacture an account failure. + The dashboard presents one OpenAI Codex card with accessible Pool/Direct controls and a separate, unchanged API-key card. `PATCH /api/providers?name=openai` persists exactly one `codexAccountMode`, clears affinity/quota cache, primes only when entering Pool, and does not refresh diff --git a/structure/runtime.md b/structure/runtime.md index 7a0231f097..94541ef247 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -9,6 +9,7 @@ | `src/cli/index.ts` | `ocx` / `opencodex` CLI. Lifecycle: init, start, stop, restart, status, sync, restore/eject, gui, service, update. Configuration: provider, account, models, combo/route, access, integrations, v2. Client launchers: Claude, OpenCode, MiniMax Code, and MiniMax CLI text. The MMX launcher owns a child-lifetime loopback path bridge from the client's hard-coded `/anthropic/v1/messages` path to the canonical `/v1/messages` data plane; the server does not expose an extra auth surface. Diagnostics: doctor, debug, observe, health. Windows adds tray. The full command surface is `src/cli/help.ts`; this table names the groups, not every verb. After help/version early exits, ordinary commands run the bounded best-effort Codex-shim auto-restore policy before dispatch. `system codex-cli-update` is the deliberate read-only exception and suppresses auto-restore for its whole namespace, including malformed invocations. Keeps the `#!/usr/bin/env bun` shebang for from-source dev (`bun run src/cli/index.ts`). | | `src/server/index.ts` | Bun server entrypoint: `startServer`, `/v1/responses` HTTP + WebSocket routing (compact handled before generic Responses), exact `POST /v1/images/generations` and `POST /v1/images/edits` routing, `/v1/models`, the Anthropic-shaped `/v1/messages` and OpenAI-shaped `/v1/chat/completions` compatibility surfaces, the Live/Realtime surface, the hosted-search relay, artifact serving, `/healthz`, the `/api/*` auth gate, the `/v1/*` JSON 404 guard, GUI fallback, the opt-in loopback-only hub-management listener, and facade re-exports for split server modules. | | `src/server/images.ts` | Standalone Images data plane: default OpenAI or explicit custom-provider selection, Codex account affinity, bounded opaque request relay, single-attempt upstream fetch, pool health recording, and safe response/cancellation relay. | +| `src/server/audio-transcriptions.ts` | Standalone multipart transcription; audio-specific key admission, bounded upload/response, stored OpenAI credential resolution and lease-bound cancellation. See [audio contracts](data-planes/inbound-compat.md#standalone-file-transcription). | | `src/config.ts` | Persisted `~/.opencodex/config.json` schema, defaults, migrations, transactions, and compatibility re-exports for split config modules. | | `src/config/paths.ts` | Resolves `OPENCODEX_HOME`, `config.json`, and owner-only directory hardening. | | `src/config/atomic-write.ts` | Shared synchronous/asynchronous temp-harden-rename writer and residual-temp failure contract. | @@ -61,7 +62,7 @@ listener, the optional unauthenticated data-loopback listener, and the optional listener. The data-loopback socket serves a fixed data-plane allowlist: Responses and its compact sibling, -the native search relay, the standalone Images POSTs, `GET /v1/models`, the realtime voice shapes, +the native search relay, the standalone Images POSTs, keyed file transcription, `GET /v1/models`, the realtime voice shapes, and the Anthropic and OpenAI chat wires the host's own local clients speak — `POST /v1/messages`, `POST /v1/messages/count_tokens`, and `POST /v1/chat/completions`. It never serves `/api/*`, `/healthz`, `/readyz`, or GUI routes, so local management discovery has to use an authenticated diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 0118a80600..63d04e6aa3 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -51,6 +51,7 @@ "antigravity-baseurl-override.test.ts": "adapters/google", "antigravity-static-catalog.test.ts": "adapters/google", "api-access-endpoints.test.ts": "server", + "audio-transcriptions.test.ts": "server", "api-catalog-route.test.ts": "server", "api-codex-log-guard-compact.test.ts": "server", "api-codex-log-guard-protection.test.ts": "server", diff --git a/tests/server/api-key-attribution.test.ts b/tests/server/api-key-attribution.test.ts index 6be92c4652..6bc42acf62 100644 --- a/tests/server/api-key-attribution.test.ts +++ b/tests/server/api-key-attribution.test.ts @@ -634,10 +634,15 @@ describe("AUTH_MATRIX is true of the running server", () => { // than admission. /v1/catalog joined this set in #809. const isGet = row.endpoint === "/v1/models" || row.endpoint === "/v1/catalog" || row.endpoint === "/v1/hub-state"; + const audio = row.endpoint === "/v1/audio/transcriptions" ? new FormData() : null; + if (audio) { + audio.append("model", "gpt-4o-transcribe"); + audio.append("file", new File([new Uint8Array([0, 0])], "sample.wav", { type: "audio/wav" })); + } const res = await fetch(new URL(row.endpoint, server.url), { method: isGet ? "GET" : "POST", - headers: { "content-type": "application/json", ...headers }, - ...(isGet ? {} : { body: JSON.stringify({ model: "test/gpt-test", input: "hi", messages: [{ role: "user", content: "hi" }] }) }), + headers: { ...(audio ? {} : { "content-type": "application/json" }), ...headers }, + ...(isGet ? {} : { body: audio ?? JSON.stringify({ model: "test/gpt-test", input: "hi", messages: [{ role: "user", content: "hi" }] }) }), }); // A 401 means the header was refused; anything else means it got past // admission (the upstream is disabled, so later failures are expected). diff --git a/tests/server/audio-transcriptions.test.ts b/tests/server/audio-transcriptions.test.ts new file mode 100644 index 0000000000..98e44e6626 --- /dev/null +++ b/tests/server/audio-transcriptions.test.ts @@ -0,0 +1,390 @@ +import { afterEach, beforeEach, describe, expect, spyOn, test } from "bun:test"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { saveConfig } from "../../src/config"; +import { saveCodexAccountCredential } from "../../src/codex/account-store"; +import { clearAccountQuota } from "../../src/codex/auth-api"; +import { clearMainAccountInfoCache } from "../../src/codex/main-account-cache"; +import { resetMainCodexAccountIdentityTrackingForTests } from "../../src/codex/account-lifecycle"; +import { waitForNativeMainStartupGate } from "../../src/codex/native-profile-startup"; +import { clearCodexUpstreamHealth, clearThreadAccountMap } from "../../src/codex/routing"; +import * as routing from "../../src/codex/routing"; +import * as authContext from "../../src/codex/auth-context"; +import { startServer } from "../../src/server"; +import { handleAudioTranscriptions, AUDIO_BODY_MAX_BYTES, AUDIO_FILE_MAX_BYTES } from "../../src/server/audio-transcriptions"; +import { resetLifecycleDrainStateForTests } from "../../src/server/lifecycle"; +import { acquireNativeMainProfileDrain, abortAndReleaseAllTurns, getActiveTurnCount, tryAdmitTurn } from "../../src/server/lifecycle"; +import type { OcxConfig } from "../../src/types"; +import { fakeChatGptJwt } from "../helpers/fake-chatgpt-jwt"; +import { installIsolatedCodexHome, type IsolatedCodexHome } from "../helpers/isolated-codex-home"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { ownedServiceHomeInspection } from "../helpers/owned-service-home-inspection"; + +const KEY = "ocx_data_audio_test_client"; +const originalFetch = globalThis.fetch; +const previousHome = process.env.OPENCODEX_HOME; +const previousToken = process.env.OPENCODEX_API_AUTH_TOKEN; +let home = ""; +let codex: IsolatedCodexHome; +let captured: Request[]; +let respond: (req: Request) => Response | Promise; +let server: ReturnType | undefined; + +function config(): OcxConfig { + return { + port: 0, hostname: "127.0.0.1", defaultProvider: "openai-apikey", openaiProviderTierVersion: 2, + providers: { "openai-apikey": { adapter: "openai-responses", baseUrl: "https://api.openai.com/v1", apiKey: "fixture-upstream-key", authMode: "key" } }, + apiKeys: [{ id: "audio-client", name: "audio", key: KEY, createdAt: "2026-09-12T00:00:00Z" }], + } as OcxConfig; +} + +function form(fields: Record = {}): FormData { + const data = new FormData(); + data.append("file", new File([new Uint8Array([82, 73, 70, 70, 0, 0])], "sample.wav", { type: "audio/wav" })); + data.append("model", "gpt-4o-transcribe"); + for (const [name, value] of Object.entries(fields)) data.set(name, value); + return data; +} + +function savePoolConfig(): OcxConfig { + const cfg = config(); + cfg.defaultProvider = "openai"; + cfg.providers = { openai: { adapter: "openai-responses", baseUrl: "https://chatgpt.com/backend-api/codex", authMode: "forward", codexAccountMode: "pool" } }; + cfg.codexAccounts = [{ id: "pool-a", email: "pool@example.test", isMain: false, chatgptAccountId: "acct-pool-a" }]; + cfg.activeCodexAccountId = "pool-a"; + saveCodexAccountCredential("pool-a", { accessToken: fakeChatGptJwt({ chatgpt_account_id: "acct-pool-a" }), refreshToken: "fixture-refresh", expiresAt: Date.now() + 3_600_000, chatgptAccountId: "acct-pool-a" }); + saveConfig(cfg); + return cfg; +} + +async function request(body: BodyInit = form(), headers: Record = {}): Promise { + server ??= startServer(0, { inspectNativeCodexOwnership: ownedServiceHomeInspection("audio fixture") }); + await waitForNativeMainStartupGate(); + return originalFetch(new URL("/v1/audio/transcriptions", server.url), { + method: "POST", body, headers: { authorization: `Bearer ${KEY}`, ...headers }, + }); +} + +beforeEach(() => { + home = mkdtempSync(join(tmpdir(), "ocx-audio-")); + process.env.OPENCODEX_HOME = home; + delete process.env.OPENCODEX_API_AUTH_TOKEN; + codex = installIsolatedCodexHome("ocx-audio-codex-"); + clearAccountQuota(); + clearCodexUpstreamHealth(); + clearThreadAccountMap(); + clearMainAccountInfoCache(); + resetMainCodexAccountIdentityTrackingForTests(); + resetLifecycleDrainStateForTests(); + captured = []; + respond = () => Response.json({ text: "synthetic transcript" }); + globalThis.fetch = (async (input: RequestInfo | URL, init?: RequestInit) => { + const req = new Request(input, init); + if (["api.openai.com", "chatgpt.com"].includes(new URL(req.url).hostname)) { + if (!req.url.endsWith("/transcribe") && !req.url.endsWith("/audio/transcriptions")) return Response.json({}); + captured.push(req); + return respond(req); + } + throw new Error("Unexpected non-fixture outbound request"); + }) as typeof fetch; + saveConfig(config()); +}); + +afterEach(async () => { + await server?.stop(true); + server = undefined; + globalThis.fetch = originalFetch; + clearAccountQuota(); + clearCodexUpstreamHealth(); + clearThreadAccountMap(); + clearMainAccountInfoCache(); + resetMainCodexAccountIdentityTrackingForTests(); + resetLifecycleDrainStateForTests(); + codex.restore(); + if (previousHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previousHome; + if (previousToken === undefined) delete process.env.OPENCODEX_API_AUTH_TOKEN; + else process.env.OPENCODEX_API_AUTH_TOKEN = previousToken; + removeTreeWithRetry(home); +}); + +describe("standalone transcription API", () => { + test("forwards multipart bytes with upstream credentials and returns only transcript", async () => { + respond = () => Response.json({ text: "synthetic transcript", internal: "must not escape" }); + const response = await request(form({ prompt: "technical terms", language: "ko" })); + expect(response.status).toBe(200); + expect(await response.json()).toEqual({ text: "synthetic transcript" }); + expect(captured).toHaveLength(1); + const upstream = captured[0]!; + expect(upstream.url).toBe("https://api.openai.com/v1/audio/transcriptions"); + expect(upstream.headers.get("authorization")).toBe("Bearer fixture-upstream-key"); + expect(upstream.headers.get("x-opencodex-api-key")).toBeNull(); + const data = await upstream.formData(); + expect(data.get("model")).toBe("gpt-4o-transcribe"); + expect(data.get("prompt")).toBe("technical terms"); + expect(data.get("language")).toBe("ko"); + expect(Array.from(new Uint8Array(await (data.get("file") as File).arrayBuffer()))).toEqual([82, 73, 70, 70, 0, 0]); + }); + + test("text format is a text response while upstream uses JSON", async () => { + const response = await request(form({ response_format: "text" })); + expect(response.headers.get("content-type")).toContain("text/plain"); + expect(await response.text()).toBe("synthetic transcript"); + }); + + test("explicit bad dedicated key cannot fall through to a valid bearer on loopback", async () => { + expect((await request(form(), { "x-opencodex-api-key": "wrong" })).status).toBe(401); + expect((await request(form(), { authorization: "Bearer wrong" })).status).toBe(401); + expect((await request(form(), { authorization: "" })).status).toBe(401); + expect(captured).toHaveLength(0); + }); + + test("hostile origin is rejected even with a valid key", async () => { + expect((await request(form(), { origin: "https://untrusted.example" })).status).toBe(403); + expect(captured).toHaveLength(0); + }); + + test("duplicate, missing and unsupported fields fail before upstream", async () => { + const duplicate = form(); duplicate.append("model", "whisper-1"); + const missing = form(); missing.delete("file"); + for (const body of [duplicate, missing, form({ model: "chat-model" }), form({ stream: "true" }), form({ response_format: "srt" })]) { + expect((await request(body)).status).toBe(400); + } + expect(captured).toHaveLength(0); + }); + + test("malformed and compressed multipart are rejected", async () => { + expect((await request("broken", { "content-type": "multipart/form-data; boundary=test" })).status).toBe(400); + expect((await request(form(), { "content-encoding": "gzip" })).status).toBe(400); + expect(captured).toHaveLength(0); + }); + + test("file and text limits are enforced", async () => { + const oversized = form(); oversized.set("file", new File([new Uint8Array(AUDIO_FILE_MAX_BYTES + 1)], "large.wav")); + expect((await request(oversized)).status).toBe(413); + expect((await request(form({ prompt: "x".repeat(16 * 1024 + 1) }))).status).toBe(413); + expect(captured).toHaveLength(0); + }); + + test("chunked body limit cancels the reader before account resolution", async () => { + let canceled = false; + const body = new ReadableStream({ + start(controller) { controller.enqueue(new Uint8Array(AUDIO_BODY_MAX_BYTES + 1)); }, + cancel() { canceled = true; }, + }); + const req = new Request("http://localhost/v1/audio/transcriptions", { method: "POST", headers: { "content-type": "multipart/form-data; boundary=x" }, body }); + const result = await handleAudioTranscriptions(req, config(), { model: "unknown", provider: "unknown" }, { kind: "configured", keyId: "audio-client", source: "bearer" }); + expect(result.status).toBe(413); + expect(canceled).toBe(true); + expect(captured).toHaveLength(0); + }); + + test("declared body limit rejects without pulling audio", async () => { + let canceled = false; + const req = new Request("http://localhost/v1/audio/transcriptions", { + method: "POST", headers: { "content-type": "multipart/form-data; boundary=x", "content-length": String(AUDIO_BODY_MAX_BYTES + 1) }, + body: new ReadableStream({ cancel() { canceled = true; } }), + }); + const response = await handleAudioTranscriptions(req, config(), { model: "unknown", provider: "unknown" }, { kind: "configured", keyId: "audio-client", source: "bearer" }); + expect(response.status).toBe(413); + expect(canceled).toBe(true); + expect(captured).toHaveLength(0); + }); + + test("client cancellation aborts the upstream request", async () => { + const controller = new AbortController(); + const ready = Promise.withResolvers(); + let aborted = false; + respond = req => new Promise((_resolve, reject) => { + req.signal.addEventListener("abort", () => { aborted = true; reject(req.signal.reason); }, { once: true }); + ready.resolve(); + }); + const pending = handleAudioTranscriptions(new Request("http://localhost/v1/audio/transcriptions", { + method: "POST", body: form(), signal: controller.signal, + }), config(), { model: "unknown", provider: "unknown" }, { kind: "configured", keyId: "audio-client", source: "bearer" }); + await ready.promise; + controller.abort(); + expect((await pending).status).toBe(499); + expect(aborted).toBe(true); + }); + + test("upload deadline cancels a stalled reader and releases its turn", async () => { + const schedule = globalThis.setTimeout; + let expire: (() => void) | undefined; + const timers = spyOn(globalThis, "setTimeout").mockImplementation(((callback: (...args: unknown[]) => void, delay?: number, ...args: unknown[]) => { + if (delay === 30_000) expire = () => callback(...args); + return schedule(callback, delay, ...args); + }) as typeof setTimeout); + let canceled = false; + const lease = tryAdmitTurn()!; + try { + const pending = handleAudioTranscriptions(new Request("http://localhost/v1/audio/transcriptions", { + method: "POST", headers: { "content-type": "multipart/form-data; boundary=x" }, + body: new ReadableStream({ cancel() { canceled = true; } }), + }), config(), { model: "unknown", provider: "unknown" }, { kind: "configured", keyId: "audio-client", source: "bearer" }, lease); + expect(expire).toBeDefined(); + expire!(); + expect((await pending).status).toBe(408); + expect(canceled).toBe(true); + expect(getActiveTurnCount()).toBe(0); + expect(captured).toHaveLength(0); + } finally { timers.mockRestore(); lease.release(); } + }); + + test("shutdown aborts upstream through the registered turn controller", async () => { + const ready = Promise.withResolvers(); + let aborted = false; + respond = req => new Promise((_resolve, reject) => { + req.signal.addEventListener("abort", () => { aborted = true; reject(req.signal.reason); }, { once: true }); + ready.resolve(); + }); + const lease = tryAdmitTurn()!; + const pending = handleAudioTranscriptions(new Request("http://localhost/v1/audio/transcriptions", { + method: "POST", body: form(), + }), config(), { model: "unknown", provider: "unknown" }, { kind: "configured", keyId: "audio-client", source: "bearer" }, lease); + await ready.promise; + abortAndReleaseAllTurns(); + expect((await pending).status).toBe(503); + expect(aborted).toBe(true); + expect(getActiveTurnCount()).toBe(0); + }); + + test("malformed responses, upstream errors and redirects remain content-free errors", async () => { + for (const reply of [new Response("not-json"), Response.json({ missing: true }), new Response("private detail", { status: 429 }), new Response("", { status: 302, headers: { location: "https://untrusted.example" } })]) { + respond = () => reply; + const response = await request(); + expect(response.status).toBe(reply.status === 429 ? 429 : 502); + expect(await response.text()).not.toContain("private detail"); + } + }); + + test("oversized upstream result is rejected and canceled", async () => { + let canceled = false; + respond = () => new Response(new ReadableStream({ + start(controller) { controller.enqueue(new Uint8Array(2 * 1024 * 1024 + 1)); }, + cancel() { canceled = true; }, + })); + expect((await request()).status).toBe(502); + expect(canceled).toBe(true); + }); + + test("stored Direct credentials replace the proxy key", async () => { + writeFileSync(join(codex.path, "auth.json"), JSON.stringify({ tokens: { access_token: "fixture-main-access", account_id: "fixture-main-account" } })); + clearMainAccountInfoCache(); + const cfg = config(); + cfg.defaultProvider = "openai"; + cfg.providers = { openai: { adapter: "openai-responses", baseUrl: "https://chatgpt.com/backend-api/codex", authMode: "forward", codexAccountMode: "direct" } }; + saveConfig(cfg); + expect((await request()).status).toBe(200); + expect(captured[0]!.url).toBe("https://chatgpt.com/backend-api/transcribe"); + expect(captured[0]!.headers.get("authorization")).toBe("Bearer fixture-main-access"); + expect(captured[0]!.headers.get("user-agent")).toBe("codex_cli_rs"); + expect(captured[0]!.headers.get("originator")).toBe("codex_cli_rs"); + expect((await captured[0]!.formData()).get("model")).toBeNull(); + }); + + test("a missing stored Direct credential fails without paid-provider fallback", async () => { + const cfg = config(); + cfg.providers.openai = { adapter: "openai-responses", baseUrl: "https://chatgpt.com/backend-api/codex", authMode: "forward", codexAccountMode: "direct" }; + saveConfig(cfg); + expect((await request()).status).toBe(401); + expect(captured).toHaveLength(0); + }); + + test("a validated explicit Direct caller remains distinct from stored main", async () => { + const cfg = config(); + cfg.defaultProvider = "openai"; + cfg.providers = { openai: { adapter: "openai-responses", baseUrl: "https://chatgpt.com/backend-api/codex", authMode: "forward", codexAccountMode: "direct" } }; + saveConfig(cfg); + const token = fakeChatGptJwt({ chatgpt_account_id: "explicit-native" }); + expect((await request(form(), { "x-opencodex-api-key": KEY, authorization: `Bearer ${token}`, "chatgpt-account-id": "explicit-native" })).status).toBe(200); + expect(captured[0]!.headers.get("authorization")).toBe(`Bearer ${token}`); + expect(captured[0]!.headers.get("chatgpt-account-id")).toBe("explicit-native"); + }); + + test("draining stored Direct profile does not dispatch transcription", async () => { + writeFileSync(join(codex.path, "auth.json"), JSON.stringify({ tokens: { access_token: "fixture-main-access", account_id: "fixture-main-account" } })); + clearMainAccountInfoCache(); + const cfg = config(); + cfg.providers.openai = { adapter: "openai-responses", baseUrl: "https://chatgpt.com/backend-api/codex", authMode: "forward", codexAccountMode: "direct" }; + saveConfig(cfg); + server = startServer(0, { inspectNativeCodexOwnership: ownedServiceHomeInspection("audio fixture") }); + await waitForNativeMainStartupGate(); + const drain = acquireNativeMainProfileDrain("audio-test"); + expect(drain).not.toBeNull(); + try { + expect((await request()).status).toBe(503); + expect(captured).toHaveLength(0); + } finally { drain?.release(); } + }); + + test("Pool substitutes the selected stored account and strips compatibility model", async () => { + savePoolConfig(); + expect((await request()).status).toBe(200); + expect(captured[0]!.headers.get("chatgpt-account-id")).toBe("acct-pool-a"); + expect(captured[0]!.headers.get("authorization")).not.toContain(KEY); + expect((await captured[0]!.formData()).get("model")).toBeNull(); + }); + + test("malformed Pool response records one failure and no provisional success", async () => { + savePoolConfig(); + respond = () => Response.json({ missing: "text" }); + const outcomes = spyOn(routing, "recordCodexUpstreamOutcome"); + try { + expect((await request()).status).toBe(502); + expect(outcomes.mock.calls.filter(call => call[1] === "pool-a").map(call => call[2])).toEqual([502]); + } finally { outcomes.mockRestore(); } + }); + + test("overall timeout records one Pool timeout after dispatch", async () => { + const cfg = savePoolConfig(); + const schedule = globalThis.setTimeout; + let expire: (() => void) | undefined; + const timers = spyOn(globalThis, "setTimeout").mockImplementation(((callback: (...args: unknown[]) => void, delay?: number, ...args: unknown[]) => { + if (delay === 120_000 && !expire) expire = () => callback(...args); + return schedule(callback, delay, ...args); + }) as typeof setTimeout); + const ready = Promise.withResolvers(); + respond = req => new Promise((_resolve, reject) => { + req.signal.addEventListener("abort", () => reject(req.signal.reason), { once: true }); + ready.resolve(); + }); + const outcomes = spyOn(routing, "recordCodexUpstreamOutcome"); + const lease = tryAdmitTurn()!; + try { + const pending = handleAudioTranscriptions(new Request("http://localhost/v1/audio/transcriptions", { method: "POST", body: form() }), cfg, + { model: "unknown", provider: "unknown" }, { kind: "configured", keyId: "audio-client", source: "bearer" }, lease); + await ready.promise; + expect(expire).toBeDefined(); + expire!(); + expect((await pending).status).toBe(504); + expect(outcomes.mock.calls.filter(call => call[1] === "pool-a").map(call => call[2])).toEqual(["timeout"]); + } finally { outcomes.mockRestore(); timers.mockRestore(); lease.release(); } + }); + + test("Pool redirect is recorded once without penalizing the account", async () => { + savePoolConfig(); + respond = () => new Response("", { status: 302, headers: { location: "https://untrusted.example" } }); + const before = routing.getCodexUpstreamHealth("pool-a"); + const outcomes = spyOn(routing, "recordCodexUpstreamOutcome"); + try { + expect((await request()).status).toBe(502); + expect(outcomes.mock.calls.filter(call => call[1] === "pool-a").map(call => call[2])).toEqual([302]); + expect(routing.getCodexUpstreamHealth("pool-a")).toEqual(before); + } finally { outcomes.mockRestore(); } + }); + + test.each(["materialization", "usability"] as const)("Pool %s failure releases the acquired context before returning", async failure => { + savePoolConfig(); + const release = spyOn(authContext, "releaseCodexAuthContextProbeLease"); + const fault = failure === "materialization" + ? spyOn(authContext, "headersForCodexAuthContext").mockImplementation(() => { throw new Error("fixture materialization failure"); }) + : spyOn(authContext, "isCodexAuthContextUsable").mockReturnValue(false); + try { + expect((await request()).status).toBe(401); + expect(release.mock.calls.some(([ctx]) => ctx?.accountId === "pool-a")).toBe(true); + expect(captured).toHaveLength(0); + } finally { fault.mockRestore(); release.mockRestore(); } + }); +});