feat(dashboard): add dictation and live voice API controls

devlog/_plan/260912_audio_apis_stack/021_streaming_checks.md

@@ -0,0 +1,39 @@
+# Streaming outcome and next layer
+
+Functional layer: PASS at `011f2dff5ca3667b88f090fe711c4b2c77efd190`, PR #4392 onto
+`codex/audio-transcription` (`71e22d967ffafbd3492935c299623c9127eaf17b`).
+Remote run [34687731903](https://github.com/lidge-jun/opencodex/actions/runs/34687731903)
+tested merge `df1c26d1`.
+
+- Audio client: 11 passing cases, job 103537614446.
+- File transcription: 24 passing cases, job 103537614468.
+- Dictation/voice ingress and lifecycle: 11 passing cases, job 103537614422.
+- Call bindings: four passing cases on Linux and macOS, job 103537614428.
+- Gates: typecheck, 1,979 dashboard tests, privacy, skill surface, release syntax
+  and CLI smoke passed, job 103537614453. Dashboard build skipped for this layer.
+- Independent inherited source reviews: Ramanujan closure PASS; Feynman final
+  PASS after native platform-key compatibility and neutral WebSocket accounting
+  corrections. No unresolved blocking finding in these bounded reviews.
+
+All checks above ran remotely. Local product tests, typecheck, build and install
+were NOT RUN for this layer, per owner instruction. Pushes used `--no-verify`.
+No real upstream audio or personal recording was used.
+
+## Separate baseline failures
+
+Linux test 2/4 (103537614443) and macOS 2/2 (103537614422) still report
+`tests/codex-integration/codex-journal.test.ts:170` (failed versus skipped restore)
+and `:528` (routing retained after compensated failed restore). These failures
+were observed before the final audio changes. They are recorded, not included
+in this feature's repair scope, following the owner's explicit decision.
+An earlier run also showed the stale-process status assertion at
+`tests/cli/cli-status-json.test.ts:893`; do not claim it repaired without evidence.
+Bun batch crashes that recovered through CI singleton retries are not runtime-fix
+evidence. Whole-run green and merge readiness are not claimed.
+
+## Next
+
+Proceed to wp3: configured endpoint metadata, separate Dictation and Live Voice
+controls, synthetic browser QA of the CI-built dashboard, and ordinary stacked
+publication. Configuration is not entitlement or observed connectivity. Leave
+all PRs open; aggregate baseline failures remain a separate publication note.

devlog/_plan/260912_audio_apis_stack/030_connections.md

@@ -2,19 +2,31 @@
 
 Depends on wp2 completed audio routes. The first two layers remain independently usable through external client examples.
 
+P revalidation: wp2 D concluded functional audio tests and source reviews PASS at
+011f2dff5c, with unrelated journal failures recorded separately. Follow that
+direction without expanding baseline repairs. C4 because transient credentials
+cross the browser/data-plane boundary. Main owns implementation; inherited
+architect and independent reviewers remain read-only. No additional cost/time
+budget was imposed. No paid calls, personal audio, service changes or merge.
+Local product suites/typecheck/build/install are prohibited. Existing remote CI
+runs the checks; browser QA reads its built artifact with synthetic routes only.
+
 ## File changes
 
 | Operation | Path | Contract change |
 | --- | --- | --- |
 | MODIFY | src/server/management/api-access.ts | extend ApiAccessEndpoints with transcription, dictationStream, live and realtimeCalls URLs plus truthful capability metadata |
-| MODIFY | src/server/management-api.ts | include audio metadata using existing management authentication |
+| EXISTING | src/server/management/oauth-account-routes.ts | /api/keys already serializes ...endpoints; no new route or management authority |
 | MODIFY | tests/server/api-access-endpoints.test.ts | URL host/protocol and capability projection tests |
 | MODIFY | gui/src/pages/api-keys-utils.ts | extend endpoint type/default/derive chain for new endpoints |
 | MODIFY | gui/src/pages/ApiKeys.tsx | consume serialized endpoint metadata through KeysResponse, CachedKeysShape, cache validation and fetchKeys |
 | MODIFY | gui/src/components/apikeys-workspace/ApiKeysWorkspace.tsx | place two unframed audio sections in existing Connections/API layout |
 | NEW | gui/src/components/apikeys-workspace/AudioApiPanel.tsx | accessible Dictation and Live Voice controls, endpoint/model display, sample copying, transient key/file controls and result/error states |
 | NEW | gui/src/audio-api-client.ts | bounded cancelable upload and socket client protocol helpers; no saved secrets |
-| NEW | gui/tests/audio-api-client.test.ts | request generation, cancellation and transcript assembly tests |
+| NEW | gui/src/audio-api-examples.ts | executable protocol examples outside component markup; localized prompt label supplied by caller |
+| MODIFY | gui/src/api.ts | narrow audio upload fetch entry bypasses management auth injection/recovery, validates exact inference path |
+| NEW | gui/tests/audio-api-client.test.ts | request generation, cancellation and protocol readiness tests |
+| NEW | gui/tests/audio-api-panel.test.tsx | real component upload, error, cancellation, missing metadata and deactivation flows |
 | MODIFY | gui/src/i18n/en.ts and every locale module | complete localized label/status/action keys |
 | MODIFY | gui/src/styles-apikeys-workspace.css | restrained aligned responsive audio sections using existing tokens |
 | MODIFY | structure/gui-and-management-api.md | metadata and control ownership/current contract |
@@ -26,19 +38,57 @@ Before: ApiAccessEndpoints contains Responses/chat/messages/models only. After:
 
 Metadata fields complete chain: creation buildApiAccessEndpoints -> JSON management response -> API page validation/mapping -> ApiEndpointInfo/AudioApiPanel. Defaults cannot claim configured availability. No new provider is registered and no audio model enters a text completion test.
 
+The optional `audio` projection has transcriptionEndpoint, dictationStreamEndpoint,
+liveEndpoint, realtimeCallsEndpoint, transcriptionModel, liveModel,
+transcriptionConfigured, dictationConfigured and liveConfigured. URL/model strings
+and booleans are validated on both network and cached payloads; endpoint origins
+and paths must match the published base after WS-to-HTTP normalization. Missing
+or invalid audio metadata leaves the audio controls unavailable, while existing
+key management keeps working. Configuration flags use canonical enabled provider
+configuration only, never credential resolution, account reads or a network probe.
+File transcription accepts canonical ChatGPT or configured OpenAI API routing;
+dictation/live GPT-Live flags require canonical ChatGPT routing (an API-key-only
+configuration does not prove access to the Codex live model).
+
 Dictation section contains model and endpoint copy actions, file input, transient API key input, transcribe/cancel, text result/copy and clear error states. Stream example names extension protocol and gives start/audio/close events. Live Voice section contains actual GPT-Live model and both WS/WebRTC connection endpoints, transient client key, a connect/disconnect test with status and observed event output. Browser WebSocket auth must use a short-lived local session mechanism or supported client protocol carrier; never expose ChatGPT credentials or persist raw keys. Do not create a fake success check or billable background probe. All test actions require a deliberate user click.
 
 UI is unframed and follows existing workspace colors/type/spacing. Icons reuse gui icons, all visible text is localized. At desktop and mobile widths long endpoint text wraps or scrolls within its own element without overlapping controls. Buttons have stable dimensions and stateful controls are keyboard reachable.
 
-Pass the existing active flag through ApiKeysWorkspace. Integrations hides panels without unmounting; requests, sockets and timers must stop on deactivation as well as unmount. Existing key rows contain only prefixes: controls use a newly generated key or an explicitly entered transient key, never pretend a key ID can authenticate. Browser voice connection uses an OpenCodex-only WebSocket protocol credential carrier accepted solely by the audio routes; exact supported carrier and precedence are documented/tested in wp2. No persistent key or query authentication.
+Pass the existing active flag through ApiKeysWorkspace. Integrations hides panels without unmounting; conditionally unmount just audio controls when inactive, preserving the rest of the workspace drafts. Requests, sockets and timers stop on deactivation, origin change and unmount. Existing key rows contain only prefixes: controls use an explicitly entered transient key, never pretend a key ID can authenticate. Key edits cancel pending work. Browser voice connection uses the OpenCodex-only WebSocket protocol credential carrier accepted solely by the audio routes; exact supported carrier and precedence are documented/tested in wp2. No persistent key or query authentication. Raw upstream messages are not rendered: show localized error categories and allowlisted event types only. Socket open is not success: wait for session.started/session.updated with a nonempty session.id. Probe sends no audio and closes after a bounded interval or explicit disconnect.
 
 ## Acceptance and publication
 
 1. API metadata correctly derives HTTPS/WSS, wildcard, IPv6 and companion-listener addresses and shows missing upstream as unavailable.
 2. Mocked browser flow uploads a fixture, receives text, copies it, cancels a pending call and displays a server error. No real audio/provider requests during agent QA.
 3. Mocked voice flow connects, observes a protocol event, disconnects and releases callbacks/timers; API keys never enter storage, screenshots or URL queries.
 4. Desktop and mobile browser screenshots are read back and corrected. Screenshot attached to UI PR with synthetic data only.
-5. Run GUI focused tests, lint:i18n, lint, build and repository typecheck/full suite before review ready; per-layer CI uses exact PR head. The final PRs fill Summary, Verification and Checklist plus ordinary stack map.
+5. Remote CI executes dashboard tests/lint/build and repository typecheck/suite; no local execution. Inspect exact-head logs and download its dashboard-preview artifact. The final PRs fill Summary, Verification and Checklist plus ordinary stack map. Existing unrelated CI failures stay separately documented; never attest whole-suite green.
+
+Conditional acceptance includes empty/malformed metadata (disabled controls, no
+request), file >25,000,000 bytes (local rejection before fetch), 401/429/503
+(localized categories, no raw body), aborted upload (no stale text), WS open
+without ready event (timeout), protocol error (failed, not connected), key/origin
+change and inactive/unmount (all resources closed, no late callback). Copy samples
+contain placeholders only, never the transient input. Metadata tests cover TLS,
+wildcard, IPv6 and loopback companion URLs. No new enforcement layer is claimed;
+browser guards are early UX validation, and server admission remains authoritative.
+
+## P/A review disposition
+
+Accepted architect CONN-META-01, WIRE-02, URL-03, UI-04 and LIFE-06. Folded
+CONN-PROBE-05 and the independent A review's three residuals:
+
+- On socket open send exactly `{"type":"session.update","session":{"instructions":"","audio":{"output":{"voice":"cove"}},"delegation":{"type":"client"}}}`.
+  Acknowledgments with closed/error/failed session status are terminal, never
+  ready. An error followed by normal close remains failed.
+- The API module exposes a narrow raw audio-upload entry. It validates the exact
+  HTTP(S) `/v1/audio/transcriptions` destination and bypasses installed management
+  authentication and 401 recovery. Connected-mode tests install that wrapper and
+  assert the typed data key survives, with no session/CSRF/machine credentials.
+- Endpoint validation rejects userinfo, query, fragment and incorrect schemes as
+  well as wrong origins/paths. Network and cache paths use the same validator.
+  Only the wp2 `opencodex-audio` / `opencodex-key.<base64url>` carrier is used;
+  observed output means allowlisted event types, not raw messages.
 
 Commands are defined by root/gui package.json. Source paths and existing stylesheet/fetch owner are revalidated at this cycle P before implementation; any renamed path is amended with exact ownership evidence. No disconnected metadata fields or fake audio model tests are acceptable.
 

devlog/_plan/260912_audio_apis_stack/031_connections_checks.md

@@ -0,0 +1,78 @@
+# Connections audio verification
+
+## Delivered surface
+
+Ordinary dependency chain remains open: [#4391](https://github.com/lidge-jun/opencodex/pull/4391)
+-> [#4392](https://github.com/lidge-jun/opencodex/pull/4392)
+-> [#4395](https://github.com/lidge-jun/opencodex/pull/4395).
+No native stack registration, merge, release, deployment or live-service restart.
+
+The top layer adds validated audio metadata, separate Dictation and Live Voice
+blocks, temporary data-key input, cancelable upload/copy, connection-only native
+session readiness, client examples and all nine dashboard locales. It does not
+add audio models to text completion tests. Invalid/missing metadata leaves existing
+key management usable. Configured availability is not account entitlement.
+
+## Executable proof
+
+Runtime/UI source head: `f5aefd88af3116bec4f2ddc72c0c6fa974f52a83`.
+Remote [run 34690242138](https://github.com/lidge-jun/opencodex/actions/runs/34690242138),
+gates job 103544165369: SUCCESS for lint, typecheck, dashboard tests, privacy,
+generated skill surface, release syntax, dashboard build and CLI smoke.
+Artifact 10296892796 is `dashboard-preview-bb0f30fbab4496b2b69bbc1e8148e59035ba3778`.
+Its GUI tree `5ccecf06a8e144e5260f4991047db6fa36f464a1` exactly matches source.
+The artifact names the tested PR merge, not a different runtime build.
+
+Local product tests, typecheck, build, dependency installation and suites:
+**NOT RUN**, per owner restriction. Every commit/push used `--no-verify`.
+The only local execution was a static Node file server for the CI-built artifact
+and browser QA through the already installed Playwright dependency. No proxy
+runtime, provider audio request, personal recording or microphone was used.
+
+## Browser matrix
+
+Invocation: static artifact server, then `.tmp/audio-browser-run.mjs` driving
+`/#integrations/keys` with intercepted synthetic management/audio routes.
+Installed agbrowse lacked its documented script command, so the existing
+agbrowse Playwright dependency drove a separate CDP browser on port 9231.
+No new browser dependency was installed.
+
+| Scenario | Observed result |
+| --- | --- |
+| Upload synthetic file | Expected transcript; typed data key only, no management/CSRF headers |
+| Copy transcript | Actual browser clipboard contained the exact transcript |
+| Live connect/disconnect | session.update, session.started with ID, session.close; no audio frames |
+| HTTP 401 | Localized error, raw provider material absent |
+| Cancel slow response | No late transcript published |
+| Leave/re-enter API tab | Pending resources released, temporary key cleared |
+| Storage inspection | No typed key in localStorage/sessionStorage |
+| Keyboard | Key input -> file input follows Tab order |
+| 1440/1024/768/390/320 | No audio-control overflow; settled screenshots read back |
+| Korean 1440/390/320 | Labels fit; no new-section overlap or clipped Korean text |
+| Runtime | No page JavaScript errors; no external network requests |
+
+Initial captures exposed mobile top-bar overlap and a two-line tablet section
+strip. Both were fixed; final screenshots below depict the corrected source.
+Mobile scroll-spy uses the same 108px reading line as section positioning.
+
+![Desktop audio controls](screenshots/audio-1440-light.png)
+![Mobile audio controls](screenshots/audio-390-light.png)
+![Narrow Korean audio controls](screenshots/audio-320-ko.png)
+
+`screenshots/transcription-layer-baseline.png` separately renders the bottom PR's
+own CI artifact (run 34687369123, merge d9771b3133863f5be1b89213029b0a8037dbcb89,
+GUI tree 999781a53536c82ccfbfd376083e945bf388c55d). That layer changes GUI test
+fixtures and asset provenance only; this capture does not claim the audio UI
+exists in the bottom layer.
+
+## Review and remaining limits
+
+Inherited security review closed metadata projection and post-readiness failures.
+Inherited component review closed endpoint styling, replacement-race and cache
+observation coverage. Subsequent xai/grok-4.6 review closed idle status and mobile
+scroll-spy alignment. Source verdicts: PASS. Rendered evidence is separate from
+source review; neither establishes real provider availability.
+
+Unrelated journal restore failures remain recorded in `021_streaming_checks.md`.
+Old/superseded or cancelled whole runs are not passing final-head evidence. No
+whole-suite-green or merge-readiness claim is made. PRs remain open for review.

docs-site/src/content/docs/fr/reference/proxy-formats.md

@@ -278,6 +278,8 @@ Voir [le guide Desktop](/fr/guides/claude-code/). Relecture thinking et cache re
 
 La liaison de compte ci-dessous concerne les clients Codex natifs. Pour la dictée et GPT-Live avec une clé API externe, consultez la [spécification audio en anglais](/reference/proxy-formats/#streaming-dictation).
 
+Connections > API keys propose deux sections, Dictée et Voix en direct. La clé de données reste uniquement en mémoire dans le formulaire. La dictée envoie le fichier choisi ; la vérification vocale attend une confirmation de session sans microphone. Une configuration présente ne garantit pas la connexion.
+
 `POST /v1/live` accepte la surface de création d'appel ChatGPT/Codex App sans cadre.
 `POST /v1/realtime/calls` accepte la surface de création d'appel OpenAI Realtime. opencodex sélectionne un
 route OpenAI-family éligible, normalise la demande de création d'appel pour l'authentification en amont

docs-site/src/content/docs/ja/reference/proxy-formats.md

@@ -204,6 +204,8 @@ API ではありません。Desktop のキー移行・復旧・切断は既存
 
 以下のアカウント連携は既存の Codex クライアント向けです。外部 API キーで利用する音声入力と GPT-Live は[英語版の音声 API 仕様](/reference/proxy-formats/#streaming-dictation)を参照してください。
 
+Connections > API keys に音声入力とリアルタイム音声の項目があります。データキーは入力欄のメモリにのみ保持されます。文字起こしは選択したファイルを送信し、音声の接続確認はマイクを使わずセッション応答を待ちます。設定済みの表示は接続成功を意味しません。
+
 `POST /v1/live` は、ChatGPT/Codex アプリのフレームレス通話作成サーフェスを受け入れます。 `POST /v1/realtime/calls` は、OpenAI Realtime 呼び出し作成サーフェスを受け入れます。 opencodex は、適格な OpenAI ファミリ ルートを選択し、アップストリーム認証モードのコール作成リクエストを正規化し、制限付き応答を中継します。
 
 コールの作成後、クライアントはサポートされている受信フォームを使用してサイドバンド WebSocket に参加できます。

docs-site/src/content/docs/ko/reference/proxy-formats.md

@@ -251,6 +251,8 @@ HTTP 400을 반환합니다. 두 경우 모두 날짜 제거나 다른 경로로
 
 아래 계정 연결 설명은 기존 Codex 클라이언트 기준입니다. 외부 API 키로 쓰는 받아쓰기와 GPT-Live는 [영문 음성 API 명세](/reference/proxy-formats/#streaming-dictation)를 따릅니다.
 
+Connections > API keys에는 받아쓰기와 실시간 음성 블록이 있습니다. 데이터 키는 입력란에만 잠시 유지됩니다. 받아쓰기는 선택한 파일을 전송하고, 음성 연결 확인은 마이크 없이 세션 응답을 기다립니다. 설정 표시는 실제 연결 성공을 뜻하지 않습니다.
+
 `POST /v1/live`는 ChatGPT/Codex App Frameless call-creation 표면을 받습니다.
 `POST /v1/realtime/calls`는 OpenAI Realtime call-creation 표면을 받습니다. opencodex는 적절한 OpenAI 계열
 경로를 선택하고, 업스트림 인증 모드에 맞게 call-creation 요청을 정규화한 뒤, 제한된 응답을 릴레이합니다.