[Realtime] Use Local Studio as a first-class realtime voice provider

[0xSero]

Goal

Add Local Studio as a first-class realtime voice provider in Litter while preserving native WebRTC audio processing, shared-Rust lifecycle/transcript state, and iOS/Android parity.

Litter already has native libwebrtc sessions and Rust-owned voice transcript/handoff state. The new path should replace mobile-owned provider credentials/configuration with Local Studio capability discovery and session negotiation; it must not create a parallel voice UI or duplicate lifecycle reducers in Swift and Kotlin. The Local Studio broker contract is tracked in sybil-solutions/local-studio#346, and authenticated transport in 0xSero/alleycat#39.

Architectural boundary

• Shared Rust owns provider selection, typed capabilities/errors, realtime session lifecycle, transcript normalization, reconnect state, and handoff orchestration.
• AppClient owns direct server operations; AppStore receives authoritative events/snapshots and remains minimal.
• Swift/Kotlin own libwebrtc peer connections, platform audio sessions/permissions, and render-only UI projection.
• Local Studio owns credentials, provider/model selection policy, ephemeral session negotiation, and teardown.
• Alleycat carries authenticated control/signaling; audio/media stays direct in the MVP.

Required behavior

• Add a typed provider/capability model that distinguishes direct OpenAI realtime from Local Studio brokered realtime without platform string parsing.
• Show only capabilities actually advertised by the selected server/model and surface explicit unavailability reasons.
• Request an ephemeral session, exchange SDP or the negotiated signaling form, apply session updates, and close idempotently.
• Persist enough shared state to recover cleanly after backgrounding/reconnect without duplicating a provider session.
• Keep existing transcript, dynamic-tool, remote handoff, mute/speaker, interruption, and Live Activity/Watch behavior working.
• Never persist or log provider secrets, SDP credentials, or controller credentials.

Acceptance criteria

• Shared Rust unit/contract tests cover provider selection, capability normalization, lifecycle, reconnect, expiry, and redaction.
• iOS and Android tests cover identical capability/error projections and native WebRTC offer/answer integration.
• Direct OpenAI realtime remains available and regression-tested until an explicit removal decision is made.
• A Local Studio + Alleycat + Litter e2e harness proves capability -> create -> media connected -> transcript/tool handoff -> background/reconnect -> close.
• Physical iOS and Android device acceptance verifies microphone permission, two-way audio, interruption, speaker/mute, and reconnect; simulator/CI alone is insufficient.
• End-to-end timings separate control negotiation, ICE/media connection, first transcript, and first audio response.
• No shared protocol parsing or lifecycle state machine is added to Swift/Kotlin.

PR plan

1. Shared Rust provider/capability and signaling contract with fixtures.
2. iOS adapter and tests using existing voice UI.
3. Android adapter and tests using existing voice UI.
4. Cross-repository e2e harness and physical-device acceptance evidence.

This issue does not supersede #182; that issue tracks a bug in the existing direct Android realtime path and should close when its own fix is accepted.

[0xSero]

Cross-repo foundation update: Local Studio contract PR sybil-solutions/local-studio#347 and Alleycat consumer-contract PR 0xSero/alleycat#40 are merged. Both deliberately keep capability advertisement and signaling disabled. The Litter shared-Rust consumer contract is in progress; the remaining acceptance for this issue is authenticated broker implementation plus native iOS/Android WebRTC end-to-end proof.

[0xSero]

Litter consumer-contract PR opened: #213. It consumes the same exact v1 golden fixture as the merged Local Studio and Alleycat foundations, stays entirely in shared Rust, and deliberately advertises/routes nothing until the broker and native media acceptance land.

[0xSero]

Ledger update: the inert shared-Rust Litter consumer-contract foundation in #213 is now merged, alongside the already-merged Local Studio #347 and Alleycat #40 contracts. This issue correctly remains open: capability advertisement is still disabled, and authenticated broker signaling plus native iOS/Android WebRTC end-to-end and physical-device acceptance have not landed.

[0xSero]

Ledger correction from #319 (merged as 0824605d).

local_studio_realtime.rs (1,237 lines) was removed. It had zero consumers and no uniffi derives, so none of it could cross into Swift or Kotlin even in principle — it was 25 pub types and 5 functions describing a schema, and nothing else.

Flagging it because merging that file was recorded as progress on this issue, but none of the behavioral work landed: provider selection, broker client, device auth, phase/snapshot extensions, reducer arms. Removing it makes the remaining scope visible rather than hiding it behind a merged file. Recoverable from history when this resumes.

Realtime voice today still goes to OpenAI, hardcoded — unchanged by #319.

[0xSero]

Still the plan for realtime voice's next phase; the blocker is upstream: Codex app-server does not expose the realtime/session surface Local Studio would need to be a drop-in peer provider, and the audit gates all voice work on device acceptance. Sequencing note: the shared voice layer (transcript/handoff normalization in store/voice.rs) is provider-agnostic, so when the upstream surface lands the work is mostly an alleycat/local-studio bridge plus UI. Keeping open as the tracked epic.

[0xSero]

Status: the foundation for this landed with the realtime rework — the peer connection now runs natively on each platform (libwebrtc) with the Rust layer owning signaling, session lifecycle, transcript state, and handoff normalization, which is exactly the substrate this feature needs. Local Studio already appears as a first-class server type in the connect flow.

What remains for this feature: Local Studio exposing (or proxying) a realtime session endpoint compatible with the same signaling contract the OpenAI realtime path uses, plus a provider selection in the voice session start path. That's server-side work in Local Studio/Alleycat first, then a small mobile-side provider selection. Keeping this open as the tracking issue; no timeline yet.