iOS glasses WebRTC client + Mac-mic agent input

[J4Joshua]

Adds the iOS glasses app and wires audio into the CourtLine agent over WebRTC.

Architecture

Human speaks → MacBook mic → agent (bot-courtline.py, MAC_MIC_INPUT=1)
   → NVidia STT → vLLM → Gradium TTS → SmallWebRTCTransport.output()
   → WebRTC → iOS app (receive-only) → Bluetooth HFP → 🕶️ glasses speaker

Why this shape: WebRTC on iOS can't reliably capture the Bluetooth-HFP glasses mic (it routes to HFP but no samples reach the server), and iOS WebRTC has no supported hook to inject externally-captured PCM into an audio track. So the agent listens via the Mac mic and speaks through the glasses — the output leg is verified working end-to-end.

Changes

• ios/ — Meta Ray-Ban Display glasses app (DAT SDK). WebRTCAudioSpike.swift is a Pipecat SmallWebRTC client that connects receive-only (enableMic = false) to the agent's /api/offer and plays the agent's voice through the glasses' HFP speaker. See ios/README.md.
• bot-courtline.py — optional Mac-microphone input behind MAC_MIC_INPUT=1: captures the local mic (sounddevice) and feeds InputAudioRawFrames into the pipeline. No-op when unset, so the existing browser/WebRTC audio-in path is unaffected.
• pyproject.toml / uv.lock — add sounddevice.

Verified

• ✅ iOS app compiles against the real SDKs (PipecatClientIOS / SmallWebRTC / WebRTC 134) — xcodebuild -sdk iphonesimulator → BUILD SUCCEEDED.
• ✅ Agent audio → glasses speaker proven on-device (downlink over WebRTC, peer-to-peer on a LAN).
• ✅ Mac-mic capture → Pipecat frames verified standalone; bot-courtline.py syntax-checks.

Pending (needs the agent's API keys / on-stack run — not CI-able)

• ⏳ Full Mac-mic → STT → LLM → TTS → glasses loop (Gradium / NVIDIA / OpenAI keys).
• ⏳ Glasses video → vision (check_camera) — the legacy WebSocket video path is left as a placeholder.

Setup

Server: cd yc-voice-agents-hackathon/server && uv sync && MAC_MIC_INPUT=1 uv run main.py
iOS: set webRTCServerURL (in WebRTCAudioSpike.swift) to the Mac's LAN IP:7860, build on a device on the same Wi-Fi.

Network: peer-to-peer WebRTC needs a routable media path (same LAN, or a TURN server on isolated networks). A Cloudflare HTTP tunnel won't carry the UDP media.

🤖 Generated with Claude Code