diff --git a/.claude/skills/project-guardrails/SKILL.md b/.claude/skills/project-guardrails/SKILL.md index d1e8782a..aecb1a23 100644 --- a/.claude/skills/project-guardrails/SKILL.md +++ b/.claude/skills/project-guardrails/SKILL.md @@ -97,7 +97,7 @@ genuinely correct, and reaching for it means it's time to stop and ask. generated from `project.yml`; edit that and run `xcodegen generate`. check.sh fails on pbxproj drift (a PreToolUse hook also blocks edits to it). - The engine has **no external SPM dependencies** (Foundation/Security/ - AVFoundation only). Don't add one to `Sources/BlurtEngine/`. + AVFoundation/CoreAudio only). Don't add one to `Sources/BlurtEngine/`. - Unit tests use **Swift Testing**, not XCTest (the `BlurtUITests` XCUITest bundle is the one exception — XCUIAutomation requires XCTest). **Never touch the real Keychain in tests** — `APIKeyStore` is the production item; use an diff --git a/AGENTS.md b/AGENTS.md index 768409f7..f42505a2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -32,10 +32,10 @@ Four reflexes before you touch anything: ### The two layers -| Layer | What it is | -| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `Sources/BlurtEngine/` | Swift package (`swift-tools-version:6.2`, `platforms: [.macOS(.v15)]`) owning the pipeline. Pure logic behind protocol seams, no AppKit-shell deps, **no external SPM deps** — Foundation/Security/AVFoundation plus toolchain modules like Synchronization, with AppKit types only at the seams. | -| `App/Blurt/` | AppKit/SwiftUI shell (Xcode project generated by XcodeGen) that wires the engine to an overlay window, the main window, a Settings scene, a menu bar item, and the trigger key. Its only package is the local `BlurtEngine` (declared in `App/Blurt/project.yml`). | +| Layer | What it is | +| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `Sources/BlurtEngine/` | Swift package (`swift-tools-version:6.2`, `platforms: [.macOS(.v15)]`) owning the pipeline. Pure logic behind protocol seams, no AppKit-shell deps, **no external SPM deps** — Foundation/Security/AVFoundation/CoreAudio plus toolchain modules like Synchronization, with AppKit types only at the seams. | +| `App/Blurt/` | AppKit/SwiftUI shell (Xcode project generated by XcodeGen) that wires the engine to an overlay window, the main window, a Settings scene, a menu bar item, and the trigger key. Its only package is the local `BlurtEngine` (declared in `App/Blurt/project.yml`). | The engine's dependency-free rule is a **rule**; the app merely happens to carry none today. The former `mxcl/AppUpdater` dependency and its in-place self-updater were removed (see [Updates](#updates)), @@ -51,7 +51,8 @@ workflow and the _why_ behind the design; the engine's README covers the _what_ ```text Sources/BlurtEngine/ the engine (dependency-free Swift package) README.md the engine's developer guide (quick start, seams, error table) - Audio/ MicCapture (+meter), SoundPack/Catalog/Store — record cues + Audio/ MicCapture (+meter/+warm), MicLiveness (mic bring-up gate), AudioRoute + (+Monitor)/AudioTransport — CoreAudio routing, SoundPack/Catalog/Store Config/ Keychain-backed API key, key terms, developer mode, DefaultsKey + PersistedSettings (every defaults key, and the reset sweep over them) FocusCapture/ Accessibility reads of the frontmost app / focused field @@ -426,13 +427,80 @@ A **fresh recorder per session** resolves the current default input device at `r is deliberate — see [Settled decisions](#settled-decisions--dont-reintroduce-these) for the `AVAudioEngine` failure it replaced. +**Bluetooth inputs are the reason for four of this actor's moving parts.** Opening the mic on +AirPods (or any Bluetooth headset) makes the system renegotiate the link into its mic-capable mode — +one to two seconds, during which the OS receives no audio at all — and that link then buffers audio +in both directions. So: + +- **`start()` does not return until the input is live.** `record()` returning `true` only means the + AudioQueue started, not that frames are arriving, so `start()` polls `recorder.currentTime` until + it advances past 0 — the recorder's clock only moves once the device delivers audio, which is what + distinguishes "route still switching" from "user is silent" (a level meter can't). Capped per + transport by **`MicLiveness`** (2.5 s Bluetooth, 300 ms otherwise) and **failing open** on timeout, + so a broken or silent mic degrades to the old behavior rather than bricking the press. This is what + stops the app cueing the user to speak into a dead mic; audio spoken during the switch cannot be + recovered by anything, because nothing ever receives it. `stopGeneration` covers the one suspension + this introduces — a teardown landing mid-wait wins, and the recorder is torn down rather than + installed. `bringingUpCapture` covers the other consequence: across the wait both `activeRecorder` + and `warm` are nil, so the warm-up paths can't infer "no capture in flight" from them (see + `canPrepareWarmRecorder`) or they'd open a second recorder onto the live input. + + **A cancel preempts the bring-up rather than queueing behind it**, which took two pieces. The + press publishes its task handle (`inFlightPress`) exactly as the pipeline publishes + `pipelineTask`, and `cancel()` treats `.connecting` like the other in-flight phases: cancel the + handle, claim `.cancelled`, return. `waitUntilLive` already honors task cancellation, so the wait + unblocks at once and `start()` throws `CancellationError` rather than installing the recorder. + The cancel flag then lives in a `Mutex` beside that handle rather than in actor state, so + **`submit(.cancel)` can record and preempt without waiting for a turn** — the app's cancel door is + `submit`, and its consumer is serial, so before this a submitted `.cancel` wasn't even _recorded_ + until the press it meant to cancel had finished. `requestCancel()` is the single place both doors + funnel through. Commands are still yielded and executed in order; only the preemption is new. + `performPress` still consumes the flag before claiming `.recording`, for the narrow window where + the cancel lands after the wait returned and there is nothing left to interrupt. + +- **The warm recorder is re-armed after every capture**, not just at launch. The cost above is paid + at `prepareToRecord()`, i.e. per session, so warming only the first one hid it for one dictation + out of N. `stop()`/`cancelCapture()` schedule a re-warm; `start()` consumes it. +- **A warm recorder is validated before reuse.** `AVAudioRecorder` resolves its device once and + never re-resolves, so `MicCapture` records the default input's identity (`AudioRoute.currentInput()`) + alongside the warm recorder and discards it when the device has changed — otherwise a recorder + warmed before the user connected their AirPods would keep recording the built-in mic. Unknown + counts as changed. +- **The warm recorder expires** (`preparedRecorderLifetime`, 60 s). A prepared recorder holds the + input device open, which is exactly what pins AirPods in the profile where _output_ audio is + degraded — so it is not held indefinitely. Back-to-back dictations land inside the window; a press + past it just prepares lazily, which is the pre-re-warm behavior. + +The re-warm and the liveness gate are complements, not alternatives: the re-warm shortens how _often_ +the profile switch is paid (a warm recorder has already held the route open), and the gate is what +keeps the app honest on the presses that pay it anyway. + +`stop()` also waits out `AudioTransport.tailLinger(forTransportType:)` (220 ms on Bluetooth, `.zero` +otherwise — the policy sits beside `MicLiveness`'s wait cap so both are unit-tested) before ending +the recording **when the +session's input is Bluetooth**, so speech still travelling over the link lands in the file instead of +being truncated — the missing last word. It runs after `.transcribing` is claimed, so it delays the +transcript, never the "it heard me" cue. Cancels take `cancelCapture()` instead, which skips both the +linger and the file read-back: the audio is being discarded, so neither is worth delaying the user's +cancel for. + +The routing facts behind all of that live in **`AudioRoute`** (`Audio/AudioRoute.swift`, internal): +which device is the default input, and its raw CoreAudio transport type. **Raw reads only** — what a +transport _means_ is **`AudioTransport.isBluetooth`** and **`MicLiveness.timeout`**, which are pure +and unit-tested, because `AudioRoute` itself needs real hardware and is excluded from the coverage +gate. Don't let a decision drift into it. Its sibling +**`AudioRouteMonitor`** (public) publishes output-route changes for the cue players — see +[Settings, persistence, and cues](#settings-persistence-and-cues). Both are excluded from the +coverage gate for the same reason +`MicCapture` is: they answer questions only real hardware can answer. + The overlay meter (`levels`) comes from the recorder's dBFS power on a ~20 Hz timer (`MicCapture.meterIntervalSeconds` — public because the pill caps its animation redraws to the same cadence and reads it from here rather than restating it), mapped to `0…1` by `linearLevel(fromPowerDB:)` and floored so room ambient reads as empty bars rather than a meter that -never rests. `levels` and `warmUp()` are part of `MicCaptureProtocol` itself with an empty-stream / -no-op default, so stubs conform with just `start()`/`stop()` while hosts still read the meter through -the seam they inject. +never rests. `levels`, `warmUp()` and `cancelCapture()` are part of `MicCaptureProtocol` itself with +empty-stream / no-op / stop-and-discard defaults, so stubs conform with just `start()`/`stop()` while +hosts still read the meter through the seam they inject. ### `AssemblyAITranscriber` — `Sources/BlurtEngine/STT/AssemblyAITranscriber.swift` @@ -502,8 +570,9 @@ stream and signposts) and `+Pipeline.swift` (the post-release transcribe→injec It exposes `press()` / `release()` / `cancel()` / `cancelRecording()`, a synchronous fire-and-forget `submit(_: Command)` mirroring those four for callback-shaped hosts (commands run in exact emit order — the tap wires straight into it, no per-callback `Task` spawning), and a -`phase: PipelinePhase` (`idle | recording | transcribing | injecting | failed | cancelled`, plus the -terminal successes `pasted` / `noTarget`). `phaseStream()` yields the current phase immediately then +`phase: PipelinePhase` (`idle | starting | recording | transcribing | injecting | failed | +cancelled`, plus the terminal successes `pasted` / `noTarget`). `phaseStream()` yields the current +phase immediately then every transition, and is **multi-observer** (one continuation per call), though hosts should still render from one consumer and project the phase into their own state. @@ -521,14 +590,23 @@ round trip, and the log wrote to the user's real `~/Library/Logs/Blurt`. STT err are wrapped in `.sttFailed`. The pipeline is just transcribe → inject, and an empty transcript returns to `.idle` without injecting. -Three perceived-latency choices to preserve: +Four perceived-latency choices to preserve: +- `press()` claims `.connecting` _before_ `mic.start()`, so the pill answers the key-down while the + mic is still coming up — but the **start chime deliberately does not**. `RecordingCueGate` keys on + `.recording`, which `MicCapture`'s liveness gate only reaches once the input route is delivering + frames, so the chime rides the connecting→recording edge by construction. That ordering is + load-bearing, not cosmetic: the chime is a "speak now" cue, on a Bluetooth route it and the press + are ~1–2 s apart, and speech in that window is **unrecoverable** (the OS receives nothing while the + profile switch is in flight). Chiming at the press is what lost the first words of the utterance. + Don't "fix" the chime to fire earlier. - `.injecting` projects to `OverlayUIState.processing`, **not** `.idle` — the shell reads an idle projection as "dismiss", so mapping this working phase to idle faded the pill out mid-dictation and blinked it back for "Pasted". - `release()` claims `.transcribing` _before_ `mic.stop()`, so the stop chime and pill switch fire at key-up rather than after the recording is read back. This ordering also closes the double-release - window `ReleaseRaceTests` pins. + window `ReleaseRaceTests` pins — and it is what lets `MicCapture`'s Bluetooth tail linger sit + inside `stop()` without the user ever waiting on it. - The press-time Accessibility context read is consumed with a bounded wait (`contextWaitBudget`, 500 ms), so an unresponsive frontmost app costs the transcript its priming, never a multi-second stall. @@ -731,8 +809,16 @@ Record cues: **`SoundPack`** is a selectable start/stop chime voice (vintage syn `App/Blurt/Blurt/Resources/Sounds/`), listed by **`SoundPackCatalog.swift`**, which is _generated_ by `scripts/generate-sounds.swift` alongside the audio. Regenerate both halves together — `check.sh`'s sound-catalog guard exists because a drift plays silence with no error. **`RecordingCueGate`** is the -pure edge detector deciding when the chimes fire; the AppKit `CueSoundPlayer` just plays what it -resolves. +pure edge detector deciding when the chimes fire — on the `.recording` edge, i.e. once audio is +actually flowing, never at the press (see the latency notes above for why that ordering is +load-bearing); the AppKit `CueSoundPlayer` just plays what it resolves. + +`CueSoundPlayer` decodes and pre-rolls the players once so the first chime never stalls the pill, and +that pre-roll is bound to the output route it was made against. Blurt's own capture invalidates it: +opening the mic flips AirPods out of their output-only profile, dropping the output format underneath +the primed players. So the player observes **`AudioRouteMonitor.outputRouteChanges`** and reloads — +the monitor watches both the default output _device_ (a user switch) and the current device's nominal +sample rate (the profile flip), re-targeting the second listener whenever the first fires. History: **`RecentDictations`** is an in-memory, newest-first ring (never written to disk) holding `capacity` (100) dictations, of which the ready window lists `displayCapacity` (3). It is deep because diff --git a/App/Blurt/Blurt/CueSoundPlayer.swift b/App/Blurt/Blurt/CueSoundPlayer.swift index 333c735c..59deb02d 100644 --- a/App/Blurt/Blurt/CueSoundPlayer.swift +++ b/App/Blurt/Blurt/CueSoundPlayer.swift @@ -11,10 +11,28 @@ final class CueSoundPlayer { /// The pack the current `startSound`/`stopSound` were decoded from, so `prime()` /// can skip re-decoding when nothing changed. `nil` until the first load. private var loadedPack: SoundPack? + /// Monotonic ticket for in-flight decodes: only the newest one installs its + /// players. This is the staleness check, **not** `loadedPack` equality — which + /// cannot discriminate two loads of the *same* pack, and a route change forces + /// exactly that. Without it, a decode already in flight when the route changed + /// could land after the re-prime and install players primed against the old + /// output route — the stall the re-prime exists to prevent. + private var loadGeneration = 0 /// Edge-detector deciding when the start/stop chimes fire. The mapping from a /// pipeline phase to a cue lives in the engine (`RecordingCueGate`), where /// `swift test` covers it; this player just plays whatever it resolves to. private var cueGate = RecordingCueGate() + /// Fires when the output route changes under the pre-rolled players. Blurt's + /// own capture is the usual cause: opening the mic flips AirPods out of their + /// output-only profile, which drops the output format the players were primed + /// against — so the very next chime is the one that stalls, and that's the + /// chime at the start of a dictation. + /// Set when the output route changes, cleared when the re-prime actually runs. + /// See `transition(for:)` for why the reload waits. + private var needsReprime = false + /// Kept alive for the app's lifetime; assignment is the use. Nil until + /// `prime()` starts it, which is also what makes starting idempotent. + private var routeObserver: Task? /// The cues are deliberate UI accents, not music — they are normalized to a /// hot peak, so play them well below full scale so they read as a soft chime @@ -36,6 +54,54 @@ final class CueSoundPlayer { /// decode never sits on the main thread during startup. func prime() { Task { await loadCurrentPack() } + startObservingRoute() + } + + /// The player is owned for the whole app session, so this never runs in + /// practice — but cancelling the observer mirrors the `[weak self]` care below + /// and documents that its lifetime is owned rather than leaked. + deinit { + routeObserver?.cancel() + } + + /// Re-primes the players whenever the output route changes, so the pre-roll + /// `prime()` bought at launch survives a device switch or a profile flip. + /// Idempotent — `prime()` runs on every "app is ready" transition, and only + /// the first call installs the observer. + /// + /// A full reload rather than a bare `prepareToPlay()`: re-creating the players + /// is the one thing guaranteed to leave them primed against the *current* + /// route, and the decode runs off the main actor like every other load. Route + /// changes are not rare — on a Bluetooth output Blurt's own capture causes one + /// per dictation burst — which is exactly why the reload is deferred rather + /// than run on the tick; see `transition(for:)`. + private func startObservingRoute() { + guard routeObserver == nil else { return } + routeObserver = Task { [weak self] in + // The monitor is owned by this task, not stored: the local keeps it alive + // for as long as the loop runs, and dropping it when the task ends is what + // deregisters its CoreAudio listeners. + let monitor = await Self.makeRouteMonitor() + for await _ in monitor.outputRouteChanges { + // Rebound per tick rather than bound once above, so the observer never + // keeps the player alive — the same reason `MicCapture`'s meter task + // rebinds across its sleep. + guard let self else { return } + // Recorded, not acted on — see `transition(for:)`. + self.needsReprime = true + } + } + } + + /// Constructs the monitor off the main actor. `nonisolated` + `async` for the + /// same reason `decode` is: this type defaults to `MainActor`, so a plain + /// initializer call would run on the main thread — and constructing it + /// registers two CoreAudio listeners and makes the process's *first* HAL call, + /// paying the client-side HAL init and the coreaudiod connection. As a stored + /// property that landed during launch, before the first frame, on every run + /// including onboarding and UI tests where no chime is ever played. + private nonisolated static func makeRouteMonitor() async -> AudioRouteMonitor { + AudioRouteMonitor() } /// Reloads the players for a newly selected pack and previews the new voice @@ -54,17 +120,22 @@ final class CueSoundPlayer { /// Decodes and installs the cue players for the current selection if they aren't /// already loaded. The `loadedPack` guard makes repeat calls cheap; the decode /// itself hops off the main actor. Returns once the players are assigned. - private func loadCurrentPack() async { + /// `force` skips the already-loaded short-circuit, for a re-prime where the + /// selection is unchanged and only the output route moved. + private func loadCurrentPack(force: Bool = false) async { let pack = SoundPackStore().soundPack - guard pack != loadedPack else { return } + guard force || pack != loadedPack else { return } + loadGeneration += 1 + let generation = loadGeneration loadedPack = pack let players = await Self.decode(pack) - // Re-check after the off-actor decode: if a newer selection was claimed while - // we were decoding (rapid pack switches), its decode owns the players now — - // dropping this stale result avoids installing players that disagree with - // `loadedPack`. `loadedPack` is written synchronously above (no await between - // read and write), so only the newest-requested load passes this guard. - guard loadedPack == pack else { return } + // Re-check after the off-actor decode: whoever asked last owns the players. + // Both are written synchronously above (no await between read and write), so + // exactly one in-flight decode passes this guard — the newest. Ticketed + // rather than compared against `loadedPack`, so two loads of the same pack + // (a rapid pack switch back, or any forced route re-prime) can still tell + // each other apart. + guard generation == loadGeneration else { return } startSound = players.start stopSound = players.stop } @@ -78,6 +149,17 @@ final class CueSoundPlayer { case .stop: play(stopSound) case nil: break } + // A pending route re-prime waits for the dictation to finish. The usual + // cause of a route change is Blurt *opening the mic* — so the tick arrives + // during the press, and reloading there would put two AAC decodes and two + // `prepareToPlay()`s alongside the bring-up, then swap `startSound` out from + // under the very chime it is meant to protect (releasing an `AVAudioPlayer` + // that may be mid-play). Deferring to the next terminal phase puts the + // reload between dictations, where it costs nothing anyone is waiting on, + // and coalesces a burst of flips into one decode. + guard phase.isTerminal, needsReprime else { return } + needsReprime = false + Task { await loadCurrentPack(force: true) } } /// The decoded, pre-rolled players for a pack. Non-`Sendable` (holds diff --git a/App/Blurt/Blurt/Overlay/OverlayPillContent.swift b/App/Blurt/Blurt/Overlay/OverlayPillContent.swift index 57015493..9c848c76 100644 --- a/App/Blurt/Blurt/Overlay/OverlayPillContent.swift +++ b/App/Blurt/Blurt/Overlay/OverlayPillContent.swift @@ -56,13 +56,21 @@ extension View { } } -/// The "Transcribing…" status line with a slow breathing pulse — the processing -/// counterpart of the recording bars' idle shimmer, so the pill keeps visibly -/// working while the app waits on the dictation API and pastes the result. Driven by -/// the same continuous-clock `TimelineView` pattern as `WaveformBars` (never a -/// one-shot state toggle). Under Reduce Motion the label holds steady at full -/// opacity — exactly the pre-animation rendering. -struct TranscribingLabel: View { +/// A status line that breathes — the pill's "working, hold on" treatment, used +/// for both waits it has: "Connecting…" while `MicCapture`'s liveness gate waits +/// for the input route to deliver frames, and "Transcribing…" while the app +/// waits on the dictation API and pastes the result. +/// +/// One view rather than two near-identical ones, so the heartbeat is shared by +/// construction instead of by a comment asking two copies of the constants to +/// stay equal. Driven by the same continuous-clock `TimelineView` pattern as +/// `WaveformBars` (never a one-shot state toggle); under Reduce Motion it holds +/// steady at full opacity, exactly the pre-animation rendering. +/// +/// `StatusLineText` is shared with the "Pasted"/"Copied" notices, so every +/// hand-off between these states reads as one continuous status line. +struct BreathingStatusLine: View { + let text: String /// Whether to run the breathing motion (off under Reduce Motion). let animated: Bool @@ -75,9 +83,7 @@ struct TranscribingLabel: View { private let minOpacity: Double = 0.55 var body: some View { - // StatusLineText is shared with the "Pasted" notice (OverlayView's `.pasted` - // case) so the processing → pasted hand-off reads as one status line. - StatusLineText("Transcribing…") + StatusLineText(text) .pulsingOpacity(period: breathPeriod, minOpacity: minOpacity, animated: animated) } } @@ -94,7 +100,7 @@ struct RecordingTag: View { // One pulse every ~1.2 s, dimming to 40% and back: the universal "recording, // right now" heartbeat. Since magenta stands in for the conventional red dot, // the pulse — not the hue — carries the live-capture cue. Driven by the same - // continuous-clock TimelineView as the waveform and TranscribingLabel (never a + // continuous-clock TimelineView as the waveform and BreathingStatusLine (never a // one-shot repeatForever toggle). private let pulsePeriod: Double = 1.2 private let minOpacity: Double = 0.4 diff --git a/App/Blurt/Blurt/Overlay/OverlayView.swift b/App/Blurt/Blurt/Overlay/OverlayView.swift index 559a4fba..ec5cf609 100644 --- a/App/Blurt/Blurt/Overlay/OverlayView.swift +++ b/App/Blurt/Blurt/Overlay/OverlayView.swift @@ -32,7 +32,7 @@ struct OverlayView: View { switch state { case .error: return Color(red: 0.62, green: 0.13, blue: 0.13) - case .recording, .processing, .pasted, .noTarget, .idle: + case .connecting, .recording, .processing, .pasted, .noTarget, .idle: return Color(white: 0.16) } } @@ -84,6 +84,20 @@ struct OverlayView: View { // background would collapse with it) keeps the pill's shape intact for // `hide()`'s pre-hide reset. Color.clear + case .connecting: + // The mic is coming up; nothing is being captured yet. Styled like + // "Transcribing…" (same status-line type, tracking, cyan --ice, and + // breathing pulse) so the connecting → recording hand-off reads as one + // status line rather than a new kind of alert — and deliberately *not* + // the `● REC` tag or the meter, which would cue the user to speak into a + // mic that isn't delivering yet. + // + // On a built-in mic this is on screen for a frame or two, inside the + // pill's own 0.08 s fade-in, so it blends into the appearance rather than + // flashing; on a Bluetooth route it holds for as long as the link takes, + // which is the whole point. + BreathingStatusLine(text: "Connecting…", animated: !reduceMotion) + .transition(.opacity) case .recording: // "● REC" tag beside the live waveform, mirroring the site demo's recording // pill (magenta tag + bars). The bars fill the width left of the tag. @@ -98,7 +112,7 @@ struct OverlayView: View { // Transcribing); cyan echoes the demo's --ice. Cross-fades like the bars. // The label breathes (slow opacity pulse) so the wait for the dictation API + // paste reads as active work rather than a frozen pill. - TranscribingLabel(animated: !reduceMotion) + BreathingStatusLine(text: "Transcribing…", animated: !reduceMotion) .transition(.opacity) case .error(let message): // "Try again" tells the user what to do; the full failure reason is too diff --git a/App/Blurt/Blurt/UITestSupport.swift b/App/Blurt/Blurt/UITestSupport.swift index 57d407d2..e0de29f7 100644 --- a/App/Blurt/Blurt/UITestSupport.swift +++ b/App/Blurt/Blurt/UITestSupport.swift @@ -159,8 +159,8 @@ extension DictationComponents { /// The all-stub pipeline used under UI testing: no mic, no network, no /// Accessibility paste. `UITestMic` inherits `MicCaptureProtocol`'s default - /// empty `levels` stream (the overlay meter isn't asserted) and no-op - /// `warmUp()`. + /// empty `levels` stream (the overlay meter isn't asserted), no-op + /// `warmUp()`, and stop-and-discard `cancelCapture()`. static func uiTest() -> DictationComponents { DictationComponents( mic: UITestMic(), diff --git a/Sources/BlurtEngine/Audio/AudioRoute.swift b/Sources/BlurtEngine/Audio/AudioRoute.swift new file mode 100644 index 00000000..bf7e8e98 --- /dev/null +++ b/Sources/BlurtEngine/Audio/AudioRoute.swift @@ -0,0 +1,121 @@ +import CoreAudio + +/// Read-only queries against the system's current audio routing — the two facts +/// about the mic that `AVFoundation` doesn't expose but the capture path needs: +/// +/// 1. **Which device is the default input**, so `MicCapture` can tell whether a +/// recorder it prepared earlier is still bound to the device the user is +/// about to speak into. `AVAudioRecorder` resolves the route at +/// `prepareToRecord()` time and never re-resolves it, so a recorder warmed +/// before the user connected their AirPods would silently record from the +/// built-in mic. +/// 2. **What transport that device is on**, since a Bluetooth link is both slow +/// to bring up (the wait `MicLiveness` caps) and buffered at the tail (the +/// linger `MicCapture.stop()` grants rather than truncating). +/// +/// Raw reads only — no policy. What a transport type *means* lives in +/// `AudioTransport` and `MicLiveness`, which are pure and unit-tested; this file +/// needs real hardware to answer anything, so it is excluded from the coverage +/// gate and must not be where a decision hides. +/// +/// Internal, not public: the app never asks these directly (it observes route +/// *changes* through `AudioRouteMonitor`), and `.periphery.yml` runs with +/// `retain_public: false`, so a `public` symbol only the engine reaches fails +/// the unused-code scan. +enum AudioRoute { + /// Identity plus link character of the default input device, read together in + /// one pass so the capture path makes a single trip through CoreAudio per + /// session rather than one per question. + struct InputSnapshot: Equatable, Sendable { + /// Which device this is. + /// + /// The `AudioDeviceID` rather than the device's persistent UID string, even + /// though IDs are in principle reusable across an unplug/replug while UIDs + /// are not. The only consumer is "is the warm recorder still bound to the + /// device about to be recorded from", and the two disagree in exactly one + /// case: the warmed device was removed and a new one took its ID inside the + /// 60 s warm window. That case fails *loudly* — the recorder is bound to a + /// device that no longer exists, so `record()` returns false and the press + /// surfaces `.audioCaptureFailed`. It cannot produce the failure the check + /// exists to prevent, which is silently recording the wrong mic. Reading the + /// UID instead would mean bridging a `CFString`, i.e. pulling Foundation + /// into a file that otherwise needs only CoreAudio, to buy a distinction + /// that changes a loud failure into a slightly louder one. + let deviceID: AudioDeviceID + /// The device's CoreAudio transport type, or nil when the read failed. + /// Interpreted by `AudioTransport.isBluetooth` and + /// `MicLiveness.timeout(forTransportType:)` — kept raw here so the policy + /// stays in the files `swift test` can reach. + let transportType: UInt32? + } + + /// The default input device as an `InputSnapshot`, or nil when there is no + /// input device (all of them unplugged or asleep) or CoreAudio refused the + /// read. Nil is the conservative answer everywhere it's consumed: an unknown + /// input invalidates a warm recorder rather than silently keeping one bound to + /// a device that may have gone away. + static func currentInput() -> InputSnapshot? { + guard let deviceID = defaultDeviceID(for: kAudioHardwarePropertyDefaultInputDevice) else { + return nil + } + return InputSnapshot(deviceID: deviceID, transportType: transportType(of: deviceID)) + } + + /// The system's current default *output* device — what `AudioRouteMonitor` + /// hangs its format listener on. Nil when there is none, or the read failed. + static func defaultOutputDeviceID() -> AudioDeviceID? { + defaultDeviceID(for: kAudioHardwarePropertyDefaultOutputDevice) + } + + // MARK: - CoreAudio addressing + + /// The system-wide audio object, which owns the default-device properties. + static var systemObject: AudioObjectID { AudioObjectID(kAudioObjectSystemObject) } + + /// A global-scope address for `selector`. Returned fresh per call rather than + /// stored, because every caller needs its own copy to pass `inout` to + /// CoreAudio. Shared with `AudioRouteMonitor`, which addresses the same object + /// graph and would otherwise restate this three-field literal. + static func globalAddress(_ selector: AudioObjectPropertySelector) -> AudioObjectPropertyAddress { + AudioObjectPropertyAddress( + mSelector: selector, + mScope: kAudioObjectPropertyScopeGlobal, + mElement: kAudioObjectPropertyElementMain) + } + + // MARK: - CoreAudio reads + + // Spelled out per property rather than shared behind a generic + // `read(_:from:initial:)`. That reads better but doesn't compile: `&value` + // on an unconstrained `T` is "forming 'UnsafeMutableRawPointer' to a variable + // of type 'T'; this is likely incorrect because 'T' may contain an object + // reference". Making it work means constraining to `BitwiseCopyable` and going + // through `withUnsafeMutableBytes` — more machinery than two five-line reads + // are worth, in a file the coverage gate can't check anyway. + + /// The device the system object reports for `selector` (a default-device + /// property). Nil covers both a failed read and the "no such device" sentinel, + /// which callers treat identically. + private static func defaultDeviceID(for selector: AudioObjectPropertySelector) -> AudioDeviceID? { + var address = globalAddress(selector) + var deviceID = AudioDeviceID(0) + var size = UInt32(MemoryLayout.size) + let status = AudioObjectGetPropertyData(systemObject, &address, 0, nil, &size, &deviceID) + // 0 is `kAudioObjectUnknown` — "there is no such device" — spelled as the + // literal so this doesn't depend on how the constant imports. + guard status == noErr, deviceID != 0 else { return nil } + return deviceID + } + + /// The device's transport type, or nil when the read failed — + /// `AudioTransport` and `MicLiveness` both treat nil as "not Bluetooth", which + /// is the conservative direction for each. + private static func transportType(of deviceID: AudioDeviceID) -> UInt32? { + var address = globalAddress(kAudioDevicePropertyTransportType) + var transport = UInt32(0) + var size = UInt32(MemoryLayout.size) + let status = AudioObjectGetPropertyData(deviceID, &address, 0, nil, &size, &transport) + guard status == noErr else { return nil } + return transport + } +} diff --git a/Sources/BlurtEngine/Audio/AudioRouteMonitor.swift b/Sources/BlurtEngine/Audio/AudioRouteMonitor.swift new file mode 100644 index 00000000..4d17d1d9 --- /dev/null +++ b/Sources/BlurtEngine/Audio/AudioRouteMonitor.swift @@ -0,0 +1,152 @@ +import CoreAudio +import Dispatch +import os + +/// Ticks whenever the system's audio **output** route changes in a way that +/// invalidates an already-pre-rolled `AVAudioPlayer`. +/// +/// This exists for the record cue chimes. `CueSoundPlayer` decodes and +/// `prepareToPlay()`s them once at launch so the first chime never stalls the +/// pill — but that pre-roll is bound to the output route it was made against, +/// and Blurt's own capture is what invalidates it: opening the mic flips AirPods +/// out of their output-only profile into the bidirectional one, which drops the +/// output format underneath the primed players. The first chime after such a +/// flip is exactly the one that stalls, which is the chime at the start of a +/// dictation. +/// +/// Two properties are watched, because "the route changed" has two shapes: +/// +/// - `kAudioHardwarePropertyDefaultOutputDevice` on the system object — the user +/// switched output devices (built-in speakers → AirPods). +/// - `kAudioDevicePropertyNominalSampleRate` on whichever device is *currently* +/// default — the same device renegotiated its format, which is what the +/// profile flip looks like from CoreAudio. That listener is re-targeted +/// whenever the first one fires, so it always tracks the live device. +/// +/// Public because the app owns the cue players; the engine's own use of +/// CoreAudio routing (`AudioRoute`) stays internal. +/// +/// `@unchecked Sendable` because the listener registrations below are confined +/// to `queue` rather than protected by a lock — see their declarations. +public final class AudioRouteMonitor: @unchecked Sendable { + private static let logger = Logger(subsystem: BlurtIdentity.subsystem, category: "AudioRoute") + + /// Fires once per observed route change. `.bufferingNewest(1)` because this is + /// an invalidation signal, not a log: a consumer that was busy through three + /// changes needs to re-prime once, not three times. + public let outputRouteChanges: AsyncStream + private let continuation: AsyncStream.Continuation + + /// The queue CoreAudio delivers every listener callback on, and the one place + /// the registrations below are touched. Serial, so a re-target triggered by a + /// default-device change can't interleave with itself. + private let queue: DispatchQueue + + /// The registered listener blocks, kept so they can be handed back to + /// CoreAudio — removal matches on block identity, so a re-created block would + /// deregister nothing. + /// + /// `nonisolated(unsafe)` rather than lock-guarded: every read and write happens + /// inside a `queue` block, including the initial registration (`init` wraps it + /// in `queue.sync` precisely so a listener can't fire before the property + /// recording it has been written). Dispatch's serial ordering supplies both the + /// exclusion and the memory barriers a lock would. + private nonisolated(unsafe) var systemListener: AudioObjectPropertyListenerBlock? + private nonisolated(unsafe) var deviceListener: (id: AudioDeviceID, block: AudioObjectPropertyListenerBlock)? + + public init() { + let (stream, continuation) = AsyncStream.makeStream(bufferingPolicy: .bufferingNewest(1)) + self.outputRouteChanges = stream + self.continuation = continuation + self.queue = DispatchQueue(label: "\(BlurtIdentity.subsystem).AudioRoute") + queue.sync { + installDefaultDeviceListener() + retargetFormatListener() + } + } + + /// The monitor is owned for the app's lifetime, so this never runs in + /// practice — but deregistering mirrors the `[weak self]` care below and + /// documents that the CoreAudio registrations are owned rather than leaked: a + /// listener left behind outlives the monitor, since CoreAudio retains the block + /// and nothing else would ever hand it back. + /// + /// Removal happens **inline, with no hop onto `queue`** — deliberately. + /// + /// A `queue.sync` here can self-deadlock. The listener blocks capture `self` + /// weakly, but `guard let self` upgrades that to a strong reference for the + /// body's duration, so while a block is running `queue` *is* an owner. If the + /// last other reference is dropped in that window, the block's release is the + /// final one and this `deinit` runs **on `queue`** — where `queue.sync` + /// deadlocks against itself. + /// + /// Inline removal is also race-free without the hop. `deinit` only runs once + /// the last reference is gone, so no block can be *inside* its `guard let self` + /// concurrently with this — a block that starts now fails the upgrade and + /// touches nothing. That leaves these reads of the queue-confined + /// registrations unopposed. (`queue` is still passed to CoreAudio, because + /// removal matches on the queue the listener was added with; that's an argument, + /// not an execution context.) + deinit { + continuation.finish() + if let systemListener { + var address = AudioRoute.globalAddress(kAudioHardwarePropertyDefaultOutputDevice) + _ = AudioObjectRemovePropertyListenerBlock( + AudioRoute.systemObject, &address, queue, systemListener) + } + if let deviceListener { + var address = AudioRoute.globalAddress(kAudioDevicePropertyNominalSampleRate) + _ = AudioObjectRemovePropertyListenerBlock( + deviceListener.id, &address, queue, deviceListener.block) + } + } + + // MARK: - Registration (queue-confined) + + /// Watches for the default output device itself changing. Registered once and + /// never re-targeted — the system object is always there. + private func installDefaultDeviceListener() { + var address = AudioRoute.globalAddress(kAudioHardwarePropertyDefaultOutputDevice) + // `[weak self]`, so CoreAudio's strong hold on the block doesn't keep the + // monitor alive forever — and so a callback landing during teardown finds + // nil rather than a half-destroyed object. + let block: AudioObjectPropertyListenerBlock = { [weak self] _, _ in + guard let self else { return } + // Re-target first, then publish: a consumer that re-primes on this tick + // should already be behind a listener pointed at the new device. + self.retargetFormatListener() + self.continuation.yield() + } + let status = AudioObjectAddPropertyListenerBlock(AudioRoute.systemObject, &address, queue, block) + guard status == noErr else { + Self.logger.error("default-output listener failed: \(status)") + return + } + systemListener = block + } + + /// Points the format listener at the current default output device, removing + /// the one on the previous device. A no-op when the device hasn't actually + /// changed, so a notification that resolves to the same device doesn't churn + /// the registration. + private func retargetFormatListener() { + let device = AudioRoute.defaultOutputDeviceID() + if let existing = deviceListener { + guard existing.id != device else { return } + var address = AudioRoute.globalAddress(kAudioDevicePropertyNominalSampleRate) + _ = AudioObjectRemovePropertyListenerBlock(existing.id, &address, queue, existing.block) + deviceListener = nil + } + guard let device else { return } + let block: AudioObjectPropertyListenerBlock = { [weak self] _, _ in + self?.continuation.yield() + } + var address = AudioRoute.globalAddress(kAudioDevicePropertyNominalSampleRate) + let status = AudioObjectAddPropertyListenerBlock(device, &address, queue, block) + guard status == noErr else { + Self.logger.error("output-format listener failed: \(status)") + return + } + deviceListener = (id: device, block: block) + } +} diff --git a/Sources/BlurtEngine/Audio/AudioTransport.swift b/Sources/BlurtEngine/Audio/AudioTransport.swift new file mode 100644 index 00000000..d1540cc7 --- /dev/null +++ b/Sources/BlurtEngine/Audio/AudioTransport.swift @@ -0,0 +1,52 @@ +import CoreAudio + +/// Classification of a CoreAudio device's transport type +/// (`kAudioDevicePropertyTransportType`). +/// +/// Split from `AudioRoute`, which reads the raw value off the hardware and is +/// excluded from the coverage gate for that reason. The *decision* — which +/// transports behave like a buffered wireless link — is pure, and two separate +/// behaviors hang off it, so it lives somewhere `swift test` can pin it: +/// +/// - `MicLiveness.timeout(forTransportType:)`, the wait cap for the mic +/// bring-up gate. +/// - `MicCapture`'s tail linger, which keeps capturing past key-up so the last +/// word doesn't get truncated by the link's buffering. +enum AudioTransport { + /// Whether the transport is a Bluetooth one. Both types count: AirPods and + /// other wireless headsets report the classic `bluetooth` transport, LE Audio + /// devices report `bluetoothLE`, and the link characteristic both callers care + /// about — a slow bring-up and a buffered tail — is the same either way. + /// + /// A nil transport (the read failed, or there is no device) answers `false`. + /// That is the conservative direction for both callers: the short wait cap, + /// and no linger. Padding every wired capture with a delay would be a worse + /// regression than losing the tail on a device we couldn't classify. + static func isBluetooth(_ transportType: UInt32?) -> Bool { + guard let transportType else { return false } + return transportType == kAudioDeviceTransportTypeBluetooth + || transportType == kAudioDeviceTransportTypeBluetoothLE + } + + /// How much longer capture runs past the key-up that ends it, for a device of + /// this transport. `.zero` for everything but Bluetooth, so the wired path + /// skips the wait entirely rather than testing a flag at the call site. + /// + /// A Bluetooth link buffers: audio the user has already spoken is still in + /// flight when `stop()` is called, and `recorder.stop()` drops it — which is + /// why the last word of a dictation goes missing on AirPods. The value is + /// deliberately shorter than a typical link's worst case: it buys back the + /// common tail without making every dictation feel sluggish. + /// + /// Lives here beside `MicLiveness.timeout(forTransportType:)` — the other + /// transport-conditional policy — rather than inside `MicCapture`, so both are + /// reachable by `swift test`. `MicCapture` needs real hardware and is excluded + /// from the coverage gate. + static func tailLinger(forTransportType transportType: UInt32?) -> Duration { + isBluetooth(transportType) ? bluetoothTailLinger : .zero + } + + /// See `tailLinger(forTransportType:)`. Exposed so a test can name it rather + /// than restate the number. + static let bluetoothTailLinger = Duration.milliseconds(220) +} diff --git a/Sources/BlurtEngine/Audio/MicCapture+Warm.swift b/Sources/BlurtEngine/Audio/MicCapture+Warm.swift new file mode 100644 index 00000000..b5cc270f --- /dev/null +++ b/Sources/BlurtEngine/Audio/MicCapture+Warm.swift @@ -0,0 +1,109 @@ +import AVFoundation +import Foundation + +// The warm-recorder lifecycle — prepare ahead of the press, validate it still +// matches the live input, and let it expire — split from `MicCapture.swift` to +// stay within the lint file-length budget, like `MicCapture+Meter`. Members it +// reaches (`warm`, `preparedGeneration`, `logger`, `removeFile`, `makeRecorder`) +// are internal rather than private for that reason: `private` is file-scoped and +// can't cross the split. +extension MicCapture { + /// Pre-create and prepare a recorder so the first `start()` skips first-time + /// hardware route discovery. Does NOT begin capture — no mic indicator. Safe to + /// call multiple times; a failure here just leaves `start()` to prepare lazily. + public func warmUp() { + guard canPrepareWarmRecorder else { return } + prepareWarmRecorder() + } + + /// Whether it is safe to open the input for a *warm* recorder right now: + /// nothing is capturing, nothing is mid-bring-up, and no warm recorder is + /// already held. A scheduled re-warm that fails this has been overtaken — + /// a press got there first — and has nothing to do. + /// + /// `bringingUpCapture` is the load-bearing term. Both `activeRecorder` and + /// `warm` are nil across `start()`'s liveness wait, so testing them alone reads + /// a live capture as "idle" and prepares a second recorder onto the open input. + var canPrepareWarmRecorder: Bool { + activeRecorder == nil && warm == nil && !bringingUpCapture + } + + /// The warm recorder if it is still bound to `input`, else nil — discarding + /// (and cleaning up after) one that isn't. + /// + /// Reuse requires *positively* confirming the device is unchanged: an + /// unreadable route on either side leaves us unable to tell, and a recorder + /// bound to the wrong device doesn't fail loudly — it records the wrong mic, or + /// silence. Paying route activation is the cheaper mistake, so unknown means + /// discard. + func takeWarmRecorder(matching input: AudioRoute.InputSnapshot?) -> AVAudioRecorder? { + guard let held = warm else { return nil } + held.expiry?.cancel() + warm = nil + guard let warmed = held.input, let input, warmed.deviceID == input.deviceID else { + Self.removeFile(at: held.recorder.url) + Self.logger.info("discarded warm recorder — input device changed since warm-up") + return nil + } + return held.recorder + } + + /// Queues a re-warm to run once the current actor turn finishes, so the caller + /// (`stop()` / `cancelCapture()`) returns before the input is re-opened. + /// + /// `warmUp()` rather than a separate re-warm entry point: the two differ only + /// in when they are called, and both answer the same question — is it safe to + /// open the input for a warm recorder right now — so they share the guard + /// rather than each keeping a copy of it. + func scheduleRewarm() { + Task { [weak self] in + await self?.warmUp() + } + } + + /// Builds a recorder, records the input it is bound to, and starts its idle + /// countdown. A failure is non-fatal: `start()` then prepares lazily, exactly + /// as it did before any warm recorder existed. + func prepareWarmRecorder() { + do { + preparedGeneration += 1 + warm = WarmRecorder( + recorder: try Self.makeRecorder(), + input: AudioRoute.currentInput(), + generation: preparedGeneration) + armPreparedRecorderExpiry(generation: preparedGeneration) + Self.logger.info("prepared a warm recorder") + } catch { + Self.logger.error("warm-up failed: \(error.localizedDescription, privacy: .public)") + } + } + + /// Arms the idle countdown for the warm recorder identified by `generation`. + /// The ticket is what makes a stale expiry harmless — see + /// `releasePreparedRecorder(generation:)`. + func armPreparedRecorderExpiry(generation: Int) { + warm?.expiry?.cancel() + warm?.expiry = Task { [weak self] in + try? await Task.sleep(for: Self.preparedRecorderLifetime) + guard !Task.isCancelled else { return } + await self?.releasePreparedRecorder(generation: generation) + } + } + + /// Tears down an idle warm recorder, freeing the input device — which is what + /// lets a Bluetooth output route return to its full-quality profile. See + /// `preparedRecorderLifetime`. + /// + /// A stale expiry — one whose recorder was consumed by a press, or replaced by + /// a later re-warm — must do nothing at all, which is what `generation` buys. + /// Cancellation alone doesn't cover it: an expiry that already passed its + /// `!Task.isCancelled` check still gets its actor turn, and would otherwise nil + /// out the *live* expiry's handle (leaving the current warm recorder with no + /// countdown at all) and tear down a recorder prepared a moment ago. + func releasePreparedRecorder(generation: Int) { + guard let held = warm, held.generation == generation else { return } + warm = nil + Self.removeFile(at: held.recorder.url) + Self.logger.info("released idle warm recorder") + } +} diff --git a/Sources/BlurtEngine/Audio/MicCapture.swift b/Sources/BlurtEngine/Audio/MicCapture.swift index 11a8bfd4..5d7230c7 100644 --- a/Sources/BlurtEngine/Audio/MicCapture.swift +++ b/Sources/BlurtEngine/Audio/MicCapture.swift @@ -15,7 +15,7 @@ public actor MicCapture: MicCaptureProtocol { // log show --predicate 'subsystem == "dev.alex.blurt"' --last 1h // Stderr is unreachable for .app bundles launched via Finder/LaunchServices, // so go through the unified logging system instead. - private static let logger = Logger(subsystem: BlurtIdentity.subsystem, category: "MicCapture") + static let logger = Logger(subsystem: BlurtIdentity.subsystem, category: "MicCapture") public nonisolated let levels: AsyncStream private nonisolated let levelsContinuation: AsyncStream.Continuation @@ -26,15 +26,85 @@ public actor MicCapture: MicCaptureProtocol { /// resampling or re-encoding pass. private static let targetSampleRate = Double(SyncSTTLimits.sampleRate) - /// Pre-prepared in `warmUp()` so the *first* dictation doesn't pay hardware - /// route discovery on the hot path; consumed by the first `start()`. Every - /// later session creates a fresh recorder instead, so a device switch is always - /// reflected. (A switch in the brief launch→first-dictation window is not worth - /// guarding against — it would cost a CoreAudio device listener for a case that - /// effectively never happens.) - private var preparedRecorder: AVAudioRecorder? + /// A recorder prepared ahead of the press so `start()` doesn't pay hardware + /// route activation on the hot path, together with everything that belongs to + /// *that* recorder: the input it is bound to, its idle countdown, and the + /// ticket that countdown carries. + /// + /// One optional rather than four parallel fields, so "a recorder without its + /// input snapshot" and "an expiry without its recorder" are unrepresentable + /// instead of merely never written. + /// + /// Filled by `warmUp()` at launch and **re-filled after every capture** (see + /// `scheduleRewarm`), because that cost is paid per session, not once: + /// `prepareToRecord()` is where the route is resolved and opened, and on a + /// Bluetooth input that means renegotiating the link into its mic-capable + /// mode — hundreds of milliseconds, sometimes over a second, during which the + /// user has pressed the key and nothing has happened. Warming only the first + /// session (the previous behavior) hid that cost for one dictation out of N. + /// + /// Still a *fresh recorder per session*, which is the invariant the + /// `AVAudioEngine` rewrite bought: the warm recorder is validated against the + /// live default input before it is used (`takeWarmRecorder`) and discarded + /// rather than reused when the device has changed underneath it. + var warm: WarmRecorder? + + /// Bumped for each warm recorder prepared, so an expiry whose recorder has + /// since been consumed or replaced can recognise itself as stale. See + /// `releasePreparedRecorder(generation:)` for why cancelling the task isn't + /// sufficient on its own. + var preparedGeneration = 0 + + /// A prepared-but-not-started recorder and the state that only makes sense + /// alongside it. + struct WarmRecorder { + let recorder: AVAudioRecorder + /// The default input it was built against. `AVAudioRecorder` resolves its + /// device once, at `prepareToRecord()`, and never re-resolves — so without + /// this a recorder warmed while the built-in mic was default would keep + /// recording from it after the user connected their AirPods. See + /// `AudioRoute.InputSnapshot.deviceID` for why identity is the device ID. + let input: AudioRoute.InputSnapshot? + /// Releases this recorder once it has gone unused for + /// `preparedRecorderLifetime`. See that constant for why holding one open + /// forever is not an option. + var expiry: Task? + /// The ticket `expiry` carries, so a stale one can identify itself. + let generation: Int + } + /// The recorder for the in-flight session; nil between `stop()` and `start()`. - private var activeRecorder: AVAudioRecorder? + var activeRecorder: AVAudioRecorder? + /// The in-flight session's input transport, sampled once at `start()`. Read by + /// `stop()` to size the tail linger — sampled at start rather than re-read at + /// stop so a device switch mid-utterance can't make the two halves of one + /// capture disagree. + private var activeTransportType: UInt32? + + /// Incremented by every `stop()` / `cancelCapture()`. `start()` snapshots it + /// before suspending in the liveness wait — its one internal suspension — and + /// re-checks after, so a teardown that interleaves during the wait wins. + /// Without it, a reentrant caller's stop saw `activeRecorder == nil`, returned + /// an empty "clean stop", and the not-yet-installed recorder kept capturing + /// (mic indicator hot, temp WAV leaked) until `start()` resumed. Unreachable + /// through `DictationSession` — its serial command queue runs release/cancel + /// only after the press turn completes — but this is a public actor, and any + /// host can call it unqueued. + private var stopGeneration = 0 + + /// True from `record()` succeeding until the capture is installed or torn + /// down — i.e. across the liveness wait. + /// + /// Needed because during that window **both** recorder slots are nil: + /// `activeRecorder` isn't installed until the wait returns (the recorder stays + /// confined to `start()` so nothing can touch it while the poll loop reads its + /// clock off-actor), and `warm` was consumed on the way in. Without this, + /// `warmUp()` — which every re-warm goes through — reads those two nils as "no + /// capture in flight" and prepares a *second* recorder onto the already-live + /// input, which is very reachable since `stop()` schedules a re-warm that can + /// land inside the next press's bring-up. + var bringingUpCapture = false + /// Polls the active recorder's meter and feeds `levels` while recording. private var meterTask: Task? /// The last value `emitLevel` put on the stream, so an unchanged tick can be @@ -56,6 +126,19 @@ public actor MicCapture: MicCaptureProtocol { /// `meterIntervalSeconds` as the `Duration` the meter task sleeps for. private static let meterInterval = Duration.seconds(meterIntervalSeconds) + /// How long a prepared-but-unused recorder is held before being torn down. + /// + /// The warm recorder is the fix for per-session route activation, but it is + /// not free to hold: a prepared recorder keeps the input device open, and an + /// open input is exactly what pins AirPods in their mic-capable profile, where + /// *output* audio is degraded. Holding one indefinitely would trade dictation + /// latency for permanently worse music. This bounds that: back-to-back + /// dictations — the case the warm recorder exists for — land well inside the + /// window, and a user who stops dictating gets their output route back shortly + /// after. The next press past the window simply prepares lazily, which is the + /// behavior that shipped before the re-warm existed. + static let preparedRecorderLifetime = Duration.seconds(60) + public init() { // The continuation is fed from a ~20 Hz meter timer; the levels stream is a // meter, not the captured signal — the consumer only renders the most recent @@ -65,26 +148,15 @@ public actor MicCapture: MicCaptureProtocol { self.levelsContinuation = continuation } - /// Pre-create and prepare a recorder so the first `start()` skips first-time - /// hardware route discovery. Does NOT begin capture — no mic indicator. Safe to - /// call multiple times; a failure here just leaves `start()` to prepare lazily. - public func warmUp() { - guard preparedRecorder == nil else { return } - do { - preparedRecorder = try Self.makeRecorder() - Self.logger.info("warmUp prepared recorder") - } catch { - Self.logger.error("warmUp failed: \(error.localizedDescription, privacy: .public)") - } - } - public func start() async throws { - // Reuse the warm recorder for the first session; otherwise build a fresh one - // bound to the current default input device. `??` only evaluates - // `makeRecorder()` when nothing was warmed, so the clear below can be - // unconditional — it is a no-op in exactly the case that could have thrown. - let recorder = try preparedRecorder ?? Self.makeRecorder() - preparedRecorder = nil + // One CoreAudio read per session, answering both questions this capture has + // about its input: whether the warm recorder is still bound to it, and + // whether its link buffers a tail worth waiting for at stop. + let input = AudioRoute.currentInput() + // Reuse the warm recorder when it is still bound to the current default + // input; otherwise build a fresh one. `??` only evaluates `makeRecorder()` + // when nothing usable was warmed. + let recorder = try takeWarmRecorder(matching: input) ?? Self.makeRecorder() // record() returns false when no usable input device is available (unplugged, // asleep, route lost). Surface that as a thrown Swift error so @@ -97,35 +169,141 @@ public actor MicCapture: MicCaptureProtocol { throw BlurtError.audioCaptureFailed(underlying: MicCaptureError.noInputDevice) } + // The input is open from here, so claim the bring-up window before the first + // suspension — see `bringingUpCapture`. `defer` clears it on every exit, + // including the abort throw below. + bringingUpCapture = true + defer { bringingUpCapture = false } + + // `record()` returning true only means the AudioQueue started — not that the + // input route is delivering frames. A Bluetooth mic spends up to a couple of + // seconds switching into its mic-capable profile first, and the OS captures + // nothing in that window, so returning here immediately cues the user to + // speak into a dead mic and the first words never reach the transcript. + // Hold — `DictationSession` keeps the pill in `.connecting` and the start + // chime waits — until the recorder's clock advances, capped per transport; + // on timeout proceed anyway (fail open), which is exactly the old behavior. + // + // The re-warm above is what makes this cheap in the common case: a warm + // recorder has already held the route open, so the wait usually returns + // immediately. This gate is what makes it *correct* when it hasn't. + let timeout = MicLiveness.timeout(forTransportType: input?.transportType) + let generationBeforeWait = stopGeneration + let gap = await MicLiveness.waitUntilLive(timeout: timeout, clock: ContinuousClock()) { + // Off-actor read (`waitUntilLive` is nonisolated), safe by confinement: + // the polls run sequentially within one task, and nothing else references + // this recorder while `start()` is suspended in the wait — it isn't + // `activeRecorder` yet, the warm slot was cleared above, and the meter + // task hasn't started. + recorder.currentTime + } + + // Two ways the bring-up can be abandoned while suspended, both ending the + // same way — tear the recorder down instead of installing it, so nothing is + // left capturing and no temp file is orphaned: + // + // - A teardown landed (`stopGeneration` moved), so the caller's stop has to + // stay a real stop rather than returning an empty "clean" one. + // - The task was cancelled, which is how a cancel preempts the wait: + // `waitUntilLive` returns as soon as it sees it, so this is the difference + // between an Escape acting now and acting in `bluetoothTimeout`. It must be + // distinguished from the timeout, which returns nil too but means "fail + // open, proceed as if live". + guard stopGeneration == generationBeforeWait, !Task.isCancelled else { + recorder.stop() + Self.removeFile(at: recorder.url) + Self.logger.info("start aborted — teardown or cancellation during the liveness wait") + throw CancellationError() + } + + if let gap { + Self.logger.info("input live after \(Int(gap.milliseconds.rounded())) ms") + } else { + Self.logger.error( + "input liveness unconfirmed after \(Int(timeout.milliseconds.rounded())) ms — proceeding") + } + activeRecorder = recorder + activeTransportType = input?.transportType lastEmittedLevel = nil Self.logger.info("start recording to \(recorder.url.lastPathComponent, privacy: .public)") startMeterTimer() } public func stop() async throws -> Data { - meterTask?.cancel() - meterTask = nil - guard let recorder = activeRecorder else { return Data() } - activeRecorder = nil + // Read before the suspension below, so this capture's decision can't be + // rewritten by whatever a later `start()` sets. + let linger = AudioTransport.tailLinger(forTransportType: activeTransportType) + guard let recorder = detachActiveRecorder() else { return Data() } + if linger > .zero { + // Keep capturing for a moment past key-up so the audio still travelling + // over the link lands in the file instead of being truncated. See + // `AudioTransport.tailLinger(forTransportType:)`. + try? await Task.sleep(for: linger) + } recorder.stop() let url = recorder.url - defer { Self.removeFile(at: url) } + defer { + Self.removeFile(at: url) + // Re-arm for the *next* press now that the device is free, so the route + // activation this session just paid for isn't paid again. Scheduled rather + // than done inline: preparing re-opens the input, which is the slow part, + // and `stop()` is on the release path the transcript waits behind. + scheduleRewarm() + } let pcm = try Self.decodePCM(fromFileAt: url) let sampleCount = pcm.count / SyncSTTLimits.bytesPerSample let durationMs = SyncSTTLimits.durationMs(ofPCMBytes: pcm.count) - Self.logger.info("stop samples=\(sampleCount) durationMs=\(durationMs)") + let lingerMs = Int(linger.milliseconds.rounded()) + Self.logger.info("stop samples=\(sampleCount) durationMs=\(durationMs) lingerMs=\(lingerMs)") return pcm } + /// Ends the capture and throws the audio away — the teardown behind + /// `DictationSession`'s cancels. + /// + /// Deliberately *not* `stop()`-and-discard: the user asked for nothing to + /// happen, so neither of `stop()`'s costs is worth paying. The Bluetooth tail + /// linger would delay the `.cancelled` phase (and with it the pill's dismissal) + /// to preserve audio about to be deleted, and reading the whole recording back + /// off disk would decode a blob with no consumer. + /// + /// Not marked `throws`, because nothing on this path can fail — a + /// non-throwing implementation satisfies the `throws` requirement fine. The + /// *requirement* keeps it, so that the protocol's stop-and-discard default + /// (which can throw, via `stop()`) still conforms and `stopAndCancel`'s + /// developer-mode failure log keeps working for hosts that take it. + public func cancelCapture() { + guard let recorder = detachActiveRecorder() else { return } + recorder.stop() + Self.removeFile(at: recorder.url) + Self.logger.info("cancelled capture, discarded audio") + scheduleRewarm() + } + + /// Ends the current capture's claim on the actor's state and hands back the + /// recorder to dispose of, or nil when there was none. + /// + /// Shared by `stop()` and `cancelCapture()` because the two must not diverge: + /// the generation bump in particular is what `start()` re-checks across its + /// liveness wait to know a teardown landed, so a third exit that forgot it + /// would let an abandoned bring-up install itself and keep capturing. + private func detachActiveRecorder() -> AVAudioRecorder? { + stopGeneration += 1 + meterTask?.cancel() + meterTask = nil + defer { activeRecorder = nil } + return activeRecorder + } + // MARK: - Recorder construction /// Build a recorder that writes mono 16-bit little-endian PCM at the target /// rate into a unique temp file. `prepareToRecord()` does the heavy route/buffer /// setup so the subsequent `record()` starts promptly. - private static func makeRecorder() throws -> AVAudioRecorder { + static func makeRecorder() throws -> AVAudioRecorder { let url = FileManager.default.temporaryDirectory .appendingPathComponent("blurt-\(UUID().uuidString).wav") let settings: [String: Any] = [ @@ -173,7 +351,11 @@ public actor MicCapture: MicCaptureProtocol { } // The dB→0...1 conversion `emitLevel` uses lives in `MicCapture+Meter.swift` - // — pure math the coverage gate counts, unlike this hardware-bound actor. + // — pure math the coverage gate counts, unlike this hardware-bound actor. The + // warm-recorder lifecycle (`warmUp`, `takeWarmRecorder`, the re-warm and its + // expiry) lives in `MicCapture+Warm.swift`, split off for the lint + // file-length budget — which is why the prepared-recorder state, `logger`, + // `removeFile` and `makeRecorder` are internal rather than private. // MARK: - File helpers @@ -195,7 +377,7 @@ public actor MicCapture: MicCaptureProtocol { return Data(bytes: channel, count: Int(buffer.frameLength) * SyncSTTLimits.bytesPerSample) } - private static func removeFile(at url: URL) { + static func removeFile(at url: URL) { try? FileManager.default.removeItem(at: url) } } diff --git a/Sources/BlurtEngine/Audio/MicCaptureProtocol.swift b/Sources/BlurtEngine/Audio/MicCaptureProtocol.swift index 9323daed..1adb65ca 100644 --- a/Sources/BlurtEngine/Audio/MicCaptureProtocol.swift +++ b/Sources/BlurtEngine/Audio/MicCaptureProtocol.swift @@ -2,12 +2,29 @@ import Foundation public protocol MicCaptureProtocol: Sendable { /// Begin capturing 16 kHz mono 16-bit PCM. Throws on permission/device failure. + /// + /// **May hold until the input device actually delivers frames.** A Bluetooth + /// route spends up to ~2.5 s switching into its mic-capable profile before any + /// audio flows, and the OS captures nothing in that window. Hosts render the + /// whole in-flight call as a distinct "connecting" state (the pipeline's + /// `.connecting` phase), with the "speak now" cues — the recording pill, the + /// start chime — arriving only on return. A conformer that returns before + /// frames flow cues the user to speak into a dead mic, and the first words of + /// the utterance are unrecoverable. func start() async throws /// Stop capture and return the captured audio as raw S16LE PCM bytes — the /// exact encoding the dictation request uploads, so no conversion pass sits on /// the release hot path. Throws if the captured audio couldn't be read back, /// so the pipeline can surface an error instead of silently dropping speech. func stop() async throws -> Data + /// Stop capture and discard the audio — the teardown behind a *cancel*, where + /// the user asked for nothing to happen. Split from `stop()` because the two + /// want opposite things: `stop()` may legitimately spend time preserving the + /// captured audio (`MicCapture` waits out a Bluetooth link's tail before + /// ending the recording), whereas a cancel must take effect immediately and + /// has nothing to preserve. Declared here (not only in the default extension) + /// so it dispatches dynamically through `any MicCaptureProtocol`. + func cancelCapture() async throws /// Loudness feed for a meter UI: `0…1`, emitted while recording. Declared on /// the protocol (with an empty-stream default below) so hosts read the meter /// through the same seam they inject — a stub without a meter satisfies it @@ -29,4 +46,12 @@ extension MicCaptureProtocol { /// No-op default: a capture with nothing to pre-open inherits this, mirroring /// `TranscriberProtocol.warmUp`. public func warmUp() async {} + + /// Stop-and-discard default, so a capture with no cancel-specific teardown + /// (every stub) conforms for free and still records the stop the way it always + /// did. Implementations override it when stopping cheaply differs from + /// stopping carefully — `MicCapture` does. + public func cancelCapture() async throws { + _ = try await stop() + } } diff --git a/Sources/BlurtEngine/Audio/MicLiveness.swift b/Sources/BlurtEngine/Audio/MicLiveness.swift new file mode 100644 index 00000000..78846cb4 --- /dev/null +++ b/Sources/BlurtEngine/Audio/MicLiveness.swift @@ -0,0 +1,75 @@ +import Foundation + +/// The pure decision half of `MicCapture.start()`'s liveness gate: how long to +/// wait for the input device to actually deliver frames, and the polling loop +/// that detects when it has. Kept out of the hardware-bound capture actor (the +/// same split as `MicCapture+Meter`) so the timeout policy and the wait's edge +/// cases are unit-tested against an injected clock. +enum MicLiveness { + /// The first re-check delay, doubling up to `maxPollInterval`. + /// + /// Geometric rather than a fixed quantum because the two cases this loop + /// serves want opposite things. `AVAudioRecorder.currentTime` is 0 the instant + /// `record()` returns, so *every* press sleeps at least once before + /// `.recording`, the start chime and the meter — on a wired mic that quantum + /// is the entire wait, and it should be as small as possible. A real Bluetooth + /// bring-up, meanwhile, runs to seconds, where a small fixed quantum is + /// hundreds of timer wakeups each doing a clock read for an answer that hasn't + /// changed. Starting at 1 ms and doubling gives the common case a ~1 ms + /// acknowledgement and the slow case ~30 wakeups instead of 250, at the cost + /// of at most one `maxPollInterval` of detection granularity out of a 1–2 s + /// wait. + static let initialPollInterval: Duration = .milliseconds(1) + + /// Ceiling for the backoff — the coarsest the loop ever gets. + static let maxPollInterval: Duration = .milliseconds(25) + + /// Wait cap for Bluetooth inputs: bringing an AirPods mic up means a profile + /// switch into the mic-capable mode that takes ~1–2 s, and the link drops back + /// to the output-only profile after an idle gap — so a dictation after a pause + /// pays it again, not just the first one. + /// + /// `MicCapture`'s re-warm shortens how *often* this is paid (it keeps the + /// input open between dictations); this cap governs what happens when it is + /// paid anyway. + static let bluetoothTimeout: Duration = .milliseconds(2500) + + /// Wait cap for every other transport (and an unreadable one): wired and + /// built-in inputs deliver frames near-instantly, so a route that hasn't + /// within this budget is broken and the gate fails open without ever making a + /// healthy mic feel laggy. + static let defaultTimeout: Duration = .milliseconds(300) + + /// The wait cap for an input device of the given CoreAudio transport type + /// (`kAudioDevicePropertyTransportType`); nil means the type couldn't be read, + /// which gets the conservative default cap. + static func timeout(forTransportType transportType: UInt32?) -> Duration { + AudioTransport.isBluetooth(transportType) ? bluetoothTimeout : defaultTimeout + } + + /// Polls `currentTime` on a backoff (see `initialPollInterval`) until it + /// advances past zero. The recorder's clock only moves once the input device + /// delivers frames, which is what distinguishes "route still switching" (clock + /// stuck at 0) from "user is silent" (clock advancing over quiet audio) — a + /// meter level can't. + /// + /// Returns the elapsed wait once frames flow, or nil when `timeout` (or a task + /// cancellation) won the race. The caller FAILS OPEN on nil — proceeding + /// exactly as if live — because a silent or broken mic must degrade to the old + /// behavior, never brick the press. + static func waitUntilLive( + timeout: Duration, + clock: some Clock, + currentTime: @escaping @Sendable () -> TimeInterval + ) async -> Duration? { + let start = clock.now + let deadline = start.advanced(by: timeout) + var interval = initialPollInterval + while currentTime() <= 0 { + guard clock.now < deadline, !Task.isCancelled else { return nil } + try? await clock.sleep(for: interval) + interval = min(interval * 2, maxPollInterval) + } + return start.duration(to: clock.now) + } +} diff --git a/Sources/BlurtEngine/Duration+Milliseconds.swift b/Sources/BlurtEngine/Duration+Milliseconds.swift new file mode 100644 index 00000000..75505ded --- /dev/null +++ b/Sources/BlurtEngine/Duration+Milliseconds.swift @@ -0,0 +1,8 @@ +extension Duration { + /// This duration in milliseconds as a Double (for latency logging). Expressed + /// as a ratio of two `Duration`s rather than reassembled from `components`, + /// which meant restating the attoseconds-per-millisecond constant by hand. + var milliseconds: Double { + self / Duration.milliseconds(1) + } +} diff --git a/Sources/BlurtEngine/Pipeline/DictationSession+Commands.swift b/Sources/BlurtEngine/Pipeline/DictationSession+Commands.swift index 8ec9bcab..23aba1b8 100644 --- a/Sources/BlurtEngine/Pipeline/DictationSession+Commands.swift +++ b/Sources/BlurtEngine/Pipeline/DictationSession+Commands.swift @@ -18,6 +18,16 @@ extension DictationSession { /// single serial consumer — see `init`). Hosts that can `await` may call the /// async methods directly instead; the two styles hit the same serial queue. public nonisolated func submit(_ command: Command) { + // A cancel acts *now*, not when its turn comes up. The consumer below is + // serial by design, so a `.cancel` submitted during a press waits for that + // press to finish before `cancel()` even runs — and since `MicCapture.start()` + // holds until the mic is delivering audio, on a Bluetooth route that is + // seconds of an Escape that appears to do nothing. Recording the intent and + // preempting the press here is what `cancel()` does for a live pipeline; + // this gives the submitted door the same reach. The command is still yielded + // in order, and still executes in order — `performCancel` is idempotent + // against a cancel already consumed. + if command == .cancel { requestCancel() } commandFeed.yield(command) } @@ -32,6 +42,58 @@ extension DictationSession { } } + /// Set synchronously by either entry point — `cancel()` or `submit(.cancel)` — + /// before taking a queue turn, so a cancel arriving while a queued release + /// hasn't yet claimed `.transcribing` deterministically wins: `performRelease` + /// consumes the request after its `mic.stop()`, before any pipeline is spawned. + /// (A release that already claimed `.transcribing` is handled by `cancel()`'s + /// synchronous path instead.) `performCancel` clears it whether or not it was + /// consumed early, and `performPress` consumes it before claiming `.recording` + /// so a cancel during the mic bring-up never yields a phantom recording. + /// + /// Internal, like `pipelineTask` and `autoReleaseTask`, because the moment the + /// request is recorded is otherwise unobservable: a test landing a cancel + /// against an in-flight press has to know it was recorded before it releases + /// that press, and the alternative — draining a fixed number of `Task.yield()`s + /// and hoping — is a budget that drains the calling task, not this actor. + nonisolated var cancelRequested: Bool { + get { cancelState.withLock { $0.requested } } + set { cancelState.withLock { $0.requested = newValue } } + } + + /// Handle to the press currently on the command queue, so a cancel can + /// **preempt** the mic bring-up rather than queue behind it — the same + /// treatment `pipelineTask` already gives `.transcribing`/`.injecting`. + /// `MicCapture.start()` can hold for seconds on a Bluetooth route, and + /// `MicLiveness.waitUntilLive` already returns early on task cancellation, so + /// cancelling this handle unblocks it at once. + /// + /// Beside `requested` under the same lock, and for the same reason: reachable + /// from `nonisolated` `submit(_:)`, whose whole job is to act without waiting + /// for a turn on an actor the in-flight press is holding. + nonisolated var inFlightPress: Task? { + get { cancelState.withLock { $0.press } } + set { cancelState.withLock { $0.press = newValue } } + } + + /// Records the cancel intent and preempts an in-flight press, synchronously + /// and from any isolation. The one place both doors funnel through, so + /// `submit(.cancel)` and `cancel()` can't drift apart. + nonisolated func requestCancel() { + let press = cancelState.withLock { state -> Task? in + state.requested = true + return state.press + } + press?.cancel() + } + + /// Drops the press handle if it is still `task`'s — a later press may already + /// have claimed the slot, and clearing that one would let its bring-up run + /// un-cancellable. + nonisolated func clearInFlightPress(_ task: Task) { + cancelState.withLock { if $0.press == task { $0.press = nil } } + } + public func cancel() async { // A cancel that lands once `.transcribing` is claimed — while the release // is still inside mic.stop(), or later with the transcribe→inject task in @@ -46,12 +108,24 @@ extension DictationSession { setPhase(.cancelled) return } + // `.connecting` gets the same treatment, for the same reason: it is in-flight + // work with a handle to cancel. The press is suspended inside + // `MicCapture.start()`'s liveness wait, which honors task cancellation and + // returns at once — so this unblocks a bring-up that could otherwise hold for + // `MicLiveness.bluetoothTimeout`. `start()` tears the recorder down and throws + // `CancellationError`, which `performPress` maps to `.cancelled` — already + // claimed here, so the pill answers the Escape immediately. + if phase == .connecting { + requestCancel() + setPhase(.cancelled) + return + } // Record the intent before taking a queue turn: a release queued ahead of // our turn consumes it the moment its mic.stop() returns (no pipeline is // ever spawned), and a press ahead in the queue is followed by our own // turn, which ends the freshly started recording. Either way the cancel is // honored in arrival order, never dropped. - cancelRequested = true + requestCancel() await enqueue { await self.performCancel() } } diff --git a/Sources/BlurtEngine/Pipeline/DictationSession+Press.swift b/Sources/BlurtEngine/Pipeline/DictationSession+Press.swift new file mode 100644 index 00000000..3f9d68ec --- /dev/null +++ b/Sources/BlurtEngine/Pipeline/DictationSession+Press.swift @@ -0,0 +1,177 @@ +import Dispatch + +// The press half of the pipeline — everything between the key going down and +// `.recording` being claimed, including the mic bring-up that `.connecting` +// covers. Split from `DictationSession.swift` to stay within the lint +// file-length budget, mirroring `+Pipeline` (the release half). Members it +// reaches are internal, not private: file-scoped access can't cross the split. +// `Dispatch`, not `Foundation`: the only thing here from outside the module is +// `contextQueue.async` (see `performPress` for why that read is off-pool). +extension DictationSession { + func performPress() async { + guard phase.isTerminal else { return } + // Refuse the press before any capture begins when the host reports a + // blocker (e.g. no API key saved): recording an utterance that can only + // fail at transcribe time would discard the user's words after the fact. + if let blocker = readinessCheck() { + setPhase(.failed(blocker)) + return + } + // Times the startup path — the mic bring-up and the context capture that now + // runs alongside it (plus the detached connection warm-up) — up to the moment + // recording actually begins. Ended on both the success and failure exits + // (`mic.start()` is the only throwing call, and it precedes `.recording`, so + // the two ends are mutually exclusive). + let pressInterval = Self.signposter.beginInterval(Self.pressSignpostName) + // Claim `.connecting` before `mic.start()`: its liveness gate holds until + // the input route actually delivers frames, which on a Bluetooth route is + // ~1–2 s. The press must be visibly acknowledged in that window without + // cueing the user to speak — the pill shows a warming-up state, and the + // start chime rides the connecting→recording edge (`RecordingCueGate`), so + // it fires only once audio is genuinely flowing. `.recording` therefore + // keeps meaning exactly what it says. + setPhase(.connecting) + do { + // Pre-open the dictation connection while the user speaks, so the first dictation after an idle + // gap doesn't pay DNS+TCP+TLS on the transcribe hot path (~170 ms cold, measured). Detached + // + fire-and-forget: it must never delay recording, and a failure is harmless (the request + // just pays setup as before); warming every press is cheap since a hot pool just reuses it. + let transcriber = transcriber + Task.detached { await transcriber.warmUp() } + // The mic bring-up runs as a child task so the whole context-capture chain + // below overlaps it instead of queueing behind it. That ordering used to be + // free — `mic.start()` returned in microseconds — but the liveness gate can + // now hold it for `MicLiveness.bluetoothTimeout`, and every one of those + // milliseconds was dead time the AX read could have used. Sequenced after + // it, the read instead landed on the *release* path, where + // `runTranscribeInject` waits up to `contextWaitBudget` for it with the + // user watching. Now it is almost always finished before the mic is even + // live. + // + // `async let`, so a cancel still reaches it: the child inherits this task's + // cancellation, which is what `cancel()`'s `.connecting` branch relies on + // to preempt the wait. + async let started: Void = mic.start() + await beginContextCapture() + // Only now join the bring-up. Everything above ran while the mic was + // coming up; the phase still flips to `.recording` only once `start()` + // returns, so the UI never claims capture that isn't live. The cost of + // starting the context work first is that a press whose mic fails has + // already set the injector's target and dispatched one AX read — both + // harmless and overwritten by the next press. + try await started + // A cancel that arrived during the bring-up, on the path where `start()` + // still returned normally — the cancel landed in the window between the + // liveness wait finishing and `.recording` being claimed, so there was + // nothing left to interrupt. (When it lands *during* the wait, `start()` + // throws `CancellationError` instead and the catch below owns it.) + // + // Either way the mic is live by now, so tear it down rather than claiming + // `.recording` and chiming "speak now" for a capture the user has already + // abandoned. `Task.isCancelled` is checked alongside the flag because the + // preempting door cancels this task without setting anything else. + if cancelRequested || Task.isCancelled { + try? await mic.cancelCapture() + // `consumeCancelRequest` claims `.cancelled` when the flag route was + // used; the task-cancellation route already claimed it in `cancel()`, so + // only fall back when nothing has. + if !consumeCancelRequest() { setPhase(.cancelled) } + Self.signposter.endInterval(Self.pressSignpostName, pressInterval) + return + } + setPhase(.recording) + Self.signposter.endInterval(Self.pressSignpostName, pressInterval) + let timeout = maxRecordingSeconds + let clock = clock + autoReleaseTask = Task { [weak self] in + try? await clock.sleep(for: .seconds(timeout)) + guard let self, !Task.isCancelled else { return } + // Enqueues like a manual key-up. If a real release already ran, the + // queued performRelease sees a non-.recording phase and drops out. + await self.release() + } + } catch { + Self.signposter.endInterval(Self.pressSignpostName, pressInterval) + // `MicCapture.start()` throws `CancellationError` when a teardown landed + // during its liveness wait — an unqueued `cancelCapture()`, which its own + // doc anticipates for hosts that don't drive the mic through this session. + // That's the user's cancel arriving by another door, not a fault: reporting + // it as `.audioCaptureFailed` would flash the pill red *and* write a + // developer-mode error-log entry for something nothing went wrong in. Same + // rule `transcribe` and `inject` already follow. `.cancelled` rather than a + // bare return, because `.connecting` is non-terminal — leaving it would + // strand the trigger's gate and swallow the next press. + // + // Consumed the same way as the exit above, not just phase-set: the + // `.connecting` branch of `cancel()` claims the phase and returns + // *without* enqueueing `performCancel`, so this is the only place left + // that can clear the request it recorded. Leaving it set let the flag + // survive into the next press, which then read it after a perfectly good + // `mic.start()` and cancelled itself. + if error is CancellationError { + if !consumeCancelRequest() { setPhase(.cancelled) } + return + } + setPhase(.failed(.audioCaptureFailed(underlying: error))) + } + } + + /// Captures the paste target and kicks off the press-time AX field-context + /// read, leaving the result in `contextStream` for `runTranscribeInject` to + /// consume. + /// + /// Called *before* the bring-up is joined, so all of it — including the + /// cross-process AX read, the expensive part — overlaps the mic coming up + /// rather than queueing behind it. Split out of `performPress` for the lint + /// function-length budget; it is one phase of the press, not a reusable step. + private func beginContextCapture() async { + // Capture the frontmost app (paste target). A cheap in-process AppKit read + // on the main actor. Lifted out of the actor first (like `transcriber` + // above) so the call is a Sendable closure rather than isolated state. + let captureFrontmost = seams.captureFrontmost + let captured = await captureFrontmost() + await injector.setTargetApp(captured.flatMap { FocusCapture.runningApp(for: $0) }) + // Key terms are read synchronously at press (cheap UserDefaults read), so + // each dictation observably re-reads Settings edits at press time. + let keyTerms = keyTermsProvider() + // Session history, read on the actor for the same reason: the capture below + // runs off-actor, so what it carries has to be a value taken now. + let recentTranscripts = recentDictations.transcriptsOldestFirst + // Kick off the AX field-context read now, while the target field still + // holds focus, but don't await it here: it's cross-process IPC into the + // frontmost app (detached — off the main actor, where it froze the + // overlay, and off this actor, where it would wedge release()/cancel()). + // runTranscribeInject consumes the result right before transcription, + // bounded by `contextWaitBudget` — so a slow AX target delays the + // transcript by at most the budget, never the recording indicator. + let (stream, contextFeed) = AsyncStream.makeStream( + of: TranscriptionContext?.self, bufferingPolicy: .bufferingNewest(1)) + contextStream = stream + // A Dispatch queue, not `Task.detached`: `captureFieldContext` is documented + // as making ~6 synchronous cross-process AX round trips, each bounded only by + // the 1 s messaging timeout, so against a beachballing frontmost app one + // press can *block* a thread for seconds. The Swift cooperative pool is sized + // to the core count and does not overcommit, so a few press/cancel cycles + // against a hung app could park every cooperative thread and stall the whole + // non-main runtime — including this actor. Dispatch overcommits, so a blocked + // capture costs a thread instead of the pool. Same reasoning as + // `DictationLog`'s serial queue. Concurrent so a hung capture can't delay the + // next press's. The body is fully synchronous and captures only Sendable + // values, so it needs no task context. + let captureFieldContext = seams.captureFieldContext + Self.contextQueue.async { + let field = captureFieldContext() + let context = TranscriptionContext( + appName: captured?.processName, + windowTitle: field.windowTitle, + fieldLabel: field.fieldLabel, + priorText: field.priorText, + selectedText: field.selectedText, + recentTranscripts: recentTranscripts, + keyTerms: keyTerms, + targetIsSecure: field.isSecure) + contextFeed.yield(context.isEmpty ? nil : context) + contextFeed.finish() + } + } +} diff --git a/Sources/BlurtEngine/Pipeline/DictationSession.swift b/Sources/BlurtEngine/Pipeline/DictationSession.swift index 5f1a3d7c..de5c56b9 100644 --- a/Sources/BlurtEngine/Pipeline/DictationSession.swift +++ b/Sources/BlurtEngine/Pipeline/DictationSession.swift @@ -1,10 +1,10 @@ import Foundation -import os +import Synchronization public actor DictationSession { /// Off-pool home for the press-time AX field read — see its use in /// `performPress` for why blocking IPC must not run on the cooperative pool. - private static let contextQueue = DispatchQueue( + static let contextQueue = DispatchQueue( label: "\(BlurtIdentity.subsystem).FieldContext", qos: .userInitiated, attributes: .concurrent) @@ -17,12 +17,14 @@ public actor DictationSession { /// the failure from the log. public internal(set) var phase: PipelinePhase = .idle - // Split for the lint file-length budget: `phaseStream()`/`setPhase`/os_signpost - // live in `+Observation`; `submit(_:)` and both cancel commands live in - // `+Commands`; the post-release transcribe→inject pipeline lives in - // `+Pipeline`; the non-protocol collaborators (focus capture, developer-mode - // log) live in `+Seams`. Members those files reach are internal, not private - // (file-scoped access can't cross the split) — including `phase`'s setter. + // Split for the lint file-length budget: `performPress` — the whole press half, + // including the mic bring-up — lives in `+Press`, mirroring the post-release + // transcribe→inject pipeline in `+Pipeline`. `submit(_:)`, both cancel commands + // and the cancel-intent accessors over `cancelState` live in `+Commands`; + // `phaseStream()`/`setPhase`/os_signpost live in `+Observation`; the + // non-protocol collaborators (focus capture, developer-mode log) live in + // `+Seams`. Members those files reach are internal, not private (file-scoped + // access can't cross the split) — including `phase`'s setter. /// Live feeds of phase changes. Each `phaseStream()` call yields the current /// phase plus every subsequent transition, so the production renderer and @@ -36,7 +38,7 @@ public actor DictationSession { /// one at a time by the task spawned in `init`. nonisolated let commandFeed: AsyncStream.Continuation - private let mic: MicCaptureProtocol + let mic: MicCaptureProtocol let transcriber: TranscriberProtocol let injector: InjectorProtocol /// Supplies the user's key terms (domain vocabulary) at press time, so each @@ -44,12 +46,12 @@ public actor DictationSession { /// (`KeytermsBoost`), not as part of the conversation context. A closure, rather /// than a stored list, so edits in Settings take effect on the next dictation /// without rebuilding the session. Defaults to reading `KeyTermsStore`. - private let keyTermsProvider: @Sendable () -> [String] + let keyTermsProvider: @Sendable () -> [String] /// Auto-releases the hotkey after this long so a held key can't run forever. /// Defaults to just under the dictation API's audio cap (see /// `SyncSTTLimits`) — recording past it would only produce audio the /// endpoint rejects, so we stop early and transcribe what we have. - private let maxRecordingSeconds: Double + let maxRecordingSeconds: Double /// Clock the auto-release timer and the context-wait budget (`+Pipeline`) /// sleep on; injectable so tests advance it. let clock: any Clock @@ -60,7 +62,7 @@ public actor DictationSession { /// key-presence check so a missing API key fails at press time, not after the /// user has spoken a whole utterance. Defaults to always-ready (no Keychain /// read), so tests and keyless hosts are unaffected unless they opt in. - private let readinessCheck: @Sendable () -> BlurtError? + let readinessCheck: @Sendable () -> BlurtError? /// Fired once with the final transcript as soon as it's produced — before /// injection, so pasted, copied, and failed-to-paste dictations all count. The /// second argument is `recentDictations` as it stands, pushed from its one owner @@ -103,19 +105,18 @@ public actor DictationSession { /// a time in arrival order — none observes another suspended mid-`mic` call. private var commandQueue: Task? - /// Set synchronously by `cancel()` before it takes its queue turn, so a - /// cancel arriving while a queued release hasn't yet claimed `.transcribing` - /// deterministically wins: `performRelease` consumes the request after its - /// `mic.stop()`, before any pipeline is spawned. (A release that already - /// claimed `.transcribing` is handled by `cancel()`'s synchronous path - /// instead.) `performCancel` clears it whether or not it was consumed early. - /// - /// Internal, like `pipelineTask` and `autoReleaseTask`, because the moment the - /// request is recorded is otherwise unobservable: a test landing a cancel - /// against an in-flight press has to know it was recorded before it releases - /// that press, and the alternative — draining a fixed number of `Task.yield()`s - /// and hoping — is a budget that drains the calling task, not this actor. - var cancelRequested = false + /// Backing store for `cancelRequested` and `inFlightPress`. A `Mutex` rather + /// than actor state because **both doors into a cancel must record it + /// synchronously**, and one of them is `nonisolated`: `submit(.cancel)` can't + /// take an actor turn, and waiting for one is exactly the bug — the command + /// consumer is serial, so a submitted cancel sits unread in the feed until the + /// press it means to cancel has finished. + let cancelState = Mutex(CancelState()) + + struct CancelState { + var requested = false + var press: Task? + } // Internal, like `pipelineTask`, so a test can witness the cancel teardown // *directly* — nil means disarmed. Asserting it through the timer's effects @@ -200,119 +201,40 @@ public actor DictationSession { } } - /// Appends `op` to the serial command queue and waits for it to run. The - /// synchronous read-then-write of `commandQueue` makes the chain order match - /// the order the public methods executed their first actor turn. + /// Appends `op` to the serial command queue and waits for it to run; the + /// ordering guarantee is `chain`'s. func enqueue(_ op: @escaping @Sendable () async -> Void) async { + await chain(op).value + } + + /// Appends `op` to the serial command queue and hands back its handle + /// *without* waiting — the half of `enqueue` a caller needs when something + /// else must be able to reach the task while it runs. The synchronous + /// read-then-write of `commandQueue` is what makes the chain order match the + /// order the public methods executed their first actor turn. + private func chain(_ op: @escaping @Sendable () async -> Void) -> Task { let previous = commandQueue let task = Task { await previous?.value await op() } commandQueue = task - await task.value + return task } public func press() async { - await enqueue { await self.performPress() } + // Published before awaiting so a cancel can preempt the mic bring-up — see + // `inFlightPress`. Cleared on the way out, but only if it's still ours. + let task = chain { await self.performPress() } + inFlightPress = task + await task.value + clearInFlightPress(task) } public func release() async { await enqueue { await self.performRelease() } } - private func performPress() async { - guard phase.isTerminal else { return } - // Refuse the press before any capture begins when the host reports a - // blocker (e.g. no API key saved): recording an utterance that can only - // fail at transcribe time would discard the user's words after the fact. - if let blocker = readinessCheck() { - setPhase(.failed(blocker)) - return - } - // Times the startup path — the concurrent focus capture + mic.start (and the - // detached connection warm-up kicked off below) — up to the moment recording - // actually begins. Ended on both the success and failure exits (mic.start is - // the only throwing call, and it precedes `.recording`, so the two ends are - // mutually exclusive). - let pressInterval = Self.signposter.beginInterval(Self.pressSignpostName) - do { - // Pre-open the dictation connection while the user speaks, so the first dictation after an idle - // gap doesn't pay DNS+TCP+TLS on the transcribe hot path (~170 ms cold, measured). Detached - // + fire-and-forget: it must never delay recording, and a failure is harmless (the request - // just pays setup as before); warming every press is cheap since a hot pool just reuses it. - let transcriber = transcriber - Task.detached { await transcriber.warmUp() } - // Capture the frontmost app (paste target) concurrently with mic startup — - // a cheap in-process AppKit read on the main actor. The phase still flips - // to .recording only after mic.start succeeds, so the UI never lies about - // whether audio is being captured. Lifted out of the actor first (like - // `transcriber` above) so the child task calls a Sendable closure rather - // than reading isolated state. - let captureFrontmost = seams.captureFrontmost - async let frontmost = captureFrontmost() - try await mic.start() - let captured = await frontmost - await injector.setTargetApp(captured.flatMap { FocusCapture.runningApp(for: $0) }) - // Key terms are read synchronously at press (cheap UserDefaults read), so - // each dictation observably re-reads Settings edits at press time. - let keyTerms = keyTermsProvider() - // Session history, read on the actor for the same reason: the capture below - // runs off-actor, so what it carries has to be a value taken now. - let recentTranscripts = recentDictations.transcriptsOldestFirst - // Kick off the AX field-context read now, while the target field still - // holds focus, but don't await it here: it's cross-process IPC into the - // frontmost app (detached — off the main actor, where it froze the - // overlay, and off this actor, where it would wedge release()/cancel()). - // runTranscribeInject consumes the result right before transcription, - // bounded by `contextWaitBudget` — so a slow AX target delays the - // transcript by at most the budget, never the recording indicator. - let (stream, contextFeed) = AsyncStream.makeStream( - of: TranscriptionContext?.self, bufferingPolicy: .bufferingNewest(1)) - contextStream = stream - // A Dispatch queue, not `Task.detached`: `captureFieldContext` is documented - // as making ~6 synchronous cross-process AX round trips, each bounded only by - // the 1 s messaging timeout, so against a beachballing frontmost app one - // press can *block* a thread for seconds. The Swift cooperative pool is sized - // to the core count and does not overcommit, so a few press/cancel cycles - // against a hung app could park every cooperative thread and stall the whole - // non-main runtime — including this actor. Dispatch overcommits, so a blocked - // capture costs a thread instead of the pool. Same reasoning as - // `DictationLog`'s serial queue. Concurrent so a hung capture can't delay the - // next press's. The body is fully synchronous and captures only Sendable - // values, so it needs no task context. - let captureFieldContext = seams.captureFieldContext - Self.contextQueue.async { - let field = captureFieldContext() - let context = TranscriptionContext( - appName: captured?.processName, - windowTitle: field.windowTitle, - fieldLabel: field.fieldLabel, - priorText: field.priorText, - selectedText: field.selectedText, - recentTranscripts: recentTranscripts, - keyTerms: keyTerms, - targetIsSecure: field.isSecure) - contextFeed.yield(context.isEmpty ? nil : context) - contextFeed.finish() - } - setPhase(.recording) - Self.signposter.endInterval(Self.pressSignpostName, pressInterval) - let timeout = maxRecordingSeconds - let clock = clock - autoReleaseTask = Task { [weak self] in - try? await clock.sleep(for: .seconds(timeout)) - guard let self, !Task.isCancelled else { return } - // Enqueues like a manual key-up. If a real release already ran, the - // queued performRelease sees a non-.recording phase and drops out. - await self.release() - } - } catch { - Self.signposter.endInterval(Self.pressSignpostName, pressInterval) - setPhase(.failed(.audioCaptureFailed(underlying: error))) - } - } - private func performRelease() async { guard phase == .recording else { return } cancelAutoRelease() @@ -357,7 +279,7 @@ public actor DictationSession { /// Consumes a cancel requested while this release held the queue, claiming the /// phase for the user's cancel. Returns whether it fired. - private func consumeCancelRequest() -> Bool { + func consumeCancelRequest() -> Bool { guard cancelRequested else { return false } cancelRequested = false setPhase(.cancelled) @@ -373,7 +295,10 @@ public actor DictationSession { func stopAndCancel() async { cancelAutoRelease() do { - _ = try await mic.stop() + // `cancelCapture`, not `stop`: the audio is being thrown away, so neither + // preserving it (the Bluetooth tail linger) nor reading it back off disk + // is worth delaying the user's cancel for. + try await mic.cancelCapture() } catch { // Stays out of the UI: the user asked for nothing to happen, and a cancel // must not flash red (same rule as `performRelease`'s "a cancel wins over diff --git a/Sources/BlurtEngine/Pipeline/MenuBarStatus.swift b/Sources/BlurtEngine/Pipeline/MenuBarStatus.swift index 7bdc0bba..88db129c 100644 --- a/Sources/BlurtEngine/Pipeline/MenuBarStatus.swift +++ b/Sources/BlurtEngine/Pipeline/MenuBarStatus.swift @@ -40,7 +40,12 @@ extension PipelinePhase { switch self { case .recording: .recording case .transcribing: .transcribing - case .idle, .injecting, .cancelled, .failed, .pasted, .noTarget: .idle + // `.connecting` reads as idle: the filled glyph means "audio is being + // captured", and during the mic bring-up it isn't yet — the same honesty + // rule that holds the start chime. The pill carries the warming-up state, + // and a distinct fourth glyph isn't worth it for something usually gone + // within a frame on a wired mic. + case .idle, .connecting, .injecting, .cancelled, .failed, .pasted, .noTarget: .idle } } } diff --git a/Sources/BlurtEngine/Pipeline/OverlayUIState.swift b/Sources/BlurtEngine/Pipeline/OverlayUIState.swift index 19b1ca9a..d20c3883 100644 --- a/Sources/BlurtEngine/Pipeline/OverlayUIState.swift +++ b/Sources/BlurtEngine/Pipeline/OverlayUIState.swift @@ -3,6 +3,14 @@ /// is unit-testable; the shell just renders whatever this resolves to. public enum OverlayUIState: Equatable, Sendable { case idle + /// The press landed but the mic isn't delivering audio yet (the pipeline's + /// `.connecting` phase — a Bluetooth route takes ~1–2 s to come up). A steady + /// state, not a notice: it holds for exactly as long as the hardware takes, + /// which is a frame or two on the built-in mic. The shell renders it as a + /// breathing "Connecting…" status line, deliberately *without* the `● REC` + /// tag or the meter — those are the "speak now" cues, and speech during the + /// bring-up is unrecoverable, so the pill must not invite it. + case connecting case recording case processing /// A dictation attempt failed. The shell shows this as a brief red flash on @@ -27,6 +35,7 @@ public enum OverlayUIState: Equatable, Sendable { public var accessibilityLabel: String { switch self { case .idle: "Blurt." + case .connecting: "Connecting to the microphone." case .recording: "Recording." case .processing: "Processing." case .error(let message): message @@ -46,7 +55,7 @@ public enum OverlayUIState: Equatable, Sendable { switch self { case .pasted: 0.8 case .error, .noTarget: 1.6 - case .idle, .recording, .processing: nil + case .idle, .connecting, .recording, .processing: nil } } } @@ -64,6 +73,11 @@ extension PipelinePhase { // content — then `.pasted` arrived and faded it back in. A visible blink at // the end of a dictation, worst when the user has switched apps mid-transcribe. case .injecting: .processing + // Its own pill state, NOT `.recording`: the phase exists precisely because + // capture hasn't begun, so projecting it onto the recording pill would put + // the `● REC` tag and a live meter on screen over a mic that isn't + // delivering yet — the "speak now" cue the whole gate exists to withhold. + case .connecting: .connecting case .recording: .recording case .transcribing: .processing // A setup blocker (a missing API key) is an expected state, not a fault: the diff --git a/Sources/BlurtEngine/Pipeline/PipelinePhase.swift b/Sources/BlurtEngine/Pipeline/PipelinePhase.swift index 93d68089..9c9f09f2 100644 --- a/Sources/BlurtEngine/Pipeline/PipelinePhase.swift +++ b/Sources/BlurtEngine/Pipeline/PipelinePhase.swift @@ -2,6 +2,19 @@ import Foundation public enum PipelinePhase: Equatable, Sendable { case idle + /// The press was accepted and the mic is being brought up: `MicCapture`'s + /// liveness gate is still waiting for the input route to deliver frames (a + /// Bluetooth profile switch takes ~1–2 s). Claimed *before* `mic.start()`, so + /// the overlay answers the keypress immediately instead of at whatever moment + /// the hardware finishes coming up. + /// + /// Deliberately distinct from `.recording` rather than folded into it. The + /// projections below present it as a warming-up state, never as live capture, + /// and the start chime waits for `.recording` — so **the user is never cued to + /// speak into a mic that isn't delivering yet**, which is how the first words + /// of an utterance went missing on AirPods. Non-terminal, so a second press + /// during it is refused like one during `.recording`. + case connecting case recording case transcribing case injecting @@ -26,7 +39,7 @@ public enum PipelinePhase: Equatable, Sendable { public var isTerminal: Bool { switch self { case .idle, .failed, .cancelled, .pasted, .noTarget: true - case .recording, .transcribing, .injecting: false + case .connecting, .recording, .transcribing, .injecting: false } } diff --git a/Sources/BlurtEngine/Pipeline/RecordingCueGate.swift b/Sources/BlurtEngine/Pipeline/RecordingCueGate.swift index fd74569e..6599add4 100644 --- a/Sources/BlurtEngine/Pipeline/RecordingCueGate.swift +++ b/Sources/BlurtEngine/Pipeline/RecordingCueGate.swift @@ -10,10 +10,19 @@ public enum RecordingCue: Equatable, Sendable { /// Edge-detector deciding when the record start/stop chimes fire. The host calls /// `cue(for:)` on *every* pipeline phase, so the gate fires `.start` only on the -/// idle→recording edge and `.stop` only on the recording→not-recording edge, +/// edge into `.recording` and `.stop` only on the recording→not-recording edge, /// staying silent while a phase repeats and across transitions between two /// non-recording phases. Value type holding a single edge bit; the host owns one /// instance for the app's lifetime. +/// +/// The edge is `.recording` specifically, **not** "a press happened". In +/// production the press first claims `.connecting` while `MicCapture`'s liveness +/// gate waits for the input route to deliver frames, so the chime rides the +/// connecting→recording edge by construction — it sounds when audio is actually +/// flowing, not when the key went down. That ordering is the point: the chime is +/// a "speak now" cue, and on a Bluetooth route the two moments are ~1–2 s apart, +/// during which nothing is captured. Chiming at the press invites the user to +/// speak into a dead mic and loses the first words of the utterance. public struct RecordingCueGate: Sendable { private var wasRecording = false diff --git a/Sources/BlurtEngine/README.md b/Sources/BlurtEngine/README.md index fedcd755..d2e06bfa 100644 --- a/Sources/BlurtEngine/README.md +++ b/Sources/BlurtEngine/README.md @@ -4,7 +4,7 @@ BlurtEngine is the Swift package that powers [Blurt](../../README.md)'s dictatio ## What you get -- **`Sources/BlurtEngine/`** — a Swift package (`swift-tools-version:6.2`, macOS 15+, Swift 6 strict concurrency) with **no external dependencies**: just Foundation, Security, AVFoundation, toolchain modules like Synchronization, and AppKit types at the seams. That dependency-free rule is deliberate and enforced — don't add SPM dependencies to the engine. +- **`Sources/BlurtEngine/`** — a Swift package (`swift-tools-version:6.2`, macOS 15+, Swift 6 strict concurrency) with **no external dependencies**: just Foundation, Security, AVFoundation, CoreAudio, toolchain modules like Synchronization, and AppKit types at the seams. That dependency-free rule is deliberate and enforced — don't add SPM dependencies to the engine. - Pure logic behind three protocol seams (`MicCaptureProtocol`, `TranscriberProtocol`, `InjectorProtocol`), so every collaborator can be stubbed in tests and replaced in a host app. - Production implementations of all three seams (`MicCapture`, `AssemblyAITranscriber`, `KeyInjector`), plus the supporting pieces a dictation product needs: Keychain-backed API-key storage, per-utterance contextual prompting (the text before the cursor, and nothing else about the user's screen), a hotkey state machine, permission checks, and UI-state projections. @@ -74,7 +74,7 @@ Key properties of the design, which your integration can rely on: - **One request per utterance, no streaming.** The dictation API returns the complete transcript — and its LLM-rewritten form — in the response body: no upload step, no job polling, no incremental deltas, no second request for the cleanup. `TranscriberProtocol.transcribe` is a single `async throws -> String`. UIs should show a "transcribing…" state and then the whole result; there is nothing to stream. - **Cleanup happens server-side, and it's optional.** The request's `llm` block asks the service to apply our own cleanup instruction (`CleanupInstruction.text` — delete disfluencies, change nothing else) to the verbatim transcript inside the same call. It is one of three steering fields on the request: `config.conversation_context` primes the _transcription_ with the dialogue that came before — the user's recent dictations, then the text before the cursor (`ConversationContext`, omitted when there is neither) — and `config.word_boost` boosts the user's key terms (`KeytermsBoost`, omitted when there are none). There is no `config.prompt`; the context field replaced it. The block is gated by the **enhanced transcripts** setting (`EnhancedTranscriptsStore`, on by default): turned off, the config omits `llm` and the verbatim transcript is pasted as spoken. The user's **custom style instructions** (`CustomStyleStore`, empty by default) are appended to that instruction via `CleanupInstruction.sendable(appending:)`, trimmed to the headroom the API's 2048 instruction cap leaves (measured in UTF-8 bytes, the conservative bound — the cap's own unit is unmeasured); blank means the base instruction goes out unchanged. The engine pastes `llm_response`, falling back to the verbatim `text` when the best-effort rewrite failed (`llm_error`) — a degradation, never a user-facing error. There is no client-side LLM pass, no styling stage, and deliberately no hook for one. -- **Latency is pre-paid where possible.** `press()` fires a detached `warmUp()` at the transcriber (pre-opening the HTTPS connection while the user speaks, ~170 ms saved cold) and kicks off the cross-process accessibility read of the focused field without awaiting it — the read is then consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`, 500 ms), so an unresponsive frontmost app costs the transcript its priming, never a multi-second stall — and never delays the recording indicator. On the way out, `release()` flips the phase to `.transcribing` _before_ reading the recorded audio back, so a host's stop cue fires at key-up rather than after the disk read. +- **Latency is pre-paid where possible, and never faked.** `press()` claims `.connecting` before it touches the mic, so a host's pill answers the keypress — but `MicCapture.start()` deliberately holds until the input device is actually delivering frames, and the _start cue_ waits for `.recording`. On a Bluetooth route those are ~1–2 s apart and the OS captures nothing in between, so cueing at the press loses the first words. `MicCapture` re-arms its prepared recorder after every capture so that route activation is usually paid between dictations rather than during one. `press()` fires a detached `warmUp()` at the transcriber (pre-opening the HTTPS connection while the user speaks, ~170 ms saved cold) and kicks off the cross-process accessibility read of the focused field without awaiting it — the read is then consumed at transcribe time with a bounded wait (`DictationSession.contextWaitBudget`, 500 ms), so an unresponsive frontmost app costs the transcript its priming, never a multi-second stall — and never delays the recording indicator. On the way out, `release()` flips the phase to `.transcribing` _before_ reading the recorded audio back, so a host's stop cue fires at key-up rather than after the disk read. - **A held trigger auto-releases.** `DictationSession` stops recording after `maxRecordingSeconds` (default `SyncSTTLimits.autoReleaseSeconds`, 115 s) so audio never exceeds what the endpoint accepts, and transcribes what it has. Clips shorter than `SyncSTTLimits.minPCMBytes` (~100 ms of audio — an accidental tap) are dropped as a silent no-op rather than sent to earn a 400. ## DictationSession @@ -97,20 +97,27 @@ For callback-shaped hosts that can't `await` — an event tap, a button action `phase` / `phaseStream()` expose the pipeline's `PipelinePhase`: ```text -idle → recording → transcribing → injecting → pasted | noTarget - │ │ - └── failed(BlurtError) / cancelled (from any stage) +idle → connecting → recording → transcribing → injecting → pasted | noTarget + │ │ + └── failed(BlurtError) / cancelled (from any stage) ``` +`connecting` is claimed the moment a press is accepted, _before_ the mic is up, so your UI can answer +the keypress instead of the hardware. It is **not** live capture: `MicCapture.start()` holds until the +input device actually delivers frames, which on a Bluetooth route takes ~1–2 s during which the OS +receives no audio at all. Present it as a warming-up state and **withhold your "speak now" cues** — +the recording indicator, the start chime — until `recording`, which follows only once audio is +genuinely being captured. Cueing at the press invites speech that nothing can capture. + - `phaseStream()` yields the current phase immediately, then every transition. It is a **multi-observer** stream: every call gets its own continuation and all of them see later transitions, so an extra consumer (a diagnostic, a second window) is safe. Still, prefer one renderer that projects the phase into your own state over a fan-out of long-lived consumers — one source of UI truth is easier to reason about than several. - `.pasted` and `.noTarget` are terminal _success_ states, not errors. `.noTarget` means transcription worked but nothing editable was focused (or the target app quit), so the text was left on the clipboard — show a quiet "copied" notice, not a failure. -- Two ready-made projections keep UI mapping out of your shell: `phase.overlayState` (`OverlayUIState`: idle / recording / processing / error(message:) / pasted / noTarget, with accessibility labels and — for the transient notices — `noticeDwellSeconds`, how long to hold one before reverting to idle) and `phase.menuBarStatus` (coarser: idle / recording / transcribing, never shows errors, with `symbolName`/`accessibilityLabel` presentation). +- Two ready-made projections keep UI mapping out of your shell: `phase.overlayState` (`OverlayUIState`: idle / connecting / recording / processing / error(message:) / pasted / noTarget, with accessibility labels and — for the transient notices — `noticeDwellSeconds`, how long to hold one before reverting to idle) and `phase.menuBarStatus` (coarser: idle / recording / transcribing, never shows errors, with `symbolName`/`accessibilityLabel` presentation — `connecting` rests at idle there rather than claiming a live mic). - Pill geometry is available too, if you're drawing something like Blurt's overlay: `OverlayPlacement` resolves how big the panel is (`panelSize(pillSize:shadowMargin:)`, sized to hold the pill plus room for its shadow) and where it goes (clearance, clamping a dragged origin back on screen), and `MeterBarGeometry` gives the level meter its shape. Build a `MeterBarRow(availableSize:)` once per layout — it resolves how many bars fit and how tall they may be — then ask it for `height(at:level:time:animated:)` per bar; `MeterBarGeometry.breathingOpacity(time:period:minOpacity:)` is the pulse the record dot and status label share. All pure math; pass `animated: false` to honor Reduce Motion. `OverlayOriginStore` persists the origin the user drags the pill to — it lives here, beside the clamping it feeds, because two keys private to the AppKit controller escaped every reset sweep. Both components must be present to read back: `double(forKey:)` reports 0 for a missing key, so a half-written pair reads as "never moved" rather than pinning the pill to an implied origin. - A "recent dictations" list has a model too. `RecentDictations` is an in-memory, newest-first ring of the last `capacity` (3) transcripts — never written to disk, empty at every launch — whose `record(_:at:)` takes an injected timestamp so tests are deterministic. `Entry.relativeLabel(now:locale:)` renders "just now" for the first `justNowThreshold` (60 s, published so a view can pick a refresh cadence against it) and the system's relative phrasing after. `reservedHeight(rowHeight:separatorThickness:)` is the `capacity` arithmetic — rows plus the `capacity - 1` separators between them — for a list that holds its height whether it shows 0 entries or 3. ### Record cues -`RecordingCueGate` is the other phase projection, and the reason the chimes don't retrigger: call `cue(for:)` with **every** phase and it returns `.start` only on the idle→recording edge, `.stop` only on the recording→not-recording edge, and `nil` while a phase repeats or when two non-recording phases follow each other. It's a value type holding one edge bit — keep a single instance for the host's lifetime. +`RecordingCueGate` is the other phase projection, and the reason the chimes don't retrigger: call `cue(for:)` with **every** phase and it returns `.start` only on the edge **into** `.recording` — in production the connecting→recording one, i.e. once the mic is actually delivering audio, never at the press — `.stop` only on the recording→not-recording edge, and `nil` while a phase repeats or when two non-recording phases follow each other. It's a value type holding one edge bit — keep a single instance for the host's lifetime. Which chime plays is a `SoundPack`: an `id`, a display `label`, and the picker `group` it belongs to. `SoundPack.catalog` is generated by `scripts/generate-sounds.swift` from Yamaha DX7 (ROM1A/ROM1B) and Roland Juno-106 factory presets; `groups` and `voices(in:)` build a sectioned picker off stored indexes rather than rescanning all 192 entries per render, and `fromPersisted(_:)` is the one decode-with-default rule (mirroring `TriggerKey.fromPersisted`) so a view reading the raw id can't disagree with `SoundPackStore`. `startFileName` / `stopFileName` give the `-start` / `-stop` stems, or `nil` for the silent `SoundPack.none`. @@ -139,16 +146,25 @@ All cases are `LocalizedError` with user-ready `errorDescription` strings, and ` ```swift func start() async throws func stop() async throws -> Data // raw S16LE mono PCM, 16 kHz, in order +func cancelCapture() async throws // stop and discard; default: stop-and-drop var levels: AsyncStream { get } // 0…1 meter; default: empty stream func warmUp() async // pre-open the device; default: no-op ``` -Only `start()`/`stop()` must be implemented — `levels` and `warmUp()` have defaults, so a stub or headless capture conforms for free while hosts still read the meter and warm the device through the same seam they inject. +Only `start()`/`stop()` must be implemented — `cancelCapture()`, `levels` and `warmUp()` have defaults, so a stub or headless capture conforms for free while hosts still read the meter and warm the device through the same seam they inject. + +`cancelCapture()` is what the session calls when a dictation is cancelled, and it exists because the two teardowns want opposite things: `stop()` may legitimately spend time preserving the audio, while a cancel has nothing to preserve and must take effect at once. Override it only if stopping cheaply differs from stopping carefully in your capture. `MicCapture` records with `AVAudioRecorder` straight to a temp 16 kHz / mono / 16-bit PCM WAV — exactly the geometry the dictation API wants — and reads it back as raw S16LE bytes on `stop()` (no float detour; the blob uploads as-is). A **fresh recorder per session** resolves the current default input device at `record()` time, which is why device switches (headset ↔ built-in) just work. Do **not** replace this with a long-lived `AVAudioEngine`/`installTap` graph: that design was tried, bound itself to one device, and failed with `-10868` or all-zero buffers on device switches. `MicCapture`'s `levels` is a ~20 Hz meter of the recorder's dBFS power mapped to `0…1` (floored at −50 dBFS so room ambient reads as silence) — feed it to a voice-bars view; it costs nothing when unobserved. Its `warmUp()` pre-creates and prepares a recorder so the first `start()` skips hardware route discovery (Blurt calls it at launch, once mic permission is granted, so warming never triggers the permission prompt). +**Bluetooth inputs get four accommodations**, because opening the mic on AirPods makes the system renegotiate the link into its mic-capable mode — one to two seconds, during which the OS receives no audio at all — and that link then buffers audio. + +The load-bearing one is that **`start()` doesn't return until the input is live.** `record()` returning `true` only means the AudioQueue started, so `start()` polls `recorder.currentTime` until it advances past 0 — the recorder's clock only moves once frames arrive, which distinguishes "route still switching" from "user is silent" (a level meter can't). The wait is capped per transport by `MicLiveness` (2.5 s Bluetooth, 300 ms otherwise) and **fails open** on timeout, so a broken or silent mic degrades to the old behavior rather than bricking the press. Speech during the switch is not recovered by this — nothing receives it — the point is to stop inviting it. + +The other three reduce how often that wait is paid, and fix the tail. `MicCapture` re-arms the warm recorder after _every_ capture rather than only at launch (the cost is per session, paid at `prepareToRecord()`); it records which device the warm recorder was built against and discards it when the default input has changed, since `AVAudioRecorder` resolves its device once and never re-resolves; and it releases an unused warm recorder after 60 s, because holding the input device open is what pins AirPods in the profile where output audio is degraded. On top of that, `stop()` keeps capturing for a further 220 ms when the input is Bluetooth, so speech still travelling over the link lands in the file instead of being truncated — the missing last word. `cancelCapture()` skips both that linger and the file read-back. + ### `TranscriberProtocol` → `AssemblyAITranscriber` ```swift diff --git a/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift b/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift index cd73fca9..ecf175a3 100644 --- a/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift +++ b/Sources/BlurtEngine/STT/AssemblyAITranscriber.swift @@ -284,14 +284,10 @@ private final class MetricsLogger: NSObject, URLSessionTaskDelegate, @unchecked } } -extension Duration { - /// This duration in milliseconds as a Double (for latency logging). Expressed - /// as a ratio of two `Duration`s rather than reassembled from `components`, - /// which meant restating the attoseconds-per-millisecond constant by hand. - fileprivate var milliseconds: Double { - self / Duration.milliseconds(1) - } -} +// `Duration.milliseconds` — the latency-logging conversion this file's request +// timing uses — moved to `Duration+Milliseconds.swift` when `MicCapture` needed +// the same thing for its liveness-gap line. It was `fileprivate` here; a second +// copy is an "invalid redeclaration", not a shadow. /// Errors specific to the AssemblyAI transport. These get wrapped in /// `BlurtError.sttFailed` before reaching the UI. diff --git a/Tests/BlurtEngineTests/DictationSessionBringUpTests.swift b/Tests/BlurtEngineTests/DictationSessionBringUpTests.swift new file mode 100644 index 00000000..23ea9118 --- /dev/null +++ b/Tests/BlurtEngineTests/DictationSessionBringUpTests.swift @@ -0,0 +1,114 @@ +import Foundation +import Testing + +@testable import BlurtEngine + +/// Behavior in the `.connecting` window — the up-to-2.5 s stretch +/// `MicCapture.start()` holds while a Bluetooth route brings the mic up. Before +/// the liveness gate, `start()` returned in microseconds and nothing could land +/// mid-press; now things can, so pin what happens when they do. +@Suite("DictationSession mic bring-up", .timeLimit(.minutes(1))) +struct DictationSessionBringUpTests { + @Test("a cancel landing during the bring-up never reaches .recording") + func cancelDuringBringUpSkipsRecording() async throws { + // The regression this guards: the press used to finish the bring-up, claim + // `.recording` and fire the start chime — cueing "speak now" for a capture + // the user had already cancelled — and only then get torn down by the queued + // cancel. `.recording` must never be published on this path, because + // `RecordingCueGate` chimes on exactly that edge. + let mic = GatedStartMic() + let session = DictationSession( + mic: mic, transcriber: StubTranscriber(mode: .transcript("never")), + injector: StubInjector(), seams: .offline) + + let stream = await session.phaseStream() + let pressed = Task { await session.press() } + await mic.waitUntilStartEntered() // press() is suspended inside mic.start() + // Direct `cancel()`, not `submit(.cancel)`: `submit`'s consumer is serial, so + // a submitted cancel cannot even be *recorded* until the press returns. See + // the note in `performPress` — this covers callers that can `await`. + let cancelled = Task { await session.cancel() } + await session.awaitCancelRequest() + await mic.allowStartToFinish() + await pressed.value + await cancelled.value + + var seen: [PipelinePhase] = [] + for await phase in stream { + seen.append(phase) + if phase.isTerminal, phase != .idle { break } + } + + #expect(seen == [.idle, .connecting, .cancelled]) + // Spelled out separately so a failure names the actual regression rather + // than just an unequal array. + #expect(!seen.contains(.recording)) + // The mic came up during the wait, so it has to have been torn down — and + // through the discarding teardown, not `stop()`. + #expect(await mic.cancelCaptureCalls == 1) + } + + @Test("a submitted cancel preempts the bring-up instead of queueing behind it") + func submittedCancelPreemptsBringUp() async throws { + // The app's cancel door is `submit`, and its consumer is serial — so a + // `.cancel` submitted during a press cannot reach `cancel()` until that press + // returns. With `MicCapture.start()` holding until the mic is live, that made + // Escape invisible for up to `MicLiveness.bluetoothTimeout`. `submit` now + // records the intent and cancels the in-flight press itself, without waiting + // for a turn on an actor the press is holding. + let mic = GatedStartMic() + let session = DictationSession( + mic: mic, transcriber: StubTranscriber(mode: .transcript("never")), + injector: StubInjector(), seams: .offline) + + let stream = await session.phaseStream() + session.submit(.press) + await mic.waitUntilStartEntered() // the consumer is now blocked inside this press + + session.submit(.cancel) + // Both effects have to be observable *before* the press is released, which is + // the whole point — neither needs the consumer to get a turn. + #expect(session.cancelRequested) + #expect(session.inFlightPress?.isCancelled == true) + + await mic.allowStartToFinish() + + var seen: [PipelinePhase] = [] + for await phase in stream { + seen.append(phase) + if phase.isTerminal, phase != .idle { break } + } + #expect(seen.last == .cancelled) + #expect(!seen.contains(.recording)) + #expect(await mic.cancelCaptureCalls == 1) + } + + @Test("a cancel that aborts the bring-up doesn't leak its request into the next press") + func abortedBringUpLeavesNoStandingCancel() async throws { + // The route where `start()` actually throws — a cancel landing *inside* the + // liveness wait, which is what the real `MicCapture` does. `cancel()`'s + // `.connecting` branch claims the phase and returns without enqueueing + // `performCancel`, so the press's catch is the only place left that can + // consume the request it recorded. When it didn't, the flag survived, and the + // *next* press read it after a perfectly good `mic.start()` and cancelled + // itself — one dead dictation for every cancelled bring-up. + let mic = GatedStartMic() + await mic.setThrowsIfCancelled(true) + let session = DictationSession( + mic: mic, transcriber: StubTranscriber(mode: .transcript("Hello world.")), + injector: StubInjector(), seams: .offline) + + let pressed = Task { await session.press() } + await mic.waitUntilStartEntered() + await session.cancel() + await mic.allowStartToFinish() + await pressed.value + + #expect(await session.phase == .cancelled) + #expect(!session.cancelRequested) + + // The proof that matters: the next press records normally. + await session.press() + #expect(await session.phase == .recording) + } +} diff --git a/Tests/BlurtEngineTests/DictationSessionTests.swift b/Tests/BlurtEngineTests/DictationSessionTests.swift index 29ffa690..dcb2f250 100644 --- a/Tests/BlurtEngineTests/DictationSessionTests.swift +++ b/Tests/BlurtEngineTests/DictationSessionTests.swift @@ -238,6 +238,32 @@ extension DictationSessionTests { #expect(await terminal == .pasted) } + @Test("press claims .connecting while the mic comes up, then .recording") + func pressPublishesConnectingBeforeRecording() async throws { + // The whole point of the phase: `mic.start()` now holds until the input + // route actually delivers frames (~1–2 s on a Bluetooth link), and the + // overlay needs something to show for that whole window — while the start + // chime deliberately waits for `.recording`. So `.connecting` must be + // *published*, not merely passed through: a `setPhase` skipped here leaves + // the pill absent for the entire bring-up, and the press looks ignored. + let fixture = makeSession() + + let stream = await fixture.session.phaseStream() + fixture.session.submit(.press) + + var seen: [PipelinePhase] = [] + for await phase in stream { + seen.append(phase) + if phase == .recording { break } + } + + // The subscription's initial yield is the current phase (.idle), then the + // press's two transitions in order. + #expect(seen == [.idle, .connecting, .recording]) + + await fixture.session.cancel() + } + @Test("cancel during active recording stops mic, discards audio, and transitions to .cancelled") func cancelDuringRecording() async throws { let fixture = makeSession(mode: .transcript("Hello")) @@ -250,10 +276,32 @@ extension DictationSessionTests { #expect(await fixture.mic.stopCalls == 1) #expect(await fixture.injector.inserted.isEmpty) } + + @Test("a cancel tears the mic down through cancelCapture, a release through stop") + func cancelUsesTheDiscardingTeardown() async throws { + // The two teardowns want opposite things, so the session must not conflate + // them. `stop()` may legitimately spend time preserving the audio — + // `MicCapture` waits out a Bluetooth link's tail before ending the + // recording — while a cancel has nothing to preserve and must take effect at + // once. Routing a cancel through `stop()` would make the user's cancel pay + // that linger to save audio it is about to delete. + let cancelled = makeSession() + await cancelled.session.press() + await cancelled.session.cancel() + #expect(await cancelled.mic.cancelCaptureCalls == 1) + + let released = makeSession() + await released.session.press() + await released.session.release() + await released.session.waitForIdle() + #expect(await released.mic.cancelCaptureCalls == 0) + #expect(await released.mic.stopCalls == 1) + } } // Guard/no-op behaviors and phase-stream supersession live in // `DictationSessionGuardTests.swift` (same collaborators and stubs), split out // to stay within the lint file-length budget. The `onTranscriptDelivered` -// side-channel tests live in `DictationSessionTranscriptTests.swift` for the +// side-channel tests live in `DictationSessionTranscriptTests.swift`, and the +// `.connecting` bring-up window in `DictationSessionBringUpTests.swift`, for the // same reason. diff --git a/Tests/BlurtEngineTests/HotkeyRaceTests.swift b/Tests/BlurtEngineTests/HotkeyRaceTests.swift index 2dc98775..b2e47e5e 100644 --- a/Tests/BlurtEngineTests/HotkeyRaceTests.swift +++ b/Tests/BlurtEngineTests/HotkeyRaceTests.swift @@ -15,7 +15,7 @@ struct HotkeyRaceTests { @Test("release during mic.start is honored, not dropped") func releaseDuringStartIsHonored() async throws { - let mic = GatedMicCapture() + let mic = GatedStartMic() let stt = StubTranscriber(mode: .transcript("hi")) let injector = StubInjector() let session = DictationSession( @@ -40,7 +40,7 @@ struct HotkeyRaceTests { @Test("cancel during mic.start is honored, not dropped") func cancelDuringStartIsHonored() async throws { - let mic = GatedMicCapture() + let mic = GatedStartMic() let stt = StubTranscriber(mode: .transcript("hi")) let injector = StubInjector() let session = DictationSession( @@ -69,7 +69,7 @@ struct HotkeyRaceTests { /// — the mic must never be started twice. @Test("a second press during mic.start is dropped, not double-started") func secondPressDuringStartIsDropped() async throws { - let mic = GatedMicCapture() + let mic = GatedStartMic() let stt = StubTranscriber(mode: .transcript("hi")) let injector = StubInjector() let session = DictationSession( @@ -98,7 +98,7 @@ struct HotkeyRaceTests { /// transcribe→inject run. @Test("cancel overrides a pending release during mic.start") func cancelOverridesPendingReleaseDuringStart() async throws { - let mic = GatedMicCapture() + let mic = GatedStartMic() let stt = StubTranscriber(mode: .transcript("hi")) let injector = StubInjector() let session = DictationSession( @@ -128,26 +128,3 @@ struct HotkeyRaceTests { #expect(await injector.inserted.isEmpty) } } - -/// Mic stub whose `start()` blocks until the test releases it, so `release()` can -/// be landed deterministically while `press()` is suspended inside `mic.start()`. -/// The entry/finish choreography lives in the shared `Gate`. -private actor GatedMicCapture: MicCaptureProtocol { - private(set) var startCalls = 0 - private(set) var stopCalls = 0 - private let gate = Gate() - - func start() async throws { - startCalls += 1 - await gate.enter() - } - - func waitUntilStartEntered() async { await gate.waitUntilEntered() } - func allowStartToFinish() async { gate.allowToFinish() } - - func stop() async throws -> Data { - stopCalls += 1 - // This suite exercises the press/release race, not the too-short guard. - return StubPCM.aboveMinimum - } -} diff --git a/Tests/BlurtEngineTests/MenuBarStatusTests.swift b/Tests/BlurtEngineTests/MenuBarStatusTests.swift index c66aa177..915c3e73 100644 --- a/Tests/BlurtEngineTests/MenuBarStatusTests.swift +++ b/Tests/BlurtEngineTests/MenuBarStatusTests.swift @@ -16,6 +16,9 @@ struct MenuBarStatusTests { (.recording, .recording), (.transcribing, .transcribing), (.idle, .idle), + // The mic is still opening, so the coarse indicator rests at idle rather + // than claiming "recording" — the one thing `.connecting` exists to prevent. + (.connecting, .idle), // Injection happens silently; the indicator rests at idle through the brief // paste rather than showing a distinct state. (.injecting, .idle), diff --git a/Tests/BlurtEngineTests/MicCaptureProtocolDefaultsTests.swift b/Tests/BlurtEngineTests/MicCaptureProtocolDefaultsTests.swift index e7a33c50..e9884469 100644 --- a/Tests/BlurtEngineTests/MicCaptureProtocolDefaultsTests.swift +++ b/Tests/BlurtEngineTests/MicCaptureProtocolDefaultsTests.swift @@ -1,18 +1,31 @@ import Foundation +import Synchronization import Testing @testable import BlurtEngine -/// The protocol's default meter and warm-up, which let a capture without either -/// (test stubs, headless hosts) conform with just `start()`/`stop()`. +/// The protocol's default meter, warm-up, and cancel teardown, which let a +/// capture without any of them (test stubs, headless hosts) conform with just +/// `start()`/`stop()`. @Suite("MicCaptureProtocol defaults") struct MicCaptureProtocolDefaultsTests { - /// Supplies only the two required capture calls, so `levels` and `warmUp()` - /// resolve to the protocol's defaults. - struct BareMic: MicCaptureProtocol { + /// Supplies only the two required capture calls, so `levels`, `warmUp()` and + /// `cancelCapture()` resolve to the protocol's defaults. + /// A `final class` over a `Mutex` rather than a struct (what this stub used to + /// be) so it can count calls: the protocol's methods are non-mutating, and + /// `Mutex` is non-copyable, so a struct can't hold one. Same shape as the test + /// support's `RecordedLog`. + final class BareMic: MicCaptureProtocol { + /// Counts the `stop()` calls the defaults route through, so the cancel + /// default can be observed. + let stops = Mutex(0) + func start() async throws {} - func stop() async throws -> Data { Data() } + func stop() async throws -> Data { + stops.withLock { $0 += 1 } + return Data() + } } @Test("default levels stream is empty and finishes immediately; warmUp is a no-op") @@ -27,4 +40,17 @@ struct MicCaptureProtocolDefaultsTests { for await _ in mic.levels { count += 1 } #expect(count == 0) } + + @Test("default cancelCapture stops and discards") + func cancelCaptureDefault() async throws { + // A capture with no cancel-specific teardown must still *end* on a cancel — + // the default is stop-and-discard, so a conformance that never heard of + // `cancelCapture` keeps the behavior it had when the session called `stop()` + // directly. (`MicCapture` overrides it to skip the tail linger and the + // read-back; that path needs real hardware, so it isn't covered here.) + let mic = BareMic() + try await mic.cancelCapture() + let stops = mic.stops.withLock { $0 } + #expect(stops == 1) + } } diff --git a/Tests/BlurtEngineTests/MicCaptureWarmTests.swift b/Tests/BlurtEngineTests/MicCaptureWarmTests.swift new file mode 100644 index 00000000..f63dbfea --- /dev/null +++ b/Tests/BlurtEngineTests/MicCaptureWarmTests.swift @@ -0,0 +1,193 @@ +import Foundation +import Testing + +@testable import BlurtEngine + +/// The warm-recorder lifecycle (`MicCapture+Warm`): prepare ahead of the press, +/// refuse to double-prepare, validate against the live input before reuse, and +/// tear down on a stale expiry ticket. +/// +/// Gated on BLURT_LIVE_AUDIO_TESTS=1, like `MicCaptureLevelsTests`, because every +/// test here goes through `MicCapture.makeRecorder()` — and that calls +/// `prepareToRecord()`, which is *the* route-activation call this whole change is +/// about. On a runner with no input device it is not merely unreliable, it is +/// hostile: it blocks the calling thread rather than suspending, so several of +/// these running concurrently occupy the cooperative pool and wedge the entire +/// `swift test` run, not just this suite. That is not a hypothetical — an +/// ungated first attempt failed three expectations here and then hung 171 +/// unrelated tests until the job's 30-minute timeout killed it. +/// +/// So this suite documents and locks the warm lifecycle for a human running it +/// on a real Mac with a real microphone; `MicCapture+Warm.swift` is excluded +/// from the coverage gate for the same reason `MicCapture.swift` is. +@Suite( + "MicCapture warm recorder (live)", + .enabled( + if: ProcessInfo.processInfo.environment["BLURT_LIVE_AUDIO_TESTS"] == "1", + "set BLURT_LIVE_AUDIO_TESTS=1 to run (needs a real microphone)"), + .tags(.liveAudio), + .timeLimit(.minutes(1))) +struct MicCaptureWarmTests { + private let builtIn = AudioRoute.InputSnapshot(deviceID: 7, transportType: nil) + private let airPods = AudioRoute.InputSnapshot(deviceID: 8, transportType: nil) + + @Test("warmUp prepares a recorder once; a second call has been overtaken and no-ops") + func warmUpPreparesOnce() async throws { + let mic = MicCapture() + #expect(await mic.canPrepareWarmRecorder) + + await mic.warmUp() + #expect(await mic.hasWarmRecorder) + // The slot is taken, so it is no longer safe to open the input for another. + #expect(await mic.canPrepareWarmRecorder == false) + + // A second warm-up — e.g. a scheduled re-warm that lost its race with a + // launch-time warmUp — must not stack a second open recorder onto the input. + let generation = await mic.preparedGeneration + await mic.warmUp() + #expect(await mic.preparedGeneration == generation) + + await mic.discardWarmRecorder() + } + + @Test("warmUp is refused across the bring-up window, when both recorder slots are nil") + func warmUpRefusedDuringBringUp() async throws { + // The regression `bringingUpCapture` exists for: across `start()`'s liveness + // wait both `activeRecorder` and `warm` are nil, so a guard reading only + // those would call a live capture "idle" and prepare a second recorder onto + // the already-open input. + let mic = MicCapture() + await mic.setBringingUpCapture(true) + await mic.warmUp() + #expect(await mic.hasWarmRecorder == false) + + await mic.setBringingUpCapture(false) + await mic.warmUp() + #expect(await mic.hasWarmRecorder) + + await mic.discardWarmRecorder() + } + + @Test("a warm recorder is reused only while still bound to the live default input") + func warmRecorderReusedWhenDeviceUnchanged() async throws { + let mic = MicCapture() + try await mic.installWarmRecorder(boundTo: builtIn) + + #expect(await mic.takeWarm(matching: builtIn)) + // Consumed either way — the take empties the slot, so the next press can't + // double-dip and the re-warm guard sees it as free again. + #expect(await mic.hasWarmRecorder == false) + #expect(await mic.canPrepareWarmRecorder) + + // Nothing held: the take answers nil rather than conjuring a recorder. + #expect(await mic.takeWarm(matching: builtIn) == false) + } + + @Test("a device change — or an unreadable route on either side — discards the warm recorder") + func warmRecorderDiscardedOnDeviceChangeOrUnknown() async throws { + // Reuse requires *positively* confirming the device is unchanged. A recorder + // bound to the wrong device doesn't fail loudly — it records the wrong mic, + // or silence — so unknown means discard: paying route activation again is + // the cheaper mistake. + let mic = MicCapture() + + // The user connected their AirPods after the warm-up. + try await mic.installWarmRecorder(boundTo: builtIn) + #expect(await mic.takeWarm(matching: airPods) == false) + #expect(await mic.hasWarmRecorder == false) + + // The route was unreadable when the recorder was warmed. + try await mic.installWarmRecorder(boundTo: nil) + #expect(await mic.takeWarm(matching: builtIn) == false) + + // The route is unreadable now, at press time. + try await mic.installWarmRecorder(boundTo: builtIn) + #expect(await mic.takeWarm(matching: nil) == false) + } + + @Test("an expiry with a stale generation ticket leaves a later warm recorder alone") + func staleExpiryTicketDoesNothing() async throws { + // Cancellation alone doesn't cover this: an expiry already past its + // cancellation check still gets its actor turn, and without the ticket it + // would tear down a recorder prepared a moment ago. + let mic = MicCapture() + try await mic.installWarmRecorder(boundTo: builtIn) + let generation = await mic.preparedGeneration + + await mic.releasePreparedRecorder(generation: generation - 1) + #expect(await mic.hasWarmRecorder) + + // The live ticket is what tears the idle recorder down, freeing the input. + await mic.releasePreparedRecorder(generation: generation) + #expect(await mic.hasWarmRecorder == false) + } + + @Test("a scheduled re-warm eventually prepares a recorder on its own turn") + func scheduledRewarmPreparesARecorder() async throws { + // `stop()`/`cancelCapture()` schedule rather than prepare inline because + // preparing re-opens the input — the slow part — and both sit on paths the + // user is waiting behind. All this can pin deterministically is the other + // half of that contract: the scheduled task does land, and prepares. + // Polled on a deadline with a real sleep between reads, NOT a bare + // `Task.yield()` spin. `scheduleRewarm` hands the work to a child task that + // needs a cooperative thread to run on, and `warmUp()` then blocks that + // thread inside CoreAudio — so a hot spin here competes with the very task + // it is waiting for, and on a machine where the recorder never materialises + // it never terminates at all. Sleeping yields the thread outright, and the + // deadline turns "never landed" into a failed expectation rather than a hang + // the suite's time limit has to clean up. + let mic = MicCapture() + let before = await mic.preparedGeneration + await mic.scheduleRewarm() + let deadline = ContinuousClock().now.advanced(by: .seconds(5)) + while await mic.preparedGeneration == before, ContinuousClock().now < deadline { + try await Task.sleep(for: .milliseconds(10)) + } + #expect(await mic.hasWarmRecorder) + + await mic.discardWarmRecorder() + } +} + +/// Actor-isolated test seams over `MicCapture`'s internal warm-recorder state. +/// Extensions because `AVAudioRecorder` is not `Sendable`, so neither the `warm` +/// slot nor `takeWarmRecorder`'s return can cross the actor boundary into a +/// test — each helper reduces it to a `Sendable` answer on the actor instead. +extension MicCapture { + /// Whether a warm recorder is currently held. + var hasWarmRecorder: Bool { warm != nil } + + /// Opens or closes the bring-up window `canPrepareWarmRecorder` guards on, + /// standing in for a `start()` suspended in its liveness wait (which needs + /// real hardware to reach). + func setBringingUpCapture(_ value: Bool) { + bringingUpCapture = value + } + + /// Installs a warm recorder bound to a *known* input — `prepareWarmRecorder` + /// with the `AudioRoute.currentInput()` read replaced by `input`, so the + /// device-identity tests don't depend on what the test machine's routing + /// happens to answer. + func installWarmRecorder(boundTo input: AudioRoute.InputSnapshot?) throws { + preparedGeneration += 1 + warm = WarmRecorder( + recorder: try Self.makeRecorder(), input: input, generation: preparedGeneration) + } + + /// Consumes the warm slot through the production validation, reporting whether + /// the recorder was reusable. A reused recorder's temp file is cleaned up here, + /// the job `start()` would otherwise inherit; the discard path already does so. + func takeWarm(matching input: AudioRoute.InputSnapshot?) -> Bool { + guard let recorder = takeWarmRecorder(matching: input) else { return false } + Self.removeFile(at: recorder.url) + return true + } + + /// Test teardown: drops the warm recorder (and its expiry countdown) so a + /// finished test doesn't leave a 60 s expiry task holding the suite's actor. + /// Through `takeWarmRecorder` — which cancels the expiry — rather than a bare + /// `warm = nil`, so teardown can't diverge from how production empties the slot. + func discardWarmRecorder() { + _ = takeWarmRecorder(matching: nil) + } +} diff --git a/Tests/BlurtEngineTests/MicLivenessTests.swift b/Tests/BlurtEngineTests/MicLivenessTests.swift new file mode 100644 index 00000000..4c0f7c95 --- /dev/null +++ b/Tests/BlurtEngineTests/MicLivenessTests.swift @@ -0,0 +1,161 @@ +import CoreAudio +import Foundation +import Synchronization +import Testing + +@testable import BlurtEngine + +/// The pure half of `MicCapture.start()`'s liveness gate: the transport-aware +/// wait cap and the poll-until-the-recorder's-clock-advances loop. Driven +/// against `TestClock` so the Bluetooth cap is exercised without waiting real +/// seconds. +@Suite("MicLiveness", .timeLimit(.minutes(1))) +struct MicLivenessTests { + @Test("Bluetooth transports get the long cap; everything else the short one") + func transportTimeouts() { + // The profile switch is the whole reason the gate exists — both Bluetooth + // transport types must get the multi-second budget. + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBluetooth) + == MicLiveness.bluetoothTimeout) + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBluetoothLE) + == MicLiveness.bluetoothTimeout) + // Wired/built-in inputs deliver frames near-instantly; a long cap there + // would make a genuinely broken mic feel like a hang. + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeBuiltIn) + == MicLiveness.defaultTimeout) + #expect( + MicLiveness.timeout(forTransportType: kAudioDeviceTransportTypeUSB) + == MicLiveness.defaultTimeout) + // An unreadable transport must not be treated as Bluetooth. + #expect(MicLiveness.timeout(forTransportType: nil) == MicLiveness.defaultTimeout) + } + + @Test("already-advancing recorder clock confirms immediately, without sleeping") + func immediateLiveness() async { + let clock = TestClock() + // Never advanced: a sleep would park forever, so returning at all proves + // the fast path never sleeps (the suite's time limit backs that up). + let gap = await MicLiveness.waitUntilLive(timeout: .seconds(1), clock: clock) { 0.1 } + #expect(gap == .zero) + } + + @Test("the poll interval doubles, so a slow bring-up isn't hundreds of wakeups") + func livenessBacksOff() async { + let clock = TestClock() + let polls = Mutex(0) + async let gap = MicLiveness.waitUntilLive(timeout: MicLiveness.bluetoothTimeout, clock: clock) { + // Stuck at 0 for the first three checks — the route still switching — then + // the recorder clock starts moving. + polls.withLock { polls in + polls += 1 + return polls < 4 ? 0 : 0.05 + } + } + // Each wait is twice the last: 1 ms, 2 ms, 4 ms. Driving the clock by the + // exact expected delay is also the assertion — `waitUntilSleeping` matches on + // the deadline, so a loop that slept a fixed quantum would hang here rather + // than quietly pass. + var expected = MicLiveness.initialPollInterval + var elapsed = Duration.zero + for _ in 1...3 { + await clock.waitUntilSleeping(for: expected) + clock.advance(by: expected) + elapsed += expected + expected = min(expected * 2, MicLiveness.maxPollInterval) + } + #expect(await gap == elapsed) + } + + @Test("the backoff is capped rather than doubling without limit") + func backoffIsCapped() async { + // Otherwise a 2.5 s Bluetooth cap would end in multi-second sleeps and + // overshoot the deadline it is supposed to respect. + let clock = TestClock() + let polls = Mutex(0) + async let gap = MicLiveness.waitUntilLive(timeout: .seconds(30), clock: clock) { + polls.withLock { polls in + polls += 1 + return polls < 12 ? 0 : 0.05 + } + } + var expected = MicLiveness.initialPollInterval + for _ in 1...11 { + await clock.waitUntilSleeping(for: expected) + clock.advance(by: expected) + expected = min(expected * 2, MicLiveness.maxPollInterval) + } + #expect(expected == MicLiveness.maxPollInterval) + #expect(await gap != nil) + } + + @Test("a clock that never advances times out with nil — the fail-open signal") + func timeoutFailsOpen() async { + // nil is what tells `MicCapture` to proceed anyway: a silent or broken mic + // must degrade to the pre-gate behavior, never brick the press. + let clock = TestClock() + let timeout = MicLiveness.initialPollInterval * 3 + async let gap = MicLiveness.waitUntilLive(timeout: timeout, clock: clock) { 0 } + var expected = MicLiveness.initialPollInterval + for _ in 1...2 { + await clock.waitUntilSleeping(for: expected) + clock.advance(by: expected) + expected = min(expected * 2, MicLiveness.maxPollInterval) + } + #expect(await gap == nil) + } +} + +/// The transport classification both the liveness cap and `MicCapture`'s tail +/// linger hang off. Pure and pinned here because `AudioRoute`, which reads the +/// raw value, needs real hardware and is excluded from the coverage gate — so +/// this is the only place the decision can be tested. +@Suite("AudioTransport") +struct AudioTransportTests { + @Test("both Bluetooth transport types count") + func bluetoothTransports() { + #expect(AudioTransport.isBluetooth(kAudioDeviceTransportTypeBluetooth)) + #expect(AudioTransport.isBluetooth(kAudioDeviceTransportTypeBluetoothLE)) + } + + @Test("the tail linger is Bluetooth-only") + func tailLinger() { + // The other transport-conditional policy, here rather than in `MicCapture` + // so it is reachable by `swift test` at all — the capture actor needs real + // hardware and is excluded from the coverage gate. + #expect( + AudioTransport.tailLinger(forTransportType: kAudioDeviceTransportTypeBluetooth) + == AudioTransport.bluetoothTailLinger) + // `.zero`, not a small duration: `stop()` skips the sleep entirely on a + // wired input rather than awaiting a nominal one. + #expect(AudioTransport.tailLinger(forTransportType: kAudioDeviceTransportTypeBuiltIn) == .zero) + #expect(AudioTransport.tailLinger(forTransportType: nil) == .zero) + } + + @Test("wired, built-in, and unreadable transports do not") + func nonBluetoothTransports() { + #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeBuiltIn)) + #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeUSB)) + #expect(!AudioTransport.isBluetooth(kAudioDeviceTransportTypeAggregate)) + // nil is the conservative answer for both consumers: the short wait cap and + // no tail linger. Padding every wired capture with a delay would be a worse + // regression than losing the tail on a device we couldn't classify. + #expect(!AudioTransport.isBluetooth(nil)) + } +} + +/// `Duration.milliseconds` backs the latency lines `MicCapture` logs for the +/// liveness gap — the field evidence for whether the gate is doing anything — +/// so a silently wrong conversion would make those logs lie. +@Suite("Duration.milliseconds") +struct DurationMillisecondsTests { + @Test func convertsWholeAndFractionalDurations() { + #expect(Duration.milliseconds(250).milliseconds == 250) + #expect(Duration.seconds(2).milliseconds == 2000) + #expect(Duration.zero.milliseconds == 0) + // Sub-millisecond durations keep their fraction rather than truncating to 0. + #expect(Duration.microseconds(500).milliseconds == 0.5) + } +} diff --git a/Tests/BlurtEngineTests/OverlayUIStateTests.swift b/Tests/BlurtEngineTests/OverlayUIStateTests.swift index f3ffb5b3..7c9f3e43 100644 --- a/Tests/BlurtEngineTests/OverlayUIStateTests.swift +++ b/Tests/BlurtEngineTests/OverlayUIStateTests.swift @@ -15,6 +15,10 @@ struct OverlayUIStateTests { /// carve-out — stay as their own tests below. static let projections: [(phase: PipelinePhase, expected: OverlayUIState)] = [ (.idle, .idle), + // Its own pill state, not `.recording`: `.connecting` exists because capture + // hasn't begun, so projecting it onto the recording pill would show the + // `● REC` tag and a live meter over a mic that isn't open yet. + (.connecting, .connecting), (.recording, .recording), (.transcribing, .processing), // `.injecting` is a *working* phase, so it must not project to `.idle`: the @@ -71,7 +75,9 @@ struct OverlayUIStateTests { // A genuine failure is not a setup state. #expect(PipelinePhase.failed(.targetAppLost).setupBlocker == nil) // Neither is any non-failed phase. - for phase in [PipelinePhase.idle, .recording, .transcribing, .injecting, .pasted, .noTarget] { + for phase in [ + PipelinePhase.idle, .connecting, .recording, .transcribing, .injecting, .pasted, .noTarget, + ] { #expect(phase.setupBlocker == nil) } // And the pill projection agrees with the classification. @@ -98,6 +104,7 @@ struct OverlayUIStateAccessibilityLabelTests { /// (echo the carried message verbatim), not a constant, so it keeps its own test. static let labels: [(state: OverlayUIState, spoken: String)] = [ (.idle, "Blurt."), + (.connecting, "Connecting to the microphone."), (.recording, "Recording."), (.processing, "Processing."), (.pasted, "Your dictation was pasted."), @@ -137,6 +144,9 @@ struct OverlayUIStateNoticeDwellTests { @Test func steadyStatesHaveNoDwell() { // Held for as long as the pipeline is in them — no auto-revert. #expect(OverlayUIState.idle.noticeDwellSeconds == nil) + // `.connecting` in particular: a dwell would auto-revert the pill to idle + // mid-press, dismissing it while the mic was still opening. + #expect(OverlayUIState.connecting.noticeDwellSeconds == nil) #expect(OverlayUIState.recording.noticeDwellSeconds == nil) #expect(OverlayUIState.processing.noticeDwellSeconds == nil) } diff --git a/Tests/BlurtEngineTests/PipelinePhaseTests.swift b/Tests/BlurtEngineTests/PipelinePhaseTests.swift index be7373de..ff9d211a 100644 --- a/Tests/BlurtEngineTests/PipelinePhaseTests.swift +++ b/Tests/BlurtEngineTests/PipelinePhaseTests.swift @@ -29,6 +29,11 @@ struct PipelinePhaseTests { @Test("active phases are not terminal") func activePhasesAreNotTerminal() { + // `.connecting` included: the press guard keys off terminality, so a + // terminal `.connecting` would let a second key-down start a second capture + // while the first is still bringing the mic up — a window that lasts ~1–2 s + // on a Bluetooth route, so it is very reachable. + #expect(!PipelinePhase.connecting.isTerminal) #expect(!PipelinePhase.recording.isTerminal) #expect(!PipelinePhase.transcribing.isTerminal) #expect(!PipelinePhase.injecting.isTerminal) diff --git a/Tests/BlurtEngineTests/RecordingCueGateTests.swift b/Tests/BlurtEngineTests/RecordingCueGateTests.swift index 8efdc0d0..b5644d15 100644 --- a/Tests/BlurtEngineTests/RecordingCueGateTests.swift +++ b/Tests/BlurtEngineTests/RecordingCueGateTests.swift @@ -2,22 +2,49 @@ import Testing @testable import BlurtEngine -/// The record start/stop chimes fire on the *edges* of the recording phase, not -/// on every phase tick. `AppCoordinator.render` calls the cue gate on every -/// pipeline phase (idle, recording, transcribing, injecting, pasted, …), so the -/// gate must fire `.start` only on the idle→recording edge and `.stop` only on -/// the recording→not-recording edge, staying silent on repeats and on +/// The record start/stop chimes fire on the *edges* of `.recording`, not on +/// every phase tick. `AppCoordinator.render` calls the cue gate on every +/// pipeline phase (idle, connecting, recording, transcribing, injecting, pasted, +/// …), so the gate must fire `.start` only on the edge into `.recording` and +/// `.stop` only on the edge out of it, staying silent on repeats and on /// transitions between two non-recording phases. Lifting that edge detection out /// of the AppKit `CueSoundPlayer` lets `swift test` cover it — the same split as /// `OverlayUIState`/`MenuBarStatus`. @Suite("RecordingCueGate") struct RecordingCueGateTests { - @Test("entering recording from idle fires the start cue") + @Test("entering recording fires the start cue") func startOnRisingEdge() { var gate = RecordingCueGate() #expect(gate.cue(for: .recording) == .start) } + @Test("the mic bring-up is silent — the chime waits for audio to actually flow") + func connectingIsSilent() { + // The load-bearing case. `.connecting` is the window where `MicCapture`'s + // liveness gate is waiting for the input route, which on a Bluetooth link is + // ~1–2 s during which the OS captures nothing. A chime here is a "speak now" + // cue for a dead mic, and the first words of the utterance are lost — the + // exact bug this ordering exists to prevent. + var gate = RecordingCueGate() + #expect(gate.cue(for: .connecting) == nil) + // It fires on the connecting→recording edge instead. + #expect(gate.cue(for: .recording) == .start) + } + + @Test("a press whose mic never comes up never chimes") + func failedBringUpIsSilent() { + // `mic.start()` throwing takes the pipeline `.connecting` → `.failed` + // without ever reaching `.recording`. No start cue was played, so no stop + // cue is owed either — and the gate must be left unlatched for the next + // press rather than thinking a capture is still running. + var gate = RecordingCueGate() + #expect(gate.cue(for: .connecting) == nil) + let failure = PipelinePhase.failed(.audioCaptureFailed(underlying: MicCaptureError.noInputDevice)) + #expect(gate.cue(for: failure) == nil) + #expect(gate.cue(for: .connecting) == nil) + #expect(gate.cue(for: .recording) == .start) + } + @Test("leaving recording fires the stop cue") func stopOnFallingEdge() { var gate = RecordingCueGate() @@ -44,12 +71,14 @@ struct RecordingCueGateTests { #expect(gate.cue(for: .failed(.apiKeyMissing)) == nil) } - @Test("a full record→stop→record cycle chimes start, stop, start again") + @Test("a full press→record→stop→press cycle chimes start, stop, start again") func fullCycle() { var gate = RecordingCueGate() + #expect(gate.cue(for: .connecting) == nil) #expect(gate.cue(for: .recording) == .start) #expect(gate.cue(for: .injecting) == .stop) #expect(gate.cue(for: .idle) == nil) + #expect(gate.cue(for: .connecting) == nil) #expect(gate.cue(for: .recording) == .start) } } diff --git a/Tests/BlurtEngineTests/Stubs/GatedStartMic.swift b/Tests/BlurtEngineTests/Stubs/GatedStartMic.swift new file mode 100644 index 00000000..4a5c7c42 --- /dev/null +++ b/Tests/BlurtEngineTests/Stubs/GatedStartMic.swift @@ -0,0 +1,50 @@ +import Foundation + +@testable import BlurtEngine + +/// Mic stub whose `start()` blocks until the test releases it, so a command can +/// be landed while the session sits in `.connecting`. +/// +/// That window is not hypothetical: `MicCapture.start()` holds until the input +/// route delivers frames, which on a Bluetooth route is up to +/// `MicLiveness.bluetoothTimeout`. Before the gate existed, `start()` returned in +/// microseconds and nothing could arrive during a press. +/// +/// The entry/finish choreography lives in the shared `Gate`; `GatedStopMic` is +/// the mirror image for the release path. `HotkeyRaceTests` drives its +/// press/release races through this one too. +actor GatedStartMic: MicCaptureProtocol { + private(set) var startCalls = 0 + private(set) var stopCalls = 0 + private(set) var cancelCaptureCalls = 0 + private let gate = Gate() + /// When set, `start()` throws `CancellationError` if the task was cancelled + /// while it was gated — what the real `MicCapture` does when a cancel lands + /// during its liveness wait. Off by default so the common gated-press tests + /// see a bring-up that simply completes. + private var throwsIfCancelled = false + + func setThrowsIfCancelled(_ value: Bool) { throwsIfCancelled = value } + + func start() async throws { + startCalls += 1 + await gate.enter() + if throwsIfCancelled, Task.isCancelled { throw CancellationError() } + } + + func waitUntilStartEntered() async { await gate.waitUntilEntered() } + func allowStartToFinish() { gate.allowToFinish() } + + func stop() async throws -> Data { + stopCalls += 1 + return StubPCM.aboveMinimum + } + + /// Counts the call and delegates, so `stopCalls` keeps meaning "the mic was + /// stopped" however the teardown was reached — the same shape as + /// `StubMicCapture`. + func cancelCapture() async throws { + cancelCaptureCalls += 1 + _ = try await stop() + } +} diff --git a/Tests/BlurtEngineTests/Stubs/StubMicCapture.swift b/Tests/BlurtEngineTests/Stubs/StubMicCapture.swift index d22466a6..bec698a8 100644 --- a/Tests/BlurtEngineTests/Stubs/StubMicCapture.swift +++ b/Tests/BlurtEngineTests/Stubs/StubMicCapture.swift @@ -5,6 +5,7 @@ import Foundation actor StubMicCapture: MicCaptureProtocol { var startCalls = 0 var stopCalls = 0 + var cancelCaptureCalls = 0 var pcmToReturn = StubPCM.aboveMinimum var startError: (any Error & Sendable)? var stopError: (any Error & Sendable)? @@ -20,6 +21,15 @@ actor StubMicCapture: MicCaptureProtocol { if let stopError { throw stopError } return pcmToReturn } + /// Overrides the protocol's stop-and-discard default only to *count* the call, + /// then delegates to `stop()` so `stopCalls` and `stopError` keep meaning what + /// they did before the cancel path had its own entry point — the suites that + /// assert a cancel stopped the mic (and that a failing stop is logged) are + /// unchanged. `MicCaptureProtocolDefaultsTests` covers the bare default. + func cancelCapture() async throws { + cancelCaptureCalls += 1 + _ = try await stop() + } func setPCM(_ pcm: Data) { pcmToReturn = pcm } func setStartError(_ error: (any Error & Sendable)?) { startError = error } func setStopError(_ error: (any Error & Sendable)?) { stopError = error } diff --git a/scripts/check.sh b/scripts/check.sh index 9a459db9..a4575b55 100755 --- a/scripts/check.sh +++ b/scripts/check.sh @@ -588,8 +588,30 @@ else # MicCapture+Meter.swift, which IS covered. Keep this # list tight — exclude only code that genuinely cannot # be exercised without hardware. + # - MicCapture+Warm.swift : the same actor's warm-recorder lifecycle, split + # out of MicCapture.swift only for the lint file-length + # budget. Every path through it runs makeRecorder(), + # i.e. prepareToRecord() — the route-activation call, the + # one thing here that genuinely needs a device. Unlike + # +Meter (pure math, covered), splitting this out moved + # hardware-bound code onto the counted side by accident: + # the pattern above pins a literal filename. Trying to + # cover it in CI didn't merely fail, it deadlocked the + # whole test run — prepareToRecord blocks its thread + # instead of suspending, so concurrent attempts drained + # the cooperative pool. Its suite is env-gated alongside + # MicCaptureLevelsTests. The transport and liveness + # *policy* it consults stays covered, in AudioTransport + # and MicLiveness. + # - AudioRoute(Monitor).swift : the CoreAudio routing reads (AudioRoute) and the + # property listeners (AudioRouteMonitor). Both answer + # questions only real hardware can answer — which device + # is default, whether its transport is Bluetooth, and + # when the user switches output — and the listener half + # can only fire on an actual route change. Same + # justification as MicCapture.swift above. COVERAGE="$(xcrun llvm-cov export -summary-only -instr-profile "$PROFDATA" "$XCTEST_BIN" \ - -ignore-filename-regex='Tests/|Audio/MicCapture\.swift' \ + -ignore-filename-regex='Tests/|Audio/MicCapture(\+Warm)?\.swift|Audio/AudioRoute(Monitor)?\.swift' \ | python3 -c 'import sys,json; print(round(json.load(sys.stdin)["data"][0]["totals"]["lines"]["percent"],2))')" echo "engine line coverage: ${COVERAGE}%" if ! awk -v c="$COVERAGE" -v min="$MIN_COVERAGE" 'BEGIN{ exit (c+0 < min+0) }'; then