Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
bc512f2
Cut AirPods dictation lag at both ends of an utterance
claude Aug 13, 2026
b086e9c
Satisfy swift-format in the new CoreAudio files
claude Aug 13, 2026
bdc7957
Drop the unused Foundation imports from the CoreAudio files
claude Aug 13, 2026
a0b6428
Gate recording on mic liveness, and stop chiming at the press
claude Aug 13, 2026
6d25ada
Remove the duplicate Duration.milliseconds extension
claude Aug 13, 2026
978ebb7
Merge origin/main into the AirPods dictation-lag branch
claude Aug 13, 2026
8039fd1
Fix five review findings from the liveness-gate work
claude Aug 13, 2026
e96fc10
Let a cancel preempt the mic bring-up instead of queueing behind it
claude Aug 13, 2026
7077aeb
Merge origin/main (engine guide moved into the package)
claude Aug 13, 2026
7382f09
Cleanup pass, plus a leaked cancel request the pass surfaced
claude Aug 13, 2026
a61c975
Take the five deferred efficiency and altitude findings
claude Aug 13, 2026
606aa38
Drop the generic CoreAudio read helper — it doesn't compile
claude Aug 13, 2026
f1aff9d
Fix three swiftlint violations from the last two commits
claude Aug 13, 2026
35be0db
Merge origin/main, taking its de-duplicated invariants section
claude Aug 13, 2026
59b5a18
Drop the now-unused os import from DictationSession.swift
claude Aug 13, 2026
0f86058
Add engine tests covering the new warm-recorder lifecycle
claude Aug 13, 2026
71247f0
Merge remote-tracking branch 'origin/main' into claude/airpods-lag-di…
claude Aug 13, 2026
81a3d81
Merge remote-tracking branch 'origin/test/airpods-coverage-lift' into…
claude Aug 13, 2026
d378778
Env-gate the warm-recorder tests; exclude MicCapture+Warm from the co…
claude Aug 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/project-guardrails/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ genuinely correct, and reaching for it means it's time to stop and ask.
generated from `project.yml`; edit that and run `xcodegen generate`. check.sh
fails on pbxproj drift (a PreToolUse hook also blocks edits to it).
- The engine has **no external SPM dependencies** (Foundation/Security/
AVFoundation only). Don't add one to `Sources/BlurtEngine/`.
AVFoundation/CoreAudio only). Don't add one to `Sources/BlurtEngine/`.
- Unit tests use **Swift Testing**, not XCTest (the `BlurtUITests` XCUITest
bundle is the one exception — XCUIAutomation requires XCTest). **Never touch
the real Keychain in tests** — `APIKeyStore` is the production item; use an
Expand Down
114 changes: 100 additions & 14 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,10 @@ Four reflexes before you touch anything:

### The two layers

| Layer | What it is |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Sources/BlurtEngine/` | Swift package (`swift-tools-version:6.2`, `platforms: [.macOS(.v15)]`) owning the pipeline. Pure logic behind protocol seams, no AppKit-shell deps, **no external SPM deps** — Foundation/Security/AVFoundation plus toolchain modules like Synchronization, with AppKit types only at the seams. |
| `App/Blurt/` | AppKit/SwiftUI shell (Xcode project generated by XcodeGen) that wires the engine to an overlay window, the main window, a Settings scene, a menu bar item, and the trigger key. Its only package is the local `BlurtEngine` (declared in `App/Blurt/project.yml`). |
| Layer | What it is |
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Sources/BlurtEngine/` | Swift package (`swift-tools-version:6.2`, `platforms: [.macOS(.v15)]`) owning the pipeline. Pure logic behind protocol seams, no AppKit-shell deps, **no external SPM deps** — Foundation/Security/AVFoundation/CoreAudio plus toolchain modules like Synchronization, with AppKit types only at the seams. |
| `App/Blurt/` | AppKit/SwiftUI shell (Xcode project generated by XcodeGen) that wires the engine to an overlay window, the main window, a Settings scene, a menu bar item, and the trigger key. Its only package is the local `BlurtEngine` (declared in `App/Blurt/project.yml`). |

The engine's dependency-free rule is a **rule**; the app merely happens to carry none today. The
former `mxcl/AppUpdater` dependency and its in-place self-updater were removed (see [Updates](#updates)),
Expand All @@ -51,7 +51,8 @@ workflow and the _why_ behind the design; the engine's README covers the _what_
```text
Sources/BlurtEngine/ the engine (dependency-free Swift package)
README.md the engine's developer guide (quick start, seams, error table)
Audio/ MicCapture (+meter), SoundPack/Catalog/Store — record cues
Audio/ MicCapture (+meter/+warm), MicLiveness (mic bring-up gate), AudioRoute
(+Monitor)/AudioTransport — CoreAudio routing, SoundPack/Catalog/Store
Config/ Keychain-backed API key, key terms, developer mode, DefaultsKey +
PersistedSettings (every defaults key, and the reset sweep over them)
FocusCapture/ Accessibility reads of the frontmost app / focused field
Expand Down Expand Up @@ -426,13 +427,80 @@ A **fresh recorder per session** resolves the current default input device at `r
is deliberate — see [Settled decisions](#settled-decisions--dont-reintroduce-these) for the
`AVAudioEngine` failure it replaced.

**Bluetooth inputs are the reason for four of this actor's moving parts.** Opening the mic on
AirPods (or any Bluetooth headset) makes the system renegotiate the link into its mic-capable mode —
one to two seconds, during which the OS receives no audio at all — and that link then buffers audio
in both directions. So:

- **`start()` does not return until the input is live.** `record()` returning `true` only means the
AudioQueue started, not that frames are arriving, so `start()` polls `recorder.currentTime` until
it advances past 0 — the recorder's clock only moves once the device delivers audio, which is what
distinguishes "route still switching" from "user is silent" (a level meter can't). Capped per
transport by **`MicLiveness`** (2.5 s Bluetooth, 300 ms otherwise) and **failing open** on timeout,
so a broken or silent mic degrades to the old behavior rather than bricking the press. This is what
stops the app cueing the user to speak into a dead mic; audio spoken during the switch cannot be
recovered by anything, because nothing ever receives it. `stopGeneration` covers the one suspension
this introduces — a teardown landing mid-wait wins, and the recorder is torn down rather than
installed. `bringingUpCapture` covers the other consequence: across the wait both `activeRecorder`
and `warm` are nil, so the warm-up paths can't infer "no capture in flight" from them (see
`canPrepareWarmRecorder`) or they'd open a second recorder onto the live input.

**A cancel preempts the bring-up rather than queueing behind it**, which took two pieces. The
press publishes its task handle (`inFlightPress`) exactly as the pipeline publishes
`pipelineTask`, and `cancel()` treats `.connecting` like the other in-flight phases: cancel the
handle, claim `.cancelled`, return. `waitUntilLive` already honors task cancellation, so the wait
unblocks at once and `start()` throws `CancellationError` rather than installing the recorder.
The cancel flag then lives in a `Mutex` beside that handle rather than in actor state, so
**`submit(.cancel)` can record and preempt without waiting for a turn** — the app's cancel door is
`submit`, and its consumer is serial, so before this a submitted `.cancel` wasn't even _recorded_
until the press it meant to cancel had finished. `requestCancel()` is the single place both doors
funnel through. Commands are still yielded and executed in order; only the preemption is new.
`performPress` still consumes the flag before claiming `.recording`, for the narrow window where
the cancel lands after the wait returned and there is nothing left to interrupt.

- **The warm recorder is re-armed after every capture**, not just at launch. The cost above is paid
at `prepareToRecord()`, i.e. per session, so warming only the first one hid it for one dictation
out of N. `stop()`/`cancelCapture()` schedule a re-warm; `start()` consumes it.
- **A warm recorder is validated before reuse.** `AVAudioRecorder` resolves its device once and
never re-resolves, so `MicCapture` records the default input's identity (`AudioRoute.currentInput()`)
alongside the warm recorder and discards it when the device has changed — otherwise a recorder
warmed before the user connected their AirPods would keep recording the built-in mic. Unknown
counts as changed.
- **The warm recorder expires** (`preparedRecorderLifetime`, 60 s). A prepared recorder holds the
input device open, which is exactly what pins AirPods in the profile where _output_ audio is
degraded — so it is not held indefinitely. Back-to-back dictations land inside the window; a press
past it just prepares lazily, which is the pre-re-warm behavior.

The re-warm and the liveness gate are complements, not alternatives: the re-warm shortens how _often_
the profile switch is paid (a warm recorder has already held the route open), and the gate is what
keeps the app honest on the presses that pay it anyway.

`stop()` also waits out `AudioTransport.tailLinger(forTransportType:)` (220 ms on Bluetooth, `.zero`
otherwise — the policy sits beside `MicLiveness`'s wait cap so both are unit-tested) before ending
the recording **when the
session's input is Bluetooth**, so speech still travelling over the link lands in the file instead of
being truncated — the missing last word. It runs after `.transcribing` is claimed, so it delays the
transcript, never the "it heard me" cue. Cancels take `cancelCapture()` instead, which skips both the
linger and the file read-back: the audio is being discarded, so neither is worth delaying the user's
cancel for.

The routing facts behind all of that live in **`AudioRoute`** (`Audio/AudioRoute.swift`, internal):
which device is the default input, and its raw CoreAudio transport type. **Raw reads only** — what a
transport _means_ is **`AudioTransport.isBluetooth`** and **`MicLiveness.timeout`**, which are pure
and unit-tested, because `AudioRoute` itself needs real hardware and is excluded from the coverage
gate. Don't let a decision drift into it. Its sibling
**`AudioRouteMonitor`** (public) publishes output-route changes for the cue players — see
[Settings, persistence, and cues](#settings-persistence-and-cues). Both are excluded from the
coverage gate for the same reason
`MicCapture` is: they answer questions only real hardware can answer.

The overlay meter (`levels`) comes from the recorder's dBFS power on a ~20 Hz timer
(`MicCapture.meterIntervalSeconds` — public because the pill caps its animation redraws to the same
cadence and reads it from here rather than restating it), mapped to `0…1` by
`linearLevel(fromPowerDB:)` and floored so room ambient reads as empty bars rather than a meter that
never rests. `levels` and `warmUp()` are part of `MicCaptureProtocol` itself with an empty-stream /
no-op default, so stubs conform with just `start()`/`stop()` while hosts still read the meter through
the seam they inject.
never rests. `levels`, `warmUp()` and `cancelCapture()` are part of `MicCaptureProtocol` itself with
empty-stream / no-op / stop-and-discard defaults, so stubs conform with just `start()`/`stop()` while
hosts still read the meter through the seam they inject.

### `AssemblyAITranscriber` — `Sources/BlurtEngine/STT/AssemblyAITranscriber.swift`

Expand Down Expand Up @@ -502,8 +570,9 @@ stream and signposts) and `+Pipeline.swift` (the post-release transcribe→injec
It exposes `press()` / `release()` / `cancel()` / `cancelRecording()`, a synchronous
fire-and-forget `submit(_: Command)` mirroring those four for callback-shaped hosts (commands run in
exact emit order — the tap wires straight into it, no per-callback `Task` spawning), and a
`phase: PipelinePhase` (`idle | recording | transcribing | injecting | failed | cancelled`, plus the
terminal successes `pasted` / `noTarget`). `phaseStream()` yields the current phase immediately then
`phase: PipelinePhase` (`idle | starting | recording | transcribing | injecting | failed |
cancelled`, plus the terminal successes `pasted` / `noTarget`). `phaseStream()` yields the current
phase immediately then
every transition, and is **multi-observer** (one continuation per call), though hosts should still
render from one consumer and project the phase into their own state.

Expand All @@ -521,14 +590,23 @@ round trip, and the log wrote to the user's real `~/Library/Logs/Blurt`. STT err
are wrapped in `.sttFailed`. The pipeline is just transcribe → inject, and an empty transcript
returns to `.idle` without injecting.

Three perceived-latency choices to preserve:
Four perceived-latency choices to preserve:

- `press()` claims `.connecting` _before_ `mic.start()`, so the pill answers the key-down while the
mic is still coming up — but the **start chime deliberately does not**. `RecordingCueGate` keys on
`.recording`, which `MicCapture`'s liveness gate only reaches once the input route is delivering
frames, so the chime rides the connecting→recording edge by construction. That ordering is
load-bearing, not cosmetic: the chime is a "speak now" cue, on a Bluetooth route it and the press
are ~1–2 s apart, and speech in that window is **unrecoverable** (the OS receives nothing while the
profile switch is in flight). Chiming at the press is what lost the first words of the utterance.
Don't "fix" the chime to fire earlier.
- `.injecting` projects to `OverlayUIState.processing`, **not** `.idle` — the shell reads an idle
projection as "dismiss", so mapping this working phase to idle faded the pill out mid-dictation and
blinked it back for "Pasted".
- `release()` claims `.transcribing` _before_ `mic.stop()`, so the stop chime and pill switch fire at
key-up rather than after the recording is read back. This ordering also closes the double-release
window `ReleaseRaceTests` pins.
window `ReleaseRaceTests` pins — and it is what lets `MicCapture`'s Bluetooth tail linger sit
inside `stop()` without the user ever waiting on it.
- The press-time Accessibility context read is consumed with a bounded wait (`contextWaitBudget`,
500 ms), so an unresponsive frontmost app costs the transcript its priming, never a multi-second
stall.
Expand Down Expand Up @@ -731,8 +809,16 @@ Record cues: **`SoundPack`** is a selectable start/stop chime voice (vintage syn
`App/Blurt/Blurt/Resources/Sounds/`), listed by **`SoundPackCatalog.swift`**, which is _generated_ by
`scripts/generate-sounds.swift` alongside the audio. Regenerate both halves together — `check.sh`'s
sound-catalog guard exists because a drift plays silence with no error. **`RecordingCueGate`** is the
pure edge detector deciding when the chimes fire; the AppKit `CueSoundPlayer` just plays what it
resolves.
pure edge detector deciding when the chimes fire — on the `.recording` edge, i.e. once audio is
actually flowing, never at the press (see the latency notes above for why that ordering is
load-bearing); the AppKit `CueSoundPlayer` just plays what it resolves.

`CueSoundPlayer` decodes and pre-rolls the players once so the first chime never stalls the pill, and
that pre-roll is bound to the output route it was made against. Blurt's own capture invalidates it:
opening the mic flips AirPods out of their output-only profile, dropping the output format underneath
the primed players. So the player observes **`AudioRouteMonitor.outputRouteChanges`** and reloads —
the monitor watches both the default output _device_ (a user switch) and the current device's nominal
sample rate (the profile flip), re-targeting the second listener whenever the first fires.

History: **`RecentDictations`** is an in-memory, newest-first ring (never written to disk) holding
`capacity` (100) dictations, of which the ready window lists `displayCapacity` (3). It is deep because
Expand Down
Loading
Loading