Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/project-guardrails/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ one, stop and ask the user first. This is the fast "don't" list; AGENTS.md's
generated from `project.yml`; edit that and run `xcodegen generate`. check.sh
fails on pbxproj drift (a PreToolUse hook also blocks edits to it).
- The engine has **no external SPM dependencies** (Foundation/Security/
AVFoundation only). Don't add one to `Sources/BlurtEngine/`.
AVFoundation/CoreAudio only). Don't add one to `Sources/BlurtEngine/`.
- Unit tests use **Swift Testing**, not XCTest (the `BlurtUITests` XCUITest
bundle is the one exception — XCUIAutomation requires XCTest). **Never touch
the real Keychain in tests** — `APIKeyStore` is the production item; use an
Expand Down
29 changes: 21 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ Four reflexes before you touch anything:

### The two layers

| Layer | What it is |
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Sources/BlurtEngine/` | Swift package (`swift-tools-version:6.2`, `platforms: [.macOS(.v15)]`) owning the pipeline. Pure logic behind protocol seams, no AppKit-shell deps, **no external SPM deps** — Foundation/Security/AVFoundation plus toolchain modules like Synchronization, with AppKit types only at the seams. |
| `App/Blurt/` | AppKit/SwiftUI shell (Xcode project generated by XcodeGen) that wires the engine to an overlay window, the main window, a Settings scene, a menu bar item, and the trigger key. Its only package is the local `BlurtEngine` (declared in `App/Blurt/project.yml`). |
| Layer | What it is |
| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Sources/BlurtEngine/` | Swift package (`swift-tools-version:6.2`, `platforms: [.macOS(.v15)]`) owning the pipeline. Pure logic behind protocol seams, no AppKit-shell deps, **no external SPM deps** — Foundation/Security/AVFoundation/CoreAudio plus toolchain modules like Synchronization, with AppKit types only at the seams. |
| `App/Blurt/` | AppKit/SwiftUI shell (Xcode project generated by XcodeGen) that wires the engine to an overlay window, the main window, a Settings scene, a menu bar item, and the trigger key. Its only package is the local `BlurtEngine` (declared in `App/Blurt/project.yml`). |

The engine's dependency-free rule is a **rule**; the app merely happens to carry none today. The
former `mxcl/AppUpdater` dependency and its in-place self-updater were removed (see [Updates](#updates)),
Expand All @@ -47,7 +47,8 @@ the design; BLURTENGINE.md covers the _what_ of the API surface.

```text
Sources/BlurtEngine/ the engine (dependency-free Swift package)
Audio/ MicCapture (+meter), SoundPack/Catalog/Store — record cues
Audio/ MicCapture (+meter, MicLiveness start gate), SoundPack/Catalog/Store —
record cues
Config/ Keychain-backed API key, key terms, developer mode, DefaultsKey +
PersistedSettings (every defaults key, and the reset sweep over them)
FocusCapture/ Accessibility reads of the frontmost app / focused field
Expand Down Expand Up @@ -382,6 +383,14 @@ A **fresh recorder per session** resolves the current default input device at `r
is deliberate — see [Settled decisions](#settled-decisions--dont-reintroduce-these) for the
`AVAudioEngine` failure it replaced.

`start()` returns only once the device is actually delivering frames: `record()` returning true just
means the AudioQueue started, and a Bluetooth input (AirPods) spends ~1–2 s switching A2DP→HFP first,
during which the OS captures nothing. `MicLiveness` polls the recorder's clock (which only advances
once frames flow — unlike the meter, this distinguishes a still-switching route from a silent user)
with a transport-aware cap (`kAudioDevicePropertyTransportType`: Bluetooth ~2.5 s, everything else
~300 ms) and **fails open** on timeout, so a broken mic degrades to the old behavior instead of
bricking the press. `DictationSession` shows this wait as the `.connecting` phase.

The overlay meter (`levels`) comes from the recorder's dBFS power on a ~20 Hz timer
(`MicCapture.meterIntervalSeconds` — public because the pill caps its animation redraws to the same
cadence and reads it from here rather than restating it), mapped to `0…1` by
Expand Down Expand Up @@ -453,8 +462,8 @@ stream and signposts) and `+Pipeline.swift` (the post-release transcribe→injec
It exposes `press()` / `release()` / `cancel()` / `cancelRecording()`, a synchronous
fire-and-forget `submit(_: Command)` mirroring those four for callback-shaped hosts (commands run in
exact emit order — the tap wires straight into it, no per-callback `Task` spawning), and a
`phase: PipelinePhase` (`idle | recording | transcribing | injecting | failed | cancelled`, plus the
terminal successes `pasted` / `noTarget`). `phaseStream()` yields the current phase immediately then
`phase: PipelinePhase` (`idle | connecting | recording | transcribing | injecting | failed |
cancelled`, plus the terminal successes `pasted` / `noTarget`). `phaseStream()` yields the current phase immediately then
every transition, and is **multi-observer** (one continuation per call), though hosts should still
render from one consumer and project the phase into their own state.

Expand All @@ -472,8 +481,12 @@ round trip, and the log wrote to the user's real `~/Library/Logs/Blurt`. STT err
are wrapped in `.sttFailed`. The pipeline is just transcribe → inject, and an empty transcript
returns to `.idle` without injecting.

Three perceived-latency choices to preserve:
Four perceived-latency choices to preserve:

- `press()` claims `.connecting` _before_ `mic.start()`, and `.recording` only after it returns —
`start()` holds until the input device actually delivers frames (`MicLiveness`; a Bluetooth
A2DP→HFP switch takes ~1–2 s, capped per transport, failing open on timeout), so the pill
acknowledges the press immediately while the start chime keeps meaning "speak now".
- `.injecting` projects to `OverlayUIState.processing`, **not** `.idle` — the shell reads an idle
projection as "dismiss", so mapping this working phase to idle faded the pill out mid-dictation and
blinked it back for "Pasted".
Expand Down
48 changes: 46 additions & 2 deletions App/Blurt/Blurt/Overlay/OverlayPillContent.swift
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ enum OverlayBrandPalette {
}

/// The shared type, tracking, and cyan color for the overlay's status-line
/// text ("Transcribing…", "Pasted", and "Copied") so they can't drift out of
/// sync.
/// text ("Connecting…", "Transcribing…", "Pasted", and "Copied") so they can't
/// drift out of sync.
struct StatusLineText: View {
let text: String

Expand Down Expand Up @@ -82,6 +82,50 @@ struct TranscribingLabel: View {
}
}

/// The "Connecting…" status line shown while the mic route comes up (the
/// engine's `.connecting` phase — a Bluetooth input takes ~1–2 s to switch
/// A2DP→HFP). Styled as a status line like `TranscribingLabel`, but with a
/// faster breath so it reads as "wait" rather than the calm transcribing
/// heartbeat. Under Reduce Motion it holds steady at full opacity.
///
/// The label holds off for `revealDelay` before appearing: a wired or built-in
/// mic clears the liveness gate in well under 100 ms, so an immediate label
/// flashed "Connecting…" mid fade-in on every fast press. A fast bring-up shows
/// only the dark capsule; the label appears only once the wait is real.
struct ConnectingLabel: View {
/// Whether to run the breathing motion (off under Reduce Motion).
let animated: Bool

/// How long a bring-up must persist before the label appears — long enough
/// that fast (wired/built-in) routes never show it, short next to the ~1–2 s
/// Bluetooth wait it exists for. `OverlayWindowController` holds the
/// `.connecting` VoiceOver announcement for the same delay, so the visual and
/// spoken feedback agree on when a bring-up is worth mentioning.
static let revealDelay: Duration = .milliseconds(200)

// Twice the tempo of TranscribingLabel's breath: busy enough to say "not
// ready yet, hold on" at a glance, while staying the same status-line idiom
// as the rest of the pill. The trough stays at TranscribingLabel's ~55%
// floor — what keeps the 10 pt cyan legible against the dark tint — so the
// faster period alone carries the distinctness.
private let breathPeriod: Double = 0.9
private let minOpacity: Double = 0.55

@State private var revealed = false

var body: some View {
StatusLineText("Connecting…")
.pulsingOpacity(period: breathPeriod, minOpacity: minOpacity, animated: animated)
.opacity(revealed ? 1 : 0)
.animation(animated ? .easeInOut(duration: 0.15) : nil, value: revealed)
.task {
try? await Task.sleep(for: Self.revealDelay)
guard !Task.isCancelled else { return }
revealed = true
}
}
}

/// The "● REC" recording tag: a pulsing magenta dot + "REC" caption, sitting to
/// the left of the waveform — the native echo of the site demo's magenta pixel
/// tag. Magenta (the brand --hot) stands in for the conventional red record dot;
Expand Down
8 changes: 7 additions & 1 deletion App/Blurt/Blurt/Overlay/OverlayView.swift
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ struct OverlayView: View {
switch state {
case .error:
return Color(red: 0.62, green: 0.13, blue: 0.13)
case .recording, .processing, .pasted, .noTarget, .idle:
case .connecting, .recording, .processing, .pasted, .noTarget, .idle:
return Color(white: 0.16)
}
}
Expand Down Expand Up @@ -84,6 +84,12 @@ struct OverlayView: View {
// background would collapse with it) keeps the pill's shape intact for
// `hide()`'s pre-hide reset.
Color.clear
case .connecting:
// The mic route is still coming up (a Bluetooth input switching profiles):
// a breathing status line, deliberately without the REC tag or waveform —
// the "speak now" cues arrive with `.recording`, once audio actually flows.
ConnectingLabel(animated: !reduceMotion)
.transition(.opacity)
case .recording:
// "● REC" tag beside the live waveform, mirroring the site demo's recording
// pill (magenta tag + bars). The bars fill the width left of the tag.
Expand Down
39 changes: 37 additions & 2 deletions App/Blurt/Blurt/Overlay/OverlayWindowController.swift
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,15 @@ final class OverlayWindowController {
// (`OverlayUIState.noticeDwellSeconds`, unit-tested there).
private var errorRevertTask: Task<Void, Never>?

// Holds the `.connecting` VoiceOver announcement until the bring-up has
// persisted past `ConnectingLabel.revealDelay` — the same hold the label
// itself applies. A fast (wired/built-in) route resolves within the delay,
// cancels this, and stays silent all the way to the start chime; only a real
// (Bluetooth-length) wait gets spoken. Without it, VoiceOver users had no
// non-visual feedback at all during a bring-up: announcements fired only for
// the dwell notices, and this non-activating panel never takes focus.
private var connectingAnnounceTask: Task<Void, Never>?

// The pill fades in fast — the appear is tied to the user's keypress, so a snappy
// ramp reads as instant response — but fades out gently. Asymmetric on purpose.
private static let appearFadeDuration: Double = 0.08
Expand Down Expand Up @@ -104,20 +113,26 @@ final class OverlayWindowController {

/// OverlayWindowController lives for the whole app session, so this never runs
/// in practice — but tearing the observer down (and cancelling any pending
/// error-flash revert) mirrors the `[weak self]` care above and documents that
/// the registrations are owned, not leaked.
/// error-flash revert or connecting announcement) mirrors the `[weak self]`
/// care above and documents that the registrations are owned, not leaked.
deinit {
if let didMoveObserver {
NotificationCenter.default.removeObserver(didMoveObserver)
}
errorRevertTask?.cancel()
connectingAnnounceTask?.cancel()
}

func show(state: OverlayUIState) {
// Any explicit state change supersedes a pending error-flash revert: a new
// press while the red pill is up should win, not get stomped back to idle.
errorRevertTask?.cancel()
errorRevertTask = nil
// Likewise a pending connecting announcement: once the state has moved on
// (to `.recording`, or a failure), announcing "Connecting" would be stale —
// and on a fast bring-up this cancel is what keeps the pill silent.
connectingAnnounceTask?.cancel()
connectingAnnounceTask = nil

// Idle means "no dictation happening" — the pill rides the pipeline and is
// hidden at rest, so fade it out. The displayed state is left untouched so
Expand All @@ -137,6 +152,24 @@ final class OverlayWindowController {
if bridge.state != state {
bridge.state = state
}
// The mic bring-up gets the same VoiceOver treatment as the dwell notices
// below — this panel never takes focus, so an announcement is the only
// non-visual channel — but held for the label's reveal delay first (see
// `connectingAnnounceTask`): a fast route flips to `.recording` within the
// delay and goes straight to the start chime.
if case .connecting = state {
connectingAnnounceTask = Task {
try? await Task.sleep(for: ConnectingLabel.revealDelay)
guard !Task.isCancelled else { return }
NSAccessibility.post(
element: NSApp as Any,
notification: .announcementRequested,
userInfo: [
.announcement: state.accessibilityLabel,
.priority: NSAccessibilityPriorityLevel.high.rawValue,
])
}
}
// The red error flash and the neutral "copied" notice are both transient: the
// pill is otherwise only up during active dictation, so they linger briefly to
// be read, then settle back to idle. Announce them for VoiceOver since this
Expand Down Expand Up @@ -171,6 +204,8 @@ final class OverlayWindowController {
func hide() {
errorRevertTask?.cancel()
errorRevertTask = nil
connectingAnnounceTask?.cancel()
connectingAnnounceTask = nil
guard panel.isVisible else {
// Still settle the content when the panel is already off screen (the pill
// may have been hidden mid-notice) — `dismissPanel` would have done it.
Expand Down
Loading
Loading