diff --git a/.claude/skills/project-guardrails/SKILL.md b/.claude/skills/project-guardrails/SKILL.md index aecb1a23..3b4e7580 100644 --- a/.claude/skills/project-guardrails/SKILL.md +++ b/.claude/skills/project-guardrails/SKILL.md @@ -79,9 +79,17 @@ genuinely correct, and reaching for it means it's time to stop and ask. menu bar, so nothing may depend on it being visible. A menu-bar-_only_ variant (no Dock icon) was tried and reverted twice for that reason — don't drop the Dock icon or add `LSUIElement`. -- The dictation trigger is a **single lone modifier** (right ⌘ default), home- - grown via `CGEventTap` + `DictationKeyGate`. No `KeyboardShortcuts` package, no - key+modifier chord. +- The dictation trigger is a **single control** — a lone right-side modifier + (right ⌘ default, and `fn` is no longer an option), or a Custom **chord** + (⌃⌥D) or extra mouse button — home-grown via `CGEventTap` + + `DictationKeyRouter`/`DictationKeyGate`. No `KeyboardShortcuts` package, and no + SPM dependency for key handling: that ban is unchanged. Chords used to be + banned here too and are now supported (Alex Kroman reversed it in the PR #144 + thread). What stays refused: a **bare** key and the left/right mouse button + (the tap is listen-only and swallows nothing, so they'd type or click into the + focused app), plus a short list of system chords like ⌘Q and ⌘⇥. A bound chord + or wheel click still reaches the app that owns it — say so in the UI + (`TriggerBinding.passThroughNote`) rather than switching to an active tap. - **Updates are download-only** — check → open the DMG in the browser → the user installs it. The `mxcl/AppUpdater` dependency and its in-place self-updater were removed; don't reintroduce a self-replacing install path, a timer-driven diff --git a/AGENTS.md b/AGENTS.md index f42505a2..960f2965 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -56,7 +56,8 @@ Sources/BlurtEngine/ the engine (dependency-free Swift package) Config/ Keychain-backed API key, key terms, developer mode, DefaultsKey + PersistedSettings (every defaults key, and the reset sweep over them) FocusCapture/ Accessibility reads of the frontmost app / focused field - Hotkey/ TriggerKey(+Store), DictationKeyGate, DictationKeyRouter + Hotkey/ TriggerKey(+Store), TriggerBinding, DictationKeyGate/Router, + DictationEventDecoder Injection/ KeyInjector (clipboard paste), SystemClipboard Permissions/ PermissionsChecker (mic + Accessibility) Pipeline/ DictationSession (actor) + phases, UI projections, geometry, log @@ -346,25 +347,25 @@ a rule that outlives its row keeps firing and keeps citing this table while enfo project has already reversed. Reworded the row? Update the anchor. Reversed the decision? Delete the rule along with the row. -| Don't | Because | -| ---------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | -| Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` uses a fresh `AVAudioRecorder` per session. | -| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. | -| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by the `llm` block on the same `/transcribe` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `ConversationContext`. | -| Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | -| Pin transcription to English, or set a language at all | Hurt non-English transcription; language is left to the model's own detection. **No `config.language_code`** either — the API documents it as defaulting to `en` and as ignored while a custom `prompt` is set, so dropping the prompt un-ignored it; detection was then measured to work with neither field set (es/fr/de/ja clips each transcribed in their own language against the live endpoint, rewrite included). Setting one would only take that away. `KeytermsWireTests` asserts the absence. | -| Bring back `config.prompt` | Replaced by `config.conversation_context` (`ConversationContext`), which is the structured field for the same job. A custom `prompt` also replaces the service's managed default and makes the API ignore `language_code`, so re-adding one silently gives up both. | -| Widen the request's context past history + the prior chunk | `ConversationContext.turns` reads exactly two fields of `TranscriptionContext` — `recentTranscripts` and `priorText`. The app name, window title, field label and selected text are captured for the paste path and the developer-mode log and stay on the machine; the app/window/field hints and the `Selected text:` block were removed, not gated. Don't add one back, and don't route that context onto the request by another path. The user's key terms are the exception that proves the rule: they _are_ sent, as the request's own `word_boost` list (`KeytermsBoost`) — never folded back into the context turns. | -| Add a "remove filler words (um, uh, like)" clause | Not in the STT model's trained instruction set — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | -| Add a keystroke-typing paste path or a length threshold | Injection is **always** clipboard paste (save → write → ⌘V → settle → restore), with the copied-to-clipboard degradation when the target is lost. | -| Add `LSUIElement` or a menu-bar-**only** mode | Blurt is a Dock app first. The `MenuBarExtra` status item is convenience layered on the Dock icon; the notch can hide a status item, so nothing may depend on it. A menu-bar-only variant was reverted twice. | -| Add a `KeyboardShortcuts` package or a key+modifier chord | The trigger is a single lone modifier, home-grown (`CGEventTap` + `DictationKeyGate`), and swallows nothing. | -| Add a self-replacing install or background auto-updater | Updates are download-only; `mxcl/AppUpdater` and its in-place updater were removed. The once-a-day launch _check_ (`AutomaticUpdateCheck`) is fine; installing for the user, or polling, is not. Extend `UpdateCheckModel`. | -| Hand-edit `Blurt.xcodeproj/project.pbxproj` | Generated from `project.yml`; `check.sh`'s drift check fails on any manual edit (a Claude PreToolUse hook also blocks it). | -| Redirect the post-build install away from `/Applications` | TCC won't register apps in DerivedData/`/tmp`, so permission toggles never appear. | -| Touch the real Keychain in tests | `APIKeyStore` is the production item — a test that writes it triggers Keychain prompts and corrupts the real item's ACL. Use an isolated service (see `KeychainStoreTests`) or `InMemoryAPIKeyStore`. | -| Add backwards-compat shims for removed types | Deleted types stay deleted — no deprecated re-exports. | +| Don't | Because | +| ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Add an external SPM dependency to the engine | Dependency-free by rule (biggest supply-chain risk); a `check.sh` guard fails on `.package(` in `Package.swift` or a `url:`/`github:` package in `project.yml`. Extend `BlurtEngine` instead. | +| Use `AVAudioEngine` / `installTap` for capture | A long-lived engine bound its input graph to one device and went stale on a mic↔built-in switch — `-10868` (`kAudioUnitErr_FormatNotSupported`) or all-zero buffers. `MicCapture` uses a fresh `AVAudioRecorder` per session. | +| Add streaming STT | The dictation API returns the full (already rewritten) text in one response; the overlay shows "Transcribing…" then the full text. | +| Add a client-side LLM cleanup pass | Cleanup is the dictation API's server-side rewrite, requested by the `llm` block on the same `/transcribe` call. No LLM Gateway client, no `StylerProtocol`, no styling stage, no second request — transcription steering belongs in `ConversationContext`. | +| Add local models or model downloads | Transcription is a remote AssemblyAI call: no on-device ASR/LLM, no model cache, no download UI. | +| Pin transcription to English, or set a language at all | Hurt non-English transcription; language is left to the model's own detection. **No `config.language_code`** either — the API documents it as defaulting to `en` and as ignored while a custom `prompt` is set, so dropping the prompt un-ignored it; detection was then measured to work with neither field set (es/fr/de/ja clips each transcribed in their own language against the live endpoint, rewrite included). Setting one would only take that away. `KeytermsWireTests` asserts the absence. | +| Bring back `config.prompt` | Replaced by `config.conversation_context` (`ConversationContext`), which is the structured field for the same job. A custom `prompt` also replaces the service's managed default and makes the API ignore `language_code`, so re-adding one silently gives up both. | +| Widen the request's context past history + the prior chunk | `ConversationContext.turns` reads exactly two fields of `TranscriptionContext` — `recentTranscripts` and `priorText`. The app name, window title, field label and selected text are captured for the paste path and the developer-mode log and stay on the machine; the app/window/field hints and the `Selected text:` block were removed, not gated. Don't add one back, and don't route that context onto the request by another path. The user's key terms are the exception that proves the rule: they _are_ sent, as the request's own `word_boost` list (`KeytermsBoost`) — never folded back into the context turns. | +| Add a "remove filler words (um, uh, like)" clause | Not in the STT model's trained instruction set — a no-op, deliberately dropped; disfluency removal is the server-side LLM rewrite's job. | +| Add a keystroke-typing paste path or a length threshold | Injection is **always** clipboard paste (save → write → ⌘V → settle → restore), with the copied-to-clipboard degradation when the target is lost. | +| Add `LSUIElement` or a menu-bar-**only** mode | Blurt is a Dock app first. The `MenuBarExtra` status item is convenience layered on the Dock icon; the notch can hide a status item, so nothing may depend on it. A menu-bar-only variant was reverted twice. | +| Add a `KeyboardShortcuts` package (or any key-binding dependency) | The trigger is home-grown (`CGEventTap` + `DictationKeyRouter`/`DictationKeyGate`) and swallows nothing: a lone right-side modifier by default, or a Custom **chord** (⌃⌥D) or extra mouse button. Chords were once forbidden here and are now supported — Alex Kroman reversed that in the PR #144 thread — but the **package** ban stands, and so does the rest of the shape: no bare key (the listen-only tap can't stop it from typing), no left/right mouse button, and a short refused list of system chords. | +| Add a self-replacing install or background auto-updater | Updates are download-only; `mxcl/AppUpdater` and its in-place updater were removed. The once-a-day launch _check_ (`AutomaticUpdateCheck`) is fine; installing for the user, or polling, is not. Extend `UpdateCheckModel`. | +| Hand-edit `Blurt.xcodeproj/project.pbxproj` | Generated from `project.yml`; `check.sh`'s drift check fails on any manual edit (a Claude PreToolUse hook also blocks it). | +| Redirect the post-build install away from `/Applications` | TCC won't register apps in DerivedData/`/tmp`, so permission toggles never appear. | +| Touch the real Keychain in tests | `APIKeyStore` is the production item — a test that writes it triggers Keychain prompts and corrupts the real item's ACL. Use an isolated service (see `KeychainStoreTests`) or `InMemoryAPIKeyStore`. | +| Add backwards-compat shims for removed types | Deleted types stay deleted — no deprecated re-exports. | Release-side invariants (hardened runtime and a secure timestamp on every nested mach-o and embedded framework, or notarization rejects the build; roll-forward-only for a bad release) live in @@ -652,50 +653,103 @@ user picks a different trigger key. ## Hotkey -The dictation trigger is a **single lone modifier key** (tap-to-toggle or hold-to-talk), implemented -in-house. Four pieces, three of them pure engine logic: +The dictation trigger is a **single control** (tap-to-toggle or hold-to-talk), implemented in-house: +a lone right-side modifier by default (right ⌘ or right ⌥), or — via the Shortcut picker's Custom +option — a **keyboard chord** (⌃⌥D) or an extra mouse button. Never a **bare** key: the tap is +listen-only and swallows nothing, so a bound letter would type into the focused app on every +dictation. A chord isn't swallowed either, which is a property the UI states rather than hides +(`TriggerBinding.passThroughNote`), and a short list of system chords (⌘Q, ⌘⇥, ⌘Space…) is refused +outright. Six pieces, five of them pure engine logic: - **`TriggerKey`** (`Hotkey/TriggerKey.swift`) — enum of the curated lone momentary modifiers usable - as the trigger (right ⌘, right ⌥, `fn`), `rawValue` = the macOS virtual keycode, plus `label` + as the trigger (right ⌘, right ⌥), `rawValue` = the macOS virtual keycode, plus `label` ("right ⌘") and the device-modifier masks the event source needs. Right-side modifiers are chosen - because a solo press rarely collides with app shortcuts. -- **`TriggerKeyStore`** — persists the chosen keycode in `UserDefaults` (`BlurtTriggerKeyCode`), + because a solo press rarely collides with app shortcuts. **`fn` was a third option and was + removed** — it is the one modifier macOS itself claims (dictation, emoji picker, the F-key row) — + and a persisted `fn` migrates to right ⌥ (`TriggerBinding.legacyFunctionKeyCode`), not to the + generic right-⌘ fallback, so those users keep a right-side one-key trigger. +- **`TriggerBinding`** (`Hotkey/TriggerBinding.swift`) — what's actually bound: + `.modifier(TriggerKey)`, `.chord(keyCode:modifiers:)`, or `.mouseButton(_:)`. Every binding encodes + into the **single `Int` slot the store has always used**, so `@AppStorage` keeps live-rendering and + nothing migrates: modifiers as their bare keycode, mouse buttons as `0x10000 + button`, and a chord + packed as `0x20000 | (modifiers << 8) | keyCode` (keycode in the low byte, the four modifier bits + above it — keycodes are 16-bit, so the tags can't collide). `fromPersisted` is the one + decode-with-default rule (garbage → right ⌘, and a structurally invalid chord counts as garbage), + while `chordBinding(forKeyCode:modifiers:)` and `mouseButtonBinding(forButton:)` are the capture + policy — what's bindable, and _why_ each refusal happens (`ChordRefusal`: `bareKey`, + `modifierOnly`, `reserved`), is engine logic with tests, not a view's private list. + **Mouse-button numbering:** the stored number is the raw 0-based `CGEvent`/`NSEvent` + `buttonNumber` (0 left, 1 right, 2 wheel/middle, 3+ side buttons) and only 2 and up are bindable; + `label` renders it 1-based the way mice are numbered for users, so stored `2` displays as + "Mouse 3" **and is the middle (wheel) click**. `ChordModifiers` is side-agnostic (⌃ is ⌃) with a + persisted bit layout and `glyphs` in Apple's ⌃⌥⇧⌘ order. +- **`TriggerKeyStore`** — persists the encoded binding in `UserDefaults` (`BlurtTriggerKeyCode`), defaulting to **right ⌘**. - **`DictationKeyGate`** — pure, clock-free state machine (`idle`/`armed`/`latched`) turning - modifier-down/up and other-key-down into `start`/`stop`/`cancel`/`none`. Recording starts the - instant the modifier goes down; on key-up a release ≥ `holdThreshold` (default 1 s) is a **hold** + trigger-down/up and other-key-down into `start`/`stop`/`cancel`/`none`. Recording starts the + instant the trigger goes down; on key-up a release ≥ `holdThreshold` (default 1 s) is a **hold** (push-to-talk → stop) while a shorter release **latches** (tap-to-toggle; next tap stops). A combo (modifier + another key, e.g. ⌘C) from idle cancels the fresh capture; over a latched recording it passes through as a normal shortcut. Callers pass monotonic timestamps, so every decision is deterministic and unit-tested. -- **`DictationKeyRouter`** — the event-routing layer over the gate: only the bound keycode's flag - changes drive the modifier, and only genuine down/up **edges** reach the gate (`flagsChanged` - deliveries re-report the bit whether or not it changed, so a repeat must not double-fire). - `reset()`/`rebind(triggerKeyCode:)` report whether they discarded a live recording the host must +- **`DictationKeyRouter`** — the event-routing layer over the gate: only events for the bound + trigger drive it (`flagsChanged` for a modifier binding, `keyDown`/`keyUp` plus each event's + modifier set for a chord, `mouseDown`/`mouseUp` for a button), and only genuine down/up **edges** + reach the gate (`flagsChanged` deliveries re-report the bit whether or not it changed, and a held + key autorepeats, so a repeat must not double-fire). A chord fires only on the **exact** required + modifier set (a superset is a different shortcut) and ends on the key's `keyUp` **or** the moment + any required modifier is released — the realistic release order for ⌃⌥D is ⌃ first, and waiting + for D's `keyUp` would leave a hold recording with nothing coming to end it. The combo rule applies + to **modifier bindings only**: ⌘C over the held trigger is a real shortcut, but a chord's own + modifiers are part of the trigger and Mouse4+K names nothing to macOS, so elsewhere a key press is + just typing and must not cancel the dictation. + `reset()`/`rebind(binding:)` report whether they discarded a live recording the host must cancel upstream, and so does **`recoverFromDroppedEvents(triggerStillHeld:)`** — the disabled-tap rule: events may have been dropped, so the gate's state survives only while the - trigger is still physically held (its key-up is still coming); otherwise that key-up was among - the losses and the gate is reset. The host passes the `CGEventSource.flagsState` read in and - keeps no decision of its own, so the rule is unit-tested rather than living in an untestable - shell `if`. + trigger is still physically held (its up-event is still coming); otherwise that up-event was among + the losses and the gate is reset. The host passes the `CGEventSource` read + (`flagsState`/`keyState`/`buttonState`, per binding family — a chord needs its key _and_ its + modifiers still down) in and keeps no decision of its own, + so the rule is unit-tested rather than living in an untestable shell `if`. +- **`DictationEventDecoder`** (`Hotkey/DictationEventDecoder.swift`) — the `CGEvent` → router-event + reduction, deliberately the one CoreGraphics-typed piece of the hotkey engine so it can be + exercised with **real `CGEvent` fixtures** (`DictationEventDecoderTests` constructs genuine + mouse/keyboard/flags events and asserts what they decode to — rows are data, so captures from a + misbehaving mouse can be appended). The router itself stays CoreGraphics-free. The app side, **`DictationKeyTap`** (`App/Blurt/Blurt/Hotkey/DictationKeyTap.swift`), reduces each -`CGEventTap` delivery (watching `flagsChanged` for the bound modifier and `keyDown` for any other -key) to a `DictationKeyRouter.Event` and owns the tap lifecycle. `AppCoordinator` calls its +`CGEventTap` delivery to a `DictationKeyRouter.Event` (via `DictationEventDecoder`) and owns the tap +lifecycle. The tap's mask covers every family any binding might need (`flagsChanged`, +`keyDown`+`keyUp` — autorepeat deliveries dropped in the decoder — and +`otherMouseDown`/`otherMouseUp`), because a tap's mask is fixed at +creation and the router ignores what the current binding doesn't care about — so rebinding never +recreates the tap. `AppCoordinator` calls its `syncAfterTerminalPhase()` on every terminal phase: a dictation can end with no key event to close the gate (the auto-release cap, or a refused/failed press), which would leave the gate `.latched` and silently swallow the user's next press — a latched `modifierDown` returns `.none`, and the `modifierUp` after it returns `.stop`, which no-ops on an already-terminal session. The tap -**swallows nothing**: a lone modifier types nothing, and combos pass through so normal shortcuts keep -working. +**swallows nothing**: a lone modifier or extra button types and clicks nothing, combos +pass through so normal shortcuts keep working, and a bound chord still reaches the app that owns it +(which is why the Settings footer says so). The trigger is editable in the Shortcut section of the setup/settings UI (`HotkeyStepView`) — a -`Picker` over `TriggerKey.allCases` that writes `TriggerKeyStore`, after which -`AppCoordinator.dictationBindingChanged()` re-reads it into the tap. For display strings, use -`TriggerKeyStore().triggerKey.label` for one-shot reads; in views that must re-render live on a -Settings change, use **`@BoundTriggerKey`** — a `DynamicProperty` in `Wizard/BoundTriggerKey.swift` wrapping -the `@AppStorage(TriggerKeyStore.defaultsKey)` + `TriggerKey.fromPersisted` pair — rather than -restating that pairing per view. The unset default belongs to `fromPersisted` (an absent keycode maps +`Picker` over `TriggerKey.allCases` plus a **Custom…** row that opens a press-to-capture sheet +(`CustomTriggerCaptureView`, same file): local `NSEvent` monitors for `keyDown` and `flagsChanged` +(the held modifiers render live as ⌃⌥… while the chord forms) plus local and global +`otherMouseDown` monitors, Esc always cancels, and each refusal shows the engine's reason — a bare +key, a modifier as the key half, a system-reserved chord, or an unbindable button. A chord that +macOS itself owns (⌘⇥, ⌘Space) never reaches a local monitor at all, which is the same reason those +are refused rather than merely discouraged. While developer mode is on, **every +event the recorder sees — accepted or refused — is appended to `capture-events.jsonl`** +(`DictationLog.appendCaptureEvent`, same switch and queue as the other two logs) with its raw button +number, keycode, flags, and autorepeat bit, so a misbehaving multi-button mouse can be diagnosed +from what it actually sent. Writes go through +`TriggerKeyStore`, after which `AppCoordinator.dictationBindingChanged()` re-reads the binding into +the tap. For display strings, use `TriggerKeyStore().triggerBinding.label` for one-shot reads; in +views that must re-render live on a Settings change, use **`@BoundTriggerBinding`** — a +`DynamicProperty` in `Wizard/BoundTriggerKey.swift` wrapping the +`@AppStorage(TriggerKeyStore.defaultsKey)` + `TriggerBinding.fromPersisted` pair — rather than +restating that pairing per view. The unset default belongs to `fromPersisted` (an absent code maps to right ⌘), so views must not re-declare `TriggerKey.rightCommand.rawValue` themselves. ## Conversation context and key terms diff --git a/App/Blurt/Blurt/Hotkey/DictationKeyTap.swift b/App/Blurt/Blurt/Hotkey/DictationKeyTap.swift index 130d9bd4..c3375b1f 100644 --- a/App/Blurt/Blurt/Hotkey/DictationKeyTap.swift +++ b/App/Blurt/Blurt/Hotkey/DictationKeyTap.swift @@ -2,21 +2,30 @@ import BlurtEngine import CoreGraphics import os -/// Drives the single lone-modifier dictation trigger from a `CGEventTap`. +/// Drives the single-key dictation trigger from a `CGEventTap`. /// -/// Watches `flagsChanged` for the bound modifier (e.g. right ⌘, keycode 54) to -/// detect down/up, and `keyDown` for any *other* key to spot a modifier combo -/// (⌘C, ⌘V…). The per-event decision lives in the engine — `DictationKeyRouter` -/// (keycode relevance + down/up edge dedup) over `DictationKeyGate` (tap/hold -/// semantics) — so this type only reduces each `CGEvent` to a router event and -/// owns the tap lifecycle. +/// The bound trigger is a `TriggerBinding`: a lone modifier (e.g. right ⌘, +/// keycode 54) watched via `flagsChanged`, a keyboard chord (⌃⌥D) watched via +/// `keyDown`/`keyUp` plus the modifier set each event reports, or an extra mouse +/// button watched via `otherMouseDown`/`otherMouseUp`. `keyDown` is also watched +/// for any *other* key to spot a modifier combo (⌘C, ⌘V…). The per-event decision lives in the +/// engine — `DictationKeyRouter` (binding relevance + down/up edge dedup) over +/// `DictationKeyGate` (tap/hold semantics) — so this type only reduces each +/// `CGEvent` to a router event and owns the tap lifecycle. The mask covers +/// every family the router can care about regardless of the current binding +/// (a tap's mask is fixed at creation, and the router ignores irrelevant +/// events), so rebinding never has to recreate the tap. /// -/// Unlike the old chord trigger, this **swallows nothing**: a lone modifier -/// types nothing into the focused app, and combos must pass through so normal -/// shortcuts keep working. The tap is therefore created `.listenOnly` — an -/// active (`.defaultTap`) tap would make macOS synchronously wait on this -/// process before delivering every keystroke system-wide, so any main-thread -/// stall in Blurt would add typing latency in *other* apps. +/// This **swallows nothing**, by design: a lone modifier or extra mouse button +/// types and clicks nothing into the focused app, and combos must pass through +/// so normal shortcuts keep working. The tap is +/// therefore created `.listenOnly` — an active (`.defaultTap`) tap would make +/// macOS synchronously wait on this process before delivering every keystroke +/// system-wide, so any main-thread stall in Blurt would add typing latency in +/// *other* apps. Two consequences the UI states rather than hides: the Custom +/// recorder refuses a **bare** key (it would type on every dictation), and a +/// bound **chord** still reaches an app that already owns it — see +/// `TriggerBinding.passThroughNote`. /// /// Main-actor (via the app target's default isolation) because everything here /// already runs on the main thread: the tap's run-loop source is added to the @@ -40,12 +49,15 @@ final class DictationKeyTap { /// a transcript already in flight — see `DictationSession.cancelRecording`. private let onRecordingDiscarded: @Sendable () -> Void - /// The engine-side event router (keycode relevance, down/up edge dedup, and + /// The engine-side event router (binding relevance, down/up edge dedup, and /// the gate's tap/hold state machine — all unit-tested in BlurtEngine). /// Seeded from the persisted binding in `init` — see the note there. private var router: DictationKeyRouter - /// The bound key's device-dependent `CGEventFlags` bit — the one CoreGraphics- - /// typed piece of the binding, so it stays here rather than in the router. + /// The bound modifier's device-dependent `CGEventFlags` bit — the one + /// CoreGraphics-typed piece of the binding, so it stays here rather than in + /// the router. Empty for chord and mouse bindings, whose down/up state rides + /// their own event types rather than one device bit (a chord reads the generic + /// modifier masks, which the decoder resolves per event). private var triggerFlag: CGEventFlags /// Monotonic reference; per-event timestamps are `reference.duration(to: now)`. @@ -68,15 +80,15 @@ final class DictationKeyTap { self.onCancel = onCancel self.onRecordingDiscarded = onRecordingDiscarded // Both halves of the binding come from the store, not a hard-coded - // `.rightCommand`: `TriggerKey.fromPersisted` owns the unset default, and - // restating it here is the same mistake `BoundTriggerKey` and `HotkeyStepView` + // `.rightCommand`: `TriggerBinding.fromPersisted` owns the unset default, and + // restating it here is the same mistake `BoundTriggerBinding` and `HotkeyStepView` // were each corrected away from — the tap would name the old key while the // picker, ready screen, and menu bar all named the new one. `refreshBinding()` // re-reads this, but nothing enforces that it runs before the first read of - // either property (`simulatePressForTesting` reads `router.triggerKeyCode`). - let key = TriggerKeyStore().triggerKey - self.router = DictationKeyRouter(triggerKeyCode: key.keyCode) - self.triggerFlag = Self.flag(for: key) + // either property (`simulatePressForTesting` reads `router.binding`). + let binding = TriggerKeyStore().triggerBinding + self.router = DictationKeyRouter(binding: binding) + self.triggerFlag = Self.flag(for: binding) } deinit { @@ -102,7 +114,10 @@ final class DictationKeyTap { CGEvent.tapEnable(tap: tap, enable: true) return true } - let mask = (1 << CGEventType.keyDown.rawValue) | (1 << CGEventType.flagsChanged.rawValue) + let mask = + (1 << CGEventType.keyDown.rawValue) | (1 << CGEventType.keyUp.rawValue) + | (1 << CGEventType.flagsChanged.rawValue) + | (1 << CGEventType.otherMouseDown.rawValue) | (1 << CGEventType.otherMouseUp.rawValue) guard let created = CGEvent.tapCreate( tap: .cgSessionEventTap, @@ -142,8 +157,8 @@ final class DictationKeyTap { /// Resets unconditionally, unlike the disabled-tap recovery above. That guard /// exists to preserve a *live* recording whose key events were dropped; here the /// dictation is already over, so there is no state worth keeping even if the - /// trigger is still physically held — a later key-up just finds - /// `modifierIsDown == false` and routes to `.none`. + /// trigger is still physically held — a later key-up just finds the router's + /// down-tracker cleared and routes to `.none`. /// /// Acts on `reset()`'s discarded-recording result the same way `refreshBinding` /// does. In the stale case the session is already terminal, so the @@ -160,14 +175,14 @@ final class DictationKeyTap { if router.reset() { onRecordingDiscarded() } } - /// Re-read the bound trigger key into the router. Call after the user - /// rebinds. The router's reset reports a discarded live recording: rebinding - /// mid-dictation means the old key's up-event will never match, so the capture - /// must be cancelled, not left to run out the auto-release cap. + /// Re-read the bound trigger into the router. Call after the user rebinds. + /// The router's reset reports a discarded live recording: rebinding + /// mid-dictation means the old trigger's up-event will never match, so the + /// capture must be cancelled, not left to run out the auto-release cap. func refreshBinding() { - let key = TriggerKeyStore().triggerKey - triggerFlag = Self.flag(for: key) - if router.rebind(triggerKeyCode: key.keyCode) { onRecordingDiscarded() } + let binding = TriggerKeyStore().triggerBinding + triggerFlag = Self.flag(for: binding) + if router.rebind(binding: binding) { onRecordingDiscarded() } } /// Callback entry point (always on the main thread — the tap's source lives on @@ -180,29 +195,43 @@ final class DictationKeyTap { // state survives that is the router's call (and unit-tested there); the only // part that has to happen here is the CoreGraphics read of whether the // trigger is physically held right now. - let triggerStillHeld = CGEventSource.flagsState(.combinedSessionState).contains(triggerFlag) - if router.recoverFromDroppedEvents(triggerStillHeld: triggerStillHeld) { + if router.recoverFromDroppedEvents(triggerStillHeld: triggerStillHeld()) { onRecordingDiscarded() } return } - guard let routed = routerEvent(type: type, event: event) else { return } + // The CGEvent -> router-event reduction lives in the engine + // (`DictationEventDecoder`) so it can be exercised with real CGEvent + // fixtures; this shim only owns the tap lifecycle around it. + guard + let routed = DictationEventDecoder.routerEvent( + type: type, event: event, triggerFlag: triggerFlag) + else { return } let now = reference.duration(to: ContinuousClock.now) dispatch(router.handle(routed, at: now)) } - /// Reduces a `CGEvent` to the router's CoreGraphics-free event shape, or nil - /// for event types the trigger doesn't care about. - private func routerEvent(type: CGEventType, event: CGEvent) -> DictationKeyRouter.Event? { - let keyCode = Int(event.getIntegerValueField(.keyboardEventKeycode)) - switch type { - case .flagsChanged: - return .flagsChanged(keyCode: keyCode, triggerFlagIsOn: event.flags.contains(triggerFlag)) - case .keyDown: - return .keyDown(keyCode: keyCode) - default: - return nil + /// The CoreGraphics read behind dropped-event recovery: is the bound trigger + /// physically down right now? Each binding family has its own state query. + private func triggerStillHeld() -> Bool { + switch router.binding { + case .modifier: + return CGEventSource.flagsState(.combinedSessionState).contains(triggerFlag) + case .chord(let keyCode, let modifiers): + // Both halves have to still be down for the press to be live: the key + // itself, and every modifier the chord requires. + let held = DictationEventDecoder.modifiers( + from: CGEventSource.flagsState(.combinedSessionState)) + return CGEventSource.keyState(.combinedSessionState, key: CGKeyCode(keyCode)) + && held.isSuperset(of: modifiers) + case .mouseButton(let button): + // `CGMouseButton` is an open C enum, so any button number constructs; a + // nil (out-of-range) read degrades to "not held", which resets the gate — + // the conservative side, since keeping state with no up-event coming + // strands the session until the auto-release cap. + guard let cgButton = CGMouseButton(rawValue: UInt32(button)) else { return false } + return CGEventSource.buttonState(.combinedSessionState, button: cgButton) } } @@ -215,18 +244,20 @@ final class DictationKeyTap { } } - /// The `CGEventFlags` bit the bound key toggles, so a `flagsChanged` event for - /// it reads as down (bit set) or up (bit clear). This is the *device-dependent* - /// per-side bit (e.g. right ⌘ only), not the generic `.maskCommand` shared by - /// both ⌘ keys — see `TriggerKey.deviceModifierMask` for why that distinction - /// keeps the down/up tracking from desyncing on keyboards with both keys held. - nonisolated static func flag(for key: TriggerKey) -> CGEventFlags { - CGEventFlags(rawValue: key.deviceModifierMask) + /// The `CGEventFlags` bit a bound modifier toggles, so a `flagsChanged` event + /// for it reads as down (bit set) or up (bit clear). This is the + /// *device-dependent* per-side bit (e.g. right ⌘ only), not the generic + /// `.maskCommand` shared by both ⌘ keys — see `TriggerKey.deviceModifierMask` + /// for why that distinction keeps the down/up tracking from desyncing on + /// keyboards with both keys held. Mouse bindings have no flag bit. + nonisolated static func flag(for binding: TriggerBinding) -> CGEventFlags { + guard case .modifier(let key) = binding else { return [] } + return CGEventFlags(rawValue: key.deviceModifierMask) } #if UITEST_HOOKS /// Test seam: drive the real gate + callback dispatch for a synthetic - /// lone-modifier press, bypassing the `CGEventTap` (whose creation needs + /// trigger press, bypassing the `CGEventTap` (whose creation needs /// Accessibility trust an automated run doesn't have). Pairs with /// `simulateReleaseForTesting()` to run the same press→hold→release path a /// real keypress would — used by the leak exercise (`scripts/leaks.sh`) so the @@ -234,17 +265,27 @@ final class DictationKeyTap { /// covered, not just the session the coordinator drives directly. func simulatePressForTesting() { _ = router.reset() - dispatch( - router.handle( - .flagsChanged(keyCode: router.triggerKeyCode, triggerFlagIsOn: true), at: .seconds(0))) + dispatch(router.handle(syntheticEvent(down: true), at: .seconds(0))) } /// Completes the synthetic cycle as a hold (past the threshold), so the gate /// emits `.stop` and `onStop` fires. func simulateReleaseForTesting() { - dispatch( - router.handle( - .flagsChanged(keyCode: router.triggerKeyCode, triggerFlagIsOn: false), at: .seconds(2))) + dispatch(router.handle(syntheticEvent(down: false), at: .seconds(2))) + } + + /// The event a real down/up of the bound trigger would reduce to, whatever + /// family the current binding belongs to. + private func syntheticEvent(down: Bool) -> DictationKeyRouter.Event { + switch router.binding { + case .modifier(let key): + return .flagsChanged(keyCode: key.keyCode, triggerFlagIsOn: down) + case .chord(let keyCode, let modifiers): + return down + ? .keyDown(keyCode: keyCode, modifiers: modifiers) : .keyUp(keyCode: keyCode) + case .mouseButton(let button): + return down ? .mouseDown(button: button) : .mouseUp(button: button) + } } #endif } diff --git a/App/Blurt/Blurt/MenuBar/MenuBarScene.swift b/App/Blurt/Blurt/MenuBar/MenuBarScene.swift index fc3b2d02..0265f9dd 100644 --- a/App/Blurt/Blurt/MenuBar/MenuBarScene.swift +++ b/App/Blurt/Blurt/MenuBar/MenuBarScene.swift @@ -30,13 +30,13 @@ struct MenuBarContent: View { @Environment(\.openSettings) private var openSettings // Observed (as the ready screen does) so the reminder line updates live when - // the dictation key is rebound in Settings — see `BoundTriggerKey`. - @BoundTriggerKey private var triggerKey + // the dictation key is rebound in Settings — see `BoundTriggerBinding`. + @BoundTriggerBinding private var trigger var body: some View { // Disabled informational row: the dictation trigger is an invisible lone - // modifier, so spell it out here as the menu bar's discoverability anchor. - Text("Tap or hold \(triggerKey.label) to dictate and paste") + // key, so spell it out here as the menu bar's discoverability anchor. + Text("Tap or hold \(trigger.label) to dictate and paste") Divider() diff --git a/App/Blurt/Blurt/Wizard/BoundTriggerKey.swift b/App/Blurt/Blurt/Wizard/BoundTriggerKey.swift index 67d73ce4..2ddb9075 100644 --- a/App/Blurt/Blurt/Wizard/BoundTriggerKey.swift +++ b/App/Blurt/Blurt/Wizard/BoundTriggerKey.swift @@ -1,22 +1,22 @@ import BlurtEngine import SwiftUI -/// Live-updating read of the bound dictation trigger key, for the views that +/// Live-updating read of the bound dictation trigger, for the views that /// only *display* it ("Tap or hold right ⌘ to blurt"). /// -/// Wraps the `@AppStorage` + `TriggerKey.fromPersisted` pair that three views — -/// the ready screen, the menu bar menu, and the permissions footer — each spelled +/// Wraps the `@AppStorage` + `TriggerBinding.fromPersisted` pair that three views +/// — the ready screen, the menu bar menu, and the permissions footer — each spelled /// out, along with a restated `TriggerKey.rightCommand.rawValue` default that /// `fromPersisted` already owns (an unset default reads as 0, which it maps to -/// right ⌘). Reading the raw keycode through `@AppStorage`, rather than calling -/// `TriggerKeyStore()` once, is what makes these views re-render when the key is -/// rebound in the separate Settings window. +/// right ⌘). Reading the raw encoded slot through `@AppStorage`, rather than +/// calling `TriggerKeyStore()` once, is what makes these views re-render when the +/// trigger is rebound in the separate Settings window. /// /// A `DynamicProperty` rather than a `View` because two of the three call sites /// need the value inside a larger sentence, not a `Text` of its own. @propertyWrapper -struct BoundTriggerKey: DynamicProperty { - @AppStorage(TriggerKeyStore.defaultsKey) private var keyCode = 0 +struct BoundTriggerBinding: DynamicProperty { + @AppStorage(TriggerKeyStore.defaultsKey) private var code = 0 - var wrappedValue: TriggerKey { TriggerKey.fromPersisted(keyCode) } + var wrappedValue: TriggerBinding { TriggerBinding.fromPersisted(code) } } diff --git a/App/Blurt/Blurt/Wizard/DeveloperSection.swift b/App/Blurt/Blurt/Wizard/DeveloperSection.swift index 7e5eb277..2f3c81dc 100644 --- a/App/Blurt/Blurt/Wizard/DeveloperSection.swift +++ b/App/Blurt/Blurt/Wizard/DeveloperSection.swift @@ -26,7 +26,8 @@ struct DeveloperSection: View { // the first path is followed by a plain space rather than a comma. Text( "Logs each dictation to \(DictationLog.defaultDisplayPath) " - + "and each failure to \(DictationLog.defaultErrorDisplayPath)" + + "each failure to \(DictationLog.defaultErrorDisplayPath) " + + "and Custom-capture input events to \(DictationLog.defaultCaptureDisplayPath)" ) .textSelection(.enabled) } diff --git a/App/Blurt/Blurt/Wizard/ReadyView.swift b/App/Blurt/Blurt/Wizard/ReadyView.swift index 1ff6fe51..639c5c4c 100644 --- a/App/Blurt/Blurt/Wizard/ReadyView.swift +++ b/App/Blurt/Blurt/Wizard/ReadyView.swift @@ -8,8 +8,9 @@ struct ReadyView: View { var coordinator: AppCoordinator var openSettings: () -> Void // Observed (not read once) so changing the dictation key in the separate - // Settings window re-renders this window's keycap live — see `BoundTriggerKey`. - @BoundTriggerKey private var triggerKey + // Settings window re-renders this window's keycap live — see + // `BoundTriggerBinding`. + @BoundTriggerBinding private var trigger var body: some View { // Sections sit 20 pt apart; the logo and shortcut readout are one idea, @@ -53,7 +54,7 @@ struct ReadyView: View { HStack(spacing: 6) { Text("Tap or hold") .foregroundStyle(.secondary) - KeyCap(label: triggerKey.label) + KeyCap(label: trigger.label) Text("to blurt") .foregroundStyle(.secondary) } diff --git a/App/Blurt/Blurt/Wizard/Steps/HotkeyStepView.swift b/App/Blurt/Blurt/Wizard/Steps/HotkeyStepView.swift index b450dec4..a86bbabd 100644 --- a/App/Blurt/Blurt/Wizard/Steps/HotkeyStepView.swift +++ b/App/Blurt/Blurt/Wizard/Steps/HotkeyStepView.swift @@ -1,35 +1,79 @@ +import AppKit import BlurtEngine import SwiftUI /// The dictation-key section of the setup/settings screen. A menu picker lets -/// the user choose which lone modifier triggers dictation; changes are persisted -/// and pushed to the event tap immediately. +/// the user choose which lone modifier triggers dictation — or "Custom…", which +/// opens a press-to-capture sheet that binds a keyboard chord (⌃⌥D) or an extra +/// mouse button instead. Changes are persisted and pushed to the event tap +/// immediately. struct HotkeyStepView: View { var coordinator: AppCoordinator - // `0` is "no keycode persisted", not a default binding: the unset default belongs - // to `TriggerKey.fromPersisted` (below), which maps any unknown keycode to right + // `0` is "no binding persisted", not a default binding: the unset default belongs + // to `TriggerBinding.fromPersisted` (below), which maps any unknown code to right // ⌘. Restating `TriggerKey.rightCommand.rawValue` here would give the empty slot // two answers, and this one would win for an unset key — so a change to the // engine's default would leave this picker showing the old binding while the - // ready screen and menu bar showed the new one. Matches `@BoundTriggerKey`. - @AppStorage(TriggerKeyStore.defaultsKey) private var triggerKeyCode = 0 + // ready screen and menu bar showed the new one. Matches `@BoundTriggerBinding`. + @AppStorage(TriggerKeyStore.defaultsKey) private var triggerCode = 0 - private var selection: Binding { + @State private var isCapturing = false + + /// What the picker's menu rows are. A custom binding shows as its own row + /// (`bound`, e.g. "⌃⌥D" or "Mouse 4") so the picker always names what's bound, + /// while "Custom…" stays a separate, always-present row whose selection only + /// opens the capture sheet — re-selecting an already-selected row fires + /// nothing in a `Picker`, so folding the two into one row would make + /// "rebind my custom trigger to a different one" unreachable. + private enum Choice: Hashable { + case modifier(TriggerKey) + /// The currently bound custom (chord or mouse-button) binding — present only + /// while one is bound. + case bound + /// The "Custom…" row: selecting it opens the capture sheet. Never reads as + /// selected, because `selection`'s getter never returns it. + case capture + } + + private var binding: TriggerBinding { TriggerBinding.fromPersisted(triggerCode) } + + /// The label for the `bound` row, present only while a custom (non-modifier) + /// binding is active — the modifiers already have rows of their own. + private var boundCustomLabel: String? { + if case .modifier = binding { return nil } + return binding.label + } + + private var selection: Binding { Binding( get: { - TriggerKey.fromPersisted(triggerKeyCode) + switch TriggerBinding.fromPersisted(triggerCode) { + case .modifier(let key): return .modifier(key) + case .chord, .mouseButton: return .bound + } }, - set: { newValue in - // Write through the store, not the raw `@AppStorage` slot: the store owns - // how a `TriggerKey` is encoded, and `@AppStorage` is here to *observe* the - // key so this view re-renders (it picks up the store's external write). - // Assigning `triggerKeyCode` directly left `TriggerKeyStore`'s setter with - // no production caller, so a change to the encoding — versioning the key, - // storing the case name, a migration — would keep `swift test` green while - // the picker silently kept writing the old form. - TriggerKeyStore().triggerKey = newValue - coordinator.dictationBindingChanged() + set: { choice in + switch choice { + case .modifier(let key): + // Write through the store, not the raw `@AppStorage` slot: the store owns + // how a `TriggerBinding` is encoded, and `@AppStorage` is here to *observe* + // the key so this view re-renders (it picks up the store's external write). + // Assigning `triggerCode` directly left `TriggerKeyStore`'s setter with + // no production caller, so a change to the encoding — versioning the key, + // storing the case name, a migration — would keep `swift test` green while + // the picker silently kept writing the old form. + TriggerKeyStore().triggerBinding = .modifier(key) + coordinator.dictationBindingChanged() + case .capture: + // Nothing is persisted yet — the sheet writes the binding on a + // successful capture, and cancelling leaves the current one in place + // (the getter re-derives the selection from the persisted slot, so + // the picker snaps back on its own). + isCapturing = true + case .bound: + break // Re-selecting what's already bound changes nothing. + } }) } @@ -40,13 +84,218 @@ struct HotkeyStepView: View { accessibilityID: UITestIdentifiers.hotkeyPicker, selection: selection ) { ForEach(TriggerKey.allCases, id: \.self) { key in - Text(key.label).tag(key) + Text(key.label).tag(Choice.modifier(key)) } + if let boundCustomLabel { + Text(boundCustomLabel).tag(Choice.bound) + } + Text("Custom…").tag(Choice.capture) } } header: { Text("Shortcut") } footer: { - Text("Tap to start and tap again to stop, or hold the key and release to dictate.") + // The pass-through caution belongs to the *binding*, so its wording lives + // in the engine (`TriggerBinding.passThroughNote`) beside the tap's + // listen-only contract rather than being restated per view — and it sits + // in the footer, not the capture sheet, because it stays true for as long + // as the binding is bound. + VStack(alignment: .leading, spacing: 4) { + Text("Tap to start and tap again to stop, or hold the key and release to dictate.") + if let note = binding.passThroughNote { + Text(note) + } + } + } + .sheet(isPresented: $isCapturing) { + CustomTriggerCaptureView { captured in + TriggerKeyStore().triggerBinding = captured + coordinator.dictationBindingChanged() + } + } + } +} + +/// The press-to-capture sheet behind the picker's "Custom…" row: it records the +/// next **keyboard chord** (modifiers plus a key, e.g. ⌃⌥D) or **extra mouse +/// button** click and binds it as the dictation trigger. Esc closes without +/// changing the binding. +/// +/// What it accepts is `TriggerBinding`'s policy, not this view's +/// (`chordBinding(forKeyCode:modifiers:)` / `mouseButtonBinding(forButton:)`), +/// and each refusal maps to one sentence here. A **bare** key is refused because +/// the dictation tap is listen-only and swallows nothing, so a bound letter +/// would type into the focused app on every dictation; a handful of +/// system-reserved chords (⌘Q, ⌘⇥, ⌘Space…) are refused because they'd fire +/// their system action underneath the dictation. Left/right clicks structurally +/// never arrive (they aren't `.otherMouseDown`, and they're how the user clicks +/// at all). +/// +/// Capture uses `NSEvent` monitors, not the dictation `CGEventTap`: local +/// monitors for `keyDown` and `flagsChanged` (returning nil so a captured or +/// refused press doesn't also type, beep, or fire its own shortcut in this +/// window) plus local and global `otherMouseDown` monitors, since mouse buttons +/// don't focus-follow. The global monitor needs the Accessibility grant, which +/// the app required before Settings was reachable. +/// +/// Note what a local monitor cannot do: while this sheet is key, a chord that +/// macOS itself owns (⌘⇥, ⌘Space) is consumed by the system before any app +/// monitor sees it — which is the same reason those chords are refused rather +/// than merely discouraged. +private struct CustomTriggerCaptureView: View { + /// Called with the captured binding after the sheet dismisses itself. + var onCapture: (TriggerBinding) -> Void + + @Environment(\.dismiss) private var dismiss + @State private var refusal: String? + /// The modifiers held right now, so the sheet shows the chord forming (⌃⌥…) + /// before the key lands — the live feedback that makes a chord recorder + /// legible rather than a blind prompt. + @State private var heldModifiers: TriggerBinding.ChordModifiers = [] + @State private var monitors: [Any] = [] + + var body: some View { + VStack(spacing: 12) { + Text("Set a custom dictation trigger") + .font(.headline) + Text("Hold modifiers and press a key (⌃⌥D), or press an extra mouse button…") + .foregroundStyle(.secondary) + .multilineTextAlignment(.center) + // The forming chord, or a placeholder of the same height so the sheet + // doesn't jump when the first modifier goes down. + Text(heldModifiers.isEmpty ? " " : heldModifiers.glyphs) + .font(.system(size: 28, weight: .regular)) + .monospaced() + if let refusal { + Text(refusal) + .font(.callout) + .foregroundStyle(.red) + .multilineTextAlignment(.center) + } + Text("Blurt never intercepts the trigger, so an app that already uses it still gets it.") + .font(.caption) + .foregroundStyle(.tertiary) + .multilineTextAlignment(.center) + Button("Cancel") { dismiss() } + } + .padding(24) + .frame(width: 360) + .onAppear(perform: startListening) + .onDisappear(perform: stopListening) + } + + private func startListening() { + let localKey = NSEvent.addLocalMonitorForEvents(matching: .keyDown) { event in + handleKeyDown(event) + return nil // consume: a captured/refused press must not also type or beep + } + let localFlags = NSEvent.addLocalMonitorForEvents(matching: .flagsChanged) { event in + heldModifiers = Self.modifiers(from: event.modifierFlags) + return nil + } + let localClick = NSEvent.addLocalMonitorForEvents(matching: .otherMouseDown) { event in + handleClick(event) + return nil + } + let globalClick = NSEvent.addGlobalMonitorForEvents(matching: .otherMouseDown) { event in + handleClick(event) + } + monitors = [localKey, localFlags, localClick, globalClick].compactMap { $0 } + } + + private func stopListening() { + for monitor in monitors { NSEvent.removeMonitor(monitor) } + monitors = [] + } + + private func handleKeyDown(_ event: NSEvent) { + let escapeKeyCode = 53 + let keyCode = Int(event.keyCode) + let modifiers = Self.modifiers(from: event.modifierFlags) + // Esc always cancels, with or without modifiers: it is the sheet's dismiss + // gesture, and ⌘⌥Esc is Force Quit — neither is a trigger worth binding. + if keyCode == escapeKeyCode { + log(event, kind: "keyDown", outcome: "cancelled") + dismiss() + return + } + switch TriggerBinding.chordBinding(forKeyCode: keyCode, modifiers: modifiers) { + case .success(let captured): + log(event, kind: "keyDown", outcome: "captured", binding: captured.label) + dismiss() + onCapture(captured) + case .failure(let reason): + log(event, kind: "keyDown", outcome: Self.outcome(for: reason)) + refusal = Self.message(for: reason) + } + } + + private func handleClick(_ event: NSEvent) { + // Left/right clicks never arrive (they're not `.otherMouseDown`), so this + // refusal covers only button numbers past what a binding can express. + guard let captured = TriggerBinding.mouseButtonBinding(forButton: event.buttonNumber) else { + log(event, kind: "otherMouseDown", outcome: "refused-button") + refusal = "That button can't be the trigger. Press a mouse button other than left or right click." + return + } + log(event, kind: "otherMouseDown", outcome: "captured", binding: captured.label) + dismiss() + onCapture(captured) + } + + /// The engine's refusal reasons as the sentences this sheet shows. Kept beside + /// the outcome tokens below so the copy and the diagnostics token for a reason + /// are chosen in one place. + private static func message(for reason: TriggerBinding.ChordRefusal) -> String { + switch reason { + case .bareKey: + return + "Hold at least one modifier (⌃, ⌥, ⇧ or ⌘). A key on its own would type into whatever " + + "app is focused every time you dictate." + case .modifierOnly: + return "Add a regular key to the modifiers — or pick right ⌘ / right ⌥ from the menu instead." + case .reserved: + return "That shortcut belongs to macOS, so it would act while you dictate. Try another." + } + } + + /// The stable diagnostics token for a refusal (see `DictationLog.CapturedInput`). + private static func outcome(for reason: TriggerBinding.ChordRefusal) -> String { + switch reason { + case .bareKey: return "refused-bare-key" + case .modifierOnly: return "refused-modifier-only" + case .reserved: return "refused-reserved-chord" } } + + /// `NSEvent.ModifierFlags` → the engine's side-agnostic chord set, mirroring + /// `DictationEventDecoder.modifiers(from:)` for the `CGEvent` side. Caps Lock + /// and `fn` are deliberately not chord modifiers. + private static func modifiers( + from flags: NSEvent.ModifierFlags + ) -> TriggerBinding.ChordModifiers { + var modifiers: TriggerBinding.ChordModifiers = [] + if flags.contains(.control) { modifiers.insert(.control) } + if flags.contains(.option) { modifiers.insert(.option) } + if flags.contains(.shift) { modifiers.insert(.shift) } + if flags.contains(.command) { modifiers.insert(.command) } + return modifiers + } + + /// Developer-mode diagnostics: every event the recorder sees — accepted or + /// refused — lands in `capture-events.jsonl` with its raw facts (button + /// number, keycode, flags, autorepeat), so a misbehaving multi-button mouse or + /// a chord the system ate can be diagnosed from what actually arrived. Gated + /// inside `appendCaptureEvent` on the same developer-mode switch as the other + /// logs; with the switch off this writes nothing. + private func log(_ event: NSEvent, kind: String, outcome: String, binding: String? = nil) { + let isKeyEvent = kind == "keyDown" + let input = DictationLog.CapturedInput( + kind: kind, + button: isKeyEvent ? nil : event.buttonNumber, + keyCode: isKeyEvent ? Int(event.keyCode) : nil, + flags: UInt64(event.modifierFlags.rawValue), + // `isARepeat` raises on non-key events, so it's only read for key ones. + isRepeat: isKeyEvent && event.isARepeat) + DictationLog.appendCaptureEvent(input, outcome: outcome, binding: binding) + } } diff --git a/App/Blurt/Blurt/Wizard/Steps/PermissionsStepView.swift b/App/Blurt/Blurt/Wizard/Steps/PermissionsStepView.swift index f4461a5a..80dfa6fd 100644 --- a/App/Blurt/Blurt/Wizard/Steps/PermissionsStepView.swift +++ b/App/Blurt/Blurt/Wizard/Steps/PermissionsStepView.swift @@ -12,8 +12,8 @@ struct PermissionsStepView: View { /// Observed rather than read once via `TriggerKeyStore()`: Settings is reachable /// with ⌘, while this page is showing, so a one-shot read left the footer naming /// the old key after a rebind until something else re-rendered the view. Same - /// `BoundTriggerKey` as `ReadyView` / `MenuBarScene`. - @BoundTriggerKey private var triggerKey + /// `BoundTriggerBinding` as `ReadyView` / `MenuBarScene`. + @BoundTriggerBinding private var trigger /// Set when the user taps a settings button so the section can show a /// "waiting for you to come back" cue until the poll sees the grant. @@ -57,7 +57,7 @@ struct PermissionsStepView: View { granted: controller.permissions.microphone, waiting: "Waiting for you to turn on Blurt under Microphone…", description: - "Blurt records only after you start dictating with \(triggerKey.label)." + "Blurt records only after you start dictating with \(trigger.label)." ) } } diff --git a/Sources/BlurtEngine/Audio/SoundPack.swift b/Sources/BlurtEngine/Audio/SoundPack.swift index f0651b85..0379b935 100644 --- a/Sources/BlurtEngine/Audio/SoundPack.swift +++ b/Sources/BlurtEngine/Audio/SoundPack.swift @@ -58,7 +58,7 @@ public struct SoundPack: Sendable, Hashable, Identifiable { /// unset or names no known pack. The single decode-with-default rule shared by /// `SoundPackStore` and the `@AppStorage` views that read the raw id directly /// (so they re-render live on a Settings change) — mirroring - /// `TriggerKey.fromPersisted`. + /// `TriggerBinding.fromPersisted`. public static func fromPersisted(_ id: String?) -> SoundPack { guard let id, let pack = find(id: id) else { return .defaultPack } return pack diff --git a/Sources/BlurtEngine/Config/DeveloperModeStore.swift b/Sources/BlurtEngine/Config/DeveloperModeStore.swift index 1dbfaeca..9c90c27c 100644 --- a/Sources/BlurtEngine/Config/DeveloperModeStore.swift +++ b/Sources/BlurtEngine/Config/DeveloperModeStore.swift @@ -2,8 +2,9 @@ import Foundation /// Persists the developer-mode switch in `UserDefaults`. Off by default; the /// Settings window's Developer section flips it. While on, each completed -/// dictation is appended to `DictationLog` — that gate is the switch's only -/// effect, so a user who never opts in has no dictation text on disk. +/// dictation, each failure, and each Custom-capture input event is appended to +/// its `DictationLog` file — those gates are the switch's only effect, so a +/// user who never opts in has nothing on disk. /// Same shape as `TriggerKeyStore` / `SoundPackStore`. public struct DeveloperModeStore { /// UserDefaults key holding the switch. Public so SwiftUI views can observe diff --git a/Sources/BlurtEngine/Hotkey/DictationEventDecoder.swift b/Sources/BlurtEngine/Hotkey/DictationEventDecoder.swift new file mode 100644 index 00000000..081340c9 --- /dev/null +++ b/Sources/BlurtEngine/Hotkey/DictationEventDecoder.swift @@ -0,0 +1,64 @@ +import CoreGraphics + +/// Reduces a `CGEventTap` delivery to `DictationKeyRouter`'s CoreGraphics-free +/// event shape — the one deliberately CoreGraphics-typed piece of the hotkey +/// engine, split out of the app's tap shim so it can be exercised with *real* +/// `CGEvent` fixtures (`DictationEventDecoderTests` constructs events with +/// `CGEvent(mouseEventSource:…)`/`CGEvent(keyboardEventSource:…)` and asserts +/// the router events they decode to). The router itself stays CoreGraphics-free +/// on purpose; this is the boundary where `CGEvent` ends. +/// +/// The type is passed alongside the event (rather than read off it) because +/// that is what a `CGEventTapCallBack` receives — the two can genuinely differ +/// during tap-disable deliveries, and the callback's `type` is the authority. +public enum DictationEventDecoder { + /// The router event a tap delivery reduces to, or nil for event types the + /// trigger doesn't care about. `triggerFlag` is the bound modifier's + /// device-dependent `CGEventFlags` bit, and is **empty under a chord or mouse + /// binding** — where the resulting `triggerFlagIsOn` is meaningless and must + /// not be read: `CGEventFlags.contains([])` is vacuously true, so an empty + /// trigger flag reports "on" for every delivery. The router honors that (only + /// `handleForModifier` looks at the field); `DictationEventDecoderTests` pins + /// it so it can't be mistaken for a bug later. + public static func routerEvent( + type: CGEventType, event: CGEvent, triggerFlag: CGEventFlags + ) -> DictationKeyRouter.Event? { + switch type { + case .flagsChanged: + return .flagsChanged( + keyCode: Int(event.getIntegerValueField(.keyboardEventKeycode)), + triggerFlagIsOn: event.flags.contains(triggerFlag), + modifiers: modifiers(from: event.flags)) + case .keyDown: + // Autorepeat is not an edge: a held chord key repeats at the system rate, + // and every repeat would otherwise walk the router's dedup. Dropped here so + // the repeat storm never reaches the engine at all. + guard event.getIntegerValueField(.keyboardEventAutorepeat) == 0 else { return nil } + return .keyDown( + keyCode: Int(event.getIntegerValueField(.keyboardEventKeycode)), + modifiers: modifiers(from: event.flags)) + case .keyUp: + return .keyUp(keyCode: Int(event.getIntegerValueField(.keyboardEventKeycode))) + case .otherMouseDown: + return .mouseDown(button: Int(event.getIntegerValueField(.mouseEventButtonNumber))) + case .otherMouseUp: + return .mouseUp(button: Int(event.getIntegerValueField(.mouseEventButtonNumber))) + default: + return nil + } + } + + /// The side-agnostic chord modifier set a `CGEvent`'s flags carry. Reads the + /// **generic** masks (`maskControl` etc.), not the per-side device bits + /// `TriggerKey` uses: a chord is a shortcut, so ⌃ is ⌃ whichever one is held. + /// Caps Lock and `fn` are deliberately not chord modifiers — Caps Lock is a + /// latch rather than a held key, and `fn` is macOS's own. + public static func modifiers(from flags: CGEventFlags) -> TriggerBinding.ChordModifiers { + var modifiers: TriggerBinding.ChordModifiers = [] + if flags.contains(.maskControl) { modifiers.insert(.control) } + if flags.contains(.maskAlternate) { modifiers.insert(.option) } + if flags.contains(.maskShift) { modifiers.insert(.shift) } + if flags.contains(.maskCommand) { modifiers.insert(.command) } + return modifiers + } +} diff --git a/Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift b/Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift index 589ee3ed..9e6d7aa4 100644 --- a/Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift +++ b/Sources/BlurtEngine/Hotkey/DictationKeyRouter.swift @@ -1,13 +1,26 @@ -/// Routes raw trigger-key events into `DictationKeyGate` and owns the three +/// Routes raw trigger events into `DictationKeyGate` and owns the three /// decisions that would otherwise sit untested in the app's event-tap shim: /// /// - **Edge dedup.** `flagsChanged` deliveries re-report the bound key's flag -/// bit whether or not it changed, so the router tracks the modifier's current +/// bit whether or not it changed, a held key autorepeats, and a held mouse +/// button can be re-reported, so the router tracks the trigger's current /// physical state and only a genuine down/up *edge* reaches the gate — /// repeated same-state deliveries must not double-fire a dictation. -/// - **Relevance.** Only the bound keycode's flag changes drive the modifier; -/// a `keyDown` for any *other* key marks a combo (e.g. ⌘C over the held -/// trigger), and the trigger's own keycode never counts as a combo. +/// - **Relevance.** Only events for the bound trigger drive the gate, and which +/// event family that is follows the binding: a modifier binding listens to +/// `flagsChanged`, a **chord** binding to `keyDown`/`keyUp` plus the modifier +/// set each event reports, a mouse binding to `mouseDown`/`mouseUp`. For a +/// **modifier** binding, a `keyDown` for any *other* key marks a combo (e.g. +/// ⌘C over the held trigger) and cancels the fresh capture; the trigger's own +/// keycode never counts as a combo. For a chord or mouse binding there is no +/// combo rule, and there must not be: a chord's own modifiers are part of the +/// trigger, and another key pressed during a latched dictation is just typing +/// — cancelling on it would punish exactly the input dictation produces. +/// - **Chord completeness.** A chord fires only while the **exact** required +/// modifier set is held (not a superset: ⇧ added to ⌃⌥D usually means the user +/// meant a different shortcut), and it ends on the bound key's `keyUp` *or* as +/// soon as any required modifier is released — releasing ⌃ while still holding +/// D ends the press, because the chord is no longer being held. /// - **Dropped-event recovery.** After the host's tap is disabled and re-enabled, /// whether the gate's state survives depends on the trigger still being held — /// see `recoverFromDroppedEvents(triggerStillHeld:)`. @@ -16,73 +29,160 @@ /// — so every decision is deterministic and unit-testable. The app-side /// `DictationKeyTap` reduces each `CGEvent` to an `Event` and forwards it here. public struct DictationKeyRouter: Sendable { - /// A keyboard event reduced to exactly what the routing decision needs, so + /// An input event reduced to exactly what the routing decision needs, so /// the router never touches `CGEvent`/`CGEventFlags` types. public enum Event: Sendable, Equatable { - /// A `flagsChanged` delivery: the keycode it reports and whether the bound + /// A `flagsChanged` delivery: the keycode it reports, whether the bound /// trigger's device-dependent flag bit is set in the event's flags (see - /// `TriggerKey.deviceModifierMask`). - case flagsChanged(keyCode: Int, triggerFlagIsOn: Bool) - /// A `keyDown` for `keyCode`. - case keyDown(keyCode: Int) + /// `TriggerKey.deviceModifierMask` — meaningful only under a modifier + /// binding), and the side-agnostic modifier set now held, which is what a + /// chord binding watches for a required modifier being released. + case flagsChanged(keyCode: Int, triggerFlagIsOn: Bool, modifiers: TriggerBinding.ChordModifiers) + /// A non-autorepeat `keyDown` for `keyCode` with the modifier set held at the + /// time — a chord's trigger-down, and the combo probe for a modifier + /// binding. (The host drops autorepeat deliveries; the edge filter here + /// backstops any that slip through.) + case keyDown(keyCode: Int, modifiers: TriggerBinding.ChordModifiers) + /// A `keyUp` for `keyCode` — a chord's trigger-up. + case keyUp(keyCode: Int) + /// An `otherMouseDown` for the given `CGEvent` button number. + case mouseDown(button: Int) + /// An `otherMouseUp` for the given `CGEvent` button number. + case mouseUp(button: Int) + + /// A `flagsChanged` delivery with no chord modifiers held — the shorthand a + /// modifier binding's call sites want, since enum cases can't carry default + /// associated values and the chord set is meaningless to them. + public static func flagsChanged(keyCode: Int, triggerFlagIsOn: Bool) -> Event { + .flagsChanged(keyCode: keyCode, triggerFlagIsOn: triggerFlagIsOn, modifiers: []) + } + + /// A `keyDown` with no modifiers held — the plain "some other key went down" + /// combo probe. + public static func keyDown(keyCode: Int) -> Event { + .keyDown(keyCode: keyCode, modifiers: []) + } } - /// The virtual keycode of the bound trigger modifier (`TriggerKey.keyCode`). - public private(set) var triggerKeyCode: Int + /// The bound trigger (`TriggerKeyStore.triggerBinding`). + public private(set) var binding: TriggerBinding private var gate: DictationKeyGate - /// The bound modifier's current physical state, so repeated `flagsChanged` - /// deliveries with an unchanged bit don't re-fire the gate. - private var modifierIsDown = false + /// The bound trigger's current physical state, so repeated same-state + /// deliveries (an unchanged flag bit, a re-reported button-down) don't + /// re-fire the gate. + private var triggerIsDown = false - public init(triggerKeyCode: Int, holdThreshold: Duration = .seconds(1)) { - self.triggerKeyCode = triggerKeyCode + public init(binding: TriggerBinding, holdThreshold: Duration = .seconds(1)) { + self.binding = binding self.gate = DictationKeyGate(holdThreshold: holdThreshold) } /// Feeds one event through the relevance/edge filters into the gate and /// returns its decision. public mutating func handle(_ event: Event, at now: Duration) -> DictationKeyGate.Action { + switch binding { + case .modifier(let key): + return handleForModifier(key, event, at: now) + case .chord(let keyCode, let modifiers): + return handleForChord(keyCode: keyCode, required: modifiers, event, at: now) + case .mouseButton(let button): + return handleForMouseButton(button, event, at: now) + } + } + + private mutating func handleForModifier( + _ key: TriggerKey, _ event: Event, at now: Duration + ) -> DictationKeyGate.Action { switch event { - case .flagsChanged(let keyCode, let triggerFlagIsOn): - guard keyCode == triggerKeyCode else { return .none } - if triggerFlagIsOn, !modifierIsDown { - modifierIsDown = true - return gate.modifierDown(at: now) - } - if !triggerFlagIsOn, modifierIsDown { - modifierIsDown = false - return gate.modifierUp(at: now) - } + case .flagsChanged(let keyCode, let triggerFlagIsOn, _): + guard keyCode == key.keyCode else { return .none } + return edge(isDown: triggerFlagIsOn, at: now) + case .keyDown(let keyCode, _): + // Another key over the held modifier is a combo (a real shortcut). + return keyCode == key.keyCode ? .none : gate.otherKeyDown() + case .keyUp, .mouseDown, .mouseUp: return .none - case .keyDown(let keyCode): - return keyCode == triggerKeyCode ? .none : gate.otherKeyDown() } } + /// A chord binding: the bound key's non-autorepeat `keyDown` while exactly the + /// required modifiers are held is trigger-down; its `keyUp`, or the release of + /// any required modifier, is trigger-up. + private mutating func handleForChord( + keyCode boundKeyCode: Int, required: TriggerBinding.ChordModifiers, + _ event: Event, at now: Duration + ) -> DictationKeyGate.Action { + switch event { + case .keyDown(let keyCode, let modifiers): + // Exact match, not a superset: ⇧ added to ⌃⌥D is a different shortcut, and + // firing on it would make the trigger unpredictable rather than generous. + guard keyCode == boundKeyCode, modifiers == required else { return .none } + return edge(isDown: true, at: now) + case .keyUp(let keyCode): + guard keyCode == boundKeyCode else { return .none } + return edge(isDown: false, at: now) + case .flagsChanged(_, _, let modifiers): + // The chord stops being held the moment a required modifier goes up — the + // key's own `keyUp` may never arrive in that order, and waiting for it + // would leave the gate armed with nothing coming. + guard triggerIsDown, !modifiers.isSuperset(of: required) else { return .none } + return edge(isDown: false, at: now) + case .mouseDown, .mouseUp: + return .none + } + } + + private mutating func handleForMouseButton( + _ boundButton: Int, _ event: Event, at now: Duration + ) -> DictationKeyGate.Action { + switch event { + case .mouseDown(let button): + return button == boundButton ? edge(isDown: true, at: now) : .none + case .mouseUp(let button): + return button == boundButton ? edge(isDown: false, at: now) : .none + case .flagsChanged, .keyDown, .keyUp: + return .none + } + } + + /// The shared edge filter: only a genuine change of the trigger's physical + /// state reaches the gate, whatever event family reported it. + private mutating func edge(isDown: Bool, at now: Duration) -> DictationKeyGate.Action { + if isDown, !triggerIsDown { + triggerIsDown = true + return gate.modifierDown(at: now) + } + if !isDown, triggerIsDown { + triggerIsDown = false + return gate.modifierUp(at: now) + } + return .none + } + /// Rebinds the trigger and resets: events already tracked belong to the old - /// key, whose up-event can no longer match. Returns whether the reset + /// binding, whose up-event can no longer match. Returns whether the reset /// discarded a live recording (see `reset()`). @discardableResult - public mutating func rebind(triggerKeyCode: Int) -> Bool { - self.triggerKeyCode = triggerKeyCode + public mutating func rebind(binding: TriggerBinding) -> Bool { + self.binding = binding return reset() } /// Recovery after the host's event tap was disabled (by timeout, or by user input /// while it was down) and re-enabled: events may have been dropped, so the gate's - /// state may no longer match the keyboard. + /// state may no longer match the input device. /// - /// `triggerStillHeld` is the caller's read of whether the trigger modifier is - /// physically down *right now* — `CGEventSource.flagsState` on the app side, the - /// one CoreGraphics-typed input, which is why it's passed in rather than read - /// here. If it is, nothing that matters was lost: the key-up is still coming and + /// `triggerStillHeld` is the caller's read of whether the trigger is physically + /// down *right now* — `CGEventSource.flagsState`/`keyState`/`buttonState` on the app side, + /// the one CoreGraphics-typed input, which is why it's passed in rather than read + /// here. If it is, nothing that matters was lost: the up-event is still coming and /// the gate is coherent, so the state is kept — resetting would discard speech the - /// user is mid-sentence on. If it isn't, the trigger's key-up may have been among - /// the dropped events, so the gate is reset. + /// user is mid-sentence on. If it isn't, the trigger's up-event may have been + /// among the dropped events, so the gate is reset. /// /// Returns whether that reset discarded a live recording the caller must cancel - /// upstream, matching `reset()` and `rebind(triggerKeyCode:)`. This lived in the + /// upstream, matching `reset()` and `rebind(binding:)`. This lived in the /// shell as a bare `if`, where nothing could test it — the app target has no test /// target and a `CGEventTap` can't be driven from XCUITest — while carrying the /// worst failure of the three decisions here: a session left in `.recording` with @@ -94,7 +194,7 @@ public struct DictationKeyRouter: Sendable { return reset() } - /// Clears the gate (and the modifier-down tracker) because the events it was + /// Clears the gate (and the trigger-down tracker) because the events it was /// tracking can no longer be trusted — the binding changed, or the host's /// event tap was disabled and events were dropped. Returns true when the /// reset discarded a live gate state (armed or latched): no future key event @@ -105,7 +205,7 @@ public struct DictationKeyRouter: Sendable { public mutating func reset() -> Bool { let discardedRecording = !gate.isIdle gate.reset() - modifierIsDown = false + triggerIsDown = false return discardedRecording } } diff --git a/Sources/BlurtEngine/Hotkey/TriggerBinding.swift b/Sources/BlurtEngine/Hotkey/TriggerBinding.swift new file mode 100644 index 00000000..65afc82e --- /dev/null +++ b/Sources/BlurtEngine/Hotkey/TriggerBinding.swift @@ -0,0 +1,300 @@ +/// What the user bound as the dictation trigger: a lone modifier key (the +/// default family — see `TriggerKey`), a **keyboard chord** (modifiers plus one +/// non-modifier key, e.g. ⌃⌥D), or an extra mouse button. +/// +/// Every binding encodes into the single persisted `Int` slot `TriggerKeyStore` +/// has always used (`BlurtTriggerKeyCode`), so existing installs migrate with no +/// data change and `@AppStorage` views keep observing one key: +/// +/// - A modifier stores its virtual keycode (54/61), exactly as before — and the +/// removed `fn` option's 63 migrates to right ⌥. +/// - A mouse button stores `mouseButtonCodeBase + buttonNumber`. +/// - A chord packs both halves into one tagged value: +/// `chordCodeBase | (modifiers << chordModifierShift) | keyCode` — the keycode +/// in the low byte, the four modifier bits above it. Virtual keycodes are +/// 16-bit and the two bases are far above them, so the three families can't +/// collide (`TriggerBindingTests` pins that). +/// +/// The curation is policy, not a parsing limit. A **bare** key is refused: the +/// event tap is listen-only and swallows nothing, so it would type into the +/// focused app on every dictation. A chord with modifiers is far less likely to +/// insert text, but it is **not** swallowed either — a chord an app already owns +/// still reaches that app, which is why the recorder also refuses a small set of +/// system-reserved chords and the UI says so plainly. Mouse buttons 0/1 are how +/// the user operates the machine, so only Mouse 3 and up are bindable. +public enum TriggerBinding: Sendable, Hashable { + /// A lone modifier (right ⌘ or right ⌥) — the original trigger family, + /// driven by `flagsChanged` events. + case modifier(TriggerKey) + /// A keyboard chord: one non-modifier key plus the exact modifier set that + /// must be held with it. Driven by `keyDown`/`keyUp` plus modifier changes. + case chord(keyCode: Int, modifiers: ChordModifiers) + /// An extra mouse button, by **raw `CGEvent`/`NSEvent` button number**, which + /// is 0-based: 0 is the left click, 1 the right click, 2 the wheel/middle + /// click, 3+ the side buttons. Bindable from 2 up. The *display* name is + /// 1-based, the way mice and their drivers number buttons for users — so the + /// stored `2` shows as "Mouse 3" and IS the middle (wheel) click. Driven by + /// `otherMouseDown`/`otherMouseUp` events. + case mouseButton(Int) + + /// The modifier keys a chord can require, as a stable bit set — the layout is + /// persisted, so the raw values are a storage contract, not an implementation + /// detail. Deliberately side-agnostic (left and right ⌃ are the same + /// requirement): a chord is a shortcut, and no shortcut in macOS distinguishes + /// the sides. `TriggerKey` keeps the per-side device bits for the lone-modifier + /// family, where telling the sides apart is the whole point. + public struct ChordModifiers: OptionSet, Sendable, Hashable { + public let rawValue: Int + + public init(rawValue: Int) { + self.rawValue = rawValue + } + + public static let control = ChordModifiers(rawValue: 1 << 0) + public static let option = ChordModifiers(rawValue: 1 << 1) + public static let shift = ChordModifiers(rawValue: 1 << 2) + public static let command = ChordModifiers(rawValue: 1 << 3) + + /// Every bit the persisted encoding can carry — the mask the decoder + /// validates against, so a stray high bit reads as garbage rather than as a + /// modifier that doesn't exist. + static let all: ChordModifiers = [.control, .option, .shift, .command] + + /// The glyphs in Apple's canonical order (⌃⌥⇧⌘), so a bound chord reads + /// the way the same shortcut reads in any macOS menu. + public var glyphs: String { + var text = "" + if contains(.control) { text += "⌃" } + if contains(.option) { text += "⌥" } + if contains(.shift) { text += "⇧" } + if contains(.command) { text += "⌘" } + return text + } + } + + /// Where the mouse-button namespace starts in the persisted slot. Virtual + /// keycodes are 16-bit, so `0x10000 + button` can never collide with one. + static let mouseButtonCodeBase = 0x10000 + + /// Where the chord namespace starts — one tag above the mouse namespace, whose + /// values stop at `mouseButtonCodeBase + maximumMouseButton`. + static let chordCodeBase = 0x20000 + /// How far the modifier bits sit above the keycode in a packed chord: the + /// keycode occupies the low byte (virtual keycodes are 0–127), the four + /// modifier bits the byte above it. + static let chordModifierShift = 8 + /// The keycode half of a packed chord. + static let chordKeyCodeMask = 0xFF + + /// `fn`'s virtual keycode, kept only as a **migration** input: `fn` was once a + /// third `TriggerKey` option and was removed, so an install that persisted it + /// decodes to right ⌥ rather than to the generic right-⌘ garbage fallback — + /// it's the other right-side modifier, so the user keeps a one-key trigger on + /// the same side of the keyboard instead of being moved onto ⌘. + static let legacyFunctionKeyCode = 63 + + /// The wheel/middle click's raw button number — the one bindable button apps + /// commonly act on themselves (open-link-in-new-tab, paste-on-middle-click). + /// Bindable, but `passThroughNote` cautions about it. + static let middleMouseButton = 2 + + /// The lowest bindable button number: 2 ("Mouse 3"). Buttons 0 and 1 are the + /// left and right click — binding either would fire dictation on ordinary + /// mousing, so the recorder refuses them. + static let minimumMouseButton = 2 + + /// The highest bindable button number. `CGEvent` reports up to 32 buttons + /// (numbers 0–31), so anything past that in the persisted slot is garbage. + static let maximumMouseButton = 31 + + /// The virtual keycodes of the modifier keys themselves. A chord's key half + /// must be a *non-modifier* key: "⌃⌥ plus ⌘" is not a chord, and a lone + /// modifier is already its own binding family. + static let modifierKeyCodes: Set = [ + 54, 55, // right ⌘, left ⌘ + 56, 60, // left ⇧, right ⇧ + 58, 61, // left ⌥, right ⌥ + 59, 62, // left ⌃, right ⌃ + 57, // Caps Lock + 63, // fn + ] + + /// System chords the recorder refuses outright: each is a shortcut macOS or + /// the frontmost app acts on *while dictation is running*, and the tap is + /// listen-only, so binding one would both trigger dictation and quit the app / + /// switch away / open Spotlight underneath it. Small and explicit rather than + /// a heuristic — a rule nobody can predict is worse than a short list. + static let reservedChords: Set = [ + packedChord(keyCode: 12, modifiers: .command), // ⌘Q — quits the app + packedChord(keyCode: 13, modifiers: .command), // ⌘W — closes the window + packedChord(keyCode: 4, modifiers: .command), // ⌘H — hides the app + packedChord(keyCode: 48, modifiers: .command), // ⌘⇥ — switches apps + packedChord(keyCode: 49, modifiers: .command), // ⌘Space — Spotlight + packedChord(keyCode: 12, modifiers: [.control, .command]), // ⌃⌘Q — locks the screen + ] + + /// Decodes the persisted slot into a binding, falling back to the right-⌘ + /// modifier for anything unrecognized — an unset slot (0), a keycode from a + /// removed option, a mouse code outside the bindable range, or a chord whose + /// packed halves are structurally invalid. The single decode-with-default rule + /// shared by `TriggerKeyStore` and the `@AppStorage` views that read the raw + /// slot directly (so they re-render live on a Settings change). + public static func fromPersisted(_ code: Int) -> TriggerBinding { + if let key = TriggerKey(rawValue: code) { return .modifier(key) } + // The one migration this decoder carries: `fn` was a removed option, and its + // keycode must land on a deliberate replacement rather than the default. + if code == legacyFunctionKeyCode { return .modifier(.rightOption) } + if let binding = mouseButtonBinding(forButton: code - mouseButtonCodeBase) { return binding } + if let binding = chordBinding(packed: code) { return binding } + return .modifier(.rightCommand) + } + + /// The binding for a captured mouse button number, or nil when that button + /// isn't bindable (0/1 are the left/right click; past 31 `CGEvent` can't + /// report it). + public static func mouseButtonBinding(forButton button: Int) -> TriggerBinding? { + (minimumMouseButton...maximumMouseButton).contains(button) ? .mouseButton(button) : nil + } + + /// Why a captured chord can't be bound, or nil when it can. The capture UI's + /// single policy check — which chords are allowed is engine logic with tests + /// rather than a view's private list, and the reasons are the copy the sheet + /// shows. + /// Conforms to `Error` because it is the failure half of the `Result` the + /// capture policy returns — a refusal is a *decision with a reason*, and the + /// reason is what the sheet turns into a sentence, so it must survive the call + /// rather than collapsing into `nil`. + public enum ChordRefusal: Error, Sendable, Hashable { + /// No modifier was held. A bare key would type into the focused app on + /// every dictation, because the tap swallows nothing. + case bareKey + /// The "key" was itself a modifier (or Caps Lock / `fn`). A lone modifier is + /// its own binding family; a chord needs a real key. + case modifierOnly + /// A system-reserved chord (`reservedChords`) — it would fire its system + /// action underneath the dictation. + case reserved + } + + /// The binding for a captured chord, or the reason it's refused. `modifiers` + /// is the exact set that must be held with `keyCode` for the trigger to fire. + public static func chordBinding( + forKeyCode keyCode: Int, modifiers: ChordModifiers + ) -> Result { + if modifierKeyCodes.contains(keyCode) { return .failure(.modifierOnly) } + if modifiers.isEmpty { return .failure(.bareKey) } + if reservedChords.contains(packedChord(keyCode: keyCode, modifiers: modifiers)) { + return .failure(.reserved) + } + return .success(.chord(keyCode: keyCode, modifiers: modifiers)) + } + + /// The packed slot value for a chord — the encoding in one place, so the + /// reserved-chord table above and `persistedValue` below can't drift apart. + static func packedChord(keyCode: Int, modifiers: ChordModifiers) -> Int { + chordCodeBase | (modifiers.rawValue << chordModifierShift) | keyCode + } + + /// The chord a packed slot value names, or nil when it isn't a structurally + /// valid chord: outside the chord namespace, no modifiers (a bare key was + /// never bindable, so a stored one is garbage), a modifier as the key half, or + /// bits set above the four modifiers. + static func chordBinding(packed code: Int) -> TriggerBinding? { + guard code >= chordCodeBase else { return nil } + let payload = code - chordCodeBase + let keyCode = payload & chordKeyCodeMask + let modifierBits = payload >> chordModifierShift + guard modifierBits == modifierBits & ChordModifiers.all.rawValue else { return nil } + let modifiers = ChordModifiers(rawValue: modifierBits) + guard !modifiers.isEmpty, !modifierKeyCodes.contains(keyCode) else { return nil } + return .chord(keyCode: keyCode, modifiers: modifiers) + } + + /// The value stored in the persisted slot; `fromPersisted` round-trips it. + /// Internal: `TriggerKeyStore` owns the write, and hosts go through it. + var persistedValue: Int { + switch self { + case .modifier(let key): return key.rawValue + case .chord(let keyCode, let modifiers): + return Self.packedChord(keyCode: keyCode, modifiers: modifiers) + case .mouseButton(let button): return Self.mouseButtonCodeBase + button + } + } + + /// Inline sentence form, e.g. "Tap or hold ⌃⌥D to dictate". Chords render as + /// their macOS glyph sequence; mouse buttons use the 1-based numbering macOS + /// and pointing-device vendors present to users (button number 3 is + /// "Mouse 4"). + public var label: String { + switch self { + case .modifier(let key): return key.label + case .chord(let keyCode, let modifiers): + return modifiers.glyphs + Self.keyLabel(forKeyCode: keyCode) + case .mouseButton(let button): return "Mouse \(button + 1)" + } + } + + /// The caution this binding deserves in the UI, or nil when it needs none. + /// + /// The tap is `.listenOnly` and **swallows nothing**, so a binding the system + /// or the focused app also acts on will do both things at once. That is + /// harmless for a lone right-side modifier (which types nothing) and for a + /// side button (which almost nothing claims), and worth saying plainly for the + /// two bindings where it bites: the wheel click, which browsers and terminals + /// use, and any chord, which the frontmost app may already own. Engine-side + /// rather than in the view so the wording is testable and the Settings footer + /// and the capture sheet can't disagree. + public var passThroughNote: String? { + switch self { + case .modifier: + return nil + case .chord: + return "Blurt doesn't intercept \(label) — an app that already uses it will still act on it." + case .mouseButton(let button): + guard button == Self.middleMouseButton else { return nil } + return + "Mouse 3 is the wheel click. Blurt doesn't intercept it, so apps that use it " + + "(opening a link in a new tab, for example) will still act on it." + } + } + + /// The display name of a chord's key half, in the form a macOS menu would use + /// (a glyph for the editing keys, the bare character for letters and digits). + /// Unknown codes render as `key ` rather than nothing, so a binding is + /// always nameable in the picker. + static func keyLabel(forKeyCode keyCode: Int) -> String { + if let name = keyLabels[keyCode] { return name } + return "key \(keyCode)" + } + + /// Virtual keycode → display name, for every key a chord can name. The values + /// are the US-layout characters and the standard glyphs; a non-US layout may + /// print a different character on the same physical key, which is the same + /// approximation every keyboard-shortcut UI on the platform makes. + private static let keyLabels: [Int: String] = { + var labels: [Int: String] = [ + // Letters, in keycode order. + 0: "A", 11: "B", 8: "C", 2: "D", 14: "E", 3: "F", 5: "G", 4: "H", 34: "I", + 38: "J", 40: "K", 37: "L", 46: "M", 45: "N", 31: "O", 35: "P", 12: "Q", + 15: "R", 1: "S", 17: "T", 32: "U", 9: "V", 13: "W", 7: "X", 16: "Y", 6: "Z", + // Digits. + 29: "0", 18: "1", 19: "2", 20: "3", 21: "4", 23: "5", 22: "6", 26: "7", + 28: "8", 25: "9", + // Punctuation. + 27: "-", 24: "=", 33: "[", 30: "]", 42: "\\", 41: ";", 39: "'", 43: ",", + 47: ".", 44: "/", 50: "`", + // Editing and navigation, as the glyphs a menu shows. + 36: "↩", 48: "⇥", 49: "Space", 51: "⌫", 53: "⎋", 71: "⌧", 76: "⌤", + 114: "Help", 115: "↖", 116: "⇞", 117: "⌦", 119: "↘", 121: "⇟", + 123: "←", 124: "→", 125: "↓", 126: "↑", + ] + // F1–F20, whose keycodes are scattered rather than contiguous. + let functionKeys = [ + 122: 1, 120: 2, 99: 3, 118: 4, 96: 5, 97: 6, 98: 7, 100: 8, 101: 9, + 109: 10, 103: 11, 111: 12, 105: 13, 107: 14, 113: 15, 106: 16, 64: 17, + 79: 18, 80: 19, 90: 20, + ] + for (code, number) in functionKeys { labels[code] = "F\(number)" } + return labels + }() +} diff --git a/Sources/BlurtEngine/Hotkey/TriggerKey.swift b/Sources/BlurtEngine/Hotkey/TriggerKey.swift index f27952ac..ef32ec3a 100644 --- a/Sources/BlurtEngine/Hotkey/TriggerKey.swift +++ b/Sources/BlurtEngine/Hotkey/TriggerKey.swift @@ -1,22 +1,21 @@ -/// A lone momentary modifier key usable as the single dictation trigger. The raw -/// value is the macOS virtual key code, so `TriggerKey(rawValue:)` decodes a -/// persisted keycode directly. Curated to right-side modifiers (rarely used in -/// app shortcuts, so a solo press maps cleanly to "dictate") and `fn`. +/// A lone momentary modifier key usable as the dictation trigger — the default +/// family of `TriggerBinding`, which owns decoding the persisted slot +/// (`TriggerBinding.fromPersisted`). The raw value is the macOS virtual key +/// code, so `TriggerKey(rawValue:)` decodes a persisted keycode directly. +/// +/// Curated to the two right-side modifiers: a solo press of either rarely +/// collides with app shortcuts. `fn` was a third option and was **removed** — +/// it is the one modifier macOS itself claims (dictation, emoji picker, the +/// F-key row), so it never belonged in a list of keys a solo press could own. +/// A persisted `fn` migrates to right ⌥ (see +/// `TriggerBinding.legacyFunctionKeyCode`); anything else a user wants beyond +/// these two is a Custom chord or mouse button. public enum TriggerKey: Int, CaseIterable, Sendable, Hashable { case rightCommand = 54 case rightOption = 61 - case function = 63 public var keyCode: Int { rawValue } - /// Decodes a persisted keycode into a `TriggerKey`, falling back to right ⌘ - /// when the value isn't one of the curated options. The single decode-with- - /// default rule shared by `TriggerKeyStore` and the `@AppStorage` views that - /// read the raw keycode directly (so they re-render live on a Settings change). - public static func fromPersisted(_ code: Int) -> TriggerKey { - TriggerKey(rawValue: code) ?? .rightCommand - } - /// The **device-dependent** modifier flag bit this key toggles, as a raw /// `CGEventFlags`/IOKit value (the app wraps it in `CGEventFlags(rawValue:)`). /// @@ -27,12 +26,15 @@ public enum TriggerKey: Int, CaseIterable, Sendable, Hashable { /// tap's down/up tracking on keyboards where both keys are in play (a leading /// suspect for the duplicate-paste reports on third-party keyboards). The /// device bit names exactly one physical side, so the bound key's own state is - /// unambiguous. `fn` has no left/right split, so it uses the secondary-fn bit. + /// unambiguous. + /// + /// A **chord** binding reads the generic masks instead, deliberately: a chord + /// is a shortcut, and no shortcut in macOS distinguishes ⌃ from ⌃ — see + /// `TriggerBinding.ChordModifiers`. public var deviceModifierMask: UInt64 { switch self { case .rightCommand: return 0x10 // NX_DEVICERCMDKEYMASK case .rightOption: return 0x40 // NX_DEVICERALTKEYMASK - case .function: return 0x80_0000 // kCGEventFlagMaskSecondaryFn } } @@ -41,7 +43,6 @@ public enum TriggerKey: Int, CaseIterable, Sendable, Hashable { switch self { case .rightCommand: return "right ⌘" case .rightOption: return "right ⌥" - case .function: return "fn" } } } diff --git a/Sources/BlurtEngine/Hotkey/TriggerKeyStore.swift b/Sources/BlurtEngine/Hotkey/TriggerKeyStore.swift index 0517f716..29392ca7 100644 --- a/Sources/BlurtEngine/Hotkey/TriggerKeyStore.swift +++ b/Sources/BlurtEngine/Hotkey/TriggerKeyStore.swift @@ -1,11 +1,12 @@ import Foundation -/// Persists the chosen dictation `TriggerKey` as its keycode in `UserDefaults`. -/// Defaults to right ⌘ when unset or when the stored code isn't one of the -/// curated options. +/// Persists the chosen dictation `TriggerBinding` in `UserDefaults`, encoded +/// into the single `Int` slot the store has always used (see `TriggerBinding` +/// for the encoding). Defaults to the right-⌘ modifier when unset or when the +/// stored code isn't one of the curated options. public struct TriggerKeyStore { - /// UserDefaults key holding the trigger keycode. Public so SwiftUI views can - /// observe it directly (e.g. `@AppStorage`) and re-render on change. + /// UserDefaults key holding the encoded trigger binding. Public so SwiftUI + /// views can observe it directly (e.g. `@AppStorage`) and re-render on change. public static let defaultsKey = DefaultsKey.triggerKeyCode.rawValue private let defaults: UserDefaults @@ -13,14 +14,14 @@ public struct TriggerKeyStore { self.defaults = defaults } - public var triggerKey: TriggerKey { + public var triggerBinding: TriggerBinding { get { - // Unset reads as 0, which isn't a curated keycode, so `fromPersisted`'s + // Unset reads as 0, which decodes to nothing, so `fromPersisted`'s // right-⌘ fallback covers both "never set" and "unknown code". - TriggerKey.fromPersisted(defaults.integer(forKey: Self.defaultsKey)) + TriggerBinding.fromPersisted(defaults.integer(forKey: Self.defaultsKey)) } nonmutating set { - defaults.set(newValue.rawValue, forKey: Self.defaultsKey) + defaults.set(newValue.persistedValue, forKey: Self.defaultsKey) } } } diff --git a/Sources/BlurtEngine/Pipeline/DictationLog+Capture.swift b/Sources/BlurtEngine/Pipeline/DictationLog+Capture.swift new file mode 100644 index 00000000..38571abb --- /dev/null +++ b/Sources/BlurtEngine/Pipeline/DictationLog+Capture.swift @@ -0,0 +1,156 @@ +import Foundation + +// The input-diagnostics half of the developer-mode log: while the Custom +// trigger capture sheet is open, every event its recorder sees — accepted or +// refused — is appended to a sibling `capture-events.jsonl`. Split from +// `DictationLog.swift` (same enum, so it shares the serial queue, the encoder, +// the developer-mode gate, and `appendLine`) because it answers a different +// question again: not "what did users say" or "what broke", but "what did that +// mouse actually send" — the diagnosis for a multi-button mouse whose extra +// buttons arrive as something unexpected (or never arrive at all), which is +// invisible from the sheet's refusal text alone. +// +// A separate file for the reason `errors.jsonl` is: the other two logs' lines +// each carry their own expected shape, and interleaving input rows would break +// anything decoding them. +extension DictationLog { + /// The raw facts of one event the capture recorder saw, exactly as the input + /// system reported them — one value rather than loose parameters, so every + /// layer from the sheet down to the entry builder hands the same bundle + /// along. What the recorder *decided* about the event (`outcome`, the bound + /// label) deliberately rides beside it, not in it. + public struct CapturedInput: Sendable { + /// Which monitor delivered the event: `keyDown` or `otherMouseDown`. + let kind: String + /// `CGEvent`/`NSEvent` button number, for mouse events (0 = left). + let button: Int? + /// macOS virtual keycode, for keyboard events. + let keyCode: Int? + /// The event's raw modifier flags. Recorded even when no decision reads + /// them: a mouse driver that ships clicks with phantom flags is exactly the + /// kind of misbehavior this log exists to catch. + let flags: UInt64 + /// Whether a keyboard event was an autorepeat delivery. + let isRepeat: Bool + + public init( + kind: String, button: Int? = nil, keyCode: Int? = nil, + flags: UInt64 = 0, isRepeat: Bool = false + ) { + self.kind = kind + self.button = button + self.keyCode = keyCode + self.flags = flags + self.isRepeat = isRepeat + } + } + + struct CaptureEventEntry: Encodable { + let ts: String + /// See `CapturedInput.kind`. + let kind: String + /// What the recorder did with the event — a stable token to aggregate on: + /// `captured`, `refused-button`, `refused-keyboard-key`, or `cancelled`. + let outcome: String + let button: Int? + let keyCode: Int? + let flags: UInt64 + let isRepeat: Bool + /// The bound trigger's label ("Mouse 4") when `outcome` is `captured`. + let binding: String? + + /// Spelled out for the reason `Entry.CodingKeys` is: a hand-written + /// `encode(to:)` means the key names are the on-disk contract, so they're + /// stated rather than left to a synthesis that no longer happens. + enum CodingKeys: String, CodingKey { + case ts, kind, outcome, button, keyCode, flags, binding + case isRepeat = "repeat" + } + + /// Hand-written so absent facts are omitted rather than written as `null` + /// (matching the other two logs), and so the two always-meaningless-when- + /// default fields (`flags` 0, `repeat` false) don't pad every line of a log + /// meant to be eyeballed. + func encode(to encoder: Encoder) throws { + var container = encoder.container(keyedBy: CodingKeys.self) + try container.encode(ts, forKey: .ts) + try container.encode(kind, forKey: .kind) + try container.encode(outcome, forKey: .outcome) + try container.encodeIfPresent(button, forKey: .button) + try container.encodeIfPresent(keyCode, forKey: .keyCode) + if flags != 0 { + try container.encode(flags, forKey: .flags) + } + if isRepeat { + try container.encode(true, forKey: .isRepeat) + } + try container.encodeIfPresent(binding, forKey: .binding) + } + } + + /// Where capture-sheet input events land. Sibling of `defaultURL` in the same + /// directory, so the one "delete my logs" gesture covers all three files. + static let defaultCaptureURL = URL.libraryDirectory.appending( + path: "Logs/Blurt/capture-events.jsonl") + + /// `defaultCaptureURL` as a home-abbreviated path, for the Developer section's + /// footer — derived here next to the URL the writer uses, for the same reason + /// as `defaultDisplayPath`: the displayed path can't drift from the write + /// target. + public static var defaultCaptureDisplayPath: String { + (defaultCaptureURL.path(percentEncoded: false) as NSString).abbreviatingWithTildeInPath + } + + /// Append one capture-sheet input event. **Gated on developer mode**, so with + /// the switch off (the default) this returns without touching the disk and the + /// capture sheet can invoke it unconditionally for every event it sees. The + /// file I/O is dispatched onto `queue`, off the event-monitor callback. + /// + /// Public because the caller is the app's capture sheet — unlike the other two + /// logs, whose writers live inside the engine. The real store and destination + /// are supplied in the body rather than as defaulted parameters: a public + /// function's default arguments may only reference public declarations, and + /// neither `DeveloperModeStore.init` nor `defaultCaptureURL` has a reason to + /// be public. Tests inject both through the internal overload below. + public static func appendCaptureEvent( + _ input: CapturedInput, outcome: String, binding: String? = nil + ) { + appendCaptureEvent( + input, outcome: outcome, binding: binding, + store: DeveloperModeStore(), to: defaultCaptureURL) + } + + /// The injectable overload behind the public entry point: `store` and `url` + /// exist to be overridden by tests, exactly as on `append`/`appendError` — + /// with both hard-coded, the gate could only be exercised by writing to the + /// real `~/Library/Logs`. + static func appendCaptureEvent( + _ input: CapturedInput, outcome: String, binding: String? = nil, + store: DeveloperModeStore, to url: URL + ) { + gated(store: store) { now in + writeCaptureEvent(input, outcome: outcome, binding: binding, to: url, now: now) + } + } + + /// One capture entry as a value, split from `writeCaptureEvent` for the same + /// reason as `makeEntry`/`makeErrorEntry`: what a row carries is assertable + /// directly rather than through a temp file and a substring search. + static func makeCaptureEntry( + _ input: CapturedInput, outcome: String, binding: String?, now: Date + ) -> CaptureEventEntry { + CaptureEventEntry( + ts: now.formatted(timestampFormat), kind: input.kind, outcome: outcome, + button: input.button, keyCode: input.keyCode, flags: input.flags, + isRepeat: input.isRepeat, binding: binding) + } + + /// The unconditional writer, named distinctly from `appendCaptureEvent` for + /// the reason `write`/`writeError` are: the gated entry point above must not + /// be bypassable by accidentally satisfying a different signature. + static func writeCaptureEvent( + _ input: CapturedInput, outcome: String, binding: String? = nil, to url: URL, now: Date + ) { + appendLine(makeCaptureEntry(input, outcome: outcome, binding: binding, now: now), to: url) + } +} diff --git a/Sources/BlurtEngine/README.md b/Sources/BlurtEngine/README.md index d2e06bfa..9bc0924e 100644 --- a/Sources/BlurtEngine/README.md +++ b/Sources/BlurtEngine/README.md @@ -119,7 +119,7 @@ genuinely being captured. Cueing at the press invites speech that nothing can ca `RecordingCueGate` is the other phase projection, and the reason the chimes don't retrigger: call `cue(for:)` with **every** phase and it returns `.start` only on the edge **into** `.recording` — in production the connecting→recording one, i.e. once the mic is actually delivering audio, never at the press — `.stop` only on the recording→not-recording edge, and `nil` while a phase repeats or when two non-recording phases follow each other. It's a value type holding one edge bit — keep a single instance for the host's lifetime. -Which chime plays is a `SoundPack`: an `id`, a display `label`, and the picker `group` it belongs to. `SoundPack.catalog` is generated by `scripts/generate-sounds.swift` from Yamaha DX7 (ROM1A/ROM1B) and Roland Juno-106 factory presets; `groups` and `voices(in:)` build a sectioned picker off stored indexes rather than rescanning all 192 entries per render, and `fromPersisted(_:)` is the one decode-with-default rule (mirroring `TriggerKey.fromPersisted`) so a view reading the raw id can't disagree with `SoundPackStore`. `startFileName` / `stopFileName` give the `-start` / `-stop` stems, or `nil` for the silent `SoundPack.none`. +Which chime plays is a `SoundPack`: an `id`, a display `label`, and the picker `group` it belongs to. `SoundPack.catalog` is generated by `scripts/generate-sounds.swift` from Yamaha DX7 (ROM1A/ROM1B) and Roland Juno-106 factory presets; `groups` and `voices(in:)` build a sectioned picker off stored indexes rather than rescanning all 192 entries per render, and `fromPersisted(_:)` is the one decode-with-default rule (mirroring `TriggerBinding.fromPersisted`) so a view reading the raw id can't disagree with `SoundPackStore`. `startFileName` / `stopFileName` give the `-start` / `-stop` stems, or `nil` for the silent `SoundPack.none`. The cue audio itself is **not** in the package — the `.m4a` files live in the app bundle, so the engine hands you stems and a host supplies the files, or its own player entirely. See [`SoundPack` ships metadata without the audio](#soundpack-ships-metadata-without-the-audio) for what that costs a package consumer. @@ -237,14 +237,16 @@ The engine carries the _decision_ half of Blurt's self-update, for the same reas ## Hotkey building blocks -The engine ships the _decision logic_ for a lone-modifier trigger; the host supplies the event source (in Blurt, a `CGEventTap` — see `App/Blurt/Blurt/Hotkey/DictationKeyTap.swift` for the reference wiring). +The engine ships the _decision logic_ for a single-control trigger — a lone modifier, a keyboard chord, or an extra mouse button; the host supplies the event source (in Blurt, a `CGEventTap` — see `App/Blurt/Blurt/Hotkey/DictationKeyTap.swift` for the reference wiring). -- **`TriggerKey`** — the curated lone modifiers usable as a trigger (right ⌘, right ⌥, `fn`), with keycodes, display labels, and the device-modifier masks the event source needs. -- **`TriggerKeyStore`** — persists the chosen key in `UserDefaults` (`BlurtTriggerKeyCode`), defaulting to right ⌘. -- **`DictationKeyGate`** — a pure, clock-free state machine that turns `modifierDown(at:)` / `modifierUp(at:)` / `otherKeyDown()` into `.start` / `.stop` / `.cancel` / `.none`. Recording starts the instant the modifier goes down; on key-up, a release held ≥ `holdThreshold` (default 1 s) is push-to-talk (stop), a shorter release latches tap-to-toggle (next tap stops). A modifier+key combo from idle cancels the fresh capture; over a latched recording it passes through as a normal shortcut. Callers pass monotonic timestamps, so every decision is deterministic and unit-tested (`DictationKeyGateTests`, `HotkeyRaceTests`). -- **`DictationKeyRouter`** — the recommended layer over the gate: reduce each raw event to `.flagsChanged(keyCode:triggerFlagIsOn:)` / `.keyDown(keyCode:)` and `handle(_:at:)` applies the filters every event source needs — only the bound keycode's flag changes count, and only genuine down/up _edges_ reach the gate (`flagsChanged` deliveries re-report the bit whether or not it changed, so a repeat must not double-start a dictation). `reset()` / `rebind(triggerKeyCode:)` clear state that can no longer be trusted (dropped events, a rebound trigger) and return whether they discarded a live recording. Unit-tested (`DictationKeyRouterTests`). +- **`TriggerKey`** — the curated lone modifiers usable as a trigger (right ⌘, right ⌥), with keycodes, display labels, and the device-modifier masks the event source needs. `fn` was removed as an option; a persisted `fn` migrates to right ⌥. +- **`TriggerBinding`** — what's bound: `.modifier(TriggerKey)`, `.chord(keyCode:modifiers:)` (⌃⌥D, side-agnostic `ChordModifiers` with ⌃⌥⇧⌘ `glyphs`), or `.mouseButton(_:)` (the raw 0-based button number, bindable from 2 up — so stored `2` displays as "Mouse 3" and is the wheel/middle click). All three encode into the store's single `Int` slot (modifiers as their bare keycode, so pre-Custom installs decode unchanged; mouse buttons and chords in disjoint tagged namespaces, a chord packing its keycode and modifier bits together); `fromPersisted(_:)` is the one decode-with-default rule (garbage → right ⌘), `chordBinding(forKeyCode:modifiers:)` / `mouseButtonBinding(forButton:)` are the capture-time policy (with `ChordRefusal` naming _why_ a chord is refused), and `passThroughNote` is the caution a host shows for bindings the listen-only tap can't keep from reaching the focused app. +- **`TriggerKeyStore`** — persists the chosen binding in `UserDefaults` (`BlurtTriggerKeyCode`), defaulting to right ⌘. +- **`DictationKeyGate`** — a pure, clock-free state machine that turns `modifierDown(at:)` / `modifierUp(at:)` / `otherKeyDown()` (trigger-down/up, whatever key or button the trigger is) into `.start` / `.stop` / `.cancel` / `.none`. Recording starts the instant the trigger goes down; on key-up, a release held ≥ `holdThreshold` (default 1 s) is push-to-talk (stop), a shorter release latches tap-to-toggle (next tap stops). A modifier+key combo from idle cancels the fresh capture; over a latched recording it passes through as a normal shortcut. Callers pass monotonic timestamps, so every decision is deterministic and unit-tested (`DictationKeyGateTests`, `HotkeyRaceTests`). +- **`DictationKeyRouter`** — the recommended layer over the gate: reduce each raw event to `.flagsChanged(keyCode:triggerFlagIsOn:modifiers:)` / `.keyDown(keyCode:modifiers:)` / `.keyUp(keyCode:)` / `.mouseDown(button:)` / `.mouseUp(button:)` and `handle(_:at:)` applies the filters every event source needs — only events for the bound trigger's own family count (`flagsChanged` for a modifier binding, key events plus their modifier set for a chord, mouse events for a button), and only genuine down/up _edges_ reach the gate (`flagsChanged` deliveries re-report the bit whether or not it changed, and a held key autorepeats, so a repeat must not double-start a dictation). A chord needs its exact modifier set and ends on the key's `keyUp` or on any required modifier being released. The other-key combo cancel applies to modifier bindings only — elsewhere, keys are just typing. `reset()` / `rebind(binding:)` clear state that can no longer be trusted (dropped events, a rebound trigger) and return whether they discarded a live recording. Unit-tested (`DictationKeyRouterTests`, `DictationKeyRouterCustomBindingTests`). +- **`DictationEventDecoder`** — the `CGEvent` → router-event reduction, the one deliberately CoreGraphics-typed piece of the hotkey engine, split out of the host's tap shim so it can be exercised with real `CGEvent` fixtures (`DictationEventDecoderTests`). -Map the router's actions onto the session with `submit`: `.start` → `submit(.press)`, `.stop` → `submit(.release)`, `.cancel` → `submit(.cancel)` — event-tap callbacks can't `await`, and `submit` preserves their emit order where per-callback `Task` spawning wouldn't. If your event source can lose key-ups (a disabled tap, a rebind), call the router's `reset()`/`rebind(triggerKeyCode:)` and recover a discarded recording with `submit(.cancelRecording)`. +Map the router's actions onto the session with `submit`: `.start` → `submit(.press)`, `.stop` → `submit(.release)`, `.cancel` → `submit(.cancel)` — event-tap callbacks can't `await`, and `submit` preserves their emit order where per-callback `Task` spawning wouldn't. If your event source can lose key-ups (a disabled tap, a rebind), call the router's `reset()`/`rebind(binding:)` and recover a discarded recording with `submit(.cancelRecording)`. ## Testing your integration diff --git a/Tests/BlurtEngineTests/DictationCaptureLogTests.swift b/Tests/BlurtEngineTests/DictationCaptureLogTests.swift new file mode 100644 index 00000000..236ed375 --- /dev/null +++ b/Tests/BlurtEngineTests/DictationCaptureLogTests.swift @@ -0,0 +1,118 @@ +import Foundation +import Testing + +@testable import BlurtEngine + +private func makeTempCaptureLogURL() -> URL { + let dir = FileManager.default.temporaryDirectory + .appendingPathComponent("BlurtCaptureLogTests-\(UUID().uuidString)", isDirectory: true) + try? FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) + return dir.appendingPathComponent("capture-events.jsonl") +} + +private func readLog(_ url: URL) -> String { + (try? String(contentsOf: url, encoding: .utf8)) ?? "" +} + +/// What one capture-sheet input event records — the raw facts a misbehaving +/// mouse is diagnosed from — and which default-valued fields stay off the line. +@Suite("DictationLog.makeCaptureEntry") +struct DictationCaptureEntryTests { + @Test("records the event's raw facts and the outcome") + func recordsRawFacts() { + let entry = DictationLog.makeCaptureEntry( + DictationLog.CapturedInput(kind: "otherMouseDown", button: 3, flags: 0x100), + outcome: "captured", binding: "Mouse 4", + now: Date(timeIntervalSince1970: 1_700_000_000)) + #expect(entry.kind == "otherMouseDown") + #expect(entry.outcome == "captured") + #expect(entry.button == 3) + #expect(entry.keyCode == nil) + #expect(entry.flags == 0x100) + #expect(entry.binding == "Mouse 4") + #expect(entry.ts.hasPrefix("2023-11-14")) + } + + @Test("a refused event carries the same raw facts, with no binding") + func refusedEventStillCarriesFacts() throws { + // The whole point of the log: the refusal text alone can't say what a + // misbehaving button actually sent. + let entry = DictationLog.makeCaptureEntry( + DictationLog.CapturedInput(kind: "keyDown", keyCode: 96, isRepeat: true), + outcome: "refused-keyboard-key", binding: nil, now: Date()) + #expect(entry.keyCode == 96) + #expect(entry.isRepeat) + #expect(entry.binding == nil) + let data = try DictationLog.makeEncoder().encode(entry) + let line = try #require(String(data: data, encoding: .utf8)) + #expect(line.contains("\"repeat\":true")) + } + + @Test("absent and default-valued fields are omitted from the line, not written as null") + func defaultsAreOmitted() throws { + let entry = DictationLog.makeCaptureEntry( + DictationLog.CapturedInput(kind: "otherMouseDown", button: 1), + outcome: "refused-button", binding: nil, now: Date()) + let data = try DictationLog.makeEncoder().encode(entry) + let object = try JSONSerialization.jsonObject(with: data) + let encoded = try #require(object as? [String: Any]) + #expect(encoded.keys.sorted() == ["button", "kind", "outcome", "ts"]) + } +} + +/// The developer-mode gate — the same privacy guarantee the other two logs +/// carry: a user who never opts in has nothing on disk, so the capture sheet +/// can log unconditionally. +@Suite("DictationLog capture-log developer-mode gate") +struct DictationCaptureLogGateTests { + @Test("with developer mode off, appendCaptureEvent writes nothing to disk") + func gateClosedWritesNothing() { + let url = makeTempCaptureLogURL() + let store = DeveloperModeStore(defaults: freshDefaults()) + DictationLog.appendCaptureEvent( + DictationLog.CapturedInput(kind: "otherMouseDown", button: 3), + outcome: "captured", store: store, to: url) + DictationLog.queue.sync {} + #expect(!FileManager.default.fileExists(atPath: url.path)) + } + + @Test("with developer mode on, appendCaptureEvent appends the event") + func gateOpenWrites() { + let url = makeTempCaptureLogURL() + let store = developerModeStore(enabled: true) + DictationLog.appendCaptureEvent( + DictationLog.CapturedInput(kind: "otherMouseDown", button: 1), + outcome: "refused-button", store: store, to: url) + DictationLog.queue.sync {} + let line = readLog(url) + #expect(line.contains("\"outcome\":\"refused-button\"")) + #expect(line.contains("\"button\":1")) + } + + /// A separate sibling file: input rows must never land in the transcript + /// corpus or the error log, whose readers expect their own shapes. + @Test("the capture log is a sibling file of the other two") + func writesToASiblingFile() { + #expect(DictationLog.defaultCaptureURL != DictationLog.defaultURL) + #expect(DictationLog.defaultCaptureURL != DictationLog.defaultErrorURL) + #expect( + DictationLog.defaultCaptureURL.deletingLastPathComponent() + == DictationLog.defaultURL.deletingLastPathComponent()) + #expect(DictationLog.defaultCaptureURL.lastPathComponent == "capture-events.jsonl") + } +} + +/// The displayed capture-log location (Developer section footer) has to name +/// the file the writer actually appends to. +@Suite("DictationLog.defaultCaptureDisplayPath") +struct DictationCaptureLogDisplayPathTests { + @Test("abbreviates the home directory and names the write target") + func matchesTheWriteTarget() { + let shown = DictationLog.defaultCaptureDisplayPath + #expect(shown.hasPrefix("~/")) + #expect(!shown.contains(NSHomeDirectory())) + #expect( + (shown as NSString).expandingTildeInPath + == DictationLog.defaultCaptureURL.path(percentEncoded: false)) + } +} diff --git a/Tests/BlurtEngineTests/DictationEventDecoderTests.swift b/Tests/BlurtEngineTests/DictationEventDecoderTests.swift new file mode 100644 index 00000000..25df89fc --- /dev/null +++ b/Tests/BlurtEngineTests/DictationEventDecoderTests.swift @@ -0,0 +1,216 @@ +import CoreGraphics +import Testing + +@testable import BlurtEngine + +/// Decode-level fixtures: **real `CGEvent`s**, constructed with the same +/// initializers macOS uses to deliver input, fed through the engine's +/// tap-delivery reduction (`DictationEventDecoder.routerEvent`) and asserted +/// against the router events they must produce. This is the layer the pure +/// router suites can't cover — that a genuine `otherMouseDown` for button 3 +/// actually carries `mouseEventButtonNumber == 3` into `.mouseDown(button: 3)`. +/// +/// The fixtures are data (button/keycode, event type, expected router event) so +/// captures from real hardware — a mouse whose extra buttons arrive as +/// something unexpected — can be appended as new rows rather than new code. +/// +/// Constructing events posts nothing and needs no event tap, no Accessibility +/// grant, and no key window, so this runs on CI. If a headless runner ever +/// proves otherwise, gate the suites with `.enabled(if:)` on an environment +/// flag the way `MicCaptureLevelsTests` gates on `BLURT_LIVE_AUDIO_TESTS` — +/// never by creating a real `CGEventTap` here. +@Suite("DictationEventDecoder") +struct DictationEventDecoderTests { + /// Right ⌘'s device-dependent bit (`TriggerKey.rightCommand.deviceModifierMask`), + /// the flag a modifier binding hands the decoder. + private static let rightCommandFlag = CGEventFlags( + rawValue: TriggerKey.rightCommand.deviceModifierMask) + + /// One replayed mouse delivery: the button and type the event is built from, + /// and the router event the decoder must produce. Append rows here for new + /// hardware captures. + struct MouseFixture: Sendable { + let button: Int + let type: CGEventType + let expected: DictationKeyRouter.Event? + } + + static let mouseFixtures: [MouseFixture] = [ + // The bindable range, down and up: middle ("Mouse 3"), the MX-style thumb + // buttons ("Mouse 4"/"Mouse 5"), and the edges of what CGEvent can report. + MouseFixture(button: 2, type: .otherMouseDown, expected: .mouseDown(button: 2)), + MouseFixture(button: 2, type: .otherMouseUp, expected: .mouseUp(button: 2)), + MouseFixture(button: 3, type: .otherMouseDown, expected: .mouseDown(button: 3)), + MouseFixture(button: 3, type: .otherMouseUp, expected: .mouseUp(button: 3)), + MouseFixture(button: 4, type: .otherMouseDown, expected: .mouseDown(button: 4)), + MouseFixture(button: 4, type: .otherMouseUp, expected: .mouseUp(button: 4)), + MouseFixture(button: 31, type: .otherMouseDown, expected: .mouseDown(button: 31)), + MouseFixture(button: 31, type: .otherMouseUp, expected: .mouseUp(button: 31)), + // Left/right clicks arrive as their own event types, which the tap's mask + // doesn't even include — the decoder must ignore them, not misread them as + // extra buttons. + MouseFixture(button: 0, type: .leftMouseDown, expected: nil), + MouseFixture(button: 0, type: .leftMouseUp, expected: nil), + MouseFixture(button: 1, type: .rightMouseDown, expected: nil), + MouseFixture(button: 1, type: .rightMouseUp, expected: nil), + ] + + private static func mouseEvent(type: CGEventType, button: Int) throws -> CGEvent { + let cgButton = try #require(CGMouseButton(rawValue: UInt32(button))) + return try #require( + CGEvent( + mouseEventSource: nil, mouseType: type, mouseCursorPosition: .zero, mouseButton: cgButton)) + } + + @Test("a real mouse CGEvent decodes to the expected router event", arguments: mouseFixtures) + func decodesMouseEvents(fixture: MouseFixture) throws { + let event = try Self.mouseEvent(type: fixture.type, button: fixture.button) + let decoded = DictationEventDecoder.routerEvent( + type: fixture.type, event: event, triggerFlag: []) + #expect(decoded == fixture.expected) + } + + @Test("a real keyDown CGEvent decodes to the combo probe with its keycode") + func decodesKeyDown() throws { + let event = try #require(CGEvent(keyboardEventSource: nil, virtualKey: 8, keyDown: true)) // C + event.flags = [] + let decoded = DictationEventDecoder.routerEvent( + type: .keyDown, event: event, triggerFlag: Self.rightCommandFlag) + #expect(decoded == .keyDown(keyCode: 8, modifiers: [])) + } + + @Test("a real keyUp CGEvent decodes to the chord's trigger-up") + func decodesKeyUp() throws { + let event = try #require(CGEvent(keyboardEventSource: nil, virtualKey: 8, keyDown: false)) + let decoded = DictationEventDecoder.routerEvent( + type: .keyUp, event: event, triggerFlag: Self.rightCommandFlag) + #expect(decoded == .keyUp(keyCode: 8)) + } + + // MARK: - Chord decode + + /// One replayed chord keyDown: the flags the event carries and the modifier set + /// they must decode to. Rows are data, so a keyboard or driver that reports + /// flags oddly can be added as a fixture rather than as new code. + struct ChordFixture: Sendable { + let flags: CGEventFlags + let expected: TriggerBinding.ChordModifiers + } + + static let chordFixtures: [ChordFixture] = [ + ChordFixture(flags: [], expected: []), + ChordFixture(flags: [.maskControl], expected: [.control]), + ChordFixture(flags: [.maskAlternate], expected: [.option]), + ChordFixture(flags: [.maskShift], expected: [.shift]), + ChordFixture(flags: [.maskCommand], expected: [.command]), + // The everyday ⌃⌥D shape, and the full house. + ChordFixture(flags: [.maskControl, .maskAlternate], expected: [.control, .option]), + ChordFixture( + flags: [.maskControl, .maskAlternate, .maskShift, .maskCommand], + expected: [.control, .option, .shift, .command]), + // The device-dependent side bits ride along with the generic mask on a real + // press; a chord is side-agnostic, so they must not change the decode. + ChordFixture( + flags: [.maskControl, CGEventFlags(rawValue: 0x2000)], expected: [.control]), + // Caps Lock and fn are deliberately not chord modifiers. + ChordFixture(flags: [.maskAlphaShift], expected: []), + ChordFixture(flags: [.maskSecondaryFn], expected: []), + ChordFixture(flags: [.maskAlphaShift, .maskCommand], expected: [.command]), + ] + + @Test("a chord keyDown decodes its modifier set from the event's flags", arguments: chordFixtures) + func decodesChordModifiers(fixture: ChordFixture) throws { + let event = try #require(CGEvent(keyboardEventSource: nil, virtualKey: 2, keyDown: true)) // D + event.flags = fixture.flags + let decoded = DictationEventDecoder.routerEvent( + type: .keyDown, event: event, triggerFlag: []) + #expect(decoded == .keyDown(keyCode: 2, modifiers: fixture.expected)) + } + + @Test("an autorepeat keyDown decodes to nothing — a repeat is not an edge") + func dropsAutorepeatKeyDown() throws { + let event = try #require(CGEvent(keyboardEventSource: nil, virtualKey: 2, keyDown: true)) + event.flags = [.maskControl, .maskAlternate] + event.setIntegerValueField(.keyboardEventAutorepeat, value: 1) + #expect(DictationEventDecoder.routerEvent(type: .keyDown, event: event, triggerFlag: []) == nil) + + // The same event without the repeat flag is the real trigger-down. + event.setIntegerValueField(.keyboardEventAutorepeat, value: 0) + #expect( + DictationEventDecoder.routerEvent(type: .keyDown, event: event, triggerFlag: []) + == .keyDown(keyCode: 2, modifiers: [.control, .option])) + } + + @Test("a partial-modifier release decodes to the flags event a chord ends on") + func decodesPartialModifierRelease() throws { + // ⌃⌥ held, then ⌃ released: the delivery reports ⌥ still down, which is what + // tells the router the chord is no longer complete. + // + // A real `triggerFlag` is passed even though this is the chord path, because + // an **empty** flag set makes `CGEventFlags.contains` vacuously true — every + // set contains the empty set — so `triggerFlagIsOn` would read `true` for + // every delivery and assert nothing. Harmless in production (a chord binding + // ignores that field entirely; `handleForChord` reads only the modifier set), + // but worth not writing into a fixture that claims to check it. + let event = try #require( + CGEvent(keyboardEventSource: nil, virtualKey: CGKeyCode(59), keyDown: false)) // left ⌃ + event.type = .flagsChanged + event.flags = [.maskAlternate] + let decoded = DictationEventDecoder.routerEvent( + type: .flagsChanged, event: event, triggerFlag: Self.rightCommandFlag) + #expect(decoded == .flagsChanged(keyCode: 59, triggerFlagIsOn: false, modifiers: [.option])) + } + + @Test("an empty trigger flag reads as always-on, which is why a chord ignores that field") + func emptyTriggerFlagIsVacuouslyOn() throws { + // Pins the sharp edge above rather than leaving it to be rediscovered: the + // tap passes `[]` as the trigger flag under a chord or mouse binding, so this + // field is meaningless there — and the router must never read it for those + // bindings (`DictationKeyRouterChordBindingTests` covers that side). + let event = try #require( + CGEvent(keyboardEventSource: nil, virtualKey: CGKeyCode(59), keyDown: true)) + event.type = .flagsChanged + event.flags = [] + let decoded = DictationEventDecoder.routerEvent( + type: .flagsChanged, event: event, triggerFlag: []) + #expect(decoded == .flagsChanged(keyCode: 59, triggerFlagIsOn: true, modifiers: [])) + } + + @Test("a flagsChanged CGEvent decodes the keycode and the trigger flag, set and cleared") + func decodesFlagsChanged() throws { + let event = try #require( + CGEvent(keyboardEventSource: nil, virtualKey: CGKeyCode(54), keyDown: true)) // right ⌘ + event.type = .flagsChanged + event.flags = [.maskCommand, Self.rightCommandFlag] + let down = DictationEventDecoder.routerEvent( + type: .flagsChanged, event: event, triggerFlag: Self.rightCommandFlag) + #expect(down == .flagsChanged(keyCode: 54, triggerFlagIsOn: true, modifiers: [.command])) + + event.flags = [] + let up = DictationEventDecoder.routerEvent( + type: .flagsChanged, event: event, triggerFlag: Self.rightCommandFlag) + #expect(up == .flagsChanged(keyCode: 54, triggerFlagIsOn: false, modifiers: [])) + } + + @Test("a flagsChanged delivery for another modifier still reports its own keycode") + func decodesOtherModifiersFlagsChanged() throws { + // Left ⌘ going down while right ⌘ is bound: the generic command bit is set, + // the right-side device bit is not — the decoder must report exactly that, + // and the router's keycode relevance filter does the ignoring. + let event = try #require( + CGEvent(keyboardEventSource: nil, virtualKey: CGKeyCode(55), keyDown: true)) // left ⌘ + event.type = .flagsChanged + event.flags = [.maskCommand] + let decoded = DictationEventDecoder.routerEvent( + type: .flagsChanged, event: event, triggerFlag: Self.rightCommandFlag) + #expect(decoded == .flagsChanged(keyCode: 55, triggerFlagIsOn: false, modifiers: [.command])) + } + + @Test("event types outside the tap's interest decode to nothing") + func ignoresIrrelevantTypes() throws { + let event = try Self.mouseEvent(type: .otherMouseDragged, button: 3) + let decoded = DictationEventDecoder.routerEvent( + type: .otherMouseDragged, event: event, triggerFlag: []) + #expect(decoded == nil) + } +} diff --git a/Tests/BlurtEngineTests/DictationKeyRouterCustomBindingTests.swift b/Tests/BlurtEngineTests/DictationKeyRouterCustomBindingTests.swift new file mode 100644 index 00000000..e26a2ec6 --- /dev/null +++ b/Tests/BlurtEngineTests/DictationKeyRouterCustomBindingTests.swift @@ -0,0 +1,263 @@ +import Testing + +@testable import BlurtEngine + +/// `DictationKeyRouter` under the Custom binding families — a keyboard chord +/// (`.chord`, riding `keyDown`/`keyUp` plus each event's modifier set) and an +/// extra mouse button (`.mouseButton`, riding `mouseDown`/`mouseUp`). The +/// modifier-binding behavior and the reset/rebind/recovery contract live in +/// `DictationKeyRouterTests`; these suites pin that the custom families share the +/// same edge filter and gate semantics, that a chord needs its **exact** modifier +/// set and ends when any required modifier is released, and that the other-key +/// combo cancel deliberately does NOT apply to either (a chord's own modifiers +/// are part of the trigger, and Mouse4+K names no shortcut to macOS). +@Suite("DictationKeyRouter chord bindings") +struct DictationKeyRouterChordBindingTests { + /// ⌃⌥D — the chord from the request that prompted this feature. + private let key = 2 + private let required: TriggerBinding.ChordModifiers = [.control, .option] + + private func chordRouter() -> DictationKeyRouter { + DictationKeyRouter(binding: .chord(keyCode: key, modifiers: required)) + } + + private func down(_ modifiers: TriggerBinding.ChordModifiers) -> DictationKeyRouter.Event { + .keyDown(keyCode: key, modifiers: modifiers) + } + + @Test("a held chord is start → stop (push-to-talk)") + func chordHoldIsStartStop() { + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(2)) == .stop) + } + + @Test("a short chord tap latches; the next tap stops") + func chordTapToToggle() { + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + #expect(router.handle(.keyUp(keyCode: key), at: .milliseconds(200)) == .none) // latched + #expect(router.handle(down(required), at: .seconds(5)) == .none) + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(5) + .milliseconds(200)) == .stop) + } + + @Test("a repeated keyDown of the held chord doesn't re-fire the gate") + func chordAutorepeatIsDeduped() { + // The host drops autorepeat deliveries, but the edge filter must hold on its + // own — a second down with no up between is not an edge. + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + #expect(router.handle(down(required), at: .milliseconds(500)) == .none) + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(2)) == .stop) + } + + @Test("the chord needs its exact modifier set — too few, too many, or wrong ones do nothing") + func chordNeedsExactModifiers() { + var router = chordRouter() + #expect(router.handle(down([]), at: .zero) == .none) // bare D types, never triggers + #expect(router.handle(down([.control]), at: .milliseconds(10)) == .none) // half the chord + #expect(router.handle(down([.command, .option]), at: .milliseconds(20)) == .none) // wrong one + // A superset is a *different* shortcut (⌃⌥⇧D), so it must not fire either. + #expect(router.handle(down([.control, .option, .shift]), at: .milliseconds(30)) == .none) + // And the exact set still works afterwards — none of the above armed the gate. + #expect(router.handle(down(required), at: .milliseconds(40)) == .start) + } + + @Test("another key's keyDown never drives or cancels a chord binding") + func chordIgnoresOtherKeys() { + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + // Typing while the chord is held (or latched) is just typing: a chord binding + // has no combo rule, so this must not cancel the dictation. + #expect(router.handle(.keyDown(keyCode: 8, modifiers: required), at: .milliseconds(50)) == .none) + #expect(router.handle(.keyUp(keyCode: 8), at: .milliseconds(60)) == .none) + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(2)) == .stop) + } + + @Test("releasing a required modifier ends the press, even before the key's keyUp") + func releasingAModifierEndsThePress() { + // The realistic release order for ⌃⌥D: the user lets go of ⌃ while D is still + // down. Waiting for D's keyUp would leave the gate armed with the chord no + // longer held — and on a hold that means the recording never stops. + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + #expect( + router.handle( + .flagsChanged(keyCode: 59, triggerFlagIsOn: false, modifiers: [.option]), + at: .seconds(2)) == .stop) + // The key's own keyUp then arrives with nothing left to end. + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(2) + .milliseconds(50)) == .none) + } + + @Test("a modifier release that leaves the chord complete changes nothing") + func extraModifierReleaseIsInert() { + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + // ⇧ was never required, so letting it go leaves ⌃⌥ held: still a live press. + #expect( + router.handle( + .flagsChanged(keyCode: 56, triggerFlagIsOn: false, modifiers: [.control, .option]), + at: .milliseconds(500)) == .none) + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(2)) == .stop) + } + + @Test("a modifier release while idle emits nothing") + func modifierReleaseWhileIdleIsInert() { + // Ordinary modifier traffic with no dictation in flight must not reach the gate. + var router = chordRouter() + #expect( + router.handle( + .flagsChanged(keyCode: 59, triggerFlagIsOn: false, modifiers: []), at: .zero) == .none) + #expect(router.handle(down(required), at: .seconds(1)) == .start) + } + + @Test("a latched chord recording survives modifier traffic and stops on the next tap") + func latchedChordSurvivesModifierRelease() { + // Tap-to-toggle: after the latch the user's hands leave the chord entirely, so + // the modifier releases that follow must not stop the recording — only the + // next deliberate tap does. + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + #expect(router.handle(.keyUp(keyCode: key), at: .milliseconds(150)) == .none) // latched + #expect( + router.handle( + .flagsChanged(keyCode: 59, triggerFlagIsOn: false, modifiers: []), + at: .milliseconds(200)) == .none) + #expect(router.handle(down(required), at: .seconds(5)) == .none) + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(5) + .milliseconds(150)) == .stop) + } + + @Test("another key's keyUp and mouse events never reach a chord binding's gate") + func chordIgnoresOtherFamilies() { + var router = chordRouter() + #expect(router.handle(.mouseDown(button: 3), at: .zero) == .none) + #expect(router.handle(.mouseUp(button: 3), at: .milliseconds(10)) == .none) + #expect(router.handle(.keyUp(keyCode: 8), at: .milliseconds(20)) == .none) + } + + @Test("dropped-event recovery keeps a held chord, discards a released one") + func recoveryForChordBinding() { + // The host's "still held" read is `keyState` AND the required modifiers via + // `flagsState`; the router's decision is the same rule as every other family. + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + let keptWhileHeld = router.recoverFromDroppedEvents(triggerStillHeld: true) + #expect(!keptWhileHeld) + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(2)) == .stop) + + #expect(router.handle(down(required), at: .seconds(3)) == .start) + let discarded = router.recoverFromDroppedEvents(triggerStillHeld: false) + #expect(discarded) + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(4)) == .none) + } + + @Test("rebinding from a chord to a modifier discards the live recording") + func rebindFromChord() { + var router = chordRouter() + #expect(router.handle(down(required), at: .zero) == .start) + let discarded = router.rebind(binding: .modifier(.rightCommand)) + #expect(discarded) + #expect(router.binding == .modifier(.rightCommand)) + // The chord's keyUp is now irrelevant; the modifier drives dictation. + #expect(router.handle(.keyUp(keyCode: key), at: .seconds(1)) == .none) + #expect( + router.handle(.flagsChanged(keyCode: 54, triggerFlagIsOn: true), at: .seconds(2)) == .start) + } +} + +@Suite("DictationKeyRouter mouse-button bindings") +struct DictationKeyRouterCustomBindingTests { + private let mouseButton = 3 // "Mouse 4" + private func mouseRouter() -> DictationKeyRouter { + DictationKeyRouter(binding: .mouseButton(mouseButton)) + } + + @Test("a held mouse button is start → stop (push-to-talk)") + func mouseHoldIsStartStop() { + var router = mouseRouter() + #expect(router.handle(.mouseDown(button: mouseButton), at: .zero) == .start) + #expect(router.handle(.mouseUp(button: mouseButton), at: .seconds(2)) == .stop) + } + + @Test("a short mouse-button click latches; the next click stops") + func mouseTapToToggle() { + var router = mouseRouter() + #expect(router.handle(.mouseDown(button: mouseButton), at: .zero) == .start) + #expect(router.handle(.mouseUp(button: mouseButton), at: .milliseconds(200)) == .none) + #expect(router.handle(.mouseDown(button: mouseButton), at: .seconds(5)) == .none) + #expect(router.handle(.mouseUp(button: mouseButton), at: .seconds(5) + .milliseconds(200)) == .stop) + } + + @Test("a repeated mouseDown of the held button doesn't re-fire the gate") + func mouseRepeatIsDeduped() { + var router = mouseRouter() + #expect(router.handle(.mouseDown(button: mouseButton), at: .zero) == .start) + #expect(router.handle(.mouseDown(button: mouseButton), at: .milliseconds(50)) == .none) + #expect(router.handle(.mouseUp(button: mouseButton), at: .seconds(2)) == .stop) + } + + @Test("a mouseUp with no tracked down is ignored") + func mouseUpWithoutDownIsIgnored() { + var router = mouseRouter() + #expect(router.handle(.mouseUp(button: mouseButton), at: .zero) == .none) + } + + @Test("other buttons never reach a mouse binding's gate") + func mouseBindingIgnoresOtherButtons() { + var router = mouseRouter() + #expect(router.handle(.mouseDown(button: 4), at: .zero) == .none) + #expect(router.handle(.mouseUp(button: 4), at: .milliseconds(50)) == .none) + } + + @Test("keyboard events never cancel a mouse binding's recording") + func mouseBindingIgnoresKeyboard() { + // Typing while a mouse-button dictation is held or latched is just typing — + // Mouse4+K names no shortcut to macOS, so the modifier bindings' combo + // cancel deliberately does not apply here. + var router = mouseRouter() + #expect(router.handle(.mouseDown(button: mouseButton), at: .zero) == .start) + #expect(router.handle(.keyDown(keyCode: 8), at: .milliseconds(100)) == .none) + #expect( + router.handle(.flagsChanged(keyCode: 54, triggerFlagIsOn: true), at: .milliseconds(200)) + == .none) + #expect(router.handle(.mouseUp(button: mouseButton), at: .seconds(2)) == .stop) + } + + @Test("rebind across families: a modifier recording ends when the trigger becomes a mouse button") + + func rebindAcrossFamilies() { + var router = DictationKeyRouter(binding: .modifier(.rightCommand)) + let down = DictationKeyRouter.Event.flagsChanged(keyCode: 54, triggerFlagIsOn: true) + #expect(router.handle(down, at: .zero) == .start) + + // Rebinding means the old trigger's up-event can never match — the caller + // must cancel the capture rather than let the auto-release cap paste it. + let discarded = router.rebind(binding: .mouseButton(mouseButton)) + #expect(discarded) + #expect(router.binding == .mouseButton(mouseButton)) + + // The old modifier is now irrelevant; the button drives dictation. + #expect( + router.handle(.flagsChanged(keyCode: 54, triggerFlagIsOn: false), at: .seconds(1)) == .none) + #expect(router.handle(.mouseDown(button: mouseButton), at: .seconds(2)) == .start) + #expect(router.handle(.mouseUp(button: mouseButton), at: .seconds(4)) == .stop) + } + + @Test("dropped-event recovery keeps a held mouse-button recording, discards a released one") + func recoveryForMouseBinding() { + // Same rule as the modifier binding, mouse family: the host's "still held" + // read comes from `CGEventSource.buttonState` instead of `flagsState`, but + // the router's decision is identical. + var router = mouseRouter() + #expect(router.handle(.mouseDown(button: mouseButton), at: .zero) == .start) + + let keptWhileHeld = router.recoverFromDroppedEvents(triggerStillHeld: true) + #expect(!keptWhileHeld) + #expect(router.handle(.mouseUp(button: mouseButton), at: .seconds(2)) == .stop) + + #expect(router.handle(.mouseDown(button: mouseButton), at: .seconds(3)) == .start) + let discarded = router.recoverFromDroppedEvents(triggerStillHeld: false) + #expect(discarded) + #expect(router.handle(.mouseUp(button: mouseButton), at: .seconds(4)) == .none) + } +} diff --git a/Tests/BlurtEngineTests/DictationKeyRouterTests.swift b/Tests/BlurtEngineTests/DictationKeyRouterTests.swift index 792abc11..de436585 100644 --- a/Tests/BlurtEngineTests/DictationKeyRouterTests.swift +++ b/Tests/BlurtEngineTests/DictationKeyRouterTests.swift @@ -3,27 +3,32 @@ import Testing @testable import BlurtEngine /// The router's three jobs on top of `DictationKeyGate` (whose tap/hold semantics -/// have their own suites): only the bound keycode's flag *edges* reach the gate -/// — `flagsChanged` deliveries re-report the bit whether or not it changed, so -/// a repeat must not double-fire — reset/rebind report whether they -/// discarded a live recording the host has to cancel upstream, and dropped-event -/// recovery decides whether a disabled-then-re-enabled tap keeps the gate's state. +/// have their own suites): only the bound trigger's genuine down/up *edges* reach +/// the gate — `flagsChanged` deliveries re-report the bit whether or not it +/// changed, and a held key can re-report its down state, so a repeat must not +/// double-fire — reset/rebind report whether they discarded a live recording the +/// host has to cancel upstream, and dropped-event recovery decides whether a +/// disabled-then-re-enabled tap keeps the gate's state. @Suite("DictationKeyRouter") struct DictationKeyRouterTests { - private let trigger = TriggerKey.rightCommand.keyCode - private let otherModifier = TriggerKey.rightOption.keyCode + private let trigger = TriggerKey.rightCommand + private let otherModifier = TriggerKey.rightOption - private func downEvent(_ keyCode: Int) -> DictationKeyRouter.Event { - .flagsChanged(keyCode: keyCode, triggerFlagIsOn: true) + private func downEvent(_ key: TriggerKey) -> DictationKeyRouter.Event { + .flagsChanged(keyCode: key.keyCode, triggerFlagIsOn: true) } - private func upEvent(_ keyCode: Int) -> DictationKeyRouter.Event { - .flagsChanged(keyCode: keyCode, triggerFlagIsOn: false) + private func upEvent(_ key: TriggerKey) -> DictationKeyRouter.Event { + .flagsChanged(keyCode: key.keyCode, triggerFlagIsOn: false) + } + + private func modifierRouter() -> DictationKeyRouter { + DictationKeyRouter(binding: .modifier(trigger)) } @Test("a held press is start → stop") func holdIsStartStop() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) #expect(router.handle(upEvent(trigger), at: .seconds(2)) == .stop) } @@ -32,7 +37,7 @@ struct DictationKeyRouterTests { func repeatedDownStateIsDeduped() { // While the trigger is held, another flags delivery can re-report its bit // still set; re-arming the gate on it would corrupt the tap/hold timing. - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) #expect(router.handle(downEvent(trigger), at: .milliseconds(50)) == .none) // The eventual release still stops the (single) dictation. @@ -41,7 +46,7 @@ struct DictationKeyRouterTests { @Test("an up-state delivery with no tracked down is ignored") func upWithoutDownIsIgnored() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(upEvent(trigger), at: .zero) == .none) } @@ -49,29 +54,41 @@ struct DictationKeyRouterTests { func otherKeycodeFlagsAreIgnored() { // E.g. right ⌥ going down while right ⌘ is bound: the delivery's flags may // even carry the trigger's bit, but the event isn't about the bound key. - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(otherModifier), at: .zero) == .none) #expect(router.handle(upEvent(otherModifier), at: .seconds(2)) == .none) } @Test("another key over a fresh press is a combo and cancels") func comboCancelsFreshCapture() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) #expect(router.handle(.keyDown(keyCode: 8), at: .milliseconds(100)) == .cancel) // ⌘C } @Test("the trigger's own keyDown is not a combo") func triggerKeyDownIsNotACombo() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() + #expect(router.handle(downEvent(trigger), at: .zero) == .start) + #expect(router.handle(.keyDown(keyCode: trigger.keyCode), at: .milliseconds(100)) == .none) + #expect(router.handle(upEvent(trigger), at: .seconds(2)) == .stop) + } + + @Test("mouse events never reach a modifier binding's gate") + func modifierBindingIgnoresOtherFamilies() { + // The tap's mask covers every family any binding might need, so a modifier + // binding sees extra-button clicks too — which may neither drive nor cancel + // its gate (only a keyDown marks a combo). + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) - #expect(router.handle(.keyDown(keyCode: trigger), at: .milliseconds(100)) == .none) + #expect(router.handle(.mouseDown(button: 3), at: .milliseconds(60)) == .none) + #expect(router.handle(.mouseUp(button: 3), at: .milliseconds(70)) == .none) #expect(router.handle(upEvent(trigger), at: .seconds(2)) == .stop) } @Test("a short tap latches; the next tap stops") func tapToToggle() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) #expect(router.handle(upEvent(trigger), at: .milliseconds(200)) == .none) // latched #expect(router.handle(downEvent(trigger), at: .seconds(5)) == .none) @@ -84,14 +101,14 @@ struct DictationKeyRouterTests { @Test("reset while idle reports nothing discarded") func resetWhileIdle() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() let discarded = router.reset() #expect(!discarded) } @Test("reset mid-recording reports the discarded recording") func resetMidRecordingReportsDiscard() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) let discarded = router.reset() #expect(discarded) @@ -102,7 +119,7 @@ struct DictationKeyRouterTests { @Test("reset over a latched recording reports the discarded recording") func resetOverLatchedReportsDiscard() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) #expect(router.handle(upEvent(trigger), at: .milliseconds(200)) == .none) // latched let discarded = router.reset() @@ -118,7 +135,7 @@ struct DictationKeyRouterTests { // the user's whole next press does nothing. `DictationKeyTap`'s // `syncAfterTerminalPhase()` calls `reset()` to clear it; this pins that a // reset genuinely restores the next press. - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) #expect(router.handle(upEvent(trigger), at: .milliseconds(200)) == .none) // latched @@ -134,10 +151,10 @@ struct DictationKeyRouterTests { func resetWhileHeldThenReleaseIsInert() { // The auto-release/failed-press case where the trigger is still physically // held when the phase goes terminal. `syncAfterTerminalPhase` resets anyway - // (the dictation is over), which clears the modifier tracker — so the release + // (the dictation is over), which clears the down tracker — so the release // that follows must route to `.none` rather than emitting a spurious `.stop`, // and the press after that must start normally. - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) router.reset() // terminal phase arrived while the key is still down @@ -146,15 +163,15 @@ struct DictationKeyRouterTests { #expect(router.handle(downEvent(trigger), at: .seconds(2)) == .start) } - @Test("rebind mid-recording discards it and switches keycodes") + @Test("rebind mid-recording discards it and switches bindings") func rebindMidRecording() { - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) // Rebinding means the old key's up-event can never match — the caller must // cancel the capture rather than let the auto-release cap paste it. - let discarded = router.rebind(triggerKeyCode: otherModifier) + let discarded = router.rebind(binding: .modifier(otherModifier)) #expect(discarded) - #expect(router.triggerKeyCode == otherModifier) + #expect(router.binding == .modifier(otherModifier)) // The old key is now irrelevant; the new one drives dictation. #expect(router.handle(downEvent(trigger), at: .seconds(1)) == .none) #expect(router.handle(downEvent(otherModifier), at: .seconds(2)) == .start) @@ -162,8 +179,8 @@ struct DictationKeyRouterTests { @Test("rebind while idle reports nothing discarded") func rebindWhileIdle() { - var router = DictationKeyRouter(triggerKeyCode: trigger) - let discarded = router.rebind(triggerKeyCode: otherModifier) + var router = modifierRouter() + let discarded = router.rebind(binding: .modifier(otherModifier)) #expect(!discarded) } @@ -172,7 +189,7 @@ struct DictationKeyRouterTests { // The tap was disabled mid-sentence. The key-up hasn't happened yet, so it is // still coming and the gate is coherent — resetting here would throw away // speech the user is in the middle of. - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) // Bound to a local rather than asserted inline: `#expect` rewrites a bare @@ -193,7 +210,7 @@ struct DictationKeyRouterTests { // will never arrive. Left latched, the session would sit in `.recording` until // the auto-release cap pasted an unprompted transcript — so the reset must // report the discarded recording for the host to cancel upstream. - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) let discarded = router.recoverFromDroppedEvents(triggerStillHeld: false) @@ -210,7 +227,7 @@ struct DictationKeyRouterTests { // The common case: the tap times out with no dictation in flight. Neither // branch may claim a recording was discarded, or the host cancels a session // that was never recording. - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() let discardedAfterRelease = router.recoverFromDroppedEvents(triggerStillHeld: false) #expect(!discardedAfterRelease) let discardedWhileHeld = router.recoverFromDroppedEvents(triggerStillHeld: true) @@ -224,7 +241,7 @@ struct DictationKeyRouterTests { // A tapped recording has no key held by definition, so `triggerStillHeld` is // false and the gate is latched — the state most at risk of being stranded, // since nothing is coming to close it. - var router = DictationKeyRouter(triggerKeyCode: trigger) + var router = modifierRouter() #expect(router.handle(downEvent(trigger), at: .zero) == .start) #expect(router.handle(upEvent(trigger), at: .milliseconds(100)) == .none) // latched diff --git a/Tests/BlurtEngineTests/TriggerBindingTests.swift b/Tests/BlurtEngineTests/TriggerBindingTests.swift new file mode 100644 index 00000000..f65c24c9 --- /dev/null +++ b/Tests/BlurtEngineTests/TriggerBindingTests.swift @@ -0,0 +1,230 @@ +import Testing + +@testable import BlurtEngine + +/// The single-`Int`-slot encoding behind `TriggerKeyStore`: both binding +/// families round-trip through `persistedValue`/`fromPersisted`, their codes +/// can't collide, and garbage decodes to the right-⌘ default rather than an +/// invalid selection. +@Suite("TriggerBinding") +struct TriggerBindingTests { + @Test("modifier bindings persist as their bare keycode — the pre-Custom encoding") + func modifierEncodingIsUnchanged() { + // Zero migration: an install that stored 54/61/63 before Custom bindings + // existed must decode to the same modifier afterwards. + #expect(TriggerBinding.modifier(.rightCommand).persistedValue == 54) + #expect(TriggerBinding.modifier(.rightOption).persistedValue == 61) + #expect(TriggerBinding.fromPersisted(54) == .modifier(.rightCommand)) + #expect(TriggerBinding.fromPersisted(61) == .modifier(.rightOption)) + } + + @Test("every binding kind round-trips through the persisted slot") + func roundTrips() { + var bindings = TriggerKey.allCases.map { TriggerBinding.modifier($0) } + bindings += (TriggerBinding.minimumMouseButton...TriggerBinding.maximumMouseButton) + .map { TriggerBinding.mouseButton($0) } + for binding in bindings { + #expect(TriggerBinding.fromPersisted(binding.persistedValue) == binding) + } + } + + @Test("the two families' persisted codes are disjoint") + func encodingsAreDisjoint() { + // The whole zero-migration scheme rests on this: a mouse code must never + // read as a modifier keycode, and vice versa. + let modifierCodes = Set(TriggerKey.allCases.map(\.rawValue)) + let mouseCodes = Set( + (TriggerBinding.minimumMouseButton...TriggerBinding.maximumMouseButton) + .map { TriggerBinding.mouseButton($0).persistedValue }) + #expect(modifierCodes.isDisjoint(with: mouseCodes)) + } + + @Test("an unset slot decodes to the right-⌘ default") + func unsetFallsBack() { + // UserDefaults reads an absent integer as 0. + #expect(TriggerBinding.fromPersisted(0) == .modifier(.rightCommand)) + } + + @Test("a persisted keycode from a removed option falls back to right ⌘") + func removedOptionsFallBack() { + // right ⌃ (62) and Caps Lock (57) were never options; anyone who had one + // saved must decode to the default rather than an invalid selection. + #expect(TriggerBinding.fromPersisted(62) == .modifier(.rightCommand)) + #expect(TriggerBinding.fromPersisted(57) == .modifier(.rightCommand)) + } + + /// `fn` was a third modifier option and was removed. Its keycode gets a + /// **deliberate** landing spot rather than the generic right-⌘ fallback: right + /// ⌥ is the other right-side modifier, so an install that had `fn` keeps a + /// one-key trigger on the same side of the keyboard. + @Test("a persisted fn (the removed option) migrates to right ⌥, not the default") + func persistedFunctionKeyMigratesToRightOption() { + #expect(TriggerBinding.legacyFunctionKeyCode == 63) + #expect(TriggerBinding.fromPersisted(63) == .modifier(.rightOption)) + // And it is genuinely a migration, not a decode: `fn` is no longer a case. + #expect(TriggerKey(rawValue: 63) == nil) + } + + // MARK: - Chords + + @Test("a chord round-trips through the packed slot, keycode and modifiers intact") + func chordRoundTrips() { + let chord = TriggerBinding.chord(keyCode: 2, modifiers: [.control, .option]) // ⌃⌥D + let packed = chord.persistedValue + #expect(packed > TriggerBinding.chordCodeBase) + #expect(TriggerBinding.fromPersisted(packed) == chord) + } + + @Test("every modifier combination round-trips for a representative key") + func everyModifierSetRoundTrips() { + let keyCode = 2 // D + for bits in 1...TriggerBinding.ChordModifiers.all.rawValue { + let modifiers = TriggerBinding.ChordModifiers(rawValue: bits) + let chord = TriggerBinding.chord(keyCode: keyCode, modifiers: modifiers) + #expect(TriggerBinding.fromPersisted(chord.persistedValue) == chord) + } + } + + @Test("chord codes are disjoint from the modifier keycodes and the mouse namespace") + func chordCodesAreDisjoint() { + let chordCodes = Set( + (1...TriggerBinding.ChordModifiers.all.rawValue).map { + TriggerBinding.chord( + keyCode: 2, modifiers: TriggerBinding.ChordModifiers(rawValue: $0) + ).persistedValue + }) + let modifierCodes = Set(TriggerKey.allCases.map(\.rawValue)) + let mouseCodes = Set( + (TriggerBinding.minimumMouseButton...TriggerBinding.maximumMouseButton) + .map { TriggerBinding.mouseButton($0).persistedValue }) + #expect(chordCodes.isDisjoint(with: modifierCodes)) + #expect(chordCodes.isDisjoint(with: mouseCodes)) + } + + @Test("a chord with no modifiers or a modifier as its key is refused at capture") + func chordCapturePolicy() { + #expect(TriggerBinding.chordBinding(forKeyCode: 2, modifiers: []) == .failure(.bareKey)) + #expect( + TriggerBinding.chordBinding(forKeyCode: 96, modifiers: []) == .failure(.bareKey)) // bare F5 + // A modifier keycode can't be a chord's key half, even with modifiers held. + #expect( + TriggerBinding.chordBinding(forKeyCode: 54, modifiers: [.control]) == .failure(.modifierOnly)) + #expect( + TriggerBinding.chordBinding(forKeyCode: 63, modifiers: [.control]) == .failure(.modifierOnly)) + // The everyday case succeeds. + #expect( + TriggerBinding.chordBinding(forKeyCode: 2, modifiers: [.control, .option]) + == .success(.chord(keyCode: 2, modifiers: [.control, .option]))) + // A modifier + F-key chord is fine — it's the *bare* F-key that was refused. + #expect( + TriggerBinding.chordBinding(forKeyCode: 96, modifiers: [.option]) + == .success(.chord(keyCode: 96, modifiers: [.option]))) + } + + @Test("system-reserved chords are refused") + func reservedChordsRefused() { + // Each would fire its system action underneath the dictation, because the + // tap swallows nothing. + #expect(TriggerBinding.chordBinding(forKeyCode: 12, modifiers: [.command]) == .failure(.reserved)) // ⌘Q + #expect(TriggerBinding.chordBinding(forKeyCode: 13, modifiers: [.command]) == .failure(.reserved)) // ⌘W + #expect(TriggerBinding.chordBinding(forKeyCode: 48, modifiers: [.command]) == .failure(.reserved)) // ⌘⇥ + #expect(TriggerBinding.chordBinding(forKeyCode: 49, modifiers: [.command]) == .failure(.reserved)) // ⌘Space + #expect( + TriggerBinding.chordBinding(forKeyCode: 12, modifiers: [.control, .command]) + == .failure(.reserved)) // ⌃⌘Q + // Adding a modifier makes it the user's own shortcut again, not the system's. + #expect( + TriggerBinding.chordBinding(forKeyCode: 12, modifiers: [.option, .command]) + == .success(.chord(keyCode: 12, modifiers: [.option, .command]))) + } + + @Test("a structurally invalid packed chord falls back to right ⌘") + func invalidChordsFallBack() { + let base = TriggerBinding.chordCodeBase + // No modifier bits: a bare key was never bindable, so a stored one is garbage. + #expect(TriggerBinding.fromPersisted(base | 2) == .modifier(.rightCommand)) + // A modifier keycode as the key half. + #expect( + TriggerBinding.fromPersisted(TriggerBinding.packedChord(keyCode: 54, modifiers: [.control])) + == .modifier(.rightCommand)) + // Bits set above the four modifiers. + #expect( + TriggerBinding.fromPersisted(base | (0x10 << TriggerBinding.chordModifierShift) | 2) + == .modifier(.rightCommand)) + } + + @Test("chord labels render as macOS glyph sequences in canonical order") + func chordLabels() { + // ⌃⌥⇧⌘ is Apple's order regardless of the order the user pressed them in. + #expect(TriggerBinding.chord(keyCode: 2, modifiers: [.option, .control]).label == "⌃⌥D") + #expect( + TriggerBinding.chord(keyCode: 2, modifiers: [.command, .shift, .option, .control]).label + == "⌃⌥⇧⌘D") + #expect(TriggerBinding.chord(keyCode: 49, modifiers: [.control]).label == "⌃Space") + #expect(TriggerBinding.chord(keyCode: 96, modifiers: [.option]).label == "⌥F5") + #expect(TriggerBinding.chord(keyCode: 36, modifiers: [.command, .shift]).label == "⇧⌘↩") + // An unmapped keycode still names itself rather than rendering as nothing. + #expect(TriggerBinding.chord(keyCode: 200, modifiers: [.control]).label == "⌃key 200") + } + + @Test("a bare keyboard keycode falls back to right ⌘ — a key alone isn't bindable") + func keyboardKeycodesFallBack() { + // Only a *chord* can carry a keyboard key, and chords live in their own + // namespace, so a bare keycode in the slot is garbage (this includes the + // F-keys an earlier revision of the Custom option stored raw). + #expect(TriggerBinding.fromPersisted(0x31) == .modifier(.rightCommand)) // Space + #expect(TriggerBinding.fromPersisted(96) == .modifier(.rightCommand)) // F5 + #expect(TriggerBinding.fromPersisted(122) == .modifier(.rightCommand)) // F1 + } + + @Test("a mouse code outside the bindable buttons falls back to right ⌘") + func outOfRangeMouseCodesFallBack() { + let base = TriggerBinding.mouseButtonCodeBase + #expect(TriggerBinding.fromPersisted(base + 0) == .modifier(.rightCommand)) // left click + #expect(TriggerBinding.fromPersisted(base + 1) == .modifier(.rightCommand)) // right click + #expect(TriggerBinding.fromPersisted(base + 32) == .modifier(.rightCommand)) // past CGEvent's range + #expect(TriggerBinding.fromPersisted(base + 2) == .mouseButton(2)) // first bindable + #expect(TriggerBinding.fromPersisted(base + 31) == .mouseButton(31)) // last bindable + } + + @Test("mouseButtonBinding(forButton:) refuses the left and right click") + func mouseBindingPolicy() { + #expect(TriggerBinding.mouseButtonBinding(forButton: 0) == nil) // left + #expect(TriggerBinding.mouseButtonBinding(forButton: 1) == nil) // right + #expect(TriggerBinding.mouseButtonBinding(forButton: 2) == .mouseButton(2)) // "Mouse 3" + #expect(TriggerBinding.mouseButtonBinding(forButton: 3) == .mouseButton(3)) // "Mouse 4" + #expect(TriggerBinding.mouseButtonBinding(forButton: 31) == .mouseButton(31)) + #expect(TriggerBinding.mouseButtonBinding(forButton: 32) == nil) // past CGEvent's range + #expect(TriggerBinding.mouseButtonBinding(forButton: -1) == nil) + } + + @Test("labels name the binding the way the UI should") + func labels() { + #expect(TriggerBinding.modifier(.rightCommand).label == "right ⌘") + // Raw button numbers are 0-based (0 left, 1 right, 2 wheel/middle); the + // display name is 1-based the way mice are numbered for users. So the stored + // 2 shows as "Mouse 3" and IS the middle click — this pins that off-by-one + // deliberately, because a label that disagrees with the raw number is + // exactly what would mislead someone reading the log or the code. + #expect(TriggerBinding.mouseButton(2).label == "Mouse 3") + #expect(TriggerBinding.mouseButton(3).label == "Mouse 4") + #expect(TriggerBinding.mouseButton(31).label == "Mouse 32") + } + + /// The tap is listen-only, so some bindings reach the focused app as well as + /// Blurt. The UI says so for exactly the two families where it bites. + @Test("the wheel click and every chord carry a pass-through caution; nothing else does") + func passThroughNotes() throws { + #expect(TriggerBinding.modifier(.rightCommand).passThroughNote == nil) + #expect(TriggerBinding.modifier(.rightOption).passThroughNote == nil) + // Button 2 is the wheel click, which browsers and terminals act on. + let wheel = try #require(TriggerBinding.mouseButton(2).passThroughNote) + #expect(wheel.contains("wheel click")) + // Side buttons: almost nothing claims them, so no caution. + #expect(TriggerBinding.mouseButton(3).passThroughNote == nil) + #expect(TriggerBinding.mouseButton(4).passThroughNote == nil) + // A chord names itself in its caution, since the app that owns it varies. + let chord = TriggerBinding.chord(keyCode: 2, modifiers: [.control, .option]) + #expect(chord.passThroughNote?.contains("⌃⌥D") == true) + } +} diff --git a/Tests/BlurtEngineTests/TriggerKeyStoreTests.swift b/Tests/BlurtEngineTests/TriggerKeyStoreTests.swift index 9ab18713..5a40cbd3 100644 --- a/Tests/BlurtEngineTests/TriggerKeyStoreTests.swift +++ b/Tests/BlurtEngineTests/TriggerKeyStoreTests.swift @@ -8,21 +8,38 @@ struct TriggerKeyStoreTests { @Test("defaults to right command when unset") func defaultsToRightCommand() { let store = TriggerKeyStore(defaults: freshDefaults()) - #expect(store.triggerKey == .rightCommand) + #expect(store.triggerBinding == .modifier(.rightCommand)) } - @Test("persists and reads back a chosen key") - func roundTrips() { + @Test("persists and reads back a chosen modifier") + func roundTripsAModifier() { let defaults = freshDefaults() let store = TriggerKeyStore(defaults: defaults) - store.triggerKey = .rightOption - #expect(TriggerKeyStore(defaults: defaults).triggerKey == .rightOption) + store.triggerBinding = .modifier(.rightOption) + #expect(TriggerKeyStore(defaults: defaults).triggerBinding == .modifier(.rightOption)) + } + + @Test("persists and reads back a custom mouse-button binding") + func roundTripsAMouseBinding() { + let defaults = freshDefaults() + let store = TriggerKeyStore(defaults: defaults) + store.triggerBinding = .mouseButton(3) // "Mouse 4" + #expect(TriggerKeyStore(defaults: defaults).triggerBinding == .mouseButton(3)) } @Test("an unknown stored code falls back to the default") func unknownFallsBack() { let defaults = freshDefaults() defaults.set(123, forKey: TriggerKeyStore.defaultsKey) - #expect(TriggerKeyStore(defaults: defaults).triggerKey == .rightCommand) + #expect(TriggerKeyStore(defaults: defaults).triggerBinding == .modifier(.rightCommand)) + } + + @Test("a modifier keycode stored before Custom bindings existed still decodes") + func preCustomSlotStillDecodes() { + // The store has always written the bare keycode for modifiers; the + // TriggerBinding encoding must keep reading those installs unchanged. + let defaults = freshDefaults() + defaults.set(61, forKey: TriggerKeyStore.defaultsKey) // right ⌥, pre-Custom form + #expect(TriggerKeyStore(defaults: defaults).triggerBinding == .modifier(.rightOption)) } } diff --git a/Tests/BlurtEngineTests/TriggerKeyTests.swift b/Tests/BlurtEngineTests/TriggerKeyTests.swift index 9096704b..23c01613 100644 --- a/Tests/BlurtEngineTests/TriggerKeyTests.swift +++ b/Tests/BlurtEngineTests/TriggerKeyTests.swift @@ -8,7 +8,6 @@ struct TriggerKeyTests { func keyCodes() { #expect(TriggerKey.rightCommand.keyCode == 54) #expect(TriggerKey.rightOption.keyCode == 61) - #expect(TriggerKey.function.keyCode == 63) } @Test("every case has a non-empty label") @@ -25,18 +24,20 @@ struct TriggerKeyTests { #expect(TriggerKey(rawValue: 999) == nil) } - @Test("a persisted right-⌃ keycode (a removed option) falls back to right ⌘") - func removedRightControlFallsBack() { - // right ⌃ (keycode 62) was dropped as an option; anyone who had it saved must - // decode to the default rather than an invalid selection. + @Test("removed options (right ⌃, Caps Lock, fn) are not modifiers") + func removedOptionsDoNotDecode() { + // right ⌃ (62) and Caps Lock (57) were never options, and `fn` (63) was + // removed as one; where each persisted keycode lands is + // `TriggerBinding.fromPersisted`'s job (see `TriggerBindingTests`). #expect(TriggerKey(rawValue: 62) == nil) - #expect(TriggerKey.fromPersisted(62) == .rightCommand) + #expect(TriggerKey(rawValue: 57) == nil) + #expect(TriggerKey(rawValue: 63) == nil) } - @Test("a persisted Caps Lock keycode (a removed option) falls back to right ⌘") - func removedCapsLockFallsBack() { - #expect(TriggerKey(rawValue: 57) == nil) - #expect(TriggerKey.fromPersisted(57) == .rightCommand) + @Test("only the two right-side modifiers are offered") + func curatedToRightSideModifiers() { + // The picker renders `allCases`, so this is what the user can choose from. + #expect(TriggerKey.allCases == [.rightCommand, .rightOption]) } // The hotkey tap reads the *device-dependent* modifier bit (which physical @@ -48,7 +49,6 @@ struct TriggerKeyTests { func deviceMasks() { #expect(TriggerKey.rightCommand.deviceModifierMask == 0x10) // NX_DEVICERCMDKEYMASK #expect(TriggerKey.rightOption.deviceModifierMask == 0x40) // NX_DEVICERALTKEYMASK - #expect(TriggerKey.function.deviceModifierMask == 0x80_0000) // kCGEventFlagMaskSecondaryFn } @Test("right-⌘ mask does not collide with the left-⌘ or generic ⌘ bit") diff --git a/scripts/check-invariants.sh b/scripts/check-invariants.sh index 92886b7e..a3c11ae7 100755 --- a/scripts/check-invariants.sh +++ b/scripts/check-invariants.sh @@ -118,7 +118,7 @@ ADVICE=( "config.prompt was replaced by config.conversation_context (ConversationContext)" "injection is always clipboard paste (save → write → ⌘V → settle → restore)" "Blurt is a Dock app first; the MenuBarExtra item is layered on, never depended on" - "the trigger is a home-grown lone modifier (CGEventTap + DictationKeyGate)" + "the trigger is home-grown (CGEventTap + DictationKeyGate); the package ban stands, chords are allowed" "updates are download-only; extend UpdateCheckModel, don't install for the user" "use an isolated service (see KeychainStoreTests) or InMemoryAPIKeyStore" "deleted types stay deleted — no deprecated re-exports" @@ -158,7 +158,7 @@ TABLE_ANCHORS=( "Bring back \`config.prompt\`" "Add a keystroke-typing paste path or a length threshold" "Add \`LSUIElement\` or a menu-bar-**only** mode" - "Add a \`KeyboardShortcuts\` package or a key+modifier chord" + "Add a \`KeyboardShortcuts\` package (or any key-binding dependency)" "Add a self-replacing install or background auto-updater" "Touch the real Keychain in tests" "Add backwards-compat shims for removed types"