Skip to content

feat: Custom dictation trigger — bind a keyboard chord or an extra mouse button - #144

Closed
claude[bot] wants to merge 8 commits into
mainfrom
feat/custom-dictation-key
Closed

feat: Custom dictation trigger — bind a keyboard chord or an extra mouse button#144
claude[bot] wants to merge 8 commits into
mainfrom
feat/custom-dictation-key

Conversation

@claude

@claude claude Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Requested by Mez, Griffin Sharp, Alex Kroman · Slack thread

What & why

Adds a fourth "Custom…" option to Settings > General > Shortcut > Dictation key that lets the user bind a keyboard chord (modifiers plus a key, e.g. ⌃⌥D) or an extra mouse button (Mouse 3 and up) as the dictation trigger, with press-to-capture UX and the exact same push-to-talk / tap-to-toggle semantics as the built-in modifier triggers. It also removes fn as a built-in option.

Before: the dictation key could only be one of three lone modifiers — right ⌘, right ⌥, or fn.
After: the menu offers right ⌘ / right ⌥ plus Custom…, which opens a sheet that records the next chord or extra-button click. The picker then names what's bound ("⌃⌥D", "Mouse 4"), and the ready screen, menu bar, and permissions footer render it live. Default stays right ⌘; nothing migrates.

Scope history (all maintainer-directed, in the linked thread): v1 allowed F-keys + mouse buttons → F-keys removed, mouse-only → chords added back as chords (not bare F-keys) and fn removed. Removed paths were deleted, not left dormant, per the repo's cleanup discipline.

Settled decision updated — @alexkroman authorized this

AGENTS.md's settled-decisions table named "a key+modifier chord" as a don't-reintroduce item. Alex, who authored that decision, asked for chords in the PR thread, so the row and its rationale now record that chords are supported and why. What has not changed:

  • No package — no KeyboardShortcuts, no SPM dependency for key handling. That ban is untouched, its import KeyboardShortcuts rule still fires, and its skill anchor is verbatim as before; only the table row's wording (and the rule's pinned slice of it) moved, updated together so check-invariants.sh --self-test passes.
  • The tap stays .listenOnly — an active tap would make macOS wait on Blurt before delivering every keystroke system-wide. So a bound chord is not swallowed: an app that already owns ⌃⌥D still acts on it. Rather than hide that, TriggerBinding.passThroughNote states it in the Settings footer and the capture sheet says it up front.
  • Still one control, never a bare key — a bare key would type into the focused app on every dictation.

What the recorder refuses

Input Result
Bare key (no modifiers) Refused — "Hold at least one modifier… a key on its own would type into whatever app is focused"
A modifier as the key half (⌃ + ⌘, Caps Lock, fn) Refused — pick a real key, or right ⌘ / right ⌥ from the menu
⌘Q, ⌘W, ⌘H, ⌘⇥, ⌘Space, ⌃⌘Q Refused — they'd fire their system action underneath the dictation (and macOS eats ⌘⇥/⌘Space before any monitor sees them, so they could never be captured anyway)
Left / right click Never arrives (not otherMouseDown)
Esc Cancels the sheet, always
Wheel click (Mouse 3) Allowed, with a caution — see below

"Is Mouse 3 always the middle click?" — yes, and the off-by-one is now pinned

The stored value is the raw 0-based CGEvent/NSEvent buttonNumber: 0 left, 1 right, 2 wheel/middle, 3+ side buttons; only 2 and up are bindable. The display name is 1-based, the way mice and their drivers number buttons for users — so stored 2 renders as "Mouse 3" and is the middle (wheel) click. That is now stated in the doc comments, asserted by a label test (mouseButton(2).label == "Mouse 3"), and consistent across UI, tests, logs, and this description. Because browsers and terminals act on the wheel click and the tap swallows nothing, it is not hard-blocked but carries its own passThroughNote ("Mouse 3 is the wheel click… apps that use it will still act on it"); side buttons get no caution because almost nothing claims them.

fn removal and migration

fn is the one modifier macOS itself claims (dictation, emoji picker, the F-key row), so it never belonged in a list of keys a solo press could own. TriggerKey is now right ⌘ / right ⌥ only and its secondary-fn mask bit is gone (chords read the generic modifier masks instead). A persisted fn (keycode 63) migrates to right ⌥, not to the generic right-⌘ fallback — it's the other right-side modifier, so those users keep a one-key trigger on the same side of the keyboard. Pinned by persistedFunctionKeyMigratesToRightOption.

How

  • TriggerBinding: .modifier(TriggerKey) / .chord(keyCode:modifiers:) / .mouseButton(Int), all in the existing single Int slot BlurtTriggerKeyCode — modifiers as bare keycodes (zero migration), mouse buttons at 0x10000 + button, chords packed as 0x20000 | (modifiers << 8) | keyCode (keycode low byte, four modifier bits above; keycodes are 16-bit so the tags can't collide — pinned by a disjointness test). One slot keeps the @AppStorage live-render pattern. ChordModifiers is a side-agnostic OptionSet with a persisted bit layout and glyphs in Apple's ⌃⌥⇧⌘ order. fromPersisted stays the single decode-with-default rule (garbage → right ⌘; a structurally invalid packed chord counts as garbage). Capture policy is engine logic: chordBinding(forKeyCode:modifiers:) returns Result<TriggerBinding, ChordRefusal> so the reason survives into the sheet's copy.
  • DictationKeyRouter: .keyDown(keyCode:modifiers:) / .keyUp(keyCode:) / .flagsChanged(…, modifiers:) / mouse events, routed per binding family through one shared edge filter. A chord fires only on the exact required modifier set and ends on the key's keyUp or the release of any required modifier (the realistic ⌃⌥D release order is ⌃ first — waiting for D's keyUp would strand a hold recording). Combo cancel remains modifier-bindings-only: a chord's own modifiers are part of the trigger, and typing during a latched dictation is just typing. DictationKeyGate is still untouched, so hold ≥ 1 s = push-to-talk and shorter = tap-to-latch behave identically for all three families.
  • DictationEventDecoder: the CGEvent → router-event reduction (engine-side so it's testable), now also mapping flags → ChordModifiers via the generic masks (⌃ is ⌃; Caps Lock and fn deliberately excluded) and dropping autorepeat key-downs.
  • DictationKeyTap: mask keyDown | keyUp | flagsChanged | otherMouseDown | otherMouseUp (fixed at creation; the router ignores what the binding doesn't need, so rebinding never recreates the tap), .listenOnly, and dropped-event recovery per family — for a chord, keyState and the required modifiers via flagsState.
  • UI: the capture sheet shows held modifiers live (⌃⌥…) as the chord forms, states the pass-through up front, and turns each engine refusal into one sentence; mouse capture is unchanged (local + global otherMouseDown monitors). The Shortcut footer appends passThroughNote for the bound binding.
  • Developer-mode capture log: ~/Library/Logs/Blurt/capture-events.jsonl records every event the recorder sees — accepted or refused, chords included — with kind, outcome token (captured, refused-bare-key, refused-modifier-only, refused-reserved-chord, refused-button, cancelled), button number, keycode, flags, autorepeat. Third file of the existing log, sharing its gate/queue/encoder; off by default, nothing on disk.
  • Tests: chord round-trips for every modifier set, namespace disjointness, invalid-packing fallbacks, capture policy + reserved chords, glyph labels, fn migration, pass-through notes; a chord router suite (hold/tap, exact-modifier matching, partial-modifier release, autorepeat dedup, no-combo rule, latched survival, recovery, cross-family rebind); real-CGEvent chord decode fixtures (flag combinations including device side bits, Caps Lock/fn exclusion, autorepeat drop, partial release). All pure — no event taps, no audio.

Notes for review:

  • No new app-target files (xcodegen isn't available here and project.pbxproj must not be hand-edited) — the capture sheet lives in HotkeyStepView.swift; new engine files are fine.
  • A local NSEvent monitor can't see chords macOS reserves — the same reason those are refused rather than discouraged. Documented in the sheet's doc comment.
  • No XCUITest for chord capture (the runner can't synthesize a chord into a sheet reliably); the decoder fixtures plus router/gate suites cover the logic.

How it was tested

Authored in a Linux sandbox — no macOS toolchain, so CI on macos-26 is the authority on green. scripts/check.sh --portable passes locally (repo-integrity guards, settled-decision invariants incl. --self-test, shell portability, actionlint/zizmor/prettier/markdownlint/shellcheck/shfmt, evals, release.test.sh). The pre-chord revision (ed121cf) was fully green on CI — required check, gate, compile, format-patch, CodeQL, Aikido; CI is running on the chord head now and I'm watching it.

  • scripts/check.sh passes (or CI will, if I'm not on a Mac) — portable subset green locally; CI runs the Swift side
  • I read AGENTS.md and this doesn't reintroduce anything deliberately removed — it updates one settled decision at the maintainer's explicit request; see above
  • Docs updated if behavior changed

Adds a fourth "Custom…" option to Settings > General > Shortcut > Dictation
key: a press-to-capture sheet that binds a lone function key (F1-F20) or an
extra mouse button (Mouse 3 and up) as the dictation trigger, with the same
tap-to-toggle / hold-to-talk semantics as the modifier triggers.

Engine:
- New TriggerBinding enum (.modifier / .key / .mouseButton) encoding every
  binding into the existing single Int slot (BlurtTriggerKeyCode): modifiers
  keep their bare keycodes (zero migration), F-keys store their raw keycodes
  (disjoint from the modifier codes), mouse buttons live at 0x10000+button.
  fromPersisted keeps the right-command fallback for garbage; the capture
  policy (which keys/buttons are bindable) lives here as tested engine logic.
- DictationKeyRouter now routes per binding family (flagsChanged for
  modifiers, keyDown/keyUp for F-keys, mouseDown/mouseUp for buttons) through
  a shared edge filter, so autorepeat and re-reported flag bits can't
  double-fire. The other-key combo cancel applies to modifier bindings only:
  cmd-C is a real shortcut, F5+K is just typing. rebind takes a
  TriggerBinding. DictationKeyGate is unchanged.
- TriggerKeyStore persists TriggerBinding; TriggerKey.fromPersisted's job
  moved to TriggerBinding.fromPersisted.

App:
- DictationKeyTap widens the tap mask (keyUp, otherMouseDown/Up), filters
  autorepeat key-downs, and answers dropped-event recovery per binding family
  (flagsState / keyState / buttonState). Still .listenOnly - it swallows
  nothing, which is exactly why printable keys and the left/right buttons are
  refused in the recorder.
- HotkeyStepView gains the Custom… row and the capture sheet (NSEvent local
  key/click monitors plus a global click monitor; Esc cancels; disallowed
  inputs are refused with an explanation).
- BoundTriggerKey.swift now wraps TriggerBinding (BoundTriggerBinding); the
  ready screen, menu bar, and permissions footer render any binding's label.

Tests: TriggerBindingTests (round-trips, disjoint namespaces, fallback,
capture policy, labels) and router suites for key and mouse bindings
(hold/tap, dedup, cross-family irrelevance, rebind, recovery).

Docs: AGENTS.md Hotkey section + the KeyboardShortcuts row's rationale cell,
the guardrails skill bullet, and the engine README now describe the single
lone *key* (modifier by default). The mechanized invariant anchors are
untouched; check-invariants.sh --self-test still passes.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZB3QrNeP732HSJzx3hxLj
@gsharp-aai
gsharp-aai requested a review from alexkroman August 14, 2026 21:51
claude added 3 commits August 14, 2026 22:06
DictationKeyRouterTests.swift grew to 419 lines with the key/mouse coverage,
tripping SwiftLint's file_length warning (400), which --strict promotes to a
failure. Move the F-key and mouse-button suites (plus the cross-family rebind
and mouse recovery cases) to DictationKeyRouterCustomBindingTests.swift; the
original file keeps the modifier-binding behavior and the reset/rebind/
recovery contract. No test added, removed, or reworded.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZB3QrNeP732HSJzx3hxLj
…sheet

With Apple's default "Use F1, F2, etc. keys as standard function keys" off, a
bare top-row press on a laptop keyboard is a systemDefined media event, not a
keyDown with an F-keycode - so it reaches neither the capture recorder nor the
trigger tap, and looks like "F-keys don't work". Held with fn (or with the
setting on, or on most external keyboards) the same key delivers a real F-key
keyDown, which captures and triggers fine.

Say so in the capture sheet's caption and in AGENTS.md's Hotkey section. No
behavior change: media keys stay out of scope by construction (the recorder
watches keyDown only, and the tap's mask has no systemDefined bit).

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZB3QrNeP732HSJzx3hxLj
…res and a capture-diagnostics log

Maintainer scope change: F-key/custom-keyboard-key interception is removed
entirely. The Custom… option now binds extra mouse buttons only (Mouse 3 and
up); the three modifier options are unchanged.

- TriggerBinding drops .key(code:) and the F-key table; the persisted encoding
  keeps modifiers as bare keycodes and mouse buttons at 0x10000+button, and a
  stored bare keycode (e.g. 96/F5 from the earlier cut) now decodes to the
  right-command default.
- DictationKeyRouter drops the .keyUp event and the key-binding family;
  keyDown remains solely the modifier bindings' combo probe. The tap's mask
  loses keyUp and the autorepeat filter (nothing consumed either any more).
- The capture sheet is mouse-only: "Press a mouse button (Mouse 3 or
  higher)…", Esc cancels, keyboard presses and unbindable buttons are refused
  with an explanation; the fn/media-key caption from the F-key revision is
  gone with the feature.

New, per maintainer request:
- DictationEventDecoder (engine): the CGEvent -> router-event reduction moved
  out of the tap shim so it is testable; DictationEventDecoderTests replays
  REAL CGEvents (CGEvent(mouseEventSource:) for otherMouseDown/Up buttons
  2/3/4/31, left/right clicks, keyboard and flagsChanged events) through it,
  with fixtures structured as data rows so captures from misbehaving hardware
  can be appended. Event construction posts nothing and needs no tap or
  Accessibility grant; if a headless runner ever disagrees, gate with
  .enabled(if:) like MicCaptureLevelsTests - never a real CGEventTap.
- Capture diagnostics: while developer mode is on, every event the capture
  recorder sees - accepted or refused - is appended to
  ~/Library/Logs/Blurt/capture-events.jsonl (DictationLog.appendCaptureEvent,
  sharing the existing gate, queue, encoder, and appendLine) with kind,
  outcome token, button number, keycode, flags, and autorepeat bit, so a
  misbehaving multi-button mouse is diagnosable from what it actually sent.
  The Developer section footer now names the third file.

Docs follow: AGENTS.md (hotkey section, repo map, the KeyboardShortcuts row's
rationale cell), the guardrails skill bullet, and the engine README are
mouse-only; the mechanized invariant anchors remain untouched and
check-invariants.sh --self-test passes.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZB3QrNeP732HSJzx3hxLj
@claude claude Bot changed the title feat: Custom dictation trigger — bind an F-key or extra mouse button feat: Custom dictation trigger — bind an extra mouse button Aug 14, 2026
claude added 3 commits August 14, 2026 22:34
…ments

A public function's default argument values may only reference public
declarations, and neither DeveloperModeStore.init nor defaultCaptureURL has a
reason to be public. Split the entry point: the public wrapper supplies the
real store and destination in its body, and an internal overload keeps the
injectable store/url parameters the gate tests drive.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZB3QrNeP732HSJzx3hxLj
SwiftLint's function_parameter_count flagged makeCaptureEntry (8 undefaulted
parameters; the rule allows 5, and --strict fails on the warning). The five
raw facts of one recorder event (kind, button, keycode, flags, autorepeat)
now travel as one CapturedInput value from the capture sheet down to the
entry builder, which is also the honest shape: they arrive together from one
NSEvent. Outcome and the bound label stay beside it — they're the recorder's
decision, not the event's facts.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZB3QrNeP732HSJzx3hxLj
…option

Two maintainer-directed changes, in one push.

1. Chords (Alex Kroman, PR #144 thread). The Custom… binding now records a
   keyboard chord — modifiers plus one non-modifier key, e.g. ⌃⌥D — alongside
   the mouse buttons already shipped. A chord is packed into the SAME single Int
   slot as everything else: 0x20000 | (modifiers << 8) | keyCode, disjoint from
   the bare modifier keycodes and the mouse namespace, so nothing migrates and
   @AppStorage keeps live-rendering. ChordModifiers is side-agnostic (a chord is
   a shortcut; no macOS shortcut distinguishes left ⌃ from right ⌃) with a
   persisted bit layout and glyphs in Apple's ⌃⌥⇧⌘ order.

   Routing: trigger-down on a non-autorepeat keyDown of the bound key while
   EXACTLY the required modifier set is held (a superset is a different
   shortcut); trigger-up on that key's keyUp OR the moment any required modifier
   is released — the realistic release order for ⌃⌥D is ⌃ first, and waiting for
   D's keyUp would leave a hold recording with nothing coming to end it. The
   other-key combo cancel stays modifier-bindings-only: a chord's own modifiers
   are part of the trigger, so it must not cancel itself, and typing during a
   latched chord dictation is just typing. keyUp is back in the tap's mask;
   autorepeat is dropped in the decoder.

   The recorder refuses a bare key (the listen-only tap can't stop it typing), a
   modifier as the key half, and a short list of system chords (⌘Q, ⌘W, ⌘H, ⌘⇥,
   ⌘Space, ⌃⌘Q) that would fire underneath the dictation; each refusal is an
   engine ChordRefusal case with the sheet's sentence beside it. The sheet shows
   modifiers live as they're held. Because the tap stays .listenOnly, a bound
   chord still reaches an app that owns it — TriggerBinding.passThroughNote says
   so in the Settings footer rather than the app pretending otherwise.

2. fn is no longer a modifier option. It is the one modifier macOS itself claims
   (dictation, emoji picker, the F-key row). TriggerKey is right ⌘ / right ⌥
   only, its secondary-fn mask bit is gone (chords read the generic masks
   instead), and a persisted fn (63) migrates to right ⌥ — the other right-side
   modifier — rather than the generic right-⌘ fallback, pinned by a test.

Also answering "is Mouse 3 always the middle click": yes. The stored number is
the raw 0-based CGEvent/NSEvent buttonNumber (0 left, 1 right, 2 wheel/middle,
3+ side) and the label is 1-based as mice are numbered for users, so stored 2
displays as "Mouse 3" and IS the wheel click. That off-by-one is now stated in
the doc comments, pinned by a label test, and — since browsers and terminals act
on the wheel click and the tap swallows nothing — carries its own
passThroughNote caution instead of being hard-blocked.

Tests: chord encode/decode round-trips for every modifier set, namespace
disjointness, invalid-packing fallbacks, capture policy and reserved chords,
glyph labels, the fn migration, pass-through notes; a chord router suite
(hold/tap, exact-modifier matching, partial-modifier release, autorepeat dedup,
no-combo rule, latched survival, recovery, rebind); and chord decode fixtures
replaying real CGEvents (flag combinations incl. device side bits, Caps Lock and
fn excluded, autorepeat dropped, partial release).

Docs: AGENTS.md's hotkey section and the KeyboardShortcuts settled-decision row
now record that chords are supported and why the package ban is untouched, with
the pinned invariant anchor updated alongside the row and the guardrails bullet
(--self-test passes).

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZB3QrNeP732HSJzx3hxLj
@claude claude Bot changed the title feat: Custom dictation trigger — bind an extra mouse button feat: Custom dictation trigger — bind a keyboard chord or an extra mouse button Aug 14, 2026
The partial-modifier-release fixture passed an EMPTY trigger flag and then
asserted triggerFlagIsOn == false. CGEventFlags.contains([]) is vacuously true
— every set contains the empty set — so the decoder correctly reported true and
the fixture was wrong; its modifier extraction ([.option]) was right all along.
This was the single failing test of 637 on CI.

Pass the real right-command bit so the field actually asserts something, and
pin the trap itself in a second test plus the decoder's doc comment: under a
chord or mouse binding the tap passes [] and triggerFlagIsOn is meaningless, so
only handleForModifier may read it. No production behavior changes — the router
already ignored that field for those bindings.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XZB3QrNeP732HSJzx3hxLj
@claude

claude Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Closing this unmerged at the maintainer's request — the work grew beyond the intended scope.


Generated by Claude Code

@claude claude Bot closed this Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant