Skip to content

v0.7.0: "Hey Vox" trigger, mid-speech command detection, earlier TTS loading - #155

Merged
JRufer merged 19 commits into
masterfrom
development
Sep 24, 2026
Merged

JRufer merged 19 commits into
masterfrom
development

Conversation

@JRufer

@JRufer JRufer commented Sep 24, 2026

Copy link
Copy Markdown
Owner

Summary

Release merge for v0.7.0. Adds a "Hey Vox" voice-command trigger, recognises commands while the user is still speaking, and loads TTS models earlier. It's a minor bump because it adds a trigger keyword, a setting and new mid-speech behaviour.

PR Summary Detail
#147 "Hey Vox" trigger keyword Works alongside "VoxCtrl" / "Vox Control", and also accepts near misses like "hey box" and "hay vox". Triggers now need word boundaries. "… control" only triggers at the start of a dictation when the first word sounds like "vox", so "Take control, say hello" is no longer sent to a Say target. Matching target names in text with characters like İ or Ü can no longer crash the delivery thread.
#148 Voice commands recognised mid-speech Early passes transcribe the first 6s of a recording, so a command's overlay appears and its TTS model starts loading before the hotkey is released. The final transcript alone decides routing. If it doesn't confirm an early command, the overlay is withdrawn. Early passes skip the hotkey's OpenAI rewrite, and are skipped entirely for the remote backend and CPU-only medium/large Whisper models. New setting: features.early_command_detection, default on.
#149 No S1-mini rewrite for spoken text Text going to a Speak target is read aloud word for word, and the first word comes sooner.
#150 TTS loads earlier, without loading at every launch Preloading now also works in always-loaded mode. The per-engine prewarm setting still decides whether the model loads at startup. Switching to always-loaded with prewarm on loads the model right away.
#151 Overlay never overlaps The target visualizer is hidden while SYSTEM RESPONDING shows. Recording takes priority over SYSTEM RESPONDING. The visualizer's fade-out still plays, and the pills stay inside the overlay window.
#152 Test fix A clipboard test ran the real wl-copy: it replaced the developer's clipboard and made piped cargo test hang.
#153 Version bump Updates Cargo.toml, Cargo.lock, package.json, package-lock.json and tauri.conf.json.

Note on 1f13d62

The history includes 1f13d62 and its revert, 29ab1f8. That commit bundled all of the above into one change and had problems the split PRs fix:

  • a billed OpenAI request on every interim pass;
  • TTS models loaded at every startup, making the prewarm toggles do nothing;
  • on-demand TTS memory mode effectively disabled;
  • an early-match fallback that could send ordinary dictation to a command target.

Together, the commit and its revert change nothing; the PRs above are what ships.

Test plan

  • cargo test for voxctrl-app, voxctrl-config, voxctrl-routing, voxctrl-tts and voxctrl-inference on the merged development: 426 pass
  • npx vitest run: 244/244 pass
  • CI (Linux + Windows cargo check/cargo test, frontend check) green on every PR above, re-run after each stacked PR was retargeted to development
  • Version reads 0.7.0 across all five version files
  • Release build with default features (moonshine/parakeet/inflect-micro); not run yet
  • On a real desktop:
    • "Hey Vox, say hello" shows the command overlay before the hotkey is released and is spoken verbatim;
    • dictation without a trigger is typed as before;
    • with on-demand TTS, the model does not load for ordinary dictation.

Tagging v0.7.0 is left to you, as with previous releases.

🤖 Generated with Claude Code

JRufer and others added 19 commits September 23, 2026 19:11
…nd overlay exclusivity

- Add "Hey Vox" trigger keyword alongside "Vox Control" with fuzzy matching and documentation updates across the UI and routing docs.
- Implement live interim transcription in the audio coordinator to detect voice commands early during active speech.
- Prewarm and preload TTS models concurrently upon early command detection, significantly reducing latency for 'Say' commands.
- Skip unnecessary S1-mini post-processing on spoken speech targets to preserve verbatim output and avoid latency.
- Hide normal target visualizers while the system response overlay is active so overlays never overlap.
- Enforce strict wake-word matching and transcript verification on voice commands to ensure normal dictation is never dropped or misclassified as a command.
- Accept "Hey Vox" (and "Hey, Vox", "Hey-Vox", fuzzy "hey box"/"hay vox")
  alongside "VoxCtrl"/"Vox Control"; exact triggers now require word
  boundaries so "voxelize" and similar never trigger.
- Only treat "<word> control" as a trigger when it opens the dictation and
  <word> sounds like "vox" ("box control", "walks control"). Previously any
  "... control" anywhere matched, so "Take control, say hello" was routed to
  a Say target.
- Match target names on an offset-preserving lowercase copy so text or
  labels with characters like "İ" can no longer panic the delivery thread.
- Add "Hey Vox" to the default custom vocabulary and the Whisper prompt, and
  update Settings copy and docs/routing.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rtup

- preload_tts() now also works in always-loaded mode, where a model without
  prewarm is loaded lazily; recording against a speech target (or an early
  spoken command) starts that first load while the user is still talking.
  The worker ignores the request when the model is already resident.
- Switching from on-demand to always-loaded loads the model immediately
  when the engine's prewarm setting is on, and logs a load failure.
- The per-engine prewarm settings still decide whether the model is loaded
  at startup (an earlier draft loaded it unconditionally, including
  Breeze-TTS-2's ~5 GB, and made the prewarm toggles do nothing).
- Voice Command Router hotkeys no longer count as speech targets, so
  on-demand mode doesn't load the model for every ordinary dictation.
- Start the TTS worker before AppState is built so tts_handle is set from
  the start, and let the speak callback use it without a runtime hop when
  it's uncontended.
- Share the "load the selected engine" match between Preload and the mode
  switch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stack the overlay's parts in a column and hide the recording visualizer
while the SYSTEM RESPONDING pill is up, so the two never overlap. Command
and MCP pills are shown alongside the visualizer during recording.

- The visualizer stays mounted while the whole overlay fades out, so its
  outro transition still plays (an earlier draft unmounted it the instant
  recording stopped). It is dropped once recording ends only when the
  overlay stays up for a command/MCP pill.
- Overlay tests restore the config store after themselves, wait out the
  overlay's 25ms repaint delay explicitly, and fail the mocked get_config
  so the store keeps its defaults instead of a status-shaped object.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ction, and overlay exclusivity"

This reverts commit 1f13d62.

The commit bundled six independent changes, several of which changed
runtime behaviour for users who never say a trigger word (a billed OpenAI
call per interim pass, TTS models loaded at every startup, on-demand TTS
memory mode defeated). It is re-landed as separate, fixed PRs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It was the one clipboard test without MockClipboardTool, so on a Wayland
desktop it ran the real wl-copy: the developer's clipboard was replaced,
and wl-copy's forked background server inherited the test binary's stdout,
so `cargo test -p voxctrl-routing | tail` hung after every test passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pills in view

- While recording, show the target visualizer and hide the SYSTEM
  RESPONDING pill. Starting to dictate interrupts playback, but the
  speaking flag can lag behind it, which briefly hid all recording feedback.
- Let the visualizer box shrink (trimmed, not scaled) so a tall custom
  overlay can't push the command/MCP pills out of the 444px window.
  Built-in visualizers (at most ~152px) plus two pills fit with room left.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The status store polls get_status every second when no status-tick events
arrive, which in tests is always. The mock replied with a fixed idle
status, so whenever a run was slow enough for a poll to land inside a
test's settle window (as on CI), speaking/recording were reset under it
and 'hides target visualizer and displays SYSTEM RESPONDING' failed. The
mock now returns the status store's current value, so a poll is a no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add "Hey Vox" trigger keyword; tighten trigger matching
Run short interim transcription passes over the opening of a recording so a
spoken command ("Hey Vox, say ...") is recognised mid-speech: the command
overlay appears early and the TTS model starts loading before release.

Compared with the first cut of this feature:
- Interim passes skip the hotkey's OpenAI post-processing, which otherwise
  made a billed request every ~400ms while recording.
- Only the first 6s of audio is transcribed early, so passes stay short,
  the final transcription never waits long behind one, and long dictations
  don't keep the model busy for their whole length.
- Interim passes run only when a command could match (a non-router target
  exists) and never for remote or CPU medium/large Whisper backends.
- The early match is a head start only. Routing is decided by the final
  transcript alone, which must carry the trigger itself, so a misheard
  interim ("hey folks" -> "hey box") can no longer send ordinary dictation
  to a command target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…detection off

- When an interim pass announced a command but the final transcript
  doesn't confirm it (no command, empty result or a transcription error),
  the command overlay is taken down right away through a new
  "command-withdrawn" event instead of lingering for its full duration.
- New features.early_command_detection setting (default on; older configs
  read as on), with a Settings → Features → Voice Commands toggle and docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Detect voice commands while the user is still speaking
Speak targets read their text aloud verbatim, so the S1-mini rewrite only
adds latency before the first word and can change what the user meant to
have spoken. Command resolution now picks the target and raw text first and
runs S1-mini once afterwards, instead of repeating the check per branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skip S1-mini cleanup for text bound for Speak targets
Start TTS model loads earlier without loading at every startup
Overlay: never show the target visualizer over SYSTEM RESPONDING
Mock wl-copy in the empty-text clipboard test
@JRufer
JRufer merged commit 648e576 into master Sep 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant