v0.7.0: "Hey Vox" trigger, mid-speech command detection, earlier TTS loading - #155
Merged
Merged
Conversation
…nd overlay exclusivity - Add "Hey Vox" trigger keyword alongside "Vox Control" with fuzzy matching and documentation updates across the UI and routing docs. - Implement live interim transcription in the audio coordinator to detect voice commands early during active speech. - Prewarm and preload TTS models concurrently upon early command detection, significantly reducing latency for 'Say' commands. - Skip unnecessary S1-mini post-processing on spoken speech targets to preserve verbatim output and avoid latency. - Hide normal target visualizers while the system response overlay is active so overlays never overlap. - Enforce strict wake-word matching and transcript verification on voice commands to ensure normal dictation is never dropped or misclassified as a command.
- Accept "Hey Vox" (and "Hey, Vox", "Hey-Vox", fuzzy "hey box"/"hay vox")
alongside "VoxCtrl"/"Vox Control"; exact triggers now require word
boundaries so "voxelize" and similar never trigger.
- Only treat "<word> control" as a trigger when it opens the dictation and
<word> sounds like "vox" ("box control", "walks control"). Previously any
"... control" anywhere matched, so "Take control, say hello" was routed to
a Say target.
- Match target names on an offset-preserving lowercase copy so text or
labels with characters like "İ" can no longer panic the delivery thread.
- Add "Hey Vox" to the default custom vocabulary and the Whisper prompt, and
update Settings copy and docs/routing.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rtup - preload_tts() now also works in always-loaded mode, where a model without prewarm is loaded lazily; recording against a speech target (or an early spoken command) starts that first load while the user is still talking. The worker ignores the request when the model is already resident. - Switching from on-demand to always-loaded loads the model immediately when the engine's prewarm setting is on, and logs a load failure. - The per-engine prewarm settings still decide whether the model is loaded at startup (an earlier draft loaded it unconditionally, including Breeze-TTS-2's ~5 GB, and made the prewarm toggles do nothing). - Voice Command Router hotkeys no longer count as speech targets, so on-demand mode doesn't load the model for every ordinary dictation. - Start the TTS worker before AppState is built so tts_handle is set from the start, and let the speak callback use it without a runtime hop when it's uncontended. - Share the "load the selected engine" match between Preload and the mode switch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stack the overlay's parts in a column and hide the recording visualizer while the SYSTEM RESPONDING pill is up, so the two never overlap. Command and MCP pills are shown alongside the visualizer during recording. - The visualizer stays mounted while the whole overlay fades out, so its outro transition still plays (an earlier draft unmounted it the instant recording stopped). It is dropped once recording ends only when the overlay stays up for a command/MCP pill. - Overlay tests restore the config store after themselves, wait out the overlay's 25ms repaint delay explicitly, and fail the mocked get_config so the store keeps its defaults instead of a status-shaped object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ction, and overlay exclusivity" This reverts commit 1f13d62. The commit bundled six independent changes, several of which changed runtime behaviour for users who never say a trigger word (a billed OpenAI call per interim pass, TTS models loaded at every startup, on-demand TTS memory mode defeated). It is re-landed as separate, fixed PRs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It was the one clipboard test without MockClipboardTool, so on a Wayland desktop it ran the real wl-copy: the developer's clipboard was replaced, and wl-copy's forked background server inherited the test binary's stdout, so `cargo test -p voxctrl-routing | tail` hung after every test passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pills in view - While recording, show the target visualizer and hide the SYSTEM RESPONDING pill. Starting to dictate interrupts playback, but the speaking flag can lag behind it, which briefly hid all recording feedback. - Let the visualizer box shrink (trimmed, not scaled) so a tall custom overlay can't push the command/MCP pills out of the 444px window. Built-in visualizers (at most ~152px) plus two pills fit with room left. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The status store polls get_status every second when no status-tick events arrive, which in tests is always. The mock replied with a fixed idle status, so whenever a run was slow enough for a poll to land inside a test's settle window (as on CI), speaking/recording were reset under it and 'hides target visualizer and displays SYSTEM RESPONDING' failed. The mock now returns the status store's current value, so a poll is a no-op. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add "Hey Vox" trigger keyword; tighten trigger matching
Run short interim transcription passes over the opening of a recording so a
spoken command ("Hey Vox, say ...") is recognised mid-speech: the command
overlay appears early and the TTS model starts loading before release.
Compared with the first cut of this feature:
- Interim passes skip the hotkey's OpenAI post-processing, which otherwise
made a billed request every ~400ms while recording.
- Only the first 6s of audio is transcribed early, so passes stay short,
the final transcription never waits long behind one, and long dictations
don't keep the model busy for their whole length.
- Interim passes run only when a command could match (a non-router target
exists) and never for remote or CPU medium/large Whisper backends.
- The early match is a head start only. Routing is decided by the final
transcript alone, which must carry the trigger itself, so a misheard
interim ("hey folks" -> "hey box") can no longer send ordinary dictation
to a command target.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…detection off - When an interim pass announced a command but the final transcript doesn't confirm it (no command, empty result or a transcription error), the command overlay is taken down right away through a new "command-withdrawn" event instead of lingering for its full duration. - New features.early_command_detection setting (default on; older configs read as on), with a Settings → Features → Voice Commands toggle and docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Detect voice commands while the user is still speaking
Speak targets read their text aloud verbatim, so the S1-mini rewrite only adds latency before the first word and can change what the user meant to have spoken. Command resolution now picks the target and raw text first and runs S1-mini once afterwards, instead of repeating the check per branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skip S1-mini cleanup for text bound for Speak targets
Start TTS model loads earlier without loading at every startup
Overlay: never show the target visualizer over SYSTEM RESPONDING
Mock wl-copy in the empty-text clipboard test
Bump version to 0.7.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Release merge for v0.7.0. Adds a "Hey Vox" voice-command trigger, recognises commands while the user is still speaking, and loads TTS models earlier. It's a minor bump because it adds a trigger keyword, a setting and new mid-speech behaviour.
İorÜcan no longer crash the delivery thread.features.early_command_detection, default on.prewarmsetting still decides whether the model loads at startup. Switching to always-loaded with prewarm on loads the model right away.wl-copy: it replaced the developer's clipboard and made pipedcargo testhang.Cargo.toml,Cargo.lock,package.json,package-lock.jsonandtauri.conf.json.Note on 1f13d62
The history includes 1f13d62 and its revert, 29ab1f8. That commit bundled all of the above into one change and had problems the split PRs fix:
Together, the commit and its revert change nothing; the PRs above are what ships.
Test plan
cargo testforvoxctrl-app,voxctrl-config,voxctrl-routing,voxctrl-ttsandvoxctrl-inferenceon the mergeddevelopment: 426 passnpx vitest run: 244/244 passcargo check/cargo test, frontend check) green on every PR above, re-run after each stacked PR was retargeted todevelopmentmoonshine/parakeet/inflect-micro); not run yetTagging
v0.7.0is left to you, as with previous releases.🤖 Generated with Claude Code