Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,7 @@ Both platforms are build-from-source only for now — there's no signed installe

- **On-device ASR** — Whisper (via `whisper.cpp`), running on the GPU. Ships a small English model by default; larger models are auto-preferred if present.
- **Local LLM cleanup** — Qwen3-4B-Instruct (via `llama.cpp`) runs as a separate worker process and cleans the transcript: removes fillers, resolves spoken self-corrections ("meet at 2… no wait, 3" → "3"), applies spoken punctuation, and formats lists/paragraphs. Deterministic gates guard against over-editing, with a raw-transcript fallback.
- **Optional cloud cleanup** — OpenAI (default) / Anthropic, behind one trait. Keys are stored in the OS keychain (macOS Keychain / Windows Credential Manager), **never in a file**.
- **Floating pill UI** — a small always-on-top bar showing idle / recording / processing states.
- **Floating pill UI** — an always-on-top bar that appears only while WhimprFlow is working (recording, cleaning up, the done flash, or an error) and disappears the moment it's idle, so it never sits on your screen at rest.
- **Personal dictionary + auto-learn** — teach it names and terms; on macOS a post-paste Accessibility observer watches for a one-word correction and learns it automatically (conservative filters to avoid junk). *Auto-learn capture is macOS-only so far.*
- **Usage stats** — words dictated, words-per-minute, day streak, time saved, 7-day activity, all stored locally.

Expand Down Expand Up @@ -116,10 +115,11 @@ why it looked like nothing was happening at all). If you still hit this:
- **macOS — "granted but still nothing" after a rebuild.** Every local
`tauri build` produces a differently-signed binary, and macOS can leave a
stale Accessibility entry for the old signature that *looks* granted but
isn't. Fix: in System Settings → Privacy & Security → Accessibility, remove
WhimprFlow with the **−** button and re-add it (or toggle it off/on), then
relaunch. WhimprFlow's pill and Hub will now show "Fn key isn't wired up"
when this happens instead of just doing nothing.
isn't. WhimprFlow now heals this itself: on launch it clears any stale TCC
entry for its bundle id, re-prompts, and opens **System Settings → Privacy &
Security → Accessibility** — just enable WhimprFlow there (and if the Hub
ever shows "Fn key isn't wired up" while the pane says it's on, click its
**Fix Accessibility** button). No relaunch needed either way.
- **Windows — Right Ctrl does nothing.** Another app may be holding a
conflicting global keyboard hook (some anti-cheat/security tools do this);
close it and relaunch WhimprFlow.
Expand Down
5 changes: 4 additions & 1 deletion dev.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
#!/bin/bash
# Run WhimprFlow in development: starts the Vite UI server + the app with hot reload.
# Run WhimprFlow in development: builds the local-LLM worker (tauri dev only
# builds the app crate), then starts the Vite UI server + the app with hot reload.
# The app loads its UI from the dev server, so the pill actually renders.
set -e
cd "$(dirname "$0")"
echo "[dev] building the local-LLM worker…"
cargo build -p whimpr-llm-worker
exec ui/node_modules/.bin/tauri dev "$@"
11 changes: 11 additions & 0 deletions scripts/build-macos.sh
Original file line number Diff line number Diff line change
Expand Up @@ -98,13 +98,24 @@ export APPLE_SIGNING_IDENTITY="$IDENTITY"
BUILD_ARGS=(build)
[ -n "$TARGET" ] && BUILD_ARGS+=(--target "$TARGET")

# The worker is NOT an externalBin (tauri-build demands a triple-suffixed file
# name that breaks dev builds); instead, drop it next to the app executable —
# `worker_bin_path()` checks that location first — sign it, and re-seal the
# bundle so Gatekeeper/notarization stay intact.
echo "==> Building the local-LLM worker…"
cargo build --release -p whimpr-llm-worker

cd "$REPO_ROOT/src-tauri"
"$TAURI" "${BUILD_ARGS[@]}"

TARGET_DIR="$REPO_ROOT/target"
[ -n "$TARGET" ] && TARGET_DIR="$TARGET_DIR/$TARGET"
APP="$TARGET_DIR/release/bundle/macos/WhimprFlow.app"
DMG="$(/usr/bin/find "$TARGET_DIR/release/bundle/dmg" -name "*.dmg" -print -quit 2>/dev/null || true)"
WORKER_DEST="$APP/Contents/MacOS/whimpr-llm-worker"
cp "$REPO_ROOT/target/release/whimpr-llm-worker" "$WORKER_DEST"
codesign --force --sign "$IDENTITY" "$WORKER_DEST"
codesign --force --sign "$IDENTITY" "$APP"

[ -d "$APP" ] || { echo "No WhimprFlow.app was produced." >&2; exit 1; }

Expand Down
11 changes: 3 additions & 8 deletions src-tauri/src/diag.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ const PLATFORM: whimpr_core::diagnostics::Platform = whimpr_core::diagnostics::P
#[cfg(not(any(target_os = "macos", target_os = "windows")))]
const PLATFORM: whimpr_core::diagnostics::Platform = whimpr_core::diagnostics::Platform::MacOs;

const OVERLAY_LABEL: &str = "whimpr_bar";
/// How long the error stays on the pill before it reverts to idle — much
/// longer than the ~500ms "done" flash, since this is the one state the user
/// actually needs time to read.
Expand All @@ -36,11 +35,6 @@ pub struct ErrorDto {
pub detail: String,
}

#[derive(Clone, Serialize)]
struct BarPayload {
state: &'static str,
}

/// Report a failure: log it, push the pill to the `error` state, broadcast
/// the message to every window, and remember it for [`last_error`].
#[allow(dead_code)] // used on macOS/Windows; inert-but-present on other targets
Expand All @@ -49,13 +43,14 @@ pub fn report(app: &AppHandle, failure: InjectionFailure) {
eprintln!("[whimpr] ⚠ {}: {}", diag.headline, diag.detail);
let dto = ErrorDto { headline: diag.headline, detail: diag.detail };
*LAST_ERROR.get_or_init(|| Mutex::new(None)).lock().unwrap() = Some(dto.clone());
let _ = app.emit_to(OVERLAY_LABEL, "whimpr://flowbar/state", BarPayload { state: "error" });
// Shared emitter: also makes the overlay window exist for the error state.
crate::emit_flowbar_state(app, "error");
let _ = app.emit("whimpr://error", dto);

let app2 = app.clone();
std::thread::spawn(move || {
std::thread::sleep(std::time::Duration::from_millis(ERROR_LINGER_MS));
let _ = app2.emit_to(OVERLAY_LABEL, "whimpr://flowbar/state", BarPayload { state: "idle" });
crate::emit_flowbar_state(&app2, "idle");
});
}

Expand Down
73 changes: 54 additions & 19 deletions src-tauri/src/hotkey.rs
Original file line number Diff line number Diff line change
Expand Up @@ -95,13 +95,16 @@ mod imp {
static CLOCK: OnceLock<Instant> = OnceLock::new();
static FN_IS_DOWN: AtomicBool = AtomicBool::new(false);
static TAP_PORT: AtomicPtr<c_void> = AtomicPtr::new(null_mut());
/// True once the global Fn CGEventTap is actually created and running —
/// distinct from `AXIsProcessTrusted`, which can report "granted" for a
/// stale TCC entry that macOS will never honor for this build's signature.
/// Drives the Hub's `hotkey_wired` status and the stale-grant Fix flow.
static TAP_LIVE: AtomicBool = AtomicBool::new(false);
/// Set once at startup if no Whisper model file exists on disk at all —
/// distinct from "still loading", so the finalize path only shows the
/// user a loud "no speech model" error for the real case, not a race
/// against the ~1s background load right after launch.
static ASR_MODEL_MISSING: AtomicBool = AtomicBool::new(false);
/// Bundle id of the app that was frontmost at record-start = the paste target.
/// Cleanup uses it to format for the medium (email vs. text vs. chat).
static TARGET_APP: OnceLock<Mutex<Option<String>>> = OnceLock::new();
static CAPTURE: OnceLock<Mutex<Option<whimpr_audio::CaptureHandle>>> = OnceLock::new();
static ASR: OnceLock<Arc<whimpr_asr::WhisperEngine>> = OnceLock::new();
Expand All @@ -112,11 +115,6 @@ mod imp {
static DICTIONARY: OnceLock<Mutex<whimpr_core::DictionaryStore>> = OnceLock::new();
static STATS: OnceLock<Mutex<whimpr_core::StatsStore>> = OnceLock::new();

#[derive(Clone, Serialize)]
struct BarPayload {
state: &'static str,
}

#[derive(Clone, Serialize)]
struct WavePayload {
bars: Vec<f32>,
Expand Down Expand Up @@ -425,7 +423,9 @@ mod imp {

fn emit_bar(app: &AppHandle, state: &'static str) {
eprintln!("[whimpr] pill -> {state}");
let _ = app.emit_to(OVERLAY_LABEL, "whimpr://flowbar/state", BarPayload { state });
// Shared emitter also toggles the overlay window: visible for every
// state except idle.
crate::emit_flowbar_state(app, state);
}

/// Feed one input into the shared state machine and enact its actions.
Expand Down Expand Up @@ -630,6 +630,19 @@ mod imp {
event
}

/// Whether the global Fn tap is live (see [`TAP_LIVE`]). `get_status`
/// surfaces this to the Hub as `hotkey_wired`.
pub fn tap_live() -> bool {
TAP_LIVE.load(Ordering::SeqCst)
}

/// Called when the Hub's "Fix Accessibility" flow resets the TCC entry: the
/// old tap (if any) is no longer meaningful until the user re-grants and a
/// fresh tap is created.
pub fn mark_tap_stale() {
TAP_LIVE.store(false, Ordering::SeqCst);
}

pub fn install(app: AppHandle) {
let _ = APP.set(app);
let _ = MACHINE.set(Mutex::new(StateMachine::new()));
Expand Down Expand Up @@ -676,16 +689,27 @@ mod imp {

// Accessibility is the ONE permission that makes the Fn CGEventTap global AND
// lets us post the Cmd+V paste into other apps. Without it, a keyboard tap is
// silently limited to frontmost-only — the exact bug. Prompt for it up front.
// silently limited to frontmost-only — the exact bug. Self-heal up front:
// "granted in System Settings but the app doesn't acknowledge it" means a
// stale TCC entry is enforcing an older build's signature, so clear it,
// re-prompt, and open the pane — the tap thread below picks the fresh grant
// up the moment it lands, with no relaunch.
if crate::paste::is_trusted() {
eprintln!("[whimpr] Accessibility granted — Fn works in every app, paste enabled");
} else {
eprintln!(
"[whimpr] ⚠ Accessibility NOT granted — Fn only works while WhimprFlow is \
frontmost and paste is disabled. Prompting; grant WhimprFlow under System \
Settings → Privacy & Security → Accessibility (no relaunch needed)."
"[whimpr] ⚠ Accessibility NOT granted — clearing any stale TCC entry, \
re-prompting, and opening System Settings → Privacy & Security → \
Accessibility (no relaunch needed)."
);
crate::paste::prompt_accessibility();
std::thread::spawn(|| {
// Let the Hub/onboarding window mount first so the user sees it
// before the Settings pane opens over it.
std::thread::sleep(Duration::from_millis(800));
if let Err(e) = crate::reset_and_prompt_accessibility() {
eprintln!("[whimpr] accessibility self-heal failed: {e}");
}
});
}
// Input Monitoring is NOT the gate for a CGEventTap — kept only as diagnostics.
eprintln!(
Expand Down Expand Up @@ -722,6 +746,13 @@ mod imp {
// picked up automatically.
let mut reported = false;
let port = loop {
// Re-check trust inside the retry loop: the Hub's "Fix" button
// resets the TCC entry, and a tap created while untrusted is
// permanently frontmost-only — keep waiting for a fresh grant.
if !crate::paste::is_trusted() {
std::thread::sleep(Duration::from_millis(500));
continue;
}
let port = unsafe {
CGEventTapCreate(
K_CG_SESSION_EVENT_TAP,
Expand All @@ -737,9 +768,8 @@ mod imp {
}
eprintln!(
"[whimpr] Fn tap null despite Accessibility — likely a stale TCC entry from \
an earlier build. Run: tccutil reset Accessibility com.whimpr.whimprflow, \
or toggle WhimprFlow off/on under System Settings → Privacy & Security → \
Accessibility. Retrying…"
an earlier build. Use the Hub's Fix button (or run: tccutil reset \
Accessibility com.whimpr.whimprflow), then re-enable WhimprFlow. Retrying…"
);
if !reported {
if let Some(app) = APP.get() {
Expand All @@ -753,6 +783,7 @@ mod imp {
eprintln!("[whimpr] Fn tap recovered — the key is live now.");
crate::diag::clear_last_error();
}
TAP_LIVE.store(true, Ordering::SeqCst);
TAP_PORT.store(port, Ordering::SeqCst);
unsafe {
let source = CFMachPortCreateRunLoopSource(null(), port, 0);
Expand All @@ -767,14 +798,14 @@ mod imp {
#[cfg(target_os = "macos")]
pub use imp::{
current_settings, dictionary_add, dictionary_entries, dictionary_learn, dictionary_remove,
history, install, rebuild_providers, stats_summary, update_settings,
history, install, mark_tap_stale, rebuild_providers, stats_summary, tap_live, update_settings,
};

// Windows uses the real (but unverified) platform layer in `crate::win`.
#[cfg(target_os = "windows")]
pub use crate::win::{
current_settings, dictionary_add, dictionary_entries, dictionary_learn, dictionary_remove,
history, install, rebuild_providers, stats_summary, update_settings,
history, install, mark_tap_stale, rebuild_providers, stats_summary, tap_live, update_settings,
};

// Other platforms (Linux, etc.): inert stubs so the crate still builds.
Expand All @@ -798,9 +829,13 @@ mod other {
pub fn dictionary_add(_correct: String, _mishears: Vec<String>) {}
pub fn dictionary_remove(_correct: &str) {}
pub fn dictionary_learn(_correct: String, _mishears: Vec<String>) {}
pub fn tap_live() -> bool {
true
}
pub fn mark_tap_stale() {}
}
#[cfg(not(any(target_os = "macos", target_os = "windows")))]
pub use other::{
current_settings, dictionary_add, dictionary_entries, dictionary_learn, dictionary_remove,
history, install, rebuild_providers, stats_summary, update_settings,
history, install, mark_tap_stale, rebuild_providers, stats_summary, tap_live, update_settings,
};
Loading