Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# Changelog

## 0.14.0 · Set up in a minute (on `main`, ships with 1.0)

- Add a first-run assistant. A new installation opens on four short screens: where to transcribe (Cloud, recommended, or on this device), then the OpenAI key — with **Create key ↗** to OpenAI's key page — or the local model (Base recommended), then the language as a code such as `de`, suggested from the system language, and optional words, and last the ChatGPT subscription for text actions (experimental, only with a Codex login), the dictation key and start at login.
- **Skip setup** on every screen closes the assistant for good in one click and applies nothing of it. Every screen says that all of it can be changed later in Settings.
- Nothing happens until **Finish**: only then is the key stored, the model downloaded, the dictation key registered and start at login written, in that order. A step that fails returns to the screen where it is chosen, with the reason; what already worked is kept, and setup is marked done only after the last step.
- Anyone already using Utterform never sees the assistant: a settings file from before it counts as set up.
- While the assistant is open, the dictation key, `utterform --toggle` and a second launch bring it to the front instead of recording, and nothing is held back to record afterwards.

See [0.14.0 release notes](docs/releases/v0.14.0.md).

## 0.13.0 · Voice messages and recordings (on `main`, ships with 1.0)

- Transcribe an audio file: the new folder button next to Settings or `O` opens one, dropping it on the window shows **Drop to transcribe** with the action and engine it will use, `utterform --file <path>` and **Open with → Utterform** in the file manager hand one over. Several files are done one after another.
Expand Down
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,16 @@ Required runtime packages:
sudo pacman -S --needed webkit2gtk-4.1 gtk3 alsa-lib libayatana-appindicator
```

## Set up in a minute

The first start asks four short questions: cloud or on this device, then the OpenAI key
(**Create key ↗** opens OpenAI's key page) or the local model, then your language as a code
such as `de` and any names to spell right, and last whether text actions may use your
ChatGPT subscription (experimental), plus the dictation key and start at login.
**Skip setup** ends it at once and changes nothing. Nothing is stored, downloaded or
registered before **Finish**, and everything can be changed later in Settings. An upgrade
from an earlier version does not show it.

## Tune it once, then stay in flow

Settings has a search field on top and five categories below it. Each setting is one
Expand Down
41 changes: 41 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,47 @@ Ghostty); macOS uses the localized name.
`recording.app_action` (action) when a rule takes over. The application is never logged:
which program someone dictates into says as much as a window title.

## First-run assistant (0.14.0)

**Who sees it.** `settings.onboarding_completed`. `AppSettings::default()` — what `load`
returns when there is no settings file — has `false`; a stored file without the field
deserializes it as `true` (`#[serde(default = "already_set_up")]`), so an upgrade never
shows the assistant. An unreadable settings file stays an error, not a new user. The
interface's `DEFAULT_SETTINGS` say `true`: the backend alone decides who is new.

**One decision for intents.** `setup` loads the settings once and sets the managed
`Onboarding` flag; `save_settings` and `complete_onboarding` keep it current.
`cli::Intent::route(setting_up)` decides for the launch intent, the second instance and the
global hotkey (`deliver_intent`): while setting up, every intent — including `--toggle`,
the hotkey and a stale autostart entry — reveals the window and none is forwarded or kept
as the startup intent, so nothing records afterwards out of nowhere. Otherwise the earlier
matrix holds (only `Show` reveals; `Autostart` reaches nothing). The interface also ignores
`remote-intent`, its keyboard shortcuts and audio files while the assistant is open, and
`main` is `inert` beneath it.

**Draft, then Finish.** `lib/OnboardingAssistant.svelte` keeps an `OnboardingDraft`
(`lib/onboarding.ts`): mode, key text, local model, one language code, vocabulary, ChatGPT,
dictation key, autostart. Moving between the four steps calls nothing that changes state.
`applyDraft` maps it onto the loaded settings, touching only those fields; a cloud model
chosen earlier survives a local choice. Finish then runs, in order: `save_settings` with
`onboarding_completed: false` (a crash resumes setup), `set_openai_api_key` if one was
typed for the cloud, `download_local_model` if the chosen model is missing (progress from
the window's existing listener), `apply_global_hotkey` if the key changed,
enable/disable autostart if the switch changed (read back, believed over the request), and
`complete_onboarding`. A failing stage returns to its step with the message; what already
succeeded is remembered and not repeated, and setup stays incomplete. **Skip** calls only
`complete_onboarding(loaded settings, skipped: true)`.

**Links.** `open_openai_api_keys` opens the compiled-in `https://platform.openai.com/api-keys`
and takes no argument, like `open_codex_setup`; no opener permission reaches the WebView.

**Diagnostics.** `onboarding.shown` (source `first_launch` or `startup_intent`),
`onboarding.intent_consumed` (source, intent), `onboarding.skipped`,
`onboarding.completed` (engine, cloud model, local model id, text provider, language and
vocabulary counts, hotkey on/off), `onboarding.finish_failed` (stage `settings`) and
`onboarding.key_page_open_failed`. The other stages fail inside their existing commands,
which log their own events. Never the key, the language, the words or a typed draft.

## What the local engine is compiled for

whisper.cpp is built from source by `whisper-rs-sys`, and ggml's CMake targets the machine doing the compiling unless told otherwise. For a release built on someone else's server that is a trap rather than an optimization: the 0.7.2 Linux binary carried the runner's AVX-512 and AMX instructions and was stopped with SIGILL on an Intel N300 while loading a model — inside `ggml_backend_cpu_device_get_extra_buffers_type`, plain C++ frame code, which is before ggml's own runtime dispatch can choose a kernel and therefore beyond its help. The bug had been there since the first release; only the pairing of build machine and user machine decided whether it fired.
Expand Down
16 changes: 15 additions & 1 deletion docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,21 @@ proposals are in [plans/v1.0-roadmap.md](plans/v1.0-roadmap.md). The plan of 202
(reliability, assistant, finish) is superseded; 0.7.13 shipped, 0.7.14
(`feat/simple-settings`) was rejected and will not be merged.

## Current handoff — 0.12.0 Action per application
## Current handoff — 0.14.0 First-run assistant

Stage 0.14.0 of [the roadmap](plans/v1.0-roadmap.md), branch
`work/utterform-0.14.0-assistant`. Jonas accepted the mock-up on 2026-11-08: four steps
(where to transcribe · key or model · language and words · ChatGPT, dictation key, start
at login), Skip on every step, nothing applied before Finish. Built on the contract of
Draft PR #39, adapted: no Live choice and no recording topic in the assistant (both stay in
Settings), and issue #40 stays with 1.0. `settings.onboarding_completed` (legacy files read
as done), `cli::Intent::route` holds every intent back while setting up,
`lib/OnboardingAssistant.svelte` with `lib/onboarding.ts`. Design in
[ARCHITECTURE.md → First-run assistant](ARCHITECTURE.md#first-run-assistant-0140),
validation and what is open for 1.0 in [TESTING.md](TESTING.md). Merged into `main` without
tag or release; ships with 1.0. Next stage: 1.0.0.

## Previous handoff — 0.12.0 Action per application

Stage 0.12.0 of [the roadmap](plans/v1.0-roadmap.md), branch `work/utterform-0.12.0-per-app`.
Jonas decided to build it small and accepted the mock-up: **Settings → Context → Per app**,
Expand Down
23 changes: 23 additions & 0 deletions docs/TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,29 @@
`PLAYWRIGHT_CHROMIUM_EXECUTABLE=/usr/bin/chromium npm run test:e2e`.
- `cmake` is needed for whisper.cpp. Without root, `mise exec cmake@latest -- cargo test` works.

## 0.14.0 First-run assistant

Rust: `settings.rs` — no file means not set up, a file without the field means set up,
explicit `false`/`true` survive a save and reload; `cli.rs` — while setting up every intent
reveals and none is forwarded, afterwards the dictation key leaves the window alone.
`lib/onboarding.test.ts`: system language, language names and typos, the starting draft,
`applyDraft` writing only its fields and keeping the cloud model, what holds Next back.
`App.test.ts`: never shown to someone set up; Skip from step 3 saves only completion and
calls no key, model, hotkey or autostart command; cloud needs a key, opens the key page,
stores the key only on Finish and before completion; a failed download returns to the
model step, stays incomplete and succeeds on retry; ChatGPT only with a Codex login; key
and autostart changes applied on Finish only; no hotkey intent or Space records beneath it.
Playwright, at 920×720 and 360×400: all four steps with screenshots, nothing sideways,
Skip and Next/Finish in view on every step, a toggle beneath it ignored, Finish writes the
local choice and does not store a key typed before switching to local; Skip in the light
theme leaves settings and the startup entry untouched.

Not exercised, to check at 1.0: the installed app on a clean profile (first launch,
first launch through `utterform --toggle` and Alt+Space, second launch while the assistant
is open, launch after Skip), a real model download and keyring write from the assistant,
the key page opening in the browser, and an upgrade from real 0.8.0 settings on Linux,
Windows and macOS.

## 0.12.0 Action per application

Unit tests in `foreground.rs`: display names from classes and program names, Utterform's
Expand Down
5 changes: 3 additions & 2 deletions docs/plans/v1.0-roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ and without its subscription:
| 2026-11-07 | 0.13.0: audio files open from a folder button next to Settings (and `O`) or by dropping on the window, which shows "Drop to transcribe" with action and engine; text to history and clipboard, never the cursor. Mock-up accepted. |
| 2026-11-07 | Settings mock-up accepted: search field on top, categories as tabs below, one row per setting. The "Context" category appears only once 0.11.0 gives it rows. |
| 2026-11-08 | 0.12.0 is built, small: Settings → Context → "Per app", empty by default; "+ App" offers the apps dictated into lately or a typed name; a rule applies only to the dictation key, everywhere else the window's action; a notification "Utterform · Email — for Thunderbird" only when a rule changes the action. Mock-up accepted. |
| 2026-11-08 | 0.14.0: four steps — where to transcribe · OpenAI key or local model · language code (system language suggested) and optional words · ChatGPT (only with Codex login), dictation key, start at login. Skip on every step, nothing applied before Finish. Mock-up accepted. |
| 2026-11-07 | 0.10.0: ChatGPT is marked "Experimental" in Settings; **no fallback** to the OpenAI key — a ChatGPT failure is shown and the plain transcript delivered. Mock-up (AI → Provider: OpenAI key \| ChatGPT) accepted. |
| 2026-11-07 | 0.11.0: context is its **own action "Context"**, no per-action switch; other actions never read context. **Linux first**: Linux reads the selection, Windows/macOS use the clipboard only — no simulated Ctrl+C, which would interrupt a terminal. |

Expand All @@ -74,7 +75,7 @@ Proposals not yet confirmed by Jonas are marked **Proposal**.
| 0.11.0 | Context: selected text and clipboard | merged, ships with 1.0 |
| 0.12.0 | Action per application | merged, ships with 1.0 |
| 0.13.0 | Audio files | merged, ships with 1.0 |
| 0.14.0 | First-run assistant | planned (was 0.8.0) |
| 0.14.0 | First-run assistant | merged, ships with 1.0 |
| 1.0.0 | Acceptance on Linux, Windows, macOS; release; WinGet | planned |

**Proposal (order):** the Omarchy backend first because it is small and the CLI already
Expand Down Expand Up @@ -174,7 +175,7 @@ clipboard; text actions apply as usual.
longer files). Check which formats the API accepts (WhatsApp `.opus`/`.ogg`).
- Local: decode to 16 kHz mono for Whisper; check Opus support of the decoder used.

### 0.14.0 — First-run assistant
### 0.14.0 — First-run assistant (decided 2026-11-08, built)

The contract on Draft PR #39 (`docs/plans/v0.8.0-onboarding-assistant.md`) stays the base,
adapted to the new Settings and the Codex choice: skippable, short welcome, OpenAI key or
Expand Down
32 changes: 32 additions & 0 deletions docs/releases/v0.14.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Utterform 0.14.0 — Set up in a minute

*Stage of the road to 1.0, merged into `main`; not released on its own. It ships with 1.0.*

Until now a new user met the main window and had to find Settings to make Utterform work:
the key, the engine, the language were all one click away but nobody said so. Now the first
start asks four short questions.

**Where to transcribe.** Cloud — GPT Transcribe, the most accurate, with an OpenAI key — or
on this device with Local Whisper, offline.

**The key or the model.** For the cloud, paste the OpenAI key; **Create key ↗** opens
OpenAI's key page. It goes into the system keyring. On this device, pick the model; Base is
recommended, and one already downloaded says so.

**The language.** Type a code such as `de`; the system language is already there, and the
field shows its name. Add names and terms Utterform should spell right, if you like.

**Text actions.** Use the ChatGPT subscription for Email, Polish, Context and the others,
marked experimental and offered only when the Codex CLI is signed in. Below it the
dictation key and start at login.

**Skip setup** on every screen ends the assistant for good and changes nothing. Whatever
it asks can be changed later in Settings, and every screen says so.

**Nothing before Finish.** Moving between the screens stores, downloads and registers
nothing. Finish stores the key, downloads the model, sets the dictation key and start at
login, and only then counts setup as done. If something fails — a download, a key another
program holds — the assistant goes back to where it is chosen and says why; Finish again
picks up where it stopped.

**Existing users** never see it: an upgrade keeps everything as it is.
73 changes: 73 additions & 0 deletions src-tauri/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,33 @@ impl Intent {
pub fn reaches_the_interface(self) -> bool {
self != Intent::Autostart
}

/// What to do with this intent, given whether the first-run assistant is
/// still open. Until setup is finished or skipped, every way of starting
/// or reaching Utterform shows the assistant and nothing records: a
/// dictation key pressed before there is a key or a model would only fail,
/// and a request kept for later would start a recording out of nowhere
/// the moment setup ends.
pub fn route(self, setting_up: bool) -> Route {
if setting_up {
return Route {
reveal: true,
forward: false,
};
}
Route {
reveal: self.raises_window(),
forward: self.reaches_the_interface(),
}
}
}

/// Where an intent goes: whether the window comes to the front, and whether
/// the interface is asked to act on it.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Route {
pub reveal: bool,
pub forward: bool,
}

/// Read the intent from a command line, ignoring the executable path and any
Expand Down Expand Up @@ -300,4 +327,50 @@ mod tests {
);
assert_eq!(file_url_path("server/share/x.opus"), None);
}

#[test]
fn while_setting_up_every_intent_shows_the_assistant_and_none_records() {
for intent in [
Intent::Show,
Intent::Toggle,
Intent::Start,
Intent::Stop,
Intent::Cancel,
Intent::Autostart,
] {
assert_eq!(
intent.route(true),
Route {
reveal: true,
forward: false
},
"{intent:?}"
);
}
}

#[test]
fn once_set_up_the_dictation_key_leaves_the_window_where_it_is() {
assert_eq!(
Intent::Toggle.route(false),
Route {
reveal: false,
forward: true
}
);
assert_eq!(
Intent::Show.route(false),
Route {
reveal: true,
forward: true
}
);
assert_eq!(
Intent::Autostart.route(false),
Route {
reveal: false,
forward: false
}
);
}
}
52 changes: 52 additions & 0 deletions src-tauri/src/commands.rs
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,8 @@ pub fn get_settings(app: AppHandle) -> Result<AppSettings, String> {
pub fn save_settings(app: AppHandle, value: AppSettings) -> Result<(), String> {
match settings::save(&app, &value) {
Ok(()) => {
app.state::<crate::Onboarding>()
.set(!value.onboarding_completed);
diagnostics::info!(
"settings.saved",
engine = value.engine.as_str(),
Expand All @@ -129,6 +131,42 @@ pub fn save_settings(app: AppHandle, value: AppSettings) -> Result<(), String> {
}
}

/// End the first-run assistant for good: `value` is written with setup marked
/// done, and dictation intents reach the interface again. A skip hands back
/// the settings as they were loaded, so nothing of the assistant is applied.
/// Only modes and switches are logged, never what was typed.
#[tauri::command]
pub fn complete_onboarding(
app: AppHandle,
mut value: AppSettings,
skipped: bool,
) -> Result<(), String> {
value.onboarding_completed = true;
if let Err(failure) = settings::save(&app, &value) {
return Err(diagnostics::failure!(
"onboarding.finish_failed",
&failure,
stage = "settings"
));
}
app.state::<crate::Onboarding>().set(false);
if skipped {
diagnostics::info!("onboarding.skipped");
} else {
diagnostics::info!(
"onboarding.completed",
engine = value.engine.as_str(),
cloud_model = value.cloud_model.as_str(),
local_model = value.local_model_id.as_deref().unwrap_or("none"),
text_provider = value.text_provider.as_str(),
languages = value.language_hints.len(),
vocabulary = value.vocabulary.len(),
hotkey = value.global_hotkey.is_some(),
);
}
Ok(())
}

#[tauri::command]
pub fn list_input_devices() -> Result<Vec<AudioDeviceInfo>, String> {
audio::list_input_devices()
Expand Down Expand Up @@ -1094,6 +1132,20 @@ pub fn open_codex_setup() -> Result<(), String> {
})
}

/// Where an OpenAI key is made. A fixed address, like the Codex page above:
/// the first-run assistant asks for this page and can name no other.
pub const OPENAI_API_KEYS_URL: &str = "https://platform.openai.com/api-keys";

#[tauri::command]
pub fn open_openai_api_keys() -> Result<(), String> {
tauri_plugin_opener::open_url(OPENAI_API_KEYS_URL, None::<&str>).map_err(|error| {
diagnostics::failure!(
"onboarding.key_page_open_failed",
&Failure::new("opener", "The OpenAI key page could not be opened").detail(error)
)
})
}

#[tauri::command]
pub fn has_openai_api_key() -> bool {
secrets::has_openai_api_key()
Expand Down
Loading
Loading