diff --git a/README.md b/README.md index fb1a421..9dc4b24 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,10 @@ Private, local text-to-speech, cross-platform. Say It turns copied text into speech with open models running entirely on your machine — your text and generated audio never leave your computer. +

+ Say It Linux desktop app — Speak tab +

+ This is a multi-platform port of [callebtc/sayit](https://github.com/callebtc/sayit) (macOS / Apple silicon), keeping its architecture and CLI surface while swapping every Apple-specific layer for portable equivalents: @@ -36,32 +40,13 @@ The Tauri v2 shell is cross-platform by design; only the playback helper Porting notes for macOS/Windows contributors are welcome — see [LINUX.md](LINUX.md) for how the port is put together and why. -## Architecture - -``` -┌──────────────┐ REST + SSE, Bearer token ┌──────────────────┐ -│ Tauri v2 app │ ◄──────────────────────────► │ sidecar (Node) │ -│ SvelteKit UI │ │ kokoro-js engine │ -│ sayit CLI │ ◄──────────────────────────► │ mpv playback │ -│ sayit-clipboard │ history, models │ -└──────────────┘ └──────────────────┘ -``` - -- **sidecar/** — per-user service: synthesis (Kokoro ONNX via kokoro-js), - playback via mpv's JSON IPC (pause / seek / speed / volume), history, model catalog, - settings. One model in memory, unloaded after 10 idle minutes (configurable). -- **app/** — SvelteKit 2 + Svelte 5 UI: speak box, transport, history, voices, - Settings marketplace for models, onboarding when none are installed. -- **cli/sayit.js** — `sayit "text"`, `printf … | sayit`, `sayit status`, - `pause`, `resume`, `stop`, `seek`, `speed`, `volume`, `voices`, `models`, `history`, `replay`. -- **src-tauri/** — tray icon, global hotkey (Ctrl+Alt+V speaks clipboard), - spawns the sidecar, hands the API token to the webview. - -## Quick install (non-developer) +## Quick start Requirements: Node ≥ 20, npm, and **mpv** for playback (falls back to `aplay`). Clipboard tools (`wl-paste` / `xclip` / `xsel`) only if you want the hotkey. +### 1. Install + ```sh curl -fsSL https://raw.githubusercontent.com/ildella/sayit/master/scripts/install.sh | bash -s -- --systemd ``` @@ -69,30 +54,62 @@ curl -fsSL https://raw.githubusercontent.com/ildella/sayit/master/scripts/instal Or from a clone of this repo: ```sh -bash scripts/install.sh # service + sayit CLI +bash scripts/install.sh # sidecar + sayit CLI bash scripts/install.sh --systemd # + start automatically at login ``` -Installs the sidecar to `~/.local/share/sayit/sidecar` and the `sayit` -command to `~/.local/bin/sayit`, then starts the daemon. Download a catalog -model from the app (or `sayit models install kokoro-q8 --use`) before -speaking; after that the app stays offline. +That puts the sidecar in `~/.local/share/sayit/sidecar` and the **CLI** at +`~/.local/bin/sayit`, then starts the daemon. Ensure `~/.local/bin` is on +your `PATH`. Download a catalog model before speaking; after that the app +stays offline. + +After pulling updates, re-run `scripts/setup-sidecar.sh` (or `npm run setup`) +and restart the service. The GUI and CLI talk to whatever is already on +port 7878 — an old sidecar will look “stuck” or reject speak. + +### 2. Run the CLI ```sh sayit models install kokoro-q8 --use # once, ~90 MB sayit "Hello from Say It" # speak -sayit status # player + engine status sayit volume 0 # silence; 1 = normal, 2 = boost sayit service status # is the daemon running? ``` -After pulling updates, re-run `scripts/setup-sidecar.sh` (or `npm run setup`) -and restart the service. The GUI and CLI talk to whatever is already on -port 7878 — an old sidecar will look “stuck” or reject speak. +`sayit` talks to the sidecar on `127.0.0.1:7878`. If the daemon is down: +`sayit service start` (or `systemctl --user start sayit` if you used +`--systemd`). -## Desktop app (GUI) +### 3. Install the agent skill -From a clone of this repo, with the sidecar already installed (`npm run setup` or `scripts/install.sh`): +The [skill](skills/sayit/SKILL.md) is from [callebtc/sayit](https://github.com/callebtc/sayit); +this port only installs it next to the Linux CLI. After `install.sh` (or +`npm run setup`): + +```sh +sayit skill install +``` + +That copies `SKILL.md` to `~/.agents/skills/sayit/` (OpenCode and other +agents that read that directory). Then tell the agent: + +```text +Load the Say It skill and use it for live spoken updates. +``` + +**Claude Code** (if you use it instead): + +```sh +mkdir -p ~/.claude/skills/sayit +cp "$(sayit skill path)" ~/.claude/skills/sayit/SKILL.md +``` + +Re-run `sayit skill install` after upgrading Say It. The sidecar must be +running (`sayit service start`) and a model installed before speech works. + +### 4. Run the desktop app + +From a clone, after the sidecar is installed (`install.sh` or `npm run setup`): ```sh npm install # once: @tauri-apps/cli @@ -100,9 +117,8 @@ npm --prefix app install # once: SvelteKit UI npm run dev # sidecar + Tauri window ``` -`npm run dev` starts the sidecar and the tray/window. If the daemon is already up (`sayit service start` or systemd), Tauri connects to it instead of spawning a second one. - -Equivalent: `npm run tauri dev` (same window; sidecar auto-spawn if port 7878 is free). +`npm run dev` (or `npm run tauri dev`) opens the tray/window. If the daemon +is already up, Tauri connects to it instead of spawning a second one. ```sh npm run tauri build # .deb / AppImage (Linux shell only) @@ -114,6 +130,61 @@ lists `~/.local/bin` first, the GNOME icon or `sayit status` may run the CLI instead of the window. Launch the GUI with `/usr/bin/sayit`, the CLI with `~/.local/bin/sayit`. +### 5. Which engine is running? + +There is one synthesis engine today: **Kokoro-82M** via kokoro-js / +onnxruntime-node (CPU). What *does* vary is the **catalog model** (q8 vs q4) +and whether it is loaded in memory. + +```sh +sayit status +``` + +Example: + +``` +state: idle +engine: loaded (kokoro-q8) +sidecar: 0.x.x +``` + +- **engine** — `loaded` / `loading…` / `unloaded`, and the active model id + (`kokoro-q8` or `kokoro-q4`). Unloaded after idle timeout; the next speak + loads it again. +- **sidecar** — installed sidecar version (and a protocol mismatch warning + if the CLI is newer than the daemon). + +List every catalog row and which one is **active**: + +```sh +sayit models +``` + +In the GUI: **Settings → Models**. The row marked in use is the engine +the next speak will load. There is no separate GPU / ROCm / CUDA switch in +this port — onnxruntime-node runs on CPU. + +## Architecture + +``` +┌──────────────┐ REST + SSE, Bearer token ┌──────────────────┐ +│ Tauri v2 app │ ◄──────────────────────────► │ sidecar (Node) │ +│ SvelteKit UI │ │ kokoro-js engine │ +│ sayit CLI │ ◄──────────────────────────► │ mpv playback │ +│ sayit-clipboard │ history, models │ +└──────────────┘ └──────────────────┘ +``` + +- **sidecar/** — per-user service: synthesis (Kokoro ONNX via kokoro-js), + playback via mpv's JSON IPC (pause / seek / speed / volume), history, model catalog, + settings. One model in memory, unloaded after 10 idle minutes (configurable). +- **app/** — SvelteKit 2 + Svelte 5 UI: speak box, transport, history, voices, + Settings marketplace for models, onboarding when none are installed. +- **cli/sayit.js** — `sayit "text"`, `printf … | sayit`, `sayit status`, + `pause`, `resume`, `stop`, `seek`, `speed`, `volume`, `voices`, `models`, `history`, `replay`, `skill path`, `skill install`. +- **src-tauri/** — tray icon, global hotkey (Ctrl+Alt+V speaks clipboard), + spawns the sidecar, hands the API token to the webview. + ## Models and first run Speech models are a **catalog**, not a silent download on first speak. @@ -134,6 +205,15 @@ sayit models rm kokoro-q4 Weights land in `~/.cache/sayit/models`. After that the app stays offline. Adding another ONNX family later is a catalog row, not a new Settings screen. +## Playback speed + +- **Speak tab slider:** 0.5×–2.5× in 0.25 steps (the default-speed setting uses the same range). +- **Player − / + buttons:** ±0.25 per press while playing, clamped to 0.5×–4.0×. +- **CLI:** `sayit speed <0.5-4>` accepts any value in the range, not just the steps. +- Pitch is preserved at every speed (mpv `scaletempo`); at 1× the audio plays + untouched, and history replays apply speed only at playback — the stored + audio file is always the original. + ## Setup (development) Requirements: Node ≥ 20, npm, **mpv** (recommended; falls back to `aplay`), @@ -192,6 +272,7 @@ you can adapt `sayit-clipboard` to use `xclip -o` (PRIMARY) instead. | `~/.cache/sayit/models` | downloaded models | | `~/.cache/sayit/audio` | synthesized WAVs | | `~/.cache/sayit/sidecar.log` | sidecar stdout/stderr (when spawned by the GUI) | +| `~/.cache/sayit/sidecar.pid` | pid of the running sidecar, used by health/recovery | ## Troubleshooting @@ -204,8 +285,15 @@ sayit models install kokoro-q8 --use Speak never fetches weights by itself. **Empty Voice menu, red connection dot, or Speak stuck on Synthesizing** — -the UI is talking to an outdated sidecar (or systemd restarted one after you -killed the process). Stop the unit, refresh the install, start again: +the UI is talking to an outdated sidecar. The GUI handles this on its own: +it verifies sidecar health and protocol at launch and every 30 seconds, and +retires + respawns a stale one (a process it cannot attribute to this install +is never touched). One caveat: if the stale sidecar is the systemd unit itself, +`Restart=on-failure` brings it back while the GUI waits for the port — stop +the unit first (`systemctl --user stop sayit`), let the GUI win, then either +keep the GUI-managed sidecar or refresh the install and start the unit again. +Without the GUI — daemon-only setups — stop the unit, refresh the install, +start again: ```sh systemctl --user stop sayit @@ -224,7 +312,8 @@ Use `~/.local/bin/sayit status`. ## API (v1) -`GET /v1/status` · `POST /v1/speak|pause|resume|stop|seek|speed|volume` · +`GET /v1/status` · `GET /v1/health` (liveness + sidecar version/protocol) · +`POST /v1/speak|pause|resume|stop|seek|speed|volume` · `GET /v1/voices|models|history|settings` · `POST /v1/models/:id/install|select` · `DELETE /v1/models/:id[/install]` · `POST /v1/history/replay` · diff --git a/cli/sayit.js b/cli/sayit.js index 51eef47..f7018e0 100644 --- a/cli/sayit.js +++ b/cli/sayit.js @@ -12,6 +12,8 @@ const settings = (() => { catch { return {}; } })(); const BASE = `http://${settings.host || '127.0.0.1'}:${settings.port || 7878}`; +// Keep in sync with PROTOCOL_VERSION in sidecar/src/config.js. +const EXPECTED_PROTOCOL = 1; async function api(method, pathName, body) { const res = await fetch(BASE + pathName, { @@ -89,6 +91,65 @@ async function isServiceUp() { return true; } +/** Raw /v1/health: { status, health } — never throws for HTTP error codes. */ +async function fetchHealth() { + const res = await fetch(`${BASE}/v1/health`, { + headers: { Authorization: `Bearer ${token}` }, + signal: AbortSignal.timeout(3000), + }); + return { status: res.status, health: res.ok ? await res.json() : null }; +} + +function isFetchTimeout(err) { + return err?.name === 'TimeoutError' || err?.name === 'AbortError'; +} + +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +/** Sidecar install dirs the CLI considers its own — same rule as the GUI. */ +function sidecarDirs() { + const here = path.dirname(fileURLToPath(import.meta.url)); + return [ + process.env.SAYIT_SIDECAR_DIR, + path.join(DATA_DIR, 'sayit', 'sidecar'), + path.resolve(here, '..', 'sidecar'), + ].filter(Boolean); +} + +function pidAlive(pid) { + try { fs.statSync(`/proc/${pid}`); return true; } catch { return false; } +} + +function pidMatchesSidecarDir(pid) { + try { + const cwd = fs.realpathSync(`/proc/${pid}/cwd`); + return sidecarDirs().some((dir) => fs.realpathSync(dir) === cwd); + } catch { return false; } +} + +/** SIGTERM a stale sidecar. Attribution rule (mirrors src-tauri main.rs): a + * token-verified health pid is trusted outright; pidfile and port-scan pids + * are fallbacks, used only while alive and only when their cwd is one of our + * sidecar dirs — never touch a process we cannot attribute. */ +async function retireStaleSidecar(healthPid) { + const port = settings.port || 7878; + const candidates = []; + if (healthPid) candidates.push({ pid: healthPid, trusted: true }); + const listener = findListeningPid(port); + if (listener) candidates.push({ pid: listener }); + try { + const pidfilePid = parseInt(fs.readFileSync(path.join(CACHE_DIR, 'sayit', 'sidecar.pid'), 'utf8').trim(), 10); + if (pidfilePid) candidates.push({ pid: pidfilePid }); + } catch { /* no pidfile */ } + for (const { pid, trusted } of candidates) { + if (pid === process.pid || !pidAlive(pid)) continue; + if (!trusted && !pidMatchesSidecarDir(pid)) continue; + try { process.kill(pid, 'SIGTERM'); } catch { /* already gone */ } + for (let i = 0; i < 50 && findListeningPid(port); i++) await sleep(100); + return; + } +} + const USAGE = `sayit — local text-to-speech Usage: @@ -107,6 +168,8 @@ Usage: sayit history Show history sayit replay Replay a history entry sayit rm Delete a history entry + sayit skill path Print the bundled agent skill path + sayit skill install Copy the skill into ~/.agents/skills/sayit sayit service status Is the sidecar daemon running? sayit service start Start the sidecar daemon (detached) sayit service stop Stop the sidecar daemon @@ -132,9 +195,15 @@ try { case 'status': { const s = await api('GET', '/v1/status'); const p = s.player; + let sidecarLine = 'unknown (outdated sidecar?)'; + try { + const h = await api('GET', '/v1/health'); + sidecarLine = h.protocol === EXPECTED_PROTOCOL ? h.version : `${h.version} (protocol mismatch)`; + } catch { /* pre-health sidecar */ } console.log(`state: ${p.playing ? (p.paused ? 'paused' : 'speaking') : 'idle'}`); if (p.playing) console.log(`position: ${p.position.toFixed(1)}s / ${p.duration.toFixed(1)}s @ ${p.speed}x`); console.log(`engine: ${s.engine.loaded ? 'loaded' : s.engine.loading ? 'loading…' : 'unloaded'} (${s.engine.model})`); + console.log(`sidecar: ${sidecarLine}`); break; } @@ -217,48 +286,124 @@ try { console.log('ok'); break; + case 'skill': { + const here = path.dirname(fileURLToPath(import.meta.url)); + const src = [ + path.join(DATA_DIR, 'sayit', 'skills', 'sayit', 'SKILL.md'), + path.resolve(here, '..', 'skills', 'sayit', 'SKILL.md'), + ].find((p) => fs.existsSync(p)); + if (!src) { + console.error('error: skill file not found. Re-run scripts/setup-sidecar.sh'); + process.exitCode = 1; + break; + } + if (args[0] === 'path') { + console.log(src); + } else if (args[0] === 'install') { + const destDir = path.join(os.homedir(), '.agents', 'skills', 'sayit'); + fs.mkdirSync(destDir, { recursive: true }); + const dest = path.join(destDir, 'SKILL.md'); + fs.copyFileSync(src, dest); + console.log(dest); + } else { + console.log('Usage: sayit skill path | install'); + process.exitCode = 1; + } + break; + } + case 'service': { const sub = args[0]; const port = settings.port || 7878; if (sub === 'status') { try { - await isServiceUp(); - const pid = findListeningPid(port); - console.log(`running${pid ? ` (pid ${pid})` : ''}`); + const { status, health } = await fetchHealth(); + if (status === 404) { + console.log('running, but outdated (no /v1/health). Update it: npm run setup, then restart the service'); + process.exitCode = 1; + } else if (status === 401) { + console.log(`something answers on port ${port} but rejects our token — stale or foreign service. Try: sayit service stop && sayit service start`); + process.exitCode = 1; + } else if (!health) { + console.log(`running, but unhealthy — /v1/health answered HTTP ${status}`); + process.exitCode = 1; + } else { + const pid = health.pid ?? findListeningPid(port); + let line = `running (sidecar ${health.version}${pid ? `, pid ${pid}` : ''})`; + if (health.protocol !== EXPECTED_PROTOCOL) { + line += ` — protocol ${health.protocol}, expected ${EXPECTED_PROTOCOL}. Update it: npm run setup, then restart the service`; + process.exitCode = 1; + } + console.log(line); + } } catch (err) { if (err.cause?.code === 'ECONNREFUSED') { console.log('stopped'); process.exitCode = 1; + } else if (isFetchTimeout(err)) { + console.log('running, but unhealthy — /v1/health timed out'); + process.exitCode = 1; } else { throw err; } } } else if (sub === 'start') { + let retire = false; + let retirePid = null; try { - await isServiceUp(); - console.log('already running'); - } catch (err) { - if (err.cause?.code !== 'ECONNREFUSED') throw err; - const dir = findSidecarDir(); - if (!dir) { - console.error('sidecar not found. Install it with: npm run setup (or set $SAYIT_SIDECAR_DIR)'); + const { status, health } = await fetchHealth(); + if (status === 200 && health?.protocol === EXPECTED_PROTOCOL) { + console.log('already running'); + break; + } + if (status === 401) { + console.error(`port ${port} is held by a service that rejects our token — not touching it. Stop it manually, then re-run: sayit service start`); process.exitCode = 1; break; } - fs.mkdirSync(path.dirname(SIDECAR_LOG), { recursive: true }); - const log = fs.openSync(SIDECAR_LOG, 'a'); - const child = spawn(process.execPath, ['src/index.js'], { - cwd: dir, - detached: true, - stdio: ['ignore', log, log], - env: process.env, - }); - child.unref(); - let up = false; - for (let i = 0; i < 20 && !up; i++) { - await new Promise((r) => setTimeout(r, 150)); - try { await isServiceUp(); up = true; } catch { /* not up yet */ } + // Token accepted (200 mismatch, 404, 5xx) or no body: ours, not healthy. + retire = true; + retirePid = health?.pid ?? null; + } catch (err) { + if (err.cause?.code === 'ECONNREFUSED') { + // nothing on the port — spawn below + } else if (isFetchTimeout(err)) { + retire = true; + } else { + throw err; } + } + if (retire) { + console.log('retiring stale sidecar…'); + await retireStaleSidecar(retirePid); + } + const dir = findSidecarDir(); + if (!dir) { + console.error('sidecar not found. Install it with: npm run setup (or set $SAYIT_SIDECAR_DIR)'); + process.exitCode = 1; + break; + } + fs.mkdirSync(path.dirname(SIDECAR_LOG), { recursive: true }); + const log = fs.openSync(SIDECAR_LOG, 'a'); + const child = spawn(process.execPath, ['src/index.js'], { + cwd: dir, + detached: true, + stdio: ['ignore', log, log], + env: process.env, + }); + child.unref(); + let up = false; + for (let i = 0; i < 20 && !up; i++) { + await sleep(150); + try { await isServiceUp(); up = true; } catch { /* not up yet */ } + } + // Honest reporting: what answers on the port must be the instance we + // spawned, not a systemd unit or leftover that won the race. + const listener = findListeningPid(port); + if (up && listener && listener !== child.pid) { + console.error(`warning: port ${port} is held by pid ${listener}, not the instance we started (${child.pid}) — a systemd unit or stale sidecar won the race. See README troubleshooting.`); + process.exitCode = 1; + } else { console.log(`started (pid ${child.pid})${up ? '' : ` — not responding yet; log: ${SIDECAR_LOG}`}`); } } else if (sub === 'stop') { diff --git a/docs/screenshots/desktop-speak.png b/docs/screenshots/desktop-speak.png new file mode 100644 index 0000000..2a9de69 Binary files /dev/null and b/docs/screenshots/desktop-speak.png differ diff --git a/scripts/setup-sidecar.sh b/scripts/setup-sidecar.sh index b87f4bd..8fa8eeb 100755 --- a/scripts/setup-sidecar.sh +++ b/scripts/setup-sidecar.sh @@ -34,9 +34,12 @@ printf '\n' cp "$SRC/cli/sayit.js" "$BIN/sayit" cp "$SRC/scripts/sayit-clipboard.sh" "$BIN/sayit-clipboard" chmod +x "$BIN/sayit" "$BIN/sayit-clipboard" +mkdir -p "$DEST/skills/sayit" +cp "$SRC/skills/sayit/SKILL.md" "$DEST/skills/sayit/SKILL.md" echo "${BOLD}✓ Sidecar${RESET} $DEST/sidecar" echo "${BOLD}✓ CLI${RESET} $BIN/sayit (+ sayit-clipboard)" +echo "${BOLD}✓ Skill${RESET} $DEST/skills/sayit/SKILL.md" echo echo "◇ Optional — systemd user service" echo " mkdir -p ~/.config/systemd/user" diff --git a/sidecar/src/config.js b/sidecar/src/config.js index 07ae83b..52483ef 100644 --- a/sidecar/src/config.js +++ b/sidecar/src/config.js @@ -16,6 +16,34 @@ export const HISTORY_FILE = path.join(DATA_DIR, 'history.json'); export const TOKEN_FILE = path.join(CONFIG_DIR, 'token'); export const SETTINGS_FILE = path.join(CONFIG_DIR, 'settings.json'); export const MPV_SOCKET = path.join(CACHE_DIR, 'mpv.sock'); +export const PID_FILE = path.join(CACHE_DIR, 'sidecar.pid'); + +/** The pidfile names the sidecar that successfully bound the port; write it + * only after listen() has bound, and remove it only when it names *this* + * process — so a crashed second instance can never unlink a healthy one's. */ +export function writeOwnPidfile() { + try { fs.writeFileSync(PID_FILE, `${process.pid}\n`); } catch { /* best effort */ } +} + +export function removeOwnPidfile() { + try { + if (fs.readFileSync(PID_FILE, 'utf8').trim() === String(process.pid)) { + fs.unlinkSync(PID_FILE); + } + } catch { /* missing or unreadable */ } +} + +// Bump when the REST surface changes in a way clients must reconcile with +// (the Tauri shell and the CLI carry their own copy to compare against). +export const PROTOCOL_VERSION = 1; + +export const SIDECAR_VERSION = (() => { + try { + return JSON.parse(fs.readFileSync(new URL('../package.json', import.meta.url), 'utf8')).version; + } catch { + return 'unknown'; + } +})(); for (const dir of [CONFIG_DIR, DATA_DIR, CACHE_DIR, MODELS_DIR, AUDIO_DIR]) { fs.mkdirSync(dir, { recursive: true }); diff --git a/sidecar/src/index.js b/sidecar/src/index.js index 9a4aa24..6ed6f2c 100644 --- a/sidecar/src/index.js +++ b/sidecar/src/index.js @@ -1,10 +1,24 @@ import { createServer } from './server.js'; -import { getToken, getSettings } from './config.js'; +import { getToken, removeOwnPidfile } from './config.js'; // Started by the Tauri app (or standalone via `npm start` / systemd user unit). const token = getToken(); createServer(); +// Clean shutdowns remove our pidfile; kill -9 / OOM leave it stale, which +// recovery code tolerates by checking the pid is alive before trusting it. +for (const signal of ['SIGTERM', 'SIGINT']) { + process.on(signal, () => { + removeOwnPidfile(); + process.exit(0); + }); +} +process.on('uncaughtException', (err) => { + removeOwnPidfile(); + console.error(err); + process.exit(1); +}); + if (process.argv.includes('--print-token')) { console.log(token); } diff --git a/sidecar/src/server.js b/sidecar/src/server.js index dd31cdf..c1b8403 100644 --- a/sidecar/src/server.js +++ b/sidecar/src/server.js @@ -3,7 +3,7 @@ import fs from 'node:fs'; import { synthesize, engineState, VOICES } from './engine.js'; import { player } from './player.js'; import { addHistory, listHistory, getHistory, deleteHistory } from './history.js'; -import { getToken, getSettings, saveSettings } from './config.js'; +import { getToken, getSettings, saveSettings, writeOwnPidfile, removeOwnPidfile, SIDECAR_VERSION, PROTOCOL_VERSION } from './config.js'; import { modelStore } from './store.js'; /** @@ -110,6 +110,17 @@ export function createServer() { try { switch (route) { + // Liveness + version handshake. Clients (Tauri shell, CLI) use this to + // tell a healthy current sidecar from a stale or foreign process that + // merely holds the port. Must not depend on the engine being loadable. + case 'GET /v1/health': + return json(res, 200, { + ok: true, + version: SIDECAR_VERSION, + protocol: PROTOCOL_VERSION, + pid: process.pid, + }); + case 'GET /v1/status': return json(res, 200, { player: player.state, @@ -239,7 +250,15 @@ export function createServer() { } }); + // EADDRINUSE and friends: drop the pidfile if it is ours, never a healthy + // instance's. The pidfile itself is written only once the port is bound. + server.on('error', (err) => { + removeOwnPidfile(); + console.error(`sayit sidecar: ${err.message}`); + process.exit(1); + }); server.listen(settings.port, settings.host, () => { + writeOwnPidfile(); console.log(`sayit sidecar listening on http://${settings.host}:${settings.port}`); }); return server; diff --git a/skills/sayit/SKILL.md b/skills/sayit/SKILL.md new file mode 100644 index 0000000..b29ff0a --- /dev/null +++ b/skills/sayit/SKILL.md @@ -0,0 +1,243 @@ +--- +name: sayit +description: Live, low-latency spoken narration for hands-free agent sessions through the sayit TTS command. Trigger immediately for commands such as “let’s use sayit,” “use sayit,” “talk me through this,” “keep me updated out loud,” “I’m AFK, speak updates,” or “give me live voice updates,” and whenever the user otherwise asks the agent to speak, narrate, or provide voice updates. Once activated, voice mode is sticky and mandatory throughout the active session—including follow-up turns, exploration, experiments, errors, recovery, and final handoffs—until the user explicitly turns it off with language such as “stop using sayit,” “turn off voice,” “no more spoken updates,” or “text only.” Prefer this skill over ad-hoc say or audio commands whenever speech output is involved. +--- + +# sayit — live-agent spoken narration + +Use `sayit` as a live companion to the work, not as a text report reader. The +listener should hear the agent begin, explore, discover, adjust, and finish in +near real time while the underlying task continues without waiting for speech. + +`sayit` owns a shared playback queue. Each invocation appends one utterance, +and queued utterances play in order without talking over one another. + +## Activation is a sticky session mode + +Treat a direct voice command as a mode switch, not a one-turn request. Examples +that activate the mode include: + +- “Let’s use sayit.” +- “Use sayit while you work.” +- “Talk me through this.” +- “Keep me updated out loud.” +- “I’m going AFK; speak your progress.” +- “Switch on voice mode.” +- “Give me live voice updates while you work.” + +After any such activation, spoken narration is mandatory throughout the active +session. Keep using it across user follow-ups, new phases of the work, errors, +retries, topic refinements, and completed subtasks. A short user message, a new +question, a topic change, a written final response, or a period of silence does +not deactivate voice mode. Do not make the user repeatedly say “use sayit.” + +Only an explicit opt-out turns it off. Examples include: + +- “Stop using sayit.” +- “Turn off voice.” +- “No more spoken updates.” +- “Text only from now on.” +- “You can stop talking now.” + +When the user turns voice mode off, acknowledge that once in text and stop +launching new utterances. Do not infer deactivation from brevity, interruption, +task completion, or a change of subject. + +Voice mode belongs to the current conversation. A separate thread, fork, or +side conversation is a new session and does not inherit voice mode unless the +user activates it there. + +## Absolute rule: speech must return instantly + +Never run `sayit` in the foreground. Never wait for playback. Never make the +task depend on TTS completion. + +Every utterance must be launched disowned in the background with its output +detached from the tool harness: + +```zsh +sayit "I’m checking the current branch and its recent commits." >/dev/null 2>&1 &! +``` + +The complete `>/dev/null 2>&1 &!` suffix is mandatory. `&!` disowns the job; +redirecting both output streams prevents an inherited pipe from keeping the +tool call open. A bare `sayit "..."`, ordinary `&`, or `2>&1 &!` without +detaching output can still block or lose playback in some harnesses. + +For the first utterance, check `command -v sayit` once. A background shell can +report exit code 0 even when dispatch failed, so treat any launch-time shell +warning or error as a failed utterance. Do not add a blocking delivery check to +normal speech. If the user reports that nothing played, troubleshoot then: +check `sayit service status`, start the service if it is stopped, and retry one +short audible confirmation before resuming narration. + +> **Coding-agent sandbox note:** Agents such as Codex may run shell commands in +> a restricted sandbox even though ordinary coding agents do not. In a +> sandboxed zsh, backgrounding may emit `nice(5) failed: operation not +> permitted`, and audio access itself may require narrow approval. Keep the +> normal command above as the default. Only when this sandbox-specific failure +> occurs, disable zsh background-job priority adjustment and retry with the +> environment's narrow audio permission: +> +> ```zsh +> unsetopt BG_NICE 2>/dev/null || true +> sayit "I’ve restored spoken updates." >/dev/null 2>&1 &! +> ``` + +After launching an utterance, immediately continue the user’s task. Do not +poll the speech engine, sleep, wait, or check completion unless the user says +they cannot hear anything and troubleshooting is now the task. + +Use a separate, tiny shell call for speech when that gives control back sooner. +If speech precedes a shell investigation in the same call, keep it as the first +line, fully detached, and start the real command immediately on the next line. + +When the sandbox note applies and audio needs approval, request it once for the +narrow `sayit` command. After approval, retain the instant-return pattern for +every update. + +## The live narration loop + +Once this skill triggers, keep voice active until the user explicitly disables +it. Repeat this small loop while working: + +1. **Orient.** Speak one short sentence at the start: what you understood and + what you are checking first. +2. **Explore.** Continue working immediately. Before a meaningful investigation + or experiment, briefly say what question it should answer. +3. **Report the event.** As soon as a useful result lands, speak the finding and + why it matters. Do not silently collect several findings for a later batch. +4. **Adapt.** If the result changes the plan, say what changed and where you are + going next. If something fails, acknowledge it promptly and narrate the + recovery. +5. **Finish.** Enqueue a complete spoken handoff, then return the written final + response immediately without waiting for the queue. + +This is event-driven narration. Speak when the listener’s mental model should +change: a step begins, evidence lands, a hypothesis is confirmed or rejected, +a decision is made, an error occurs, the plan pivots, or the work completes. +Do not narrate trivial commands, every file opened, or repetitive checks. + +## Cadence and latency + +The audio should feel attached to the work in progress, not delayed until the +end. + +- Speak the opening before or alongside the first substantive tool action. +- During active exploration, aim for a useful update every 15–30 seconds. Do + not let more than about 45 seconds of active work pass in silence. If a tool + may run longer, say what is running and what result you are waiting for + before starting it. +- Speak a discovery immediately after the result that supports it. Do not read + five files, run three commands, and only then narrate the combined result. +- Keep ordinary updates to one breath: usually 8–30 words or one to two short + sentences. +- Do not flood the queue. If narration has fallen behind the work, drop stale + low-value updates and enqueue one current-state summary. Real-time relevance + matters more than exhaustive play-by-play. +- If the user interrupts or redirects the task, acknowledge the pivot aloud + promptly and stop narrating the superseded work. + +## What to say + +Useful live updates include: + +- “I found the branch’s core idea: consensus happens before any member exposes + a signature share. I’m tracing the wallet side next.” +- “That test failed because the fixture is stale, not because the new path is + broken. I’m regenerating the fixture and rerunning the focused case.” +- “The first approach adds a second state owner, so I’m discarding it. I’m + checking whether the existing journal can own the transition instead.” +- “The focused checks pass. I’m doing the standalone compatibility check now.” + +Speak concise conclusions and decision-relevant rationale. Do not expose hidden +chain-of-thought, sensitive data, secrets, personal information, raw logs, or +large code fragments. “This failed because the server rejected the stale +token” is useful; a private internal monologue is not. + +## Final spoken handoff + +The final narration is different from a progress update: it must be a complete, +self-contained summary for a listener who may have missed earlier audio. + +Lead with “I’m done” or the actual terminal state, then include all important +information: + +- the outcome; +- the central findings or design; +- material changes made; +- verification performed and its result; +- unresolved risks, blockers, or tests not run; and +- the most useful next step, when one exists. + +Aim for roughly 120–250 spoken words. If the handoff needs more, split it into +two or three clearly ordered topical utterances. Enqueue them back-to-back and +return the written final response immediately. Never wait for the summary to +finish playing. A final handoff closes the current task, not voice mode; narrate +the next user follow-up unless they explicitly opt out. + +## Write for the ear + +Speech is not Markdown read aloud. + +- Lead with the headline, then the detail. The listener cannot skim. +- Use ordinary sentences, not bullets, tables, headings, file paths, or code. +- Expand or space acronyms that TTS may mangle: “H T L C,” “V T X O,” “D K G,” + or “B I P 47.” +- Translate identifiers into words. Say “the verify keys command,” not a long + snake-case function name. +- Speak numbers and units naturally: “three of five members,” “one thousand + sats,” or “commit c five two d f.” +- Use signposts for structure: “There are three findings. First…” + +## Keep the written channel + +Voice is the live experience; text is the durable record. Continue sending +normal concise commentary and a written final response. Put exact commands, +code, links, file paths, tables, and anything the user may need to copy in text. +Mirror critical conclusions in both channels, but adapt each for its medium +instead of reading the written message verbatim. + +## Conversational voice mode + +Treat spoken sessions as dialogue. Answer the user aloud promptly. Ask at most +one blocking question at a time and speak it as well as writing it. Interpret +dictated typos generously. When a correction contradicts the current course, +confirm the new direction aloud before pivoting. + +## Shell safety + +Wrap the utterance in double quotes. Inside it, avoid double quotes, backticks, +shell variables, command substitutions, and exclamation marks. Rewrite the +sentence in plain language rather than risking shell interpretation. Do not +combine a speech launch with unrelated globs or cleanup commands that could +abort the line before the utterance is queued. + +## Availability and fallback + +Check `command -v sayit` once when needed. If it is unavailable on macOS, use +the built-in `say` command with the same instant-return contract: + +```zsh +say "I’ll keep you updated while I investigate." >/dev/null 2>&1 &! +``` + +If neither command exists, say so in text and continue the task without +pretending audio was delivered. + +## Anti-patterns + +- Silent research followed by one giant spoken report. +- Foreground speech or any wait for playback. +- Treating the TTS process as part of task success. +- Treating a completed task or written final response as automatic voice-mode + deactivation. +- Narrating every command until the queue lags behind reality. +- Reading Markdown, raw diffs, logs, or paths aloud. +- Saying “still working” without naming the current question or new evidence. +- Ending with a vague “done” that omits verification, limitations, or the + actual result. + +The success test is simple: the user can look away, understand where the work +is going from short timely updates, and hear a complete trustworthy summary at +the end—while the agent never pauses its work for speech. diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index 2168af9..4c3686f 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -915,6 +915,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "b1f9e3d69d39e4862ffed03ed071a76f9a13ba1d9109d355b0f0aa6b15e393c4" dependencies = [ "futures-core", + "futures-sink", ] [[package]] @@ -2781,7 +2782,9 @@ dependencies = [ "base64 0.22.1", "bytes", "encoding_rs", + "futures-channel", "futures-core", + "futures-util", "h2", "http", "http-body", @@ -2953,6 +2956,7 @@ name = "sayit" version = "0.1.0" dependencies = [ "dirs", + "libc", "reqwest 0.12.28", "serde", "serde_json", diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index b79cdc2..5c9d608 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -14,9 +14,12 @@ tauri = { version = "2", features = ["tray-icon"] } tauri-plugin-global-shortcut = "2" serde = { version = "1", features = ["derive"] } serde_json = "1" -reqwest = { version = "0.12", features = ["json", "rustls-tls"] } +reqwest = { version = "0.12", features = ["json", "rustls-tls", "blocking"] } dirs = "6" +[target.'cfg(unix)'.dependencies] +libc = "0.2" + [profile.release] lto = true codegen-units = 1 diff --git a/src-tauri/src/main.rs b/src-tauri/src/main.rs index 152a43c..09e78de 100644 --- a/src-tauri/src/main.rs +++ b/src-tauri/src/main.rs @@ -6,12 +6,21 @@ use std::path::PathBuf; use std::process::{Child, Command, Stdio}; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Mutex}; +use std::time::{Duration, Instant}; use tauri::menu::{Menu, MenuItem}; use tauri::tray::TrayIconBuilder; use tauri::{AppHandle, Manager}; use tauri_plugin_global_shortcut::{GlobalShortcutExt, ShortcutState}; +/// Must match `PROTOCOL_VERSION` in sidecar/src/config.js. A sidecar serving +/// a different value is stale (or foreign) and gets retired and respawned. +const PROTOCOL_VERSION: u32 = 1; +const SIDECAR_PORT: u16 = 7878; +const BASE_URL: &str = "http://127.0.0.1:7878"; + fn config_dir() -> PathBuf { dirs::config_dir() .unwrap_or_else(|| PathBuf::from(".")) @@ -57,7 +66,7 @@ fn read_clipboard() -> Option { async fn speak_text(text: String) -> Result<(), String> { let client = reqwest::Client::new(); client - .post("http://127.0.0.1:7878/v1/speak") + .post(format!("{BASE_URL}/v1/speak")) .bearer_auth(read_token()) .json(&serde_json::json!({ "text": text })) .send() @@ -118,14 +127,244 @@ fn spawn_sidecar() -> Option { .ok() } -fn main() { - let mut sidecar: Option = None; +#[derive(serde::Deserialize)] +struct SidecarHealth { + #[serde(default)] + version: String, + protocol: u32, + #[serde(default)] + pid: Option, +} + +/// GET /v1/health. Ok only on 200 with a parseable body; anything else +/// (nothing on the port, a hung server, wrong token, or a pre-health +/// sidecar answering 404) reads as "no healthy current sidecar". +fn sidecar_health() -> Option { + let client = reqwest::blocking::Client::builder() + .timeout(Duration::from_secs(2)) + .build() + .ok()?; + let res = client + .get(format!("{BASE_URL}/v1/health")) + .bearer_auth(read_token()) + .send() + .ok()?; + if !res.status().is_success() { + return None; + } + res.json().ok() +} + +fn port_has_listener() -> bool { + std::net::TcpStream::connect(("127.0.0.1", SIDECAR_PORT)).is_ok() +} + +/// Install dirs the app considers its own, in spawn priority order. +fn sidecar_dirs() -> Vec { + let mut dirs = Vec::new(); + if let Ok(dir) = std::env::var("SAYIT_SIDECAR_DIR") { + dirs.push(PathBuf::from(dir)); + } + dirs.push(data_dir().join("sidecar")); + dirs +} + +#[cfg(target_os = "linux")] +fn pid_is_our_sidecar(pid: u32) -> bool { + if pid == std::process::id() { + return false; + } + let Ok(cwd) = std::fs::canonicalize(format!("/proc/{pid}/cwd")) else { + return false; + }; + sidecar_dirs() + .iter() + .filter_map(|dir| dir.canonicalize().ok()) + .any(|dir| cwd == dir) +} + +#[cfg(target_os = "linux")] +fn pid_alive(pid: u32) -> bool { + std::path::Path::new(&format!("/proc/{pid}")).exists() +} - // Only spawn if no service is already answering. - let already_running = ureq_get_status(); - if !already_running { - sidecar = spawn_sidecar(); +#[cfg(target_os = "linux")] +fn read_pidfile() -> Option { + std::fs::read_to_string(dirs::cache_dir()?.join("sayit").join("sidecar.pid")) + .ok()? + .trim() + .parse() + .ok() +} + +/// Pure-/proc port scan, same approach as cli/sayit.js findListeningPid(). +#[cfg(target_os = "linux")] +fn find_listening_pid(port: u16) -> Option { + let want = format!("{port:04X}"); + let mut inodes = Vec::new(); + for table in ["/proc/net/tcp", "/proc/net/tcp6"] { + let Ok(text) = std::fs::read_to_string(table) else { continue }; + for line in text.lines().skip(1) { + let cols: Vec<&str> = line.split_whitespace().collect(); + if cols.len() < 10 { + continue; + } + if cols[1].split(':').next_back() == Some(want.as_str()) && cols[3] == "0A" { + inodes.push(cols[9].to_string()); + } + } + } + for entry in std::fs::read_dir("/proc").ok()?.flatten() { + let Some(name) = entry.file_name().to_str().map(String::from) else { + continue; + }; + if !name.bytes().all(|b| b.is_ascii_digit()) { + continue; + } + let Ok(fds) = std::fs::read_dir(entry.path().join("fd")) else { + continue; + }; + for fd in fds.flatten() { + let Ok(target) = std::fs::read_link(fd.path()) else { + continue; + }; + if inodes + .iter() + .any(|inode| target == PathBuf::from(format!("socket:[{inode}]"))) + { + return name.parse().ok(); + } + } } + None +} + +/// Terminate a stale sidecar. Attribution rule (mirrored by cli/sayit.js): a +/// pid from a token-verified /v1/health answer is trusted outright; pidfile +/// and port-scan pids are fallbacks, used only while still alive and only +/// after confirming their cwd is one of our sidecar dirs — never touch a +/// process we cannot attribute (upstream PR #21's rule). +#[cfg(target_os = "linux")] +fn retire_stale_sidecar(health_pid: Option) { + if let Some(pid) = health_pid { + if pid != std::process::id() && pid_alive(pid) { + retire_pid(pid); + return; + } + } + // Fallbacks, strongest first: the actual port listener, then the pidfile. + for pid in find_listening_pid(SIDECAR_PORT).into_iter().chain(read_pidfile()) { + if pid != std::process::id() && pid_alive(pid) && pid_is_our_sidecar(pid) { + retire_pid(pid); + return; + } + } +} + +#[cfg(target_os = "linux")] +fn retire_pid(pid: u32) { + eprintln!("sayit: retiring stale sidecar pid {pid}"); + unsafe { libc::kill(pid as i32, libc::SIGTERM) }; + for _ in 0..50 { + if !port_has_listener() { + return; + } + std::thread::sleep(Duration::from_millis(100)); + } + // A systemd unit with Restart=on-failure keeps the port cycling here; + // the README troubleshooting says to stop the unit in that case. + eprintln!("sayit: port {SIDECAR_PORT} still busy 5s after SIGTERM"); +} + +#[cfg(not(target_os = "linux"))] +fn retire_stale_sidecar(_health_pid: Option) {} + +fn wait_until_healthy(timeout: Duration) -> bool { + let deadline = Instant::now() + timeout; + while Instant::now() < deadline { + if let Some(h) = sidecar_health() { + if h.protocol == PROTOCOL_VERSION { + return true; + } + } + std::thread::sleep(Duration::from_millis(300)); + } + false +} + +/// Archive a freshly spawned sidecar, reaping any child handle we still held +/// so recovery after a hang cannot leave a zombie around until exit. +fn store_sidecar(sidecar: &Arc>>, child: Child) { + if let Some(mut old) = sidecar.lock().unwrap().take() { + let _ = old.kill(); + let _ = old.wait(); + } + *sidecar.lock().unwrap() = Some(child); +} + +/// Ensure a healthy current-protocol sidecar answers on the port: retire a +/// stale one, then spawn and poll. Runs once at startup (short wait so the +/// window is never blocked long) and from the watcher (longer wait). +fn recover_sidecar(sidecar: &Arc>>, wait: Duration) -> bool { + let health = sidecar_health(); + match &health { + Some(h) if h.protocol == PROTOCOL_VERSION => return true, + Some(_) => eprintln!( + "sayit: port {SIDECAR_PORT} serves an outdated protocol (sidecar {})", + health.as_ref().map(|h| h.version.as_str()).unwrap_or("?") + ), + None if port_has_listener() => { + eprintln!("sayit: port {SIDECAR_PORT} answers but /v1/health fails") + } + None => {} + } + retire_stale_sidecar(health.and_then(|h| h.pid)); + if !port_has_listener() { + if let Some(child) = spawn_sidecar() { + store_sidecar(sidecar, child); + } + } + wait_until_healthy(wait) +} + +/// Watch sidecar health; on failure recover, at most twice per disconnected +/// period (upstream PR #21's bound). Any healthy check resets the counter. +fn recovery_watcher(sidecar: Arc>>, shutdown: Arc) { + let mut failures = 0u32; + while !shutdown.load(Ordering::SeqCst) { + std::thread::sleep(Duration::from_secs(30)); + if shutdown.load(Ordering::SeqCst) { + return; + } + match sidecar_health() { + Some(h) if h.protocol == PROTOCOL_VERSION => failures = 0, + _ => { + if failures >= 2 { + continue; // gave up for this period; manual fixes reset us via health + } + failures += 1; + eprintln!("sayit: sidecar unhealthy, recovery attempt {failures}/2"); + if recover_sidecar(&sidecar, Duration::from_secs(15)) { + failures = 0; + } + } + } + } +} + +fn main() { + let sidecar = Arc::new(Mutex::new(None::)); + let shutdown = Arc::new(AtomicBool::new(false)); + + // Short startup wait: the watcher retries with a longer budget, so a slow + // or unattributable recovery must not delay the window here. + recover_sidecar(&sidecar, Duration::from_secs(5)); + + std::thread::spawn({ + let sidecar = Arc::clone(&sidecar); + let shutdown = Arc::clone(&shutdown); + move || recovery_watcher(sidecar, shutdown) + }); tauri::Builder::default() .plugin( @@ -169,16 +408,17 @@ fn main() { .invoke_handler(tauri::generate_handler![get_token]) .build(tauri::generate_context!()) .expect("error while building Say It") - .run(move |_app, event| { - if let tauri::RunEvent::Exit = event { - if let Some(mut child) = sidecar.take() { - let _ = child.kill(); + .run({ + let sidecar = Arc::clone(&sidecar); + let shutdown = Arc::clone(&shutdown); + move |_app, event| { + if let tauri::RunEvent::Exit = event { + shutdown.store(true, Ordering::SeqCst); + if let Some(mut child) = sidecar.lock().unwrap().take() { + let _ = child.kill(); + let _ = child.wait(); + } } } }); } - -fn ureq_get_status() -> bool { - // Cheap health check: is something already listening on 7878? - std::net::TcpStream::connect("127.0.0.1:7878").is_ok() -}