Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

claude-code-talkback

Local, offline voice output for Claude Code on WSL2. Claude talks back: it speaks a summary of every turn, narrates progress mid-turn while it works, and calls out loud when it's blocked waiting on you — so you can walk away from the screen.

Everything runs locally. Speech is Kokoro-82M neural TTS via kokoro-onnx on CPU, with Windows SAPI as a robotic-but-reliable fallback. The only network call is an optional headless Haiku request (through your existing claude CLI auth) to summarize long turns.

Measured on a mid-range CPU: voice starts ~1–2 s after a short turn ends, ~3 s for long summarized turns. A naive implementation of the same idea sits at 10–15 s — most of this repo is the latency engineering that closes that gap.

How it works

Claude Code hooks (settings.json)
│
├─ Stop ──────────────► hooks/speak-summary.py
│                        │  quiescence gate + flush shortcut (transcript race)
│                        │  short turn → speak verbatim
│                        │  long turn  → speak opener NOW, Haiku summary
│                        │              renders in a background thread and
│                        │              speaks as the tail (no dead silence)
│                        ▼
├─ PostToolUse ───────► hooks/speak-summary.py --mid-turn
│                        (speaks each completed text block while the
│                         turn is still running)
│
└─ Notification ──────► hooks/speak-notify.py
                         ("Claude needs permission" / "waiting for input")

Both speakers call:
tts/kokoro_say.py ── sentence-chunked synthesis: first chunk plays while
 │                   the rest renders; persistent PowerShell player process
 │                   (no per-sentence process spawn)
 └─► tts/kokoro_daemon.py ── keeps the onnx model loaded behind a unix
                             socket; auto-spawned, self-healing, idles out
                             after 30 min. Without it every call pays ~1.5 s
                             of model load.

Why the latency is low:

Technique Saves
Sentence-chunked synthesis (first chunk ≈1 sentence) ~5 s on long text
Model daemon (skip onnx load per call) ~1.5 s every call
Persistent player, spawned before synthesis starts ~1 s + inter-sentence gaps
Transcript flush shortcut (last_assistant_message match) ~3 s gate wait
Opener-overlap on summarized turns ~13 s of silence while Haiku runs

State and safety properties

  • Never re-speaks, never drops speech. Per-transcript state counts spoken blocks; it is written after playback, atomically, so a killed playback re-speaks next turn (re-speak beats lost speech). Mid-turn and Stop share the same state and re-check it under an fcntl lock, so they never double-speak.
  • Playback is serialized by a lock file — except notifications, which deliberately bypass it (a permission alert should not queue behind a 20 s summary).
  • Kill switch: touch ~/.claude/tts-off mutes everything, delete it to re-enable.
  • Recursion guard: the headless summarizer session carries an env var that makes every hook in it exit silently, so the summarizer can never trigger itself.
  • Content never dropped: if Kokoro fails mid-stream (it has a known IndexError on punctuation-sparse ~480-char input — chunking also keeps text out of that zone), the failed chunk is rescued via SAPI; whole-call failures fall back to SAPI for the full text.
  • Everything logs one line per event to ~/.claude/tts.log (self-capping). Look there first when something misbehaves.

Install

Requirements: Windows 10/11 with WSL2 + WSLg, Python 3.10+, and the Claude Code CLI. Tested on Kali and Ubuntu WSL.

  1. Copy the scripts into ~/.claude (the hooks assume this layout):

    git clone https://github.com/writtenonwater99/claude-code-talkback
    cd claude-code-talkback
    mkdir -p ~/.claude/hooks ~/.claude/tts
    cp hooks/*.py ~/.claude/hooks/
    cp tts/*.py ~/.claude/tts/
  2. Set up the TTS venv and model (~330 MB, fully offline afterwards):

    cd ~/.claude/tts
    python3 -m venv venv
    venv/bin/pip install kokoro-onnx soundfile
    wget https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/kokoro-v1.0.onnx
    wget https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/voices-v1.0.bin
  3. Wire the hooks: merge examples/settings-hooks.json into the "hooks" section of ~/.claude/settings.json.

  4. Restart your Claude Code session. Hook config is snapshotted at session start — edits to settings.json do nothing to a running session.

  5. Optional, for Claude Code's native /voice input on WSL: sudo apt install sox libsox-fmt-pulse (run in a plain terminal; sudo can't prompt inside a Claude Code session).

Sanity check without a session:

~/.claude/tts/venv/bin/python ~/.claude/tts/kokoro_say.py "If you can hear this, playback works."

WSL gotchas this repo already handles

These cost real debugging time; they're encoded in the scripts so you don't hit them:

  • WSLg audio-out stutters. Mic input through WSLg PulseAudio is fine; playback crackles under buffer pressure. Fix: write the wav to a Windows-readable temp dir and play it Windows-native via PowerShell. The Windows temp path is auto-detected once and cached (CLAUDE_TTS_WIN_TMP overrides).
  • Media.SoundPlayer can't read float wavs — soundfile's default. Everything is written PCM_16.
  • The Stop hook can fire before the final message is flushed to the transcript JSONL. A naive "read last message" speaks the previous turn. The quiescence gate + last_assistant_message shortcut handle this.
  • PostToolUse catch-all matcher is "*", not ".*".
  • PowerShell treats Unicode curly quotes as string delimiters — the SAPI path normalizes and escapes them, so an apostrophe in prose can't break out of the command string.

Tuning

Constants at the top of the scripts: voice (af_heart), speed (1.15), spoken budget per turn (480 chars), summary model (claude-haiku-4-5), daemon idle timeout (KOKORO_DAEMON_IDLE_S). The summarizer runs claude -p --safe-mode; if the CLI is missing, an offline first-sentence-per-paragraph extraction is used instead.

Tests

python3 -m pytest hooks/test_speak_summary.py

The suite (40 tests) covers the flush race, state atomicity, lock contention, kill-switch timing, the overlap path, and the text-to-speech cleaning rules. Tests never play audio.

License

MIT. Kokoro-82M weights are Apache-2.0 (hexgrad); kokoro-onnx is MIT (thewh1teagle).

About

Claude Code talks back: local offline TTS (Kokoro) for WSL2 with ~1-3s latency. Turn summaries, mid-turn narration, permission alerts.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages