🎙️ Claude Voice AI

Real-time voice conversations for the Claude CLI.

Talk to Claude naturally from your terminal — speak, and hear it think back.

---

claude-voice is a native voice layer on top of the Claude CLI, not a speech-to-text wrapper. You speak; your words are transcribed and sent to claude; the streamed response is printed live and spoken aloud sentence by sentence as it is generated. Start talking again at any time and it stops, listens, and hands your interruption straight to Claude — like pair programming out loud.

✨ Features

• 🎤 Speak to Claude — continuous listening with silence detection, or push-to-talk.
• 🧠 Wraps the Claude CLI — uses your existing claude install and session; no Claude API key required.
• 🔊 Speaks as it thinks — responses are streamed to TTS one complete sentence at a time, so audio starts almost immediately.
• ✋ Barge-in / interruption — cut in mid-sentence; playback and generation stop instantly and it starts listening again.
• 🔌 Pluggable providers — Groq Whisper (default) or any OpenAI-compatible STT; ElevenLabs or Sarvam TTS. Add your own.
• 🩺 doctor — one command verifies your CLI, keys, mic, speaker, and network.
• 🧩 Extensible — an event-driven core with a plugin API for wake-words, clipboard, memory, notifications, MCP, and more.
• 🎨 Polished, branded TUI — an Ink-powered Claude Voice AI interface: a gradient-branded header with a persistent watermark, a spinner that shimmers through the brand palette, a scrollback transcript, live token streaming, colored role labels (❯ You, ✦ Claude), and a status bar with a mic meter + timings.
• ⌨️ Talk or type — press t to type a message instead of speaking, so you can use it even without a working mic.

🚀 Quick start

# 1. Install globally
npm install -g @aayushdebugging/claude-voice

# 2. Make sure the Claude CLI is installed and logged in
#    https://claude.com/claude-code
claude --version

# 3. Set your provider keys (see "Configuration" below)
export GROQ_API_KEY="…"        # speech-to-text
export ELEVENLABS_API_KEY="…"  # text-to-speech

# 4. Check everything is wired up
claude-voice doctor

# 5. Talk to Claude
claude-voice

You'll also need a microphone recorder backend and audio output:

Platform	Install
macOS	brew install sox (and brew install ffmpeg for gapless streaming)
Debian/Ubuntu	sudo apt-get install sox libsox-fmt-all alsa-utils ffmpeg
Windows	Install SoX and add it to PATH

ffmpeg (its ffplay) is optional but recommended: it enables gapless streamed speech. Without it, playback falls back to afplay/aplay clips (still works, with a tiny seam between sentences).

See the Installation guide for details and troubleshooting.

🎛️ Usage

claude-voice                      # start a conversation (continuous mode)
claude-voice --model opus         # pick a Claude model (opus | sonnet | fable)
claude-voice --voice river        # pick a TTS voice
claude-voice --stt groq           # choose the speech-to-text provider
claude-voice --tts sarvam         # choose the text-to-speech provider (elevenlabs | sarvam)
claude-voice --push-to-talk       # press SPACE to talk instead of continuous
claude-voice --no-speak           # text only, no spoken responses
claude-voice --language hi        # set STT + TTS language (ISO-639-1) or "auto"
claude-voice --speed 1.5          # speech rate multiplier, 0.5–3.0 (1 = natural)
claude-voice --no-stream          # speak the whole reply at once (default: as it streams)
claude-voice --fast-speech        # lowest latency: start after the first clause (may sound choppier)
claude-voice --local              # fully-local, free stack (whisper.cpp + Kokoro)

In-session keys (default push-to-talk):

Key	Action
SPACE	tap to start talking → it auto-stops when you finish speaking and transcribes. Tap again to send immediately, or to interrupt while Claude is replying.
t	type a message instead of speaking (Enter sends, Esc cancels, ↑/↓ recalls previous prompts)
/	open the command palette
q / Ctrl-C	quit

Command palette (press /):

Command	Action
/help	show keys & commands
/clear	clear the transcript
/mute · /speak	toggle spoken responses on the fly
/voice <name>	switch the TTS voice live (e.g. /voice karun)
/model <name>	switch the Claude model (opus/sonnet/fable) for the next turn
/speed <rate>	set the speech rate live — 0.5–3.0 (e.g. /speed 1.5 or /speed 2x)
/lang <code>	switch language for STT + TTS (e.g. /lang hi, /lang auto)
/stream	toggle speaking-as-Claude-writes vs speaking the whole reply at once
/quit	exit

While listening, a live mic-level meter in the status bar fills green as it hears you, so you always know the mic is working. Claude's answer streams to the screen as it's written and is spoken as it streams — text is chunked one sentence at a time and each sentence is synthesized ahead of playback, so audio starts moments after the first sentence and tracks the text as it appears. Sentences are the unit (rather than clauses) so the TTS engine voices internal punctuation — commas, dashes, colons — as natural pauses: splitting on every comma makes engines like Kokoro pad each fragment with a beat of trailing silence, i.e. an audible stop at each mark. For the lowest possible first-word latency at the cost of choppier prosody, --fast-speech chunks at clause boundaries instead. Generation, synthesis, and playback all run in parallel and never block each other.

For gapless streaming, install ffmpeg (brew install ffmpeg): the reply is fed sentence-by-sentence into a single persistent audio stream (ffplay), which waits during pauses rather than reopening the device — so it sounds like one continuous answer. Without a streaming player it falls back to playing a few back-to-back clips via afplay (a small seam between clips), and --no-stream (or /stream) speaks the whole reply at once with no seams at all. sox play is intentionally not used for streaming — it cuts off on the first pause.

While Claude generates, the status bar shows a live word count and words/sec, and each reply is tagged with its time and length (e.g. Claude · 2.4s · 45w).

The rich UI needs a real terminal. When output is piped or CLAUDE_VOICE_PLAIN=1 is set, it falls back to a plain streaming UI.

Commands

Command	What it does
claude-voice / claude-voice chat	Start a live voice conversation.
claude-voice serve	Host a remote voice client on your Wi-Fi — talk to Claude from your phone.
claude-voice say "<text>"	Speak a phrase aloud with the configured voice (no mic needed).
claude-voice config	View or edit configuration.
claude-voice local	Set up / check the fully-local, free stack (whisper.cpp + Kokoro).
claude-voice doctor	Diagnose your setup (CLI, keys, mic, speaker, network).
claude-voice update	Check for and install the latest version.
claude-voice version	Print the installed version.

⬆️ Updating

Already installed it? Upgrade to the latest published version:

# global install:
npm install -g @aayushdebugging/claude-voice@latest

# or, if it's a project dependency:
npm update @aayushdebugging/claude-voice

From v0.1.2 onward you can also self-update in place:

claude-voice update          # checks npm and installs the newest version
claude-voice update --check  # just check, don't install
claude-voice version         # what you're on now

Note: npm update takes a bare package name (no @latest); use npm install …@latest to force the newest, or npm update … to move within your saved range.

🔒 Fully local & free (no API keys)

Run entirely offline with open-source models — no cloud, no keys, $0:

• STT: whisper.cpp (Metal-accelerated on Apple Silicon)
• TTS: Kokoro (kokoro-onnx, CPU, no PyTorch)

It's a few one-time steps. claude-voice local downloads the models and reports status; you install the native runtimes once and start the two local servers (npm can't install compiled binaries, and the servers are long-running, so those stay in your hands).

1. Install the runtimes (one-time — native/Python, not from npm):

Platform	Command
macOS	brew install sox ffmpeg whisper-cpp uv
Debian/Ubuntu	sudo apt-get install sox ffmpeg · then build whisper.cpp and install uv

2. Download the models + check what's needed:

claude-voice local

This auto-downloads the whisper model (~150 MB, with a progress bar), writes the Kokoro server script to ~/.claude-voice/kokoro-server.py, and prints a ✔/✖ status report with the exact command for anything still missing. (--no-download prints the model's curl command instead of fetching it.)

3. Start the two servers — each in its own terminal, since they keep running:

# speech-to-text (whisper.cpp)
whisper-server -m ~/.claude-voice/models/ggml-base.en.bin --host 127.0.0.1 --port 8081

# text-to-speech (Kokoro — downloads its ~340 MB model on first run)
uv run --with kokoro-onnx --with numpy python ~/.claude-voice/kokoro-server.py --port 8880

4. Talk, pointed at the local stack:

claude-voice --local
# …or make it the permanent default:
claude-voice config --set stt=whispercpp --set tts=kokoro --set voice=af_heart

Re-run claude-voice local any time to see live status (which servers are up, which models exist). If you see whisper.cpp request failed: fetch failed, a server just isn't running yet — start it with step 3.

Any OpenAI-compatible STT/TTS server works too (point providers.whispercpp.baseUrl / providers.kokoro.baseUrl at it).

📱 Talk from your phone (remote)

Run a token-protected voice session on your machine and drive it from your phone on the same Wi-Fi — hands-free with headphones:

claude-voice serve

It prints a link and a QR code. Scan it (or open the link) on your phone:

1. Accept the one-time certificate warning (it's a self-signed cert — needed so the phone's browser will allow microphone access on the LAN).
2. Allow microphone access.
3. Hold the button, speak, release. Your words are transcribed on your machine, sent to Claude, and the reply streams back as text and plays aloud on the phone. Tap Stop to cut off a reply, or type instead.

Safe by default. A remote session runs with all of Claude's tools disabled — it can talk, but it can't run shell commands, edit files, or read your disk. The link carries a secret token, but anyone on your network who has it can talk to Claude on your machine, so only share it with yourself.

claude-voice serve --host 127.0.0.1   # keep it on this machine only
claude-voice serve --port 8080        # change the port
claude-voice serve --max-clients 2    # cap simultaneous devices
claude-voice serve --allow-tools      # DANGER: let remote prompts run tools

Please read SECURITY.md before using --allow-tools or exposing the server on an untrusted network.

🔑 Configuration

Configuration lives at ~/.claude-voice/config.json. API keys are never stored there — they are read from environment variables:

Provider	Env var
Groq (STT, default)	GROQ_API_KEY
OpenAI (STT)	OPENAI_API_KEY
ElevenLabs (TTS, default)	ELEVENLABS_API_KEY
Sarvam (TTS)	SARVAM_API_KEY

{
  "stt": "groq",              // "groq" | "openai"
  "tts": "elevenlabs",        // "elevenlabs" | "sarvam"
  "voice": "aria",            // TTS voice name
  "model": "opus",            // Claude model alias
  "pushToTalk": false,        // continuous vs push-to-talk
  "autoSpeak": true,          // speak responses aloud
  "streamSpeech": true,       // speak as Claude writes (vs the whole reply at once)
  "voicePrompt": "…",         // steers Claude to answer in a voice-friendly way ("" disables)
  "language": "auto",         // STT + TTS language (ISO-639-1) or "auto"
  "speechRate": 1.0,          // speech rate multiplier, 0.5–3.0 (1 = natural)
  "silenceTimeoutMs": 1500,   // silence that ends an utterance
  "micSensitivity": 150,      // lower = picks up quieter mics; higher = ignores noise
  "sampleRate": 16000
}

Manage it from the CLI:

claude-voice config                       # print current config + path
claude-voice config --edit                # open in $EDITOR
claude-voice config --set voice=river --set model=sonnet
claude-voice config --get providers.groq.model
claude-voice config --reset               # restore defaults

🧠 How it works

🎤 Microphone → 📝 Speech-to-Text → 🤖 Claude CLI (stream-json)
        ↑                                     ↓
   barge-in                          🔎 Sentence parser
        ↑                              ↓            ↓
  🔊 Speakers ← 🗣️ TTS queue ←──────────┘     🖥️ Terminal (live tokens)

Claude's output is streamed token-by-token. A streaming sentence parser accumulates tokens until a sentence completes (., ?, !, or a paragraph break), then hands each finished sentence to an interruptible speech queue so audio begins while Claude is still generating. Every stage communicates over a typed event bus, so the UI, plugins, and providers stay decoupled.

Read the full Architecture guide.

🧩 Programmatic use

claude-voice is also a library. The CLI is a thin layer over these primitives:

import { createSession, loadConfig } from '@aayushdebugging/claude-voice';

const config = await loadConfig({ model: 'opus', voice: 'river' });
const { conversation, bus } = createSession({ config });

bus.on('SpeechRecognized', ({ text }) => console.log('You said:', text));
bus.on('ClaudeToken', ({ text }) => process.stdout.write(text));

await conversation.start(); // Ctrl-C to stop

See the API documentation for the full surface, including how to write custom STT/TTS providers and plugins.

🩺 Troubleshooting

Run claude-voice doctor first — it pinpoints most issues. Common ones:

• "Claude CLI not found" — install it from https://claude.com/claude-code and ensure claude is on your PATH.
• "recording backend not found" / "Microphone capture needs sox" — install sox (brew install sox; see the table above).
• "No microphone audio detected" — either sox isn't installed, or your terminal lacks mic permission. Install sox, then grant access under System Settings → Privacy & Security → Microphone (to the terminal app you launched from — e.g. Terminal, iTerm, or VS Code) and restart it.
• "whisper.cpp request failed: fetch failed" (or the same for kokoro) — the local STT/TTS server isn't running. Start it (see Fully local & free, step 3) or run claude-voice local to see which servers are up. fetch failed = nothing is listening on the configured port.
• "speaker module unavailable" — the optional native audio module failed to build; install your platform's build tools and reinstall. --no-speak works meanwhile.
• 401 from a provider — double-check the corresponding *_API_KEY.

🔐 Security

claude-voice drives the Claude CLI — a capable agent — from your voice. Local sessions have full tool access (like your terminal); remote (serve) sessions disable all tools by default so a spoken or injected prompt can't run commands or touch your files. Before exposing a server or using --allow-tools, read SECURITY.md for the threat model and safe-use guidance. To report a vulnerability, see the reporting section there (please don't file a public issue).

🤝 Contributing

Contributions are very welcome! See the Contributing guide for the dev setup, coding standards, and how to add a provider or plugin.

📄 License

MIT © claude-voice contributors