VoiceCue

VoiceCue is a Windows-first, local-first voice-following teleprompter built with Electron, React, and TypeScript. It follows speech instead of a fixed scroll speed.

The current public version focuses on desktop architecture, secure IPC boundaries, reproducible builds, automated testing, local Markdown script management, deterministic transcript matching, and a bilingual floating teleprompter interface. Local realtime mode uses CPU streaming recognition for Chinese/English voice following. Local Whisper remains available and selected by default; Cloud transcription remains optional.

Portable users: double-click VoiceCue-portable\VoiceCue.exe. No Python, Node, venv or API key is needed. Open a saved script, select Local realtime (CPU, Chinese / English) for low-latency following, start the presentation, then show the Prompter and speak. The first model download requires internet access; a cached model loads offline. The input meter, transcript and highlighted reading position show the three stages of the pipeline.

Developer setup uses an isolated Python environment:

python -m venv .venv-local-asr
.\.venv-local-asr\Scripts\python.exe -m pip install -r local-asr\requirements.txt
npm run dev

Whisper uses multilingual small, attempts actual GPU warmup and falls back to CPU INT8 when acceleration is unavailable. GPU runtime libraries are not bundled. Streaming uses an independently licensed, pinned bilingual Zipformer model and does not need a GPU. See docs/local-speech.md and docs/portable-asr-verification.md for evidence and limitations.

Streaming capture retains at most three seconds of pending audio, plus one in-flight batch. Brief backlog is merged without losing samples. If the pending limit is exceeded, the UI explicitly reports skipped queued audio and the recognizer resets its context before continuing. This is teleprompter recovery, not lossless meeting transcription. Initial acquisition from the middle can require multiple complete utterances. Physical microphone acceptance is still outstanding.

Mock mode needs no network or cloud account. Real speech requires Main-process environment variables (never use VITE_ for secrets):

$env:VOICECUE_ASR_BASE_URL = 'https://api.openai.com'
$env:VOICECUE_ASR_API_KEY = 'your-local-secret'
$env:VOICECUE_ASR_MODEL = 'gpt-4o-mini-transcribe'
npm run dev

The base URL is configurable for compatible providers. Cloud audio is uploaded as bounded PCM WAV chunks. See docs/real-speech.md and .env.example.

Requirements and commands

Windows is the primary development platform. Install Node.js 22.12+ (Node.js 24 LTS recommended), npm and Git. Initial installation requires network access to the npm registry and Electron binary downloads.

git clone https://github.com/zms618/VoiceCue.git
cd VoiceCue
npm install
npm run dev

For a reproducible install after cloning, use npm ci with the committed lockfile. npm run dev builds Main and Preload, starts Vite on loopback and opens Electron. Close the window and stop the development server with Ctrl+C when finished.

Electron 44 downloads its binary on demand. The project's postinstall runs the official install-electron command so electron-vite can find it immediately. If GitHub binary downloads are unreachable, Electron documents this optional mirror (set only for the current PowerShell session):

$env:ELECTRON_MIRROR = 'https://npmmirror.com/mirrors/electron/'
npm install

The installer verifies the downloaded binary against Electron's checksums.

npm run build   # Strict type checks and production bundles in out/
npm start       # Open the production build in Electron
npm test        # Vitest matching and IPC boundary tests, single run
npm run lint   # ESLint, zero warnings allowed
npm run typecheck

build produces runnable application bundles for local use.

Windows portable acceptance build

For a ready-to-run Windows development/acceptance copy, create the isolated Python environment once, then run:

python -m venv .venv-local-asr
npm run package:win

The script verifies the project, builds the faster-whisper worker with PyInstaller, stages the installed Electron runtime with an app.asar, and writes the portable application to VoiceCue-portable\VoiceCue.exe. Close this portable copy before repackaging; the script checks for its running processes before updating files. The bundled worker runs automatically as a child process; users install neither Node nor Python. Keep the whole portable folder together. Whisper models live in VoiceCue-portable\models and are preserved by packaging. The first Local Speech start may download the model there; later starts load the cache offline.

Local availability starts that exact bundled worker and loads the cached model; it does not infer readiness from a file or non-empty directory. Startup failures, missing models and ready state are displayed separately. Diagnostic events are written to %APPDATA%\VoiceCue\logs\local-asr.log, without audio or transcript contents.

npm run package:asr rebuilds only the worker when local-asr/ changes. Normal UI/Core changes can use npm run package:win; the command intentionally rebuilds the worker as a simple, reliable default. Development mode continues to use npm run dev and the project-local Python environment.

Stack and architecture

Electron, TypeScript (strict), React, Vite via electron-vite, Vitest and ESLint. electron.vite.config.ts is the single Vite configuration for three explicit entry points; a separate vite.config.ts is unnecessary.

• src/main/: desktop lifecycle, main window and IPC handler. Main owns application metadata.
• src/preload/: contextBridge exposes typed script, session, speech and Prompter operations.
• src/renderer/: React script library/editor using only the narrow preload API.
• src/core/script/: authoritative script model and deterministic Markdown parser.
• src/main/storage/: local JSON persistence under Electron's per-user data directory.
• src/main/speech/: provider contract, lifecycle service and deterministic mock provider.
• src/main/session/: narrow orchestration from parsed script through ReadingTracker.
• src/core/matching/: normalization, edit distance, local fuzzy matching, reading tracking, minimal input types and validated configuration. Its dedicated strict TypeScript check excludes DOM and Node ambient types.
• src/shared/: cross-process types and the sole IPC channel constant.
• tests/: matching behavior and bootstrap IPC tests; fixtures/presentation.ts contains an original 1,559-character, 82-sentence Chinese presentation.

Security

Context isolation, sandboxing and web security are enabled; renderer Node integration is disabled. No filesystem or raw Electron IPC API is exposed. The Main handler validates the sender and top-level frame. New windows, navigation and non-audio permission requests are denied. A CSP restricts content to the application, with loopback WebSocket access and inline styles for Vite development. Production loads the bundled local HTML. Optional cloud credentials stay in Main.

ESLint restricts privileged imports in Renderer and Core/Shared. Core and Shared must remain free of Electron, React, Node and DOM dependencies.

See the matching core design for API examples, scoring, transcript semantics, test scenarios and known limitations. See script management for Markdown syntax, indexing, storage and process boundaries. See speech pipeline for provider lifecycle, fixtures, session state and future source separation.

Current direction and verification

See the source audit and phased product decisions and P0 portable verification. The reference repository has no explicit source license at the audited revision; this iteration borrows design ideas without copying its source.

Settings, hotkeys, click-through and Meeting/Q&A providers are planned phases, not implemented product claims. Existing slide metadata stays supported; PPT automation is not a current priority.

Developers can reproduce the packaged speech smoke test:

powershell.exe -NoProfile -File scripts/create-smoke-audio.ps1
node scripts/smoke-portable.mjs verification/speech-en.wav verification/speech-en.md en-US

This launches the actual portable EXE from a different working directory with an isolated Chromium profile and a synthetic microphone WAV. It verifies real Whisper transcripts, cursor progress, overlay highlighting and session restart. Physical microphone acceptance remains a separate manual check.