Per-character TTS in Story Mode quotes (Web Speech API)

[adhit-r]

Skills: Web Speech API · TypeScript · per-speaker voice mapping
Time: ~6 hours
Good for: Voice-UI devs · accessibility advocates · audio engineers
Difficulty: Intermediate

---

Context

Story Mode currently surfaces quotes as text-on-card during each beat. Adding TTS narration would make stories accessible to low-vision users AND give them more cinematic weight. The Web Speech API's SpeechSynthesis is built into browsers — no model download, no copyright issue (synthetic voices aren't actor likenesses).

Goal

Each StoryBeat.quote is read aloud when its beat starts (provided audio is enabled and the user opted in). Different speakers get different voice characteristics — pitch, rate, perceived "weight" — chosen for canon resonance, NOT impersonation.

Where to start

• components/cinematic/StoryMode.tsx — beat changes already trigger an effect; hook TTS there.
• New lib/audio/tts.ts — speaker → SpeechSynthesisUtterance config (rate, pitch, voice).
• Suggested mappings (canon-evocative, not actor-faithful):
  • Anakin / Luke: default rate, mid pitch
  • Obi-Wan: slightly slower, slight pitch lift
  • Yoda: very slow, low pitch
  • Vader: very slow, very low pitch
  • Padmé / Leia: default rate, slight pitch lift
  • Palpatine: slow, low, raspy (try voice filter for "gravelly" available voices)
• Honor existing audio toggle. If audio is off, no TTS.
• Stop in-flight utterance on beat advance, pause, end-story.

Acceptance criteria

• TTS opt-in alongside the existing audio toggle (don't autoplay synthesized speech without user gesture)
• Per-speaker variation audible
• Stops cleanly on Esc / pause / next-beat
• Reduced-motion is irrelevant; reduced-data optionally suppresses
• Browser support: SpeechSynthesis is in all browsers but voice availability varies — pick by lang === 'en-US' and pitch/rate, not by named voice
• Captions stay readable even with TTS active

Reference

https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis