Skip to content

Voice: stream spoken replies from a server voice (TTS) with sentence-level buffering #392

Description

@SecureCloudGroup

Spoken replies use the browser's system voices; the optional server voice (Server voice model under Settings → Local models → Voice, e.g. kokoro) is fetched per sentence as a whole audio blob. Streaming the server voice (start playing before the sentence finishes synthesizing) would cut the pause before the first spoken word on long replies.

Constraints: the Speaker chain (web/src/lib/audio/speaker.ts) must keep its idle/stop semantics — conversation mode reopens the mic on idle, and Stop must cut audio immediately; the microphone must stay closed while audio plays (an open capture makes the OS duck the voice).

Done means: with a server voice configured, the first audible word of a multi-sentence reply arrives within ~1 s of the first sentence completing, Stop cuts it within 200 ms, and conversation mode still reopens the mic only after the last chunk ends.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is neededvoiceDictation, spoken replies, conversation mode

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions