Skip to content

clone a friend’s voice from 15s of voice notes for text-to-voice messages - #67

Closed
rafapolo wants to merge 4 commits into
crmne:mainfrom
rafapolo:feature/voice-clone-unlock
Closed

rafapolo wants to merge 4 commits into
crmne:mainfrom
rafapolo:feature/voice-clone-unlock

Conversation

@rafapolo

@rafapolo rafapolo commented Sep 18, 2026

Copy link
Copy Markdown

Summary

  • Once a friend's incoming voice notes add up to 15s total, build a local reference clip and unlock playing their text messages back in their cloned voice
  • Synthesis is fully offline/on-device via vox's Chatterbox Turbo backend (zero-shot cloning from the reference clip), on demand per message and cached after first play
  • Per-friend "voice playback: Auto" mode pre-synthesizes new incoming text in the background (never auto-plays) so playback is instant when tapped

Test plan

  • cargo test --lib (289 passed)
  • cargo clippy --all-targets (clean)
  • Manual run through the unlock → play → auto-mode flow with real voice notes (not yet done — reviewer heads-up: the runtime cloning/playback path, including the vox/Chatterbox model download, is unverified beyond compilation and unit tests)

Once a friend's incoming voice notes add up to 15s, build a local
reference clip and unlock playing their text messages back in a
cloned voice, synthesized on demand and fully offline via vox's
Chatterbox Turbo backend. Adds a per-friend auto-synthesize mode
that pre-warms new text messages in the background without
auto-playing them.
@rafapolo
rafapolo marked this pull request as ready for review September 18, 2026 10:01
@rafapolo rafapolo changed the title WIP: clone a friend's voice after 15s of voice notes clone a friend's voice after 15s of voice notes Sep 18, 2026
@rafapolo rafapolo changed the title clone a friend's voice after 15s of voice notes clone a friend's voice after 15s of voice notes to play as next text messages Sep 18, 2026
@rafapolo rafapolo changed the title clone a friend's voice after 15s of voice notes to play as next text messages clone a friend's voice after 15s of voice notes to play as text messages Sep 18, 2026
@rafapolo rafapolo changed the title clone a friend's voice after 15s of voice notes to play as text messages clone a friend’s voice from 15s of voice notes for text-to-voice messages Sep 18, 2026
ort-sys (ONNX Runtime, pulled in by vox's chatterbox backend)
downloads prebuilt binaries from a CDN during its build script,
which the sandboxed, network-less Flatpak build cannot do. Gate
vox/hound behind a non-default `voice-clone` Cargo feature so the
Flatpak build (default features only) skips them entirely, while
regular CI still exercises the real cloning code via --all-features.
@crmne

crmne commented Sep 18, 2026

Copy link
Copy Markdown
Owner

Voice cloning and the substantial ONNX/runtime dependencies are outside Zapfast’s intended scope, and they introduce substantial privacy, consent, and maintenance concerns.

@crmne crmne closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants