Repository navigation
feat: put text-to-speech providers on one contract and open narration to all of them - #356
Merged
Merged
Conversation
… to all of them - `SpeechGenerator` (ai-providers/src/speech): `synthesize(request)` returns audio bytes with their container (ElevenLabs and OpenAI MP3, Kokoro WAV) or throws `ProviderError`. `createSpeechGenerator(provider, key)` opens one by name. - ElevenLabs and OpenAI TTS go through `providerRequest` instead of their own 429 loops; models resolve through the catalog, which now lists ElevenLabs Multilingual v2 and Flash v2.5, OpenAI tts-1/tts-1-hd, and Kokoro. ElevenLabs refuses text over a model's character limit, and OpenAI refuses voices tts-1 does not have, before any request. - `tts-resolve` builds on the registry, so scenes and builds get provider error kinds and the model that spoke. - `vibe generate narration` and the `generate_narration` tool were ElevenLabs-only; they now take `--provider auto|elevenlabs|openai|kokoro` plus `--model` and `--speed`. `writeAudioFile` converts when the output extension does not match the audio (Kokoro WAV to `narration.mp3`). - A missing key is an auth error (exit 4) rather than a generic failure. - runway.md no longer lists the removed Runway image model. Claude-Session: https://claude.ai/code/session_011S1BLTaFctZNQzGERpkCuA
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Speech counterpart of the video (#349-#353) and image (#355) provider work.
What changes
SpeechGenerator(ai-providers/src/speech):synthesize(request)returns audio bytes with their container (ElevenLabs and OpenAI MP3, Kokoro WAV), or throwsProviderError.createSpeechGenerator(provider, key)opens a provider by name.providerRequestinstead of their own 429 loops.tts-1/tts-1-hd(shutting down 2027-01-06), and Kokoro.tts-1does not have.tts-resolveon the registry, so scenes and builds get provider error kinds and the model that spoke.vibe generate narrationand thegenerate_narrationtool were ElevenLabs-only, even though the MCP server's own instructions tell agents to offer Kokoro, OpenAI, or ElevenLabs. They now take--provider auto|elevenlabs|openai|kokoro,--model, and--speed.writeAudioFileconverts with ffmpeg when the output extension doesn't match the audio, for example Kokoro WAV tonarration.mp3.Verification
auth.quota,tts-1voices, Kokoro's empty-text refusal.tts-resolvetests now run against the registry.pnpm build,lint,typecheck, tests (providers 239, CLI 1343 including integration, MCP 73),providers:checkwith 0 warnings, pre-push gate.vibe generate narrationwith-p openai: 3.77 s MP3.-p kokoro -o kokoro.mp3: WAV converted to a real MP3, 3.13 s.-p elevenlabs -m flash: Flash v2.5, 2.83 s.AUTH_ERROR, exit 4.https://claude.ai/code/session_011S1BLTaFctZNQzGERpkCuA