Great comparison

[jerkstorecaller]

Thanks for making this, it's so nice to be able to sample all the TTSes in one place without having to get the models. It's appreciated. I have an ARM system and almost no TTSes work out of the box.

Some random thoughts:

• A new TTS just came out, dots.tts 2B. If the demos aren't cherry-picked this is the best TTS I've ever heard, including best voice cloning and best expressivity. You really want to add this. https://rednote-hilab.github.io/dots.tts-demo/
• Could you add MOSS TTS 1.5 which came out last month? It's not as impressive but it's pretty solid.
• Why are there only partial entries on some prompts? For example prompt 5 has way fewer entries than prompt 3. Does it take a long time to do a full run with all supported models?
• Is there some source that tracks WER for all models on the same benchmark? And if so, would you want to add it in a column on the Listen page, next to the Play Audio button for each model? This way, it helps quickly see the trade-offs of a great-sounding model. For example your 'TLDR best cloning choice,' OmniVoice, on Prompt 3 voice clone skips the entire beginning of the sentence (5 words). It feels wrong to make this your Great comparison #1 recommendation when it has such a glaring flaw, I don't care how good the voice is.

[5uck1ess]

@jerkstorecaller

Thanks for posting this. I will address each point, and let me know if you have any other questions.

• dots.tts 2B, I havent seen this model, the demos do sound good. I will begin the process of adding it in.
• MOSS TTS, looking into adding moss 1.5 as it is an upgrade to what I already have. (version 1.0 which sounded pretty good)
• Partial entries are because prompt 5 is French and a lot of the models are English-only, so they get skipped on that prompt. Prompts 1-4 are English and have fuller coverage.
• WER / objective metric is something I tried adding and am working on. Yes the OmniVoice has issues but for cloning i was going strictly by cloning fidelity but artifacts should be taken into account as well. Will fix. That rating will definitely swap once public voting system is introduced (wrapping up design for it)

Really appreciate you taking the time.

[5uck1ess]

@jerkstorecaller
dots.tts added
moss tts 1.5 added (to my ears and objectively kind of a minor regression but some prompts work better)
WER, SIM and UTMOS metrics added. Omnivoice cloning ranking corrected.

[jerkstorecaller]

Nice, thanks.

MOSS 1.5 is way better-sounding than 1.0, but still noticeably behind Higgs v3 (the best one IMO).

dots.tts turned out to be really unimpressive, I guess they cherry-picked the stuff they put in the demo. Fell for it again!

• P1 clone: speaks too fast
• P4 clone: drops/hallucinates words
• P5 default: very heavy foreign accent but still perfectly coherent, it's likely the default dots.tts voice just doesn't have french "sounds"
• P5 clone: this is complete gibberish, not a single recognizable french word

Later when I have a holiday I'll spend some time getting all that stuff to run on my ARM AI system (DGX Spark) and hopefully be able to contribute something, perhaps a different multi-lingual section that tests various things. There's some surprising undocumented utility there, for example if you want to generate english lines for someone who is intended to speak english with a foreign accent, like Hercule Poirot. Or an english native who is trying to speak french but really sucks at it (some models that don't claim to support french are actually decent at this). In these cases the TTS that makes them sound like a native speaker is actually counter-productive.

Can you tell me how long it takes you to do a full test run for a single prompt after everything is installed? Just an estimate.

[5uck1ess]

Rough estimate, splitting it out properly since the question was specifically about a single prompt:

One-time setup is the real cost. You need a HuggingFace account + API key set up ahead of time, and to accept the terms on a few gated models, otherwise those get skipped. Then it builds a virtual env per model and downloads all the weights — that part is network-bound and the bulk of the time (a lot of GB across ~40+ models, so several hours depending on your connection).

Once everything's installed and cached, an actual run of one prompt across all the models is only ~15–25 min on my Linux 3090. It's dominated by loading each model, not the generation — individual clips render in seconds; the time goes into cold-starting 40+ models one after another (the slow ones, like Mars5, take ~70s just to spin up). Running the full 5-prompt sweep is what stretches it into the several-hours range, since the slower models re-generate for every prompt.

So: hours of one-time setup + downloads, then ~15–25 min per single-prompt pass after that.

---

One thing to flag for your DGX Spark specifically, since it's ARM (GB10): those numbers are from my x86 + 3090 box and a couple of things will be different.

If you do get a multilingual/accent section working on it I'd be happy to take a PR — that Poirot / "english-speaker-faking-french" idea is a genuinely interesting axis the current prompts don't cover.