Add results from Hugging Face's Open TTS Leaderboard
#30
by bezzam HF Staff - opened
README.md
CHANGED
|
@@ -169,6 +169,25 @@ Supertonic 3 runs fast on CPU, even compared with larger baselines measured on A
|
|
| 169 |
|
| 170 |
At about 99M parameters across the public ONNX assets, Supertonic 3 is much smaller than 0.7B to 2B class open TTS systems. The smaller model size is a practical advantage for download size, startup time, and on-device inference.
|
| 171 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 172 |
## Supported Languages
|
| 173 |
|
| 174 |
| Code | Language | Code | Language | Code | Language | Code | Language |
|
|
|
|
| 169 |
|
| 170 |
At about 99M parameters across the public ONNX assets, Supertonic 3 is much smaller than 0.7B to 2B class open TTS systems. The smaller model size is a practical advantage for download size, startup time, and on-device inference.
|
| 171 |
|
| 172 |
+
### Open TTS Leaderboard
|
| 173 |
+
|
| 174 |
+
As of **September 9, 2026**, on Hugging Face's [Open TTS Leaderboard](https://huggingface.co/spaces/hf-audio/open_tts_leaderboard) — which scores open-source TTS models on the same texts across two multilingual eval sets ([Seed-TTS eval](https://github.com/BytedanceSpeech/seed-tts-eval) and [CV3-Eval](https://github.com/QwenAudio/CV3-Eval)) — Supertonic 3 ranks:
|
| 175 |
+
|
| 176 |
+
- 🥈 **2nd on intelligibility (WER)**, averaged across its supported evaluated languages (English, French, Spanish, German, Italian, Japanese, Korean, Russian); 🥇 **1st on WER** specifically for **Korean**.
|
| 177 |
+
- 🥈 **2nd on RTFx** (generation speed).
|
| 178 |
+
- 📈 On the **Pareto frontier** for both **WER vs RTFx** and **WER vs Model size**.
|
| 179 |
+
|
| 180 |
+
<p align="center">
|
| 181 |
+
<img width="45%" alt="Top 10 models by WER" src="https://huggingface.co/datasets/bezzam/tts_leaderboard_screenshots/resolve/main/supertonic-3/top_wer.png" />
|
| 182 |
+
<img width="45%" alt="Top 10 models by RTFx" src="https://huggingface.co/datasets/bezzam/tts_leaderboard_screenshots/resolve/main/supertonic-3/top_rtfx.png" />
|
| 183 |
+
</p>
|
| 184 |
+
|
| 185 |
+
<p align="center">
|
| 186 |
+
<img width="45%" alt="WER vs RTFx Pareto frontier" src="https://huggingface.co/datasets/bezzam/tts_leaderboard_screenshots/resolve/main/supertonic-3/wer_rtfx.png" />
|
| 187 |
+
<img width="45%" alt="WER vs model size Pareto frontier" src="https://huggingface.co/datasets/bezzam/tts_leaderboard_screenshots/resolve/main/supertonic-3/wer_size.png" />
|
| 188 |
+
</p>
|
| 189 |
+
|
| 190 |
+
|
| 191 |
## Supported Languages
|
| 192 |
|
| 193 |
| Code | Language | Code | Language | Code | Language | Code | Language |
|