license: apache-2.0
language:
- en
pipeline_tag: text-to-speech
library_name: onnx
base_model:
- hexgrad/Kokoro-82M
- gradium/phonon-31m-en
- sahilmahendrakar/Paradee-8M-v1.1
tags:
- text-to-speech
- tts
- onnx
- webassembly
- wasm
- on-device
- browser
- cpu
- tiny
- distillation
- kokoro
- phonon
widget:
- text: >-
This is where the page began: one small voice, made right here in your
browser, faster than you can blink.
example_title: Heart · US female
output:
url: >-
https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/heart.mp3
- text: >-
No server is reading this to you. Every word is being made on your device,
as you listen.
example_title: Wren · US female
output:
url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/wren.mp3
- text: >-
Take a breath and settle in. I can read you anything you like, as slowly
as you like.
example_title: Everly · US female
output:
url: >-
https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/everly.mp3
- text: >-
Your text never leaves this tab. It turns into speech right here, in a few
milliseconds.
example_title: Hadley · US female
output:
url: >-
https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hadley.mp3
- text: Put the kettle on, find a comfy chair, and I'll read you the next chapter.
example_title: Bryony · UK female
output:
url: >-
https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/bryony.mp3
- text: >-
Right, let's get straight to it. Tell me what to say, and I'll say it with
a smile.
example_title: Maeve · UK female
output:
url: >-
https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/maeve.mp3
- text: >-
Give me a headline, and I'll make it sound like the most important news of
the day.
example_title: Miles · US male
output:
url: >-
https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/miles.mp3
- text: >-
Hey, I'm running in your browser right now, and honestly? I'm having a
great time.
example_title: Trevor · US male
output:
url: >-
https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/trevor.mp3
- text: >-
Good evening. Shall we begin? I'll be narrating tonight's story, clearly
and precisely.
example_title: Edwin · UK male
output:
url: >-
https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/edwin.mp3
- text: Lean in a little. I'll keep it gentle, and I'll keep it short.
example_title: Hugh · UK male
output:
url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hugh.mp3
mesm-tts: ten tiny English voices, 100 to 300 times faster than real time on one CPU core
Ten English text-to-speech voices, small enough to ship in a web page and fast enough to stream on one CPU core. No GPU, no server, and no onnxruntime in the browser: a 40 kB WebAssembly engine runs them. Try all ten in the mesmer.tools TTS Generator.
| Size | 2.3 to 6.9 MB per voice, vs. 325 MB (Kokoro-82M) and 55 to 103 MB (Phonon-31m-en) |
| Native, 1 CPU core | Heart 234x to 292x real time (about 150x to 180x faster than Kokoro-82M); Phonon voices 82x to 106x (about 30x faster than Phonon-31m-en) |
| Browser, 1 CPU core | Heart 149x to 201x, Phonon voices 55x to 80x; first audio after 10 to 55 ms |
| Memory | 100 to 117 MB peak for the whole process (Heart), vs. 770 to 849 MB with onnxruntime-web; the engine is 40 kB |
| iPhone 13 Pro | about 50x real time (Heart), about 33x (Phonon voices) |
| Quality | Heart: UTMOS 4.46, WER 5.6% (Kokoro-82M: 4.52, 5.7%). Every Phonon voice is statistically tied with its teacher on WER |
Listen
Each voice reading its own line, as a browser tab plays it.
| Voice | Sample |
|---|---|
| Heart · US female · the default | |
| Wren · US female · Phonon's built-in | |
| Everly · US female · soft | |
| Hadley · US female · calm | |
| Bryony · UK female · warm | |
| Maeve · UK female · bright | |
| Miles · US male · resonant | |
| Trevor · US male · lively | |
| Edwin · UK male · crisp | |
| Hugh · UK male · gentle |
Student vs. original. The same passage, read by each small model and by its teacher:
The lighthouse keeper climbed the stairs one last time. Below him, the harbor lights blinked awake, one by one, and the evening ferry sounded its horn. He had kept this light burning for thirty-one years. Tonight, for the first time, he wasn't sure he was ready to let it go.
| Voice | Read by | Sample |
|---|---|---|
| Heart | mesm-tts | |
| Kokoro-82M | ||
| Wren | mesm-tts | |
| Phonon-31m-en | ||
| Everly | mesm-tts | |
| Phonon-31m-en | ||
| Hadley | mesm-tts | |
| Phonon-31m-en | ||
| Bryony | mesm-tts | |
| Phonon-31m-en | ||
| Maeve | mesm-tts | |
| Phonon-31m-en | ||
| Miles | mesm-tts | |
| Phonon-31m-en | ||
| Trevor | mesm-tts | |
| Phonon-31m-en | ||
| Edwin | mesm-tts | |
| Phonon-31m-en | ||
| Hugh | mesm-tts | |
| Phonon-31m-en |
No model saw this passage in training, and each original is level-matched to its student. For reference: Paradee-8M v1.1 reading it, the model Heart started from.
The voices
| Voice | Learned from | ONNX file | Native | Browser |
|---|---|---|---|---|
| Heart lite (default) | Kokoro-82M af_heart |
2.64 MB | 274x | 185x |
| Heart v2f (best scores) | 3.42 MB | 234x | 149x | |
| Heart v2g (streams first) | 2.32 MB | 292x | 201x | |
| Wren | Phonon-31m-en, built-in voice | 5.34 MB | 105x | 80x |
| Everly, Hadley, Bryony, Maeve, Miles, Trevor, Hugh | Phonon-31m-en + phonon-voices | 5.34 MB | 104x to 106x | 80x |
| Edwin | 6.50 MB | 82x | 55x |
x real time on 1 core: onnxruntime 1.31 natively; pdn relaxed SIMD in Chrome and Edge (Firefox and Safari: 70 to 85% of that).
Quick start
Python
pip install -r requirements.txt # about 0.5 GB, no PyTorch
python tts.py "Hello there. How are you?" --voice hadley --out hello.wav # --list for voices; Heart takes --model v2g
from tts import MesmTTS # downloads the voice's ONNX file on first use
tts = MesmTTS("miles")
audio = tts.synth("Good evening.") # float32, 24 kHz
tts.save("Good evening.", "evening.wav")
Browser, no install
<script type="module">
import { loadParadee, relaxedSimdSupported } from "https://mesmer.tools/paradee/pv/pv.mjs";
const tts = await loadParadee({
model: "https://huggingface.co/mesmertech/mesm-tts/resolve/main/web/voices/hadley.pdn",
base: "https://mesmer.tools/paradee/v2/",
wasm: `https://mesmer.tools/paradee/pv/engine/${relaxedSimdSupported() ? "pdn_r" : "pdn"}.wasm`,
join: { gapScale: 0.496, leadKeepS: 0.02 }, // this voice's "join" in voices.json
espeakUrl: "https://mesmer.tools/paradee/espeak.mjs", // eSpeak NG for unknown words (GPL)
});
const audio = await tts.synth("Your text never leaves this tab."); // Float32Array, 24 kHz
for await (const chunk of tts.stream("Long text streams, sentence by sentence.")) { /* play chunk */ }
</script>
Heart uses the v2 bundle: loadParadee({ model: "v2f_lite" }) from https://mesmer.tools/paradee/v2/paradee.mjs. To
self-host, serve the repo root with python -m http.server 8000 and open http://localhost:8000/web/demo.html. Bundlers and
workers: web/v2/README.md.
Node, no dependencies
node web/say.mjs "Good evening. Shall we begin?" evening.wav edwin # --list for voices
Raw ONNX
Every file in onnx/ has Paradee-8M v1.1's interface:
| name | type | |
|---|---|---|
input_ids (in) |
int64 [1, T] |
0, phoneme ids (tokenizer.json), 0; T ≤ 512 |
speed (in) |
float32 [1] |
1.0 = trained rate |
waveform (out) |
float32 [1, samples] |
mono, 24 kHz |
import numpy as np, onnxruntime as ort
s = ort.InferenceSession("onnx/voices/hadley.onnx", providers=["CPUExecutionProvider"])
wav = s.run(None, {"input_ids": ids, "speed": np.ones(1, np.float32)})[0][0] # ids: np.int64 [1, T]
Benchmarks
Both images are drawn by assets/make_images.py from the repo's own samples and benchmark files; teachers' speeds are scaled
from interleaved runs. Full tables and protocols: benchmarks/README.md.
Native speed (onnxruntime 1.31, 1 thread)
| Model | x real time | vs. Paradee-8M v1.1 |
|---|---|---|
| Paradee-8M v1.1 int8 | 42.3 | 1.00 |
| Heart v2g | 291.5 | 6.89x |
| Heart v2f lite | 273.8 | 6.47x |
| Heart v2f | 234.1 | 5.53x |
| Phonon voices (Edwin) | 104.1 to 105.8 (82.0) | 2.46x to 2.50x (1.94x) |
Teachers, same core: Kokoro-82M (PyTorch) 1.04x real time, Phonon-31m-en (f32) 1.7x to 2.9x.
Browser speed (1 thread, V8)
| Model | onnxruntime-web | pdn (relaxed SIMD) | First audio | Cold start |
|---|---|---|---|---|
| Paradee-8M v1.1 | 15.1x | 3.6 s | ||
| Heart v2g | 96.5x | 201.3x | 10 ms | 60 ms |
| Heart v2f lite | 84.8x | 185.2x | 20 ms | 75 ms |
| Heart v2f | 73.4x | 148.7x | 29 ms | 84 ms |
| Phonon voices (Edwin) | 80.0x (54.7x) | 55 ms | 111 to 115 ms |
Peak process memory for Heart: 100 to 117 MB with pdn (38 to 40 kB brotli) and 770 to 849 MB with onnxruntime-web.
Phones
| Device | Heart | Phonon voices | v1.1 on onnxruntime-web |
|---|---|---|---|
| iPhone 13 Pro, Safari (live page, measured by hand) | ~50x | ~33x | |
| Desktop core slowed about 6x (low-end phone proxy) | 32x (v2g) | 2.6x |
Quality: Heart (TEST-200, held out, 3 seeds)
| Paradee-8M v1.1 | Heart v2f | Heart v2f lite | Heart v2g | Kokoro-82M | |
|---|---|---|---|---|---|
| UTMOS ↑ | 4.391 | 4.462 | 4.453 | 4.453 | 4.520 |
| WER ↓ | 5.85% | 5.65% | 5.64% | 5.48% | 5.67% |
| WER, capped ↓ | 5.17% | 4.91% | 4.84% | 4.91% |
Heart is significantly more natural than v1.1 (UTMOS +0.06 to +0.07) and statistically tied with v1.1 and Kokoro-82M on WER.
Quality: Phonon voices (DEV-400, vs. their own teachers)
| Voice | UTMOS, student / teacher | WER, student / teacher | Capped WER, student − teacher, 95% CI (pp) |
|---|---|---|---|
| Wren | 4.417 / 4.452 | 6.24% / 6.11% | [−0.26, +0.61] |
| Everly | 4.346 / 4.434 | 6.09% / 6.12% | [−0.54, +0.29] |
| Hadley | 4.469 / 4.496 | 5.98% / 6.10% | [−0.41, +0.51] |
| Bryony | 4.172 / 4.331 | 6.95% / 7.43% | [−0.78, +0.21] |
| Maeve | 4.391 / 4.442 | 6.31% / 6.20% | [−0.39, +0.59] |
| Miles | 4.258 / 4.365 | 6.17% / 6.45% | [−0.49, +0.36] |
| Trevor | 4.323 / 4.425 | 6.23% / 6.17% | [−0.30, +0.55] |
| Edwin | 4.216 / 4.389 | 6.05% / 6.16% | [−0.56, +0.34] |
| Hugh | 4.267 / 4.367 | 6.83% / 6.58% | [−0.35, +0.64] |
No capped-WER difference is significant. DEV-400 is held out but was used for model selection, so these are development numbers.
How we tested
- Texts: TEST-200, the 200 held-out WikiText-103 sentences on Paradee-8M v1.1's card (source), looked at once per release; DEV-400 for selection; XDEV-1600 for small WER gaps.
- WER: Whisper-base, also capped at 3 errors per sentence, with Whisper large-v3-turbo as a guard.
- Naturalness: UTMOS22-strong, a MOS predictor, not a listening test.
- Significance: paired per-sentence bootstrap; a difference counts only if its 95% CI excludes zero and it beats seed noise.
- Speed: one reserved i7-14700K P-core at 4.5 GHz, 1 thread, interleaved with the baseline, median of 3 rounds.
How it was made
A team of AI coding agents, run from T3 Code, built the models, the engine and the evaluation harness in about a week on one desktop GPU. An orchestrator wrote the brief ("Beat Paradee-8M v1.1 on all three axes at once: faster, smaller, and better") and spawned specialist agents; the human set goals, listened, and decided. Short prompts changed course:
- "The final model runs in the browser, on mobile and laptop." Speed was judged in the browser from then on.
- "Think beyond the runtime." onnxruntime-web's 21.6 MB WASM dwarfed a 3 MB model, so an agent wrote pdn.
- "Add more voices: four female, four male." Seven more Kokoro voices passed the agents' checks, but the human found them "not good enough", so the project switched teachers to Phonon-31m-en.
The rules: TEST was never used to choose, and every gain had to clear seed noise.
Files
| Path | What |
|---|---|
onnx/heart/, onnx/voices/ |
Heart's v2f_lite, v2f, v2g (int8 math; *_noint8 twins: fp32 math), and the nine Phonon voices |
web/ |
the browser bundle mesmer.tools serves: v2/ (Heart), pv/ (Phonon voices), voices/*.pdn, demo.html, say.mjs |
tts.py, voices.json |
Python inference; per-voice files, sha256, level, joins, quality |
samples/, assets/, benchmarks/ |
the clips above, the images, full benchmark tables and raw JSON |
Limitations
- English only: US-English misaki phonemes, the UK voices too.
- Below the teachers: Heart about 0.06 UTMOS under Kokoro, the Phonon voices 0.03 to 0.17 under Phonon. No human MOS study.
- Rare words (about 3.5% of DEV words) need eSpeak NG (GPL-3.0, not included).
- At most 512 phoneme ids (roughly 25 s) per call; longer sentences are split.
- Not bit-reproducible: the models draw their own noise.
- Speeds come from one desktop core; the iPhone figure was measured by hand.
Responsible use
Don't use these synthetic voices to impersonate real people or mislead listeners; disclose synthetic speech.
License and credits
Apache-2.0 (LICENSE, NOTICE) covers the models, the engine and all code here. Built on these Apache-2.0 projects:
- Kokoro-82M by hexgrad: Heart's teacher (
af_heart). - Paradee-8M v1.1 by Sahil Mahendrakar: Heart builds on it and its training code.
- Phonon-31m-en and phonon-voices by Gradium: the teachers of the other nine voices. Phonon builds on Kyutai's Pocket TTS and the Mimi codec. This repo contains no Phonon, Pocket TTS or Mimi weights, and we used only the Apache-2.0 31m model, not the CC BY-NC 40m models.
- misaki by hexgrad: the phonemizer, ported to JavaScript in
web/*/frontend/.
Built by mesmer.tools.