README.md · mesmertech/mesm-tts at main

mesm-tts / README.md

Model card: one player per row in the student vs. original table

aa59535 verified about 20 hours ago

19.8 kB

---
license: apache-2.0
language:
- en
pipeline_tag: text-to-speech
library_name: onnx
base_model:
- hexgrad/Kokoro-82M
- gradium/phonon-31m-en
- sahilmahendrakar/Paradee-8M-v1.1
tags:
- text-to-speech
- tts
- onnx
- webassembly
- wasm
- on-device
- browser
- cpu
- tiny
- distillation
- kokoro
- phonon
widget:
- text: "This is where the page began: one small voice, made right here in your browser, faster than you can blink."
  example_title: "Heart · US female"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/heart.mp3
- text: "No server is reading this to you. Every word is being made on your device, as you listen."
  example_title: "Wren · US female"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/wren.mp3
- text: "Take a breath and settle in. I can read you anything you like, as slowly as you like."
  example_title: "Everly · US female"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/everly.mp3
- text: "Your text never leaves this tab. It turns into speech right here, in a few milliseconds."
  example_title: "Hadley · US female"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hadley.mp3
- text: "Put the kettle on, find a comfy chair, and I'll read you the next chapter."
  example_title: "Bryony · UK female"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/bryony.mp3
- text: "Right, let's get straight to it. Tell me what to say, and I'll say it with a smile."
  example_title: "Maeve · UK female"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/maeve.mp3
- text: "Give me a headline, and I'll make it sound like the most important news of the day."
  example_title: "Miles · US male"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/miles.mp3
- text: "Hey, I'm running in your browser right now, and honestly? I'm having a great time."
  example_title: "Trevor · US male"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/trevor.mp3
- text: "Good evening. Shall we begin? I'll be narrating tonight's story, clearly and precisely."
  example_title: "Edwin · UK male"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/edwin.mp3
- text: "Lean in a little. I'll keep it gentle, and I'll keep it short."
  example_title: "Hugh · UK male"
  output:
    url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hugh.mp3
---
# mesm-tts: ten tiny English voices, 100 to 300 times faster than real time on one CPU core

![mesm-tts: waveforms of the ten voices; 290× faster than real time, 2.3 MB smallest voice, 10 ms to first sound](assets/banner.png)

Ten English text-to-speech voices, small enough to ship in a web page and fast enough to stream on one CPU core. No GPU, no
server, and no onnxruntime in the browser: a 40 kB WebAssembly engine runs them. Try all ten in the
[mesmer.tools TTS Generator](https://mesmer.tools/free-tools/tts-generator).

| | |
|---|---|
| **Size** | 2.3 to 6.9 MB per voice, vs. 325 MB (Kokoro-82M) and 55 to 103 MB (Phonon-31m-en) |
| **Native, 1 CPU core** | Heart **234x to 292x** real time (about 150x to 180x faster than Kokoro-82M); Phonon voices **82x to 106x** (about 30x faster than Phonon-31m-en) |
| **Browser, 1 CPU core** | Heart **149x to 201x**, Phonon voices **55x to 80x**; first audio after 10 to 55 ms |
| **Memory** | 100 to 117 MB peak for the whole process (Heart), vs. 770 to 849 MB with onnxruntime-web; the engine is 40 kB |
| **iPhone 13 Pro** | about **50x** real time (Heart), about **33x** (Phonon voices) |
| **Quality** | Heart: UTMOS 4.46, WER 5.6% (Kokoro-82M: 4.52, 5.7%). Every Phonon voice is statistically tied with its teacher on WER |

## Listen

Each voice reading its own line, as a browser tab plays it.

<table>
<tr><th>Voice</th><th>Sample</th></tr>
<tr><td><b>Heart</b> · US female · the default</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/heart.mp3"></audio></td></tr>
<tr><td><b>Wren</b> · US female · Phonon's built-in</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/wren.mp3"></audio></td></tr>
<tr><td><b>Everly</b> · US female · soft</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/everly.mp3"></audio></td></tr>
<tr><td><b>Hadley</b> · US female · calm</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hadley.mp3"></audio></td></tr>
<tr><td><b>Bryony</b> · UK female · warm</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/bryony.mp3"></audio></td></tr>
<tr><td><b>Maeve</b> · UK female · bright</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/maeve.mp3"></audio></td></tr>
<tr><td><b>Miles</b> · US male · resonant</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/miles.mp3"></audio></td></tr>
<tr><td><b>Trevor</b> · US male · lively</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/trevor.mp3"></audio></td></tr>
<tr><td><b>Edwin</b> · UK male · crisp</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/edwin.mp3"></audio></td></tr>
<tr><td><b>Hugh</b> · UK male · gentle</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hugh.mp3"></audio></td></tr>
</table>

**Student vs. original.** The same passage, read by each small model and by its teacher:

> *The lighthouse keeper climbed the stairs one last time. Below him, the harbor lights blinked awake, one by one, and the
> evening ferry sounded its horn. He had kept this light burning for thirty-one years. Tonight, for the first time, he wasn't
> sure he was ready to let it go.*

<table>
<tr><th>Voice</th><th>Read by</th><th>Sample</th></tr>
<tr><td rowspan="2"><b>Heart</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/heart.mp3"></audio></td></tr>
<tr><td>Kokoro-82M</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/heart_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Wren</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/wren.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/wren_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Everly</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/everly.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/everly_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Hadley</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/hadley.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/hadley_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Bryony</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/bryony.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/bryony_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Maeve</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/maeve.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/maeve_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Miles</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/miles.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/miles_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Trevor</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/trevor.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/trevor_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Edwin</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/edwin.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/edwin_original.mp3"></audio></td></tr>
<tr><td rowspan="2"><b>Hugh</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/hugh.mp3"></audio></td></tr>
<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/hugh_original.mp3"></audio></td></tr>
</table>

No model saw this passage in training, and each original is level-matched to its student. For reference:
[Paradee-8M v1.1 reading it](https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/heart_paradee_v1.1.mp3),
the model Heart started from.

## The voices

| Voice | Learned from | ONNX file | Native | Browser |
|---|---|---:|---:|---:|
| **Heart** lite (default) | Kokoro-82M `af_heart` | 2.64 MB | **274x** | **185x** |
| Heart v2f (best scores) | | 3.42 MB | 234x | 149x |
| Heart v2g (streams first) | | 2.32 MB | **292x** | **201x** |
| **Wren** | Phonon-31m-en, built-in voice | 5.34 MB | 105x | 80x |
| **Everly, Hadley, Bryony, Maeve, Miles, Trevor, Hugh** | Phonon-31m-en + phonon-voices | 5.34 MB | 104x to 106x | 80x |
| **Edwin** | | 6.50 MB | 82x | 55x |

x real time on 1 core: onnxruntime 1.31 natively; pdn relaxed SIMD in Chrome and Edge (Firefox and Safari: 70 to 85% of that).

## Quick start

### Python

```bash
pip install -r requirements.txt      # about 0.5 GB, no PyTorch
python tts.py "Hello there. How are you?" --voice hadley --out hello.wav    # --list for voices; Heart takes --model v2g
```

```python
from tts import MesmTTS              # downloads the voice's ONNX file on first use
tts = MesmTTS("miles")
audio = tts.synth("Good evening.")    # float32, 24 kHz
tts.save("Good evening.", "evening.wav")
```

### Browser, no install

```html
<script type="module">
import { loadParadee, relaxedSimdSupported } from "https://mesmer.tools/paradee/pv/pv.mjs";
const tts = await loadParadee({
  model: "https://huggingface.co/mesmertech/mesm-tts/resolve/main/web/voices/hadley.pdn",
  base: "https://mesmer.tools/paradee/v2/",
  wasm: `https://mesmer.tools/paradee/pv/engine/${relaxedSimdSupported() ? "pdn_r" : "pdn"}.wasm`,
  join: { gapScale: 0.496, leadKeepS: 0.02 },                 // this voice's "join" in voices.json
  espeakUrl: "https://mesmer.tools/paradee/espeak.mjs",       // eSpeak NG for unknown words (GPL)
});
const audio = await tts.synth("Your text never leaves this tab.");   // Float32Array, 24 kHz
for await (const chunk of tts.stream("Long text streams, sentence by sentence.")) { /* play chunk */ }
</script>
```

Heart uses the `v2` bundle: `loadParadee({ model: "v2f_lite" })` from `https://mesmer.tools/paradee/v2/paradee.mjs`. To
self-host, serve the repo root with `python -m http.server 8000` and open `http://localhost:8000/web/demo.html`. Bundlers and
workers: [`web/v2/README.md`](web/v2/README.md).

### Node, no dependencies

```bash
node web/say.mjs "Good evening. Shall we begin?" evening.wav edwin     # --list for voices
```

### Raw ONNX

Every file in `onnx/` has Paradee-8M v1.1's interface:

| name | type | |
|---|---|---|
| `input_ids` (in) | int64 `[1, T]` | `0`, phoneme ids (`tokenizer.json`), `0`; T ≤ 512 |
| `speed` (in) | float32 `[1]` | 1.0 = trained rate |
| `waveform` (out) | float32 `[1, samples]` | mono, 24 kHz |

```python
import numpy as np, onnxruntime as ort
s = ort.InferenceSession("onnx/voices/hadley.onnx", providers=["CPUExecutionProvider"])
wav = s.run(None, {"input_ids": ids, "speed": np.ones(1, np.float32)})[0][0]   # ids: np.int64 [1, T]
```

## Benchmarks

![File size against speed on one CPU core: Heart and the Phonon voices versus Kokoro-82M, Phonon-31m-en and Paradee-8M v1.1](assets/speed_vs_size.png)

Both images are drawn by `assets/make_images.py` from the repo's own samples and benchmark files; teachers' speeds are scaled
from interleaved runs. Full tables and protocols: [`benchmarks/README.md`](benchmarks/README.md).

### Native speed (onnxruntime 1.31, 1 thread)

| Model | x real time | vs. Paradee-8M v1.1 |
|---|---:|---:|
| Paradee-8M v1.1 int8 | 42.3 | 1.00 |
| **Heart v2g** | **291.5** | **6.89x** |
| **Heart v2f lite** | **273.8** | **6.47x** |
| Heart v2f | 234.1 | 5.53x |
| Phonon voices (Edwin) | 104.1 to 105.8 (82.0) | 2.46x to 2.50x (1.94x) |

Teachers, same core: Kokoro-82M (PyTorch) 1.04x real time, Phonon-31m-en (f32) 1.7x to 2.9x.

### Browser speed (1 thread, V8)

| Model | onnxruntime-web | pdn (relaxed SIMD) | First audio | Cold start |
|---|---:|---:|---:|---:|
| Paradee-8M v1.1 | 15.1x | | | 3.6 s |
| **Heart v2g** | 96.5x | **201.3x** | **10 ms** | **60 ms** |
| **Heart v2f lite** | 84.8x | **185.2x** | 20 ms | 75 ms |
| Heart v2f | 73.4x | 148.7x | 29 ms | 84 ms |
| Phonon voices (Edwin) | | 80.0x (54.7x) | 55 ms | 111 to 115 ms |

Peak process memory for Heart: 100 to 117 MB with pdn (38 to 40 kB brotli) and 770 to 849 MB with onnxruntime-web.

### Phones

| Device | Heart | Phonon voices | v1.1 on onnxruntime-web |
|---|---:|---:|---:|
| **iPhone 13 Pro**, Safari (live page, measured by hand) | **~50x** | **~33x** | |
| Desktop core slowed about 6x (low-end phone proxy) | 32x (v2g) | | 2.6x |

### Quality: Heart (TEST-200, held out, 3 seeds)

| | Paradee-8M v1.1 | Heart v2f | Heart v2f lite | Heart v2g | Kokoro-82M |
|---|---:|---:|---:|---:|---:|
| UTMOS ↑ | 4.391 | **4.462** | 4.453 | 4.453 | 4.520 |
| WER ↓ | 5.85% | 5.65% | 5.64% | **5.48%** | 5.67% |
| WER, capped ↓ | 5.17% | 4.91% | **4.84%** | 4.91% | |

Heart is significantly more natural than v1.1 (UTMOS +0.06 to +0.07) and statistically tied with v1.1 and Kokoro-82M on WER.

### Quality: Phonon voices (DEV-400, vs. their own teachers)

| Voice | UTMOS, student / teacher | WER, student / teacher | Capped WER, student − teacher, 95% CI (pp) |
|---|---:|---:|---:|
| Wren | 4.417 / 4.452 | 6.24% / 6.11% | [−0.26, +0.61] |
| Everly | 4.346 / 4.434 | 6.09% / 6.12% | [−0.54, +0.29] |
| Hadley | 4.469 / 4.496 | 5.98% / 6.10% | [−0.41, +0.51] |
| Bryony | 4.172 / 4.331 | 6.95% / 7.43% | [−0.78, +0.21] |
| Maeve | 4.391 / 4.442 | 6.31% / 6.20% | [−0.39, +0.59] |
| Miles | 4.258 / 4.365 | 6.17% / 6.45% | [−0.49, +0.36] |
| Trevor | 4.323 / 4.425 | 6.23% / 6.17% | [−0.30, +0.55] |
| Edwin | 4.216 / 4.389 | 6.05% / 6.16% | [−0.56, +0.34] |
| Hugh | 4.267 / 4.367 | 6.83% / 6.58% | [−0.35, +0.64] |

No capped-WER difference is significant. DEV-400 is held out but was used for model selection, so these are development numbers.

### How we tested

- **Texts:** TEST-200, the 200 held-out WikiText-103 sentences on Paradee-8M v1.1's card ([source](https://github.com/sahilmahendrakar/paradee)), looked at once per release; DEV-400 for selection; XDEV-1600 for small WER gaps.
- **WER:** Whisper-base, also capped at 3 errors per sentence, with Whisper large-v3-turbo as a guard.
- **Naturalness:** UTMOS22-strong, a MOS predictor, not a listening test.
- **Significance:** paired per-sentence bootstrap; a difference counts only if its 95% CI excludes zero and it beats seed noise.
- **Speed:** one reserved i7-14700K P-core at 4.5 GHz, 1 thread, interleaved with the baseline, median of 3 rounds.

## How it was made

A team of AI coding agents, run from [T3 Code](https://t3.codes), built the models, the engine and the evaluation harness in
about a week on one desktop GPU. An orchestrator wrote the brief (*"Beat Paradee-8M v1.1 on all three axes at once: faster,
smaller, and better"*) and spawned specialist agents; the human set goals, listened, and decided. Short prompts changed course:

- *"The final model runs in the browser, on mobile and laptop."* Speed was judged in the browser from then on.
- *"Think beyond the runtime."* onnxruntime-web's 21.6 MB WASM dwarfed a 3 MB model, so an agent wrote pdn.
- *"Add more voices: four female, four male."* Seven more Kokoro voices passed the agents' checks, but the human found them
  *"not good enough"*, so the project switched teachers to Phonon-31m-en.

The rules: TEST was never used to choose, and every gain had to clear seed noise.

## Files

| Path | What |
|---|---|
| `onnx/heart/`, `onnx/voices/` | Heart's `v2f_lite`, `v2f`, `v2g` (int8 math; `*_noint8` twins: fp32 math), and the nine Phonon voices |
| `web/` | the browser bundle mesmer.tools serves: `v2/` (Heart), `pv/` (Phonon voices), `voices/*.pdn`, `demo.html`, `say.mjs` |
| `tts.py`, `voices.json` | Python inference; per-voice files, sha256, level, joins, quality |
| `samples/`, `assets/`, `benchmarks/` | the clips above, the images, full benchmark tables and raw JSON |

## Limitations

- English only: US-English misaki phonemes, the UK voices too.
- Below the teachers: Heart about 0.06 UTMOS under Kokoro, the Phonon voices 0.03 to 0.17 under Phonon. No human MOS study.
- Rare words (about 3.5% of DEV words) need eSpeak NG (GPL-3.0, not included).
- At most 512 phoneme ids (roughly 25 s) per call; longer sentences are split.
- Not bit-reproducible: the models draw their own noise.
- Speeds come from one desktop core; the iPhone figure was measured by hand.

## Responsible use

Don't use these synthetic voices to impersonate real people or mislead listeners; disclose synthetic speech.

## License and credits

Apache-2.0 (`LICENSE`, `NOTICE`) covers the models, the engine and all code here. Built on these Apache-2.0 projects:

- [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad: Heart's teacher (`af_heart`).
- [Paradee-8M v1.1](https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.1) by Sahil Mahendrakar: Heart builds on it and its
  [training code](https://github.com/sahilmahendrakar/paradee).
- [Phonon-31m-en](https://huggingface.co/gradium/phonon-31m-en) and [phonon-voices](https://huggingface.co/gradium/phonon-voices)
  by Gradium: the teachers of the other nine voices. Phonon builds on Kyutai's [Pocket TTS](https://github.com/kyutai-labs/pocket-tts)
  and the Mimi codec. This repo contains no Phonon, Pocket TTS or Mimi weights, and we used only the Apache-2.0 31m model, not
  the CC BY-NC 40m models.
- [misaki](https://github.com/hexgrad/misaki) by hexgrad: the phonemizer, ported to JavaScript in `web/*/frontend/`.

Built by [mesmer.tools](https://mesmer.tools).

	---
	license: apache-2.0
	language:
	- en
	pipeline_tag: text-to-speech
	library_name: onnx
	base_model:
	- hexgrad/Kokoro-82M
	- gradium/phonon-31m-en
	- sahilmahendrakar/Paradee-8M-v1.1
	tags:
	- text-to-speech
	- tts
	- onnx
	- webassembly
	- wasm
	- on-device
	- browser
	- cpu
	- tiny
	- distillation
	- kokoro
	- phonon
	widget:
	- text: "This is where the page began: one small voice, made right here in your browser, faster than you can blink."
	example_title: "Heart · US female"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/heart.mp3
	- text: "No server is reading this to you. Every word is being made on your device, as you listen."
	example_title: "Wren · US female"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/wren.mp3
	- text: "Take a breath and settle in. I can read you anything you like, as slowly as you like."
	example_title: "Everly · US female"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/everly.mp3
	- text: "Your text never leaves this tab. It turns into speech right here, in a few milliseconds."
	example_title: "Hadley · US female"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hadley.mp3
	- text: "Put the kettle on, find a comfy chair, and I'll read you the next chapter."
	example_title: "Bryony · UK female"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/bryony.mp3
	- text: "Right, let's get straight to it. Tell me what to say, and I'll say it with a smile."
	example_title: "Maeve · UK female"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/maeve.mp3
	- text: "Give me a headline, and I'll make it sound like the most important news of the day."
	example_title: "Miles · US male"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/miles.mp3
	- text: "Hey, I'm running in your browser right now, and honestly? I'm having a great time."
	example_title: "Trevor · US male"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/trevor.mp3
	- text: "Good evening. Shall we begin? I'll be narrating tonight's story, clearly and precisely."
	example_title: "Edwin · UK male"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/edwin.mp3
	- text: "Lean in a little. I'll keep it gentle, and I'll keep it short."
	example_title: "Hugh · UK male"
	output:
	url: https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hugh.mp3
	---

	# mesm-tts: ten tiny English voices, 100 to 300 times faster than real time on one CPU core

	![mesm-tts: waveforms of the ten voices; 290× faster than real time, 2.3 MB smallest voice, 10 ms to first sound](assets/banner.png)

	Ten English text-to-speech voices, small enough to ship in a web page and fast enough to stream on one CPU core. No GPU, no
	server, and no onnxruntime in the browser: a 40 kB WebAssembly engine runs them. Try all ten in the
	[mesmer.tools TTS Generator](https://mesmer.tools/free-tools/tts-generator).

	\| \| \|
	\|---\|---\|
	\| Size \| 2.3 to 6.9 MB per voice, vs. 325 MB (Kokoro-82M) and 55 to 103 MB (Phonon-31m-en) \|
	\| Native, 1 CPU core \| Heart 234x to 292x real time (about 150x to 180x faster than Kokoro-82M); Phonon voices 82x to 106x (about 30x faster than Phonon-31m-en) \|
	\| Browser, 1 CPU core \| Heart 149x to 201x, Phonon voices 55x to 80x; first audio after 10 to 55 ms \|
	\| Memory \| 100 to 117 MB peak for the whole process (Heart), vs. 770 to 849 MB with onnxruntime-web; the engine is 40 kB \|
	\| iPhone 13 Pro \| about 50x real time (Heart), about 33x (Phonon voices) \|
	\| Quality \| Heart: UTMOS 4.46, WER 5.6% (Kokoro-82M: 4.52, 5.7%). Every Phonon voice is statistically tied with its teacher on WER \|

	## Listen

	Each voice reading its own line, as a browser tab plays it.

	<table>
	<tr><th>Voice</th><th>Sample</th></tr>
	<tr><td><b>Heart</b> · US female · the default</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/heart.mp3"></audio></td></tr>
	<tr><td><b>Wren</b> · US female · Phonon's built-in</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/wren.mp3"></audio></td></tr>
	<tr><td><b>Everly</b> · US female · soft</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/everly.mp3"></audio></td></tr>
	<tr><td><b>Hadley</b> · US female · calm</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hadley.mp3"></audio></td></tr>
	<tr><td><b>Bryony</b> · UK female · warm</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/bryony.mp3"></audio></td></tr>
	<tr><td><b>Maeve</b> · UK female · bright</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/maeve.mp3"></audio></td></tr>
	<tr><td><b>Miles</b> · US male · resonant</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/miles.mp3"></audio></td></tr>
	<tr><td><b>Trevor</b> · US male · lively</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/trevor.mp3"></audio></td></tr>
	<tr><td><b>Edwin</b> · UK male · crisp</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/edwin.mp3"></audio></td></tr>
	<tr><td><b>Hugh</b> · UK male · gentle</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/hugh.mp3"></audio></td></tr>
	</table>

	Student vs. original. The same passage, read by each small model and by its teacher:

	> *The lighthouse keeper climbed the stairs one last time. Below him, the harbor lights blinked awake, one by one, and the
	> evening ferry sounded its horn. He had kept this light burning for thirty-one years. Tonight, for the first time, he wasn't
	> sure he was ready to let it go.*

	<table>
	<tr><th>Voice</th><th>Read by</th><th>Sample</th></tr>
	<tr><td rowspan="2"><b>Heart</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/heart.mp3"></audio></td></tr>
	<tr><td>Kokoro-82M</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/heart_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Wren</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/wren.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/wren_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Everly</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/everly.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/everly_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Hadley</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/hadley.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/hadley_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Bryony</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/bryony.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/bryony_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Maeve</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/maeve.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/maeve_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Miles</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/miles.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/miles_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Trevor</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/trevor.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/trevor_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Edwin</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/edwin.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/edwin_original.mp3"></audio></td></tr>
	<tr><td rowspan="2"><b>Hugh</b></td><td>mesm-tts</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/hugh.mp3"></audio></td></tr>
	<tr><td>Phonon-31m-en</td><td><audio controls preload="none" src="https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/hugh_original.mp3"></audio></td></tr>
	</table>

	No model saw this passage in training, and each original is level-matched to its student. For reference:
	[Paradee-8M v1.1 reading it](https://huggingface.co/mesmertech/mesm-tts/resolve/main/samples/passage/heart_paradee_v1.1.mp3),
	the model Heart started from.

	## The voices

	\| Voice \| Learned from \| ONNX file \| Native \| Browser \|
	\|---\|---\|---:\|---:\|---:\|
	\| Heart lite (default) \| Kokoro-82M `af_heart` \| 2.64 MB \| 274x \| 185x \|
	\| Heart v2f (best scores) \| \| 3.42 MB \| 234x \| 149x \|
	\| Heart v2g (streams first) \| \| 2.32 MB \| 292x \| 201x \|
	\| Wren \| Phonon-31m-en, built-in voice \| 5.34 MB \| 105x \| 80x \|
	\| Everly, Hadley, Bryony, Maeve, Miles, Trevor, Hugh \| Phonon-31m-en + phonon-voices \| 5.34 MB \| 104x to 106x \| 80x \|
	\| Edwin \| \| 6.50 MB \| 82x \| 55x \|

	x real time on 1 core: onnxruntime 1.31 natively; pdn relaxed SIMD in Chrome and Edge (Firefox and Safari: 70 to 85% of that).

	## Quick start

	### Python

	```bash
	pip install -r requirements.txt # about 0.5 GB, no PyTorch
	python tts.py "Hello there. How are you?" --voice hadley --out hello.wav # --list for voices; Heart takes --model v2g
	```

	```python
	from tts import MesmTTS # downloads the voice's ONNX file on first use
	tts = MesmTTS("miles")
	audio = tts.synth("Good evening.") # float32, 24 kHz
	tts.save("Good evening.", "evening.wav")
	```

	### Browser, no install

	```html
	<script type="module">
	import { loadParadee, relaxedSimdSupported } from "https://mesmer.tools/paradee/pv/pv.mjs";
	const tts = await loadParadee({
	model: "https://huggingface.co/mesmertech/mesm-tts/resolve/main/web/voices/hadley.pdn",
	base: "https://mesmer.tools/paradee/v2/",
	wasm: `https://mesmer.tools/paradee/pv/engine/${relaxedSimdSupported() ? "pdn_r" : "pdn"}.wasm`,
	join: { gapScale: 0.496, leadKeepS: 0.02 }, // this voice's "join" in voices.json
	espeakUrl: "https://mesmer.tools/paradee/espeak.mjs", // eSpeak NG for unknown words (GPL)
	});
	const audio = await tts.synth("Your text never leaves this tab."); // Float32Array, 24 kHz
	for await (const chunk of tts.stream("Long text streams, sentence by sentence.")) { /* play chunk */ }
	</script>
	```

	Heart uses the `v2` bundle: `loadParadee({ model: "v2f_lite" })` from `https://mesmer.tools/paradee/v2/paradee.mjs`. To
	self-host, serve the repo root with `python -m http.server 8000` and open `http://localhost:8000/web/demo.html`. Bundlers and
	workers: [`web/v2/README.md`](web/v2/README.md).

	### Node, no dependencies

	```bash
	node web/say.mjs "Good evening. Shall we begin?" evening.wav edwin # --list for voices
	```

	### Raw ONNX

	Every file in `onnx/` has Paradee-8M v1.1's interface:

	\| name \| type \| \|
	\|---\|---\|---\|
	\| `input_ids` (in) \| int64 `[1, T]` \| `0`, phoneme ids (`tokenizer.json`), `0`; T ≤ 512 \|
	\| `speed` (in) \| float32 `[1]` \| 1.0 = trained rate \|
	\| `waveform` (out) \| float32 `[1, samples]` \| mono, 24 kHz \|

	```python
	import numpy as np, onnxruntime as ort
	s = ort.InferenceSession("onnx/voices/hadley.onnx", providers=["CPUExecutionProvider"])
	wav = s.run(None, {"input_ids": ids, "speed": np.ones(1, np.float32)})[0][0] # ids: np.int64 [1, T]
	```

	## Benchmarks

	![File size against speed on one CPU core: Heart and the Phonon voices versus Kokoro-82M, Phonon-31m-en and Paradee-8M v1.1](assets/speed_vs_size.png)

	Both images are drawn by `assets/make_images.py` from the repo's own samples and benchmark files; teachers' speeds are scaled
	from interleaved runs. Full tables and protocols: [`benchmarks/README.md`](benchmarks/README.md).

	### Native speed (onnxruntime 1.31, 1 thread)

	\| Model \| x real time \| vs. Paradee-8M v1.1 \|
	\|---\|---:\|---:\|
	\| Paradee-8M v1.1 int8 \| 42.3 \| 1.00 \|
	\| Heart v2g \| 291.5 \| 6.89x \|
	\| Heart v2f lite \| 273.8 \| 6.47x \|
	\| Heart v2f \| 234.1 \| 5.53x \|
	\| Phonon voices (Edwin) \| 104.1 to 105.8 (82.0) \| 2.46x to 2.50x (1.94x) \|

	Teachers, same core: Kokoro-82M (PyTorch) 1.04x real time, Phonon-31m-en (f32) 1.7x to 2.9x.

	### Browser speed (1 thread, V8)

	\| Model \| onnxruntime-web \| pdn (relaxed SIMD) \| First audio \| Cold start \|
	\|---\|---:\|---:\|---:\|---:\|
	\| Paradee-8M v1.1 \| 15.1x \| \| \| 3.6 s \|
	\| Heart v2g \| 96.5x \| 201.3x \| 10 ms \| 60 ms \|
	\| Heart v2f lite \| 84.8x \| 185.2x \| 20 ms \| 75 ms \|
	\| Heart v2f \| 73.4x \| 148.7x \| 29 ms \| 84 ms \|
	\| Phonon voices (Edwin) \| \| 80.0x (54.7x) \| 55 ms \| 111 to 115 ms \|

	Peak process memory for Heart: 100 to 117 MB with pdn (38 to 40 kB brotli) and 770 to 849 MB with onnxruntime-web.

	### Phones

	\| Device \| Heart \| Phonon voices \| v1.1 on onnxruntime-web \|
	\|---\|---:\|---:\|---:\|
	\| iPhone 13 Pro, Safari (live page, measured by hand) \| ~50x \| ~33x \| \|
	\| Desktop core slowed about 6x (low-end phone proxy) \| 32x (v2g) \| \| 2.6x \|

	### Quality: Heart (TEST-200, held out, 3 seeds)

	\| \| Paradee-8M v1.1 \| Heart v2f \| Heart v2f lite \| Heart v2g \| Kokoro-82M \|
	\|---\|---:\|---:\|---:\|---:\|---:\|
	\| UTMOS ↑ \| 4.391 \| 4.462 \| 4.453 \| 4.453 \| 4.520 \|
	\| WER ↓ \| 5.85% \| 5.65% \| 5.64% \| 5.48% \| 5.67% \|
	\| WER, capped ↓ \| 5.17% \| 4.91% \| 4.84% \| 4.91% \| \|

	Heart is significantly more natural than v1.1 (UTMOS +0.06 to +0.07) and statistically tied with v1.1 and Kokoro-82M on WER.

	### Quality: Phonon voices (DEV-400, vs. their own teachers)

	\| Voice \| UTMOS, student / teacher \| WER, student / teacher \| Capped WER, student − teacher, 95% CI (pp) \|
	\|---\|---:\|---:\|---:\|
	\| Wren \| 4.417 / 4.452 \| 6.24% / 6.11% \| [−0.26, +0.61] \|
	\| Everly \| 4.346 / 4.434 \| 6.09% / 6.12% \| [−0.54, +0.29] \|
	\| Hadley \| 4.469 / 4.496 \| 5.98% / 6.10% \| [−0.41, +0.51] \|
	\| Bryony \| 4.172 / 4.331 \| 6.95% / 7.43% \| [−0.78, +0.21] \|
	\| Maeve \| 4.391 / 4.442 \| 6.31% / 6.20% \| [−0.39, +0.59] \|
	\| Miles \| 4.258 / 4.365 \| 6.17% / 6.45% \| [−0.49, +0.36] \|
	\| Trevor \| 4.323 / 4.425 \| 6.23% / 6.17% \| [−0.30, +0.55] \|
	\| Edwin \| 4.216 / 4.389 \| 6.05% / 6.16% \| [−0.56, +0.34] \|
	\| Hugh \| 4.267 / 4.367 \| 6.83% / 6.58% \| [−0.35, +0.64] \|

	No capped-WER difference is significant. DEV-400 is held out but was used for model selection, so these are development numbers.

	### How we tested

	- Texts: TEST-200, the 200 held-out WikiText-103 sentences on Paradee-8M v1.1's card ([source](https://github.com/sahilmahendrakar/paradee)), looked at once per release; DEV-400 for selection; XDEV-1600 for small WER gaps.
	- WER: Whisper-base, also capped at 3 errors per sentence, with Whisper large-v3-turbo as a guard.
	- Naturalness: UTMOS22-strong, a MOS predictor, not a listening test.
	- Significance: paired per-sentence bootstrap; a difference counts only if its 95% CI excludes zero and it beats seed noise.
	- Speed: one reserved i7-14700K P-core at 4.5 GHz, 1 thread, interleaved with the baseline, median of 3 rounds.

	## How it was made

	A team of AI coding agents, run from [T3 Code](https://t3.codes), built the models, the engine and the evaluation harness in
	about a week on one desktop GPU. An orchestrator wrote the brief (*"Beat Paradee-8M v1.1 on all three axes at once: faster,
	smaller, and better"*) and spawned specialist agents; the human set goals, listened, and decided. Short prompts changed course:

	- "The final model runs in the browser, on mobile and laptop." Speed was judged in the browser from then on.
	- "Think beyond the runtime." onnxruntime-web's 21.6 MB WASM dwarfed a 3 MB model, so an agent wrote pdn.
	- "Add more voices: four female, four male." Seven more Kokoro voices passed the agents' checks, but the human found them
	"not good enough", so the project switched teachers to Phonon-31m-en.

	The rules: TEST was never used to choose, and every gain had to clear seed noise.

	## Files

	\| Path \| What \|
	\|---\|---\|
	\| `onnx/heart/`, `onnx/voices/` \| Heart's `v2f_lite`, `v2f`, `v2g` (int8 math; `*_noint8` twins: fp32 math), and the nine Phonon voices \|
	\| `web/` \| the browser bundle mesmer.tools serves: `v2/` (Heart), `pv/` (Phonon voices), `voices/*.pdn`, `demo.html`, `say.mjs` \|
	\| `tts.py`, `voices.json` \| Python inference; per-voice files, sha256, level, joins, quality \|
	\| `samples/`, `assets/`, `benchmarks/` \| the clips above, the images, full benchmark tables and raw JSON \|

	## Limitations

	- English only: US-English misaki phonemes, the UK voices too.
	- Below the teachers: Heart about 0.06 UTMOS under Kokoro, the Phonon voices 0.03 to 0.17 under Phonon. No human MOS study.
	- Rare words (about 3.5% of DEV words) need eSpeak NG (GPL-3.0, not included).
	- At most 512 phoneme ids (roughly 25 s) per call; longer sentences are split.
	- Not bit-reproducible: the models draw their own noise.
	- Speeds come from one desktop core; the iPhone figure was measured by hand.

	## Responsible use

	Don't use these synthetic voices to impersonate real people or mislead listeners; disclose synthetic speech.

	## License and credits

	Apache-2.0 (`LICENSE`, `NOTICE`) covers the models, the engine and all code here. Built on these Apache-2.0 projects:

	- [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M) by hexgrad: Heart's teacher (`af_heart`).
	- [Paradee-8M v1.1](https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.1) by Sahil Mahendrakar: Heart builds on it and its
	[training code](https://github.com/sahilmahendrakar/paradee).
	- [Phonon-31m-en](https://huggingface.co/gradium/phonon-31m-en) and [phonon-voices](https://huggingface.co/gradium/phonon-voices)
	by Gradium: the teachers of the other nine voices. Phonon builds on Kyutai's [Pocket TTS](https://github.com/kyutai-labs/pocket-tts)
	and the Mimi codec. This repo contains no Phonon, Pocket TTS or Mimi weights, and we used only the Apache-2.0 31m model, not
	the CC BY-NC 40m models.
	- [misaki](https://github.com/hexgrad/misaki) by hexgrad: the phonemizer, ported to JavaScript in `web/*/frontend/`.

	Built by [mesmer.tools](https://mesmer.tools).