af_heart_arm_tts
A distilled single-voice, single-language text-to-speech model (af_heart,
English), built for Arm inference on NVIDIA GB10 (DGX Spark).
Trained, exported and served entirely by GPL-free code. No piper, no GPL dependency in any runtime path.
Code, benchmarks and the Arm CPU↔GPU cooperative pipeline: https://github.com/dlyog/arm-cooperative-tts
Why it is small
Kokoro-82M is general purpose — 11 voices, multiple languages. Measured from its own module tree, most of that capacity is unnecessary for one voice:
| Kokoro-82M component | params | needed for one voice? |
|---|---|---|
predictor (prosody) |
16.20 M | no |
decoder.decode (style conditioning) |
27.94 M | no — multi-voice machinery |
decoder.encode |
5.66 M | no |
bert + text_encoder |
11.90 M | ours is smaller |
decoder.generator (vocoder) |
19.69 M | yes — ours is 3.76 M |
Measurements
DGX Spark GB10, idle GPU, 20 held-out sentences, each engine in its own
process. Produced by dgx-spark/benchmark.py in the repo; raw output is
committed as dgx-spark/results.json.
| engine | RTF | latency | peak RSS | size |
|---|---|---|---|---|
| this model — Arm CPU | 0.03481 | 92.4 ms | 396 MB | 68.5 MB |
| this model — Arm CPU → GPU | 0.01596 | 43.3 ms | 1974 MB | 68.5 MB |
| Kokoro-82M — GPU | 0.01421 | 40.2 ms | 3239 MB | 326 MB |
| Kokoro-82M — Arm CPU | 0.34894 | 984.6 ms | 2775 MB | 326 MB |
On the Arm CPU: 10.0× faster than Kokoro, 7.0× less memory, 4.8× smaller. The cooperative path also runs in 1.6× less memory than Kokoro on the GPU.
Full detail, including word and character error rates, is in
EVAL.md. Audio you can listen to is in SAMPLES.md.
Cooperative split
| stage | time | share |
|---|---|---|
| Arm CPU — encoder, duration predictor, flow | 15.85 ms | 59.2% |
| handoff — unified memory | 0.1215 ms | 0.45% |
| Blackwell GPU — vocoder | 10.79 ms | 40.3% |
Stated plainly: the hybrid path is 0.89× Kokoro-GPU — slightly slower. Running a whole model on CUDA beats splitting it. The Arm result that stands alone is the CPU comparison.
Files
| file | size | needed for | what it is |
|---|---|---|---|
af_heart.onnx |
68.5 MB | CPU inference | the full model |
af_heart.onnx.json |
5 KB | everything | sample rate, phoneme map, inference scales |
encoder_prefix.onnx |
54.1 MB | hybrid | the Arm CPU half — encoder, duration predictor, flow |
af_heart_decoder.pt |
14.4 MB | hybrid | the GPU half — the 3.76M vocoder |
Everything here totals ~138 MB — smaller than the 327 MB Kokoro-82M checkpoint it is compared against.
encoder_prefix.onnx is a subgraph of af_heart.onnx, exported separately so
it can run on the Arm CPU while the vocoder runs on the GPU.
The 834 MB training checkpoint is deliberately not published. Inference
reads exactly one thing out of it — the vocoder — which ships on its own as
af_heart_decoder.pt. Distributing the checkpoint would have made this repo
~1 GB, three times the size of the baseline this model is meant to undercut,
to ship discriminator and optimizer state that inference never touches. To
train your own, use the pipeline in the code repository on your own audio.
Usage
git clone https://github.com/dlyog/arm-cooperative-tts
cd arm-cooperative-tts
bash run.sh setup --full # venv, deps, and these weights
bash run.sh demo
Direct use:
from tts import TTSModel
m = TTSModel("models/af_heart.onnx") # auto-tunes threads for asymmetric Arm
m.synthesize_to_wav("Hello there.", "out.wav")
Requires espeak-ng installed. It is invoked as a separate process, never
linked, which is what keeps its GPL terms from propagating.
Inference settings
Ships with noise_scale: 0.0, noise_w: 0.6 rather than piper's 0.667 / 0.8.
Chosen by sweeping both and scoring each with a Whisper ASR round-trip.
Training
- Data: 4,104 clips, 3.0000 hours, 24 kHz mono — synthesized by Kokoro-82M
(
af_heart), i.e. knowledge distillation - Architecture: VITS, 3.76 M vocoder (ResBlock1, 256 channels), MRD enabled
- Trainer: GPL-free, adapted from MIT sources — see the repo's
training/ - Hardware: NVIDIA DGX Spark (GB10), 374 epochs / 48,000 steps
Limitations
Quality is below Kokoro's. Because the training audio came from Kokoro, Kokoro is a hard ceiling — a student can approach its teacher, not exceed it.
WER 0.0362 / CER 0.0308 on the held-out set, against Kokoro's 0.0299 /
0.0321 measured the same way — character error rate is lower than the
teacher's, word error rate within 1.2×. The two sentences we miss are
single-word confusions (sugar → "shower", report → "record").
See EVAL.md.
WER measures only whether words are recognizable. It is blind to naturalness,
prosody and tonal artifacts. Listen to samples/.
English only, by design. Not validated beyond ~6.25 s per chunk — the longest clip in the training data — so longer text should be chunked by sentence, which the repo does automatically.
Citation
@software{chawdhury2026afheartarmtts,
author = {Chawdhury, Tarun Kumar},
title = {af\_heart\_arm\_tts: A Distilled Single-Voice TTS Model
for Arm Platforms},
year = {2026},
url = {https://huggingface.co/dlyog/af_heart_arm_tts},
organization = {DLYog Lab Research Services LLC}
}
Tarun Kumar Chawdhury, DLYog Lab Research Services LLC. af_heart_arm_tts: A Distilled Single-Voice TTS Model for Arm Platforms. 2026. https://huggingface.co/dlyog/af_heart_arm_tts
License and attribution
Copyright © 2026 Tarun Kumar Chawdhury, DLYog Lab Research Services LLC. Apache 2.0.
- Kokoro-82M (Apache-2.0) — teacher model for the training audio, and the comparison baseline. This work is not affiliated with or endorsed by it.
- Model definitions adapted from jaywalnut310/vits (MIT) and jik876/hifi-gan (MIT).
- espeak-ng (GPL-3.0) is invoked as a separate process only, never linked.