DviTTS-Nano

Text-to-speech for English and six Indian languages that runs on a phone or a laptop CPU, with no GPU and no PyTorch. One folder of ONNX files plus a short reference runner; the same files serve Android, iOS, desktop and servers through ONNX Runtime.

Base model: the 0.5B language model at the start of DviTTS is Fun-CosyVoice3-0.5B-2512 (FunAudioLLM / Alibaba, Apache 2.0), a model that already knows how to produce speech. We retrained it to emit IndiCodec ids and to speak six Indian languages.

This is the small sibling of DviTTS: a 110M stage 1 distilled from that 0.5B model, a one-step stage 2, and a quantized decoder. It speaks in 18 named preset voices. Voice cloning from your own recording needs the full model.

The DviTTS family:

Model Size Runs on Voices Get it
DviTTS 0.5B stage 1 + 130M flow + IndiCodec decoder CUDA GPU (A10G: 0.37x real time) clone from 3–10 s, or 18 presets pip install
DviTTS-Nano (this repo) 110M stage 1 + 1-step flow, ONNX phone or laptop CPU, no PyTorch 18 presets Android app + code · demo

More samples from both models: the demo page.

Samples

Preset voices, one take each (seed 0), not picked from several. Made with the Python runner on a laptop CPU (ONNX Runtime path). Every output carries the AudioSeal watermark.

VoiceLanguageDviTTS-Nano
emmaEnglish
It might rain this evening, so please take an umbrella when you leave the house.
jamesEnglish
The meeting starts at 10:30 on Monday, so please be on time.
ananyaHindi
आज शाम बारिश हो सकती है, इसलिए घर से निकलते समय छाता साथ ले जाइए।
arjunHindi
आज शाम बारिश हो सकती है, इसलिए घर से निकलते समय छाता साथ ले जाइए।
snehaMarathi
आज संध्याकाळी पाऊस पडण्याची शक्यता आहे, त्यामुळे घरातून निघताना छत्री सोबत घ्या.
arnabBengali
আজ সন্ধ্যায় বৃষ্টি হতে পারে, তাই বাড়ি থেকে বেরোনোর সময় ছাতা সঙ্গে নিন।
jasleenPunjabi
ਅੱਜ ਸ਼ਾਮ ਨੂੰ ਮੀਂਹ ਪੈ ਸਕਦਾ ਹੈ, ਇਸ ਲਈ ਘਰੋਂ ਨਿਕਲਦੇ ਸਮੇਂ ਛਤਰੀ ਨਾਲ ਲੈ ਜਾਓ।
kunalGujarati
આજે સાંજે વરસાદ પડી શકે છે, તેથી ઘરેથી નીકળતી વખતે છત્રી સાથે લઈ જજો.
subhashreeOdia
ଆଜି ସନ୍ଧ୍ୟାରେ ବର୍ଷା ହୋଇପାରେ, ତେଣୁ ଘରୁ ବାହାରିବା ବେଳେ ଛତା ସାଙ୍ଗରେ ନିଅନ୍ତୁ।

Quick start (Python, any CPU)

Python 3.11 or newer (ONNX Runtime 1.32 has no wheels for Python 3.10).

pip install onnxruntime==1.32.0 tokenizers numpy num2words indic-numtowords
hf download prakharkhatri/dvitts-nano --local-dir dvitts-nano
python dvitts-nano/dvitts_onnx.py dvitts-nano "नमस्ते, आप कैसे हैं?" --voice ananya --out hello.wav
import sys; sys.path.insert(0, "dvitts-nano")
from dvitts_onnx import DviTTS

tts = DviTTS("dvitts-nano", threads=4)
print(tts.voices())                                   # 18 named presets; voices/index.json has language and source
y, sr = tts.say("The meeting starts at 10:30, so please be on time.", voice="emma")

Laptops and desktops (x86): optional speed-up

ONNX Runtime is the only required runtime. On x86 Windows and Linux, stage 1 can also run on llama.cpp from s1_q8_0.gguf (the same stage 1 in llama.cpp's 8-bit format), about 1.3x faster. The runner picks this path by itself when llama-cpp-python is installed; without it, it prints one warning and keeps working on ONNX Runtime.

pip install --no-deps -r dvitts-nano/requirements-laptop.txt   # maintainer's prebuilt CPU wheel, sha256-pinned
pip install typing-extensions diskcache jinja2

PyPI carries only the source of llama-cpp-python (a plain pip install needs a C++ compiler), so requirements-laptop.txt points at the project's own prebuilt wheels for v0.3.36 (Windows x64, Linux x64) and pins their sha256; pip refuses any other file.

ARM machines (Apple Silicon Macs, Windows on ARM, phones) stay on the ONNX path: the 8-bit graphs that are slow on x86 run fast on ARM, and the runner only switches to llama.cpp by itself on x86. llama.cpp on an Apple Silicon Mac is opt-in and not yet measured: build it with CPU repack off (on newer ARM chips it gave wrong output in our tests), CMAKE_ARGS="-DGGML_CPU_REPACK=OFF" pip install llama-cpp-python==0.3.36 --no-binary llama-cpp-python --no-cache-dir, then pass DviTTS(..., s1_gguf="s1_q8_0.gguf").

Phone-only use does not need the laptop files: hf download prakharkhatri/dvitts-nano --local-dir dvitts-nano --exclude "*.gguf" --exclude "wm_fp32.onnx".

Android

A Kotlin library (say(text, voice), ONNX Runtime for Android, model download with md5 check, playback) and a demo app are on GitHub, with the demo APK on its releases page. The Kotlin code is a line-by-line port of dvitts_onnx.py: on the same random draws it gives the same token ids, the same speech ids and the same audio (checked on 36,214 number words, 2,273 front-end texts, 2,277 tokenizer cases and 7 whole calls in four languages, two of them with the gap retry firing).

Benchmarks (unseen speakers only)

The phone build (these ONNX files and this runner, watermark and trim on) on AWS x86 CPUs, 4 threads, seed per item. Same pairs and scorers as the DviTTS card: Whisper large-v3 for English (WER), IndicConformer for Indian languages (CER). Each clip's error is capped at 100 % before averaging. The full model's numbers are shown for comparison. Nano does not clone from a recording: for these benchmarks each unseen prompt was pre-encoded with the IndiCodec encoder (part of DviTTS, not in this folder), the same way the presets were made.

Set n On-device DviTTS (GPU)
Seed-TTS test-en, WER 1,088 2.65 2.04
CMU Arctic, WER 350 2.74 1.45
LibriTTS-R test, WER 200 3.63 3.07
IndicVoices Hindi, CER 300 3.52 3.53
FLEURS Hindi, CER 100 5.16 4.52

Other Indian languages, FLEURS 100 per language, CER, prompts cut to 10 s (the same job as the DviTTS language table).

Language n On-device DviTTS (GPU)
Marathi 100 3.70 2.57
Gujarati 100 4.54 3.76
Bengali 100 3.54 2.69
Punjabi 100 5.54 5.23
Odia 100 3.75 3.11

Lines of 180+ characters (236 across 6 languages, chunked at 120 characters): mean CER 7.27 vs 6.98 for DviTTS. Punjabi long lines are the weak spot (20.7 %). Clips that never ended: 0. A take with a 2 s silence inside is made once more and the shorter gap kept; this fired on 5 of 100 Punjabi FLEURS lines and on no other language.

Speed

Device Threads Real-time factor Peak memory
Phone, Snapdragon 8 Gen 3 (8-core CPU) 4 0.61 877 MB
Mac, Apple M1 Pro 4 0.27 973 MB
Laptop, Ryzen 7 (x86), llama.cpp stage 1* 8 0.63 770 MB

Real-time factor is compute time per second of audio, watermark on; under 1.0 is faster than real time. * Without llama.cpp (ONNX Runtime only) the same laptop runs at 0.81.

Watermark

Every output carries an inaudible AudioSeal mark, the same as DviTTS; check a clip with dvitts --detect clip.wav (see the DviTTS card). If you deploy this model, label its output as AI-generated where your local rules require it.

Limitations

  • Preset voices only; no cloning in this folder (use DviTTS for cloning).
  • Cross-language (English text with an Indian preset, or the reverse) works, but sounds less like the picked voice.
  • Indian languages other than Hindi learned from fewer speakers; Punjabi is the weakest.
  • English text uses rule-based number reading (no NeMo), so unusual formats (times with am/pm, units) may be read plainly.
  • No emotion or style control.
  • No streaming; on a phone the first audio of one sentence takes about 3 s.

Intended use and ethical considerations

For research and building speech products. Not for impersonation, fraud, or presenting synthetic speech as a real person.

Preset voices are speakers from three public datasets (Rasa, IndicVoices-R, LibriTTS-R; CC BY 4.0), credited in Acknowledgements and voices/index.json. Display names (Ananya, Emma, …) are labels, not the speakers' names.

Do not use these voices to imitate the real speakers, or to make anyone believe they are hearing them. Audio from this model carries an AudioSeal watermark by default.

Reproducibility

config.json holds every rule and constant. manifest.json holds the md5 of every file. Test lists, seeds and scorers are as in the DviTTS card.

Citation

If you use DviTTS-Nano, please cite:

@misc{khatri2026dvittsnano,
  title        = {DviTTS-Nano: On-Device Text-to-Speech for English and Six Indian Languages},
  author       = {Khatri, Prakhar},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/prakharkhatri/dvitts-nano}}
}

DviTTS-Nano builds on CosyVoice 3 (its base model) and AudioSeal (its watermark); please cite them too:

@article{du2025cosyvoice,
  title   = {CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training},
  author  = {Du, Zhihao and Gao, Changfeng and Wang, Yuxuan and Yu, Fan and Zhao, Tianyu and Wang, Hao and Lv, Xiang and Wang, Hui and Shi, Xian and An, Keyu and others},
  journal = {arXiv preprint arXiv:2505.17589},
  year    = {2025}
}

@article{sanroman2024proactive,
  title   = {Proactive Detection of Voice Cloning with Localized Watermarking},
  author  = {San Roman, Robin and Fernandez, Pierre and Elsahar, Hady and D{\'e}fossez, Alexandre and Furon, Teddy and Tran, Tuan},
  journal = {ICML},
  year    = {2024}
}

Acknowledgements

  • Base model: Fun-CosyVoice3-0.5B-2512 (FunAudioLLM, Alibaba; Apache 2.0) and its Qwen2 text tokenizer.
  • Watermark: AudioSeal (Meta; MIT). Silence trim: Silero VAD (MIT).
  • Text front end: num2words (LGPL-2.1, used unmodified) and indic-numtowords (AI4Bharat; MIT).
  • Training data (CC BY 4.0): Emilia-YODAS English subset (Amphion), LibriTTS and LibriTTS-R, VCTK, and from AI4Bharat Rasa, IndicVoices-R and IndicVoices (training split).
  • Preset voices: speakers from Rasa, IndicVoices-R and LibriTTS-R (CC BY 4.0). They belong to those speakers and must not be used to impersonate them.
  • Files in the folder from MIT projects (AudioSeal, Silero VAD, indic-numtowords) ship with their licence texts in THIRD_PARTY_LICENSES.

Licence

Apache 2.0 (LICENSE).

Downloads last month
33
GGUF
Model size
0.1B params
Architecture
qwen2
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for prakharkhatri/dvitts-nano

Quantized
(1)
this model

Space using prakharkhatri/dvitts-nano 1

Collection including prakharkhatri/dvitts-nano

Paper for prakharkhatri/dvitts-nano