DviTTS
Zero-shot text-to-speech for English and six Indian languages: Hindi, Marathi, Bengali, Punjabi, Gujarati and Odia. Give it 3–10 s of someone's speech and its transcript and it reads new text in that voice, or pick one of 18 named voices.
Base model: stage 1 starts from Fun-CosyVoice3-0.5B-2512 (FunAudioLLM / Alibaba, Apache 2.0), a 0.5B language model that already knows how to produce speech. We retrained it to emit IndiCodec ids and to speak six Indian languages.
The DviTTS family:
| Model | Size | Runs on | Voices | Get it |
|---|---|---|---|---|
| DviTTS (this repo) | 0.5B stage 1 + 130M flow + IndiCodec decoder | CUDA GPU (A10G: 0.37x real time) | clone from 3–10 s, or 18 presets | pip install (below) |
| DviTTS-Nano | 110M stage 1 + 1-step flow, ONNX | phone or laptop CPU, no PyTorch | 18 presets | Android app + code · demo |
More samples from both models: the demo page.
Samples
Prompts are speakers the model never saw in training, from LibriTTS-R test-clean and FLEURS. Each output is one take (seed 0), not picked from several. Made with the released package (pip install) on a laptop RTX 3050. Every output carries the AudioSeal watermark.
| Language | Prompt (unseen speaker) | DviTTS |
|---|---|---|
| English prompt: LibriTTS-R | It might rain this evening, so please take an umbrella when you leave the house. | |
| Hindi prompt: FLEURS | आज शाम बारिश हो सकती है, इसलिए घर से निकलते समय छाता साथ ले जाइए। | |
| Marathi prompt: FLEURS | आज संध्याकाळी पाऊस पडण्याची शक्यता आहे, त्यामुळे घरातून निघताना छत्री सोबत घ्या. | |
| Bengali prompt: FLEURS | আজ সন্ধ্যায় বৃষ্টি হতে পারে, তাই বাড়ি থেকে বেরোনোর সময় ছাতা সঙ্গে নিন। | |
| Punjabi prompt: FLEURS | ਅੱਜ ਸ਼ਾਮ ਨੂੰ ਮੀਂਹ ਪੈ ਸਕਦਾ ਹੈ, ਇਸ ਲਈ ਘਰੋਂ ਨਿਕਲਦੇ ਸਮੇਂ ਛਤਰੀ ਨਾਲ ਲੈ ਜਾਓ। | |
| Gujarati prompt: FLEURS | આજે સાંજે વરસાદ પડી શકે છે, તેથી ઘરેથી નીકળતી વખતે છત્રી સાથે લઈ જજો. | |
| Odia prompt: FLEURS | ଆଜି ସନ୍ଧ୍ୟାରେ ବର୍ଷା ହୋଇପାରେ, ତେଣୁ ଘରୁ ବାହାରିବା ବେଳେ ଛତା ସାଙ୍ଗରେ ନିଅନ୍ତୁ। | |
| Hindi prompt, English text | The meeting starts at 10:30 on Monday, so please be on time. |
Install
python -m venv dvitts-env && source dvitts-env/bin/activate # a fresh environment: the package pins its torch range
pip install "git+https://github.com/codeprakhar25/dvitts@v1.0.0"
Python 3.10+ and a CUDA GPU; installs PyTorch 2.6–2.8 (RTX 50-series needs 2.7+; not yet tested there). Weights download from this repo on first use; the codec comes from IndiCodec.
Windows: PyPI's Windows torch is CPU-only, so install the CUDA build first, then the package:
pip install torch==2.8.0 torchaudio==2.8.0 --index-url https://download.pytorch.org/whl/cu128.
Use
dvitts "नमस्ते, आप कैसे हैं?" --voice ananya -o hello.wav
dvitts "Please call me back after the meeting." --prompt me.wav --prompt-text "exact words spoken in me.wav" -o out.wav
dvitts --list-voices
import soundfile as sf
from dvitts import DviTTS
tts = DviTTS.from_pretrained("prakharkhatri/dvitts")
y, sr = tts.say("ananya", "कल सुबह हम स्टेशन पर मिलेंगे।") # a preset voice
sf.write("hello.wav", y, sr)
w, wsr = sf.read("me.wav", dtype="float32") # clone: 3-10 s of clean speech
v = tts.voice(w, wsr, "exact words spoken in me.wav")
y, sr = tts.say(v, "The meeting starts at 10:30, so please be on time.")
y, sr = tts.say("sneha", "आज संध्याकाळी पाऊस पडण्याची शक्यता आहे.", lang="mr") # Devanagari reads as Hindi unless lang="mr"
for chunk, sr, info in tts.say_stream(v, "First sentence. Second sentence. Third sentence."):
play(chunk, sr) # sentence by sentence: first audio early
Named voices (labels, not the speakers' names):
| Language | Female | Male |
|---|---|---|
| English | emma, grace | james, oliver |
| Hindi | ananya, kavya | arjun, rohan |
| Marathi | sneha | aditya |
| Bengali | ishita | arnab |
| Punjabi | jasleen | gurpreet |
| Gujarati | hetal | kunal |
| Odia | subhashree | debasis |
Languages
FLEURS, 100 sentences per language, unseen speakers as prompts (prompts cut to 10 s), CER in % by IndicConformer on our audio and on the real recording of the same sentence.
| Language | CER DviTTS | CER real recording |
|---|---|---|
| Hindi | 2.3 | 3.8 |
| Marathi | 2.6 | 2.8 |
| Bengali | 2.7 | 3.8 |
| Punjabi | 5.2 | 6.2 |
| Gujarati | 3.8 | 3.6 |
| Odia | 3.1 | 4.0 |
Tamil, Telugu, Kannada and Malayalam are to be added.
Benchmarks (unseen speakers only)
Whisper large-v3 for English, IndicConformer for Hindi; prompt transcripts come from ASR of the prompt (a user supplies only audio); seed per item; watermark on; AWS A10G.
| Set | n | CER % | WER % |
|---|---|---|---|
| Seed-TTS test-en | 1,088 | 0.82 | 2.04 |
| CMU Arctic | 350 | 0.36 | 1.45 |
| LibriTTS-R test | 200 | 0.86 | 3.07 |
| IndicVoices Hindi | 300 | 3.53 | 8.92 |
| FLEURS Hindi | 100 | 4.52 | 9.01 |
FLEURS Hindi here is a different 100-sentence sample and prompt pairing from the language table above. Dead air of 1.5 s or more: 3 of 2,038 clips (all IndicVoices Hindi); clips that never ended: 0.
LibriSpeech-PC test-clean (F5-TTS protocol)
The F5-TTS cross-sentence list (1,127 pairs, 4–10 s targets), the protocol used by F5-TTS, ZipVoice and OmniVoice. Same audio scored both ways; "real" is the real target recording through the same ASR.
| WER % (HuBERT-large) | WER % (Whisper large-v3) | |
|---|---|---|
| DviTTS | 2.05 | 2.12 |
| Real recording | 1.89 | 2.37 |
MiniMax multilingual test set, Hindi (voice cloning)
MiniMaxAI/TTS-Multilingual-Test-Set Hindi: 100 lines, 2 Common Voice prompt speakers, Whisper large-v3
(language="hi"). How the text is normalised changes the number a lot, so three scorings of the same audio are given.
| Scoring | WER % | CER % |
|---|---|---|
| Whole words | 16.82 | 5.90 |
| Whisper's stock text normaliser (removes vowel signs, as most papers' scripts do) | 8.67 | — |
| IndicConformer instead of Whisper | 3.10 | 1.18 |
Published Whisper-large-v3 numbers on this set: OmniVoice 4.33, ElevenLabs Multilingual v2 5.83, MiniMax-Speech 6.96. Papers rarely say which Hindi normaliser they used, so compare with care. 9 of the 100 lines contain digits, which the front end reads as words and Whisper often writes back as digits.
Speed and memory
A10G, bf16 stage 1 on a CUDA graph, 16-step flow, trim and watermark on:
| Real-time factor (Seed-TTS) | 0.37 |
| First audio, one sentence | 1.54 s English, 2.73 s Hindi (say_stream starts after the first sentence) |
| GPU memory, speaking | 2.2 GB allocated, ~3.2 GB in nvidia-smi |
| GPU memory, cloning a new voice | peak 4.0 GB for ~0.6 s (the prompt encoder visits the GPU, then goes back to CPU RAM) |
A 6 GB GPU runs presets; cloning is tight. CPU (device="cpu") works but is slow: use DviTTS-Nano on CPUs.
Defaults
| Step | Default | Change it | Why |
|---|---|---|---|
| Text normalisation | on | normalize=False |
digits are read as digits otherwise |
| Long text | split at sentence ends (over 500 English / 400 Indian characters, or ~25 s of speech) | say_long(..., gap_s=) |
one long call drops its opening and loops |
| Prompt over 10 s | cut at the last pause before 10 s | pass 10 s or less | the model sees 10 s of prompt |
| Prompt under 3 s | warning | — | 2–3 s prompts raise English word error by 6–8 points |
| Edge trim | on | DviTTS(..., trim=False) |
removes silence at the clip edges |
Watermark
Every output carries an inaudible AudioSeal mark. Check a clip with
dvitts --detect clip.wav. If you deploy this model, label its output as AI-generated where your local rules require it.
Limitations
- No emotion or style control.
- Five of the six Indian languages learned from fewer speakers; voices in those languages may clone less well.
- Noisy prompts can give a take full of loud noise (rare). Use clean speech.
- English number reading uses simple rules unless
pip install "dvitts[en-norm]"(NeMo text processing, Linux).
Intended use and ethical considerations
For research and for building speech products with voices you have permission to use. Not for impersonation, fraud, or cloning anyone's voice without their consent.
The preset voices are speakers from Rasa and IndicVoices-R (AI4Bharat) and LibriTTS-R, all CC BY 4.0, credited in Acknowledgements. Display names (Ananya, Emma, …) are labels, not the speakers' names.
Do not use these voices to imitate the real speakers, or to make anyone believe they are hearing them. Clone only voices you have permission to use. Audio from this model carries an AudioSeal watermark by default.
Citation
If you use DviTTS, please cite:
@misc{khatri2026dvitts,
title = {DviTTS: Text-to-Speech for English and Six Indian Languages},
author = {Khatri, Prakhar},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/prakharkhatri/dvitts}}
}
DviTTS builds on CosyVoice 3 (its base model) and AudioSeal (its watermark); please cite them too:
@article{du2025cosyvoice,
title = {CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training},
author = {Du, Zhihao and Gao, Changfeng and Wang, Yuxuan and Yu, Fan and Zhao, Tianyu and Wang, Hao and Lv, Xiang and Wang, Hui and Shi, Xian and An, Keyu and others},
journal = {arXiv preprint arXiv:2505.17589},
year = {2025}
}
@article{sanroman2024proactive,
title = {Proactive Detection of Voice Cloning with Localized Watermarking},
author = {San Roman, Robin and Fernandez, Pierre and Elsahar, Hady and D{\'e}fossez, Alexandre and Furon, Teddy and Tran, Tuan},
journal = {ICML},
year = {2024}
}
Acknowledgements
- Base model: Fun-CosyVoice3-0.5B-2512 (FunAudioLLM, Alibaba; Apache 2.0) and its Qwen2 text tokenizer.
- Watermark: AudioSeal (Meta; MIT). Silence trim: Silero VAD (MIT).
- Text front end: num2words (LGPL-2.1, used unmodified) and indic-numtowords (AI4Bharat; MIT); optional NeMo text processing (NVIDIA; Apache 2.0) for English.
- Training data (CC BY 4.0): Emilia-YODAS English subset (Amphion), LibriTTS and LibriTTS-R, VCTK, and from AI4Bharat Rasa, IndicVoices-R and IndicVoices (training split).
- Preset voices: speakers from Rasa, IndicVoices-R and LibriTTS-R (CC BY 4.0). They belong to those speakers and must not be used to impersonate them.
- Samples on this card: prompts from FLEURS (Google; CC BY 4.0) and LibriTTS-R test-clean, speakers not seen in training.
Licence
Apache 2.0 (LICENSE).
- Downloads last month
- 30