--- license: apache-2.0 pipeline_tag: text-to-speech tags: [text-to-speech, tts, indic, voice-cloning, multilingual] language: [kn, hi, ta, te, ml, mr, bn, gu, pa, or, as, ne, ur, sa, ks, sd, kok, mai, mni, brx, doi, sat, bho, mag, hne] --- # Volt-TTS Volt-TTS is a text-to-speech model for 25 Indian languages, finetuned from [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) by the Qwen team. It speaks in two built-in voices, or clones any voice from a short reference clip. This is a preview release. ## Highlights - **25 Indian languages** in one model, chosen per line. - **Two built-in voices**: Arjun (male) and Meera (female), both speaking every language. - **Voice cloning** from a 5-15 s clip, with or without its transcript, in any language. ## Samples Built-in voices (Arjun, Meera) with default settings. | Voice | Language | Text | Audio | |---|---|---|---| | Arjun | Kannada | ಬೆಂಗಳೂರು ಕರ್ನಾಟಕದ ರಾಜಧಾನಿ. ಇಲ್ಲಿನ ಹವಾಮಾನ ವರ್ಷವಿಡೀ ಹಿತವಾಗಿರುತ್ತದೆ. ಉದ್ಯಾನಗಳು, ಕೆರೆಗಳು ಮತ್ತು ತಂತ್ರಜ್ಞಾನ ಕಂಪನಿಗಳಿಂದ ಈ ನಗರ ದೇಶದಾದ್ಯಂತ ಪ್ರಸಿದ್ಧವಾಗಿದೆ. | | | Meera | Hindi | भारत में हर कुछ किलोमीटर पर भाषा और खाना बदल जाता है। यही विविधता इस देश को खास बनाती है, और हर यात्रा में कुछ नया सीखने को मिलता है। | | | Meera | Tamil | சென்னை மெரினா கடற்கரை உலகின் நீளமான கடற்கரைகளில் ஒன்று. மாலை நேரத்தில் குடும்பங்கள் இங்கே வந்து காற்று வாங்கி, சுண்டல் சாப்பிட்டு மகிழ்கிறார்கள். | | | Arjun | Telugu | హైదరాబాద్ బిర్యానీ రుచికి ప్రపంచమంతా అభిమానులు ఉన్నారు. చార్మినార్ దగ్గర సాయంత్రం నడుస్తూ, వేడి వేడి ఇరానీ చాయ్ తాగడం ఒక మంచి అనుభవం. | | | Meera | Bengali | কলকাতার দুর্গাপূজা শুধু একটি উৎসব নয়, এটি শহরের প্রাণ। প্রতিটি পাড়ায় সুন্দর প্যান্ডেল সাজানো হয়, আর রাতভর মানুষ ঠাকুর দেখতে বের হয়। | | | Arjun | Malayalam | കേരളത്തിലെ കായലുകളിലൂടെയുള്ള ഹൗസ്ബോട്ട് യാത്ര വളരെ മനോഹരമാണ്. പച്ചപ്പ് നിറഞ്ഞ തീരങ്ങളും ശാന്തമായ വെള്ളവും മനസ്സിന് വലിയ സമാധാനം നൽകുന്നു. | | | Meera | Marathi | पुण्यातला पाऊस काही वेगळाच असतो. डोंगरावरून येणारे ढग, गरम चहा आणि भजी, यामुळे पावसाळा हा सगळ्यांचा आवडता ऋतू आहे. | | ## Model details | | | |---|---| | Parameters | 2.02B (the 1.93B base plus 93M for the new Indic token embeddings) | | Base model | [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) | | Languages | 25 Indian languages | | Speech codec | 12.5 Hz (from the base model) | | Precision | bf16 | | License | Apache-2.0 | ## Languages Pass the value in the `language=` column to `generate_voice_clone`. | Language | `language=` | Language | `language=` | |---|---|---|---| | Assamese | `"assamese"` | Malayalam | `"malayalam"` | | Bengali | `"bengali"` | Manipuri | `"manipuri"` | | Bhojpuri | `"bhojpuri"` | Marathi | `"marathi"` | | Bodo | `"bodo"` | Nepali | `"nepali"` | | Chhattisgarhi | `"chhattisgarhi"` | Odia | `"odia"` | | Dogri | `"dogri"` | Punjabi | `"punjabi"` | | Gujarati | `"gujarati"` | Sanskrit | `"sanskrit"` | | Hindi | `"hindi"` | Santali | `"santali"` | | Kannada | `"kannada"` | Sindhi | `"sindhi"` | | Kashmiri | `"kashmiri"` | Tamil | `"tamil"` | | Konkani | `"konkani"` | Telugu | `"telugu"` | | Magahi | `"magahi"` | Urdu | `"urdu"` | | Maithili | `"maithili"` | | | ## Quickstart Needs a CUDA GPU with about 8 GB of memory. Warnings about SoX and the tokenizer class can be ignored. ```bash pip install -U qwen-tts soundfile ``` **Built-in voices** ```python import soundfile as sf import torch from huggingface_hub import hf_hub_download from safetensors import safe_open from qwen_tts import Qwen3TTSModel from qwen_tts.inference.qwen3_tts_model import VoiceClonePromptItem tts = Qwen3TTSModel.from_pretrained("openarcane/Volt-TTS", device_map="cuda:0", dtype=torch.bfloat16) def speaker(name): """A built-in voice: "arjun" (male) or "meera" (female).""" path = hf_hub_download("openarcane/Volt-TTS", f"speakers/{name}.safetensors") with safe_open(path, "pt") as f: return [VoiceClonePromptItem(ref_code=f.get_tensor("ref_code"), ref_spk_embedding=f.get_tensor("ref_spk_embedding"), x_vector_only_mode=False, icl_mode=True, ref_text=f.metadata()["ref_text"])] wavs, sr = tts.generate_voice_clone( text=("ಬೆಂಗಳೂರು ಕರ್ನಾಟಕದ ರಾಜಧಾನಿ. ಇಲ್ಲಿನ ಹವಾಮಾನ ವರ್ಷವಿಡೀ ಹಿತವಾಗಿರುತ್ತದೆ. " "ಉದ್ಯಾನಗಳು, ಕೆರೆಗಳು ಮತ್ತು ತಂತ್ರಜ್ಞಾನ ಕಂಪನಿಗಳಿಂದ ಈ ನಗರ ದೇಶದಾದ್ಯಂತ ಪ್ರಸಿದ್ಧವಾಗಿದೆ."), language="kannada", voice_clone_prompt=speaker("arjun")) sf.write("arjun_kannada.wav", wavs[0], sr) wavs, sr = tts.generate_voice_clone( text="आज मौसम बहुत अच्छा है। सुबह हल्की बारिश हुई थी, इसलिए हवा में ठंडक है। चलो, शाम को पार्क में टहलने चलते हैं।", language="hindi", voice_clone_prompt=speaker("meera")) sf.write("meera_hindi.wav", wavs[0], sr) ``` **Voice cloning.** Give a 5-15 s clip of the voice and what is said in it. The clip can be in any language. ```python wavs, sr = tts.generate_voice_clone( text=("ఈ రోజు వాతావరణం చాలా బాగుంది. ఉదయం కొంచెం వర్షం పడింది, అందుకే గాలి చల్లగా ఉంది. " "సాయంత్రం పార్కుకు నడకకు వెళ్దాం."), language="telugu", ref_audio="my_voice.wav", # path, URL or base64 ref_text="Transcript of my_voice.wav.", # no transcript? drop it and pass x_vector_only_mode=True ) sf.write("my_voice_telugu.wav", wavs[0], sr) ``` **Batch.** Pass lists to speak several lines at once, each in its own language: ```python lines = [("இன்று வானிலை நன்றாக உள்ளது. காலையில் லேசான மழை பெய்ததால் காற்று குளிர்ச்சியாக இருக்கிறது. " "மாலையில் கடற்கரைக்குச் செல்லலாம்.", "tamil"), ("আজ আবহাওয়া খুব সুন্দর। সকালে হালকা বৃষ্টি হয়েছিল, তাই বাতাস ঠান্ডা। চলো, সন্ধ্যায় পার্কে হাঁটতে যাই।", "bengali"), ("आज हवामान खूप छान आहे. सकाळी हलका पाऊस पडला, त्यामुळे हवेत गारवा आहे. चला, संध्याकाळी बागेत फिरायला जाऊ.", "marathi"), ("Volt speaks twenty five Indian languages and English. Give it a short clip of any voice, " "and it reads your text in that voice, in the language you choose.", "english")] wavs, sr = tts.generate_voice_clone(text=[t for t, _ in lines], language=[l for _, l in lines], voice_clone_prompt=speaker("meera") * len(lines)) for (_, lang), w in zip(lines, wavs): sf.write(f"meera_{lang}.wav", w, sr) ``` The default sampling settings (`temperature=0.9, top_k=50, repetition_penalty=1.05`) come from `generation_config.json`. Pass any of them to `generate_voice_clone` to change them, e.g. `temperature=0.7`. Call `torch.manual_seed(...)` first for a repeatable result. ## Benchmark (FLEURS, 8 languages) We benchmarked on **FLEURS** (Google), using its real **audio and text**. Each language uses test items from the FLEURS test split. For each item, the voice reference is a real FLEURS recording with its transcript, and the model reads a different FLEURS test sentence in that voice. None of this data was used in training. **Note:** these are early, small-scale numbers from one ASR system, and they leave out English loanwords, so they are lower than a strict score. Counting every word, CER is higher, for example Hindi 7.5 %, Malayalam 2.8 % and Tamil 4.0 %. Treat them as a rough guide, not a full evaluation. | Language | CER | WER | |---|---|---| | Hindi | 1.0 % | 4.1 % | | Malayalam | 1.7 % | 13.5 % | | Kannada | 2.0 % | 9.3 % | | Telugu | 2.3 % | 10.0 % | | Odia | 2.7 % | 12.4 % | | Tamil | 2.8 % | 15.2 % | | Punjabi | 3.1 % | 8.4 % | | Assamese | 5.6 % | 21.8 % | | **Average** | **2.7 %** | **11.8 %** | - **CER / WER:** the generated audio was transcribed with ElevenLabs Scribe v3, with the language forced. Each transcript was compared with the target sentence after Unicode normalisation. CER is computed per language, then averaged. - **English loanwords:** Scribe often writes English loanwords in Latin script even when they are spoken correctly, which the metric counts as errors. These regions are excluded from both the reference and the transcript. Raw CER with them included is higher, for example Hindi 7.5 %, Malayalam 2.8 % and Tamil 4.0 %. - **Speaker similarity:** the cloned voice scores above 0.94 cosine similarity to the reference speaker (Malayalam 0.942, Tamil 0.943; WavLM-base-plus-sv embeddings). Two halves of the same real recording score about 0.96, the practical ceiling. - Sampling settings are the defaults above (temperature 0.9, top-k 50), so every run differs slightly. ## Training - **Method**: LoRA finetuning (rank 32, alpha 64, 40.5M trainable parameters) on the attention, MLP, text projection and output layers; the new token embeddings were trained in full. - **Data**: Rasa (22 languages), SYSPIN (9 languages) and LibriTTS-R (English): 1.18M clips, about 2,240 hours, sampled so smaller languages are not drowned out. - **Tokenizer**: extended with 45,622 Indic pieces (vocab 197,298) and 25 new language ids. - **Voice cloning**: about half the clips were trained with a reference clip and its transcript, the rest with the speaker embedding only. - **Validation**: held-out set of 2,291 clips, loss 2.31 (base model before training: 4.22). ## Limitations - Preview release. Quality varies by language and speaker; low-resource languages are less tested. - No text normaliser yet: numbers, symbols and maths may be read wrongly. - Cross-lingual cloning (for example an English voice speaking Kannada) works but was not trained explicitly. ## License and credits Apache-2.0. Volt-TTS is a finetune of [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) by the Qwen team. It uses the `qwen-tts` package, which is why the config says `qwen3_tts`. Training data: Rasa (AI4Bharat), SYSPIN (IISc / ARTPARK) and LibriTTS-R. Please follow their licenses when using this model. The built-in voices Arjun and Meera each come from one SYSPIN clip.