|
Download README.md from openarcane/Volt-TTS: direct link, hf CLI and curl.
- Browser
- Download file 13.4 kB
-
https://huggingface.co/openarcane/Volt-TTS/resolve/main/README.md
- Command line
-
hf download hf://openarcane/Volt-TTS/README.md
-
curl -L -o README.md https://huggingface.co/openarcane/Volt-TTS/resolve/main/README.md
13.4 kB
| license: apache-2.0 | |
| pipeline_tag: text-to-speech | |
| tags: [text-to-speech, tts, indic, voice-cloning, multilingual] | |
| language: [kn, hi, ta, te, ml, mr, bn, gu, pa, or, as, ne, ur, sa, ks, sd, kok, mai, mni, brx, doi, sat, bho, mag, hne] | |
| # Volt-TTS | |
| <video src="https://huggingface.co/openarcane/Volt-TTS/resolve/main/Volt-TTS-teaser.mp4" controls autoplay muted loop playsinline width="100%"></video> | |
| Volt-TTS is a text-to-speech model for 25 Indian languages, finetuned from | |
| [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) by the Qwen team. It speaks in | |
| two built-in voices, or clones any voice from a short reference clip. This is a preview release. | |
| ## Highlights | |
| - **25 Indian languages** in one model, chosen per line. | |
| - **Two built-in voices**: Arjun (male) and Meera (female), both speaking every language. | |
| - **Voice cloning** from a 5-15 s clip, with or without its transcript, in any language. | |
| ## Samples | |
| Built-in voices (Arjun, Meera) with default settings. | |
| | Voice | Language | Text | Audio | | |
| |---|---|---|---| | |
| | Arjun | Kannada | ಬೆಂಗಳೂರು ಕರ್ನಾಟಕದ ರಾಜಧಾನಿ. ಇಲ್ಲಿನ ಹವಾಮಾನ ವರ್ಷವಿಡೀ ಹಿತವಾಗಿರುತ್ತದೆ. ಉದ್ಯಾನಗಳು, ಕೆರೆಗಳು ಮತ್ತು ತಂತ್ರಜ್ಞಾನ ಕಂಪನಿಗಳಿಂದ ಈ ನಗರ ದೇಶದಾದ್ಯಂತ ಪ್ರಸಿದ್ಧವಾಗಿದೆ. | <audio controls src="https://huggingface.co/openarcane/Volt-TTS/resolve/main/samples/arjun_kannada.wav"></audio> | | |
| | Meera | Hindi | भारत में हर कुछ किलोमीटर पर भाषा और खाना बदल जाता है। यही विविधता इस देश को खास बनाती है, और हर यात्रा में कुछ नया सीखने को मिलता है। | <audio controls src="https://huggingface.co/openarcane/Volt-TTS/resolve/main/samples/meera_hindi.wav"></audio> | | |
| | Meera | Tamil | சென்னை மெரினா கடற்கரை உலகின் நீளமான கடற்கரைகளில் ஒன்று. மாலை நேரத்தில் குடும்பங்கள் இங்கே வந்து காற்று வாங்கி, சுண்டல் சாப்பிட்டு மகிழ்கிறார்கள். | <audio controls src="https://huggingface.co/openarcane/Volt-TTS/resolve/main/samples/meera_tamil.wav"></audio> | | |
| | Arjun | Telugu | హైదరాబాద్ బిర్యానీ రుచికి ప్రపంచమంతా అభిమానులు ఉన్నారు. చార్మినార్ దగ్గర సాయంత్రం నడుస్తూ, వేడి వేడి ఇరానీ చాయ్ తాగడం ఒక మంచి అనుభవం. | <audio controls src="https://huggingface.co/openarcane/Volt-TTS/resolve/main/samples/arjun_telugu.wav"></audio> | | |
| | Meera | Bengali | কলকাতার দুর্গাপূজা শুধু একটি উৎসব নয়, এটি শহরের প্রাণ। প্রতিটি পাড়ায় সুন্দর প্যান্ডেল সাজানো হয়, আর রাতভর মানুষ ঠাকুর দেখতে বের হয়। | <audio controls src="https://huggingface.co/openarcane/Volt-TTS/resolve/main/samples/meera_bengali.wav"></audio> | | |
| | Arjun | Malayalam | കേരളത്തിലെ കായലുകളിലൂടെയുള്ള ഹൗസ്ബോട്ട് യാത്ര വളരെ മനോഹരമാണ്. പച്ചപ്പ് നിറഞ്ഞ തീരങ്ങളും ശാന്തമായ വെള്ളവും മനസ്സിന് വലിയ സമാധാനം നൽകുന്നു. | <audio controls src="https://huggingface.co/openarcane/Volt-TTS/resolve/main/samples/arjun_malayalam.wav"></audio> | | |
| | Meera | Marathi | पुण्यातला पाऊस काही वेगळाच असतो. डोंगरावरून येणारे ढग, गरम चहा आणि भजी, यामुळे पावसाळा हा सगळ्यांचा आवडता ऋतू आहे. | <audio controls src="https://huggingface.co/openarcane/Volt-TTS/resolve/main/samples/meera_marathi.wav"></audio> | | |
| ## Model details | |
| | | | | |
| |---|---| | |
| | Parameters | 2.02B (the 1.93B base plus 93M for the new Indic token embeddings) | | |
| | Base model | [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) | | |
| | Languages | 25 Indian languages | | |
| | Speech codec | 12.5 Hz (from the base model) | | |
| | Precision | bf16 | | |
| | License | Apache-2.0 | | |
| ## Languages | |
| Pass the value in the `language=` column to `generate_voice_clone`. | |
| | Language | `language=` | Language | `language=` | | |
| |---|---|---|---| | |
| | Assamese | `"assamese"` | Malayalam | `"malayalam"` | | |
| | Bengali | `"bengali"` | Manipuri | `"manipuri"` | | |
| | Bhojpuri | `"bhojpuri"` | Marathi | `"marathi"` | | |
| | Bodo | `"bodo"` | Nepali | `"nepali"` | | |
| | Chhattisgarhi | `"chhattisgarhi"` | Odia | `"odia"` | | |
| | Dogri | `"dogri"` | Punjabi | `"punjabi"` | | |
| | Gujarati | `"gujarati"` | Sanskrit | `"sanskrit"` | | |
| | Hindi | `"hindi"` | Santali | `"santali"` | | |
| | Kannada | `"kannada"` | Sindhi | `"sindhi"` | | |
| | Kashmiri | `"kashmiri"` | Tamil | `"tamil"` | | |
| | Konkani | `"konkani"` | Telugu | `"telugu"` | | |
| | Magahi | `"magahi"` | Urdu | `"urdu"` | | |
| | Maithili | `"maithili"` | | | | |
| ## Quickstart | |
| Needs a CUDA GPU with about 8 GB of memory. Warnings about SoX and the tokenizer class can be ignored. | |
| ```bash | |
| pip install -U qwen-tts soundfile | |
| ``` | |
| **Built-in voices** | |
| ```python | |
| import soundfile as sf | |
| import torch | |
| from huggingface_hub import hf_hub_download | |
| from safetensors import safe_open | |
| from qwen_tts import Qwen3TTSModel | |
| from qwen_tts.inference.qwen3_tts_model import VoiceClonePromptItem | |
| tts = Qwen3TTSModel.from_pretrained("openarcane/Volt-TTS", device_map="cuda:0", dtype=torch.bfloat16) | |
| def speaker(name): | |
| """A built-in voice: "arjun" (male) or "meera" (female).""" | |
| path = hf_hub_download("openarcane/Volt-TTS", f"speakers/{name}.safetensors") | |
| with safe_open(path, "pt") as f: | |
| return [VoiceClonePromptItem(ref_code=f.get_tensor("ref_code"), ref_spk_embedding=f.get_tensor("ref_spk_embedding"), | |
| x_vector_only_mode=False, icl_mode=True, ref_text=f.metadata()["ref_text"])] | |
| wavs, sr = tts.generate_voice_clone( | |
| text=("ಬೆಂಗಳೂರು ಕರ್ನಾಟಕದ ರಾಜಧಾನಿ. ಇಲ್ಲಿನ ಹವಾಮಾನ ವರ್ಷವಿಡೀ ಹಿತವಾಗಿರುತ್ತದೆ. " | |
| "ಉದ್ಯಾನಗಳು, ಕೆರೆಗಳು ಮತ್ತು ತಂತ್ರಜ್ಞಾನ ಕಂಪನಿಗಳಿಂದ ಈ ನಗರ ದೇಶದಾದ್ಯಂತ ಪ್ರಸಿದ್ಧವಾಗಿದೆ."), | |
| language="kannada", voice_clone_prompt=speaker("arjun")) | |
| sf.write("arjun_kannada.wav", wavs[0], sr) | |
| wavs, sr = tts.generate_voice_clone( | |
| text="आज मौसम बहुत अच्छा है। सुबह हल्की बारिश हुई थी, इसलिए हवा में ठंडक है। चलो, शाम को पार्क में टहलने चलते हैं।", | |
| language="hindi", voice_clone_prompt=speaker("meera")) | |
| sf.write("meera_hindi.wav", wavs[0], sr) | |
| ``` | |
| **Voice cloning.** Give a 5-15 s clip of the voice and what is said in it. The clip can be in any language. | |
| ```python | |
| wavs, sr = tts.generate_voice_clone( | |
| text=("ఈ రోజు వాతావరణం చాలా బాగుంది. ఉదయం కొంచెం వర్షం పడింది, అందుకే గాలి చల్లగా ఉంది. " | |
| "సాయంత్రం పార్కుకు నడకకు వెళ్దాం."), | |
| language="telugu", | |
| ref_audio="my_voice.wav", # path, URL or base64 | |
| ref_text="Transcript of my_voice.wav.", # no transcript? drop it and pass x_vector_only_mode=True | |
| ) | |
| sf.write("my_voice_telugu.wav", wavs[0], sr) | |
| ``` | |
| **Batch.** Pass lists to speak several lines at once, each in its own language: | |
| ```python | |
| lines = [("இன்று வானிலை நன்றாக உள்ளது. காலையில் லேசான மழை பெய்ததால் காற்று குளிர்ச்சியாக இருக்கிறது. " | |
| "மாலையில் கடற்கரைக்குச் செல்லலாம்.", "tamil"), | |
| ("আজ আবহাওয়া খুব সুন্দর। সকালে হালকা বৃষ্টি হয়েছিল, তাই বাতাস ঠান্ডা। চলো, সন্ধ্যায় পার্কে হাঁটতে যাই।", "bengali"), | |
| ("आज हवामान खूप छान आहे. सकाळी हलका पाऊस पडला, त्यामुळे हवेत गारवा आहे. चला, संध्याकाळी बागेत फिरायला जाऊ.", "marathi"), | |
| ("Volt speaks twenty five Indian languages and English. Give it a short clip of any voice, " | |
| "and it reads your text in that voice, in the language you choose.", "english")] | |
| wavs, sr = tts.generate_voice_clone(text=[t for t, _ in lines], language=[l for _, l in lines], | |
| voice_clone_prompt=speaker("meera") * len(lines)) | |
| for (_, lang), w in zip(lines, wavs): | |
| sf.write(f"meera_{lang}.wav", w, sr) | |
| ``` | |
| The default sampling settings (`temperature=0.9, top_k=50, repetition_penalty=1.05`) come from | |
| `generation_config.json`. Pass any of them to `generate_voice_clone` to change them, e.g. `temperature=0.7`. | |
| Call `torch.manual_seed(...)` first for a repeatable result. | |
| ## Benchmark (FLEURS, 8 languages) | |
| We benchmarked on **FLEURS** (Google), using its real **audio and text**. Each language uses test items from | |
| the FLEURS test split. For each item, the voice reference is a real FLEURS recording with its transcript, and the | |
| model reads a different FLEURS test sentence in that voice. None of this data was used in training. | |
| **Note:** these are early, small-scale numbers from one ASR system, and they leave out English loanwords, so | |
| they are lower than a strict score. Counting every word, CER is higher, for example Hindi | |
| 7.5 %, Malayalam 2.8 % and Tamil 4.0 %. Treat them as a rough guide, not a full evaluation. | |
| | Language | CER | WER | | |
| |---|---|---| | |
| | Hindi | 1.0 % | 4.1 % | | |
| | Malayalam | 1.7 % | 13.5 % | | |
| | Kannada | 2.0 % | 9.3 % | | |
| | Telugu | 2.3 % | 10.0 % | | |
| | Odia | 2.7 % | 12.4 % | | |
| | Tamil | 2.8 % | 15.2 % | | |
| | Punjabi | 3.1 % | 8.4 % | | |
| | Assamese | 5.6 % | 21.8 % | | |
| | **Average** | **2.7 %** | **11.8 %** | | |
| - **CER / WER:** the generated audio was transcribed with ElevenLabs Scribe v3, with the language forced. Each | |
| transcript was compared with the target sentence after Unicode normalisation. CER is computed per language, | |
| then averaged. | |
| - **English loanwords:** Scribe often writes English loanwords in Latin script even when they are spoken | |
| correctly, which the metric counts as errors. These regions are excluded from both the reference and the | |
| transcript. Raw CER with them included is higher, for example Hindi 7.5 %, Malayalam 2.8 % and Tamil 4.0 %. | |
| - **Speaker similarity:** the cloned voice scores above 0.94 cosine similarity to the reference speaker | |
| (Malayalam 0.942, Tamil 0.943; WavLM-base-plus-sv embeddings). Two halves of the same real recording score | |
| about 0.96, the practical ceiling. | |
| - Sampling settings are the defaults above (temperature 0.9, top-k 50), so every run differs slightly. | |
| ## Training | |
| - **Method**: LoRA finetuning (rank 32, alpha 64, 40.5M trainable parameters) on the attention, MLP, text | |
| projection and output layers; the new token embeddings were trained in full. | |
| - **Data**: Rasa (22 languages), SYSPIN (9 languages) and LibriTTS-R (English): 1.18M clips, about 2,240 hours, | |
| sampled so smaller languages are not drowned out. | |
| - **Tokenizer**: extended with 45,622 Indic pieces (vocab 197,298) and 25 new language ids. | |
| - **Voice cloning**: about half the clips were trained with a reference clip and its transcript, the rest with the | |
| speaker embedding only. | |
| - **Validation**: held-out set of 2,291 clips, loss 2.31 (base model before training: 4.22). | |
| ## Limitations | |
| - Preview release. Quality varies by language and speaker; low-resource languages are less tested. | |
| - No text normaliser yet: numbers, symbols and maths may be read wrongly. | |
| - Cross-lingual cloning (for example an English voice speaking Kannada) works but was not trained explicitly. | |
| ## License and credits | |
| Apache-2.0. Volt-TTS is a finetune of [Qwen3-TTS-12Hz-1.7B-Base](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) | |
| by the Qwen team. It uses the `qwen-tts` package, which is why the config says `qwen3_tts`. | |
| Training data: Rasa (AI4Bharat), SYSPIN (IISc / ARTPARK) and LibriTTS-R. Please follow their licenses when using | |
| this model. The built-in voices Arjun and Meera each come from one SYSPIN clip. | |