README.md

metadata

license: cc-by-sa-4.0
language:
  - ne
  - en
pipeline_tag: text-to-speech
tags:
  - text-to-speech
  - voice-cloning
  - nepali
  - f5-tts
base_model: ai4bharat/IndicF5

Swar: open Nepali voice cloning

Swar speaks Nepali the way it is spoken in Nepal and clones a voice from 20 to 30 seconds of someone talking. It reads Nepali, English, and Nepali mixed with English. By Naamche Labs.

Try it in your browser, no setup: Swar Playground: nine ready voices, or clone your own from 30 seconds of speech, free to try. Run it yourself: Colab notebook (free T4 GPU). Code and paper: github.com/Naamche-Labs/swar.

Files

• podvoices_ema.pt: the default model (F5-TTS Base architecture, EMA weights). Every result below is this file.
• swar_v6_ema.pt: a second checkpoint for English-heavy text, trained further with 96 h of English (LibriTTS-R). Mixed Nepali-English errors 5.24% (default 5.6%), but pure Nepali is weaker (NepTTS-Bench pronunciation error 4.04%, FLEURS cloning errors 5.41%). Use it with ENGLISH_LATIN=1.
• vocab.txt: character vocabulary (IndicF5).
• kriti_onnx/: Kriti, our Nepali speech recogniser (MIT), used to transcribe uploads and check takes.
• presets/: ready voices Maya (female) and Aarav (male), built only from model-made audio.
• app/: the cloner app (Gradio) used by the notebook.

Results (NepTTS-Bench, 205 sentences, one take)

System	Character error	Pronunciation error	UTMOS
Swar, studio voice	6.19	2.92	3.82
Swar, Maya voice	7.45	4.52	4.09
OmniVoice (same prompt)	6.68	4.32	3.93
Indic-Speak Srijana	10.25	8.12	3.97
ElevenLabs v3	15.70	12.91

Nepali Voice Cloning Benchmark (our open benchmark on FLEURS, v1.1, clean track, 64 distinct prompt + sentence pairs from 4 unseen voices): Swar errors 4.72% and voice match 0.697, OmniVoice 5.08% and 0.683, IndicF5 8.30% and 0.669.

Nepali mixed with English (30 held-out sentences): Swar 5.6% character errors, OmniVoice 11.4%. UTMOS is an automatic predictor, not a listener test.

Training data

Initialised from ai4bharat/IndicF5 (MIT). Fine-tuned on AI4Bharat Rasa and IndicVoices-R (CC BY 4.0), Google FLEURS (CC BY 4.0), OpenSLR 43 and 54 (CC BY-SA 4.0), LibriTTS-R (CC BY 4.0), and podcast speech used with the speakers' permission (not released). The weights are released under CC BY-SA 4.0 to respect the share-alike data.

Responsible use

Only clone a voice with that person's permission. Do not use Swar to impersonate anyone, to deceive, or to commit fraud.