| --- |
| language: |
| - sa |
| - hi |
| - kn |
| - ta |
| - te |
| - ml |
| license: mit |
| library_name: onnx |
| pipeline_tag: text-to-speech |
| datasets: |
| - ai4bharat/Rasa |
| tags: |
| - text-to-speech |
| - streaming-tts |
| - ninada |
| - sanskrit |
| - vedic |
| - indic |
| - kannada |
| - tamil |
| - telugu |
| - malayalam |
| - vits |
| - hifi-gan |
| - onnxruntime-web |
| - wasm |
| --- |
| |
| # `Ninada` (निनाद) v4.0 — 24,000 Hz Native Per-Language Studio VITS Specialists |
|
|
| `Ninada` (*Nināda* — Sanskrit `निनाद`, deep resonant sound) v4.0 is a **24,000 Hz Native Per-Language Studio Neural TTS** suite (**`~27.9 MB` active W8A32+FP16 ONNX per language, `372.7 ms` native CPU median TTFA, `0.197x` RTF**) with **6 dedicated Studio Male + 6 dedicated Studio Female specialists** (`Sanskrit [sa]`, `Kannada [kn]`, `Hindi [hi]`, `Tamil [ta]`, `Telugu [te]`, `Malayalam [ml]`) trained end-to-end on each language's own native single-speaker `24,000 Hz` studio recordings (`ai4bharat/Rasa`) and native Unicode script tokens. |
|
|
| - **Live Off-Main-Thread Browser `AudioWorklet` Studio**: [https://hemanth.github.io/ninada/](https://hemanth.github.io/ninada/) |
| - **GitHub Repository & Streaming SDKs (JS + Python)**: [https://github.com/hemanth/ninada](https://github.com/hemanth/ninada) |
| - **Curated 24 kHz Studio Split Manifests**: [https://huggingface.co/datasets/gnumanth/ninada-sanskrit-indic](https://huggingface.co/datasets/gnumanth/ninada-sanskrit-indic) |
|
|
| ## Architecture & Native Dataset-Matched Training |
| - **Zero Cross-Language Accent Bleed**: Every language (`sa`, `kn`, `hi`, `ta`, `te`, `ml`) has its own dedicated `24,000 Hz` `StudioVitsModel` (`RelativePositionTransformer` + `StochasticDurationPredictor` Rational-Quadratic Spline + `ResidualCouplingBlock` normalizing flow + `HiFi-GAN v1` `[8, 8, 2, 2]` upsampling generator, `HOP_LENGTH = 256`) trained strictly on that language's own native `ai4bharat/Rasa` single-speaker `24 kHz` studio recordings (`13700` total studio training clips across Male & Female speakers). |
| - **100% Native Script Orthography (`vocabs.json`)**: |
| - `kn`: Native Kannada script (`85` tokens) trained on `ai4bharat/Rasa` Kannada (`SYSPIN/vits_Kannada_{Male,Female}` frozen 500k-step `TextEncoder` + `24 kHz` studio vocoder adaptation). |
| - `sa`: Classical Sanskrit Devanagari (`144` tokens including `ऋ`, `ॠ`, `ऌ`, `ळ`, `ऽ`, `ः`, `।`, `॥`, `ॐ`) trained exclusively on `ai4bharat/Rasa` Sanskrit (`SYSPIN/vits_Marathi_{Male,Female}` warm-start — zero Hindi dataset or Hindi voice sharing). |
| - `te`: Native Telugu script (`110` tokens) trained on `ai4bharat/Rasa` Telugu (`SYSPIN/vits_Telugu_{Male,Female}` warm-start). |
| - `hi`: Native Hindi Devanagari (`120` tokens) trained on `ai4bharat/Rasa` Hindi (`SYSPIN/vits_Hindi_{Male,Female}` warm-start). |
| - `ta`: Native Tamil script (`61` tokens including `ழ`, `ற`, `ன`) trained on `ai4bharat/Rasa` Tamil (`SYSPIN/vits_Kannada_{Male,Female}` Brahmic `+0x0100` embedding isomorphism). |
| - `ml`: Native Malayalam script (`101` tokens including atomic chillus `ൺ ൻ ർ ൽ ൾ ൿ`) trained on `ai4bharat/Rasa` Malayalam (`SYSPIN/vits_Kannada_{Male,Female}` Brahmic `-0x0080` embedding isomorphism). |
| - **`~27.9 MB` Browser W8A32+FP16 ONNX + Offline `CacheStorage`**: Each exported `onnx/{sa,kn,hi,ta,te,ml}.onnx` and `onnx/{sa,kn,hi,ta,te,ml}_female.onnx` is `~27.9 MB` (`0.61 dB` deterministic MCD parity vs PyTorch FP32), persisted in `CacheStorage` + `IndexedDB` (`0` network refetches on reload or tab revisit) and streamed via `24 kHz` `AudioWorklet` ring buffer with `0` underruns. |
|
|
| ## Held-Out 24 kHz Studio Test Leaderboard (`N = 240` Utterances; `B = 1,000` Bootstrap 95% CIs) |
|
|
| | System | Active Footprint (↓) | Sample Rate | Coverage (↑) | TTFA p50 (ms, ↓) | Mean RTF (↓) | MCD 0–8k (dB, ↓) | Studio 8–12k HF Err (dB, ↓) | Log-F0 RMSE (c, ↓) | Dur MAE (s, ↓) | |
| |---|---:|---:|---:|---:|---:|---:|---:|---:|---:| |
| | Base `facebook/mms-tts-hin` (Raw) | 138.8 MB | 16 kHz | 35.0% | 1.0 | 0.030x | `7.97 [7.69, 8.24]` | `66.24 [65.48, 66.95]` | `1018.47 [987.17, 1050.48]` | `3.21 [2.89, 3.54]` | |
| | Meta MMS 5-Specialist Fleet (`5x` VITS) | 694.0 MB | 16 kHz | 100.0% | 60.2 | 0.024x | `5.09 [5.03, 5.15]` | `65.90 [65.49, 66.33]` | `676.29 [661.53, 690.29]` | `1.34 [1.21, 1.48]` | |
| | `Ninada` Tier-2 Zero-Dep Klatt Fallback | 0.0 MB | 24 kHz | 100.0% | 104.0 | 0.0305x | `10.38 [10.26, 10.50]` | `41.94 [41.27, 42.56]` | `613.77 [597.33, 630.07]` | `0.98 [0.88, 1.08]` | |
| | `Ninada` v4.0 Untuned Warm-Start (`step=0`) | 112.0 MB | 24 kHz | 100.0% | 189.2 | 0.047x | `4.60 [4.54, 4.65]` | `11.01 [10.68, 11.35]` | `639.61 [626.39, 653.49]` | `0.95 [0.86, 1.04]` | |
| | **`Ninada` v4.0 Native Studio PyTorch (FP32)** | 112.0 MB | **24 kHz** | **100.0%** | **45.3** | **0.040x** | **`3.40 [3.37, 3.43]`** | **`6.34 [6.16, 6.52]`** | **`529.20 [514.68, 542.48]`** | **`0.57 [0.50, 0.63]`** | |
| | **`Ninada` v4.0 Native W8A32+FP16 ONNX (Browser)** | **27.9 MB** | **24 kHz** | **100.0%** | **372.7** | **0.197x** | **`3.39 [3.36, 3.43]`** | **`6.37 [6.19, 6.57]`** | **`530.38 [517.00, 543.32]`** | **`0.56 [0.50, 0.62]`** | |
|
|
| ## 24 kHz Streaming Quickstart |
|
|
| ```js |
| import ninada from 'ninada'; |
| |
| const voice = await ninada('kannada', { |
| models: { |
| kn: 'https://huggingface.co/gnumanth/ninada/resolve/main/onnx/kn.onnx', |
| sa: 'https://huggingface.co/gnumanth/ninada/resolve/main/onnx/sa.onnx', |
| }, |
| }); |
| for await (const chunk of voice.stream('ನಮಸ್ಕಾರ, ಇಂದು ಬೆಂಗಳೂರಿನಲ್ಲಿ ಹವಾಮಾನ ತುಂಬಾ ಚೆನ್ನಾಗಿದೆ.')) { |
| playPcm24k(chunk.pcm, chunk.sampleRate); // 24,000 Hz native Kannada studio stream! |
| } |
| ``` |
|
|
| ```python |
| from ninada import Ninada |
| |
| tts = Ninada(lang="sa") |
| for chunk in tts.stream("प्रज्ञापूर्वकव्यापारवत्त्वं जीवत्वम् । यथार्थज्ञानं प्रमाणम् ॥"): |
| play(chunk["pcm"], chunk["sample_rate"]) # 24,000 Hz float32 native Sanskrit PCM |
| ``` |
|
|
