bharathkumarK commited on
Commit
59fe770
·
verified ·
1 Parent(s): 35b2a3b

Rewrite Maya 2 Native model card and embed launch video

Browse files

Reorganize the card around the video, multilingual samples, quickstart, evaluation, and credits. Preserve model files and all runtime identifiers.

Files changed (3) hide show
  1. .gitattributes +1 -0
  2. Maya Final with Changes.mp4 +3 -0
  3. README.md +347 -203
.gitattributes CHANGED
@@ -46,3 +46,4 @@ samples/or_ahana.wav filter=lfs diff=lfs merge=lfs -text
46
  samples/pa_malini.wav filter=lfs diff=lfs merge=lfs -text
47
  samples/ta_abhay.wav filter=lfs diff=lfs merge=lfs -text
48
  samples/te_nivedita.wav filter=lfs diff=lfs merge=lfs -text
 
 
46
  samples/pa_malini.wav filter=lfs diff=lfs merge=lfs -text
47
  samples/ta_abhay.wav filter=lfs diff=lfs merge=lfs -text
48
  samples/te_nivedita.wav filter=lfs diff=lfs merge=lfs -text
49
+ Maya[[:space:]]Final[[:space:]]with[[:space:]]Changes.mp4 filter=lfs diff=lfs merge=lfs -text
Maya Final with Changes.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57d84c690b339c2b77641ee42d95d82c756eff43c2d59b567df9882378d228d8
3
+ size 30109557
README.md CHANGED
@@ -26,263 +26,407 @@ tags:
26
  - mimi
27
  ---
28
 
29
- # Maya3 TTS
30
 
31
- Maya3 is an open-weights, streaming text-to-speech model for **11 Indic languages and English**, with
32
- **20 built-in voices** and **zero-shot voice cloning** from a 3-10 s reference clip. It is served by the `maya3`
33
- inference engine (continuous batching, CUDA graphs, OpenAI-compatible HTTP + WebSocket streaming):
34
- code at **https://github.com/MayaResearch/maya2-native**.
35
 
36
- **[Samples](#samples) · [Usage](#usage) · [Evaluation](#evaluation) · [Limitations](#limitations) · [Code](https://github.com/MayaResearch/maya2-native)**
37
 
38
- > **Release 1.0 (2026-10-25).** Weights are bf16 safetensors (5.0 GB, 2.50 B unique parameters).
39
 
40
- ## Overview
41
 
42
- - **Languages:** Assamese, Bengali, English, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu.
43
- - **Voices:** 20 named voices selected with `voice="Name"`. No reference audio is needed. See [Voices](#voices).
44
- - **Voice cloning:** zero-shot, from a 3-10 s clip plus its transcript. Cloning works across languages: a Hindi reference can speak Tamil.
45
- - **Streaming:** the first audio arrives in about 14 ms at B=1 on an H100. On one H100 NVL the server sustains 224 concurrent streams with 0 playback stalls (see [Evaluation](#evaluation)).
46
- - **Text normaliser (bundled):** digits, currency, dates, times, URLs, e-mail addresses and ALL-CAPS names are expanded in the request language. The server applies it by default.
47
- - **Inline tags:** `[laughs]`, `[sighs]`, `[whispers]`, `[sad]` and `[excited]` can appear in the text. They are best effort.
48
 
49
- ## Samples
50
 
51
- Twelve clips from the release benchmark, one per language, each in a different voice. They are unedited outputs of the server with its
52
- default settings (normaliser on, sampled decoding), picked from clips that all three ASR judges transcribed with under 2 % CER;
53
- see [Evaluation](#evaluation) for the averages. The text column is the input as sent, before normalisation.
54
 
55
- | language | voice | text | audio |
56
- |---|---|---|---|
57
- | Assamese | Nakul | এন ই এফ টি লেনদেনটো সম্পূৰ্ণ হ'বলৈ প্ৰায় 2 ঘণ্টা 30 মিনিট সময় লাগিব পাৰে। | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/as_nakul.wav"></audio> |
58
- | Bengali | Jahnavi | 12/08/2023 তারিখে আপনার 1,500 টাকার refund process হয়ে গেছে, status update চেক করুন। | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/bn_jahnavi.wav"></audio> |
59
- | English | Siddhant | The EMI for your home loan is due by 12-10-24; please ensure your account has sufficient funds. | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/en_siddhant.wav"></audio> |
60
- | Gujarati | Vaidehi | જી બિલકુલ, મને સિસ્ટમમાં દેખાય છે કે તમારું ₹2,500 નું પેમેન્ટ 15મી તારીખે સક્સેસફુલ થઈ ગયું છે. | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/gu_vaidehi.wav"></audio> |
61
- | Hindi | Ritvik | क्या आपका वर्तमान पता 45, एमजी रोड, अंधेरी ईस्ट, मुंबई 400069 है? यदि हाँ, तो 1 दबाएं। | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/hi_ritvik.wav"></audio> |
62
- | Kannada | Satvik | ಹೌದಾ... ಅದು ನಿಜಕ್ಕೂ ಕಷ್ಟ. ನಿಮ್ಮ ಅಂತಿಮ ಬಿಲ್ ₹45,000 ಆಗಿದೆ, ಅದರಲ್ಲಿ ಇನ್ಶೂರೆನ್ಸ್ ಕವರ್ ಆಗಿದೆಯಾ ಅಂತ ಚೆಕ್ ಮಾಡ್ತೀನಿ. | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/kn_satvik.wav"></audio> |
63
- | Malayalam | Anirban | റദ്ദാക്കിയ ടിക്കറ്റിന്റെ റീഫണ്ട് തുകയായ ₹1,800 മൂന്ന് ദിവസത്തിനുള്ളിൽ ലഭിക്കും. | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/ml_anirban.wav"></audio> |
64
- | Marathi | Charvi | मला 98765 43210 या नंबरवर OTP आला नाही, कृपया पुन्हा resend करा. | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/mr_charvi.wav"></audio> |
65
- | Odia | Ahana | ଦୟାକରି ମନେରଖନ୍ତୁ ଯେ EMI ପ୍ରତ୍ୟେକ ମାସର 5 ତାରିଖରେ କଟିବ ଏବଂ ଏହା ଆପଣଙ୍କର 3ୟ କିସ୍ତି। | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/or_ahana.wav"></audio> |
66
- | Punjabi | Malini | NEFT ਰਾਹੀਂ ਫੰਡ ਟ੍ਰਾਂਸਫਰ 00:30 ਤੋਂ 23:30 ਤੱਕ ਉਪਲਬਧ ਹਨ। | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/pa_malini.wav"></audio> |
67
- | Tamil | Abhay | கடந்த நிதியாண்டில் வங்கியின் மொத்த லாபம் ₹3.5 கோடி, இதில் ₹45 லட்சம் முதலீட்டாளர்களுக்கு வழங்கப்பட்டது. | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/ta_abhay.wav"></audio> |
68
- | Telugu | Nivedita | సేవింగ్స్ తో పోలిస్తే ఫిక్స్డ్ డిపాజిట్ వడ్డీ నిష్పత్తి 3:2 గా ఉండి, 7.25% ఇస్తుంది. | <audio controls preload="none" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/te_nivedita.wav"></audio> |
69
 
70
- Tamil by Abhay is cross-lingual: Abhay was trained on English and Hindi only.
71
 
72
- ## Architecture
 
 
 
 
 
 
73
 
74
- | part | details |
75
  |---|---|
76
- | backbone | Qwen3-1.7B decoder (28 layers, hidden 2048, GQA 16/8), extended tokenizer with language and audio tokens |
77
- | depth decoder | 555 M-parameter transformer that predicts codebooks 1-31 of each frame, level by level |
78
- | audio codec | [kyutai/mimi](https://huggingface.co/kyutai/mimi): 32 codebooks x 2048 entries, 12.5 Hz (80 ms frames), 24 kHz mono |
79
- | speaker anchor | 192-dim ECAPA embedding ([speechbrain/spkrec-ecapa-voxceleb](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb)) -> MLP, injected into the backbone |
80
- | prompt | `<|lang:xx|>` + optional reference (text + Mimi codes) or a voice name `{Name}: ` + target text + `<|audio|>`. Generation stops on a learned end-of-audio token. |
 
 
81
 
82
- Per step, the backbone predicts codebook 0 and the depth decoder fills in the other 31 levels. The Mimi decoder
83
- turns each frame into 80 ms of audio, so audio can be streamed frame by frame.
84
 
85
- **Files in this repo**
86
 
87
- | path | content |
 
 
 
 
88
  |---|---|
89
- | `model-0000{1,2}-of-00002.safetensors`, `model.safetensors.index.json` | Maya3 weights in bf16. The tied `lm_head` is stored once as an alias of `embed_tokens`. |
90
- | `maya3_config.json` | resolved model config (speaker anchor, depth decoder), format version, aliases |
91
- | `backbone/` | Qwen3 `config.json` and tokenizer only. There are no weights here: every weight is in the Maya3 safetensors. |
92
- | `voices/voices.json` | the 20 named voices (name, gender, age band, languages, description, conditioning mode) |
93
- | `samples/` | the 12 sample clips above (24 kHz WAV) |
94
-
95
- Mimi and ECAPA are downloaded from their own repos on first use.
96
-
97
- ## Languages
98
-
99
- Character error rate (CER, %) on the 6,000-clip release benchmark (500 clips per language), measured with three ASR
100
- judges. The other columns are the real-failure rate and the voice-identity accuracy (see [Evaluation](#evaluation)).
101
-
102
- | code | language | script | CER ASR-1 | CER Qwen3-ASR | CER ASR-2 (held out) | real fail % | identity % |
103
- |---|---|---|---:|---:|---:|---:|---:|
104
- | `as` | Assamese | Bengali-Assamese | 5.41 | 6.59 | 5.54 | 0.20 | 96.4 |
105
- | `bn` | Bengali | Bengali | 2.60 | 2.83 | 2.68 | 0.00 | 98.5 |
106
- | `en` | English | Latin | 9.69 | 1.74 | 1.43 | 0.20 | 99.6 |
107
- | `gu` | Gujarati | Gujarati | 2.25 | 3.76 | 2.26 | 0.20 | 98.9 |
108
- | `hi` | Hindi | Devanagari | 1.52 | 2.05 | 1.57 | 0.00 | 99.6 |
109
- | `kn` | Kannada | Kannada | 2.28 | 2.72 | 2.23 | 0.00 | 97.9 |
110
- | `ml` | Malayalam | Malayalam | 3.75 | 4.20 | 4.03 | 0.40 | 99.4 |
111
- | `mr` | Marathi | Devanagari | 1.66 | 2.09 | 1.54 | 0.20 | 99.1 |
112
- | `or` | Odia | Odia | 3.45 | 4.33 | 3.54 | 0.00 | 97.4 |
113
- | `pa` | Punjabi | Gurmukhi | 2.51 | 3.32 | 2.66 | 0.40 | 99.1 |
114
- | `ta` | Tamil | Tamil | 4.42 | 4.55 | 4.33 | 0.20 | 98.1 |
115
- | `te` | Telugu | Telugu | 2.29 | 2.64 | 2.53 | 0.00 | 99.4 |
116
-
117
- - **English, ASR-1:** a few clips with spoken URLs ("W W W") can make ASR-1 loop, which inflates its English CER. The other two judges are the better guide for English.
118
- - **Assamese is the weakest language** (see [Limitations](#limitations)).
119
 
120
- ## Voices
121
 
122
- Select a voice with `voice="<Name>"` (case-insensitive). Every voice can be asked to speak any of the 12 languages.
123
- The languages listed are those with at least 1 h of training audio for that voice. Other languages work
124
- cross-lingually but sound less native.
125
-
126
- | voice | gender | age | languages (>= 1 h trained) | description | identity % (bench) | CER % (Qwen3-ASR) |
127
- |---|---|---|---|---|---:|---:|
128
- | Ahana | female | young | bn en gu hi kn ml mr or pa ta te | Young female voice, bright, youthful | 94.2 | 3.83 |
129
- | Vaidehi | female | adult | bn en gu hi kn ml mr or pa ta te | Adult female voice, calm, gentle | 98.6 | 3.47 |
130
- | Malini | female | mature | bn en gu hi kn ml mr or pa ta te | Mature female voice, steady narrator | 95.0 | 3.37 |
131
- | Charvi | female | young | bn en gu hi kn ml mr or pa ta te | Young female voice, energetic, upbeat | 99.8 | 3.36 |
132
- | Jahnavi | female | adult | bn en gu hi kn ml mr or ta te | Adult female voice, warm, friendly | 99.6 | 3.00 |
133
- | Anirban | male | adult | bn en gu hi kn ml mr or pa ta te | Adult male voice, steady narrator | 98.4 | 4.21 |
134
- | Nakul | male | young | bn en gu hi kn ml mr or pa ta te | Young male voice, bright, youthful | 99.8 | 3.74 |
135
- | Vasant | male | mature | bn en gu hi kn ml mr or ta te | Mature male voice, steady narrator | 100.0 | 3.35 |
136
- | Siddhant | male | adult | bn en gu hi kn ml mr or pa ta te | Adult male voice, clear, professional | 99.8 | 3.57 |
137
- | Satvik | male | young | bn en gu hi kn ml mr or pa ta te | Young male voice, warm, friendly | 99.6 | 3.31 |
138
- | Abhay | male | adult | en hi | Adult male voice, natural conversational style; trained on English, Hindi | 99.2 | 2.97 |
139
- | Nivedita | female | adult | en hi | Adult female voice, natural conversational style; trained on English, Hindi | 100.0 | 2.49 |
140
- | Ishwari | female | adult | hi | Adult female voice, natural conversational style; trained on Hindi | 100.0 | 4.34 |
141
- | Ritvik | male | adult | hi | Adult male voice, natural conversational style; trained on Hindi | 100.0 | 1.98 |
142
- | Vrinda | female | young | hi | Young female voice, natural conversational style; trained on Hindi | 97.9 | 1.92 |
143
- | Sreekar | male | adult | en hi te | Adult male voice, natural conversational style; trained on English, Hindi, Telugu | 100.0 | 1.85 |
144
- | Kayal | female | adult | en ta | Adult female voice, natural conversational style; trained on English, Tamil | 100.0 | 4.21 |
145
- | Shaunak | male | young | en hi | Young male voice, natural conversational style; trained on English, Hindi | 100.0 | 1.75 |
146
- | Akshat | male | adult | en hi | Adult male voice, natural conversational style; trained on English, Hindi | 100.0 | 0.76 |
147
- | Tanushree | female | adult | en hi | Adult female voice, natural conversational style; trained on English, Hindi | 100.0 | 1.78 |
148
-
149
- All 20 voices are **synthetic personas**. They do not represent and must not be presented as real people.
150
-
151
- **Prompt format.** A named voice is rendered by the engine as `<|lang:xx|>{Name}: <your text><|audio|>`. Pass only
152
- your text and `voice="Name"`. Do not type the `{Name}: ` prefix yourself, and do not combine a named voice with your
153
- own reference audio.
154
-
155
- ## Usage
156
 
157
  ```bash
158
- pip install "maya3[server] @ git+https://github.com/MayaResearch/maya2-native" --extra-index-url https://download.pytorch.org/whl/cu128
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
159
  ```
160
 
 
 
 
 
 
 
 
 
 
161
  ```python
162
  from maya3 import Maya3TTS
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
163
  import soundfile as sf
 
164
 
165
- tts = Maya3TTS.from_pretrained("maya-research/maya2-native", max_seats=16) # downloads this repo + Mimi + ECAPA
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
166
 
167
- wav = tts.generate("नमस्ते, आपका ऑर्डर कल पहुँच जाएगा।", lang="hi", voice="Ahana")
168
- sf.write("ahana_hi.wav", wav, 24000) # float32, 24 kHz mono
169
 
170
- # zero-shot cloning: 3-10 s reference + its exact transcript (use only voices you have consent for)
171
- tts.register_voice("my_voice", ref_audio="ref.wav", ref_text="<exact transcript>", lang="hi")
172
- wav = tts.generate("வணக்கம், எப்படி இருக்கிறீர்கள்?", lang="ta", voice="my_voice") # cross-lingual
173
 
174
- for pcm in tts.stream("Your OTP is 482913.", lang="en", voice="Siddhant", normalize=True):
175
- ... # int16 PCM chunks as they are generated
 
 
176
  ```
177
 
178
- Server (OpenAI-compatible `POST /v1/audio/speech` and `WS /v1/tts/stream`):
179
 
180
  ```bash
181
- python -m maya3.server --ckpt maya-research/maya2-native --port 8080 --max-seats 64
182
- curl -sN localhost:8080/v1/audio/speech -H 'content-type: application/json' \
183
- -d '{"input":"Your order has shipped.","language":"en","voice":"Ahana","response_format":"wav"}' > out.wav
 
184
  ```
185
 
186
- You need an NVIDIA GPU with a CUDA 12.8 driver. One stream needs about 8 GB of GPU memory, and 16 streams about
187
- 13 GB. See the GitHub README for the memory table, the WebSocket protocol and the tuning options.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
188
 
189
  ## Evaluation
190
 
191
- **Release benchmark ("bench6k").**
192
- - **Data:** 6000 held-out texts: 500 per language over 31 content categories (numbers, dates, currency, addresses, URLs, code-mix, names, tongue twisters, conversational, 1-50-word lengths and more). None of these texts were used in training.
193
- - **Setup:** every voice is assigned evenly across languages and categories. Generation goes through the public `maya3.server` with its defaults: text normaliser on, engine guards on, sampled decoding, one seed per row. Long texts are chunked as in production. 20.3 h of audio were generated.
194
- - **Judges: three ASR systems.**
195
- - ASR-1 is an in-house Indic ASR and the primary judge.
196
- - ASR-2 is a second in-house Indic ASR that is held out: it was never used in training, reward or checkpoint selection.
197
- - [Qwen3-ASR](https://huggingface.co/Qwen) is the third.
198
- - **CER:** utterance CER against the spoken-form reference, with harness re-speaking and script harmonisation. Scorer v2 additionally harmonises Latin and native-script spellings in code-mixed spans.
199
- - **Real failure:** a clip counts as a real failure only when **two independent ASRs agree** that it is broken (ASR-1 and Qwen3-ASR): the length cap was hit, both see a loop, both see a large overrun with insertions, or both have CER > 50 %. One-ASR disagreements are treated as ASR noise. We also report the stricter 3-ASR rate.
200
- - **Identity:** the ECAPA embedding of each clip (>= 1 s) is compared with the 20 voice centroids, and the clip counts as correct when the nearest centroid is the intended voice.
201
-
202
- | metric | value |
203
- |---|---|
204
- | CER, ASR-1 / Qwen3-ASR / ASR-2 (held out) | **3.49 / 3.40 / 2.86 %** |
205
- | CER, scorer v2 (code-mix harmonised) | 2.35 / 2.22 / 1.73 % |
206
- | median per-clip CER, ASR-1 | 0.60 % |
207
- | real failures (2 ASRs agree), v1 / v2 scorer | **0.15 % / 0.03 %** (all 3 ASRs: 0.25 %) |
208
- | never stopped (length cap) | 0.00 % |
209
- | loops / truncations | 0.02 % / 0.13 % |
210
- | voice name spoken aloud (named voices) | 0.00 % |
211
- | voice identity (nearest of 20 centroids) | **98.6 %** (excluding Assamese: 98.8 %); mean ECAPA similarity 0.778 |
212
-
213
- Hardest categories, ASR-1 CER (scorer v1 / v2): web addresses 29.06 / 20.34 %, heavy code-mix
214
- 16.74 / 4.11 %, code-mix 10.60 / 3.51 %,
215
- symbols 6.34 / 2.49 %. Much of the code-mix CER is a script-spelling mismatch rather
216
- than a mispronunciation, which is why the v2 scorer is much lower.
217
-
218
- **Named voices (dedicated eval, 1248 rows, trained languages):**
219
- - identity 98.9 % against a ceiling of 98.0 % on real held-out recordings of the same voices;
220
- - cross-lingual identity 98.8 %;
221
- - same-voice vs different-voice cosine gap 0.392;
222
- - name spoken 0.00 %.
223
-
224
- **Zero-shot cloning (`icl_spk`):** supported; a cloning benchmark for this checkpoint will be published separately.
225
-
226
- **Streaming throughput and latency.** The whole bench was served as 224 concurrent WebSocket streams on
227
- **one H100 NVL (95 GB)**, with 224 server seats, bf16 backbone and fp16 depth decoder:
228
-
229
- | metric | value |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
230
  |---|---|
231
- | throughput | **323 s of audio per second** |
232
- | time to first audio, steady state, p50 / p90 / p99 | **111 / 134 / 242 ms** |
233
- | time to first audio, whole run incl. ramp and drain, p50 / p99 | 115 / 774 ms |
234
- | per-stream real-time factor at full load, p50 | 0.68 |
235
- | playback stalls / errors | 0 / 0 |
236
- | single stream (B=1, H100) | TTFA about 14 ms, RTF about 0.13 (7x faster than real time) |
 
237
 
238
- The whole-run p99 includes a drain tail in which the shortest 1-2-word texts are queued last. This tail is
239
- admission-bound, not a model property.
 
 
 
 
 
 
 
240
 
241
  ## Limitations
242
 
243
- - **Assamese is the weakest language.** CER is 5.41 % (ASR-1) and identity 96.4 %. None of the 20 voices has an hour of Assamese training audio, so Assamese is spoken cross-lingually and some voices drift towards a Bengali-like timbre.
244
- - **Very short prompts (1-2 words).** CER is acceptable (1.97 % / 2.28 %), but voice identity is much less stable: 95.5 % / 93.6 %. A one-word clip carries little speaker information, and the ECAPA judge is also unreliable at about 1 s. For short agent turns, prefer a slightly longer phrase.
245
- - **Code-mixed text** (Hinglish, Tanglish, ...) and text that mixes scripts has the highest CER (16.74 % on heavy code-mix). Pronunciation of English words inside Indic sentences varies by voice.
246
- - **Web addresses, symbols and long digit strings** depend on the bundled normaliser. Turning it off (`normalize=false`) makes them much worse. Unusual formats may still be read character by character.
247
- - **Cloning similarity.** Zero-shot clones are recognisably close to the reference but not identical, and noisy or very short references reduce it. Cross-lingual clones keep the timbre but can carry an accent from the reference language.
248
- - **Named voices** were trained with their name in the prompt. No benchmark clip spoke the name aloud, but it cannot be ruled out. Voices trained mostly on Hindi and English (see the table) sound less native in other languages.
249
- - **Emotion and event tags** are best effort and were not a focus of this release.
250
- - **Sampling.** Output is sampled. The same seed, text and voice is reproducible only when the request runs alone, because batching changes the random stream.
251
- - **Loop guard.** The engine has a loop guard and a duration cap, both on by default. They can end a genuinely repetitive text early. The quality numbers above are measured with the defaults.
252
 
253
- ## Intended use and misuse
254
 
255
- **Intended use:** voice interfaces and agents, accessibility, reading and narration, IVR and notifications, and
256
- research on multilingual Indic speech synthesis.
257
 
258
- **Out of scope / prohibited:**
259
- - **Impersonation:** do not use Maya3 to imitate a real person without their explicit consent, or to deceive anyone about who is speaking.
260
- - **Voice cloning without consent:** clone only voices you own or have documented permission to use. The reference speaker must have consented to having their voice cloned.
261
- - **Fraud and abuse:** this includes scams, social engineering, harassment, defeating voice-based authentication, political or electoral disinformation, and non-consensual content.
262
- - **Undisclosed synthetic speech:** tell listeners when speech is synthetic, especially in calls, agents and media.
263
 
264
- There is no audio watermark in this release.
265
- To report misuse: TBD.
266
 
267
- ## Training data
 
268
 
269
- Maya3 was trained on a mixture of **licensed and synthetic speech** in the 12 languages, with transcripts, plus
270
- text-only data used to warm up the backbone for the new languages.
271
- - The 20 named voices come from synthetic and licensed voice data, rendered or recorded for this purpose.
272
- - The preference stage (DPO) used the model's own generations, ranked by ASR agreement and speaker similarity.
273
- - No evaluation text from the benchmark above was used in training.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
274
 
275
  ## Citation
276
 
277
  ```bibtex
278
- @misc{maya3tts,
279
- title = {Maya3: Streaming Multilingual Text-to-Speech for Indic Languages},
280
- author = {Maya Research},
281
  year = {2026},
282
- howpublished = {\url{https://huggingface.co/maya-research/maya2-native}}
 
283
  }
284
  ```
285
 
286
- Maya3 builds on [Qwen3](https://huggingface.co/Qwen) (backbone), [Mimi](https://huggingface.co/kyutai/mimi)
287
- (codec) and [SpeechBrain ECAPA-TDNN](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb) (speaker embedding).
288
- Please also cite them. The backbone is initialised from [Qwen3-1.7B-Base](https://huggingface.co/Qwen/Qwen3-1.7B-Base) (Apache-2.0); Mimi is CC-BY-4.0 and SpeechBrain ECAPA is Apache-2.0.
 
 
 
 
 
 
 
 
 
 
26
  - mimi
27
  ---
28
 
29
+ <div align="center">
30
 
31
+ # Maya 2 Native
 
 
 
32
 
33
+ **Open speech. Native voices.**
34
 
35
+ Streaming text-to-speech for **11 Indic languages and English**.
36
 
37
+ **12 languages · 20 voices · Voice cloning · Apache-2.0**
38
 
39
+ [Listen](#listen-in-12-languages) · [Get started](#quickstart) · [Explore voices](#voices) · [Evaluation](#evaluation) · [Code](https://github.com/MayaResearch/maya2-native)
 
 
 
 
 
40
 
41
+ </div>
42
 
43
+ Maya 2 Native brings streaming speech generation to the languages people use every day. Built by **Maya Research**, it generates **24 kHz speech** in 12 languages, with **20 built-in voices** and **zero-shot voice cloning from 3–10 seconds of reference audio**. A voice can speak across languages: a Hindi reference can become Tamil speech.
 
 
44
 
45
+ Use it to give a voice agent its next turn, read a story in a familiar language, or build a multilingual interface you can host yourself. The release includes **open weights and inference code**, with a Python API, an OpenAI-compatible speech endpoint, and WebSocket streaming. Continuous batching and CUDA graphs support concurrent generation.
 
 
 
 
 
 
 
 
 
 
 
 
 
46
 
47
+ ## Watch Maya in action
48
 
49
+ <video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/maya-research/maya2-native/resolve/main/Maya%20Final%20with%20Changes.mp4" type="video/mp4">
50
+ <a href="https://drive.google.com/file/d/1WRd_e8TztpvX1n-DbOBlDj-zP89WE4oX/view">Watch the video on Google Drive</a>
51
+ </video>
52
+
53
+ [Open the video](https://huggingface.co/maya-research/maya2-native/resolve/main/Maya%20Final%20with%20Changes.mp4) · [View on Google Drive](https://drive.google.com/file/d/1WRd_e8TztpvX1n-DbOBlDj-zP89WE4oX/view)
54
+
55
+ ## At a glance
56
 
57
+ | Model at a glance | |
58
  |---|---|
59
+ | Languages | Assamese, Bengali, English, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, Telugu |
60
+ | Model size | 2.50B unique parameters; approximately 5.0 GB of BF16 weights |
61
+ | Audio | 24 kHz, mono; streaming in 80 ms audio frames |
62
+ | Voices | 20 built-in voices; zero-shot and cross-lingual voice cloning |
63
+ | Interfaces | Python, OpenAI-compatible speech endpoint, WebSocket streaming |
64
+ | License | Apache-2.0; see [component licenses](#license) |
65
+ | Checkpoint release | 1.0 · October 25, 2026 |
66
 
67
+ > **For developers:** the model is **Maya 2 Native**. The existing package, Python class, and configuration filename remain `maya3`, `Maya3TTS`, and `maya3_config.json`. Use the identifiers shown in the examples.
 
68
 
69
+ ## Listen in 12 languages
70
 
71
+ Listen to one example in each supported language. These are existing, unedited outputs from the release benchmark, generated with text normalization enabled and sampled decoding. The displayed text is the original input, before normalization.
72
+
73
+ **Selection:** these twelve clips were chosen from outputs with character error rate below 2% under all three ASR judges. They illustrate the model's capabilities; the [full benchmark results](#evaluation) describe average performance and harder cases.
74
+
75
+ | Language, voice, and input | Audio |
76
  |---|---|
77
+ | **Hindi · Ritvik**<br>क्या आपका वर्तमान पता 45, एमजी रोड, अंधेरी ईस्ट, मुंबई 400069 है? यदि हाँ, तो 1 दबाएं। | <audio controls preload="none" aria-label="Hindi, Ritvik" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/hi_ritvik.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/hi_ritvik.wav) |
78
+ | **Assamese · Nakul**<br>এন ই এফ টি লেনদেনটো সম্পূৰ্ণ হ'বলৈ প্ৰায় 2 ঘণ্টা 30 মিনিট সময় লাগিব পাৰে। | <audio controls preload="none" aria-label="Assamese, Nakul" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/as_nakul.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/as_nakul.wav) |
79
+ | **Bengali · Jahnavi**<br>12/08/2023 তারিখে আপনার 1,500 টাকার refund process হয়ে গেছে, status update চেক করুন। | <audio controls preload="none" aria-label="Bengali, Jahnavi" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/bn_jahnavi.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/bn_jahnavi.wav) |
80
+ | **English · Siddhant**<br>The EMI for your home loan is due by 12-10-24; please ensure your account has sufficient funds. | <audio controls preload="none" aria-label="English, Siddhant" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/en_siddhant.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/en_siddhant.wav) |
81
+ | **Gujarati · Vaidehi**<br>જી બિલકુલ, મને સિસ્ટમમાં દેખાય છે કે તમારું ₹2,500 નું પેમેન્ટ 15મી તારીખે સક્સેસફુલ થઈ ગયું છે. | <audio controls preload="none" aria-label="Gujarati, Vaidehi" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/gu_vaidehi.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/gu_vaidehi.wav) |
82
+ | **Kannada · Satvik**<br>ಹೌದಾ... ಅದು ನಿಜಕ್ಕೂ ಕಷ್ಟ. ನಿಮ್ಮ ಅಂತಿಮ ಬಿಲ್ ₹45,000 ಆಗಿದೆ, ಅದರಲ್ಲಿ ಇನ್ಶೂರೆನ್ಸ್ ಕವರ್ ಆಗಿದೆಯಾ ಅಂತ ಚೆಕ್ ಮಾಡ್ತೀನಿ. | <audio controls preload="none" aria-label="Kannada, Satvik" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/kn_satvik.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/kn_satvik.wav) |
83
+ | **Malayalam · Anirban**<br>റദ്ദാക്കിയ ടിക്കറ്റിന്റെ റീഫണ്ട് തുകയായ ₹1,800 മൂന്ന് ദിവസത്തിനുള്ളിൽ ലഭിക്കും. | <audio controls preload="none" aria-label="Malayalam, Anirban" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/ml_anirban.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/ml_anirban.wav) |
84
+ | **Marathi · Charvi**<br>मला 98765 43210 या नंबरवर OTP आला नाही, कृपया पुन्हा resend करा. | <audio controls preload="none" aria-label="Marathi, Charvi" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/mr_charvi.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/mr_charvi.wav) |
85
+ | **Odia · Ahana**<br>ଦୟାକରି ମନେରଖନ୍ତୁ ଯେ EMI ପ୍ରତ୍ୟେକ ମାସର 5 ତାରିଖରେ କଟିବ ଏବଂ ଏହା ଆପଣଙ୍କର 3ୟ କିସ୍ତି। | <audio controls preload="none" aria-label="Odia, Ahana" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/or_ahana.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/or_ahana.wav) |
86
+ | **Punjabi · Malini**<br>NEFT ਰਾਹੀਂ ਫੰਡ ਟ੍ਰਾਂਸਫਰ 00:30 ਤੋਂ 23:30 ਤੱਕ ਉਪਲਬਧ ਹਨ। | <audio controls preload="none" aria-label="Punjabi, Malini" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/pa_malini.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/pa_malini.wav) |
87
+ | **Tamil · Abhay**<br>கடந்த நிதியாண்டில் வங்கியின் மொத்த லாபம் ₹3.5 கோடி, இதில் ₹45 லட்சம் முதலீட்டாளர்களுக்கு வழங்கப்பட்டது. | <audio controls preload="none" aria-label="Tamil, Abhay" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/ta_abhay.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/ta_abhay.wav) |
88
+ | **Telugu · Nivedita**<br>సేవింగ్స్ తో పోలిస్తే ఫిక్స్డ్ డిపాజిట్ వడ్డీ నిష్పత్తి 3:2 గా ఉండి, 7.25% ఇస్తుంది. | <audio controls preload="none" aria-label="Telugu, Nivedita" src="https://huggingface.co/maya-research/maya2-native/resolve/main/samples/te_nivedita.wav"></audio><br>[Open WAV](https://huggingface.co/maya-research/maya2-native/resolve/main/samples/te_nivedita.wav) |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
89
 
90
+ The Tamil sample uses **Abhay**, whose listed training languages are English and Hindi. The Telugu sample uses **Nivedita**, also trained on English and Hindi. Both demonstrate cross-lingual synthesis. Assamese is cross-lingual for every built-in voice; see [limitations](#limitations).
91
 
92
+ ## Quickstart
93
+
94
+ Use **Linux, Python 3.10–3.12, and an NVIDIA GPU with a driver supporting CUDA 12.8**. Start with `max_seats=1`. The documented single-seat loading peak is approximately 9.2 GB plus CUDA context; a 12 GB or larger GPU provides more room. See [hardware and memory](#hardware-and-memory) before increasing concurrency.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
95
 
96
  ```bash
97
+ python -m venv .venv
98
+ source .venv/bin/activate
99
+ pip install "maya3[server] @ git+https://github.com/MayaResearch/maya2-native" \
100
+ --extra-index-url https://download.pytorch.org/whl/cu128
101
+ ```
102
+
103
+ ### Generate speech
104
+
105
+ ```python
106
+ import soundfile as sf
107
+ from maya3 import Maya3TTS
108
+
109
+ tts = Maya3TTS.from_pretrained("maya-research/maya2-native", max_seats=1)
110
+
111
+ audio = tts.generate(
112
+ "नमस्ते, आपका ऑर्डर कल पहुँच जाएगा।",
113
+ lang="hi",
114
+ voice="Ahana",
115
+ normalize=True,
116
+ )
117
+ sf.write("speech.wav", audio, 24000)
118
+ tts.close()
119
  ```
120
 
121
+ `generate()` returns a mono NumPy waveform in `float32`. Model weights, the Mimi codec, and the ECAPA speaker encoder are downloaded on first use. For offline inference, cache all three first, then set `HF_HUB_OFFLINE=1`.
122
+
123
+ **Normalization differs by interface:** the HTTP and WebSocket servers enable it by default. The Python API requires `normalize=True`. It expands numbers, currency, dates, times, URLs, email addresses, and other written forms into spoken text. Check the output for unusual formats.
124
+
125
+ <details>
126
+ <summary><strong>Stream audio as it is generated</strong></summary>
127
+
128
+ ### Stream speech
129
+
130
  ```python
131
  from maya3 import Maya3TTS
132
+
133
+ tts = Maya3TTS.from_pretrained("maya-research/maya2-native", max_seats=1)
134
+ try:
135
+ with open("speech.pcm", "wb") as output:
136
+ for chunk in tts.stream(
137
+ "Your order has shipped and will arrive tomorrow.",
138
+ lang="en",
139
+ voice="Siddhant",
140
+ normalize=True,
141
+ ):
142
+ output.write(chunk.astype("<i2", copy=False).tobytes())
143
+ finally:
144
+ tts.close()
145
+ ```
146
+
147
+ `stream()` yields signed 16-bit PCM arrays at 24 kHz, mono. `astream()` provides the asynchronous interface. Concurrent requests are batched by the engine; size `max_seats` to your hardware.
148
+
149
+ </details>
150
+
151
+ ### Clone a voice across languages
152
+
153
+ Use a clean, single-speaker reference clip and its exact transcript. Obtain the speaker's permission before cloning their voice. The reference language and the output language can differ.
154
+
155
+ ```python
156
  import soundfile as sf
157
+ from maya3 import Maya3TTS
158
 
159
+ tts = Maya3TTS.from_pretrained("maya-research/maya2-native", max_seats=1)
160
+ tts.register_voice(
161
+ "my_voice",
162
+ ref_audio="reference.wav", # 3–10 seconds recommended
163
+ ref_text="<exact transcript of reference.wav>",
164
+ lang="hi", # reference language
165
+ )
166
+ audio = tts.generate(
167
+ "வணக்கம், எப்படி இருக்கிறீர்கள்?",
168
+ lang="ta", # output language
169
+ voice="my_voice",
170
+ normalize=True,
171
+ )
172
+ sf.write("cloned_voice_tamil.wav", audio, 24000)
173
+ tts.close()
174
+ ```
175
+
176
+ The default cloning mode, `icl_spk`, combines the reference audio with an ECAPA speaker embedding. Clones approximate the reference voice; similarity and accent vary. A dedicated cloning benchmark is not included in this release's reported results.
177
 
178
+ <details>
179
+ <summary><strong>Run a local HTTP and WebSocket server</strong></summary>
180
 
181
+ ### Serve over HTTP or WebSocket
 
 
182
 
183
+ ```bash
184
+ python -m maya3.server \
185
+ --ckpt maya-research/maya2-native \
186
+ --host 127.0.0.1 --port 8080 --max-seats 1
187
  ```
188
 
189
+ After `GET /health` reports ready, generate speech:
190
 
191
  ```bash
192
+ curl --fail-with-body -sS http://127.0.0.1:8080/v1/audio/speech \
193
+ -H 'Content-Type: application/json' \
194
+ -d '{"input":"Your order has shipped.","language":"en","voice":"Ahana","response_format":"wav"}' \
195
+ --output speech.wav
196
  ```
197
 
198
+ The server provides `POST /v1/audio/speech`, `WS /v1/tts/stream`, and voice registration endpoints. It has no built-in authentication; add authentication and access controls before exposing it beyond a trusted environment. See the [GitHub README](https://github.com/MayaResearch/maya2-native#serve-the-model) for clients and protocol details.
199
+
200
+ </details>
201
+
202
+ ## Voices
203
+
204
+ Select a built-in voice by name, for example `voice="Vaidehi"`. Names are case-insensitive, and no reference audio is needed. All 20 names describe synthetic voice personas.
205
+
206
+ The training-language column lists languages with **at least one hour of training audio for that voice**. A voice can be requested in any supported language, but performance outside its listed languages is cross-lingual and may sound less native.
207
+
208
+ | Voice | Profile | Training languages, ≥1 hour |
209
+ |---|---|---|
210
+ | **Ahana** | Young female voice, bright, youthful | `bn en gu hi kn ml mr or pa ta te` |
211
+ | **Vaidehi** | Adult female voice, calm, gentle | `bn en gu hi kn ml mr or pa ta te` |
212
+ | **Malini** | Mature female voice, steady narrator | `bn en gu hi kn ml mr or pa ta te` |
213
+ | **Charvi** | Young female voice, energetic, upbeat | `bn en gu hi kn ml mr or pa ta te` |
214
+ | **Jahnavi** | Adult female voice, warm, friendly | `bn en gu hi kn ml mr or ta te` |
215
+ | **Anirban** | Adult male voice, steady narrator | `bn en gu hi kn ml mr or pa ta te` |
216
+ | **Nakul** | Young male voice, bright, youthful | `bn en gu hi kn ml mr or pa ta te` |
217
+ | **Vasant** | Mature male voice, steady narrator | `bn en gu hi kn ml mr or ta te` |
218
+ | **Siddhant** | Adult male voice, clear, professional | `bn en gu hi kn ml mr or pa ta te` |
219
+ | **Satvik** | Young male voice, warm, friendly | `bn en gu hi kn ml mr or pa ta te` |
220
+ | **Abhay** | Adult male voice, natural conversational style | `en hi` |
221
+ | **Nivedita** | Adult female voice, natural conversational style | `en hi` |
222
+ | **Ishwari** | Adult female voice, natural conversational style | `hi` |
223
+ | **Ritvik** | Adult male voice, natural conversational style | `hi` |
224
+ | **Vrinda** | Young female voice, natural conversational style | `hi` |
225
+ | **Sreekar** | Adult male voice, natural conversational style | `en hi te` |
226
+ | **Kayal** | Adult female voice, natural conversational style | `en ta` |
227
+ | **Shaunak** | Young male voice, natural conversational style | `en hi` |
228
+ | **Akshat** | Adult male voice, natural conversational style | `en hi` |
229
+ | **Tanushree** | Adult female voice, natural conversational style | `en hi` |
230
+
231
+ Language codes: `as` Assamese · `bn` Bengali · `en` English · `gu` Gujarati · `hi` Hindi · `kn` Kannada · `ml` Malayalam · `mr` Marathi · `or` Odia · `pa` Punjabi · `ta` Tamil · `te` Telugu.
232
+
233
+ Pass only the speech text and the voice name to the API. The engine constructs the voice prompt internally; adding `Ahana:` to your text is unnecessary. Use either a built-in voice or your own reference audio for a request, rather than combining them.
234
 
235
  ## Evaluation
236
 
237
+ The release was evaluated on **6,000 held-out texts across 12 languages**, producing **20.3 hours of speech**. Three ASR judges measured intelligibility; a speaker-embedding classifier measured consistency among the 20 built-in voices.
238
+
239
+ **These are Maya Research's release measurements.** ASR error rates and embedding similarity capture specific aspects of quality; they do not replace human listening or measure every aspect of naturalness. Serving measurements below describe the tested hardware and workload.
240
+
241
+ ### Benchmark design
242
+
243
+ The release benchmark, **bench6k**, contains **6,000 held-out texts: 500 per language across 31 content categories**, including conversation, names, numbers, dates, currency, addresses, URLs, code-mixed text, tongue twisters, and prompts of 1–50 words. The release documentation states that these evaluation texts were excluded from training.
244
+
245
+ The benchmark produced **20.3 hours of speech** through the server, with normalization and engine guards enabled, sampled decoding, and one seed per row. Voices were distributed across languages and categories; long inputs used the serving stack's text chunking.
246
+
247
+ Three ASR systems transcribed the generated speech:
248
+
249
+ - **ASR-1:** Maya's in-house Indic ASR and primary judge.
250
+ - **Qwen3-ASR:** a third ASR judge.
251
+ - **ASR-2:** a second in-house Indic ASR, held out from training, reward generation, and checkpoint selection.
252
+
253
+ **Character error rate (CER)** compares ASR transcripts with spoken-form references. Scorer v1 uses the benchmark's normalization and script harmonization. Scorer v2 additionally harmonizes Latin and native-script spellings in code-mixed spans. The two scorers evaluate the same generated audio; lower v2 scores reflect a scoring change.
254
+
255
+ | Overall CER ↓ | ASR-1 | Qwen3-ASR | ASR-2, held out |
256
+ |---|---:|---:|---:|
257
+ | Scorer v1 | 3.49% | 3.40% | 2.86% |
258
+ | Scorer v2, additional code-mix harmonization | 2.35% | 2.22% | 1.73% |
259
+
260
+ ### Results by language
261
+
262
+ CER below uses **scorer v1**. Voice identity is nearest-centroid classification among the 20 built-in voices, using ECAPA embeddings from clips of at least one second. It is a closed-set identity metric, separate from zero-shot clone similarity.
263
+
264
+ | Language | Code | CER ASR-1 ↓ | CER Qwen3-ASR ↓ | CER ASR-2 ↓ | Voice identity ↑ |
265
+ |---|---|---:|---:|---:|---:|
266
+ | Assamese | `as` | 5.41% | 6.59% | 5.54% | 96.4% |
267
+ | Bengali | `bn` | 2.60% | 2.83% | 2.68% | 98.5% |
268
+ | English | `en` | 9.69% | 1.74% | 1.43% | 99.6% |
269
+ | Gujarati | `gu` | 2.25% | 3.76% | 2.26% | 98.9% |
270
+ | Hindi | `hi` | 1.52% | 2.05% | 1.57% | 99.6% |
271
+ | Kannada | `kn` | 2.28% | 2.72% | 2.23% | 97.9% |
272
+ | Malayalam | `ml` | 3.75% | 4.20% | 4.03% | 99.4% |
273
+ | Marathi | `mr` | 1.66% | 2.09% | 1.54% | 99.1% |
274
+ | Odia | `or` | 3.45% | 4.33% | 3.54% | 97.4% |
275
+ | Punjabi | `pa` | 2.51% | 3.32% | 2.66% | 99.1% |
276
+ | Tamil | `ta` | 4.42% | 4.55% | 4.33% | 98.1% |
277
+ | Telugu | `te` | 2.29% | 2.64% | 2.53% | 99.4% |
278
+
279
+ ASR-1's English CER is substantially higher than the other two judges. The release analysis attributes part of this difference to ASR-1 looping on spoken URLs such as “W W W.” All three results are retained to show judge disagreement.
280
+
281
+ ### Failures and voice consistency
282
+
283
+ The benchmark's **consensus failure rate** requires agreement between ASR-1 and Qwen3-ASR under the release's failure rules, covering looping, substantial insertions, very high CER, and duration-limit events.
284
+
285
+ | Release measurement | Result |
286
+ |---|---:|
287
+ | Consensus failure rate, scorer v1 | 0.15% |
288
+ | Consensus failure rate, scorer v2 | 0.03% |
289
+ | Outputs reaching the absolute length cap | 0.00% |
290
+ | Loops / truncations | 0.02% / 0.13% |
291
+ | Built-in voice name spoken aloud | 0.00% |
292
+ | Voice identity, nearest of 20 centroids | 98.6% |
293
+ | Mean ECAPA similarity | 0.778 |
294
+
295
+ Zero observed events apply to this finite benchmark. A consensus failure rate does not count every pronunciation error or one-judge disagreement.
296
+
297
+ A separate 1,248-row named-voice evaluation reports 98.9% identity in trained languages and 98.8% cross-lingual identity. The same classifier scored 98.0% on held-out recordings of the source voices. These are task-specific measurements, rather than evidence that generated speech is more natural than recorded speech.
298
+
299
+ ### Harder inputs
300
+
301
+ | Category | ASR-1 CER, v1 | ASR-1 CER, v2 |
302
+ |---|---:|---:|
303
+ | Web addresses | 29.06% | 20.34% |
304
+ | Heavy code-mix | 16.74% | 4.11% |
305
+ | Code-mix | 10.60% | 3.51% |
306
+ | Symbols | 6.34% | 2.49% |
307
+
308
+ The code-mix gap illustrates the effect of script spelling on ASR-based scoring. Pronunciation and intelligibility still need listening evaluation, especially for names, URLs, and critical numbers.
309
+
310
+ ### Streaming performance
311
+
312
+ The reported release serving run used **one H100 NVL with 95 GB reported memory**, **224 concurrent WebSocket streams**, a BF16 backbone, and an FP16 depth decoder.
313
+
314
+ | Metric | Reported result |
315
+ |---|---:|
316
+ | Concurrent streams | 224 |
317
+ | Aggregate throughput | 323 seconds of audio per wall-clock second |
318
+ | Time to first audio, steady state, p50 / p90 / p99 | 111 / 134 / 242 ms |
319
+ | Time to first audio, complete run including ramp and drain, p50 / p99 | 115 / 774 ms |
320
+ | Per-stream real-time factor at full load, p50 | 0.68 |
321
+ | Observed playback stalls / serving errors | 0 / 0 |
322
+
323
+ A separate single-stream H100 measurement reports approximately **14 ms to first audio** and **0.13 real-time factor**. It uses a different load condition from the 224-stream run. Real-time factor is generation time divided by audio duration; lower is faster.
324
+
325
+ The release analysis attributes the complete-run latency tail partly to admission queuing during the drain phase. Latency depends on hardware, load, input length, cached state, and the serving environment; these measurements do not establish public internet or telephone latency.
326
+
327
+ ## Hardware and memory
328
+
329
+ The engine documentation reports the following GPU allocator measurements with a BF16 backbone, FP16 depth decoder, and **2,048 KV-cache tokens per seat**. Add approximately **0.5 GB for the CUDA context** and allow additional deployment headroom.
330
+
331
+ | Concurrent seats | Steady GPU memory | Peak during loading |
332
+ |---|---:|---:|
333
+ | 1 | 7.7 GB | 9.2 GB |
334
+ | 8 | 9.7 GB | 10.9 GB |
335
+ | 16 | 11.7 GB | 12.8 GB |
336
+ | 32 | 15.8 GB | 16.5 GB |
337
+
338
+ `max_seats` controls allocated serving capacity and memory, even before every seat is busy. These figures describe this configuration; a checkpoint's download size is smaller than its total runtime memory requirement. See [engine documentation](https://github.com/MayaResearch/maya2-native/blob/main/docs/ENGINE.md) for configuration and hardware-specific measurements.
339
+
340
+ <details>
341
+ <summary><strong>Architecture and audio generation</strong></summary>
342
+
343
+ ## Architecture
344
+
345
+ Maya 2 Native generates discrete audio tokens autoregressively, then decodes them into speech with Mimi.
346
+
347
+ | Component | Description |
348
  |---|---|
349
+ | Text and audio backbone | Initialized from [Qwen3-1.7B-Base](https://huggingface.co/Qwen/Qwen3-1.7B-Base); 28 layers, hidden size 2,048, 16 attention heads and 8 KV heads; extended vocabulary for language and audio tokens |
350
+ | Depth decoder | Approximately 555M parameters; predicts codebooks 1–31 after the backbone predicts codebook 0 |
351
+ | Audio codec | [Kyutai Mimi](https://huggingface.co/kyutai/mimi); 32 codebooks with 2,048 entries each, at 12.5 frames per second |
352
+ | Speaker conditioning | 192-dimensional [SpeechBrain ECAPA-TDNN](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb) embedding projected into the backbone; reference-audio conditioning for cloning |
353
+ | Output | 24 kHz mono speech, 80 ms per audio frame |
354
+
355
+ At each step, the backbone predicts the first codebook, the depth decoder fills the remaining codebooks, and Mimi decodes an audio frame. The engine can emit that frame while generation continues. An end-of-audio token controls normal stopping; duration and loop guards provide additional controls.
356
 
357
+ </details>
358
+
359
+ ## Training data and procedure
360
+
361
+ The release documentation describes a mixture of **licensed and synthetic speech with transcripts** across the 12 languages, plus text-only data used to adapt the backbone to the languages. The built-in voice personas use synthetic and licensed voice data prepared for this purpose.
362
+
363
+ The preference-optimization stage used **direct preference optimization (DPO)** on model-generated candidates ranked by ASR agreement and speaker similarity. Benchmark texts were excluded from training, according to the release documentation.
364
+
365
+ Dataset identities, total training hours, detailed per-language composition, and the full training configuration are not disclosed in this release. Automated scores should be considered alongside these disclosure limits.
366
 
367
  ## Limitations
368
 
369
+ - **Uneven language coverage.** Assamese has the highest CER in the release benchmark. No built-in voice has at least one hour of Assamese training data; some outputs drift toward a Bengali-like timbre. Other cross-lingual combinations can also carry the voice's training-language accent.
370
+ - **Very short turns.** One- or two-word prompts are less stable in voice identity, and speaker-embedding evaluation is less reliable on very short clips. A slightly longer phrase may help short agent responses.
371
+ - **Code-mixed and written-form inputs.** English words inside Indic sentences vary by voice. URLs, symbols, dates, and long digit sequences remain difficult, even with normalization. Listen to application-specific examples before relying on them.
372
+ - **Voice cloning.** Similarity is approximate. Noise, multiple speakers, short references, and inaccurate transcripts can degrade results. Dedicated zero-shot cloning scores are not reported here.
373
+ - **Emotion and events.** Tags such as `[laughs]`, `[sighs]`, `[whispers]`, `[sad]`, and `[excited]` are best effort and were not the focus of this release.
374
+ - **Generation and guards.** Sampling can produce errors. The same seed is reproducible for an isolated request, but batching changes the random stream. Loop and duration guards can end genuinely repetitive text early. No benchmark clip spoke a built-in voice name aloud, but this remains a possible failure mode.
375
+ - **Evaluation scope.** Results use automated ASR and speaker-embedding judges. This card does not report a human naturalness study, comprehensive dialect or demographic evaluation, confidence intervals, or a controlled comparison with other TTS models.
 
 
376
 
377
+ ## Intended use and responsible deployment
378
 
379
+ Maya 2 Native is intended for voice interfaces and agents, accessibility, reading and narration, IVR, notifications, and research on multilingual speech generation.
 
380
 
381
+ Clone only voices you own or have explicit permission to use. Disclose synthetic speech to listeners, especially in calls and media. Do not use the model for deceptive impersonation, fraud, harassment, non-consensual content, disinformation, or attempts to defeat voice authentication. Built-in voice names identify synthetic personas and should not be presented as real people.
 
 
 
 
382
 
383
+ **This release does not add an audio watermark.** Applications should implement the consent, disclosure, access controls, and abuse monitoring appropriate to their use. Consult the repository's [security guidance](https://github.com/MayaResearch/maya2-native/blob/main/SECURITY.md) for deployment considerations.
 
384
 
385
+ <details>
386
+ <summary><strong>Files in this repository</strong></summary>
387
 
388
+ ## Files in this repository
389
+
390
+ | Path | Contents |
391
+ |---|---|
392
+ | `model-00001-of-00002.safetensors`, `model-00002-of-00002.safetensors` | BF16 model weight shards |
393
+ | `model.safetensors.index.json` | Weight-to-shard mapping |
394
+ | `maya3_config.json` | Resolved model configuration and serialization metadata |
395
+ | `backbone/` | Backbone configuration and tokenizer; backbone weights are included in the main model shards |
396
+ | `voices/voices.json` | Built-in voice metadata and conditioning modes |
397
+ | `samples/` | The twelve WAV examples above |
398
+
399
+ Tied weights are stored once and represented through aliases. Mimi and the ECAPA speaker encoder are resolved from their own repositories on first use.
400
+
401
+ </details>
402
+
403
+ ## License
404
+
405
+ The model weights are released under **Apache-2.0**, as specified in this repository's license metadata. The [inference code](https://github.com/MayaResearch/maya2-native) is also Apache-2.0.
406
+
407
+ The upstream components retain their own licenses: **Qwen3-1.7B-Base** (Apache-2.0), **Kyutai Mimi** (CC-BY-4.0), and **SpeechBrain ECAPA-TDNN** (Apache-2.0). Preserve their notices and attribution when redistributing or using these components.
408
 
409
  ## Citation
410
 
411
  ```bibtex
412
+ @misc{maya2native2026,
413
+ title = {Maya 2 Native: Streaming Multilingual Text-to-Speech},
414
+ author = {{Maya Research}},
415
  year = {2026},
416
+ url = {https://huggingface.co/maya-research/maya2-native},
417
+ note = {Release 1.0}
418
  }
419
  ```
420
 
421
+ ## Credits and acknowledgments
422
+
423
+ **Created by [Maya Research](https://huggingface.co/maya-research).**
424
+
425
+ Maya 2 Native builds on the work of the open research community. We thank:
426
+
427
+ - **The Qwen team** for [Qwen3-1.7B-Base](https://huggingface.co/Qwen/Qwen3-1.7B-Base), which initializes the text and audio backbone.
428
+ - **Kyutai** for [Mimi](https://huggingface.co/kyutai/mimi), the neural audio codec used to represent and decode speech.
429
+ - **The SpeechBrain team** for [ECAPA-TDNN](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb), used for speaker conditioning and voice-identity evaluation.
430
+ - **The Hugging Face community** for the tools and infrastructure that make sharing models and building on them easier.
431
+
432
+ Please cite Maya 2 Native and acknowledge the upstream work relevant to your use.