AGENT_GUIDE.md · Infinix-ai/maya2-native at main

maya2-native / AGENT_GUIDE.md

Infinix-ai

bharathkumarK

Duplicate from maya-research/maya2-native

58644b6 9 days ago

preview code

Raw History Blame Contribute Delete

9.29 kB

# Maya 2 Native: agent integration guide

Reviewed: November 3, 2026. Model: `https://huggingface.co/maya-research/maya2-native`. This guide describes the downloadable checkpoint and reference inference engine. It contains no credentials and does not require a hosted Maya API account.

## Choose the model

Maya 2 Native converts written text into spoken audio. It is intended for streaming Indian-language voice agents, accessibility, reading, narration, IVR and permitted voice cloning. It is not ASR or a text-chat LLM. Speech recognition and conversational orchestration are separate pipeline components.

Supported languages and Python `lang` values:

| Language | Code |
|---|---|
| Assamese | `as` |
| Bengali | `bn` |
| English | `en` |
| Gujarati | `gu` |
| Hindi | `hi` |
| Kannada | `kn` |
| Malayalam | `ml` |
| Marathi | `mr` |
| Odia | `or` |
| Punjabi | `pa` |
| Tamil | `ta` |
| Telugu | `te` |

## Identity and source of truth

- Public model name: **Maya 2 Native**; aliases: **Maya2 Native**, **Maya-2-Native**.
- Model ID: `maya-research/maya2-native`.
- Package: `maya3`; loader: `from maya3 import Maya3TTS`.
- Configuration file: `maya3_config.json`.
- [Full model card](https://huggingface.co/maya-research/maya2-native/resolve/main/README.md), [inference-engine source](https://github.com/MayaResearch/maya2-native), [resolved model configuration](https://huggingface.co/maya-research/maya2-native/resolve/main/maya3_config.json).
- `model-discovery.json` is a descriptive manifest, not a standard framework plugin or executable tool definition.

The implementation retains its earlier internal `maya3` identifiers. Use them exactly as shown; changing them to `maya2` breaks the documented imports. This naming does not indicate a separate public Maya 3 release.

## Runtime and installation

The reference engine targets Linux, Python 3.10–3.12 and an NVIDIA GPU whose driver supports CUDA 12.8. The documented single-seat loading peak is approximately 9.2 GB plus CUDA context; a 12 GB or larger GPU provides more room. Increase capacity only after checking [memory measurements](https://huggingface.co/maya-research/maya2-native#hardware-and-memory).

```bash
python -m venv .venv
source .venv/bin/activate
pip install "maya3[server] @ git+https://github.com/MayaResearch/maya2-native" \
  --extra-index-url https://download.pytorch.org/whl/cu128
```

This installs the engine from the official source repository. A generic Transformers text-to-speech pipeline is not the documented loader. Do not infer CPU or Apple MPS support from the presence of Safetensors weights.

## Generate or stream speech

```python
from maya3 import Maya3TTS
import soundfile as sf
tts = Maya3TTS.from_pretrained("maya-research/maya2-native", max_seats=1)
try:
    waveform = tts.generate(
        "नमस्ते, आपका ऑर्डर कल पहुँच जाएगा।",
        lang="hi", voice="Ahana", normalize=True,
    )
    sf.write("speech.wav", waveform, 24000)
    with open("speech.pcm", "wb") as output:
        for pcm in tts.stream(
            "నమస్కారం, మీరు ఎలా ఉన్నారు?",
            lang="te", voice="Sreekar", normalize=True,
        ):
            output.write(pcm.astype("<i2", copy=False).tobytes())
finally:
    tts.close()
```

- `generate()` returns a float32 NumPy waveform, 24 kHz mono.
- `stream()` yields signed int16 PCM arrays; serialized chunks above are little-endian PCM (`s16le`).
- `astream()` is the asynchronous streaming interface.
- Raw PCM has no WAV header; a playback client needs 24,000 Hz, one channel and signed 16-bit little-endian samples.
- Model audio frames are 80 ms. Transport chunks and first-audio latency are separate quantities.
- `max_seats` allocates concurrent engine capacity and memory, including when seats are idle.

## Voice selection and cloning

Select one of the 20 checkpoint voices by name, case-insensitively. The authoritative [voice file](https://huggingface.co/maya-research/maya2-native/resolve/main/voices/voices.json) lists training-language coverage; the [model-card table](https://huggingface.co/maya-research/maya2-native#voices) explains the at-least-one-hour criterion. This coverage does not mean every voice has equal quality in all languages.

Built-in voice IDs: Ahana, Vaidehi, Malini, Charvi, Jahnavi, Anirban, Nakul, Vasant, Siddhant, Satvik, Abhay, Nivedita, Ishwari, Ritvik, Vrinda, Sreekar, Kayal, Shaunak, Akshat, Tanushree.

No built-in voice has at least one hour of Assamese training audio. Assamese generation is cross-lingual; pronunciation and timbre need listening evaluation. For any language, prefer a voice with relevant coverage when available.

```python
tts.register_voice(
    "my_voice", ref_audio="reference.wav",
    ref_text="<exact transcript of reference.wav>", lang="hi",
)
audio = tts.generate(
    "வணக்கம், எப்படி இருக்கிறீர்கள்?",
    lang="ta", voice="my_voice", normalize=True,
)
```

Use a clean single-speaker reference, 3–10 seconds recommended, with the speaker's permission. Reference and output languages may differ. Pass the voice separately; do not insert a voice-name prefix into the speech text or combine a built-in named voice with your own reference audio. Cloning similarity is approximate.

## Local HTTP and WebSocket contract

```bash
python -m maya3.server --ckpt maya-research/maya2-native \
  --host 127.0.0.1 --port 8080 --max-seats 1
```

Wait for `GET http://127.0.0.1:8080/health` to return HTTP 200. During loading it returns 503. Then:

```bash
curl --fail-with-body -sS http://127.0.0.1:8080/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"model":"maya3","input":"नमस्ते।","language":"hi","voice":"Ahana","response_format":"wav","normalize":true}' \
  --output speech.wav
```

| Interface | Contract |
|---|---|
| `POST /v1/audio/speech` | OpenAI-compatible speech shape with Maya extensions such as `language` and `normalize` |
| `WS /v1/tts/stream` | Local `metadata`, `speak`, `audio`, `end`, `cancel` and error messages; context IDs identify requests |
| HTTP `pcm` | Streaming raw PCM s16le, 24 kHz mono |
| HTTP `wav` | Streamed WAV wrapper around PCM; streaming header sizes may be unspecified |
| HTTP `mp3` / `opus` | Encoded streaming output, subject to installed encoder support |
| HTTP `flac` | Sent after generation completes |
| `GET /v1/voices` | Inspect registered voices |
| `POST /v1/voices` | Register a permitted reference voice |
| Local speed | `1.0`; do not copy hosted API speed ranges into this engine |

Use the [server source and protocol](https://raw.githubusercontent.com/MayaResearch/maya2-native/main/maya3/server.py) for complete messages and error handling, and [client examples](https://github.com/MayaResearch/maya2-native/tree/main/examples) for callers. HTTP and WebSocket servers normalize by default; Python callers explicitly pass `normalize=True`. Explicit language selection avoids ambiguous script detection for shared scripts or code-mixed input.

The local WebSocket protocol accepts a `speak` request and returns generated audio incrementally. This is not the hosted Maya API's protocol. A voice-agent application must handle text aggregation, user turns, cancellation and discarding buffered playback. The local server has no built-in authentication; deployment access controls belong to the application environment.

## Hosted API and framework distinction

The downloadable model and Maya's hosted service have different contracts. Hosted voice IDs such as Ananya or Arjun do not appear in this checkpoint's built-in roster. Do not substitute hosted endpoints, credentials or voice IDs into the examples above.

[Maya Research's Pipecat integration](https://github.com/MayaResearch/pipecat-maya) documents the hosted service and its supported Pipecat version. Check its current README before choosing an adapter. The reference engine's OpenAI-compatible endpoint can be used by suitable clients, but it is not proof of a maintained native adapter for every voice-agent framework.

## Offline use, licence and evaluation

First use downloads the model, [Mimi codec](https://huggingface.co/kyutai/mimi) and [ECAPA speaker encoder](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb). Cache all three and the installed dependencies before enabling `HF_HUB_OFFLINE=1` in an offline environment.

Model weights and inference code are Apache-2.0. Mimi is CC-BY-4.0; other upstream licences and notices remain applicable. See the [licence section](https://huggingface.co/maya-research/maya2-native#license) and [credits](https://huggingface.co/maya-research/maya2-native#credits-and-acknowledgments).

The full card distinguishes Maya Research's automated release evaluation from external human-preference rankings. Hardware-specific serving results are not latency promises for a public API, phone call or different GPU. Dynamic leaderboard claims require a date, language and filter; consult the [current benchmark source](https://voicearena.com/tts-leaderboard/hindi). Very short turns, Assamese, URLs, symbols, digits and heavily code-mixed input need application-specific evaluation. Emotion and event tags are best effort.

	# Maya 2 Native: agent integration guide

	Reviewed: November 3, 2026. Model: `https://huggingface.co/maya-research/maya2-native`. This guide describes the downloadable checkpoint and reference inference engine. It contains no credentials and does not require a hosted Maya API account.

	## Choose the model

	Maya 2 Native converts written text into spoken audio. It is intended for streaming Indian-language voice agents, accessibility, reading, narration, IVR and permitted voice cloning. It is not ASR or a text-chat LLM. Speech recognition and conversational orchestration are separate pipeline components.

	Supported languages and Python `lang` values:

	\| Language \| Code \|
	\|---\|---\|
	\| Assamese \| `as` \|
	\| Bengali \| `bn` \|
	\| English \| `en` \|
	\| Gujarati \| `gu` \|
	\| Hindi \| `hi` \|
	\| Kannada \| `kn` \|
	\| Malayalam \| `ml` \|
	\| Marathi \| `mr` \|
	\| Odia \| `or` \|
	\| Punjabi \| `pa` \|
	\| Tamil \| `ta` \|
	\| Telugu \| `te` \|

	## Identity and source of truth

	- Public model name: Maya 2 Native; aliases: Maya2 Native, Maya-2-Native.
	- Model ID: `maya-research/maya2-native`.
	- Package: `maya3`; loader: `from maya3 import Maya3TTS`.
	- Configuration file: `maya3_config.json`.
	- [Full model card](https://huggingface.co/maya-research/maya2-native/resolve/main/README.md), [inference-engine source](https://github.com/MayaResearch/maya2-native), [resolved model configuration](https://huggingface.co/maya-research/maya2-native/resolve/main/maya3_config.json).
	- `model-discovery.json` is a descriptive manifest, not a standard framework plugin or executable tool definition.

	The implementation retains its earlier internal `maya3` identifiers. Use them exactly as shown; changing them to `maya2` breaks the documented imports. This naming does not indicate a separate public Maya 3 release.

	## Runtime and installation

	The reference engine targets Linux, Python 3.10–3.12 and an NVIDIA GPU whose driver supports CUDA 12.8. The documented single-seat loading peak is approximately 9.2 GB plus CUDA context; a 12 GB or larger GPU provides more room. Increase capacity only after checking [memory measurements](https://huggingface.co/maya-research/maya2-native#hardware-and-memory).

	```bash
	python -m venv .venv
	source .venv/bin/activate
	pip install "maya3[server] @ git+https://github.com/MayaResearch/maya2-native" \
	--extra-index-url https://download.pytorch.org/whl/cu128
	```

	This installs the engine from the official source repository. A generic Transformers text-to-speech pipeline is not the documented loader. Do not infer CPU or Apple MPS support from the presence of Safetensors weights.

	## Generate or stream speech

	```python
	from maya3 import Maya3TTS
	import soundfile as sf

	tts = Maya3TTS.from_pretrained("maya-research/maya2-native", max_seats=1)
	try:
	waveform = tts.generate(
	"नमस्ते, आपका ऑर्डर कल पहुँच जाएगा।",
	lang="hi", voice="Ahana", normalize=True,
	)
	sf.write("speech.wav", waveform, 24000)
	with open("speech.pcm", "wb") as output:
	for pcm in tts.stream(
	"నమస్కారం, మీరు ఎలా ఉన్నారు?",
	lang="te", voice="Sreekar", normalize=True,
	):
	output.write(pcm.astype("<i2", copy=False).tobytes())
	finally:
	tts.close()
	```

	- `generate()` returns a float32 NumPy waveform, 24 kHz mono.
	- `stream()` yields signed int16 PCM arrays; serialized chunks above are little-endian PCM (`s16le`).
	- `astream()` is the asynchronous streaming interface.
	- Raw PCM has no WAV header; a playback client needs 24,000 Hz, one channel and signed 16-bit little-endian samples.
	- Model audio frames are 80 ms. Transport chunks and first-audio latency are separate quantities.
	- `max_seats` allocates concurrent engine capacity and memory, including when seats are idle.

	## Voice selection and cloning

	Select one of the 20 checkpoint voices by name, case-insensitively. The authoritative [voice file](https://huggingface.co/maya-research/maya2-native/resolve/main/voices/voices.json) lists training-language coverage; the [model-card table](https://huggingface.co/maya-research/maya2-native#voices) explains the at-least-one-hour criterion. This coverage does not mean every voice has equal quality in all languages.

	Built-in voice IDs: Ahana, Vaidehi, Malini, Charvi, Jahnavi, Anirban, Nakul, Vasant, Siddhant, Satvik, Abhay, Nivedita, Ishwari, Ritvik, Vrinda, Sreekar, Kayal, Shaunak, Akshat, Tanushree.

	No built-in voice has at least one hour of Assamese training audio. Assamese generation is cross-lingual; pronunciation and timbre need listening evaluation. For any language, prefer a voice with relevant coverage when available.

	```python
	tts.register_voice(
	"my_voice", ref_audio="reference.wav",
	ref_text="<exact transcript of reference.wav>", lang="hi",
	)
	audio = tts.generate(
	"வணக்கம், எப்படி இருக்கிறீர்கள்?",
	lang="ta", voice="my_voice", normalize=True,
	)
	```

	Use a clean single-speaker reference, 3–10 seconds recommended, with the speaker's permission. Reference and output languages may differ. Pass the voice separately; do not insert a voice-name prefix into the speech text or combine a built-in named voice with your own reference audio. Cloning similarity is approximate.

	## Local HTTP and WebSocket contract

	```bash
	python -m maya3.server --ckpt maya-research/maya2-native \
	--host 127.0.0.1 --port 8080 --max-seats 1
	```

	Wait for `GET http://127.0.0.1:8080/health` to return HTTP 200. During loading it returns 503. Then:

	```bash
	curl --fail-with-body -sS http://127.0.0.1:8080/v1/audio/speech \
	-H 'Content-Type: application/json' \
	-d '{"model":"maya3","input":"नमस्ते।","language":"hi","voice":"Ahana","response_format":"wav","normalize":true}' \
	--output speech.wav
	```

	\| Interface \| Contract \|
	\|---\|---\|
	\| `POST /v1/audio/speech` \| OpenAI-compatible speech shape with Maya extensions such as `language` and `normalize` \|
	\| `WS /v1/tts/stream` \| Local `metadata`, `speak`, `audio`, `end`, `cancel` and error messages; context IDs identify requests \|
	\| HTTP `pcm` \| Streaming raw PCM s16le, 24 kHz mono \|
	\| HTTP `wav` \| Streamed WAV wrapper around PCM; streaming header sizes may be unspecified \|
	\| HTTP `mp3` / `opus` \| Encoded streaming output, subject to installed encoder support \|
	\| HTTP `flac` \| Sent after generation completes \|
	\| `GET /v1/voices` \| Inspect registered voices \|
	\| `POST /v1/voices` \| Register a permitted reference voice \|
	\| Local speed \| `1.0`; do not copy hosted API speed ranges into this engine \|

	Use the [server source and protocol](https://raw.githubusercontent.com/MayaResearch/maya2-native/main/maya3/server.py) for complete messages and error handling, and [client examples](https://github.com/MayaResearch/maya2-native/tree/main/examples) for callers. HTTP and WebSocket servers normalize by default; Python callers explicitly pass `normalize=True`. Explicit language selection avoids ambiguous script detection for shared scripts or code-mixed input.

	The local WebSocket protocol accepts a `speak` request and returns generated audio incrementally. This is not the hosted Maya API's protocol. A voice-agent application must handle text aggregation, user turns, cancellation and discarding buffered playback. The local server has no built-in authentication; deployment access controls belong to the application environment.

	## Hosted API and framework distinction

	The downloadable model and Maya's hosted service have different contracts. Hosted voice IDs such as Ananya or Arjun do not appear in this checkpoint's built-in roster. Do not substitute hosted endpoints, credentials or voice IDs into the examples above.

	[Maya Research's Pipecat integration](https://github.com/MayaResearch/pipecat-maya) documents the hosted service and its supported Pipecat version. Check its current README before choosing an adapter. The reference engine's OpenAI-compatible endpoint can be used by suitable clients, but it is not proof of a maintained native adapter for every voice-agent framework.

	## Offline use, licence and evaluation

	First use downloads the model, [Mimi codec](https://huggingface.co/kyutai/mimi) and [ECAPA speaker encoder](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb). Cache all three and the installed dependencies before enabling `HF_HUB_OFFLINE=1` in an offline environment.

	Model weights and inference code are Apache-2.0. Mimi is CC-BY-4.0; other upstream licences and notices remain applicable. See the [licence section](https://huggingface.co/maya-research/maya2-native#license) and [credits](https://huggingface.co/maya-research/maya2-native#credits-and-acknowledgments).

	The full card distinguishes Maya Research's automated release evaluation from external human-preference rankings. Hardware-specific serving results are not latency promises for a public API, phone call or different GPU. Dynamic leaderboard claims require a date, language and filter; consult the [current benchmark source](https://voicearena.com/tts-leaderboard/hindi). Very short turns, Assamese, URLs, symbols, digits and heavily code-mixed input need application-specific evaluation. Emotion and event tags are best effort.