--- license: apache-2.0 library_name: transformers pipeline_tag: text-to-speech base_model: Zyphra/ZONOS3-Mini language: - en - fr - es - de - it - pt - nl - pl - ru - sv - tr - uk - cs - da - fi - el - hu - "no" - ro - sk - bg - zh - ja - ko - ar - he - vi - id - th - tl - ms - ka - hi - bn - ta - te - mr - gu - kn - ml - pa - or - ur tags: - text-to-speech - tts - speech-synthesis - multilingual - voice-cloning - zonos - zonos3 - zyphra --- # ZONOS3-Mini (native 🤗 Transformers) [ZONOS3-Mini](https://huggingface.co/Zyphra/ZONOS3-Mini) is Zyphra's compact multilingual text-to-speech model (43 languages, about 100 million active parameters per token). This repo holds the checkpoint for the native `transformers` port (`zonos3_mini`, no `trust_remote_code`): the language model, the MECT-B2 speaker encoder, the SuperBPE tokenizer, the processor and the 27 release voices in `voices/`. The processor loads the [ZAC-1](https://huggingface.co/Zyphra/ZAC-1-hf) codec as its audio tokenizer. Model details, samples and evaluations: the [blog post](https://www.zyphra.com/our-work/zonos3-mini) and [technical report](https://arxiv.org/abs/2611.16525). ## Usage ```python from huggingface_hub import hf_hub_download from safetensors.torch import load_file from transformers import AutoModelForTextToWaveform, AutoProcessor model_id = "Zyphra/ZONOS3-Mini-transformers" processor = AutoProcessor.from_pretrained(model_id) model = AutoModelForTextToWaveform.from_pretrained(model_id, device_map="auto") voice = load_file(hf_hub_download(model_id, "voices/Savannah.safetensors")) conversation = [{"role": "user", "content": "Hello! This is ZONOS3-Mini running in transformers."}] inputs = processor.apply_chat_template( conversation, tokenize=True, return_dict=True, processor_kwargs={"speaker_reference_codes": [voice["codes"]]}, ).to(model.device) audio = processor.decode(model.generate(**inputs)) processor.save_audio(audio, "zonos3_mini.wav") ``` To clone a voice from a clip, put the clip (44.1 kHz) in the user turn: ```python conversation = [ { "role": "user", "content": [ {"type": "text", "text": "This voice was cloned from a short clip."}, {"type": "audio", "path": "my_voice.wav"}, ], } ] inputs = processor.apply_chat_template( conversation, tokenize=True, return_dict=True, processor_kwargs={"sampling_rate": 44100} ).to(model.device) audio = processor.decode(model.generate(**inputs)) ``` The clip is encoded into reference codes by ZAC-1 by default; `speaker_mode="embedding"` conditions on its speaker embedding instead, and `speaker_guidance=True` with `guidance_scale > 1` in `generate` strengthens the voice. The documentation covers conditions (loudness, fmax, speaking rate), batching, `torch.compile`, the pipeline and training. Install `zyphra-textnorm` to read numbers, dates, currencies and units aloud as the model was trained; without it, the text is used as written. ## Voices `voices/.safetensors` holds each release voice's reference codes (`codes`, `[frames, 10]`) and speaker embedding (`embedding`, `[192]`); `voices/voices.json` lists their language, style and gender. ## Architecture | Component | ZONOS3-Mini | |---|---| | Backbone | 11 layers, dimension 1,024; latent MoE with 64 experts, top-2 routing from layer 1 | | Sequence mixer | 9 Gated DeltaNet layers + 2 global attention layers, Full AttnRes; context 32,768 positions | | Text | SuperBPE, 8,192 entries + 4 special tokens | | Audio tokens | 15 Hz; 10 codebooks of 4,096 entries (1 semantic + 9 acoustic); hop 2,940 samples at 44.1 kHz | | Speaker prompt | codec reference prefix (up to 60 s) or a 192-d MECT-B2 embedding | | Codec | [ZAC-1](https://huggingface.co/Zyphra/ZAC-1-hf) `keep2_plain` decoder | The language model is stored in bfloat16 (the release dtype); the speaker encoder stays in float32. ## License Apache-2.0. ## Citation ```bibtex @misc{zyphra2026zonos3mini, title = {{ZONOS3-Mini Technical Report}}, author = {Gabriel Clark and Sofian Mejjoute and George Close and Beren Millidge}, year = {2026}, eprint = {2611.16525}, archivePrefix = {arXiv}, primaryClass = {cs.SD}, url = {https://arxiv.org/abs/2611.16525}, } ```