--- library_name: transformers language: - en - fr - es - de - it - pt - nl - pl - ru - sv - tr - uk - cs - da - fi - el - hu - no - ro - sk - bg - zh - ja - ko - ar - he - vi - id - th - tl - ms - ka - hi - bn - ta - te - mr - gu - kn - ml - pa - or - ur tags: - llama-cpp - llama-tts - text-to-speech - tts - speech-synthesis - voice-cloning - multilingual - gguf - zonos - zonos3 - zyphra - wav license: apache-2.0 base_model: - Zyphra/ZONOS3-Mini pipeline_tag: text-to-speech --- # **ZONOS3-Mini-GGUF** > **ZONOS3-Mini** is Zyphra's open-source, multilingual text-to-speech model. It supports 43 languages and offers voice cloning from a reference clip or a single speaker embedding. It has about 800M total parameters, with roughly 100M active per token. The backbone is an 11-layer hybrid of 9 Gated DeltaNet layers and 2 global attention layers, with a latent mixture-of-experts (64 experts, top-2 routing). It generates 15 Hz, 10-codebook audio tokens, and a causal iSTFT head renders them to 44.1 kHz audio as they are produced. Users can design a voice by shaping age, accent and vocal texture, and can control loudness, bandwidth and speaking rate. Text is normalized with NeMo for nine languages and tokenized with an 8,192-entry SuperBPE tokenizer. A single generation can run up to 10 minutes. The card reports a median time to first audio of 80 ms with 8 concurrent requests on an H100, and peak throughput of 457 seconds of audio per second. The model runs in real time on phones and laptops, including in the browser via WebGPU. It can also be served with SGLang-Omni through an OpenAI-compatible `/v1/audio/speech` endpoint, or run with native runtimes (CUDA, MLX for Apple Silicon, and C++/ggml for CPU, CUDA, Metal or Vulkan) that load the provided GGUF files. The weights are released under Apache 2.0. ## Model Files | File Name | Quant Type | File Size | File Link | Description | |-----------|------------|-----------|-----------|-------------| | ZONOS3-Mini.BF16.gguf | BF16 | 1.61 GB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.BF16.gguf) | Full BF16 weights. Highest quality, largest file size. | | ZONOS3-Mini.Q2_K.gguf | Q2_K | 300 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q2_K.gguf) | Lowest quality, smallest file size, only for extreme resource constraints. | | ZONOS3-Mini.Q3_K_L.gguf | Q3_K_L | 420 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q3_K_L.gguf) | Lower quality but usable, good for very low RAM availability. | | ZONOS3-Mini.Q3_K_M.gguf | Q3_K_M | 387 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q3_K_M.gguf) | Low quality. | | ZONOS3-Mini.Q4_0.gguf | Q4_0 | 456 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q4_0.gguf) | Standard 4-bit quantization. | | ZONOS3-Mini.Q4_K_M.gguf | Q4_K_M | 484 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q4_K_M.gguf) | Good quality, default size for most use cases, *recommended*. | | ZONOS3-Mini.Q4_K_S.gguf | Q4_K_S | 457 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q4_K_S.gguf) | Slightly lower quality with more space savings, *recommended*. | | ZONOS3-Mini.Q5_0.gguf | Q5_0 | 555 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q5_0.gguf) | Standard 5-bit quantization. | | ZONOS3-Mini.Q5_K_M.gguf | Q5_K_M | 570 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q5_K_M.gguf) | High quality, *recommended*. | | ZONOS3-Mini.Q5_K_S.gguf | Q5_K_S | 555 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q5_K_S.gguf) | High quality with slight space savings, *recommended*. | | ZONOS3-Mini.Q6_K.gguf | Q6_K | 661 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q6_K.gguf) | Very high quality, near perfect, *recommended*. | | ZONOS3-Mini.Q8_0.gguf | Q8_0 | 855 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q8_0.gguf) | Extremely high quality, generally unneeded but max available standard quant. | | ZONOS3-Mini.mmproj-bf16.gguf | mmproj-bf16 | 689 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.mmproj-bf16.gguf) | Multimodal projection file in BF16 format. Used for audio/voice capabilities. | ## Run with llama.cpp ### Install dependencies and clone llama.cpp ```python !pip install transformers huggingface-hub !git clone https://github.com/ggml-org/llama.cpp.git %cd /data/llama.cpp ``` ### Download GGUF model files Download the BF16 model and BF16 multimodal projector: ```python from huggingface_hub import snapshot_download # Target local directory local_dir = "./model/ZONOS3-Mini-GGUF" # Download only the specified files snapshot_download( repo_id="prithivMLmods/ZONOS3-Mini-GGUF", local_dir=local_dir, allow_patterns=[ "ZONOS3-Mini.BF16.gguf", "ZONOS3-Mini.mmproj-bf16.gguf", ], resume_download=True, ) ``` ### Build llama-tts Build the TTS CLI with the CUDA backend enabled: ```bash !cmake --build . --config Release -j$(nproc) --target llama-tts ``` ### Text-to-speech generation (GPU offloading) Run inference using `llama-tts` with GPU offloading enabled: ```bash !/data/llama.cpp/build/bin/llama-tts \ -m /data/llama.cpp/model/ZONOS3-Mini-GGUF/ZONOS3-Mini.BF16.gguf \ -mm /data/llama.cpp/model/ZONOS3-Mini-GGUF/ZONOS3-Mini.mmproj-bf16.gguf \ -p "ZONOS3-Mini is Zyphra’s compact, open-source text-to-speech model, delivering natural reliable speech and high-fidelity voice cloning with just 100 million active parameters. It runs in real time on phones and laptops, including directly in the browser." \ -ngl 99 \ -n 1000 \ --seed 42 \ --output /data/llama.cpp/model/ZONOS3-Mini-GGUF/out.wav ``` ### Output The generated audio file will be saved at: ```text /data/llama.cpp/model/ZONOS3-Mini-GGUF/out.wav ``` You can play the generated WAV file using an audio player or a notebook audio widget. ## llama.cpp LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp Quantized using llama.cpp by [prithivMLmods](https://huggingface.co/prithivMLmods).