Instructions to use prithivMLmods/ZONOS3-Mini-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use prithivMLmods/ZONOS3-Mini-GGUF with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-to-speech", model="prithivMLmods/ZONOS3-Mini-GGUF")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("prithivMLmods/ZONOS3-Mini-GGUF", device_map="auto") - Zonos
How to use prithivMLmods/ZONOS3-Mini-GGUF with Zonos:
# pip install git+https://github.com/Zyphra/Zonos.git import torchaudio from zonos.model import Zonos from zonos.conditioning import make_cond_dict model = Zonos.from_pretrained("prithivMLmods/ZONOS3-Mini-GGUF", device="cuda") wav, sr = torchaudio.load("speaker.wav") # 5-10s reference clip speaker = model.make_speaker_embedding(wav, sr) cond = make_cond_dict(text="Hello, world!", speaker=speaker, language="en-us") codes = model.generate(model.prepare_conditioning(cond)) audio = model.autoencoder.decode(codes)[0].cpu() torchaudio.save("sample.wav", audio, model.autoencoder.sampling_rate) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use prithivMLmods/ZONOS3-Mini-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
Use Docker
docker model run hf.co/prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use prithivMLmods/ZONOS3-Mini-GGUF with Ollama:
ollama run hf.co/prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use prithivMLmods/ZONOS3-Mini-GGUF with Docker Model Runner:
docker model run hf.co/prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
- Lemonade
How to use prithivMLmods/ZONOS3-Mini-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.ZONOS3-Mini-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
ZONOS3-Mini-GGUF
ZONOS3-Mini is Zyphra's open-source, multilingual text-to-speech model. It supports 43 languages and offers voice cloning from a reference clip or a single speaker embedding. It has about 800M total parameters, with roughly 100M active per token. The backbone is an 11-layer hybrid of 9 Gated DeltaNet layers and 2 global attention layers, with a latent mixture-of-experts (64 experts, top-2 routing). It generates 15 Hz, 10-codebook audio tokens, and a causal iSTFT head renders them to 44.1 kHz audio as they are produced. Users can design a voice by shaping age, accent and vocal texture, and can control loudness, bandwidth and speaking rate. Text is normalized with NeMo for nine languages and tokenized with an 8,192-entry SuperBPE tokenizer. A single generation can run up to 10 minutes. The card reports a median time to first audio of 80 ms with 8 concurrent requests on an H100, and peak throughput of 457 seconds of audio per second. The model runs in real time on phones and laptops, including in the browser via WebGPU. It can also be served with SGLang-Omni through an OpenAI-compatible
/v1/audio/speechendpoint, or run with native runtimes (CUDA, MLX for Apple Silicon, and C++/ggml for CPU, CUDA, Metal or Vulkan) that load the provided GGUF files. The weights are released under Apache 2.0.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| ZONOS3-Mini.BF16.gguf | BF16 | 1.61 GB | Link | Full BF16 weights. Highest quality, largest file size. |
| ZONOS3-Mini.Q2_K.gguf | Q2_K | 300 MB | Link | Lowest quality, smallest file size, only for extreme resource constraints. |
| ZONOS3-Mini.Q3_K_L.gguf | Q3_K_L | 420 MB | Link | Lower quality but usable, good for very low RAM availability. |
| ZONOS3-Mini.Q3_K_M.gguf | Q3_K_M | 387 MB | Link | Low quality. |
| ZONOS3-Mini.Q4_0.gguf | Q4_0 | 456 MB | Link | Standard 4-bit quantization. |
| ZONOS3-Mini.Q4_K_M.gguf | Q4_K_M | 484 MB | Link | Good quality, default size for most use cases, recommended. |
| ZONOS3-Mini.Q4_K_S.gguf | Q4_K_S | 457 MB | Link | Slightly lower quality with more space savings, recommended. |
| ZONOS3-Mini.Q5_0.gguf | Q5_0 | 555 MB | Link | Standard 5-bit quantization. |
| ZONOS3-Mini.Q5_K_M.gguf | Q5_K_M | 570 MB | Link | High quality, recommended. |
| ZONOS3-Mini.Q5_K_S.gguf | Q5_K_S | 555 MB | Link | High quality with slight space savings, recommended. |
| ZONOS3-Mini.Q6_K.gguf | Q6_K | 661 MB | Link | Very high quality, near perfect, recommended. |
| ZONOS3-Mini.Q8_0.gguf | Q8_0 | 855 MB | Link | Extremely high quality, generally unneeded but max available standard quant. |
| ZONOS3-Mini.mmproj-bf16.gguf | mmproj-bf16 | 689 MB | Link | Multimodal projection file in BF16 format. Used for audio/voice capabilities. |
Run with llama.cpp
Install dependencies and clone llama.cpp
!pip install transformers huggingface-hub
!git clone https://github.com/ggml-org/llama.cpp.git
%cd /data/llama.cpp
Download GGUF model files
Download the BF16 model and BF16 multimodal projector:
from huggingface_hub import snapshot_download
# Target local directory
local_dir = "./model/ZONOS3-Mini-GGUF"
# Download only the specified files
snapshot_download(
repo_id="prithivMLmods/ZONOS3-Mini-GGUF",
local_dir=local_dir,
allow_patterns=[
"ZONOS3-Mini.BF16.gguf",
"ZONOS3-Mini.mmproj-bf16.gguf",
],
resume_download=True,
)
Build llama-tts
Build the TTS CLI with the CUDA backend enabled:
!cmake --build . --config Release -j$(nproc) --target llama-tts
Text-to-speech generation (GPU offloading)
Run inference using llama-tts with GPU offloading enabled:
!/data/llama.cpp/build/bin/llama-tts \
-m /data/llama.cpp/model/ZONOS3-Mini-GGUF/ZONOS3-Mini.BF16.gguf \
-mm /data/llama.cpp/model/ZONOS3-Mini-GGUF/ZONOS3-Mini.mmproj-bf16.gguf \
-p "ZONOS3-Mini is Zyphraβs compact, open-source text-to-speech model, delivering natural reliable speech and high-fidelity voice cloning with just 100 million active parameters. It runs in real time on phones and laptops, including directly in the browser." \
-ngl 99 \
-n 1000 \
--seed 42 \
--output /data/llama.cpp/model/ZONOS3-Mini-GGUF/out.wav
Output
The generated audio file will be saved at:
/data/llama.cpp/model/ZONOS3-Mini-GGUF/out.wav
You can play the generated WAV file using an audio player or a notebook audio widget.
llama.cpp
LLM inference in C/C++ β https://github.com/ggml-org/llama.cpp
Quantized using llama.cpp by prithivMLmods.
- Downloads last month
- 339
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Model tree for prithivMLmods/ZONOS3-Mini-GGUF
Base model
Zyphra/ZONOS3-Mini
docker model run hf.co/prithivMLmods/ZONOS3-Mini-GGUF: