Text-to-Speech
Transformers
GGUF
Zonos
llama-cpp
llama-tts
tts
speech-synthesis
voice-cloning
multilingual
zonos3
zyphra
wav
Instructions to use prithivMLmods/ZONOS3-Mini-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use prithivMLmods/ZONOS3-Mini-GGUF with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-to-speech", model="prithivMLmods/ZONOS3-Mini-GGUF")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("prithivMLmods/ZONOS3-Mini-GGUF", device_map="auto") - Zonos
How to use prithivMLmods/ZONOS3-Mini-GGUF with Zonos:
# pip install git+https://github.com/Zyphra/Zonos.git import torchaudio from zonos.model import Zonos from zonos.conditioning import make_cond_dict model = Zonos.from_pretrained("prithivMLmods/ZONOS3-Mini-GGUF", device="cuda") wav, sr = torchaudio.load("speaker.wav") # 5-10s reference clip speaker = model.make_speaker_embedding(wav, sr) cond = make_cond_dict(text="Hello, world!", speaker=speaker, language="en-us") codes = model.generate(model.prepare_conditioning(cond)) audio = model.autoencoder.decode(codes)[0].cpu() torchaudio.save("sample.wav", audio, model.autoencoder.sampling_rate) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use prithivMLmods/ZONOS3-Mini-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
Use Docker
docker model run hf.co/prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use prithivMLmods/ZONOS3-Mini-GGUF with Ollama:
ollama run hf.co/prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use prithivMLmods/ZONOS3-Mini-GGUF with Docker Model Runner:
docker model run hf.co/prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
- Lemonade
How to use prithivMLmods/ZONOS3-Mini-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull prithivMLmods/ZONOS3-Mini-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.ZONOS3-Mini-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: transformers
|
| 3 |
+
---
|
| 4 |
+
|
| 5 |
+
# **ZONOS3-Mini-GGUF**
|
| 6 |
+
|
| 7 |
+
> **ZONOS3-Mini** is Zyphra's open-source, multilingual text-to-speech model. It supports 43 languages and offers voice cloning from a reference clip or a single speaker embedding. It has about 800M total parameters, with roughly 100M active per token. The backbone is an 11-layer hybrid of 9 Gated DeltaNet layers and 2 global attention layers, with a latent mixture-of-experts (64 experts, top-2 routing). It generates 15 Hz, 10-codebook audio tokens, and a causal iSTFT head renders them to 44.1 kHz audio as they are produced. Users can design a voice by shaping age, accent and vocal texture, and can control loudness, bandwidth and speaking rate. Text is normalized with NeMo for nine languages and tokenized with an 8,192-entry SuperBPE tokenizer. A single generation can run up to 10 minutes. The card reports a median time to first audio of 80 ms with 8 concurrent requests on an H100, and peak throughput of 457 seconds of audio per second. The model runs in real time on phones and laptops, including in the browser via WebGPU. It can also be served with SGLang-Omni through an OpenAI-compatible `/v1/audio/speech` endpoint, or run with native runtimes (CUDA, MLX for Apple Silicon, and C++/ggml for CPU, CUDA, Metal or Vulkan) that load the provided GGUF files. The weights are released under Apache 2.0, and a hosted version is available on Zyphra Cloud. Evaluation details are in the blog post and the technical report (arXiv 2611.16525).
|
| 8 |
+
|
| 9 |
+
| File Name | Quant Type | File Size | File Link | Description |
|
| 10 |
+
|-----------|------------|-----------|-----------|-------------|
|
| 11 |
+
| ZONOS3-Mini.BF16.gguf | BF16 | 1.61 GB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.BF16.gguf) | Full BF16 weights. Highest quality, largest file size. |
|
| 12 |
+
| ZONOS3-Mini.Q2_K.gguf | Q2_K | 300 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q2_K.gguf) | Lowest quality, smallest file size, only for extreme resource constraints. |
|
| 13 |
+
| ZONOS3-Mini.Q3_K_L.gguf | Q3_K_L | 420 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q3_K_L.gguf) | Lower quality but usable, good for very low RAM availability. |
|
| 14 |
+
| ZONOS3-Mini.Q3_K_M.gguf | Q3_K_M | 387 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q3_K_M.gguf) | Low quality. |
|
| 15 |
+
| ZONOS3-Mini.Q4_0.gguf | Q4_0 | 456 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q4_0.gguf) | Standard 4-bit quantization. |
|
| 16 |
+
| ZONOS3-Mini.Q4_K_M.gguf | Q4_K_M | 484 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q4_K_M.gguf) | Good quality, default size for most use cases, *recommended*. |
|
| 17 |
+
| ZONOS3-Mini.Q4_K_S.gguf | Q4_K_S | 457 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q4_K_S.gguf) | Slightly lower quality with more space savings, *recommended*. |
|
| 18 |
+
| ZONOS3-Mini.Q5_0.gguf | Q5_0 | 555 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q5_0.gguf) | Standard 5-bit quantization. |
|
| 19 |
+
| ZONOS3-Mini.Q5_K_M.gguf | Q5_K_M | 570 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q5_K_M.gguf) | High quality, *recommended*. |
|
| 20 |
+
| ZONOS3-Mini.Q5_K_S.gguf | Q5_K_S | 555 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q5_K_S.gguf) | High quality with slight space savings, *recommended*. |
|
| 21 |
+
| ZONOS3-Mini.Q6_K.gguf | Q6_K | 661 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q6_K.gguf) | Very high quality, near perfect, *recommended*. |
|
| 22 |
+
| ZONOS3-Mini.Q8_0.gguf | Q8_0 | 855 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.Q8_0.gguf) | Extremely high quality, generally unneeded but max available standard quant. |
|
| 23 |
+
| ZONOS3-Mini.mmproj-bf16.gguf | mmproj-bf16 | 689 MB | [Link](https://huggingface.co/prithivMLmods/ZONOS3-Mini-GGUF/blob/main/ZONOS3-Mini.mmproj-bf16.gguf) | Multimodal projection file in BF16 format. Used for audio/voice capabilities. |
|
| 24 |
+
|