ZONOS3-Mini-swift-ptq (MLX)

Zyphra's ZONOS3-Mini β€” real-time multilingual text-to-speech at 44.1 kHz β€” converted for mlx-zonos3-tts-swift, a Swift-MLX port for Apple Silicon. On an M5 Max it runs at about 9–10Γ— real time with first audio in tens of milliseconds, 1.6 GB resident and 2.8 GB peak.

Why this repository exists next to mlx-community/ZONOS3-Mini

mlx-community/ZONOS3-Mini is the conversion for mlx-audio's Python port, and it is the right choice if that is what you run. This one is laid out for the Swift package: tensor names and metadata follow Zyphra's own GGUF release, the quantized tensors are stored as MLX affine quantization, and the 27 voices ship as one voices.json. Neither port loads the other's files.

What is in it

file contents
lm/model.safetensors the language model: Zyphra's post-training-quantized backbone (gguf/backbone-ptq.gguf β€” 4-bit experts, 8-bit sequence mixers), plus the logits head at 8-bit
codec/model.safetensors the keep2_plain decoder (gguf/zac-keep2.gguf), fp16 / fp32
speaker_encoder/model.safetensors the MECT-B2 speaker encoder (gguf/spk-encoder.gguf), fp32
config.json every metadata key of the three GGUFs, and the bit width of each quantized tensor
tokenizer.json the SuperBPE tokenizer embedded in the backbone GGUF
voices.json the 27 default voices' 192-d speaker embeddings

The conversion changes no values. GGUF Q4_0 / Q8_0 blocks are repacked bit-for-bit into MLX's affine form (group size 32); nothing is re-quantized. The one computed tensor is the logits head: Zyphra's MLX server quantizes it to Q8_0 when it loads a quantized backbone, and this file stores the result of that same call. The converter is Tools/convert_gguf.py in the Swift repository.

Not included: the full-size full_mr decoder, the raw-reference encoder and its SSL teacher, the speech-edit backbone and adapters, and the steering bundle. The Swift port does not use them yet.

Source: Zyphra/ZONOS3-Mini at revision 7146a686b62bcb0661528fa300984e815e10152b; runtime reference github.com/Zyphra/ZONOS3-Mini at 4c66ceac.

Use

let tts = try Zonos3TTS(directory: checkpointURL)
let voice = try tts.embedSpeaker(pcm44100: referenceSamples)       // or a voice from voices.json, or nil
let result = try tts.synthesize(text: "Hello from ZONOS3 Mini.", speakerEmbedding: voice)

Through MLXEngine it is the MLXZonos3TTS package on the tts capability, which downloads this repository itself.

Limits

  • Text is spoken as written. The Swift port has no text normalization; write numbers, dates and abbreviations out.
  • Voice cloning is approximate. It clones from a 192-d speaker embedding; similarity to the reference is lower than larger cloning models reach. The raw-reference mode of the original is not ported.
  • Duration targets, delivery steering, accents and speech editing are not ported.
  • A seed is reproducible in Swift but does not reproduce the original runtime's takes.
  • Clone a voice only with its owner's consent.

Licence

Model weights: Apache-2.0, Zyphra, Inc. The conversion adds no terms. The Swift code is MIT and derives from Zyphra's MIT-licensed runtime; the gated-delta Metal kernel it uses is MIT, from Apple's mlx-lm.

Downloads last month
110
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for mlx-community/ZONOS3-Mini-swift-ptq

Finetuned
(5)
this model