drama-3-MLX-5bit

Built with Fish Audio. An MLX conversion of fishaudio/drama-3 (revision e0a9218) for Apple Silicon. It runs on stock mlx-audio; no fork is needed.

This is an unofficial community conversion. Fish Audio has not endorsed or reviewed it.

Recipe	Experts 5-bit affine, group size 64; attention, DeltaNet and shared-expert projections, semantic head and tower layers 8-bit (group size 64); the 9 codebook heads, router, embeddings and norms bf16
Weights on disk	26.3 GB (+1.6 GB codec)
Peak memory (M5 Max, clips ≤ 28 s)	40 GB
Speed (M5 Max)	2.8× real time
Recommended Mac	64 GB+

Usage

pip install "mlx-audio @ git+https://github.com/Blaizzy/mlx-audio@dad2087f790dfe96b07d5aac1405db4b578b411f"

import mlx.core as mx
from mlx_audio.tts.utils import load_model

model = load_model("fcmeyer/drama-3-MLX-5bit")
mx.random.seed(0)
for result in model.generate(
    text="<|speaker:0|>Did you hear that? <|speaker:1|>[whispering] Keep your voice down.",
    temperature=1.0, top_p=0.9, top_k=30,   # Fish Audio's benchmark settings
):
    ...  # result.audio (mx.array), model.sample_rate == 44100

Or from the command line:

python -m mlx_audio.tts.generate --model fcmeyer/drama-3-MLX-5bit \
  --text "Hello from drama 3." --temperature 1.0 --top_p 0.9 --top_k 30

Trade-off vs 4bit-HQ. The experts get one more bit, and the dense layers and tower drop from bf16 to 8-bit while the 9 codebook heads stay bf16. The net effect is about half the semantic-token KL (paired difference −0.0096 [−0.0106, −0.0086]), 28% lower head KL, and about 24% higher speed, for 1.8 GB more weights and peak memory.

How it compares

All upstream-loadable MLX quants of drama-3 below were run through the same harness on the same machine with the same sampler (T=1.0, top-p 0.9, top-k 30, fixed seeds). The harness and raw results are public.

Model	Main KL	Main top-1	Head KL (mean of 9)	Head top-1	Failures	WER/CER	Peak GB	RTF
MLX bf16 (reference)	0.0000 [-0.0000, 0.0000]	100.0 [100.0, 100.0]	0.0000 [-0.0000, 0.0000]	100.0 [100.0, 100.0]	20/500 (4.0% [1.6, 6.8])	3.9 [2.2, 5.9]%	84.9	2.01×
8-bit experts, rest bf16 (not published)	0.0046 [0.0043, 0.0049]	93.7 [93.0, 94.4]	0.0025 [0.0024, 0.0027]	93.5 [93.1, 93.9]	31/500 (6.2% [3.6, 9.4])	4.8 [3.0, 6.9]%	54.7	2.08×
6-bit experts, rest bf16 (not published)	0.0062 [0.0055, 0.0070]	92.6 [91.7, 93.4]	0.0032 [0.0029, 0.0035]	92.9 [92.4, 93.4]	36/500 (7.2% [4.2, 10.6])	5.6 [3.5, 7.9]%	46.6	2.03×
this repo	0.0094 [0.0088, 0.0102]	90.3 [89.3, 91.2]	0.0059 [0.0056, 0.0063]	90.6 [90.0, 91.2]	19/500 (3.8% [1.8, 6.0])	3.3 [1.9, 4.8]%	40.4	2.80×
fcmeyer/drama-3-MLX-4bit-HQ	0.0190 [0.0177, 0.0205]	86.7 [85.6, 87.8]	0.0082 [0.0076, 0.0089]	89.0 [88.3, 89.7]	23/500 (4.6% [2.6, 6.6])	4.0 [2.6, 5.4]%	38.6	2.26×
mlx-community/fish-audio-drama-3-4bit	0.0204 [0.0190, 0.0219]	86.3 [85.1, 87.4]	0.0101 [0.0094, 0.0108]	87.7 [87.0, 88.5]	22/500 (4.4% [2.2, 7.2])	3.5 [1.9, 5.5]%	36.3	3.02×
fcmeyer/drama-3-MLX-3bit-HQ	0.0352 [0.0318, 0.0390]	81.8 [80.5, 83.3]	0.0146 [0.0132, 0.0163]	85.4 [84.4, 86.4]	18/500 (3.6% [1.8, 5.6])	3.1 [1.9, 4.5]%	33.8	2.72×
Nav8ha/drama-3-mlx-mixed-3bit	0.0660 [0.0610, 0.0713]	75.4 [73.8, 77.1]	0.0271 [0.0246, 0.0297]	80.4 [79.1, 81.7]	15/500 (3.0% [1.2, 5.2])	3.0 [1.6, 4.6]%	32.2	3.11×

• Main KL / Head KL: teacher-forced KL(bf16 ‖ quant) for the semantic token and for the 9 residual-codebook heads, averaged over 64 bf16-generated sequences (32 prompts × 2 seeds). Lower means closer to the unquantized MLX model. Brackets give the 95% bootstrap CI.
• Top-1: how often the quant's most likely token matches bf16's.
• Failures: share of 500 free-running clips (100 prompts × 5 seeds; English, dialogue, cues, es/de/fr/zh/ja) that hit the 600-frame cap, covered < 60% of the text, or had WER/CER ≥ 50%. Almost all failures, bf16 included, are stalls: the model speaks the first line, then produces silence instead of ending. Brackets give a 95% bootstrap CI over prompts.
• WER/CER: Whisper large-v3-turbo against the prompt, with speaker tags and cues removed. CER is used for zh/ja. The CI is bootstrapped over prompts.
• Peak GB / RTF: peak MLX memory and audio seconds per wall-clock second over the 500 clips on an M5 Max (128 GB).
• All clips were generated from text alone, without a reference voice. Voice-cloned output has not been evaluated.
• The reference is MLX bf16. Per the mlx-audio port, MLX bf16 itself matches the PyTorch FP8 original at about 86% semantic top-1.

Paired differences

Every model is scored on the same 64 sequences, so the KL difference between two models can be measured sequence by sequence. That removes the spread between easy and hard sequences, and the intervals get much tighter than in the table above. Positive means further from bf16 than this repo.

Model	Main KL − this repo	Head KL − this repo
8-bit experts, rest bf16 (not published)	-0.0049 [-0.0054, -0.0044]	-0.0034 [-0.0036, -0.0031]
6-bit experts, rest bf16 (not published)	-0.0032 [-0.0039, -0.0026]	-0.0027 [-0.0030, -0.0024]
fcmeyer/drama-3-MLX-4bit-HQ	0.0096 [0.0086, 0.0106]	0.0023 [0.0019, 0.0028]
mlx-community/fish-audio-drama-3-4bit	0.0110 [0.0100, 0.0120]	0.0041 [0.0037, 0.0046]
fcmeyer/drama-3-MLX-3bit-HQ	0.0257 [0.0229, 0.0291]	0.0087 [0.0075, 0.0101]
Nav8ha/drama-3-mlx-mixed-3bit	0.0566 [0.0521, 0.0614]	0.0211 [0.0191, 0.0235]

Choosing a quant

• Best balance of quality, speed and size: this repo. It is closer to bf16 than every 4-bit and 3-bit quant here on both the semantic token and the heads (the paired intervals above exclude zero), and only mlx-community 4-bit and Nav8ha 3-bit are faster.
• Closer still to bf16: the 6- and 8-bit experts-only quants (not published) cut the remaining KL by a third to a half, but need 46–55 GB peak and run at about 2× real time.
• Smaller at similar speed: drama-3-MLX-3bit-HQ needs 6.6 GB less peak memory and runs at 2.7× real time, but its semantic-token KL is about 3.7× this repo's.
• Smallest or fastest: mlx-community 4-bit (36 GB peak, 3.0×) and Nav8ha 3-bit (32 GB, 3.1×). Both drift further from bf16 under teacher forcing; Nav8ha's 3-bit by a lot.
• None of the free-running metrics (failures, WER) separate the quants at this sample size. Teacher-forced KL is the only measure here that does.

Limitations and known behaviour

• Stalls. In about 3–7% of clips, at every precision including bf16, the model speaks the first line or sentence and then produces silence until the length limit instead of ending. It is much more common in multi-speaker dialogue and in prompts with cues. If you see it, re-run with another seed, or stop generation in your own code once the output has been silent for a second or two.
• No reference voice was used. Every clip was generated from text alone. Voice-cloned output, speaker similarity and long multi-chunk texts have not been evaluated.
• WER is near Whisper's floor (3–6%), so it is a sanity check, not a ranking.
• One machine. Speed and peak memory were measured on a single M5 Max (128 GB); other Macs will differ. Speed is mostly limited by memory bandwidth plus a fixed per-frame overhead in the generation loop.
• Sampler. The numbers above use T=1.0, top-p 0.9, top-k 30, set explicitly. mlx-audio's own defaults differ; pass the parameters if you want to reproduce them.

Reproducing

python -m mlx_audio.tts.models.fish_drama3.convert --src drama-3 --dst out --expert-bits 5 --dense-bits 8 --tower-bits 8 --head-bits 0

--head-bits comes from a small converter patch that is not in upstream mlx-audio yet: convert-head-bits.patch in this repo, against the commit above (git apply convert-head-bits.patch). It only changes how the checkpoint is written; the result loads on stock mlx-audio.

Eval harness: eval/ in this repo, with raw per-clip results in eval/results/.

License and use restrictions

These Fish Audio Materials are licensed under the Fish Audio Community License, Version 1.0. See NOTICE for the changes made in this conversion. In short:

• Research, non-commercial use, and commercial use below USD 500k annual revenue are permitted. Above that, you need a separate license from Fish Audio.
• Offering it as a hosted TTS/voice-cloning service needs a separate license, whatever your revenue.
• Do not clone the voice of an identifiable person without their consent. Do not use it to impersonate, deceive or harass anyone. Do not remove watermarks or provenance signals.
• Do not use its outputs to train or distill other speech or generative models.

Read the full LICENSE. This summary is not legal advice.