Community NVFP4 quant, pruned to 7,369 experts (runs on one NVIDIA GB10)
#8
by plotarmordev - opened
Hi Fish Audio team, and thanks for releasing Drama 3.
We published an unofficial community quant: https://huggingface.co/plotarmordev/drama-3-NVFP4-Pruned
- The routed experts are pruned from 10,240 to 7,369 and stored as 4-bit NVFP4 (GPTQ). Everything else, the codec
included, is byte-identical to Drama 3 at revision e0a9218. - Weights and codec take 20.76 GiB on disk against 38.63 GiB for the original.
- On one NVIDIA GB10, over 24 English texts, word error rate, failed takes and speaker similarity were within noise of
the original. First audio came sooner; a cold start took more than four times as long. - It does not load in Transformers or SGLang-Omni. It runs on a public branch of TensorFold's native engine; the card
has the build, serve and request commands: https://github.com/plotarmordev/TensorFold/tree/drama-nvfp4 - The full comparison, raw records and commands:
https://github.com/plotarmordev/drama-tensorfold/tree/main/results/full-vs-quant-01
It is released under the Fish Audio Community License with your NOTICE, and the card says plainly that Fish Audio
did not make or endorse it. If you would like anything changed on the card, we will fix it.