apiplant/drama-q4

Drama 3 int4 (drama-rs format)

A 4-bit quantization of Fish Audio Drama 3 that runs on a single 24 GB GPU with the drama-rs Rust + CUDA engine.

Built with Fish Audio. This is an unofficial derivative; it is not made, sponsored or endorsed by Fish Audio. It is distributed under the same Fish Audio Community License as the original — see NOTICE for the attribution and a description of the modifications.

These files are in the drama-rs layout and are not loadable with transformers or SGLang-Omni.

What's quantized

tensor	format
routed MoE experts (FP8 in the original)	int4, asymmetric, group 64, f16 scale + min, MSE clip search
attention / DeltaNet / shared expert / codebook decoder linears	int8, symmetric, group 128
lm_head	the 4097 sampleable rows only (EOS + semantic band), bf16
embeddings, router, norms	bf16 / f32
audio codec	original weights, weight norm folded, safetensors

About 22 GB on disk; ~21.4 GiB of GPU memory with the codec loaded.

Quality

Compared against the original FP8/bf16 checkpoint with an fp32 PyTorch reference, teacher-forced on a real voice-cloning prompt (12 frames):

	KL(original ‖ int4)	top-1 agreement
semantic token	0.012	92%
codebook heads 1–9	~0.006 each	75–100%

At the default sampling settings (temperature 1.0, top-k 30, top-p 0.9) this difference is negligible; generated speech transcribes back to the input text exactly.

Speed (RTX 4090)

• ~5.2 ms per frame, ~190 frames/s (21.5 frames = 1 s of 44.1 kHz audio): ~7–9× real time end to end
• time to first audio ~50–190 ms with streaming

Usage

git clone https://github.com/apiplant/drama-rs && cd drama-rs
cargo build --release   # needs the CUDA toolkit; DRAMA_CUDA_ARCH defaults to sm_89
hf download apiplant/drama-q4 --local-dir models/drama-3-q4

./target/release/drama speak --model models/drama-3-q4 --text "Hello there." --play

# voice cloning
./target/release/drama speak --model models/drama-3-q4 --play \
  --ref-wav voice.wav --ref-text "Exactly what is said in voice.wav." \
  --text "<|speaker:0|> [sighs] Fine. If we miss it, it's on you."

License and acceptable use

Same terms as the original: the Fish Audio Community License, Version 1.0. Research and non-commercial use is free; commercial use is limited to organizations under USD 500,000 annual revenue, and offering a hosted TTS / voice-cloning service requires a separate license from Fish Audio. You must not clone the voice of an identifiable person without their consent, impersonate anyone to deceive, or use the model or its outputs to train other speech generation models. Read the full LICENSE before use.