markldn/Dia2-2B-GGUF

Dia2-2B GGUF: F16, Q8_0 and Q4_0

Converted from Nari Labs Dia2-2B. This is a community conversion for markldn/audio.cpp-dia2. The linked native runtime is required; ordinary llama.cpp does not synthesize this custom architecture. See build, cloning and streaming instructions.

All weight variants are at the repository root. Choose one Dia2 weight file and keep the shared codec plus config/tokenizer sidecars beside it. F16 uses floating-point weights; Q8_0 and Q4_0 quantize the two-dimensional Dia2 matrices in 32-value blocks. One-dimensional normalization weights retain F32 precision. The codec always stays F16.

Dia2 file	Size, decimal	Precision
dia2-f16.gguf	3.839 GB	16-bit floating point
dia2-q8_0.gguf	2.040 GB	Q8_0
dia2-q4_0.gguf	1.080 GB	Q4_0

mimi-f16.gguf adds 0.193 GB. Download config.json, tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, special_tokens_map.json and added_tokens.json as well. Quantized files save download/storage space; runtime VRAM also includes caches and temporary buffers and can differ from file size. Q4_0 trades more fidelity for size; F16/Q8_0 are preferable when memory permits. These are Q8_0/Q4_0, not Q8_K or Q4_K_M.

Run on AMD GPU1

git clone https://github.com/markldn/audio.cpp-dia2
cd audio.cpp-dia2
./serve/build-dia2.sh hip
hf download markldn/Dia2-2B-GGUF --local-dir models/Dia2-2B-GGUF
build/dia2-hip/bin/audiocpp_cli --task tts --family dia2 \
  --model models/Dia2-2B-GGUF/dia2-q8_0.gguf \
  --backend hip --device 1 --threads 2 \
  --request-option dia2.cfg_scale=2.0 --request-option dia2.seed=12345 \
  --text 'Hello there. This is a speech test.' --out speech.wav

Change the weight filename to select F16 or Q4_0. --backend cpu is also available. The recommended guidance setting here is 2.0 for 2B. The model is English; generation uses complete text input with incremental audio output, mono 24 kHz. Optional reference-audio cloning requires a WAV reference and transcript; exact word timestamps can replace estimated prefix timing. Supported parenthetical vocal cues are sample-dependent and can harm intelligibility, so use them sparingly.

Validation

All three variants executed on GPU1, AMD RX 9070 16 GB, including temporal inference, all31 depth stages,32-codebook Mimi, synthetic audio-conditioned prefix warmup and cached-graph repeat checks. F16 passed numerical comparison against the official PyTorch implementation with F16-rounded LM weights; see parity-report.json. Quantized outputs are intentionally lossy: quantization-validation.json records their tensor formats, differences from F16 and a short generated-speech transcription. A smoke test is not a comprehensive evaluation of voice quality.

Private cloned voice recordings and transcripts are excluded. The source runtime also implements [S1]/[S2] dialogue, audio-prefix cloning and progressive PCM/SSE output. Live/incremental text input and speech-to-speech input are not implemented. Vulkan builds pass compilation but GPU inference validation here uses ROCm/HIP.

Speed on GPU1 (RX9070 16GB)

Same text and seed for all variants; warm full-WAV time is the median of3 repeats. Time per10s audio is normalized from the actual generated length, which varies by quantization. These are unconditioned requests: voice-reference cloning adds prefix preparation latency. Every streaming run matched its full PCM exactly. The short cold warmup took about7–9s including loading and synthesis; it is not model-load time alone. Peak VRAM includes runtime caches/temporary allocations and idle driver usage. GPU0 stayed essentially idle.

Precision	Warm seconds per10s audio	First audio, seconds	Peak VRAM, GB
F16	10.84	1.17	6.08
Q8_0	10.10	1.09	7.06
Q4_0	10.41	1.17	3.92

In this benchmark, ASR recognized the Q4_0 greeting with errors ("Blue there" / "Below there") while the F16/Q8_0 samples were closer to the text. This is a limited smoke check, not a general quality ranking.

See speed-benchmark.json; the sample WAV files use the same unconditioned benchmark text. Numbers describe this machine and workload, not a general performance guarantee. Q8_0 may require more runtime VRAM than F16 in this backend despite the smaller weight file.

Sources and licenses

• Dia2 weights/config/tokenizer: nari-labs/Dia2-2B, revision 7abae125471a73b0fc6b9d413cb15f4ae1e771d8, Apache-2.0. See LICENSE-Dia2; the repository-level license metadata refers to this component.
• Shared codec: Kyutai Mimi, revision 89091b3e466eb6a9d11e537bf26b144f194978f7, Creative Commons Attribution 4.0: https://creativecommons.org/licenses/by/4.0/. Mimi retains its separate license. Conversion changes tensor names, storage precision and rotary projection layout for the native runtime. Attribution: Kyutai, Mimi neural audio codec.
• Runtime: fork of 0xShug0/audio.cpp, retaining upstream attribution and license.

This is an experimental community conversion, not an official Nari Labs or Kyutai release.