AXERA-TECH/Qwen3-TTS-0.6B-CustomVoice-AX8860-llama.cpp

Qwen3-TTS-12Hz-0.6B-CustomVoice on AX8860 (llama.cpp + ggml-axera, 2026-11-09)

在爱芯 AX8860 的 NPU（4 核）上用 llama.cpp 运行 Qwen3-TTS-12Hz-0.6B-CustomVoice 语音合成：九个内置音色，可选风格指令。 Qwen3-TTS-12Hz-0.6B-CustomVoice speech synthesis on the 4-core NPU of the Axera AX8860: nine built-in speakers and an optional style instruction.

性能 / Performance

AX8860，SDK V0.5.1，NPU 4 核。本版只更新板端 bin，kernels 与 GGUF 不变；以下为发布目录实测 / 4 cores; this release only updates the board binaries, the kernels and GGUFs are unchanged; measured with the release directory:

项目 / Item	结果 / Result
一段中文，20.9 s 语音，进程总耗时（含加载与首次声码器转换）/ one Chinese paragraph, 20.9 s of speech, total process time	25 s
稳定段 talker + codec / steady-state talker + codec	约 / about 40 帧/s（25 ms/帧 per 80 ms frame）
每进程加载与权重转换 / load and weight conversion, once per process	约 / about 14 s

文件 / Files

文件 / File	说明 / Description
qwen3-tts-0.6b-customvoice-bf16.gguf	talker 权重（GGUF BF16，1.1 GB，md5 e2346116107e6418a05e720baa4665fa）/ talker weights
mmproj-qwen3-tts-0.6b-customvoice-bf16.gguf	code predictor 与声码器权重（GGUF BF16，575 MB，md5 294c4dc42e3caaa0572a21df43e1be2b）/ code predictor and codec decoder weights
bin/	llama.cpp 板端程序 llama-tts（静态链接，只依赖系统 /opt/lib 与 libc）/ the board program (static; needs only the system /opt/lib and libc)
kernels/	NPU kernel 4 个：talker prefill、talker decode（含 codec 输出头）、code predictor、声码器 + manifest.json / four kernels: talker prefill, talker decode (with the codec head), code predictor and codec decoder + manifest.json
run.sh	运行脚本 / run script

权重只来自 GGUF：kernel 不含权重，加载时从 GGUF 转换（talker 与 code predictor MXFP8，声码器 BF16）。 The weights come only from the GGUFs: the kernels hold none; they are converted at load (talker and code predictor MXFP8, codec decoder BF16).

运行 / Run

环境 / Requirements：AX8860，SDK V0.5.0 或更新（cat /proc/ax_proc/version）/ SDK V0.5.0 or newer.

./bin/llama-tts -m qwen3-tts-0.6b-customvoice-bf16.gguf \
  --mmproj mmproj-qwen3-tts-0.6b-customvoice-bf16.gguf \
  -p "今天天气很好，我们一起去公园散步吧。" --tts-speaker vivian --tts-lang zh -o out.wav

--tts-speaker 必填（例如 vivian、dylan、eric 等九个内置音色）；--tts-instruct 可选，0.6B 会忽略并给出警告（与官方一致）。--tts-lang 支持 zh en de it pt es ja ko fr ru 与 auto。 --tts-speaker is required (nine built-in speakers such as vivian, dylan, eric); --tts-instruct is optional and ignored with a warning by 0.6B (matching the official wrapper). --tts-lang accepts zh en de it pt es ja ko fr ru and auto.

直接调用程序时设置 /opt/lib 与 kernels/ / when calling the program directly: LD_LIBRARY_PATH=/opt/lib GGML_AXERA_KERNEL_DIR=<repo>/kernels，-ngl 99 -c 4096，top-k 不超过 50 / top-k up to 50。

正确性 / Correctness

• 中文长句合成后用 Qwen3-ASR-1.7B 回转，与原文逐字一致；英文测试句回转得到 "The quick brown fox jumps over the lazy dog." / the long Chinese output was transcribed back by Qwen3-ASR-1.7B exactly, and the English test sentence transcribed back verbatim.

已知限制 / Known limits

• 运行时独占 NPU 4 核 / takes all 4 NPU cores; do not run other NPU programs alongside.
• 上下文 4096；top-k 最大 50 / context 4096; top-k at most 50.
• 贪心解码（--temp 0）的语音偏长、停顿多，建议用默认采样 / greedy decoding gives long, pause-heavy speech; keep the default sampling.

源码 / Sources

• llama.cpp：ZHEQIUSHUI/llama.cpp axera-dev @ 2f81e2416（ggml/src/ggml-axera/models/qwen3、qwen3_tts；语音编码器 tools/mtmd/models/qwen3tts-encode.cpp）
• kernel：ZHEQIUSHUI/ax-dsl-kernels main @ b60fc30（models/qwen3_tts）

更新 / Changelog

• 2026-11-09：更新板端 bin（llama.cpp 2f81e2416）；kernels 与 GGUF 不变 / new board binaries; kernels and GGUF unchanged.