AXERA-TECH/Qwen3-TTS-1.7B-VoiceDesign-AX8860-llama.cpp

Qwen3-TTS-12Hz-1.7B-VoiceDesign on AX8860 (llama.cpp + ggml-axera, 2026-11-09)

在爱芯 AX8860 的 NPU（4 核）上用 llama.cpp 运行 Qwen3-TTS-12Hz-1.7B-VoiceDesign 语音合成：用一段自然语言描述设计音色。 Qwen3-TTS-12Hz-1.7B-VoiceDesign speech synthesis on the 4-core NPU of the Axera AX8860: design a voice from a natural-language description.

性能 / Performance

AX8860，SDK V0.5.1，NPU 4 核。本版只更新板端 bin，kernels 与 GGUF 不变；以下为发布目录实测 / 4 cores; this release only updates the board binaries, the kernels and GGUFs are unchanged; measured with the release directory:

项目 / Item	结果 / Result
一段中文，22.2 s 语音，进程总耗时（含加载与首次声码器转换）/ one Chinese paragraph, 22.2 s of speech, total process time	36 s
稳定段 talker + codec / steady-state talker + codec	约 / about 32 帧/s（31 ms/帧 per 80 ms frame）
每进程加载与权重转换 / load and weight conversion, once per process	约 / about 25 s

文件 / Files

文件 / File	说明 / Description
qwen3-tts-1.7b-voicedesign-bf16.gguf	talker 权重（GGUF BF16，3.2 GB，md5 c0c13fc8bb1c946f33739d4538b5056e）/ talker weights
mmproj-qwen3-tts-1.7b-voicedesign-bf16.gguf	code predictor 与声码器权重（GGUF BF16，645 MB，md5 6200b0762d600203d46a52df6a3d9a91）/ code predictor and codec decoder weights
bin/	llama.cpp 板端程序 llama-tts（静态链接，只依赖系统 /opt/lib 与 libc）/ the board program (static; needs only the system /opt/lib and libc)
kernels/	NPU kernel 4 个：talker prefill、talker decode（含 codec 输出头）、code predictor、声码器 + manifest.json / four kernels: talker prefill, talker decode (with the codec head), code predictor and codec decoder + manifest.json
run.sh	运行脚本 / run script

权重只来自 GGUF：kernel 不含权重，加载时从 GGUF 转换（talker 与 code predictor MXFP8，声码器 BF16）。 The weights come only from the GGUFs: the kernels hold none; they are converted at load (talker and code predictor MXFP8, codec decoder BF16).

运行 / Run

环境 / Requirements：AX8860，SDK V0.5.0 或更新（cat /proc/ax_proc/version）/ SDK V0.5.0 or newer.

./bin/llama-tts -m qwen3-tts-1.7b-voicedesign-bf16.gguf \
  --mmproj mmproj-qwen3-tts-1.7b-voicedesign-bf16.gguf \
  -p "今天天气很好，我们一起去公园散步吧。" --tts-lang zh \
  --tts-instruct "A calm female voice, warm and measured." -o out.wav

--tts-instruct 必填，用自然语言描述想要的音色；VoiceDesign 不接受 --tts-speaker 或参考录音。--tts-lang 支持 zh en de it pt es ja ko fr ru 与 auto。 --tts-instruct is required and describes the target voice in natural language; VoiceDesign rejects --tts-speaker and reference audio. --tts-lang accepts zh en de it pt es ja ko fr ru and auto.

直接调用程序时设置 /opt/lib 与 kernels/ / when calling the program directly: LD_LIBRARY_PATH=/opt/lib GGML_AXERA_KERNEL_DIR=<repo>/kernels，-ngl 99 -c 4096，top-k 不超过 50 / top-k up to 50。

正确性 / Correctness

• 中文长句（按描述设计的音色）合成后用 Qwen3-ASR-1.7B 回转，与原文逐字一致 / the long Chinese output in the designed voice was transcribed back exactly by Qwen3-ASR-1.7B.

已知限制 / Known limits

• 运行时独占 NPU 4 核 / takes all 4 NPU cores; do not run other NPU programs alongside.
• 上下文 4096；top-k 最大 50 / context 4096; top-k at most 50.
• 贪心解码（--temp 0）的语音偏长、停顿多，建议用默认采样 / greedy decoding gives long, pause-heavy speech; keep the default sampling.

源码 / Sources

• llama.cpp：ZHEQIUSHUI/llama.cpp axera-dev @ 0104880be（ggml/src/ggml-axera/models/qwen3、qwen3_tts；语音编码器 tools/mtmd/models/qwen3tts-encode.cpp）

更新 / Changelog

• 2026-11-09：更新板端 bin（llama.cpp 2f81e2416）；kernels 与 GGUF 不变 / new board binaries; kernels and GGUF unchanged.