license: apache-2.0
library_name: audio.cpp
base_model: Soul-AILab/SoulX-FlashHead-1_3B
base_model_relation: quantized
tags:
- audio.cpp
- gguf
- talking-head
- image-to-video
- audio-to-video
SoulX-FlashHead Lite GGUF
Follow or star audio.cpp for the C++ runtime and model integration updates.
GGUF packaging of Soul-AILab/SoulX-FlashHead-1_3B.
The model generates streaming talking-portrait video from a reference image and speech audio.
This repository packages Lite only, including its DiT, LTX video VAE and
Wav2Vec2 speech encoder.
Upstream checkpoint revision: 5fbd2e44ba08ab2deb28bd55cff2c15523e51ba5.
Model architecture and weights are by Soul-AILab; the C++ integration is in audio.cpp.
cuDNN is strongly recommended for CUDA: about 2× faster warm inference, with lower peak VRAM. Prebuilt audio.cpp binaries do not include cuDNN support. Build from source with cuDNN and
-DGGML_CUDA_CUDNN_CONV3D=ON, then select--session-option 'soulx_flashhead.exec_plan={"vae":{"impl":"cudnn"}}'. The default GGML path remains available without cuDNN, including in prebuilts.
English streaming demo
Generated from a single reference portrait and 42.24 seconds of English narration using audio.cpp: Lite Q8, 512×512, four denoising steps, 25 FPS, Vulkan backend. Audio is processed in 0.96-second chunks with continuous motion between chunks.
Inputs: English audio, transcript, and portrait. The portrait is from the upstream example. The English narration was generated with Qwen3-TTS.
audio.cpp CUDA performance
Measured on an RTX 5090 with the Q8 package, 512×512 output, four denoising steps, and the 42.24-second English demo above (44 streaming chunks). Debug build, eight CPU threads; the two modes were run sequentially.
| VAE implementation | Warm RTF ↓ | Full-run RTF ↓ | Peak VRAM |
|---|---|---|---|
| cuDNN (recommended) | 0.215 | 0.430 | 5,168 MiB |
| GGML (default) | 0.490 | 0.706 | 9,152 MiB |
RTF is generation time divided by video duration; lower is faster. Warm RTF covers the full audio-encoder, DiT, VAE, color-correction and RGB-output pipeline, excluding the first two chunks and CLI artifact serialization. Full-run RTF also includes initialization, cold chunks and CLI artifact serialization. Both exclude model loading and MP4 encoding. Peak VRAM is sampled per-process GPU memory. The cuDNN VAE uses F16 activations with F32 accumulation.
Weights and usage
| File | Components |
|---|---|
SoulX-FlashHead-Lite-Q8_0.gguf |
Q8 DiT, BF16 LTX VAE, F32 Wav2Vec2, embedded configuration and model contract |
SoulX-FlashHead-Lite-BF16.gguf |
BF16 DiT and LTX VAE, F32 Wav2Vec2, embedded configuration and model contract |
audiocpp_cli --family soulx_flashhead --task gen \
--model SoulX-FlashHead-Lite-Q8_0.gguf --backend cuda \
--audio speech.wav --request-option reference_image_file=portrait.png \
--mode streaming --out-dir outputs/flashhead \
--log --log-file outputs/flashhead/run.log
For a cuDNN-enabled source build, add the following option to the command above:
--session-option 'soulx_flashhead.exec_plan={"vae":{"impl":"cudnn"}}'
Lite emits 24 RGB24 frames per chunk at 25 FPS. Streaming input is mono 16 kHz; offline input is mixed down and resampled when needed. Default output is 512×512 with four denoising steps. Chunks retain motion continuity from the preceding output.
See the usage and conversion documentation.