AudioLift
Audio-Driven Full-Head Gaussian Avatar via Multi-view Video Generation
This repository provides the two fine-tuned models used by AudioLift. The weights are publicly downloadable without a Hugging Face login.
| Model | File | Purpose |
|---|---|---|
| AudioLift-mvdiffusion | AudioLift-mvdiffusion.pt |
Generate five synchronized video views from a portrait and speech audio. |
| AudioLift-gaussianlifter | AudioLift-gaussianlifter.pt |
Reconstruct and render a Gaussian head from the five generated views. |
The original selected checkpoint bytes are preserved. Only the public filenames have changed. File checksums are recorded in SHA256SUMS.
Usage
Use the AudioLift inference entrypoints in the
AudioLift repository.
The entrypoints download missing models to the local checkpoints/ directory
and verify the fine-tuned checkpoint checksums before loading them.
To download both models and their required base assets before inference, run
python scripts/download_models.py from an installed AudioLift environment.
See the installation guide
for environment setup and validation status.
AudioLift-mvdiffusion also requires the Wan and wav2vec2 assets from StableAvatar. These are downloaded from the upstream model repository. AudioLift-gaussianlifter requires the camera and model configuration files supplied with the AudioLift code.
These are inference checkpoints. Training code is not included in the public AudioLift code release.
Licenses and acknowledgements
See model licenses before use. The Gaussian reconstruction weights retain FaceLift's Adobe Research License for noncommercial research.
Built on StableAvatar, Wan and FaceLift.