--- license: other license_name: audiolift-component-licenses license_link: LICENSE.md base_model: - FrancisRing/StableAvatar - wlyu/OpenFaceLift tags: - audiolift - multi-view - audio-driven-animation - gaussian-splatting --- # AudioLift **Audio-Driven Full-Head Gaussian Avatar via Multi-view Video Generation** This repository provides the two fine-tuned models used by AudioLift. The weights are publicly downloadable without a Hugging Face login. | Model | File | Purpose | | --- | --- | --- | | AudioLift-mvdiffusion | `AudioLift-mvdiffusion.pt` | Generate five synchronized video views from a portrait and speech audio. | | AudioLift-gaussianlifter | `AudioLift-gaussianlifter.pt` | Reconstruct and render a Gaussian head from the five generated views. | The original selected checkpoint bytes are preserved. Only the public filenames have changed. File checksums are recorded in [SHA256SUMS](SHA256SUMS). ## Usage Use the AudioLift inference entrypoints in the [AudioLift repository](https://github.com/ZhuoyuanJiang/AudioLift). The entrypoints download missing models to the local `checkpoints/` directory and verify the fine-tuned checkpoint checksums before loading them. To download both models and their required base assets before inference, run `python scripts/download_models.py` from an installed AudioLift environment. See the [installation guide](https://github.com/ZhuoyuanJiang/AudioLift/blob/main/docs/installation.md) for environment setup and validation status. AudioLift-mvdiffusion also requires the Wan and wav2vec2 assets from [StableAvatar](https://huggingface.co/FrancisRing/StableAvatar). These are downloaded from the upstream model repository. AudioLift-gaussianlifter requires the camera and model configuration files supplied with the AudioLift code. These are inference checkpoints. Training code is not included in the public AudioLift code release. ## Licenses and acknowledgements See [model licenses](LICENSE.md) before use. The Gaussian reconstruction weights retain FaceLift's Adobe Research License for noncommercial research. Built on [StableAvatar](https://github.com/Francis-Rings/StableAvatar), [Wan](https://github.com/Wan-Video/Wan2.1) and [FaceLift](https://github.com/weijielyu/FaceLift).