AudioLift / README.md
DanJZY's picture
Document public AudioLift checkpoint downloads
6d29bb6 verified
|
Raw History Blame Contribute Delete
2.25 kB
---
license: other
license_name: audiolift-component-licenses
license_link: LICENSE.md
base_model:
- FrancisRing/StableAvatar
- wlyu/OpenFaceLift
tags:
- audiolift
- multi-view
- audio-driven-animation
- gaussian-splatting
---
# AudioLift
**Audio-Driven Full-Head Gaussian Avatar via Multi-view Video Generation**
This repository provides the two fine-tuned models used by AudioLift.
The weights are publicly downloadable without a Hugging Face login.
| Model | File | Purpose |
| --- | --- | --- |
| AudioLift-mvdiffusion | `AudioLift-mvdiffusion.pt` | Generate five synchronized video views from a portrait and speech audio. |
| AudioLift-gaussianlifter | `AudioLift-gaussianlifter.pt` | Reconstruct and render a Gaussian head from the five generated views. |
The original selected checkpoint bytes are preserved. Only the public filenames
have changed. File checksums are recorded in [SHA256SUMS](SHA256SUMS).
## Usage
Use the AudioLift inference entrypoints in the
[AudioLift repository](https://github.com/ZhuoyuanJiang/AudioLift).
The entrypoints download missing models to the local `checkpoints/` directory
and verify the fine-tuned checkpoint checksums before loading them.
To download both models and their required base assets before inference, run
`python scripts/download_models.py` from an installed AudioLift environment.
See the [installation guide](https://github.com/ZhuoyuanJiang/AudioLift/blob/main/docs/installation.md)
for environment setup and validation status.
AudioLift-mvdiffusion also requires the Wan and wav2vec2 assets from
[StableAvatar](https://huggingface.co/FrancisRing/StableAvatar). These are downloaded
from the upstream model repository. AudioLift-gaussianlifter requires the camera
and model configuration files supplied with the AudioLift code.
These are inference checkpoints. Training code is not included in the public
AudioLift code release.
## Licenses and acknowledgements
See [model licenses](LICENSE.md) before use. The Gaussian reconstruction weights
retain FaceLift's Adobe Research License for noncommercial research.
Built on [StableAvatar](https://github.com/Francis-Rings/StableAvatar),
[Wan](https://github.com/Wan-Video/Wan2.1) and
[FaceLift](https://github.com/weijielyu/FaceLift).