AudioLift / README.md
DanJZY's picture
Document public AudioLift checkpoint downloads
6d29bb6 verified
|
Raw History Blame Contribute Delete
2.25 kB
metadata
license: other
license_name: audiolift-component-licenses
license_link: LICENSE.md
base_model:
  - FrancisRing/StableAvatar
  - wlyu/OpenFaceLift
tags:
  - audiolift
  - multi-view
  - audio-driven-animation
  - gaussian-splatting

AudioLift

Audio-Driven Full-Head Gaussian Avatar via Multi-view Video Generation

This repository provides the two fine-tuned models used by AudioLift. The weights are publicly downloadable without a Hugging Face login.

Model File Purpose
AudioLift-mvdiffusion AudioLift-mvdiffusion.pt Generate five synchronized video views from a portrait and speech audio.
AudioLift-gaussianlifter AudioLift-gaussianlifter.pt Reconstruct and render a Gaussian head from the five generated views.

The original selected checkpoint bytes are preserved. Only the public filenames have changed. File checksums are recorded in SHA256SUMS.

Usage

Use the AudioLift inference entrypoints in the AudioLift repository. The entrypoints download missing models to the local checkpoints/ directory and verify the fine-tuned checkpoint checksums before loading them. To download both models and their required base assets before inference, run python scripts/download_models.py from an installed AudioLift environment. See the installation guide for environment setup and validation status.

AudioLift-mvdiffusion also requires the Wan and wav2vec2 assets from StableAvatar. These are downloaded from the upstream model repository. AudioLift-gaussianlifter requires the camera and model configuration files supplied with the AudioLift code.

These are inference checkpoints. Training code is not included in the public AudioLift code release.

Licenses and acknowledgements

See model licenses before use. The Gaussian reconstruction weights retain FaceLift's Adobe Research License for noncommercial research.

Built on StableAvatar, Wan and FaceLift.