AudioLift

Audio-Driven Full-Head Gaussian Avatar via Multi-view Video Generation

This repository provides the two fine-tuned models used by AudioLift. The weights are publicly downloadable without a Hugging Face login.

Model File Purpose
AudioLift-mvdiffusion AudioLift-mvdiffusion.pt Generate five synchronized video views from a portrait and speech audio.
AudioLift-gaussianlifter AudioLift-gaussianlifter.pt Reconstruct and render a Gaussian head from the five generated views.

The original selected checkpoint bytes are preserved. Only the public filenames have changed. File checksums are recorded in SHA256SUMS.

Usage

Use the AudioLift inference entrypoints in the AudioLift repository. The entrypoints download missing models to the local checkpoints/ directory and verify the fine-tuned checkpoint checksums before loading them. To download both models and their required base assets before inference, run python scripts/download_models.py from an installed AudioLift environment. See the installation guide for environment setup and validation status.

AudioLift-mvdiffusion also requires the Wan and wav2vec2 assets from StableAvatar. These are downloaded from the upstream model repository. AudioLift-gaussianlifter requires the camera and model configuration files supplied with the AudioLift code.

These are inference checkpoints. Training code is not included in the public AudioLift code release.

Licenses and acknowledgements

See model licenses before use. The Gaussian reconstruction weights retain FaceLift's Adobe Research License for noncommercial research.

Built on StableAvatar, Wan and FaceLift.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for DanJZY/AudioLift

Finetuned
(1)
this model