MegaAvatar / README.md
Gaojunyao's picture
Update README.md
c69fc84 verified
|
Raw History Blame Contribute Delete
2.69 kB
---
pipeline_tag: image-to-video
tags:
- video-generation
- talking-avatar
- human-animation
- audio-driven
- smpl-x
- wan2.2
---
<h1 align="center">MegaAvatar: Controllable Talking Avatar Generation</h1>
<div align="center">
**Junyao Gao\*<sup>1,2</sup> &nbsp; Sibo Liu\*<sup>1</sup> &nbsp; Weidong Zhang<sup>1</sup> &nbsp; Cairong Zhao<sup>2‡</sup> &nbsp; Jun Zhang<sup>1‡</sup>**
<sup>1</sup>Tencent AIPD &nbsp; <sup>2</sup>Tongji University &nbsp; \*Equal contribution &nbsp; ‡Corresponding authors
[![GitHub](https://img.shields.io/badge/GitHub-MegaAvatar-black)](https://github.com/Jeoyal/MegaAvatar)
[![arXiv](https://img.shields.io/badge/arXiv-Paper-red)](https://arxiv.org/abs/2609.39273)
[![Dataset](https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-Dataset-yellow)](https://huggingface.co/datasets/Gaojunyao/MegaAvatar)
</div>
---
MegaAvatar is a controllable talking avatar generation framework built on top of **Wan2.2-TI2V-5B**.
Given a reference portrait image, MegaAvatar generates talking avatar videos with:
- controllable body and head motion using SMPL-X;
- speech-synchronized lip motion and facial expressions using audio;
- identity-preserving facial appearance.
MegaAvatar supports both **SMPL-X-driven** and **audio-driven** generation.
## Model Download
```bash
hf download Gaojunyao/MegaAvatar --include "checkpoints/*" --local-dir .
```
The released checkpoints include:
```text
checkpoints/
├── megaavatar.safetensors
└── audio2smplx/
```
- `megaavatar.safetensors`: MegaAvatar video generation model.
- `audio2smplx/`: audio-to-SMPL-X components.
## Usage
Please refer to the official GitHub repository for installation, inference, and training instructions:
https://github.com/Jeoyal/MegaAvatar
MegaAvatar supports:
```text
Reference Image + SMPL-X + Audio → Talking Avatar Video
Reference Image + Audio → SMPL-X → Talking Avatar Video
```
## Related Resources
- Code: https://github.com/Jeoyal/MegaAvatar
- Model: https://huggingface.co/Gaojunyao/MegaAvatar
- Dataset: https://huggingface.co/datasets/Gaojunyao/MegaAvatar
- Paper: https://arxiv.org/abs/2609.39273
## Acknowledgements
This project benefits from [FantasyTalking](https://github.com/Fantasy-AMAP/fantasy-talking), [SpeakerVid-5M-Code](https://github.com/Dorniwang/SpeakerVid-5M-Code), and [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio).
## Citation
```bibtex
@article{gao2026megaavatar,
title={MegaAvatar: Controllable Talking Avatar Generation},
author={Gao, Junyao and Liu, Sibo and Zhang, Weidong and Zhao, Cairong and Zhang, Jun},
journal={arXiv preprint arXiv:2609.39273},
year={2026}
}
```