Instructions to use ZhengmingYu/DMAD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ZhengmingYu/DMAD with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("ZhengmingYu/DMAD") prompt = "A man with short gray hair plays a red electric guitar." output = pipe(prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
Model card: title, badges and teaser
Browse files- .gitattributes +1 -0
- README.md +17 -8
- assets/teaser.gif +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
assets/teaser.gif filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -13,14 +13,23 @@ tags:
|
|
| 13 |
- minimax-h3
|
| 14 |
---
|
| 15 |
|
| 16 |
-
# DMAD
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
[
|
| 22 |
-
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
| File | Checkpoint | Size |
|
| 26 |
|------|------------|------|
|
|
|
|
| 13 |
- minimax-h3
|
| 14 |
---
|
| 15 |
|
| 16 |
+
# DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
|
| 17 |
+
|
| 18 |
+
### 4-step MiniMax-H3 students for joint audio-video generation
|
| 19 |
+
|
| 20 |
+
<!-- TODO: paper and code links -->
|
| 21 |
+
[](https://yzmblog.github.io/DMAD/)
|
| 22 |
+
[](https://arxiv.org/abs/TODO)
|
| 23 |
+
[](https://github.com/TODO)
|
| 24 |
+
|
| 25 |
+
<p align="center">
|
| 26 |
+
<img src="https://huggingface.co/ZhengmingYu/DMAD/resolve/main/assets/teaser.gif" alt="Videos generated by the 4-step DMAD student of MiniMax-H3 (video only; every clip also has generated audio)">
|
| 27 |
+
</p>
|
| 28 |
+
|
| 29 |
+
This repository holds the **4-step DMAD students of [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3)
|
| 30 |
+
(33B, text-to-audio-video)**: rank-128 LoRAs on the H3 transformer that turn the 50-step teacher into a 4-step
|
| 31 |
+
generator of 1344x768 video with native stereo audio. The inference code is in the
|
| 32 |
+
[code repository](https://github.com/TODO).
|
| 33 |
|
| 34 |
| File | Checkpoint | Size |
|
| 35 |
|------|------------|------|
|
assets/teaser.gif
ADDED
|
Git LFS Details
|