Instructions to use ZhengmingYu/DMAD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use ZhengmingYu/DMAD with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("MiniMaxAI/MiniMax-H3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("ZhengmingYu/DMAD") prompt = "A man with short gray hair plays a red electric guitar." output = pipe(prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Inference
- Notebooks
- Google Colab
- Kaggle
Model card: 16 GB / 22 GB GPUs (ComfyUI workflows at a 14 GiB cap, inference.py --low-vram)
Browse files
README.md
CHANGED
|
@@ -62,7 +62,7 @@ token-refiner blocks (312 modules). alpha = rank = 128. The safetensors metadata
|
|
| 62 |
|
| 63 |
The inference code lives in the [code repository](https://github.com/Yzmblog/DMAD): `inference.py` with the sampler
|
| 64 |
the paper used (re-noise step rule) and a Diffusers-pipeline example; its README covers the environment. Sampling
|
| 65 |
-
settings: 4 steps, time shift 12 (video) and 2 (audio), no classifier-free guidance, 124 frames at 24 fps.
|
| 66 |
|
| 67 |
```bash
|
| 68 |
git clone https://github.com/Yzmblog/DMAD.git && cd DMAD # code + environment setup (see its README)
|
|
@@ -120,6 +120,13 @@ ComfyUI-Manager), which both workflows include. Without it, 5 s (length 124) fit
|
|
| 120 |
the one made with the memory node. Times are end to end (model loading, text encoding, sampling, decoding) on an H200
|
| 121 |
with the PyTorch allocator capped at 24 GiB; a consumer GPU is slower, and sampling time scales linearly with steps.
|
| 122 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 123 |
## Wan2.1 (text-to-video)
|
| 124 |
|
| 125 |
Full generators (EMA, spectral norm folded in) in the `.pth` format of
|
|
|
|
| 62 |
|
| 63 |
The inference code lives in the [code repository](https://github.com/Yzmblog/DMAD): `inference.py` with the sampler
|
| 64 |
the paper used (re-noise step rule) and a Diffusers-pipeline example; its README covers the environment. Sampling
|
| 65 |
+
settings: 4 steps, time shift 12 (video) and 2 (audio), no classifier-free guidance, 124 frames at 24 fps. `--low-vram` runs it on a 16 GB GPU.
|
| 66 |
|
| 67 |
```bash
|
| 68 |
git clone https://github.com/Yzmblog/DMAD.git && cd DMAD # code + environment setup (see its README)
|
|
|
|
| 120 |
the one made with the memory node. Times are end to end (model loading, text encoding, sampling, decoding) on an H200
|
| 121 |
with the PyTorch allocator capped at 24 GiB; a consumer GPU is slower, and sampling time scales linearly with steps.
|
| 122 |
|
| 123 |
+
**16 GB and 22 GB GPUs.** Both workflows also run unchanged with the GPU memory capped at 14 GiB (room for a 16 GB
|
| 124 |
+
card's desktop and CUDA context): ComfyUI's dynamic VRAM keeps less of the model resident and streams more of it from
|
| 125 |
+
system memory, and the videos are bit-identical to the 24 GiB runs. About 46 GiB of system memory is in use, so 64 GB
|
| 126 |
+
of RAM is recommended. Outside ComfyUI, `inference.py --low-vram` in the
|
| 127 |
+
[code repository](https://github.com/Yzmblog/DMAD) makes 5 s and 15 s videos in under 14 GiB of GPU memory, bit-identical
|
| 128 |
+
to its larger-GPU runs.
|
| 129 |
+
|
| 130 |
## Wan2.1 (text-to-video)
|
| 131 |
|
| 132 |
Full generators (EMA, spectral norm folded in) in the `.pth` format of
|