Image-Text-to-Video
MiniMax H3
hyperflow-h3
lora
video-generation
audio-video-generation
text-to-video
image-to-video
text-to-audio-video
image-to-audio-video
reference-to-audio-video
flow-matching
distillation
self-distillation
few-step
Instructions to use VideoRebirthAI/hyperflow with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Inference
- Notebooks
- Google Colab
- Kaggle
docs: write approximate timings as about instead of tilde
Browse files
README.md
CHANGED
|
@@ -46,7 +46,7 @@ holds the LoRA only; the loader and examples live at
|
|
| 46 |
| Adapter | PEFT LoRA, rank 256 / alpha 256, on attention, feed-forward and both time embedders (316 modules); 2.8 GB |
|
| 47 |
| Sampling | 8 forward passes on a fixed sigma grid stored in the file (video shift 12, audio shift 3) |
|
| 48 |
| Workflows | `t2va`, `fl2va` and `ref2va`, one file for all three (`ref2va` loads it onto `transformer_ref/`) |
|
| 49 |
-
| Speed-up |
|
| 50 |
| Optional | Ulysses context parallel on up to 4 GPUs; FlashAttention-3; NVIDIA Sol-Attn sparse attention |
|
| 51 |
| License | [MiniMax H3 Community License Agreement](LICENSE); code on GitHub is Apache-2.0. See [License](#license) |
|
| 52 |
|
|
@@ -133,15 +133,15 @@ hyperflow-h3-fl2va --prompt "..." --image first.png --gpus 1 # the official si
|
|
| 133 |
|
| 134 |
HyperFlow (8 NFE, 9 sigma points) against the base pipeline at Diffusers' default (49 NFE, 50 sigma points) on the
|
| 135 |
same clip: `fl2va`, 124 frames, 1344x768, seed 0, dense SDPA attention. Pipeline time covers text encoding, denoising
|
| 136 |
-
and decoding; process start-up and weight loading add
|
| 137 |
|
| 138 |
| | Base, 49 NFE | HyperFlow, 8 NFE | Speed-up |
|
| 139 |
|---|---|---|---|
|
| 140 |
-
| 4x H200, Ulysses degree 4 |
|
| 141 |
-
| 1x H200, auto CPU offload |
|
| 142 |
|
| 143 |
The speed-up is below the 49:8 NFE ratio because text encoding and VAE decoding, which HyperFlow leaves unchanged,
|
| 144 |
-
now take much of the run. Peak accelerator memory is
|
| 145 |
single-GPU row uses `memory_reserve_margin="24GB"`, not the official recipe's 12 GB: on a 141 GB H200 a smaller margin
|
| 146 |
evicts the 10 GB VAE instead of the 62 GB text encoder when the DiT arrives and starves the denoiser. An 80 GB card
|
| 147 |
is untested.
|
|
@@ -196,7 +196,7 @@ meta.base_model_revision, meta.sigmas, meta.gate
|
|
| 196 |
|
| 197 |
## Intended use and limitations
|
| 198 |
|
| 199 |
-
- A drop-in speed-up for MiniMax-H3 inference on diffusers: same inputs, same outputs (video + stereo audio),
|
| 200 |
faster end to end. It is a distilled few-step sampler, so compare against the base Diffusers pipeline on your own
|
| 201 |
prompts before adopting it (`--baseline` in the example scripts runs both on the same seed).
|
| 202 |
- Prompt following, languages, biases and failure modes are the base model's, which the distillation is designed to
|
|
|
|
| 46 |
| Adapter | PEFT LoRA, rank 256 / alpha 256, on attention, feed-forward and both time embedders (316 modules); 2.8 GB |
|
| 47 |
| Sampling | 8 forward passes on a fixed sigma grid stored in the file (video shift 12, audio shift 3) |
|
| 48 |
| Workflows | `t2va`, `fl2va` and `ref2va`, one file for all three (`ref2va` loads it onto `transformer_ref/`) |
|
| 49 |
+
| Speed-up | about 3x end to end against the 49-NFE Diffusers baseline (about 60 s vs about 175 s per clip on 4x H200) |
|
| 50 |
| Optional | Ulysses context parallel on up to 4 GPUs; FlashAttention-3; NVIDIA Sol-Attn sparse attention |
|
| 51 |
| License | [MiniMax H3 Community License Agreement](LICENSE); code on GitHub is Apache-2.0. See [License](#license) |
|
| 52 |
|
|
|
|
| 133 |
|
| 134 |
HyperFlow (8 NFE, 9 sigma points) against the base pipeline at Diffusers' default (49 NFE, 50 sigma points) on the
|
| 135 |
same clip: `fl2va`, 124 frames, 1344x768, seed 0, dense SDPA attention. Pipeline time covers text encoding, denoising
|
| 136 |
+
and decoding; process start-up and weight loading add about 30 s.
|
| 137 |
|
| 138 |
| | Base, 49 NFE | HyperFlow, 8 NFE | Speed-up |
|
| 139 |
|---|---|---|---|
|
| 140 |
+
| 4x H200, Ulysses degree 4 | about 175 s | about 60 s | 2.9x |
|
| 141 |
+
| 1x H200, auto CPU offload | about 395 s | about 130 s | 3.0x |
|
| 142 |
|
| 143 |
The speed-up is below the 49:8 NFE ratio because text encoding and VAE decoding, which HyperFlow leaves unchanged,
|
| 144 |
+
now take much of the run. Peak accelerator memory is about 80 GB per card either way (the LoRA adds 2.8 GB). The
|
| 145 |
single-GPU row uses `memory_reserve_margin="24GB"`, not the official recipe's 12 GB: on a 141 GB H200 a smaller margin
|
| 146 |
evicts the 10 GB VAE instead of the 62 GB text encoder when the DiT arrives and starves the denoiser. An 80 GB card
|
| 147 |
is untested.
|
|
|
|
| 196 |
|
| 197 |
## Intended use and limitations
|
| 198 |
|
| 199 |
+
- A drop-in speed-up for MiniMax-H3 inference on diffusers: same inputs, same outputs (video + stereo audio), about 3x
|
| 200 |
faster end to end. It is a distilled few-step sampler, so compare against the base Diffusers pipeline on your own
|
| 201 |
prompts before adopting it (`--baseline` in the example scripts runs both on the same seed).
|
| 202 |
- Prompt following, languages, biases and failure modes are the base model's, which the distillation is designed to
|