Hugging Face's logo

VideoRebirthAI
/
hyperflow-h3-flash

Image-Text-to-Video
MiniMax H3
Diffusers
Safetensors
GGUF
English
Chinese
hyperflow-h3
audio-video-generation
text-to-video
image-to-video
reference-to-video
few-step
int8
comfyui
stable-diffusion.cpp
unsloth
Model card Files
xet
 Community
1
hyperflow-h3-flash / README.md
videorebirth's picture
videorebirth
HyperFlow H3 Flash 1.0.0: INT8 convrot and GGUF for fl2va and ref2va
9e9ed14
preview code
|
Raw History Blame Contribute Delete
19.8 kB
---
license: other
license_name: minimax-h3-community-license-agreement
license_link: LICENSE
base_model:
- MiniMaxAI/MiniMax-H3
language:
- en
- zh
library_name: minimax-h3
pipeline_tag: image-text-to-video
tags:
- hyperflow-h3
- minimax-h3
- audio-video-generation
- text-to-video
- image-to-video
- reference-to-video
- few-step
- int8
- gguf
- comfyui
- stable-diffusion.cpp
- unsloth
---
# HyperFlow H3 Flash (14B)
> By downloading or using these weights, you accept the [MiniMax H3 Community License Agreement](LICENSE). The Agreement restricts territory (not licensed in the EU, UK, South Korea, or the US without prior authorization from MiniMax) and includes an Acceptable Use Policy. See [License](#license). This repository is not an official MiniMax product.
HyperFlow H3 Flash is [Video Rebirth](https://www.videorebirth.com/)'s compressed, few-step version of [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3). It generates video and synchronized stereo audio together.
**Showcase:** See [HyperFlow H3 Flash generated video and audio examples](https://www.videorebirth.com/lp/hyperflow-flash).
- **Output:** a 5.2 to 15 s video at 24 fps (1344×768 by default) with a 32 kHz stereo soundtrack.
- **Sampling:** 4, 6, or 8 steps without classifier-free guidance (CFG). The default is 6 steps.
- **Speed (5.2 s clip, 6 steps):** 8.7 s on 4× B200, 36 s on 1× H200, and 92 s on an RTX 5090.
- **Real time on 4× Blackwell GPUs:** [NVIDIA SOL-H3](https://github.com/NVlabs/Sana/tree/sol-engine/models/minimax_h3/HyperFlow-Flash) runs HyperFlow H3 Flash in real time on 4× B-series GPUs. SOL-H3 is our inference-acceleration partner and supports this model from day 0.
- **Model size:** 13.7B (`fl2va`) or 13.9B (`ref2va`) parameters, less than half of the ~31B MiniMax-H3 transformer.
- **Minimum footprint:** `Q4_K_M` is **9.1 GB** (`Q8_0` is **14.9 GB**). It runs on 24 GB and 32 GB GPUs in PCs with 16 GB of system RAM. 16 GB GPUs are not measured yet.
Code, setup guides, and ComfyUI workflows are in [`Video-Rebirth/hyperflow-h3-flash`](https://github.com/Video-Rebirth/hyperflow-h3-flash). For the 8-step LoRA on the full-size MiniMax-H3 transformer, see [`VideoRebirthAI/hyperflow`](https://huggingface.co/VideoRebirthAI/hyperflow) ([GitHub](https://github.com/Video-Rebirth/hyperflow)).
### Supported Tasks
| Task | Input | Hub partition | ComfyUI template | GGUF support | Python INT8 support |
|---|---|---|---|---|---|
| `t2va` | Text prompt | `fl2va_*` | `hyperflow_t2v` | Yes | Yes |
| `fl2va` | Prompt + first keyframe, or first and last keyframes | `fl2va_*` | `hyperflow_i2v`, `hyperflow_flf2v` | Yes | Yes |
| `ref2va` | Prompt + up to 12 references | `ref2va_*` | `hyperflow_ref2v` | Images only | Images, videos, and audio |
---
## Choose a Weight Format
**Decision rule:** Use **GGUF** on any PC with 64 GB of system RAM or less. Use **INT8 convrot** on data-center GPUs, or on 24 GB+ RTX GPUs with at least 69 GiB of system RAM.
| Feature | GGUF | INT8 convrot |
|---|---|---|
| **Hub folders** | `fl2va_gguf/`, `ref2va_gguf/` | `fl2va_int8_convrot/`, `ref2va_int8_convrot/` |
| **Precision & size** | `Q3_K_M` to `Q8_0` (7.2–15.2 GB) | W8A8 INT8 + low-rank (~16 GB) |
| **Runtime** | ComfyUI, Unsloth Studio, `stable-diffusion.cpp` | [`hyperflow-h3-flash`](https://github.com/Video-Rebirth/hyperflow-h3-flash) CLI and `diffusers` |
| **Target GPU** | Consumer GPUs; measured on **24 GB and 32 GB** RTX GPUs (16 GB GPUs not measured yet) | **H100 / H200 / B200 / B300** (1–8 GPUs), or **24 GB+ RTX GPUs** |
| **System RAM** | **16 GB+** | **~16 GiB** without offload; **≥ 69 GiB** (`fp8` text encoder) or **≥ 91 GiB** (`bf16`) with offload |
| **`ref2va` inputs** | Images | Images, videos, and audio |
---
## Repository Layout and Companion Files
```text
fl2va_int8_convrot/ config.json, diffusion_pytorch_model-0000{1..3}-of-00003.safetensors (+ index), model code
ref2va_int8_convrot/ config.json, diffusion_pytorch_model-0000{1..4}-of-00004.safetensors (+ index), model code
fl2va_gguf/ minimax_h3_fl2va_hyperflow_flash_14b-<QUANT>.gguf (6 quant levels)
ref2va_gguf/ minimax_h3_ref2va_hyperflow_flash_14b-<QUANT>.gguf (6 quant levels)
```
This repository stores only the compressed transformers (`transformer` for `fl2va`, `transformer_ref` for `ref2va`). Each runtime loads the shared text encoder, video VAE, audio VAE, tokenizer, and processor from their upstream repositories:
| Runtime | Companion text encoder and VAEs |
|---|---|
| **`hyperflow-h3-flash` (Python)** | Downloads automatically from [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3), or from [`Qwen/Qwen3-VL-32B-Instruct-FP8`](https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct-FP8) with `--text-encoder-precision fp8`. Never downloads the 62 GB base transformer. |
| **ComfyUI & stable-diffusion.cpp** | [`Comfy-Org/MiniMax-H3`](https://huggingface.co/Comfy-Org/MiniMax-H3): `text_encoders/qwen3vl_32b_minimax_h3_*.safetensors` (`nvfp4_awq` 15.7 GB, `int8_convrot` 27.1 GB, or `bf16` 51.5 GB), `vae/minimax_h3_video_vae_fp16.safetensors`, `vae/minimax_h3_audio_vae_fp32.safetensors` |
| **Unsloth Studio** | Downloads automatically from [`unsloth/MiniMax-H3-GGUF`](https://huggingface.co/unsloth/MiniMax-H3-GGUF) |
---
## GGUF (ComfyUI, Unsloth Studio, stable-diffusion.cpp)
| Quant | `fl2va_gguf/` | `ref2va_gguf/` | Recipe | When to use |
|---|---:|---:|---|---|
| **`Q4_K_M`** | **9.1 GB** | **9.2 GB** | `Q4_K` (lifted layers `Q6_K`) | **Recommended default for 24 GB GPUs** |
| `Q8_0` | 14.9 GB | 15.2 GB | `Q8_0` | Closest to full precision when VRAM and RAM fit |
| `Q6_K` | 11.6 GB | 11.8 GB | `Q6_K` | High quality, smaller than `Q8_0` |
| `Q5_K_M` | 10.3 GB | 10.5 GB | `Q5_K` (lifted layers `Q6_K`) | Balanced intermediate quant |
| `Q4_K_S` | 8.0 GB | 8.1 GB | `Q4_K` | Compact 4-bit quant |
| `Q3_K_M` | 7.2 GB | 7.4 GB | `Q3_K` (lifted layers `Q5_K`) | Smallest footprint |
Full setup instructions are in [`integrations/README.md`](https://github.com/Video-Rebirth/hyperflow-h3-flash/blob/main/integrations/README.md). Summary:
- **ComfyUI:**
1. Clone [ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF) and [ComfyUI-HyperFlow-Flash](https://github.com/Video-Rebirth/ComfyUI-HyperFlow-Flash) into `ComfyUI/custom_nodes/`.
2. Put the `.gguf` file in `models/diffusion_models/` and download the 15.7 GB `qwen3vl_32b_minimax_h3_nvfp4_awq` text encoder.
3. On 16 GB or 32 GB RAM PCs, start ComfyUI with `python main.py --fast-disk`.
4. Open **Workflow > Browse Templates > ComfyUI-HyperFlow-Flash** and select `hyperflow_t2v`, `hyperflow_i2v`, `hyperflow_flf2v`, or `hyperflow_ref2v`.
- **Unsloth Studio:** Point `SD_CLI_PATH` at the patched `sd-cli` binary (`export SD_CLI_PATH=/path/to/stable-diffusion.cpp/build/bin/sd-cli`), run `unsloth studio`, and load `VideoRebirthAI/hyperflow-h3-flash` (`fl2va_gguf/` or `ref2va_gguf/`). Studio downloads the companion files and runs the 6-step schedule at `CFG = 1`.
- **stable-diffusion.cpp:** Apply [`integrations/stable-diffusion.cpp/hyperflow-two-time.patch`](https://github.com/Video-Rebirth/hyperflow-h3-flash/tree/main/integrations/stable-diffusion.cpp) to upstream commit `3be2378`, build, and run:
```bash
sd-cli -M vid_gen \
--diffusion-model minimax_h3_fl2va_hyperflow_flash_14b-Q4_K_M.gguf \
--llm qwen3vl_32b_minimax_h3_bf16.safetensors \
--vae minimax_h3_video_vae_fp16.safetensors \
--audio-vae minimax_h3_audio_vae_fp32.safetensors \
--sampling-method euler --cfg-scale 1.0 --steps 6 \
-W 1344 -H 768 --video-frames 124 --fps 24 -s 42 \
-p "A chef slides a steaming bowl of ramen across a wooden counter; rain taps on the window." \
-o output.webm
```
Add `-i first.png` (and `--end-img last.png`) for keyframes. Use `minimax_h3_ref2va_...` with `--ref-image a.png --ref-image b.png` for reference images. Add `--offload-to-cpu` on smaller GPUs.
---
## INT8 convrot (`hyperflow-h3-flash` & `diffusers`)
### Install
```bash
pip install git+https://github.com/Video-Rebirth/hyperflow-h3-flash
pip install "hyperflow-h3-flash[hopper] @ git+https://github.com/Video-Rebirth/hyperflow-h3-flash" # H100 / H200: FlashAttention-3
pip install "hyperflow-h3-flash[fp8] @ git+https://github.com/Video-Rebirth/hyperflow-h3-flash" # --text-encoder-precision fp8
```
- **Requirements:** Python 3.10+, PyTorch 2.6+ with CUDA and Triton (`pip install triton-windows` on native Windows), an Ampere or newer NVIDIA GPU, and `diffusers >= 0.40`. Multi-GPU (`--gpus` > 1) requires Linux or WSL2.
- **First run:** Downloads ~90 GB (~68 GB with `--text-encoder-precision fp8`), covering the text encoder, VAEs, and the 16 GB INT8 transformer for the selected task. Triton INT8 kernels compile on first use (~1 min).
### Command Line
```bash
# Text to video + audio (t2va)
hyperflow-h3-flash --prompt "A chef slides a steaming bowl of ramen across a wooden counter; rain taps on the window." \
--output ramen.mp4
# First keyframe, or first + last keyframe (fl2va)
hyperflow-h3-flash --prompt "..." --image first.png --last-image last.png --output clip.mp4
# Reference images, videos, and audio clips in prompt order (ref2va)
hyperflow-h3-flash --prompt "<Picture 1> walks through the scene of <Picture 2> ..." \
--reference person.png --reference scene.png --output clip.mp4
# 15 s clip across 4 GPUs; or fp8 text encoder to save ~22 GiB system RAM when offloading
hyperflow-h3-flash --prompt "..." --num-frames 360 --gpus 4
hyperflow-h3-flash --prompt "..." --steps 4 --text-encoder-precision fp8
```
| Option | Values (default) | Description |
|---|---|---|
| `--steps` | `4`, `6` (`6`), `8` | `4` for speed, `6` balanced, `8` for quality |
| `--num-frames` | `124`–`362` (`124`) | 5.2 to 15 s at 24 fps; rounds up to `17n + 5` |
| `--gpus` | `1`, `2`, `4`, `7`, `8`, `auto` | Context and VAE tile parallelism under `torchrun` |
| `--offload` | `auto` (`auto`), `none`, `model`, `stream` | `none` needs ≥ 90 GiB VRAM; `model`/`stream` need ≥ 91 GiB RAM (`bf16`) or ≥ 69 GiB (`fp8`) |
| `--text-encoder-precision` | `bf16` (`bf16`), `fp8` | `fp8` saves ~22 GiB system RAM when offloading (requires `[fp8]`) |
| `--attention` | `auto` (`auto`), `sdpa`, `fa3` | Uses FlashAttention-3 (`fa3`) on Hopper when `kernels` is installed |
| `--block-chunk` | `0` (`0`), `16384`, ... | Slice token-wise block ops (`--block-chunk 16384` fits 15 s mixed `ref2va` on 24 GB GPUs) |
### Python API (`load_pipeline`)
```python
import torch
from diffusers.utils.export_utils import encode_video
from hyperflow_h3_flash import load_pipeline
pipe = load_pipeline("t2va") # "fl2va": pass image=... (and last_image=...); "ref2va": pass references=[...]
out = pipe(
prompt="A chef slides a steaming bowl of ramen across a wooden counter; rain taps on the window.",
num_frames=124,
num_inference_steps=6, # 4, 6, or 8
generator=torch.Generator().manual_seed(0),
output=["videos", "audio", "sampling_rate"],
)
encode_video(
out["videos"][0],
fps=24,
output_path="ramen.mp4",
audio=out["audio"][0],
audio_sample_rate=out["sampling_rate"],
)
```
### Standalone Transformer (`AutoModel`)
Each INT8 folder ships `hyperflow_h3_flash_transformer.py` and `hyperflow_h3_flash_int8.py`. `AutoModel` loads the transformer without installing `hyperflow-h3-flash`:
```python
import torch
from diffusers import AutoModel
model = AutoModel.from_pretrained(
"VideoRebirthAI/hyperflow-h3-flash",
subfolder="fl2va_int8_convrot", # or "ref2va_int8_convrot"
trust_remote_code=True,
torch_dtype=torch.bfloat16,
)
model.pad_lowrank_ranks() # pad low-rank dimensions to multiples of 64 for aligned GEMMs
model.to("cuda")
# Optional on Hopper GPUs (requires `pip install kernels`):
model.set_attention_backend("_flash_3_hub")
```
> Note: The transformer requires `endpoint_timestep` on every forward pass for two-time `(t, r)` conditioning. Use `load_pipeline` or `hyperflow_h3_flash.flash_blocks` to run the full sampling loop.
---
## Benchmarks and Memory
All benchmarks measure end-to-end generation (text encoding, 6-step transformer, and video and audio VAE decode) at **1344×768 and 6 steps**. Clip lengths: `5.2 s` = 124 frames; `15 s` = 362 frames.
### 1. Consumer RTX GPUs (`5.2 s / 15 s` total time)
| GPU | GGUF `Q4_K_M` (ComfyUI)<br>≥ 64 GB RAM | GGUF `--fast-disk`<br>32 GB PC | GGUF `--fast-disk`<br>16 GB PC | INT8 (`hyperflow-h3-flash`)¹<br>`--offload stream` |
|---|---|---|---|---|
| **RTX 5090 (32 GB)** | 99 s / 575 s | 118 s / 618 s | 130 s / 615 s | 92 s / 551 s |
| **RTX 4090 (24 GB)** | 157 s / 830 s | 151 s / 815 s | 153 s / 812 s | 130 s / 750 s |
| **RTX 3090 (24 GB)** | 315 s / 1835 s | 318 s / 1869 s | 325 s / 1866 s | 266 s / 1684 s |
| *Peak VRAM (5.2 s / 15 s)* | *5090: 28 / 27 GiB; 24 GB: 23 / 22 GiB* | *same as ≥ 64 GB* | *same as ≥ 64 GB* | *12.0–12.5 / 21.8–22.4 GiB* |
- **GGUF columns:** Use the 15.7 GB `nvfp4_awq` text encoder. `--fast-disk` streams unresident weights from the OS file page cache instead of pinned RAM, keeping anonymous RSS at ~8–16 GiB.
- **¹ INT8 column:** Requires ≥ 91 GiB system RAM with the `bf16` text encoder, or ~69 GiB with `--text-encoder-precision fp8`.
**Reference to video (`ref2va`) on one RTX GPU (`5.2 s / 15 s` total time):**
| GPU | INT8, 2 images | INT8, mixed² | GGUF `match`, 2 images | GGUF `max`, 2 images |
|---|---|---|---|---|
| **RTX 5090 (32 GB)** | 266 s / 914 s | 503 s / 1318 s | 135 s / 660 s | 295 s / 995 s³ |
| **RTX 4090 (24 GB)** | 319 s / 1167 s | 630 s / 1702 s⁴ | 194 s / 885 s | 398 s / 1275 s³ |
| **RTX 3090 (24 GB)** | 726 s / 2763 s | 1441 s / 4033 s⁴ | 401 s / 2059 s | 885 s / 2995 s³ |
- **² Mixed:** One reference image, one 5.2 s reference video with soundtrack, and one audio clip. **³** Start ComfyUI with `--disable-cuda-malloc` for `max` at 15 s. **⁴** Pass `--block-chunk 16384` on 24 GB GPUs for 15 s mixed `ref2va`. On 32 GB / 16 GB PCs with `--fast-disk`, 2-image `ref2v` (`match`) runs in `139 s / 663 s` and `157 s / 684 s` on RTX 5090, `180 s / 866 s` and `178 s / 876 s` on RTX 4090, and `385 s / 2010 s` and `384 s / 2025 s` on RTX 3090.
### 2. Data-Center GPUs (`hyperflow-h3-flash` INT8)
| GPUs | `--offload` | 5.2 s (124 frames) | 15 s (362 frames) | Peak VRAM / GPU (5.2 s / 15 s) | System RAM / GPU |
|---|---|---|---|---|---|
| 1× H200 (141 GB) | `none` | 36 s | 229 s | 80 / 91 GiB | 16 GiB |
| 2× H200 (141 GB) | `none` | 24 s | 119 s | — / 71 GiB | 16 GiB |
| 4× H200 (141 GB) | `none` | 12.6 s | 66 s | 47 / 59 GiB | 16 GiB |
| 4× B200 | `none` | 8.7 s | — | 46 GiB | 16 GiB |
The table measures the `hyperflow-h3-flash` package. For real time on 4× B-series GPUs, use [NVIDIA SOL-H3](https://github.com/NVlabs/Sana/tree/sol-engine/models/minimax_h3/HyperFlow-Flash).
---
## Step Schedules and Prompting
### Step Grids
Both partitions and both weight formats share three unshifted sigma grids (`video_shift = 12`, `audio_shift = 3`):
| Steps | Mode | Raw sigmas |
|---:|---|---|
| `4` | Speed | `1, 0.681872, 0.205019, 0.036496, 0` |
| `6` | Balanced (default) | `1, 0.825217, 0.471725, 0.147401, 0.049626, 0.015517, 0` |
| `8` | Quality | `1, 0.931506, 0.839236, 0.703462, 0.5, 0.296538, 0.160764, 0.068494, 0` |
### Prompts
- Describe the subject, action, setting, camera movement, and sound in one prompt. Do not use a negative prompt (`CFG = 1.0`).
- For `ref2va`, name inputs in prompt order as `<Picture i>`, `<Video i>`, and `<Audio i>`. Limits: 9 images, 3 videos, 3 audio clips, and 12 references total. Audio references require at least one image or video reference. GGUF runtimes accept image references only; video and audio references require the Python package.
---
## How It Works
1. **Structural pruning and low-rank factorization:** `fl2va` drops 3 of 50 blocks (6, 21, 22) and factors 102 linear layers (13.7B parameters). `ref2va` drops 1 of 50 blocks (9) and factors 114 linear layers (13.9B parameters).
2. **Rotated W8A8 INT8 (`convrot`):** 180 linear layers are quantized to W8A8 INT8 behind a block-diagonal Hadamard rotation. Kernels dispatch to fused Triton (`sm_80`–`sm_90`, `sm_120`), `cuBLASLt` INT8 (`sm_100`), or BF16 tensor cores (`sm_103` B300).
3. **Two-time `(t, r)` conditioning:** Every token receives the current noise level `t` and the step endpoint `r`. This enables 4-to-8-step sampling without CFG.
---
## Acknowledgements
Thanks to the MiniMax team for releasing [MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3), to [Unsloth](https://huggingface.co/unsloth/MiniMax-H3-GGUF) for the MiniMax-H3 GGUF layout and naming convention, to [`WAS/Minimax-h3_Singularity_v2_pruned`](https://huggingface.co/WAS/Minimax-h3_Singularity_v2_pruned) for pioneering rotated INT8 on MiniMax-H3, and to [stable-diffusion.cpp](https://github.com/leejet/stable-diffusion.cpp), [ComfyUI](https://github.com/Comfy-Org/ComfyUI), and [ComfyUI-GGUF](https://github.com/city96/ComfyUI-GGUF).
## Intended Use and Limitations
- HyperFlow H3 Flash provides fast, reduced-memory joint video and stereo audio generation across data-center and consumer GPUs (4–8 steps, 7.2–16 GB transformer weights).
- Prompt adherence, supported languages, biases, and failure modes follow the [MiniMax-H3 base model](https://huggingface.co/MiniMaxAI/MiniMax-H3).
- Every use of these weights is bound by the license terms below, including territorial restrictions and the Acceptable Use Policy.
## License
The transformer weights in this repository (both INT8 convrot and GGUF) are a Model Derivative of MiniMax-H3 and are distributed under the [MiniMax H3 Community License Agreement](LICENSE) with no additional terms. By downloading or using them, you accept that Agreement:
- **Territory:** Licensed worldwide **excluding the European Union, the United Kingdom, the Republic of Korea, and the United States of America**. Individuals and organizations in those jurisdictions must obtain MiniMax's prior authorization before downloading ([application form](https://platform.minimax.io/h3-license); [License Q&A](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/docs/QA-about-License.md)).
- **Redistribution:** Include a copy of the Agreement and the [`NOTICE`](NOTICE) file, clearly mark modified files, and do not impose additional or different terms (Section III).
- **AI-Generated Content Disclosure:** Publicly distributed content generated with these weights must be clearly disclosed as AI-generated (Acceptable Use Policy, item 12).
- **Commercial Use:** Products or services exceeding $20 million USD in annual revenue require prior written authorization from MiniMax (`api@minimax.io`). Products and services built with the weights must prominently display `"MiniMax H3"` on their user interface (Section IV.2).
- **Hosted Services & Use Restrictions:** Hosted services must implement the safeguards in Section V.5 and bind end users to Section V and the Acceptable Use Policy (Exhibit A). Outputs may not be used to train or improve other AI models.
- **Upstream Text:** [`LICENSE`](LICENSE) reproduces the August 2, 2026 Agreement text; the version in the [upstream repository](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE) governs.
See [`NOTICE`](NOTICE) for the architectural and numerical modifications from MiniMax-H3. The inference code on [GitHub](https://github.com/Video-Rebirth/hyperflow-h3-flash) is licensed separately under Apache-2.0 (ComfyUI custom node under GPL-3.0; `stable-diffusion.cpp` patch under MIT).
HyperFlow H3 Flash is developed by Video Rebirth and is not an official MiniMax product. Powered by MiniMax H3.