Text-to-Video
Diffusers
Safetensors
MiniMaxH3ModularPipeline
video-generation
audio
fastvideo
nvfp4
qad
Instructions to use FastVideo/FastH3-QAD-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use FastVideo/FastH3-QAD-v2 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("FastVideo/FastH3-QAD-v2", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
[docs]: make the V2.1 model card self-contained
Browse filesDescribe V2.1 and its QAD recipe directly; remove the V2 comparison table and the earlier-run narrative.
README.md
CHANGED
|
@@ -5,36 +5,32 @@ tags: [video-generation, text-to-video, audio, fastvideo, nvfp4, qad]
|
|
| 5 |
---
|
| 6 |
# FastH3 V2.1, 8-step, NVFP4 for one GPU (QAD)
|
| 7 |
|
| 8 |
-
FastH3 V2.1 is
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
(VSA) gates. The q/k/v/out and MLP weights were trained with QAD. The VSA gates and all norms are V2's, byte-identical to the
|
| 16 |
-
Consumer release. Attention itself runs in bf16 sparse attention (VSA, tile 64, sparsity 0.8), as in V2.
|
| 17 |
- **Text encoder:** NVFP4 Qwen3-VL trimmed to the 50 layers H3 reads.
|
| 18 |
- **VAE:** LynnReal lightweight video VAE with Kijai's INT8 weights; H3 audio VAE.
|
| 19 |
- **Sampling:** `fastvideo_inference.json` holds the 8-step DMD schedule, which FastVideo reads automatically.
|
| 20 |
|
| 21 |
-
##
|
| 22 |
-
|
| 23 |
-
QAD trains the NVFP4 student to match the bf16 V2 teacher, step by step along the teacher's 8-step trajectory (data-free:
|
| 24 |
-
text prompts only, 300 updates, 8 prompts per update, lr 5e-6, 25% of rows on the student's own trajectory). Only the
|
| 25 |
-
quantized linears were trained. The VSA gates and norms were frozen: training the gates improved the metric but caused
|
| 26 |
-
ghosting on fast motion.
|
| 27 |
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
-
|
| 31 |
-
|---|---|---|
|
| 32 |
-
| Mean (video + audio) | 0.431 | **0.351** |
|
| 33 |
-
| Video | 0.471 | 0.382 |
|
| 34 |
-
| Audio | 0.391 | 0.319 |
|
| 35 |
|
| 36 |
-
|
| 37 |
-
|
|
|
|
|
|
|
|
|
|
| 38 |
|
| 39 |
## Machines with less than 64 GB of system RAM
|
| 40 |
|
|
|
|
| 5 |
---
|
| 6 |
# FastH3 V2.1, 8-step, NVFP4 for one GPU (QAD)
|
| 7 |
|
| 8 |
+
FastH3 V2.1 is an 8-step video-and-audio model with NVFP4 weights, trained with **quantization-aware distillation
|
| 9 |
+
(QAD)**. It runs on one Blackwell GPU (RTX 5090, RTX PRO 6000) or a DGX Spark.
|
| 10 |
+
|
| 11 |
+
- **Transformer:** 50 blocks, NVFP4 W4A4 on all attention q/k/v/out and MLP linears, plus the sparse-attention
|
| 12 |
+
(VSA) gates. The q/k/v/out and MLP weights were trained with QAD. The VSA gates and all norms stay in BF16.
|
| 13 |
+
- **Attention:** block-sparse VSA with 64-token tiles and sparsity 0.8. The checkpoint was trained with BF16 VSA.
|
| 14 |
+
FastVideo also exposes an FP4 VSA inference lane on Blackwell.
|
|
|
|
|
|
|
| 15 |
- **Text encoder:** NVFP4 Qwen3-VL trimmed to the 50 layers H3 reads.
|
| 16 |
- **VAE:** LynnReal lightweight video VAE with Kijai's INT8 weights; H3 audio VAE.
|
| 17 |
- **Sampling:** `fastvideo_inference.json` holds the 8-step DMD schedule, which FastVideo reads automatically.
|
| 18 |
|
| 19 |
+
## Quantization-aware distillation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
+
QAD trains the NVFP4 student against a frozen BF16 teacher along the teacher's 8-step trajectory. The student runs
|
| 22 |
+
the deployed 4-bit weights and activations, so it learns weights that work with the quantized forward pass. Training
|
| 23 |
+
uses text prompts only: 300 steps, learning rate 5e-6, with 25% of rows on the student's own trajectory. Only the
|
| 24 |
+
quantized linears train. The VSA gates and norms are frozen because changing them changes which video tiles the
|
| 25 |
+
model attends to, not just the arithmetic of attention.
|
| 26 |
|
| 27 |
+
Error against the BF16 teacher on 30 held-out prompts (relative L2 of the x0 prediction, lower is better):
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
|
| 29 |
+
| Metric | V2.1 (QAD) |
|
| 30 |
+
|---|---:|
|
| 31 |
+
| Mean (video + audio) | 0.351 |
|
| 32 |
+
| Video | 0.382 |
|
| 33 |
+
| Audio | 0.319 |
|
| 34 |
|
| 35 |
## Machines with less than 64 GB of system RAM
|
| 36 |
|