| --- |
| license: other |
| license_name: minimax-h3-license |
| license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE |
| base_model: MiniMaxAI/MiniMax-H3 |
| tags: |
| - svdquant |
| - w4a4 |
| - int4 |
| - video |
| - text-to-video |
| - quantized |
| --- |
| |
| # MiniMax-H3 · SVDQuant W4A4 (int4, rank 32, GPTQ) |
|
|
| 4-bit **weights and activations** for the MiniMax-H3 31B video+audio |
| transformer — true [SVDQuant](http://arxiv.org/abs/2411.05007) (ICLR 2025 |
| Spotlight): activation outliers absorbed into a 16-bit rank-32 low-rank |
| branch, the residual GPTQ-rounded to int4, activations quantized to int4 |
| per-token at runtime, executed on fused CUTLASS tensor-core kernels. This |
| is not weight-only quantization. |
|
|
| The text conditioner (Qwen3-VL 31B) ships W4A16+GPTQ in the same release: |
| **every stored weight is int4**; only norms, embeddings, and the visual |
| tower stay bf16, matching MiniMax's own quantization recipe. |
|
|
| ## Measured (NVIDIA A100 80GB, 124 frames @ 24fps, 960x544) |
|
|
| | | BF16 | this release | factor | |
| |---|---|---|---| |
| | DiT checkpoint | 61.7 GB | **19.6 GB** | 3.15x | |
| | generation wall-clock | 484 s | **369 s** | **1.31x faster** | |
| | vs unfused reference dequant | 2427 s | 369 s | 6.6x | |
| | quantized GEMM (layer level) | — | — | 1.37-1.38x | |
|
|
| **All-resident configuration** (this release's DiT + TE together, |
| no CPU offload — unreachable for BF16 on one 80GB card): |
|
|
| | | BF16 (offloaded) | all-int4 resident | factor | |
| |---|---|---|---| |
| | generation wall-clock | 484 s | **318 s** | **1.52x faster** | |
| | pipeline VRAM | 65 GB peak, offload churn | 48.9 GB steady, 54.3 peak | fits | |
| | DiT + TE weights on disk | 123.8 GB | 37.6 GB | 3.3x | |
|
|
| Conversion cost: 44 min for the DiT (18 calib + 26 GPTQ) on one A100. |
| Kernel outputs agree with the fp32 reference oracle to 1.5-2.1% (the |
| bf16-vs-fp32 activation-rounding delta) at every layer shape. |
|
|
| Quality: same-seed renders are visually indistinguishable from BF16 |
| (samples in this repo). On the Z-Image anchor, the same pipeline's GPTQ |
| checkpoint scores **better LPIPS than the officially published nunchaku |
| checkpoint** (0.288 vs 0.334). |
|
|
| ## Before / after (same seed, sound on) |
|
|
| | BF16 original | this release (all-int4) | |
| |---|---| |
| | <video controls src="https://huggingface.co/ModelsLab/MiniMax-H3-svdquant-int4_r32/resolve/main/samples/p0_bf16.mp4"></video> | <video controls src="https://huggingface.co/ModelsLab/MiniMax-H3-svdquant-int4_r32/resolve/main/samples/p0_ours_int4.mp4"></video> | |
| | <video controls src="https://huggingface.co/ModelsLab/MiniMax-H3-svdquant-int4_r32/resolve/main/samples/p1_bf16.mp4"></video> | <video controls src="https://huggingface.co/ModelsLab/MiniMax-H3-svdquant-int4_r32/resolve/main/samples/p1_ours_int4.mp4"></video> | |
|
|
| Prompts: a red fox in a snowy pine forest; a saxophonist under neon in |
| light rain. Video and soundtrack are generated jointly by the model. |
|
|
| ## Use |
|
|
| ```bash |
| pip install git+https://github.com/ModelsLab/svdquant git+https://github.com/rootonchair/nunchaku-lite |
| ``` |
|
|
| ```python |
| import svdquant |
| transformer = svdquant.load_model("minimax-h3-packed.safetensors") # this repo's file |
| # drop into the diffusers ModularPipeline in place of the BF16 transformer |
| ``` |
|
|
| Requires an int4-tensor-core GPU (sm_75-89: RTX 20/30/40, A100) and |
| torch >= 2.11. An NVFP4 sibling for RTX 50-series (sm_120 has no int4 |
| path) is planned from a fresh BF16 pass — int4 and fp4 grids do not nest, |
| so transcoding is never used. |
|
|
| ## Honest notes |
|
|
| 1. Attention stays bf16 — at video sequence lengths it bounds the |
| end-to-end speedup (Amdahl); the 1.31x reflects that. |
| 2. The `token_refiner` (2 blocks, ~2% of params) stays bf16, following |
| MiniMax's own int8 recipe. |
| 3. The AWQ repack of the 50 modulation layers re-derives scales; groups |
| GPTQ pushed to -8 take one extra bounded rounding. |
|
|
| ## Independently verified (cold download) |
|
|
| Reproduced on a **separate, fresh A100 80GB** by downloading only this |
| repo's files plus the base model — the exact user path: |
|
|
| | | BF16 (offloaded) | this release (resident) | |
| |---|---|---| |
| | generation | 496.7 s | **350.9 s (1.42x)** | |
| | peak VRAM | 65.2 GB | 54.6 GB (49.2 steady) | |
|
|
| Raw log: `verified.json` in this repo. First-build A100 measured 1.52x; |
| host-to-host variance puts the honest claim at **1.4-1.5x**. |
|
|
| ## Credits and license |
|
|
| Weights derive from [MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3) |
| and inherit its license. Method: SVDQuant (Li et al., MIT HAN Lab). |
| Kernels and packed layout: [nunchaku](https://github.com/nunchaku-ai/nunchaku), |
| [nunchaku-lite](https://github.com/rootonchair/nunchaku-lite) and |
| [diffuse-compressor](https://github.com/rootonchair/diffuse-compressor) |
| by rootonchair (Apache-2.0, vendored with attribution). Quantized with |
| [svdquant](https://github.com/ModelsLab/svdquant) by ModelsLab. |
|
|
