INC4AI's picture
Update README.md
eddba7f verified
|
Raw History Blame Contribute Delete
12 kB
---
pipeline_tag: text-to-video
license: other
license_name: minimax-h3-community-license-agreement
license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE
library_name: minimax-h3
base_model: MiniMaxAI/MiniMax-H3
base_model_relation: quantized
tags:
- text-to-video
- text-to-audio-video
- image-to-video
- audio-video-generation
- multimodal
- mxfp8
- fp8
- quantized
- auto-round
- vllm
---
# MiniMax-H3 β€” MXFP8 **DiT + Text-Encoder**
An **MXFP8 (OCP microscaling FP8)** weight-quantized build of
[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) β€” the omni-modal
(video + native stereo audio) generative DiT β€” produced with
[Intel auto-round](https://github.com/intel/auto-round).
** *both* diffusion transformers **and** the Qwen3-VL **text encoder**
are quantized to MXFP8. (The companion repo `MiniMax-H3-mxfp8-native` quantizes only the two DiTs
and keeps the text encoder in BF16 β€” see "Relationship to the other variants".) The large **AdaLN**
branch of each DiT is deliberately **kept in BF16**.
**TL;DR**
- Each DiT **~62 GB (bf16) β†’ ~43.65 GB (β‰ˆ0.71Γ—, βˆ’29 %)**; AdaLN (β‰ˆ13 B params/DiT) stays BF16.
- Text encoder (Qwen3-VL, 64 decoder layers) **~63 GB (bf16) β†’ ~34 GB (β‰ˆ0.54Γ—, βˆ’46 %)**: the
`model.language_model.layers` linears are MXFP8 (448 Γ— F8_E4M3 + 448 Γ— U8 scale); the vision tower,
embeddings and norms stay BF16.
- Serves end-to-end on **2Γ—H200 (TP2)** and produces video **with synchronized audio**. H200 runs the
FP8 weights through a Marlin **dequant** kernel (correct, memory-saving, **not** faster).
---
## Showcase β€” official T2VA example
Reproduction of the official MiniMax-H3 **Text-to-Audio-Video (T2VA)** showcase
([`assets/t2va.mp4`](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/assets/t2va.mp4)),
generated on **our own vLLM-Omni stack with identical settings** (1344Γ—768, 24 fps, 32 steps,
10 s, seed 0, flow-shift 12 / audio-flow-shift 3.0, task `t2va`) so the **only** variable is the
quantization. Prompt = the official H3-Context-IR "starship captain" T2VA prompt.
| bf16 (reference) | MXFP8 DiT+TE (this model) |
|---|---|
| <video controls width="100%" src="assets/t2va_official_bf16.mp4"></video> [`assets/t2va_official_bf16.mp4`](assets/t2va_official_bf16.mp4) | <video controls width="100%" src="assets/t2va_official_mxfp8_full.mp4"></video> [`assets/t2va_official_mxfp8_full.mp4`](assets/t2va_official_mxfp8_full.mp4) |
Both clips: 1344Γ—768, 24 fps, 243 frames (10.12 s), video **+ stereo audio**, matching the official
`t2va.mp4` geometry. The MXFP8 clip is faithful to the prompt (starship bridge; silver-haired female
captain close-up) β€” the quantized text encoder is genuinely active.
Quantitative quality (dense93 VBench) is in the [Quality](#quality--vbench-dense93-h200-2026-10-14-n93)
section below: `imaging_quality` **0.6284** (this model, DiT+TE) vs **0.6306** (bf16), i.e. βˆ’0.35%.
---
## What was quantized
H3 has **two** DiTs sharing one architecture, plus one shared text encoder:
| Component | Role | Quantized? |
|---|---|---|
| `transformer` (FL2VA / T2VA DiT, 33.12 B) | video+audio denoiser | βœ… MXFP8 (from **BF16**) |
| `Ref2VA` DiT (33.12 B) | reference-conditioned denoiser | βœ… MXFP8 (from **BF16**) |
| **`text_encoder`** (Qwen3-VL) | text / vision encoding | βœ… **MXFP8** (language-model layers) |
| `video_vae` / `audio_vae` | video / audio decode | ❌ unchanged |
| `adaln_proj.linear` (Γ—52/DiT) | AdaLN modulation (~13 B) | ❌ **kept BF16 on purpose** |
**Per DiT β€” quantized Linears** (6 per block Γ— 52 = **312**):
`attn.{to_q,to_k,to_v,to_out.0}` + `ff.net.0.proj` (SwiGLU gate+up) + `ff.net.2` (down).
Per DiT: **312 Γ— F8_E4M3** weights + **312 Γ— U8** E8M0 block scales + **314 Γ— BF16** (AdaLN, norms,
embedders, biases). AdaLN stays BF16 because vLLM-Omni precomputes and **caches** its modulation β€”
pure inference never runs it through a matmul, so quantizing it would cost accuracy for zero benefit.
**Text encoder β€” quantized Linears** (7 per layer Γ— 64 = **448**):
every `model.language_model.layers.N.{self_attn.{q,k,v,o}_proj, mlp.{gate,up,down}_proj}`.
**448 Γ— F8_E4M3** weights (β‰ˆ29.06 GiB) + **448 Γ— U8** block scales (β‰ˆ0.91 GiB) + **610 Γ— BF16**
(β‰ˆ4.01 GiB: vision tower, embeddings, final norm, lm_head). The vision encoder is **not** quantized.
**Quant format** (per component `config.json β†’ quantization_config`):
`quant_method=auto-round`, `data_type=mx_fp`, `bits=8`, `group_size=32`, `sym=True`;
activations `act_data_type=mx_fp`, `act_bits=8`, `act_dynamic=True`, `act_group_size=32`;
DiT `block_name_to_quantize="token_refiner.blocks,blocks"`, TE
`block_name_to_quantize="model.language_model.layers"`, `packing_format="auto_round:llm_compressor"`;
`autoround_version=0.17.0`.
---
## Size / compression
| Component (per instance) | bf16 | MXFP8 | ratio |
|---|---:|---:|---:|
| DiT (`transformer`, Γ—2) | ~62 GiB | **43.65 GiB** | 0.71Γ— (βˆ’29 %) |
| text encoder (Qwen3-VL) | ~63 GiB | **~34 GiB** | 0.54Γ— (βˆ’46 %) |
Per-DiT breakdown (safetensors header scan): 312 Γ— F8_E4M3 = 18.66 GiB, 312 Γ— U8 = 0.58 GiB,
314 Γ— BF16 (AdaLN + norms + embedders) = 24.34 GiB, 12 Γ— F32 = 0.06 GiB β†’ **43.65 GiB**.
Because the source is already **BF16** and AdaLN (~55 % of bytes) stays BF16, the per-DiT ratio is a
modest ~0.71Γ—; the text encoder, with no cached-modulation carve-out, compresses further (~0.54Γ—).
**On-disk total β‰ˆ 142 GiB** for this repo (both partitions). The two pipeline folders share **one**
34 GiB quantized text encoder via hardlinks, so the TE is stored **once** rather than duplicated
---
## Directory layout
Native, serve-ready layout (what vLLM-Omni's *combined* mode loads). All entries are **real files**
(materialized via hardlinks; `find -type l` is empty):
```
MiniMax-H3-mxfp8-full-native/
β”œβ”€β”€ model_index.json # pipeline manifest
β”œβ”€β”€ README.md # this card
β”œβ”€β”€ assets/ # showcase mp4s
β”œβ”€β”€ transformer/config.json # ~1 KB shim: carries DiT quantization_config for auto-detect
β”œβ”€β”€ FL2VA/ # self-contained T2VA/FL2VA pipeline
β”‚ β”œβ”€β”€ transformer/ # 11 shards, MXFP8 DiT (+ .weight_scale)
β”‚ β”œβ”€β”€ text_encoder/ # 7 shards, MXFP8 Qwen3-VL (+ quantization_config)
β”‚ β”œβ”€β”€ video_vae/ audio_vae/ tokenizer/ processor/ model_index.json
└── Ref2VA/ # same structure; text_encoder shares inodes with FL2VA
```
Each partition's `text_encoder` is the quantized Qwen3-VL (its `config.json` carries the
`quantization_config` with `data_type=mx_fp`), **not** the BF16 encoder shipped by the base model.
---
---
## Serving (vLLM-Omni, 2Γ—H200 / TP2)
```bash
CUDA_VISIBLE_DEVICES=2,7 \
VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve intel-ai/MiniMax-H3-MXFP8-CT-AutoRound --omni \
--host 0.0.0.0 --port 8092 --trust-remote-code \
--num-gpus 2 --tensor-parallel-size 2 --text-encoder-tp-size 2 \
--enforce-eager --diffusion-attention-backend CUDNN_ATTN \
--init-timeout 1800 --stage-init-timeout 1200
# no --task-type β‡’ combined (loads FL2VA + Ref2VA; required for T2VA)
```
- **TP2 is mandatory**: `create_weights` allocates the BF16 skeleton first (two DiTs β‰ˆ 176 GB), so a
single 141 GB H200 OOMs; `--enable-cpu-offload` is incompatible (Marlin repack is CUDA-only).
### Required vLLM-Omni patch (text encoder)
Serving the **quantized text encoder** needs a small, un-upstreamed patch to
`vllm_omni/diffusion/models/minimax_h3/encoder.py` (4 fixes):
1. **Component-level quant routing** β€” read the serialized TE `config.json β†’ quantization_config`
(`data_type=mx_fp`) and build a text-encoder quant config scoped to
`text_encoder.text_model.layers` (the pipeline otherwise passes the DiT config, so the TE loads
unquantized).
2. **RowParallel scale-aware loader** β€” shard the grouped `weight_scale` (in/32) on the same dim as
the weight, so scales stay aligned under TP.
3. **Suffix-agnostic fused mapping** β€” fuse `q/k/v β†’ qkv_proj` and `gate/up β†’ gate_up_proj` for both
`.weight` **and** `.weight_scale`.
4. **Post-load repack** β€” run `process_weights_after_loading` on the TE's quantized Linears after
device placement (the pipeline skips the generic `post_load_weights`, and MarlinMxfp8 repack needs
CUDA).
Without the patch: ~700 `weight_scale has no target parameter` warnings, the TE falls back to an
unquantized/zeroed path, and the model ignores the prompt.
*(The two DiTs additionally need the serving fixes: a top-level `transformer/config.json`
quant anchor, grouped-qkv / fc1 scale reordering in `load_weights`, and TP2 without cpu-offload.)*
---
## Hardware note (important)
H200 (Hopper) has **no native MXFP8 tensor core**; vLLM-Omni runs these weights through a Marlin
weight-only **dequant** GEMM (documented as `MXFP8 W8A16 GEMM via Marlin`): it dequantizes the MXFP8
weight (E4M3 + E8M0 block scale) inside the GEMM and multiplies against **unquantized bf16
activations**. It is functionally correct and memory-saving, but **not faster** than bf16 on Hopper.
The checkpoints' `act_data_type=mx_fp / act_bits=8` is quantization-recipe metadata that is **not
exercised on H200** (activations stay bf16). True W8A8 (FP8 activations on a native microscaling
tensor core) requires Blackwell/B300. Dequant does not change stored values, so accuracy matches a
native kernel.
---
## Quality β€” VBench (dense93, H200, 2026-10-14, n=93)
Both models generate the **same 93 dense-prompt videos** (512Γ—288, 16 steps, 24 fps, 4 s, seeds
1000+i) on the **same vLLM-Omni code path**, then are scored with VBench `imaging_quality`
(MUSIQ-SPAQ, no-reference per-frame).
**β‘  `imaging_quality` (0–1, higher = better):**
| Model | `imaging_quality` | /100 | vs bf16 |
|---|---|---|---|
| bf16 (native, DiT+TE bf16) | **0.6306** | 63.06 | β€” |
| MXFP8 **DiT-only** (`-native`) | 0.6313 | 63.13 | +0.12% |
| **MXFP8 DiT+TE (this model)** | **0.6284** | 62.84 | **βˆ’0.35%** |
Quantizing the **text encoder as well** keeps no-reference image quality essentially on par with bf16
(Ξ” = βˆ’0.0022, βˆ’0.35%); the Β±0.3 scatter is diffusion sampling noise, not a quality collapse.
**β‘‘ Reference metrics vs bf16 (same prompt + same seed ⟹ frame-aligned; PSNR↑ / SSIM↑ / LPIPS↓):**
| Pair | PSNR (dB) | SSIM | LPIPS |
|---|---|---|---|
| MXFP8 DiT+TE vs bf16 | β‰ˆ 20.05ΒΉ | 0.652 | 0.254 |
ΒΉ Mean over 92 of 93 clips. One clip (*"Van Gogh's selfie"*) has a pixel-identical first frame β†’ a
per-frame PSNR of `inf` that poisons that video's dB-average; the remaining frames diverge normally,
so it is excluded from the PSNR mean (SSIM/LPIPS are unaffected). These are **not** per-pixel-identical
because a ~2–3% MXFP8 weight perturbation is amplified by the 16-step denoiser into a *different but
equally plausible* sample (same scene/soundscape, pixels slightly shifted) β€” divergence grows with
scene complexity/motion, consistent with the DiT-only ablation.
---
## Limitations
- Not faster than bf16 on Hopper; needs 2 GPUs to serve.
- Serving the quantized text encoder requires the un-upstreamed `encoder.py` patch above.
- VAEs and AdaLN kept BF16; TE vision tower kept BF16.
- Native/diffusers dual packaging means serving needs the conversion + the fixes above.
## License & attribution
Inherits the base model's license: **MiniMax-H3 Community License Agreement** (`license: other`,
see [LICENSE](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE)). This is a derivative
(quantized) checkpoint of
[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3); all model credit belongs to
MiniMax. Quantization by Intel [auto-round](https://github.com/intel/auto-round) (zero-shot RTN,
MXFP8) with [PR #2481](https://github.com/intel/auto-round/pull/2481). Please also comply with the
upstream MiniMax-H3 usage terms.