|
Download README.md from intel-ai/MiniMax-H3-MXFP8-CT-AutoRound: direct link, hf CLI and curl.
- Browser
- Download file 12 kB
-
https://huggingface.co/intel-ai/MiniMax-H3-MXFP8-CT-AutoRound/resolve/main/README.md
- Command line
-
hf download hf://intel-ai/MiniMax-H3-MXFP8-CT-AutoRound/README.md
-
curl -L -o README.md https://huggingface.co/intel-ai/MiniMax-H3-MXFP8-CT-AutoRound/resolve/main/README.md
12 kB
| pipeline_tag: text-to-video | |
| license: other | |
| license_name: minimax-h3-community-license-agreement | |
| license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE | |
| library_name: minimax-h3 | |
| base_model: MiniMaxAI/MiniMax-H3 | |
| base_model_relation: quantized | |
| tags: | |
| - text-to-video | |
| - text-to-audio-video | |
| - image-to-video | |
| - audio-video-generation | |
| - multimodal | |
| - mxfp8 | |
| - fp8 | |
| - quantized | |
| - auto-round | |
| - vllm | |
| # MiniMax-H3 β MXFP8 **DiT + Text-Encoder** | |
| An **MXFP8 (OCP microscaling FP8)** weight-quantized build of | |
| [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) β the omni-modal | |
| (video + native stereo audio) generative DiT β produced with | |
| [Intel auto-round](https://github.com/intel/auto-round). | |
| ** *both* diffusion transformers **and** the Qwen3-VL **text encoder** | |
| are quantized to MXFP8. (The companion repo `MiniMax-H3-mxfp8-native` quantizes only the two DiTs | |
| and keeps the text encoder in BF16 β see "Relationship to the other variants".) The large **AdaLN** | |
| branch of each DiT is deliberately **kept in BF16**. | |
| **TL;DR** | |
| - Each DiT **~62 GB (bf16) β ~43.65 GB (β0.71Γ, β29 %)**; AdaLN (β13 B params/DiT) stays BF16. | |
| - Text encoder (Qwen3-VL, 64 decoder layers) **~63 GB (bf16) β ~34 GB (β0.54Γ, β46 %)**: the | |
| `model.language_model.layers` linears are MXFP8 (448 Γ F8_E4M3 + 448 Γ U8 scale); the vision tower, | |
| embeddings and norms stay BF16. | |
| - Serves end-to-end on **2ΓH200 (TP2)** and produces video **with synchronized audio**. H200 runs the | |
| FP8 weights through a Marlin **dequant** kernel (correct, memory-saving, **not** faster). | |
| --- | |
| ## Showcase β official T2VA example | |
| Reproduction of the official MiniMax-H3 **Text-to-Audio-Video (T2VA)** showcase | |
| ([`assets/t2va.mp4`](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/assets/t2va.mp4)), | |
| generated on **our own vLLM-Omni stack with identical settings** (1344Γ768, 24 fps, 32 steps, | |
| 10 s, seed 0, flow-shift 12 / audio-flow-shift 3.0, task `t2va`) so the **only** variable is the | |
| quantization. Prompt = the official H3-Context-IR "starship captain" T2VA prompt. | |
| | bf16 (reference) | MXFP8 DiT+TE (this model) | | |
| |---|---| | |
| | <video controls width="100%" src="assets/t2va_official_bf16.mp4"></video> [`assets/t2va_official_bf16.mp4`](assets/t2va_official_bf16.mp4) | <video controls width="100%" src="assets/t2va_official_mxfp8_full.mp4"></video> [`assets/t2va_official_mxfp8_full.mp4`](assets/t2va_official_mxfp8_full.mp4) | | |
| Both clips: 1344Γ768, 24 fps, 243 frames (10.12 s), video **+ stereo audio**, matching the official | |
| `t2va.mp4` geometry. The MXFP8 clip is faithful to the prompt (starship bridge; silver-haired female | |
| captain close-up) β the quantized text encoder is genuinely active. | |
| Quantitative quality (dense93 VBench) is in the [Quality](#quality--vbench-dense93-h200-2026-10-14-n93) | |
| section below: `imaging_quality` **0.6284** (this model, DiT+TE) vs **0.6306** (bf16), i.e. β0.35%. | |
| --- | |
| ## What was quantized | |
| H3 has **two** DiTs sharing one architecture, plus one shared text encoder: | |
| | Component | Role | Quantized? | | |
| |---|---|---| | |
| | `transformer` (FL2VA / T2VA DiT, 33.12 B) | video+audio denoiser | β MXFP8 (from **BF16**) | | |
| | `Ref2VA` DiT (33.12 B) | reference-conditioned denoiser | β MXFP8 (from **BF16**) | | |
| | **`text_encoder`** (Qwen3-VL) | text / vision encoding | β **MXFP8** (language-model layers) | | |
| | `video_vae` / `audio_vae` | video / audio decode | β unchanged | | |
| | `adaln_proj.linear` (Γ52/DiT) | AdaLN modulation (~13 B) | β **kept BF16 on purpose** | | |
| **Per DiT β quantized Linears** (6 per block Γ 52 = **312**): | |
| `attn.{to_q,to_k,to_v,to_out.0}` + `ff.net.0.proj` (SwiGLU gate+up) + `ff.net.2` (down). | |
| Per DiT: **312 Γ F8_E4M3** weights + **312 Γ U8** E8M0 block scales + **314 Γ BF16** (AdaLN, norms, | |
| embedders, biases). AdaLN stays BF16 because vLLM-Omni precomputes and **caches** its modulation β | |
| pure inference never runs it through a matmul, so quantizing it would cost accuracy for zero benefit. | |
| **Text encoder β quantized Linears** (7 per layer Γ 64 = **448**): | |
| every `model.language_model.layers.N.{self_attn.{q,k,v,o}_proj, mlp.{gate,up,down}_proj}`. | |
| **448 Γ F8_E4M3** weights (β29.06 GiB) + **448 Γ U8** block scales (β0.91 GiB) + **610 Γ BF16** | |
| (β4.01 GiB: vision tower, embeddings, final norm, lm_head). The vision encoder is **not** quantized. | |
| **Quant format** (per component `config.json β quantization_config`): | |
| `quant_method=auto-round`, `data_type=mx_fp`, `bits=8`, `group_size=32`, `sym=True`; | |
| activations `act_data_type=mx_fp`, `act_bits=8`, `act_dynamic=True`, `act_group_size=32`; | |
| DiT `block_name_to_quantize="token_refiner.blocks,blocks"`, TE | |
| `block_name_to_quantize="model.language_model.layers"`, `packing_format="auto_round:llm_compressor"`; | |
| `autoround_version=0.17.0`. | |
| --- | |
| ## Size / compression | |
| | Component (per instance) | bf16 | MXFP8 | ratio | | |
| |---|---:|---:|---:| | |
| | DiT (`transformer`, Γ2) | ~62 GiB | **43.65 GiB** | 0.71Γ (β29 %) | | |
| | text encoder (Qwen3-VL) | ~63 GiB | **~34 GiB** | 0.54Γ (β46 %) | | |
| Per-DiT breakdown (safetensors header scan): 312 Γ F8_E4M3 = 18.66 GiB, 312 Γ U8 = 0.58 GiB, | |
| 314 Γ BF16 (AdaLN + norms + embedders) = 24.34 GiB, 12 Γ F32 = 0.06 GiB β **43.65 GiB**. | |
| Because the source is already **BF16** and AdaLN (~55 % of bytes) stays BF16, the per-DiT ratio is a | |
| modest ~0.71Γ; the text encoder, with no cached-modulation carve-out, compresses further (~0.54Γ). | |
| **On-disk total β 142 GiB** for this repo (both partitions). The two pipeline folders share **one** | |
| 34 GiB quantized text encoder via hardlinks, so the TE is stored **once** rather than duplicated | |
| --- | |
| ## Directory layout | |
| Native, serve-ready layout (what vLLM-Omni's *combined* mode loads). All entries are **real files** | |
| (materialized via hardlinks; `find -type l` is empty): | |
| ``` | |
| MiniMax-H3-mxfp8-full-native/ | |
| βββ model_index.json # pipeline manifest | |
| βββ README.md # this card | |
| βββ assets/ # showcase mp4s | |
| βββ transformer/config.json # ~1 KB shim: carries DiT quantization_config for auto-detect | |
| βββ FL2VA/ # self-contained T2VA/FL2VA pipeline | |
| β βββ transformer/ # 11 shards, MXFP8 DiT (+ .weight_scale) | |
| β βββ text_encoder/ # 7 shards, MXFP8 Qwen3-VL (+ quantization_config) | |
| β βββ video_vae/ audio_vae/ tokenizer/ processor/ model_index.json | |
| βββ Ref2VA/ # same structure; text_encoder shares inodes with FL2VA | |
| ``` | |
| Each partition's `text_encoder` is the quantized Qwen3-VL (its `config.json` carries the | |
| `quantization_config` with `data_type=mx_fp`), **not** the BF16 encoder shipped by the base model. | |
| --- | |
| --- | |
| ## Serving (vLLM-Omni, 2ΓH200 / TP2) | |
| ```bash | |
| CUDA_VISIBLE_DEVICES=2,7 \ | |
| VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \ | |
| vllm serve intel-ai/MiniMax-H3-MXFP8-CT-AutoRound --omni \ | |
| --host 0.0.0.0 --port 8092 --trust-remote-code \ | |
| --num-gpus 2 --tensor-parallel-size 2 --text-encoder-tp-size 2 \ | |
| --enforce-eager --diffusion-attention-backend CUDNN_ATTN \ | |
| --init-timeout 1800 --stage-init-timeout 1200 | |
| # no --task-type β combined (loads FL2VA + Ref2VA; required for T2VA) | |
| ``` | |
| - **TP2 is mandatory**: `create_weights` allocates the BF16 skeleton first (two DiTs β 176 GB), so a | |
| single 141 GB H200 OOMs; `--enable-cpu-offload` is incompatible (Marlin repack is CUDA-only). | |
| ### Required vLLM-Omni patch (text encoder) | |
| Serving the **quantized text encoder** needs a small, un-upstreamed patch to | |
| `vllm_omni/diffusion/models/minimax_h3/encoder.py` (4 fixes): | |
| 1. **Component-level quant routing** β read the serialized TE `config.json β quantization_config` | |
| (`data_type=mx_fp`) and build a text-encoder quant config scoped to | |
| `text_encoder.text_model.layers` (the pipeline otherwise passes the DiT config, so the TE loads | |
| unquantized). | |
| 2. **RowParallel scale-aware loader** β shard the grouped `weight_scale` (in/32) on the same dim as | |
| the weight, so scales stay aligned under TP. | |
| 3. **Suffix-agnostic fused mapping** β fuse `q/k/v β qkv_proj` and `gate/up β gate_up_proj` for both | |
| `.weight` **and** `.weight_scale`. | |
| 4. **Post-load repack** β run `process_weights_after_loading` on the TE's quantized Linears after | |
| device placement (the pipeline skips the generic `post_load_weights`, and MarlinMxfp8 repack needs | |
| CUDA). | |
| Without the patch: ~700 `weight_scale has no target parameter` warnings, the TE falls back to an | |
| unquantized/zeroed path, and the model ignores the prompt. | |
| *(The two DiTs additionally need the serving fixes: a top-level `transformer/config.json` | |
| quant anchor, grouped-qkv / fc1 scale reordering in `load_weights`, and TP2 without cpu-offload.)* | |
| --- | |
| ## Hardware note (important) | |
| H200 (Hopper) has **no native MXFP8 tensor core**; vLLM-Omni runs these weights through a Marlin | |
| weight-only **dequant** GEMM (documented as `MXFP8 W8A16 GEMM via Marlin`): it dequantizes the MXFP8 | |
| weight (E4M3 + E8M0 block scale) inside the GEMM and multiplies against **unquantized bf16 | |
| activations**. It is functionally correct and memory-saving, but **not faster** than bf16 on Hopper. | |
| The checkpoints' `act_data_type=mx_fp / act_bits=8` is quantization-recipe metadata that is **not | |
| exercised on H200** (activations stay bf16). True W8A8 (FP8 activations on a native microscaling | |
| tensor core) requires Blackwell/B300. Dequant does not change stored values, so accuracy matches a | |
| native kernel. | |
| --- | |
| ## Quality β VBench (dense93, H200, 2026-10-14, n=93) | |
| Both models generate the **same 93 dense-prompt videos** (512Γ288, 16 steps, 24 fps, 4 s, seeds | |
| 1000+i) on the **same vLLM-Omni code path**, then are scored with VBench `imaging_quality` | |
| (MUSIQ-SPAQ, no-reference per-frame). | |
| **β `imaging_quality` (0β1, higher = better):** | |
| | Model | `imaging_quality` | /100 | vs bf16 | | |
| |---|---|---|---| | |
| | bf16 (native, DiT+TE bf16) | **0.6306** | 63.06 | β | | |
| | MXFP8 **DiT-only** (`-native`) | 0.6313 | 63.13 | +0.12% | | |
| | **MXFP8 DiT+TE (this model)** | **0.6284** | 62.84 | **β0.35%** | | |
| Quantizing the **text encoder as well** keeps no-reference image quality essentially on par with bf16 | |
| (Ξ = β0.0022, β0.35%); the Β±0.3 scatter is diffusion sampling noise, not a quality collapse. | |
| **β‘ Reference metrics vs bf16 (same prompt + same seed βΉ frame-aligned; PSNRβ / SSIMβ / LPIPSβ):** | |
| | Pair | PSNR (dB) | SSIM | LPIPS | | |
| |---|---|---|---| | |
| | MXFP8 DiT+TE vs bf16 | β 20.05ΒΉ | 0.652 | 0.254 | | |
| ΒΉ Mean over 92 of 93 clips. One clip (*"Van Gogh's selfie"*) has a pixel-identical first frame β a | |
| per-frame PSNR of `inf` that poisons that video's dB-average; the remaining frames diverge normally, | |
| so it is excluded from the PSNR mean (SSIM/LPIPS are unaffected). These are **not** per-pixel-identical | |
| because a ~2β3% MXFP8 weight perturbation is amplified by the 16-step denoiser into a *different but | |
| equally plausible* sample (same scene/soundscape, pixels slightly shifted) β divergence grows with | |
| scene complexity/motion, consistent with the DiT-only ablation. | |
| --- | |
| ## Limitations | |
| - Not faster than bf16 on Hopper; needs 2 GPUs to serve. | |
| - Serving the quantized text encoder requires the un-upstreamed `encoder.py` patch above. | |
| - VAEs and AdaLN kept BF16; TE vision tower kept BF16. | |
| - Native/diffusers dual packaging means serving needs the conversion + the fixes above. | |
| ## License & attribution | |
| Inherits the base model's license: **MiniMax-H3 Community License Agreement** (`license: other`, | |
| see [LICENSE](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE)). This is a derivative | |
| (quantized) checkpoint of | |
| [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3); all model credit belongs to | |
| MiniMax. Quantization by Intel [auto-round](https://github.com/intel/auto-round) (zero-shot RTN, | |
| MXFP8) with [PR #2481](https://github.com/intel/auto-round/pull/2481). Please also comply with the | |
| upstream MiniMax-H3 usage terms. | |