--- pipeline_tag: text-to-video license: other license_name: minimax-h3-community-license-agreement license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE library_name: minimax-h3 base_model: MiniMaxAI/MiniMax-H3 base_model_relation: quantized tags: - text-to-video - text-to-audio-video - image-to-video - audio-video-generation - multimodal - mxfp8 - fp8 - quantized - auto-round - vllm --- # MiniMax-H3 — MXFP8 **DiT + Text-Encoder** An **MXFP8 (OCP microscaling FP8)** weight-quantized build of [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) — the omni-modal (video + native stereo audio) generative DiT — produced with [Intel auto-round](https://github.com/intel/auto-round). ** *both* diffusion transformers **and** the Qwen3-VL **text encoder** are quantized to MXFP8. (The companion repo `MiniMax-H3-mxfp8-native` quantizes only the two DiTs and keeps the text encoder in BF16 — see "Relationship to the other variants".) The large **AdaLN** branch of each DiT is deliberately **kept in BF16**. **TL;DR** - Each DiT **~62 GB (bf16) → ~43.65 GB (≈0.71×, −29 %)**; AdaLN (≈13 B params/DiT) stays BF16. - Text encoder (Qwen3-VL, 64 decoder layers) **~63 GB (bf16) → ~34 GB (≈0.54×, −46 %)**: the `model.language_model.layers` linears are MXFP8 (448 × F8_E4M3 + 448 × U8 scale); the vision tower, embeddings and norms stay BF16. - Serves end-to-end on **2×H200 (TP2)** and produces video **with synchronized audio**. H200 runs the FP8 weights through a Marlin **dequant** kernel (correct, memory-saving, **not** faster). --- ## Showcase — official T2VA example Reproduction of the official MiniMax-H3 **Text-to-Audio-Video (T2VA)** showcase ([`assets/t2va.mp4`](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/assets/t2va.mp4)), generated on **our own vLLM-Omni stack with identical settings** (1344×768, 24 fps, 32 steps, 10 s, seed 0, flow-shift 12 / audio-flow-shift 3.0, task `t2va`) so the **only** variable is the quantization. Prompt = the official H3-Context-IR "starship captain" T2VA prompt. | bf16 (reference) | MXFP8 DiT+TE (this model) | |---|---| | [`assets/t2va_official_bf16.mp4`](assets/t2va_official_bf16.mp4) | [`assets/t2va_official_mxfp8_full.mp4`](assets/t2va_official_mxfp8_full.mp4) | Both clips: 1344×768, 24 fps, 243 frames (10.12 s), video **+ stereo audio**, matching the official `t2va.mp4` geometry. The MXFP8 clip is faithful to the prompt (starship bridge; silver-haired female captain close-up) — the quantized text encoder is genuinely active. Quantitative quality (dense93 VBench) is in the [Quality](#quality--vbench-dense93-h200-2026-10-14-n93) section below: `imaging_quality` **0.6284** (this model, DiT+TE) vs **0.6306** (bf16), i.e. −0.35%. --- ## What was quantized H3 has **two** DiTs sharing one architecture, plus one shared text encoder: | Component | Role | Quantized? | |---|---|---| | `transformer` (FL2VA / T2VA DiT, 33.12 B) | video+audio denoiser | ✅ MXFP8 (from **BF16**) | | `Ref2VA` DiT (33.12 B) | reference-conditioned denoiser | ✅ MXFP8 (from **BF16**) | | **`text_encoder`** (Qwen3-VL) | text / vision encoding | ✅ **MXFP8** (language-model layers) | | `video_vae` / `audio_vae` | video / audio decode | ❌ unchanged | | `adaln_proj.linear` (×52/DiT) | AdaLN modulation (~13 B) | ❌ **kept BF16 on purpose** | **Per DiT — quantized Linears** (6 per block × 52 = **312**): `attn.{to_q,to_k,to_v,to_out.0}` + `ff.net.0.proj` (SwiGLU gate+up) + `ff.net.2` (down). Per DiT: **312 × F8_E4M3** weights + **312 × U8** E8M0 block scales + **314 × BF16** (AdaLN, norms, embedders, biases). AdaLN stays BF16 because vLLM-Omni precomputes and **caches** its modulation — pure inference never runs it through a matmul, so quantizing it would cost accuracy for zero benefit. **Text encoder — quantized Linears** (7 per layer × 64 = **448**): every `model.language_model.layers.N.{self_attn.{q,k,v,o}_proj, mlp.{gate,up,down}_proj}`. **448 × F8_E4M3** weights (≈29.06 GiB) + **448 × U8** block scales (≈0.91 GiB) + **610 × BF16** (≈4.01 GiB: vision tower, embeddings, final norm, lm_head). The vision encoder is **not** quantized. **Quant format** (per component `config.json → quantization_config`): `quant_method=auto-round`, `data_type=mx_fp`, `bits=8`, `group_size=32`, `sym=True`; activations `act_data_type=mx_fp`, `act_bits=8`, `act_dynamic=True`, `act_group_size=32`; DiT `block_name_to_quantize="token_refiner.blocks,blocks"`, TE `block_name_to_quantize="model.language_model.layers"`, `packing_format="auto_round:llm_compressor"`; `autoround_version=0.17.0`. --- ## Size / compression | Component (per instance) | bf16 | MXFP8 | ratio | |---|---:|---:|---:| | DiT (`transformer`, ×2) | ~62 GiB | **43.65 GiB** | 0.71× (−29 %) | | text encoder (Qwen3-VL) | ~63 GiB | **~34 GiB** | 0.54× (−46 %) | Per-DiT breakdown (safetensors header scan): 312 × F8_E4M3 = 18.66 GiB, 312 × U8 = 0.58 GiB, 314 × BF16 (AdaLN + norms + embedders) = 24.34 GiB, 12 × F32 = 0.06 GiB → **43.65 GiB**. Because the source is already **BF16** and AdaLN (~55 % of bytes) stays BF16, the per-DiT ratio is a modest ~0.71×; the text encoder, with no cached-modulation carve-out, compresses further (~0.54×). **On-disk total ≈ 142 GiB** for this repo (both partitions). The two pipeline folders share **one** 34 GiB quantized text encoder via hardlinks, so the TE is stored **once** rather than duplicated --- ## Directory layout Native, serve-ready layout (what vLLM-Omni's *combined* mode loads). All entries are **real files** (materialized via hardlinks; `find -type l` is empty): ``` MiniMax-H3-mxfp8-full-native/ ├── model_index.json # pipeline manifest ├── README.md # this card ├── assets/ # showcase mp4s ├── transformer/config.json # ~1 KB shim: carries DiT quantization_config for auto-detect ├── FL2VA/ # self-contained T2VA/FL2VA pipeline │ ├── transformer/ # 11 shards, MXFP8 DiT (+ .weight_scale) │ ├── text_encoder/ # 7 shards, MXFP8 Qwen3-VL (+ quantization_config) │ ├── video_vae/ audio_vae/ tokenizer/ processor/ model_index.json └── Ref2VA/ # same structure; text_encoder shares inodes with FL2VA ``` Each partition's `text_encoder` is the quantized Qwen3-VL (its `config.json` carries the `quantization_config` with `data_type=mx_fp`), **not** the BF16 encoder shipped by the base model. --- --- ## Serving (vLLM-Omni, 2×H200 / TP2) ```bash CUDA_VISIBLE_DEVICES=2,7 \ VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \ vllm serve intel-ai/MiniMax-H3-MXFP8-CT-AutoRound --omni \ --host 0.0.0.0 --port 8092 --trust-remote-code \ --num-gpus 2 --tensor-parallel-size 2 --text-encoder-tp-size 2 \ --enforce-eager --diffusion-attention-backend CUDNN_ATTN \ --init-timeout 1800 --stage-init-timeout 1200 # no --task-type ⇒ combined (loads FL2VA + Ref2VA; required for T2VA) ``` - **TP2 is mandatory**: `create_weights` allocates the BF16 skeleton first (two DiTs ≈ 176 GB), so a single 141 GB H200 OOMs; `--enable-cpu-offload` is incompatible (Marlin repack is CUDA-only). ### Required vLLM-Omni patch (text encoder) Serving the **quantized text encoder** needs a small, un-upstreamed patch to `vllm_omni/diffusion/models/minimax_h3/encoder.py` (4 fixes): 1. **Component-level quant routing** — read the serialized TE `config.json → quantization_config` (`data_type=mx_fp`) and build a text-encoder quant config scoped to `text_encoder.text_model.layers` (the pipeline otherwise passes the DiT config, so the TE loads unquantized). 2. **RowParallel scale-aware loader** — shard the grouped `weight_scale` (in/32) on the same dim as the weight, so scales stay aligned under TP. 3. **Suffix-agnostic fused mapping** — fuse `q/k/v → qkv_proj` and `gate/up → gate_up_proj` for both `.weight` **and** `.weight_scale`. 4. **Post-load repack** — run `process_weights_after_loading` on the TE's quantized Linears after device placement (the pipeline skips the generic `post_load_weights`, and MarlinMxfp8 repack needs CUDA). Without the patch: ~700 `weight_scale has no target parameter` warnings, the TE falls back to an unquantized/zeroed path, and the model ignores the prompt. *(The two DiTs additionally need the serving fixes: a top-level `transformer/config.json` quant anchor, grouped-qkv / fc1 scale reordering in `load_weights`, and TP2 without cpu-offload.)* --- ## Hardware note (important) H200 (Hopper) has **no native MXFP8 tensor core**; vLLM-Omni runs these weights through a Marlin weight-only **dequant** GEMM (documented as `MXFP8 W8A16 GEMM via Marlin`): it dequantizes the MXFP8 weight (E4M3 + E8M0 block scale) inside the GEMM and multiplies against **unquantized bf16 activations**. It is functionally correct and memory-saving, but **not faster** than bf16 on Hopper. The checkpoints' `act_data_type=mx_fp / act_bits=8` is quantization-recipe metadata that is **not exercised on H200** (activations stay bf16). True W8A8 (FP8 activations on a native microscaling tensor core) requires Blackwell/B300. Dequant does not change stored values, so accuracy matches a native kernel. --- ## Quality — VBench (dense93, H200, 2026-10-14, n=93) Both models generate the **same 93 dense-prompt videos** (512×288, 16 steps, 24 fps, 4 s, seeds 1000+i) on the **same vLLM-Omni code path**, then are scored with VBench `imaging_quality` (MUSIQ-SPAQ, no-reference per-frame). **① `imaging_quality` (0–1, higher = better):** | Model | `imaging_quality` | /100 | vs bf16 | |---|---|---|---| | bf16 (native, DiT+TE bf16) | **0.6306** | 63.06 | — | | MXFP8 **DiT-only** (`-native`) | 0.6313 | 63.13 | +0.12% | | **MXFP8 DiT+TE (this model)** | **0.6284** | 62.84 | **−0.35%** | Quantizing the **text encoder as well** keeps no-reference image quality essentially on par with bf16 (Δ = −0.0022, −0.35%); the ±0.3 scatter is diffusion sampling noise, not a quality collapse. **② Reference metrics vs bf16 (same prompt + same seed ⟹ frame-aligned; PSNR↑ / SSIM↑ / LPIPS↓):** | Pair | PSNR (dB) | SSIM | LPIPS | |---|---|---|---| | MXFP8 DiT+TE vs bf16 | ≈ 20.05¹ | 0.652 | 0.254 | ¹ Mean over 92 of 93 clips. One clip (*"Van Gogh's selfie"*) has a pixel-identical first frame → a per-frame PSNR of `inf` that poisons that video's dB-average; the remaining frames diverge normally, so it is excluded from the PSNR mean (SSIM/LPIPS are unaffected). These are **not** per-pixel-identical because a ~2–3% MXFP8 weight perturbation is amplified by the 16-step denoiser into a *different but equally plausible* sample (same scene/soundscape, pixels slightly shifted) — divergence grows with scene complexity/motion, consistent with the DiT-only ablation. --- ## Limitations - Not faster than bf16 on Hopper; needs 2 GPUs to serve. - Serving the quantized text encoder requires the un-upstreamed `encoder.py` patch above. - VAEs and AdaLN kept BF16; TE vision tower kept BF16. - Native/diffusers dual packaging means serving needs the conversion + the fixes above. ## License & attribution Inherits the base model's license: **MiniMax-H3 Community License Agreement** (`license: other`, see [LICENSE](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE)). This is a derivative (quantized) checkpoint of [`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3); all model credit belongs to MiniMax. Quantization by Intel [auto-round](https://github.com/intel/auto-round) (zero-shot RTN, MXFP8) with [PR #2481](https://github.com/intel/auto-round/pull/2481). Please also comply with the upstream MiniMax-H3 usage terms.