---
pipeline_tag: text-to-video
license: other
license_name: minimax-h3-community-license-agreement
license_link: https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE
library_name: minimax-h3
base_model: MiniMaxAI/MiniMax-H3
base_model_relation: quantized
tags:
  - text-to-video
  - text-to-audio-video
  - image-to-video
  - audio-video-generation
  - multimodal
  - mxfp8
  - fp8
  - quantized
  - auto-round
  - vllm
---

# MiniMax-H3 — MXFP8 (weight-quantized, vLLM-Omni native layout)

An **MXFP8 (OCP microscaling FP8)** weight-quantized build of
[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3) — the omni-modal
(video + native stereo audio) generative DiT — produced with
[Intel auto-round](https://github.com/intel/auto-round) (zero-shot RTN) and converted to the
**native fused layout** that vLLM-Omni serves. Both diffusion transformers are quantized; the
large **AdaLN** branch is deliberately **kept in BF16**.

**TL;DR**
- Each DiT **~62 GB (bf16) → ~43.65 GB (≈0.71×, −29 %)**; AdaLN (≈13 B params/DiT) stays BF16 because inference caches it and it never hits a matmul.
- Quality is effectively unchanged: VBench `imaging_quality` **0.6306 (bf16) → 0.6313 (mxfp8)**, Δ **+0.12 %** over 93 prompts.
- Serves end-to-end on **2×H200 (TP2)** and produces video **with synchronized audio**. H200 runs the FP8 weights through a Marlin **dequant** kernel (correct, memory-saving, **not** faster).

> ⚠️ Research / evaluation artifact. H3 has native/diffusers dual packaging; this repo is the
> **native fused** checkpoint (serve-ready). A diffusers-layout quant output also exists at
> `MiniMax-H3-mxfp8/` (not directly servable — see below).

---

## Showcase — official T2VA example, bf16 vs MXFP8

Reproduction of the official MiniMax-H3 **Text-to-Audio-Video (T2VA)** showcase
([`assets/t2va.mp4`](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/assets/t2va.mp4)),
generated on **our own vLLM-Omni stack with identical settings** (1344×768, 24 fps, 32 steps,
10 s, seed 0, flow-shift 12 / audio-flow-shift 3.0, task `t2va`) so the **only** variable is the
quantization. Prompt = the official H3-Context-IR "starship captain" T2VA prompt.

| bf16 (reference) | MXFP8 (this model) |
|---|---|
| <video controls width="100%" src="assets/t2va_official_bf16.mp4"></video> [`assets/t2va_official_bf16.mp4`](assets/t2va_official_bf16.mp4) | <video controls width="100%" src="assets/t2va_official_mxfp8.mp4"></video> [`assets/t2va_official_mxfp8.mp4`](assets/t2va_official_mxfp8.mp4) |

Both clips: 1344×768, 24 fps, 243 frames (10.12 s), video **+ stereo audio**, matching the
official `t2va.mp4` geometry (1344×768 / 24 fps / 10.12 s).

| Metric (this single example) | bf16 | MXFP8 |
|---|---|---|
| VBench `imaging_quality` | 0.6671 | 0.6614 |
| PSNR (mxfp8 vs bf16) | — | 14.69 dB |
| SSIM (mxfp8 vs bf16) | — | 0.5589 |
| LPIPS (mxfp8 vs bf16) | — | 0.3804 |

**How to read the PSNR/SSIM/LPIPS.** They are *moderate*, not near-identical, because the two
videos are **not** pixel-for-pixel copies. A ~2–3 % MXFP8 weight perturbation, compounded over the
denoising steps, nudges the sampler onto a **different but equally valid trajectory** — same scene,
motion and soundscape, slightly different pixels. The matching VBench `imaging_quality` (and the
93-prompt result below) confirm there is **no systematic quality loss**; the reference-metric
distance is sampling chaos, not degradation. (The distance is a little larger than Wan2.2's here
because this clip is longer/higher-fps, giving more room for trajectory divergence.)

---

## What was quantized

H3 has **two** DiTs sharing one architecture:

| Component | Role | Quantized? |
|---|---|---|
| `transformer` | FL2VA / T2VA DiT (33.12 B) | ✅ MXFP8 (from **BF16**) |
| `transformer_ref` (diffusers layout) / `Ref2VA` | Ref2VA DiT (33.12 B) | ✅ MXFP8 (from **BF16**) |
| `text_encoder` (Qwen3-VL) | text/vision encoding | ❌ kept bf16 |
| `vae` / `audio_vae` | video / audio decode | ❌ unchanged |
| `adaln_proj.linear` (×52/DiT) | AdaLN modulation (~13 B) | ❌ **kept BF16 on purpose** |

**Quantized Linears** — 6 per block × 52 = **312 per DiT**:
`attn.{to_q,to_k,to_v,to_out.0}` + `ff.net.0.proj` (SwiGLU gate+up) + `ff.net.2` (down).
Per DiT: **312 × F8_E4M3** weights + **312 × U8** E8M0 block scales + **314 × BF16** (AdaLN,
norms, embedders, biases).

**Why AdaLN stays BF16.** The official card notes ~13 B params live in AdaLN, and because AdaLN
modulation can be **precomputed and cached**, pure inference never loads them into a matmul
(vLLM-Omni ships `adaln_cache.py` and caches by default). Quantizing them would cost accuracy for
zero runtime benefit, so they are pinned to BF16 via
`extra_config: {".*adaln_proj\\.linear.*": {bits: 16, data_type: float, act_bits: 16}}`.

**Quant format** (per `transformer/config.json`):
`quant_method=auto-round`, `data_type=mx_fp`, `bits=8`, `group_size=32`, `sym=True`;
activations `act_data_type=mx_fp`, `act_bits=8`, `act_dynamic=True`, `act_group_size=32`;
`block_name_to_quantize="token_refiner.blocks,blocks"`; `autoround_version=0.17.0`.

### Reproduce the quantization

Unlike Wan2.2, H3 is a **Modular Diffusers** pipeline, so the one-line CLI does **not** work — you
must build the pipeline programmatically and hand it to auto-round. Outline
(`scripts/quant_h3_mxfp8_v2.py`):

```python
pipe = MiniMaxH3ModularPipeline.from_pretrained(LOCAL)            # offline
for name in ("transformer", "transformer_ref"):                   # both DiTs
    t = MiniMaxH3Transformer3DModel.from_pretrained(f"{LOCAL}/{name}", torch_dtype=torch.bfloat16)
    pipe.update_components(**{name: t})
ar = AutoRound(pipe, scheme="MXFP8", iters=0, ignore_layers="adaln_proj.linear")
ar.quantize_and_save("MiniMax-H3-mxfp8", format="auto_round")     # RTN, both DiTs
```

- `iters=0` ⇒ zero-shot **RTN**; `ignore_layers="adaln_proj.linear"` pins all 52 AdaLN layers to BF16.
- Needs the dual-transformer zero-shot fix ([auto-round PR #2481](https://github.com/intel/auto-round/pull/2481)); `transformer_ref` is then auto-detected and quantized too.
- The diffusers-layout output (`MiniMax-H3-mxfp8/`) is converted to this **native fused layout** with `scripts/convert_h3_mxfp8_to_native.py` (head-grouped qkv fusion + fc1 gate/up + synchronized U8 scale reordering). See "Serving" for the three fixes that make it serve correctly.

---

## Size / compression (safetensors header scan, per DiT)

| dtype | #tensors | Size | Meaning |
|---|---:|---:|---|
| F8_E4M3 | 312 | 18.66 GiB | quantized weights (attn + ff) |
| U8 (E8M0) | 312 | 0.58 GiB | per-32 block scales |
| BF16 | 314 | 24.34 GiB | AdaLN (52) + norms + embedders + biases |
| F32 | 12 | 0.06 GiB | proj_in / audio_proj_in / time embedder |
| **Total** | 950 | **43.65 GiB** | vs ~62 GiB bf16 ⇒ **0.71× (−29 %)** |

Because the source is already **BF16** (not FP32 like Wan2.2) and AdaLN (~55 % of bytes) stays
BF16, the per-DiT ratio is a modest ~0.71× rather than ~4×.

---

## Directory layout — how this differs from the base `MiniMax-H3`

This repo is a **serve-ready, self-contained** slice of the base model: **233 GB vs ~465 GB**. The
reduction is **not** mostly from quantization — it is from dropping redundant copies that the base
ships for the modular/diffusers distribution but vLLM-Omni never loads.

The base `MiniMax-H3` is **double-packaged**: it stores both (a) a top-level *modular* component
library (`transformer/`, `transformer_ref/`, `text_encoder/`, `vae/`, `audio_vae/`, `tokenizer/`,
`processor/`, `scheduler/`, …) **and** (b) two already-self-contained pipelines (`FL2VA/`, `Ref2VA/`),
each bundling its *own* transformer + text_encoder + VAEs + tokenizer. The two sets are the **same
weights duplicated** — e.g. the 63 GiB text encoder exists **3×** and the DiT ~4×.

vLLM-Omni serves only from the two pipeline folders (combined mode loads `FL2VA` + `Ref2VA`), so
this repo keeps just that one copy and removes the rest.

| Top-level entry | base `MiniMax-H3` (~465 G) | this repo (233 G) | why |
|---|---|---|---|
| `FL2VA/`, `Ref2VA/` | 135 G each | **117 G each** | kept; transformer swapped to MXFP8 (62→44 G) |
| `transformer/` | 62 G (bf16, 14 shards) | **config.json only (~1 KB)** | shim: carries `quantization_config` so the quant is auto-detected; weights live under the pipelines |
| `transformer_ref/`, `text_encoder/`, `vae/`, `audio_vae/`, `tokenizer/`, `processor/`, `scheduler/`, `audio_scheduler/` | present | **removed** | redundant duplicates of what's already inside `FL2VA/` + `Ref2VA/` |
| `docs/`, `scripts/`, `modular_model_index.json` | present | **removed** | not used for inference |
| `assets/`, `README.md`, `model_index.json` | official | **our showcase + this card** | model card + bf16-vs-MXFP8 samples |

Inside each pipeline, only `transformer/` changes vs the base: **13 bf16 shards → 11 MXFP8 shards**,
tensors are **fused into the vLLM-Omni native naming** (`blocks.N.attn.qkv_proj.weight`, `fc1`
gate/up) each with a paired `.weight_scale`, plus a `quantization_config.json`. The non-quantized
components (`text_encoder/`, `video_vae/`, `audio_vae/`, `tokenizer/`, `processor/`) are **byte-identical
copies** of the base — materialized as real files (no symlinks) so the directory is upload-ready.

> One intrinsic redundancy remains: `FL2VA/` and `Ref2VA/` each carry their own 63 GiB text encoder
> (identical content). De-duplicating would need a symlink/shared path, which conflicts with being a
> self-contained, uploadable checkpoint, so both copies are kept on purpose.

---

## Quality — VBench `imaging_quality` (dense93, n = 93)

Both models generated the same 93 prompts on the **same code path** (the non-modular *combined*
runner, differing only in transformer weights) with the **same recipe** (512×288, 16:9, 24 fps,
16 steps, flow-shift 12, duration 4.0 s, audio-flow-shift 3.0, task `t2va`, `seed=1000+row`), then
scored with the same VBench `imaging_quality`.

| Model | `imaging_quality` | /100 | vs bf16 |
|---|---|---|---|
| bf16 (native) | **0.6306** | 63.06 | — |
| **MXFP8 (this model)** | **0.6313** | 63.13 | **+0.12 %** |

Mean difference is **+0.0007** (mxfp8 marginally higher). Per-video deltas are near-symmetric
(mxfp8 higher on 49/93, lower on 44/93; mean |Δ| ≈ 3.41 on the 0–100 scale) while per-video scores
span 26.8–80.0 — the jitter is diffusion sampling non-determinism (amplified by the low-res,
short, few-step recipe), not a quantization floor.

---

## Serving (vLLM-Omni, 2×H200 / TP2)

```bash
CUDA_VISIBLE_DEVICES=6,7 \
VLLM_WORKER_MULTIPROC_METHOD=spawn VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve /path/to/MiniMax-H3-mxfp8-native --omni \
  --host 0.0.0.0 --port 8092 --trust-remote-code \
  --num-gpus 2 --tensor-parallel-size 2 --text-encoder-tp-size 2 \
  --enforce-eager --diffusion-attention-backend CUDNN_ATTN
# no --task-type ⇒ combined (loads FL2VA + Ref2VA; required for T2VA)
```

- **TP2 is mandatory**: `create_weights` allocates the BF16 skeleton first (two DiTs ≈ 176 GB), so a single 141 GB H200 OOMs; `--enable-cpu-offload` is incompatible (Marlin repack is CUDA-only).
- Ready in ~5–6 min; `GET /health` → 200; ~81 GB/GPU; **0 weight_scale skips** (quantization active).
- `duration` must be in **[4, 15] s**. Submit to `/v1/videos/sync`.

**Three fixes required for correct MXFP8 serving** (carried as local, un-upstreamed patches):
1. a top-level `transformer/config.json` carrying `quantization_config` so vLLM-Omni auto-detects the quant (native weights live under `FL2VA/Ref2VA`);
2. `load_weights` must apply the **same** grouped-qkv / fc1 gate-up reordering to `.weight_scale` as to `.weight` (otherwise scales misalign and output is noise);
3. TP2 + no cpu-offload (above).

---

## Hardware note (important)

H200 (Hopper) has **no native MXFP8 tensor core**; vLLM-Omni runs these weights through a Marlin
weight-only **dequant** GEMM — functionally correct and memory-saving, but ~2.5× slower than bf16
(~13 s vs ~5 s per 4 s/512×288 clip). Use on H200 to **validate accuracy**; native MXFP8 speedups
require Blackwell/B300. Dequant does not change stored values, so accuracy matches a native kernel.

**Runtime is W8A16, not W8A8.** The Marlin kernel is documented as `MXFP8 W8A16 GEMM via Marlin`:
it dequantizes the MXFP8 *weight* (E4M3 + E8M0 block scale) inside the GEMM and multiplies it
against **unquantized bf16 activations** (`a_scales=None`; a 1-byte/fp8 activation path explicitly
raises `"Marlin W8A8 is not supported."`). The checkpoint's `act_data_type=mx_fp / act_bits=8 /
act_dynamic=True` is **quantization-recipe metadata that is not exercised on H200** — activations
stay bf16. True W8A8 (FP8 activations on a native microscaling tensor core) requires Blackwell/B300.

---

## Limitations

- Not faster than bf16 on Hopper; needs 2 GPUs to serve.
- Only the two DiTs are quantized (text encoder, VAEs, AdaLN kept BF16).
- Evaluated with no-reference `imaging_quality` on short 512×288 clips; other dimensions not swept.
- Native/diffusers dual packaging means serving needs the conversion + three fixes above.

## License & attribution

Inherits the base model's license: **MiniMax-H3 Community License Agreement** (`license: other`,
see [LICENSE](https://huggingface.co/MiniMaxAI/MiniMax-H3/blob/main/LICENSE)). This is a derivative
(quantized) checkpoint of
[`MiniMaxAI/MiniMax-H3`](https://huggingface.co/MiniMaxAI/MiniMax-H3); all model credit belongs to
MiniMax. Quantization by Intel [auto-round](https://github.com/intel/auto-round) (zero-shot RTN,
MXFP8, AdaLN kept BF16) with [PR #2481](https://github.com/intel/auto-round/pull/2481). Please also
comply with the upstream MiniMax-H3 usage terms.