| --- |
| license: other |
| license_name: minimax-h3-community-license-agreement |
| license_link: https://huggingface.co/nynxz/MiniMax-H3-PTD-LoRAs/blob/main/LICENSE-MiniMax-H3.txt |
| base_model: |
| - epfl-vita/PTD-weights |
| - taobao-content-ai/MiniMax-H3-PTD-LoRAs |
| - taobao-content-ai/MiniMax-H3-PTD-RefAsymOPD-LoRAs |
| - MiniMaxAI/MiniMax-H3 |
| base_model_relation: quantized |
| pipeline_tag: text-to-video |
| library_name: comfyui |
| tags: |
| - comfyui |
| - video-generation |
| - audio-generation |
| - distillation |
| - few-step |
| - lora |
| --- |
| |
| # MiniMax-H3 PTD adapters for ComfyUI (shrunk) |
|
|
| EPFL VITA Lab's **PTD** few-step adapters for MiniMax-H3 |
| ([epfl-vita/PTD-weights](https://huggingface.co/epfl-vita/PTD-weights)), converted from the released |
| DCP training checkpoints to ComfyUI safetensors and made 2–4.5× smaller. The per-hop adapters are the |
| only thing PTD adds to the base model, so these files are all you need besides the |
| original MiniMax-H3 transformer, text encoder and VAEs. |
|
|
| It also has int8 versions of taobao-content-ai's **Ref2VA** PTD adapters |
| ([taobao-content-ai/MiniMax-H3-PTD-LoRAs](https://huggingface.co/taobao-content-ai/MiniMax-H3-PTD-LoRAs), |
| 4 and 8 steps, reference images → video + audio), and of their 8- and 6-step **RefAsymOPD** sibling |
| ([taobao-content-ai/MiniMax-H3-PTD-RefAsymOPD-LoRAs](https://huggingface.co/taobao-content-ai/MiniMax-H3-PTD-RefAsymOPD-LoRAs), |
| same format, trained for better backgrounds and sharpness), at half the size of their bf16 releases. |
|
|
| Put the files in `ComfyUI/models/ptd/`. |
|
|
| | File | Size | vs. original | Adapter error (max / mean) | Loader | |
| |---|---:|---:|---|---| |
| | `ptd/ptd_h3_4step_bf16.safetensors` | 5.55 GB | 48% | 0.23% / 0.19%, identical at runtime¹ | official **or** compact | |
| | `ptd/ptd_h3_4step_int8.safetensors` | 2.83 GB | 25% | 1.13% / 0.85% | compact | |
| | `ptd/ptd_h3_4step_int8_r5.safetensors` | 2.52 GB | 22% | 5.17% / 4.88% | compact | |
| | `ptd/ptd_h3_6step_bf16.safetensors` | 8.32 GB | 48% | 0.23% / 0.19%, identical at runtime¹ | official **or** compact | |
| | `ptd/ptd_h3_6step_int8.safetensors` | 4.25 GB | 25% | 1.10% / 0.87% | compact | |
| | `ptd/ptd_h3_6step_int8_r5.safetensors` | 3.89 GB | 23% | 5.17% / 4.83% | compact | |
| | `ptd/ptd_h3_ref2va_4step_int8.safetensors` | 2.86 GB | 51%² | 0.93% / 0.44% | compact | |
| | `ptd/ptd_h3_ref2va_8step_int8.safetensors` | 5.68 GB | 51%² | 1.18% / 0.47% | compact | |
| | `ptd/ptd_h3_ref2va_refasymopd_8step_int8.safetensors` | 5.68 GB | 51%² | 0.90% / 0.52% | compact | |
| | `ptd/ptd_h3_ref2va_refasymopd_6step_int8.safetensors` | 4.24 GB | 51%² | 0.86% / 0.47% | compact³ | |
|
|
| Originals: 11.5 GB (4-step), 17.2 GB (6-step), float32. Adapter error is the relative Frobenius error |
| `‖ΔW' − ΔW‖ / ‖ΔW‖` of every per-hop delta `ΔW = B·A` against the original float32 checkpoint, |
| measured on all 1248 (4-step) / 1872 (6-step) adapters after loading each file the way ComfyUI does. |
| The dense output-head adapters and the reference head buffers are kept float32 and exact in every file. |
|
|
| ¹ The official ComfyUI hook casts each block's factors to the block's compute dtype (bf16) before use, |
| so bf16 storage rounds them earlier but gives the same numbers. |
|
|
| ² Of taobao-content-ai's bf16 files (5.58 GB / 11.11 GB); error measured the same way against them, over |
| 1248 / 2496 / 1872 adapters (4 / 8 / 6 steps; the RefAsymOPD 8-step file is 11.11 GB, the 6-step 8.31 GB). |
| Their PDD output-head banks are kept unchanged. |
|
|
| ³ The RefAsymOPD 6-step file was first released with `pdd_num_steps: 32, pdd_block_size: 4`, the 8-step |
| file's values, though it holds a 6-row head bank (one head per hop), so no loader accepted it. We reported it |
| ([discussion #3](https://huggingface.co/taobao-content-ai/MiniMax-H3-PTD-RefAsymOPD-LoRAs/discussions/3)) and |
| taobao-content-ai corrected it to `pdd_num_steps: 6, pdd_block_size: 1` in commit `d1f88c8`; the tensors are |
| unchanged. This file carries the same corrected values, so it matches their current release. The 6-step has |
| not yet been render-compared against taobao's node the way the 4- and 8-step Ref2VA files were. |
|
|
| **Which one?** `int8` is the recommended default. It's a quarter of the original size, needs a quarter |
| of the host RAM, and its error sits far below the margins discussed below. `int8_r5` saves a further 11% |
| for about 5× the error. `bf16` is the one the unmodified official nodes can load. For Ref2VA, use |
| `ptd_h3_ref2va_4step_int8`, or an 8-step file for sharper detail at twice the steps; |
| `ptd_h3_ref2va_refasymopd_8step_int8` is taobao's improved 8-step at the same cost, and |
| `ptd_h3_ref2va_refasymopd_6step_int8` its faster 6-step. |
|
|
| ## Install |
|
|
| 1. Clone the official PTD pack into `custom_nodes`: |
|
|
| ```sh |
| cd ComfyUI/custom_nodes |
| git clone --branch release/comfyui-inference --single-branch https://github.com/vita-epfl/PTD-inference ComfyUI-PTD |
| ``` |
|
|
| 2. Clone the compact loader next to it: |
|
|
| ```sh |
| git clone https://github.com/Nynxz/ComfyUI-PTD-Compact |
| ``` |
|
|
| 3. Put the adapter files in `ComfyUI/models/ptd/`. |
|
|
| No pip install is needed: the compact node puts the official checkout on the import path itself. If |
| ComfyUI happens to import the official pack first, its log shows a harmless `No module named 'ptd'` |
| error, and the compact node then registers the PTD nodes itself. |
|
|
| Use **Load PTD MiniMax-H3 Adapters (compact)** for every file here. It is the only loader that takes the |
| int8 files, the Ref2VA files, the pruned base, and DynamicVRAM. With it in the graph, PTD Sampler and |
| PTD Sigmas (official or compact) handle both releases. The official *Load PTD MiniMax-H3 Adapters* node also loads |
| the bf16 files, but only on the full base with `--disable-dynamic-vram`. |
|
|
| ## Workflow |
|
|
| Start from the official example, `examples/comfyui/ptd_h3_t2v.json` in the PTD pack, and swap in the |
| compact loader: |
|
|
| `Load Diffusion Model → Load PTD MiniMax-H3 Adapters (compact) → Basic Guider + PTD Sampler + PTD Sigmas |
| → SamplerCustomAdvanced → Separate AV Latent → VAE Decode / VAE Decode Audio → Create Video (24 fps)` |
|
|
| - **Step count and schedule:** PTD Sigmas outputs the checkpoint's fixed grid, so there is no steps, |
| scheduler or denoise to set. |
| - **Basic Guider, no CFG:** the student was distilled to run without it. |
| - **Resolution:** MiniMax H3 Image to Video at 1344 × 768 and 124 frames, which is what PTD was trained |
| and validated on. Other sizes may work. |
| - **No ModelSamplingMiniMaxH3 / shift node:** PTD sets the video and audio shifts itself on every call. |
| A shift node placed after the loader breaks its audio clock. |
| - **Don't** use a regular KSampler or scheduler, truncate the schedule, set denoise below 1, or start |
| from a non-empty latent. |
| - **Prompt style:** `integrated_multimodal_description: [Shot 1] …`, then `overall_soundscape: …` and |
| `non_diegetic_music: …`, as in the example. |
| - **Ref2VA files:** load a **ref2va** base and condition with **MiniMax H3 Reference to Video** (reference |
| images + prompt) instead; the prompt format is in taobao-content-ai's `asset/prompts_ref2va_test.json`. |
|
|
| ## Loader details |
|
|
| The loader lives at [Nynxz/ComfyUI-PTD-Compact](https://github.com/Nynxz/ComfyUI-PTD-Compact); see its |
| README for options (`exact` / `fast` kernels) and DynamicVRAM notes. It keeps the bank as int8 at its |
| true ranks in host RAM: about 2.8 GB for 4-step, against 5.5 GB for bf16 and 11.5 GB for fp32. Each |
| adapted layer copies only its active hop's factors to the GPU. Everything else is the official |
| `ptd_comfy` code: base checks, model clone, raw refiner, hop routing, audio clock and sampler. |
|
|
| ## Which base model |
|
|
| Each release needs the base it was trained on. The EPFL files were distilled from the original MiniMax-H3, |
| which is Comfy-Org's **fl2va** (its output heads match PTD's reference copies bit for bit). The Ref2VA files |
| were trained on MiniMax's `transformer_ref`, which is Comfy-Org's **ref2va** (its adapted layers are |
| byte-identical to it). |
|
|
| | Comfy-Org `diffusion_models/` file | Works | Notes | |
| |---|---|---| |
| | `minimax_h3_fl2va_bf16` | yes | the validated base | |
| | `minimax_h3_fl2va_pruned_bf16` | yes, compact loader | adapted layers byte-identical; AdaLN curve within 0.03% of the original at PTD's step times | |
| | `minimax_h3_fl2va_int8_convrot`, `minimax_h3_fl2va_pruned_int8_convrot`, `minimax_h3_fl2va_pruned_fp8_scaled` | yes | runs; the base quantization error (~0.5–1% of each weight) is larger than PTD's whole delta (~0.1–0.2%), so compare against bf16 if quality matters | |
| | `minimax_h3_fl2va_pruned_w6a8` | no | its video head is quantized, so PTD cannot verify the base | |
| | `minimax_h3_ref2va_bf16`, `…_pruned_bf16`, `…_int8_convrot`, `…_pruned_int8_convrot`, `…_pruned_fp8_scaled` | for the `ref2va` files | same checks: adapted layers byte-identical to `transformer_ref`; AdaLN curve within 0.026% at the Ref2VA step times | |
|
|
| The **pruned** base differs from the full one only in its time path: `time_embedder` and the full |
| `adaln_proj` weights are replaced by an 8-wide curve table. Every layer PTD adapts and both output heads |
| are byte-identical. Measured on blocks 0, 25, 49 and the final layer, the curve reproduces the original |
| AdaLN modulation to at most 0.026% at all of PTD's 4- and 6-step times (and at the Ref2VA 4- and 8-step |
| times on the ref2va base). That is 10× below the original's |
| own bf16 rounding (0.2%). The official loader refuses curve bases only because they were never |
| validated. The compact loader accepts them for every file here, the bf16 ones included. |
|
|
| ## With other LoRAs |
|
|
| Ordinary MiniMax-H3 LoRAs stack with PTD in either order. Use *LoraLoaderModelOnly* before or after |
| the PTD loader. A LoRA is a weight patch merged into the base as usual. PTD adds each hop's delta |
| through forward hooks on the layer outputs, on top of whatever the weights are, and keeps no |
| GPU-resident copy. The loader's test suite ([tests](https://github.com/Nynxz/ComfyUI-PTD-Compact/tree/main/tests)) checks this |
| on a real native H3 model in ComfyUI: both orders, every adapted layer kind (fused QKV, out, SwiGLU fc1, |
| fc2, refiner), each hop. A LoRA trained against the 50-step base may still respond differently on the |
| few-step student; that's a property of the model, not of these files. |
|
|
| ## How they were shrunk |
|
|
| The analysis that picked these encodings, on the 4-step set (the 6-step set gives the same picture): |
|
|
| - **The adapters use nearly all of their rank.** To hold 99% of its energy the median adapter needs |
| 99 of its 128 ranks (105 for 6-step), and a quarter need 110 or more. Truncating rank alone buys little: |
| 12% smaller costs ~5% error. |
| - **The hops are unrelated.** The cosine similarity between one layer's deltas at two different hops |
| has a median of 0.01 (99th percentile 0.37), so a basis shared across hops, or "mean + per-hop |
| residual", gains nothing. |
| - **The deltas are tiny next to the base.** On the base blocks sampled (0–3, 22–25, 47–49), |
| `‖ΔW‖/‖W‖` has a median of 0.09–0.17% per layer kind (max 1.8%, an fc2). A 1% adapter error is |
| therefore ~10⁻⁵ of the weight, far below the bf16 rounding the base already lives with (~0.1–0.2%). |
| This is the margin the int8 encoding relies on. |
| - **int8 with one scale per row** costs ~0.6–1.1% adapter error. Finer groups (128/64/32) improve it |
| only to ~0.5%, not worth the extra scales. |
| - **Rank allocation, compared at equal size** (4-step, bf16 sizes): |
|
|
| | Target | Size | Uniform rank (max err) | Waterfill on ‖·‖_F (max / total) | Waterfill relative (max / total) | Per-layer ceiling (max / total) | |
| |---|---:|---|---|---|---| |
| | 2% | 5.39 GB | 8.8% | 9.3% / 1.1% | 4.3% / 1.7% | **2.0% / 1.7%** | |
| | 5% | 4.91 GB | 20.9% | 16.4% / 3.6% | 8.9% / 4.5% | **5.0% / 4.8%** | |
| | 10% | 3.97 GB | 34.9% | 24.3% / 8.0% | 14.0% / 9.1% | **10.0% / 9.6%** | |
| |
| Waterfilling on absolute Frobenius error gives the lowest total error but concentrates it in a few |
| layers (up to 16–24%). The per-layer ceiling (each adapter keeps the smallest rank with relative |
| error ≤ ε, via exact SVD of `B·A`) bounds every layer instead, and `_r5` uses it with ε = 5%. Truncated |
| factors are `B' = U_r S_r^½`, `A' = S_r^½ V_rᵀ`, then int8. |
|
|
| End-to-end renders have since been run on an RTX 5090 with the pruned int8 bases: on the same seed, |
| swapping bf16 for int8 adapters, the int8 base for bf16, or `exact` for `fast` kernels each changes the |
| video by about as much as rounding alone does. The adapter-level errors above are exact; judge quality on |
| your own prompts. |
|
|
| The scripts are in `tools/`: `analyze.py` (spectra, hop similarity, storage errors), `allocate.py` |
| (strategy comparison and rank files), `export.py`, `verify.py`. |
|
|
| ## License |
|
|
| These files are Model Derivatives under the MiniMax H3 Community License Agreement, modified from |
| EPFL VITA Lab's PTD checkpoints and taobao-content-ai's Ref2VA PTD LoRAs (plain and RefAsymOPD). See [LICENSE-MiniMax-H3.txt](LICENSE-MiniMax-H3.txt) and |
| [NOTICE](NOTICE). The license covers its Applicable Territory only, which excludes the European Union, |
| the United Kingdom, the Republic of Korea and the United States. |
|
|
| MiniMax H3 is licensed under the MiniMax H3 Community License Agreement, Copyright © 2026 MiniMax. All Rights Reserved. |
|
|
| ```bibtex |
| @article{feng2026ptd, |
| title = {Parametric Trajectory Distillation for Few-Step Video Generation}, |
| author = {Feng, Lan and Karkus, Peter and Igl, Maximilian and Berner, Julius and Chen, Yuxiao and |
| Tan, Shuhan and Alahi, Alexandre and Ivanovic, Boris and Pavone, Marco}, |
| journal = {arXiv preprint arXiv:2610.11498}, |
| year = {2026} |
| } |
| ``` |
|
|
