Buckets:
| <meta charset="utf-8" /><meta name="hf:doc:metadata" content="{"title":"Comfy Quantization","local":"comfy-quantization","sections":[{"title":"INT8 ConvRot","local":"int8-convrot","sections":[],"depth":2},{"title":"Load a checkpoint","local":"load-a-checkpoint","sections":[],"depth":2},{"title":"Checkpoint format and conversion","local":"checkpoint-format-and-conversion","sections":[],"depth":2},{"title":"MiniMax-H3","local":"minimax-h3","sections":[{"title":"Video VAE","local":"video-vae","sections":[],"depth":3}],"depth":2}],"depth":1}"/> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/entry/start.e5y4TNOs.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/chunks/TqhBDs8Z.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/chunks/Bupkjr9f.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/entry/app.C6u5ts0s.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/chunks/CyvZ3FS5.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/chunks/l0QnnOED.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/chunks/DvMogaAc.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/chunks/DsnmJJEf.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/nodes/0.CZhScj6h.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/chunks/CoSrdQax.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/nodes/316.BBGYi7XR.js" rel="modulepreload"> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/chunks/BithDya1.js" rel="modulepreload"> | |
| <!--8vwy03--><meta name="hf:doc:metadata" content="{"title":"Comfy Quantization","local":"comfy-quantization","sections":[{"title":"INT8 ConvRot","local":"int8-convrot","sections":[],"depth":2},{"title":"Load a checkpoint","local":"load-a-checkpoint","sections":[],"depth":2},{"title":"Checkpoint format and conversion","local":"checkpoint-format-and-conversion","sections":[],"depth":2},{"title":"MiniMax-H3","local":"minimax-h3","sections":[{"title":"Video VAE","local":"video-vae","sections":[],"depth":3}],"depth":2}],"depth":1}"/><!----> | |
| <link href="/docs/diffusers/pr_15209/en/_app/immutable/assets/0.tn0RQdqM.css" rel="modulepreload"> <!--[--><!--[0--><!--[--><!--[0--><!--[--><p></p> <div class="items-center shrink-0 min-w-[100px] max-sm:min-w-[50px] justify-end ml-auto flex" style="float: inline-end; margin-inline-start: 10px; display: inline-flex; position: relative; z-index: 10;"><div dir="ltr" class="inline-flex rounded-md max-sm:rounded-sm"><button class="inline-flex items-center gap-1 h-7 max-sm:h-7 px-2 max-sm:px-1.5 text-sm font-medium text-gray-800 border border-r-0 rounded-l-md max-sm:rounded-l-sm border-gray-200 bg-white hover:shadow-inner dark:border-gray-850 dark:bg-gray-950 dark:text-gray-200 dark:hover:bg-gray-800" aria-live="polite"><span class="inline-flex items-center justify-center rounded-md p-0.5 max-sm:p-0 hover:text-gray-800 dark:hover:text-gray-200"><svg class="sm:size-3.5 size-3" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg><!----></span> <span dir="auto">Copy page</span></button> <button class="inline-flex items-center justify-center w-6 max-sm:w-5 h-7 max-sm:h-7 disabled:pointer-events-none text-sm text-gray-500 hover:text-gray-700 dark:hover:text-white rounded-r-md max-sm:rounded-r-sm border border-l transition border-gray-200 bg-white hover:shadow-inner dark:border-gray-850 dark:bg-gray-950 dark:text-gray-200 dark:hover:bg-gray-800" aria-haspopup="menu" aria-expanded="false" aria-label="Open copy menu"><svg class="transition-transform text-gray-400 overflow-visible sm:size-3.5 size-3 rotate-0" width="1em" height="1em" viewBox="0 0 12 7" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M1 1L6 6L11 1" stroke="currentColor"></path></svg><!----></button></div> <!--[-1--><!--]--></div><!----> <!--[0--><h1 class="relative group"><a id="comfy-quantization" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#comfy-quantization"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->Comfy Quantization<!----></span></h1><!--]--><!----> <p><a href="https://github.com/Comfy-Org/comfy-quants" rel="nofollow">Comfy Quants</a> exports prequantized checkpoints for ComfyUI-compatible | |
| loaders. Diffusers loads supported checkpoints with <a href="/docs/diffusers/pr_15209/en/api/quantization#diffusers.ComfyQuantizationConfig">ComfyQuantizationConfig</a> and uses <a href="https://github.com/Comfy-Org/comfy-kitchen" rel="nofollow">Comfy-Kitchen</a> for inference.</p> <p>Currently, this integration supports native <code>int8_tensorwise</code> linear layers with optional ConvRot rotation. Other | |
| Comfy quantization formats are not supported.</p> <!--[1--><h2 class="relative group"><a id="int8-convrot" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#int8-convrot"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->INT8 ConvRot<!----></span></h2><!--]--><!----> <p><a href="https://arxiv.org/abs/2512.03673" rel="nofollow">ConvRot</a> uses group-wise regular Hadamard rotations to spread activation outliers. | |
| Native Comfy INT8 checkpoints store rotated INT8 weights, FP32 scales, and per-layer rotation metadata. At inference, <a href="https://github.com/Comfy-Org/comfy-kitchen" rel="nofollow">Comfy-Kitchen</a> rotates and dynamically quantizes activations before INT8 | |
| matrix multiplication. Weights stay in INT8 throughout inference.</p> <p>Install the optional runtime dependency:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg><!----> <div class=" absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0 "><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent;"></div> Copied</div><!----></button><!----></div> <pre class="language-bash "><!---->pip install <span class="hljs-string">"comfy-kitchen>=0.2.41"</span> accelerate<!----></pre></div><!----> <p>The native CUDA backend requires a CUDA 13.0 or newer PyTorch build and an NVIDIA GPU with INT8 tensor cores | |
| (compute capability 7.5 or newer). Comfy-Kitchen selects the available backend; unsupported configurations may use | |
| its eager backend. Memory savings do not guarantee faster inference.</p> <!--[1--><h2 class="relative group"><a id="load-a-checkpoint" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#load-a-checkpoint"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->Load a checkpoint<!----></span></h2><!--]--><!----> <p>Use <a href="/docs/diffusers/pr_15209/en/api/quantization#diffusers.ComfyQuantizationConfig">ComfyQuantizationConfig</a> with <code>from_single_file</code> to load an existing native Comfy INT8 checkpoint. <a href="https://huggingface.co/Comfy-Org/Z-Image-Turbo" rel="nofollow">Z-Image Turbo</a>, <a href="https://huggingface.co/Comfy-Org/Qwen-Image-2.1" rel="nofollow">Qwen-Image 2.1</a>, and pruned or unpruned <a href="https://huggingface.co/Comfy-Org/MiniMax-H3" rel="nofollow">MiniMax-H3</a> FL2VA and REF2VA transformers are supported.</p> <p>The INT8 linear runtime is shared across architectures. Loading another model also requires a compatible | |
| checkpoint converter that maps its weights, scales, and rotation metadata, and a model implementation that uses | |
| the floating compute dtype for activations.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg><!----> <div class=" absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0 "><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent;"></div> Copied</div><!----></button><!----></div> <pre class="language-python "><!----><span class="hljs-keyword">import</span> torch | |
| <span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> ComfyQuantizationConfig, ZImagePipeline, ZImageTransformer2DModel | |
| transformer = ZImageTransformer2DModel.from_single_file( | |
| <span class="hljs-string">"https://huggingface.co/Comfy-Org/Z-Image-Turbo/blob/main/split_files/diffusion_models/z_image_turbo_int8_convrot.safetensors"</span>, | |
| config=<span class="hljs-string">"Tongyi-MAI/Z-Image-Turbo"</span>, | |
| subfolder=<span class="hljs-string">"transformer"</span>, | |
| quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16), | |
| dtype=torch.bfloat16, | |
| ) | |
| pipe = ZImagePipeline.from_pretrained( | |
| <span class="hljs-string">"Tongyi-MAI/Z-Image-Turbo"</span>, transformer=transformer, dtype=torch.bfloat16 | |
| ) | |
| pipe.enable_model_cpu_offload() | |
| image = pipe( | |
| <span class="hljs-string">"A small orange cat sitting on a mossy stone wall in a mountain village"</span>, | |
| height=<span class="hljs-number">1024</span>, | |
| width=<span class="hljs-number">1024</span>, | |
| num_inference_steps=<span class="hljs-number">8</span>, | |
| guidance_scale=<span class="hljs-number">0.0</span>, | |
| generator=torch.Generator(<span class="hljs-string">"cpu"</span>).manual_seed(<span class="hljs-number">0</span>), | |
| ).images[<span class="hljs-number">0</span>]<!----></pre></div><!----> <p>For Qwen-Image 2.1, use <code>QwenImage21Transformer2DModel</code> with <a href="https://huggingface.co/Comfy-Org/Qwen-Image-2.1/blob/main/diffusion_models/qwen_image_2.1_int8_convrot.safetensors" rel="nofollow">Comfy-Org’s INT8 checkpoint</a> and the configuration from <a href="https://huggingface.co/Qwen/Qwen-Image-2.1" rel="nofollow">Qwen/Qwen-Image-2.1</a>. | |
| Pass the loaded transformer to <code>QwenImage21Pipeline</code>.</p> <p>To reduce host memory as well as VRAM peaks, encode prompts in a separate process, save the embeddings, and run | |
| generation with <code>text_encoder=None</code>. Supply the embeddings and their masks on the execution device. Keep the | |
| VAE offloaded during denoising and enable VAE tiling when needed.</p> <!--[1--><h2 class="relative group"><a id="checkpoint-format-and-conversion" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#checkpoint-format-and-conversion"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->Checkpoint format and conversion<!----></span></h2><!--]--><!----> <p>Each quantized linear layer must have an INT8 <code>[out_features, in_features]</code> weight, an FP32 scalar or per-output-row <code>weight_scale</code>, and a uint8 JSON <code>comfy_quant</code> tensor declaring <code>"format": "int8_tensorwise"</code>. Rotated layers require | |
| per-output-row scales, <code>"convrot": true</code>, and a power-of-four <code>convrot_groupsize</code> that divides the input width.</p> <p>The per-layer layout follows the <a href="https://github.com/Comfy-Org/comfy-quants/blob/main/docs/formats/int8_tensorwise.md" rel="nofollow">native Comfy INT8 format</a>. | |
| The older <code>int8_w8a8</code> export uses a different marker for the retired ComfyUI-INT8-Fast loader and is not supported.</p> <p>The quantizer reads these markers and replaces matching linear layers independently of the model architecture. | |
| Checkpoint conversion still maps the source model’s layer names to Diffusers names and handles fused projections. | |
| The loader splits scales alongside fused QKV and gate/up weights, and preserves the associated rotation metadata. | |
| Unquantized layers use the requested precision, except for modules the model keeps in FP32. BF16 and FP16 | |
| computation are supported.</p> <p>The Comfy format identifies a layer’s quantization scheme; it does not define a universal mapping between model | |
| architectures. New model support requires checking its existing single-file converter for scale and marker | |
| preservation, projection row ordering, and floating activation dtypes. MXFP8, NVFP4, and INT4 use different scale | |
| or packing layouts and require format-specific conversion before they can be supported.</p> <p>This integration loads prequantized single-file transformer and MiniMax-H3 video VAE checkpoints. It does not support | |
| the older ComfyUI-INT8-Fast marker format, other Comfy quantization formats, on-load quantization, training, LoRA, | |
| attention projection fusion, or <code>save_pretrained</code> for quantized models.</p> <!--[1--><h2 class="relative group"><a id="minimax-h3" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#minimax-h3"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->MiniMax-H3<!----></span></h2><!--]--><!----> <p>Load the pruned or unpruned Comfy transformer with the same config.</p> <p>Pruned Comfy checkpoints replace the timestep MLP with an <code>adaln_t_table</code> and lower-rank modulation projections. <code>from_single_file</code> infers <code>adaln_curve_grid</code> and <code>time_embed_dim</code> from that table. Curve interpolation and projection | |
| run in FP32 without applying SiLU; the denoising stream retains its requested precision. Quantized unpruned AdaLN | |
| projections use their floating compute dtype for activations.</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg><!----> <div class=" absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0 "><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent;"></div> Copied</div><!----></button><!----></div> <pre class="language-python "><!----><span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> ComfyQuantizationConfig, MiniMaxH3Transformer3DModel | |
| transformer = MiniMaxH3Transformer3DModel.from_single_file( | |
| <span class="hljs-string">"https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors"</span>, | |
| config=<span class="hljs-string">"MiniMaxAI/MiniMax-H3"</span>, | |
| subfolder=<span class="hljs-string">"transformer"</span>, | |
| quantization_config=ComfyQuantizationConfig(), | |
| )<!----></pre></div><!----> <p>For the unpruned FL2VA model, select <a href="https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_int8_convrot.safetensors" rel="nofollow"><code>minimax_h3_fl2va_int8_convrot.safetensors</code></a>. | |
| QKV splits and SwiGLU gate/value | |
| reordering apply to the weights and their per-row scales together. The published Comfy files use the default | |
| stacked QKV layout; a file merged directly from the original interleaved shards requires <code>qkv_layout="interleaved"</code>.</p> <p>For REF2VA, select <a href="https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_ref2va_pruned_int8_convrot.safetensors" rel="nofollow"><code>minimax_h3_ref2va_pruned_int8_convrot.safetensors</code></a> or <a href="https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_ref2va_int8_convrot.safetensors" rel="nofollow"><code>minimax_h3_ref2va_int8_convrot.safetensors</code></a>, | |
| load the configuration from <code>subfolder="transformer_ref"</code>, and pass | |
| the loaded model as the modular pipeline’s <code>transformer_ref</code> component.</p> <p>MiniMax-H3 uses the existing modular video/audio workflow. Its INT8 transformer is still large, so phase prompt | |
| encoding, denoising, and VAE decoding separately when host memory is limited, and offload transformer blocks as | |
| needed. See the <a href="../api/pipelines/minimax_h3">MiniMax-H3 pipeline guide</a> for packing and sampling constraints.</p> <!--[2--><h3 class="relative group"><a id="video-vae" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#video-vae"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->Video VAE<!----></span></h3><!--]--><!----> <p>Load the <a href="https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/vae/minimax_h3_video_vae_int8_convrot.safetensors" rel="nofollow">Comfy-Org INT8 video VAE</a> with the native single-file loader:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5 text-gray-600 " title="code excerpt" type="button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg><!----> <div class=" absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0 "><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent;"></div> Copied</div><!----></button><!----></div> <pre class="language-python "><!----><span class="hljs-keyword">import</span> torch | |
| <span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> AutoencoderKLMiniMaxH3, ComfyQuantizationConfig | |
| vae = AutoencoderKLMiniMaxH3.from_single_file( | |
| <span class="hljs-string">"https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/vae/minimax_h3_video_vae_int8_convrot.safetensors"</span>, | |
| config=<span class="hljs-string">"MiniMaxAI/MiniMax-H3"</span>, | |
| subfolder=<span class="hljs-string">"vae"</span>, | |
| quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16), | |
| dtype=torch.bfloat16, | |
| ) | |
| vae.to(<span class="hljs-string">"cuda"</span>)<!----></pre></div><!----> <p>The decoder transformer projections retain their native INT8 weights and FP32 scales. The encoder, latent | |
| convolutions, normalization, residual anchors, and unquantized boundary projections stay in FP32; quantized | |
| projection biases and computation use the requested compute dtype. Keep the mixed precision established by the | |
| loader: move the VAE with <code>vae.to("cuda")</code> without passing a floating dtype.</p> <p>The video VAE stores QKV rows interleaved by attention head. The loader separates these into Q, K, and V projections, | |
| reordering their weights, scales, and biases together, and also reorders SwiGLU gate/value rows. Checkpoint <code>latents_mean</code> and <code>latents_std</code> tensors override the corresponding configuration values. Encode/decode operate | |
| on unnormalized latents: normalize encoded latents with <code>(latents - mean) / std</code> before denoising, and undo that | |
| normalization before decoding. The existing MiniMax-H3 workflow handles this normalization.</p> <p>Spatial tiling remains enabled by default. This support targets the video VAE checkpoint linked above; the H3 | |
| audio and SOL VAEs use different architectures.</p> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/diffusers/blob/main/docs/source/en/quantization/comfy.md" target="_blank"><svg class="mr-1" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M31,16l-7,7l-1.41-1.41L28.17,16l-5.58-5.59L24,9l7,7z"></path><path d="M1,16l7-7l1.41,1.41L3.83,16l5.58,5.59L8,23l-7-7z"></path><path d="M12.419,25.484L17.639,6.552l1.932,0.518L14.351,26.002z"></path></svg><!----> <span><span class="underline">Update</span> on GitHub</span></a><!----> <p></p><!--]--><!----><!--]--><!--]--><!--]--> <!--[-1--><!--]--><!--]--> | |
| <script> | |
| { | |
| __sveltekit_m76s4z = { | |
| base: "/docs/diffusers/pr_15209/en", | |
| assets: "/docs/diffusers/pr_15209/en" | |
| }; | |
| const element = document.currentScript.parentElement; | |
| Promise.all([ | |
| import("/docs/diffusers/pr_15209/en/_app/immutable/entry/start.e5y4TNOs.js"), | |
| import("/docs/diffusers/pr_15209/en/_app/immutable/entry/app.C6u5ts0s.js") | |
| ]).then(([kit, app]) => { | |
| kit.start(app, element, { | |
| node_ids: [0, 316], | |
| data: [null,null], | |
| form: null, | |
| error: null | |
| }); | |
| }); | |
| } | |
| </script> | |
Xet Storage Details
- Size:
- 29.1 kB
- Xet hash:
- b32d7a9f295ea96f90db93d7fd3e4da449bc2ee29d44864626d9003c8ed8b86e
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.