Buckets:
| # Comfy Quantization | |
| [Comfy Quants](https://github.com/Comfy-Org/comfy-quants) exports prequantized checkpoints for ComfyUI-compatible | |
| loaders. Diffusers loads supported checkpoints with [ComfyQuantizationConfig](/docs/diffusers/pr_15209/en/api/quantization#diffusers.ComfyQuantizationConfig) and uses | |
| [Comfy-Kitchen](https://github.com/Comfy-Org/comfy-kitchen) for inference. | |
| Currently, this integration supports native `int8_tensorwise` linear layers with optional ConvRot rotation. Other | |
| Comfy quantization formats are not supported. | |
| ## INT8 ConvRot | |
| [ConvRot](https://arxiv.org/abs/2512.03673) uses group-wise regular Hadamard rotations to spread activation outliers. | |
| Native Comfy INT8 checkpoints store rotated INT8 weights, FP32 scales, and per-layer rotation metadata. At inference, | |
| [Comfy-Kitchen](https://github.com/Comfy-Org/comfy-kitchen) rotates and dynamically quantizes activations before INT8 | |
| matrix multiplication. Weights stay in INT8 throughout inference. | |
| Install the optional runtime dependency: | |
| ```bash | |
| pip install "comfy-kitchen>=0.2.41" accelerate | |
| ``` | |
| The native CUDA backend requires a CUDA 13.0 or newer PyTorch build and an NVIDIA GPU with INT8 tensor cores | |
| (compute capability 7.5 or newer). Comfy-Kitchen selects the available backend; unsupported configurations may use | |
| its eager backend. Memory savings do not guarantee faster inference. | |
| ## Load a checkpoint | |
| Use [ComfyQuantizationConfig](/docs/diffusers/pr_15209/en/api/quantization#diffusers.ComfyQuantizationConfig) with `from_single_file` to load an existing native Comfy INT8 checkpoint. | |
| [Z-Image Turbo](https://huggingface.co/Comfy-Org/Z-Image-Turbo), | |
| [Qwen-Image 2.1](https://huggingface.co/Comfy-Org/Qwen-Image-2.1), and pruned or unpruned | |
| [MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3) FL2VA and REF2VA transformers are supported. | |
| The INT8 linear runtime is shared across architectures. Loading another model also requires a compatible | |
| checkpoint converter that maps its weights, scales, and rotation metadata, and a model implementation that uses | |
| the floating compute dtype for activations. | |
| ```python | |
| import torch | |
| from diffusers import ComfyQuantizationConfig, ZImagePipeline, ZImageTransformer2DModel | |
| transformer = ZImageTransformer2DModel.from_single_file( | |
| "https://huggingface.co/Comfy-Org/Z-Image-Turbo/blob/main/split_files/diffusion_models/z_image_turbo_int8_convrot.safetensors", | |
| config="Tongyi-MAI/Z-Image-Turbo", | |
| subfolder="transformer", | |
| quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16), | |
| dtype=torch.bfloat16, | |
| ) | |
| pipe = ZImagePipeline.from_pretrained( | |
| "Tongyi-MAI/Z-Image-Turbo", transformer=transformer, dtype=torch.bfloat16 | |
| ) | |
| pipe.enable_model_cpu_offload() | |
| image = pipe( | |
| "A small orange cat sitting on a mossy stone wall in a mountain village", | |
| height=1024, | |
| width=1024, | |
| num_inference_steps=8, | |
| guidance_scale=0.0, | |
| generator=torch.Generator("cpu").manual_seed(0), | |
| ).images[0] | |
| ``` | |
| For Qwen-Image 2.1, use `QwenImage21Transformer2DModel` with | |
| [Comfy-Org's INT8 checkpoint](https://huggingface.co/Comfy-Org/Qwen-Image-2.1/blob/main/diffusion_models/qwen_image_2.1_int8_convrot.safetensors) | |
| and the configuration from [Qwen/Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1). | |
| Pass the loaded transformer to `QwenImage21Pipeline`. | |
| To reduce host memory as well as VRAM peaks, encode prompts in a separate process, save the embeddings, and run | |
| generation with `text_encoder=None`. Supply the embeddings and their masks on the execution device. Keep the | |
| VAE offloaded during denoising and enable VAE tiling when needed. | |
| ## Checkpoint format and conversion | |
| Each quantized linear layer must have an INT8 `[out_features, in_features]` weight, an FP32 scalar or per-output-row | |
| `weight_scale`, and a uint8 JSON `comfy_quant` tensor declaring `"format": "int8_tensorwise"`. Rotated layers require | |
| per-output-row scales, `"convrot": true`, and a power-of-four `convrot_groupsize` that divides the input width. | |
| The per-layer layout follows the [native Comfy INT8 format](https://github.com/Comfy-Org/comfy-quants/blob/main/docs/formats/int8_tensorwise.md). | |
| The older `int8_w8a8` export uses a different marker for the retired ComfyUI-INT8-Fast loader and is not supported. | |
| The quantizer reads these markers and replaces matching linear layers independently of the model architecture. | |
| Checkpoint conversion still maps the source model's layer names to Diffusers names and handles fused projections. | |
| The loader splits scales alongside fused QKV and gate/up weights, and preserves the associated rotation metadata. | |
| Unquantized layers use the requested precision, except for modules the model keeps in FP32. BF16 and FP16 | |
| computation are supported. | |
| The Comfy format identifies a layer's quantization scheme; it does not define a universal mapping between model | |
| architectures. New model support requires checking its existing single-file converter for scale and marker | |
| preservation, projection row ordering, and floating activation dtypes. MXFP8, NVFP4, and INT4 use different scale | |
| or packing layouts and require format-specific conversion before they can be supported. | |
| This integration loads prequantized single-file transformer and MiniMax-H3 video VAE checkpoints. It does not support | |
| the older ComfyUI-INT8-Fast marker format, other Comfy quantization formats, on-load quantization, training, LoRA, | |
| attention projection fusion, or `save_pretrained` for quantized models. | |
| ## MiniMax-H3 | |
| Load the pruned or unpruned Comfy transformer with the same config. | |
| Pruned Comfy checkpoints replace the timestep MLP with an `adaln_t_table` and lower-rank modulation projections. | |
| `from_single_file` infers `adaln_curve_grid` and `time_embed_dim` from that table. Curve interpolation and projection | |
| run in FP32 without applying SiLU; the denoising stream retains its requested precision. Quantized unpruned AdaLN | |
| projections use their floating compute dtype for activations. | |
| ```python | |
| from diffusers import ComfyQuantizationConfig, MiniMaxH3Transformer3DModel | |
| transformer = MiniMaxH3Transformer3DModel.from_single_file( | |
| "https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors", | |
| config="MiniMaxAI/MiniMax-H3", | |
| subfolder="transformer", | |
| quantization_config=ComfyQuantizationConfig(), | |
| ) | |
| ``` | |
| For the unpruned FL2VA model, select | |
| [`minimax_h3_fl2va_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_int8_convrot.safetensors). | |
| QKV splits and SwiGLU gate/value | |
| reordering apply to the weights and their per-row scales together. The published Comfy files use the default | |
| stacked QKV layout; a file merged directly from the original interleaved shards requires `qkv_layout="interleaved"`. | |
| For REF2VA, select | |
| [`minimax_h3_ref2va_pruned_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_ref2va_pruned_int8_convrot.safetensors) | |
| or [`minimax_h3_ref2va_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_ref2va_int8_convrot.safetensors), | |
| load the configuration from `subfolder="transformer_ref"`, and pass | |
| the loaded model as the modular pipeline's `transformer_ref` component. | |
| MiniMax-H3 uses the existing modular video/audio workflow. Its INT8 transformer is still large, so phase prompt | |
| encoding, denoising, and VAE decoding separately when host memory is limited, and offload transformer blocks as | |
| needed. See the [MiniMax-H3 pipeline guide](../api/pipelines/minimax_h3) for packing and sampling constraints. | |
| ### Video VAE | |
| Load the [Comfy-Org INT8 video VAE](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/vae/minimax_h3_video_vae_int8_convrot.safetensors) | |
| with the native single-file loader: | |
| ```python | |
| import torch | |
| from diffusers import AutoencoderKLMiniMaxH3, ComfyQuantizationConfig | |
| vae = AutoencoderKLMiniMaxH3.from_single_file( | |
| "https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/vae/minimax_h3_video_vae_int8_convrot.safetensors", | |
| config="MiniMaxAI/MiniMax-H3", | |
| subfolder="vae", | |
| quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16), | |
| dtype=torch.bfloat16, | |
| ) | |
| vae.to("cuda") | |
| ``` | |
| The decoder transformer projections retain their native INT8 weights and FP32 scales. The encoder, latent | |
| convolutions, normalization, residual anchors, and unquantized boundary projections stay in FP32; quantized | |
| projection biases and computation use the requested compute dtype. Keep the mixed precision established by the | |
| loader: move the VAE with `vae.to("cuda")` without passing a floating dtype. | |
| The video VAE stores QKV rows interleaved by attention head. The loader separates these into Q, K, and V projections, | |
| reordering their weights, scales, and biases together, and also reorders SwiGLU gate/value rows. Checkpoint | |
| `latents_mean` and `latents_std` tensors override the corresponding configuration values. Encode/decode operate | |
| on unnormalized latents: normalize encoded latents with `(latents - mean) / std` before denoising, and undo that | |
| normalization before decoding. The existing MiniMax-H3 workflow handles this normalization. | |
| Spatial tiling remains enabled by default. This support targets the video VAE checkpoint linked above; the H3 | |
| audio and SOL VAEs use different architectures. | |
Xet Storage Details
- Size:
- 9.38 kB
- Xet hash:
- 52443eb72b52f11f7d249feb1c5c3350a0113e93b9257572152f47e41a73ecdd
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.