Buckets:

|
download
raw
9.38 kB
# Comfy Quantization
[Comfy Quants](https://github.com/Comfy-Org/comfy-quants) exports prequantized checkpoints for ComfyUI-compatible
loaders. Diffusers loads supported checkpoints with [ComfyQuantizationConfig](/docs/diffusers/pr_15209/en/api/quantization#diffusers.ComfyQuantizationConfig) and uses
[Comfy-Kitchen](https://github.com/Comfy-Org/comfy-kitchen) for inference.
Currently, this integration supports native `int8_tensorwise` linear layers with optional ConvRot rotation. Other
Comfy quantization formats are not supported.
## INT8 ConvRot
[ConvRot](https://arxiv.org/abs/2512.03673) uses group-wise regular Hadamard rotations to spread activation outliers.
Native Comfy INT8 checkpoints store rotated INT8 weights, FP32 scales, and per-layer rotation metadata. At inference,
[Comfy-Kitchen](https://github.com/Comfy-Org/comfy-kitchen) rotates and dynamically quantizes activations before INT8
matrix multiplication. Weights stay in INT8 throughout inference.
Install the optional runtime dependency:
```bash
pip install "comfy-kitchen>=0.2.41" accelerate
```
The native CUDA backend requires a CUDA 13.0 or newer PyTorch build and an NVIDIA GPU with INT8 tensor cores
(compute capability 7.5 or newer). Comfy-Kitchen selects the available backend; unsupported configurations may use
its eager backend. Memory savings do not guarantee faster inference.
## Load a checkpoint
Use [ComfyQuantizationConfig](/docs/diffusers/pr_15209/en/api/quantization#diffusers.ComfyQuantizationConfig) with `from_single_file` to load an existing native Comfy INT8 checkpoint.
[Z-Image Turbo](https://huggingface.co/Comfy-Org/Z-Image-Turbo),
[Qwen-Image 2.1](https://huggingface.co/Comfy-Org/Qwen-Image-2.1), and pruned or unpruned
[MiniMax-H3](https://huggingface.co/Comfy-Org/MiniMax-H3) FL2VA and REF2VA transformers are supported.
The INT8 linear runtime is shared across architectures. Loading another model also requires a compatible
checkpoint converter that maps its weights, scales, and rotation metadata, and a model implementation that uses
the floating compute dtype for activations.
```python
import torch
from diffusers import ComfyQuantizationConfig, ZImagePipeline, ZImageTransformer2DModel
transformer = ZImageTransformer2DModel.from_single_file(
"https://huggingface.co/Comfy-Org/Z-Image-Turbo/blob/main/split_files/diffusion_models/z_image_turbo_int8_convrot.safetensors",
config="Tongyi-MAI/Z-Image-Turbo",
subfolder="transformer",
quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16),
dtype=torch.bfloat16,
)
pipe = ZImagePipeline.from_pretrained(
"Tongyi-MAI/Z-Image-Turbo", transformer=transformer, dtype=torch.bfloat16
)
pipe.enable_model_cpu_offload()
image = pipe(
"A small orange cat sitting on a mossy stone wall in a mountain village",
height=1024,
width=1024,
num_inference_steps=8,
guidance_scale=0.0,
generator=torch.Generator("cpu").manual_seed(0),
).images[0]
```
For Qwen-Image 2.1, use `QwenImage21Transformer2DModel` with
[Comfy-Org's INT8 checkpoint](https://huggingface.co/Comfy-Org/Qwen-Image-2.1/blob/main/diffusion_models/qwen_image_2.1_int8_convrot.safetensors)
and the configuration from [Qwen/Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1).
Pass the loaded transformer to `QwenImage21Pipeline`.
To reduce host memory as well as VRAM peaks, encode prompts in a separate process, save the embeddings, and run
generation with `text_encoder=None`. Supply the embeddings and their masks on the execution device. Keep the
VAE offloaded during denoising and enable VAE tiling when needed.
## Checkpoint format and conversion
Each quantized linear layer must have an INT8 `[out_features, in_features]` weight, an FP32 scalar or per-output-row
`weight_scale`, and a uint8 JSON `comfy_quant` tensor declaring `"format": "int8_tensorwise"`. Rotated layers require
per-output-row scales, `"convrot": true`, and a power-of-four `convrot_groupsize` that divides the input width.
The per-layer layout follows the [native Comfy INT8 format](https://github.com/Comfy-Org/comfy-quants/blob/main/docs/formats/int8_tensorwise.md).
The older `int8_w8a8` export uses a different marker for the retired ComfyUI-INT8-Fast loader and is not supported.
The quantizer reads these markers and replaces matching linear layers independently of the model architecture.
Checkpoint conversion still maps the source model's layer names to Diffusers names and handles fused projections.
The loader splits scales alongside fused QKV and gate/up weights, and preserves the associated rotation metadata.
Unquantized layers use the requested precision, except for modules the model keeps in FP32. BF16 and FP16
computation are supported.
The Comfy format identifies a layer's quantization scheme; it does not define a universal mapping between model
architectures. New model support requires checking its existing single-file converter for scale and marker
preservation, projection row ordering, and floating activation dtypes. MXFP8, NVFP4, and INT4 use different scale
or packing layouts and require format-specific conversion before they can be supported.
This integration loads prequantized single-file transformer and MiniMax-H3 video VAE checkpoints. It does not support
the older ComfyUI-INT8-Fast marker format, other Comfy quantization formats, on-load quantization, training, LoRA,
attention projection fusion, or `save_pretrained` for quantized models.
## MiniMax-H3
Load the pruned or unpruned Comfy transformer with the same config.
Pruned Comfy checkpoints replace the timestep MLP with an `adaln_t_table` and lower-rank modulation projections.
`from_single_file` infers `adaln_curve_grid` and `time_embed_dim` from that table. Curve interpolation and projection
run in FP32 without applying SiLU; the denoising stream retains its requested precision. Quantized unpruned AdaLN
projections use their floating compute dtype for activations.
```python
from diffusers import ComfyQuantizationConfig, MiniMaxH3Transformer3DModel
transformer = MiniMaxH3Transformer3DModel.from_single_file(
"https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors",
config="MiniMaxAI/MiniMax-H3",
subfolder="transformer",
quantization_config=ComfyQuantizationConfig(),
)
```
For the unpruned FL2VA model, select
[`minimax_h3_fl2va_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_int8_convrot.safetensors).
QKV splits and SwiGLU gate/value
reordering apply to the weights and their per-row scales together. The published Comfy files use the default
stacked QKV layout; a file merged directly from the original interleaved shards requires `qkv_layout="interleaved"`.
For REF2VA, select
[`minimax_h3_ref2va_pruned_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_ref2va_pruned_int8_convrot.safetensors)
or [`minimax_h3_ref2va_int8_convrot.safetensors`](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_ref2va_int8_convrot.safetensors),
load the configuration from `subfolder="transformer_ref"`, and pass
the loaded model as the modular pipeline's `transformer_ref` component.
MiniMax-H3 uses the existing modular video/audio workflow. Its INT8 transformer is still large, so phase prompt
encoding, denoising, and VAE decoding separately when host memory is limited, and offload transformer blocks as
needed. See the [MiniMax-H3 pipeline guide](../api/pipelines/minimax_h3) for packing and sampling constraints.
### Video VAE
Load the [Comfy-Org INT8 video VAE](https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/vae/minimax_h3_video_vae_int8_convrot.safetensors)
with the native single-file loader:
```python
import torch
from diffusers import AutoencoderKLMiniMaxH3, ComfyQuantizationConfig
vae = AutoencoderKLMiniMaxH3.from_single_file(
"https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/vae/minimax_h3_video_vae_int8_convrot.safetensors",
config="MiniMaxAI/MiniMax-H3",
subfolder="vae",
quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16),
dtype=torch.bfloat16,
)
vae.to("cuda")
```
The decoder transformer projections retain their native INT8 weights and FP32 scales. The encoder, latent
convolutions, normalization, residual anchors, and unquantized boundary projections stay in FP32; quantized
projection biases and computation use the requested compute dtype. Keep the mixed precision established by the
loader: move the VAE with `vae.to("cuda")` without passing a floating dtype.
The video VAE stores QKV rows interleaved by attention head. The loader separates these into Q, K, and V projections,
reordering their weights, scales, and biases together, and also reorders SwiGLU gate/value rows. Checkpoint
`latents_mean` and `latents_std` tensors override the corresponding configuration values. Encode/decode operate
on unnormalized latents: normalize encoded latents with `(latents - mean) / std` before denoising, and undo that
normalization before decoding. The existing MiniMax-H3 workflow handles this normalization.
Spatial tiling remains enabled by default. This support targets the video VAE checkpoint linked above; the H3
audio and SOL VAEs use different architectures.

Xet Storage Details

Size:
9.38 kB
·
Xet hash:
52443eb72b52f11f7d249feb1c5c3350a0113e93b9257572152f47e41a73ecdd

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.