Buckets:
Comfy Quantization
Comfy Quants exports prequantized checkpoints for ComfyUI-compatible loaders. Diffusers loads supported checkpoints with ComfyQuantizationConfig and uses Comfy-Kitchen for inference.
Currently, this integration supports native int8_tensorwise linear layers with optional ConvRot rotation. Other
Comfy quantization formats are not supported.
INT8 ConvRot
ConvRot uses group-wise regular Hadamard rotations to spread activation outliers. Native Comfy INT8 checkpoints store rotated INT8 weights, FP32 scales, and per-layer rotation metadata. At inference, Comfy-Kitchen rotates and dynamically quantizes activations before INT8 matrix multiplication. Weights stay in INT8 throughout inference.
Install the optional runtime dependency:
pip install "comfy-kitchen>=0.2.41" accelerate
The native CUDA backend requires a CUDA 13.0 or newer PyTorch build and an NVIDIA GPU with INT8 tensor cores (compute capability 7.5 or newer). Comfy-Kitchen selects the available backend; unsupported configurations may use its eager backend. Memory savings do not guarantee faster inference.
Load a checkpoint
Use ComfyQuantizationConfig with from_single_file to load an existing native Comfy INT8 checkpoint.
Z-Image Turbo,
Qwen-Image 2.1, and pruned or unpruned
MiniMax-H3 FL2VA and REF2VA transformers are supported.
The INT8 linear runtime is shared across architectures. Loading another model also requires a compatible checkpoint converter that maps its weights, scales, and rotation metadata, and a model implementation that uses the floating compute dtype for activations.
import torch
from diffusers import ComfyQuantizationConfig, ZImagePipeline, ZImageTransformer2DModel
transformer = ZImageTransformer2DModel.from_single_file(
"https://huggingface.co/Comfy-Org/Z-Image-Turbo/blob/main/split_files/diffusion_models/z_image_turbo_int8_convrot.safetensors",
config="Tongyi-MAI/Z-Image-Turbo",
subfolder="transformer",
quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16),
dtype=torch.bfloat16,
)
pipe = ZImagePipeline.from_pretrained(
"Tongyi-MAI/Z-Image-Turbo", transformer=transformer, dtype=torch.bfloat16
)
pipe.enable_model_cpu_offload()
image = pipe(
"A small orange cat sitting on a mossy stone wall in a mountain village",
height=1024,
width=1024,
num_inference_steps=8,
guidance_scale=0.0,
generator=torch.Generator("cpu").manual_seed(0),
).images[0]
For Qwen-Image 2.1, use QwenImage21Transformer2DModel with
Comfy-Org's INT8 checkpoint
and the configuration from Qwen/Qwen-Image-2.1.
Pass the loaded transformer to QwenImage21Pipeline.
To reduce host memory as well as VRAM peaks, encode prompts in a separate process, save the embeddings, and run
generation with text_encoder=None. Supply the embeddings and their masks on the execution device. Keep the
VAE offloaded during denoising and enable VAE tiling when needed.
Checkpoint format and conversion
Each quantized linear layer must have an INT8 [out_features, in_features] weight, an FP32 scalar or per-output-row
weight_scale, and a uint8 JSON comfy_quant tensor declaring "format": "int8_tensorwise". Rotated layers require
per-output-row scales, "convrot": true, and a power-of-four convrot_groupsize that divides the input width.
The per-layer layout follows the native Comfy INT8 format.
The older int8_w8a8 export uses a different marker for the retired ComfyUI-INT8-Fast loader and is not supported.
The quantizer reads these markers and replaces matching linear layers independently of the model architecture. Checkpoint conversion still maps the source model's layer names to Diffusers names and handles fused projections. The loader splits scales alongside fused QKV and gate/up weights, and preserves the associated rotation metadata. Unquantized layers use the requested precision, except for modules the model keeps in FP32. BF16 and FP16 computation are supported.
The Comfy format identifies a layer's quantization scheme; it does not define a universal mapping between model architectures. New model support requires checking its existing single-file converter for scale and marker preservation, projection row ordering, and floating activation dtypes. MXFP8, NVFP4, and INT4 use different scale or packing layouts and require format-specific conversion before they can be supported.
This integration loads prequantized single-file transformer and MiniMax-H3 video VAE checkpoints. It does not support
the older ComfyUI-INT8-Fast marker format, other Comfy quantization formats, on-load quantization, training, LoRA,
attention projection fusion, or save_pretrained for quantized models.
MiniMax-H3
Load the pruned or unpruned Comfy transformer with the same config.
Pruned Comfy checkpoints replace the timestep MLP with an adaln_t_table and lower-rank modulation projections.
from_single_file infers adaln_curve_grid and time_embed_dim from that table. Curve interpolation and projection
run in FP32 without applying SiLU; the denoising stream retains its requested precision. Quantized unpruned AdaLN
projections use their floating compute dtype for activations.
from diffusers import ComfyQuantizationConfig, MiniMaxH3Transformer3DModel
transformer = MiniMaxH3Transformer3DModel.from_single_file(
"https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/diffusion_models/minimax_h3_fl2va_pruned_int8_convrot.safetensors",
config="MiniMaxAI/MiniMax-H3",
subfolder="transformer",
quantization_config=ComfyQuantizationConfig(),
)
For the unpruned FL2VA model, select
minimax_h3_fl2va_int8_convrot.safetensors.
QKV splits and SwiGLU gate/value
reordering apply to the weights and their per-row scales together. The published Comfy files use the default
stacked QKV layout; a file merged directly from the original interleaved shards requires qkv_layout="interleaved".
For REF2VA, select
minimax_h3_ref2va_pruned_int8_convrot.safetensors
or minimax_h3_ref2va_int8_convrot.safetensors,
load the configuration from subfolder="transformer_ref", and pass
the loaded model as the modular pipeline's transformer_ref component.
MiniMax-H3 uses the existing modular video/audio workflow. Its INT8 transformer is still large, so phase prompt encoding, denoising, and VAE decoding separately when host memory is limited, and offload transformer blocks as needed. See the MiniMax-H3 pipeline guide for packing and sampling constraints.
Video VAE
Load the Comfy-Org INT8 video VAE with the native single-file loader:
import torch
from diffusers import AutoencoderKLMiniMaxH3, ComfyQuantizationConfig
vae = AutoencoderKLMiniMaxH3.from_single_file(
"https://huggingface.co/Comfy-Org/MiniMax-H3/blob/main/vae/minimax_h3_video_vae_int8_convrot.safetensors",
config="MiniMaxAI/MiniMax-H3",
subfolder="vae",
quantization_config=ComfyQuantizationConfig(compute_dtype=torch.bfloat16),
dtype=torch.bfloat16,
)
vae.to("cuda")
The decoder transformer projections retain their native INT8 weights and FP32 scales. The encoder, latent
convolutions, normalization, residual anchors, and unquantized boundary projections stay in FP32; quantized
projection biases and computation use the requested compute dtype. Keep the mixed precision established by the
loader: move the VAE with vae.to("cuda") without passing a floating dtype.
The video VAE stores QKV rows interleaved by attention head. The loader separates these into Q, K, and V projections,
reordering their weights, scales, and biases together, and also reorders SwiGLU gate/value rows. Checkpoint
latents_mean and latents_std tensors override the corresponding configuration values. Encode/decode operate
on unnormalized latents: normalize encoded latents with (latents - mean) / std before denoising, and undo that
normalization before decoding. The existing MiniMax-H3 workflow handles this normalization.
Spatial tiling remains enabled by default. This support targets the video VAE checkpoint linked above; the H3 audio and SOL VAEs use different architectures.
Xet Storage Details
- Size:
- 9.38 kB
- Xet hash:
- 52443eb72b52f11f7d249feb1c5c3350a0113e93b9257572152f47e41a73ecdd
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.