Instructions to use intel-ai/Qwen-Image-2.1-MXFP8DiT-AutoRound with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use intel-ai/Qwen-Image-2.1-MXFP8DiT-AutoRound with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("intel-ai/Qwen-Image-2.1-MXFP8DiT-AutoRound", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("intel-ai/Qwen-Image-2.1-MXFP8DiT-AutoRound", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]Model Details
This model is an MXFP8 (W8A8, group_size 32, symmetric) conversion of Qwen/Qwen-Image-2.1. Only the image transformer is quantized; the Qwen3-VL text encoder and VAE stay in BF16.
- Quantized (MXFP8): 32 DiT blocks ×
attn.to_q/to_k/to_v/to_out.0andimg_mlp.proj/gate_layer/out(224 Linear layers). - Kept in BF16: Qwen3-VL text encoder, VAE, DiT entry/exit layers (
img_in,txt_in,time_text_embed,modulation,norm_out,proj_out) and all norms. - Checkpoint scope: transformer 7.0 GB (from 14.2 GB); text encoder/VAE unchanged.
How to Use
Environment
uv pip install "diffusers==0.42.1" "transformers==5.19.0" "auto-round==0.17.0" accelerate
Diffusers Usage
import torch
from diffusers import DiffusionPipeline, PipelineQuantizationConfig, AutoRoundConfig
pipe = DiffusionPipeline.from_pretrained(
"<org>/Qwen-Image-2.1-MXFP8-RTN-AutoRound",
quantization_config=PipelineQuantizationConfig(
quant_mapping={
"transformer": AutoRoundConfig(
bits=8, group_size=32, data_type="mx_fp", backend="auto"
)
}
),
dtype=torch.bfloat16,
device_map="cuda",
)
image = pipe(
prompt='A neon shop sign that reads "QWEN IMAGE 2.1", rainy night, reflections on wet pavement',
width=2048, height=2048,
num_inference_steps=40,
generator=torch.Generator("cuda").manual_seed(42),
).images[0]
image.save("t2i_example.png")
vLLM-Omni Note
vLLM-Omni's MXFP8 has two kernel paths. The Marlin kernel is W8A16
(input_quant_key=None) and silently ignores the checkpoint's activation
quantization; with it the RGBA result collapses (CLIP 22.1). The
FlashInfer kernel is W8A8 (kMxfp8Dynamic) and recovers RGBA 34.7.
Enable FlashInfer with matching flashinfer-python / flashinfer-cubin
versions and CUDA_HOME / nvcc / ninja, or use diffusers. Check that
layer.quant_method.scheme.kernel is a FlashInfer*Mxfp8LinearKernel, not
MarlinMxfp8LinearKernel.
Evaluation
Run note: this directory's
gate.jsonis the raw Marlin W8A16 run (NO-GO, RGBA 22.1). The authoritative diffusers W8A8 evaluation isresults/qwen-image-2.1-mxfp8-diffusers/(GO, T2I/RGBA 45.38/34.37, average 39.88). FlashInfer W8A8 also recovers RGBA to 34.7.
Official model-card examples, generated by diffusers at 2048×2048, 40 steps,
seed 42, true_cfg_scale=1.0 (no CFG). CLIP score is image-text cosine ×100
(higher is better).
| Example | BF16 | MXFP8 | Recovery |
|---|---|---|---|
| Text-to-Image (neon sign) | 46.08 | 45.39 | 98.51% |
| RGBA (transparent dragon sticker) | 33.65 | 34.37 | 102.14% |
| Average | 39.86 | 39.88 | 100.03% |
Gate: GO (average drop 0.01).
Pixel/alpha comparison (same seed and scheduler):
- T2I: PSNR 17.96 dB, SSIM 0.778. Sampling is stochastic, so pixels differ; CLIP is within 0.7 of BF16.
- RGBA: alpha MAE 0.043, alpha IoU@0.5 0.851; transparency structure is preserved.
Generate the Model
auto-round \
--model_name Qwen/Qwen-Image-2.1 \
--scheme MXFP8 --iters 0 --format auto_round \
--output_dir Qwen-Image-2.1-MXFP8-RTN-AutoRound \
--disable_opt_rtn --disable_model_free \
--dataset coco2014 --num_inference_steps 40 --guidance_scale 1.0
Evaluation (examples/run_qwen_image_2.1_mxfp8_diffusers.yaml):
aqa run examples/run_qwen_image_2.1_mxfp8_diffusers.yaml
Ethical Considerations and Limitations
The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs. Therefore, before deploying any applications of this model, developers should perform safety testing.
Caveats and Recommendations
- vLLM-Omni MXFP8 scheme: the default Marlin fallback is W8A16 and makes
RGBA much worse for this W8A8 checkpoint. Enable the FlashInfer W8A8 kernel
(matching
flashinfer-cubin) or use diffusers. - RTN (iters=0) was used; no calibration or tuning.
- MXFP8 inference is experimental in auto-round/diffusers; speed is not necessarily better than BF16.
- Diffusers
AutoRoundConfigmust be given explicitbits=8, group_size=32, data_type="mx_fp"; the defaults are W4G128.
Disclaimer
The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.
Cite
@article{cheng2023optimize,
title={Optimize weight rounding via signed gradient descent for the quantization of llms},
author={Cheng, Wenhua and Zhang, Weiwei and Shen, Haihao and Cai, Yiyang and He, Xin and Lv, Kaokao and Liu, Yi},
journal={arXiv preprint arXiv:2309.05516},
year={2023}
}
Signed-off-by: {member name} <{member email}>
- Downloads last month
- 44
Model tree for intel-ai/Qwen-Image-2.1-MXFP8DiT-AutoRound
Base model
Qwen/Qwen-Image-2.1


