Qwen-Image-2.1-Turbo-INT8

An INT8 conversion of Qwen-Image-2.1-Turbo for LightX2V's int8-metal backend on Apple M5 or M6. It supports text-to-image generation and image editing with the Turbo model's 8-step sampling schedule.

This directory contains the complete model. Set LightX2V's model_path to this directory; a separate copy of the original BF16 model is not required. Quantization applies only to selected DiT Linear weights. The text encoder, VAE, and other components retain their original contents.

`apple_silicon_metal/`

This folder provides the precompiled GPU kernels required by LightX2V's int8-metal backend. Keep it alongside the model when deploying on Apple M5 or M6.

Kernel source code is available in LightX2V / lightx2v_kernel / apple.

apple_silicon_metal/
├── int8_k4096_n4096.metallib
├── int8_k4096_n12288.metallib
└── int8_k12288_n4096.metallib

These files contain the Metal kernels for online activation quantization and INT8 matrix multiplication. The kernels take BF16 activations, quantize them to INT8 per token, multiply them by the INT8 weights, and produce BF16 output after applying the activation and weight scales.

Model weights and scales are stored separately under transformer/. The .metallib files contain GPU code and do not contain model weights.

What each file covers

For X[M,K] @ W[K,N], K is the input feature dimension, N is the output feature dimension, and M is the number of activation rows.

Library	K	N	DiT Linear layers
`int8_k4096_n4096.metallib`	4096	4096	Attention Q, K, V, and output projections
`int8_k4096_n12288.metallib`	4096	12288	MLP up and gate projections
`int8_k12288_n4096.metallib`	12288	4096	MLP down projection

All three libraries are required to cover this model's quantized DiT layers. Each library is shared across Transformer blocks with the same (K,N) dimensions.

M remains dynamic. Different image resolutions change the activation row count M without changing the layer dimensions K and N, so a new library is not needed for each resolution.

Usage with LightX2V

Requirements

Apple M5 or M6 with PyTorch MPS enabled.
macOS 26.4 or later. The included .metallib files target a minimum deployment version of 26.4.
Python 3.10 or later and PyTorch 2.14 or later, with torch.mps.load_metallib available.
LightX2V code containing the int8-metal backend, Qwen-Image-2.1 MPS configuration, and launch scripts.

Clone LightX2V and enter the repository:

git clone https://github.com/ModelTC/LightX2V.git
cd LightX2V

Install LightX2V's inference dependencies according to its project instructions. Then install the Apple kernel package from the LightX2V repository root, using the same Python environment:

pip install ./lightx2v_kernel/apple

Before running either script below, update lightx2v_path and model_path in the script to your local paths. Adjust --prompt and, for image editing, --image_path as needed.

Text-to-image

bash scripts/platforms/mps/qwen_image_21_turbo_t2i_int8.sh

Image editing

bash scripts/platforms/mps/qwen_image_21_turbo_i2i_int8.sh

Rebuilding the Metal libraries

The included libraries have been validated on Apple M5. Other chip and OS combinations require separate validation. To rebuild the libraries, install the Metal Toolchain and run this command from the LightX2V repository root:

python lightx2v_kernel/apple/python/lightx2v_kernel_apple/int8_gemm/compile_metallib.py \
    4096x4096 4096x12288 12288x4096 \
    --output-dir /path/to/Qwen-Image-2.1-Turbo-INT8/apple_silicon_metal