LightX2V Lightning for Qwen-Image-2.1

A 4-step distilled LoRA for Qwen-Image-2.1, supporting text-to-image generation and image editing. Training was performed entirely on 16 Huawei Ascend 910B NPUs, using DMD in LightX2V.

For inference on heterogeneous hardware platforms, refer to the LightX2V platform scripts.

Both examples use the training noise schedule: four steps with dynamic exponential shifting and no terminal stretching. Training and LightX2V inference keep latents in FP32; these BF16 Diffusers examples can produce different numerical results.

Text-to-image

Run this example from the directory containing pytorch_lora_weights.safetensors:

import torch
from diffusers import FlowMatchEulerDiscreteScheduler, QwenImage21Pipeline

pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
).to("cuda")
pipe.load_lora_weights(
    ".", weight_name="pytorch_lora_weights.safetensors", adapter_name="lightning"
)

pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
    pipe.scheduler.config,
    num_train_timesteps=1000,
    use_dynamic_shifting=True,
    time_shift_type="exponential",
    base_image_seq_len=256,
    max_image_seq_len=8192,
    base_shift=0.5,
    max_shift=0.9,
    shift_terminal=None,
)

image = pipe(
    prompt='A coffee shop entrance with a chalkboard reading "Qwen Coffee, $2 per cup" and a neon sign displaying "通义千问".',
    width=1024,
    height=1024,
    num_inference_steps=4,
    sigmas=[1.0, 0.75, 0.5, 0.25],
    true_cfg_scale=1.0,
    use_kv_cache=False,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]
image.save("t2i.png")

Image editing

Run this example from the directory containing pytorch_lora_weights.safetensors and input.png:

import torch
from PIL import Image
from diffusers import FlowMatchEulerDiscreteScheduler, QwenImage21Pipeline

pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
).to("cuda")
pipe.load_lora_weights(
    ".", weight_name="pytorch_lora_weights.safetensors", adapter_name="lightning"
)

pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
    pipe.scheduler.config,
    num_train_timesteps=1000,
    use_dynamic_shifting=True,
    time_shift_type="exponential",
    base_image_seq_len=256,
    max_image_seq_len=8192,
    base_shift=0.5,
    max_shift=0.9,
    shift_terminal=None,
)

image = pipe(
    prompt="Change the red jacket to dark blue. Keep the person's identity, pose, background and lighting unchanged.",
    image=Image.open("input.png"),
    width=1024,
    height=1024,
    num_inference_steps=4,
    sigmas=[1.0, 0.75, 0.5, 0.25],
    true_cfg_scale=1.0,
    use_kv_cache=False,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]
image.save("edit.png")

For multiple references, use image=[Image.open("reference_1.png"), Image.open("reference_2.png")].

Comparison

Same prompts, seeds and output sizes; CFG is disabled for both models. The 40-step base-model images are from an earlier run. Click an example name to view its prompt.