Instructions to use PrunaAI/Pruna-Q-Image-2.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use PrunaAI/Pruna-Q-Image-2.1 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Qwen/Qwen-Image-2.1", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("PrunaAI/Pruna-Q-Image-2.1") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
update README to have less changes to the previous version
Browse files
README.md
CHANGED
|
@@ -5,15 +5,14 @@ base_model: Qwen/Qwen-Image-2.1
|
|
| 5 |
library_name: diffusers
|
| 6 |
pipeline_tag: text-to-image
|
| 7 |
tags:
|
| 8 |
-
- qwen
|
| 9 |
-
- image-generation
|
| 10 |
-
- image-editing
|
| 11 |
-
- rgba
|
| 12 |
-
- lora
|
| 13 |
-
- distilled
|
| 14 |
-
- few-step
|
| 15 |
---
|
| 16 |
-
|
| 17 |
<!-- header start -->
|
| 18 |
<!-- 200823 -->
|
| 19 |
<p align="center">
|
|
@@ -42,42 +41,49 @@ tags:
|
|
| 42 |
<img src="https://img.shields.io/badge/Performance%20Models-Try%20them%20now-9334E9?style=plastic&logo=data:image/svg%2bxml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI0MTEiIGhlaWdodD0iNDc3IiBmaWxsPSJub25lIj48cGF0aCBmaWxsPSIjZmZmIiBkPSJNMjMuNzAzIDMwMC41MzJjLTEuODQyLS42NzQtNC4xNDItMS4wNTctNS43MDMtMi4wMjktMTIuNTU4LTcuODQ2LTguODM3LTI1LjE3MiA1LjI1NS0yOC4zNjFhLjY2LjY2IDAgMCAwIC4zNDgtLjIwMS42LjYgMCAwIDAgLjE1Ni0uMzZxNi40OS04MS45ODYgNjguNTc3LTEzNC42MjlhLjM0LjM0IDAgMCAwIC4wOTItLjEyNS4zMy4zMyAwIDAgMC0uMDI2LS4zLjMzLjMzIDAgMCAwLS4xMTItLjEwOGMtNC4xMTUtMi40OTctOC42NC00Ljc1LTEyLjM1My03LjQ0M2ExOTAgMTkwIDAgMCAxLTE4Ljc5NC0xNS41MjJjLTIuOTM2LTIuNzY4LTMuNzAzLTUuODkxLjU4LTcuNzdhLjUuNSAwIDAgMCAuMjgtLjYwOEM1MyA3NS45NSA0OC40MTcgNDguNTk5IDQ1LjUxOCAxOS42NXEtLjM5My0zLjg1Mi0uMjcxLTYuMzJhLjc1Ljc1IDAgMCAxIC4xMzQtLjQuNy43IDAgMCAxIC4zMjQtLjI1NXEyLjEwNC0uODQyIDUuODQ0LTEuNDMxYzU4LjExNC05LjEzNiAxMjcuMDA5IDguNDI1IDE0NC4zMDcgNzIuNjQ1cS4xMDMuMzY0LjE3OCAwIDUuMDk2LTI0LjA5NyA0LjYzOC00OC4xOTMtLjA0Ny0yLjc4NyAxLjg3OS00Ljc4OCAxMC40MDctMTAuOCAyNC43OTgtOC42ODdjNy40NzEgMS4wOTUgNy43NDIgNC42MzggNy4yMDkgMTEuMWEzMzQgMzM0IDAgMCAxLTExLjg4NCA2NC43MDYuMzg4LjM4OCAwIDAgMCAuMTgyLjQ0NC40LjQgMCAwIDAgLjE2NC4wNTFjOTIuNDIxIDguNDUzIDE1OC44NzUgNzkuNDg5IDE2NS4wODQgMTcxLjI3NGExIDEgMCAwIDAgLjEyOS40MjQuODMuODMgMCAwIDAgLjI5MS4yOTZjMy43NSAyLjE3IDYuMiAyLjIxNiA4Ljg1NSA2LjU5MiA1LjA2OCA4LjMzMiAxLjQ1IDIwLjQzMS04LjU5MyAyMi40N2ExLjcxIDEuNzEgMCAwIDAtMS4zNjUgMS40NjhxLTkuMTczIDc1LjM4My02NC43NjIgMTIzLjM3MWMtMjguNjIyIDI0LjcwNC02NS43NDQgMzkuNzQ5LTEwMy40NjQgNDIuMzMtMTAwLjM1OSA2Ljg2My0xODQuNDg2LTY1LjYyMi0xOTUuMDA1LTE2NS41N2EuOC44IDAgMCAwLS4xNTQtLjM5NS43NC43NCAwIDAgMC0uMzMzLS4yNU0xNTEuMzM4IDk3LjY0M2wxOS4wNjYgMTkuMzc1YS41LjUgMCAwIDAgLjE1LjEuNS41IDAgMCAwIC4xNzYuMDMxLjQzLjQzIDAgMCAwIC4zMTktLjE1cTEwLjg0Ny0xMi40NjUgNy40NDMtMjguMTU1Yy00LjUzNS0yMC44Ni0xOC41OTgtMzcuNjY0LTM3LjI3MS00OC4zODktMjEuMDY3LTEyLjY4OS00Ny44NjUtMTUuOTA1LTcyLjI4LTE0Ljg0cS0uNjE3LjAzLS4xNC40MjJsNjMuNDcyIDUzLjMyNmEuNC40IDAgMCAwIC4xNDQuMDcyLjM0LjM0IDAgMCAwIC4zMDEtLjA2OC4zLjMgMCAwIDAgLjA4OC0uMTI2IDQ3IDQ3IDAgMCAwIDMuMjgyLTEzLjA4MXEuMDIzLS4yMzMuMTI0LS40MzN0LjI2OC0uMzM0YzIuOTU1LTIuNDc4IDQuMTE1LjQ1OCA0LjY5NCAyLjg1MnEyLjMyIDkuNTM3LjE4NyAxOS4yNDRhMS40OCAxLjQ4IDAgMCAwIC40NCAxLjQwM3ptLTYuNTI2IDkuMjk1LTkuMDI0LTEwLjI3NmEuNy43IDAgMCAwLS4zNDMtLjIwNy44Ni44NiAwIDAgMC0uNDMzIDBxLTExLjY4IDMuMDIxLTIzLjExNC0uODIyYy01LjgxNi0zLjg1Mi0uNzU4LTUuNTE3IDIuNTktNS44MTZxNS45MTgtLjUzMyAxMS43MzUtMi41MDZhLjMuMyAwIDAgMCAuMTI3LS4wOC4zLjMgMCAwIDAgLjA3My0uMTMzLjM0LjM0IDAgMCAwIC4wMDEtLjE1NC4zNC4zNCAwIDAgMC0uMDctLjEzOGMtMTkuNDU5LTIxLjE1LTQwLjc2LTQwLjI0NS02Mi4yNzUtNTkuMjY0cS0uMzU2LS4zMTgtLjMxOC4xNWEzMDggMzA4IDAgMCAwIDguMzg3IDUxLjgwMmM0LjU5MiAxOC4zMDggMTMuNzU1IDM0LjE1NyAzMS44NDggNDEuODcycTI2LjI0NyAxMS4xODIgNTMuNzI5IDIuODMzYS41ODMuNTgzIDAgMCAwIC4yOTktLjg4OCAyMzUgMjM1IDAgMCAwLTEzLjIxMi0xNi4zNzNtNDUuMTI1IDQ5Ljc0NXEtMTIuNTk1IDEuMzE4LTIzLjQ0MS01LjU4M2MtMy4wNzctNC44OTkgMi4wNjYtOC41OTMgNS40NDItMTEuMTgzYS41ODQuNTg0IDAgMCAwIC4xODctLjcwMSA4LjQ2IDguNDYgMCAwIDAtMy40ODgtMy44NjIuODEuODEgMCAwIDAtLjgyMy4wMDljLTE2LjY5MSA5Ljk3Ny0zNC40ODUgMTAuNzgyLTUzLjE4NiA2Ljk2N3EtMS41Ny0uMzItMi44NTIuNjQ1LTYxLjQ1MSA0Ni4zNy02OS42NyAxMjMuOTEzLS42ODMgNi40MzMtMS44MDYgOC44MjdjLTIuOTQ1IDYuMzAyLTcuNzcgNy44NDUtMTQuMjIyIDkuNDcycS0uNDQ5LjExMyAwIC4yMzRjOS4yNzYgMi4zMSAxNC42OCA2LjE0MyAxNi4wNzQgMTYuNDAxIDguNDYyIDYyLjQ5IDQ4Ljk0IDExNy44NjQgMTEwLjM4MyAxMzguMzQxYTEwNCAxMDQgMCAwIDAgMTEuMDA2IDMuNTA3YzkyLjY0NSAyMy45MDkgMTgxLjIzMi0zMy42NjIgMjAyLjgwNC0xMjUuMDU0cTEuNjA5LTYuNzkgMy40NS0xOS4wODVjMS4xMDQtNy4zOTYgNS42OTUtMTIuMTE4IDEzLjE0Ny0xMy41NjguNDEyLS4wODQgMS45MDgtLjc1Ny4yODEtLjk5MS04LjA1MS0xLjE0MS0xMi4yNC02LjM4Ni0xMi45ODgtMTQuMTI4cS0xLjA3NS0xMS4yNS0yLjUxNi0xOS4wMzhjLTExLjc3Mi02My44OTItNTYuMzgzLTExNC41NzItMTE5LjI4NS0xMzEuNDZxLTE1LjEyLTMuODgtMzAuOTg3LTUuMjQ1YS4zODYuMzg2IDAgMCAwLS40MDIuMjUyIDE2OC42IDE2OC42IDAgMCAxLTExLjM1MiAyNC44NTRjLTQuMTMzIDcuNDA2LTguMTYzIDE1LjQ5NC0xNS43NTYgMTYuNDc2Ii8+PGNpcmNsZSBjeD0iMTQwLjA2NSIgY3k9IjI3My4wMyIgcj0iNTAuNjE4IiBzdHJva2U9IiNmZmYiIHN0cm9rZS13aWR0aD0iOS4yMDMiLz48cGF0aCBmaWxsPSIjZmZmIiBkPSJNMTQ2Ljk2NyAyNDIuMDkyYzE1LjY3MiAwIDI4LjM3NiAxMy45NjMgMjguMzc2IDMxLjE4OHMtMTIuNzA0IDMxLjE4OS0yOC4zNzYgMzEuMTg5LTI4LjM3Ny0xMy45NjQtMjguMzc3LTMxLjE4OWMwLTIuNjI1LjI5Ny01LjE3NC44NTItNy42MSAxLjYwNyAzLjcyNyA1LjMxMyA2LjMzOCA5LjYyOSA2LjMzOCA1Ljc4OSAwIDEwLjQ4MS00LjY5MyAxMC40ODItMTAuNDgycy00LjY5My0xMC40ODEtMTAuNDgyLTEwLjQ4MWMtLjc2MSAwLTEuNTA0LjA4My0yLjIxOS4yMzcgNS4xMzktNS42NzYgMTIuMjUzLTkuMTkgMjAuMTE1LTkuMTkiLz48Y2lyY2xlIGN4PSIyNjkuOTMyIiBjeT0iMjczLjAzIiByPSI1MC42MTgiIHN0cm9rZT0iI2ZmZiIgc3Ryb2tlLXdpZHRoPSI5LjIwMyIvPjxwYXRoIGZpbGw9IiNmZmYiIGQ9Ik0yNzYuODM0IDI0Mi4wOTJjMTUuNjcyIDAgMjguMzc2IDEzLjk2MyAyOC4zNzYgMzEuMTg4cy0xMi43MDQgMzEuMTg5LTI4LjM3NiAzMS4xODktMjguMzc3LTEzLjk2NC0yOC4zNzctMzEuMTg5YzAtMi42MjUuMjk3LTUuMTc0Ljg1My03LjYxIDEuNjA2IDMuNzI3IDUuMzEyIDYuMzM4IDkuNjI4IDYuMzM4IDUuNzg5IDAgMTAuNDgyLTQuNjkzIDEwLjQ4Mi0xMC40ODJzLTQuNjkzLTEwLjQ4MS0xMC40ODItMTAuNDgxYy0uNzYxIDAtMS41MDQuMDgzLTIuMjE5LjIzNyA1LjEzOS01LjY3NiAxMi4yNTQtOS4xOSAyMC4xMTUtOS4xOSIvPjxwYXRoIHN0cm9rZT0iI2ZmZiIgc3Ryb2tlLXdpZHRoPSI5LjIwMyIgZD0iTTE3Ny4xMzUgMzQ1LjMyOGMuODk5LS4yNCAyLjgwMS0uMTk3IDYuMzA0LjQ2NCA2LjMxMSAxLjE5IDE2LjU4NSA0LjIyMSAyNi4xMjIgNC4yMjEgOS40NTIgMCAxNy4xMTgtMS45MjYgMjEuMzgzLTIuNjAzLjc3LS4xMjMgMS4zODUtLjE5OSAxLjg3OC0uMjM3LS4zNCAxMy44NDMtMTIuODkzIDI3LjcxLTI3Ljg2OCAyNy43MS0xNS4xNzIgMC0yNy44NzYtMTMuODM1LTI3Ljg3Ni0yOC44MTEgMC0uMzU3LjAyOS0uNTk2LjA1Ny0uNzQ0WiIvPjwvc3ZnPg==" alt="Performance Models">
|
| 43 |
</a>
|
| 44 |
</p>
|
|
|
|
| 45 |
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
|
| 50 |
-
|
| 51 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
|
|
|
|
| 53 |
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
`[Qwen/Qwen-Image-2.1](https://huggingface.co/Qwen/Qwen-Image-2.1)` **generate
|
| 58 |
-
and edit images in only 6 steps. The adapters load on top of the base
|
| 59 |
model, so the pipeline, text encoder, and VAE stay unchanged. Training is based
|
| 60 |
on DMD. Improved using Qwen.**
|
| 61 |
|
| 62 |
> **v0.2: two new 6-step adapters.** A basic distillation for faithful generation
|
| 63 |
> and editing, and a first attempt at distilling prompt upsampling into the LoRA.
|
| 64 |
-
> The
|
| 65 |
-
|
| 66 |
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
## Variants
|
| 69 |
|
|
|
|
|
|
|
| 70 |
Pick **basic** for faithful base-model behavior, or **prompt-upsampled** for
|
| 71 |
improved visual quality from a short prompt.
|
| 72 |
|
| 73 |
-
|
| 74 |
-
|
|
| 75 |
-
|
|
| 76 |
-
|
|
| 77 |
-
| **Prompt-upsampled v0.2** | `p_q_image_2.1_upsampled_6_steps_v0.2.safetensors` | 6 | **Improved visual quality.** A first attempt at distilling prompt upsampling into the adapter. |
|
| 78 |
-
| v0.1, 8-step | `p_q_image_2.1_8step_v0.1.safetensors` | 8 | **Higher quality** of the two original adapters. |
|
| 79 |
-
| v0.1, 5-step | `p_q_image_2.1_5step_v0.1.safetensors` | 5 | **Higher speed**, with noticeably lower visual quality. |
|
| 80 |
-
|
| 81 |
|
| 82 |
**Basic** is a pure 6-step distillation of the base model, without any additional
|
| 83 |
post-training. It is intended as a drop-in replacement for the base model in
|
|
@@ -90,11 +96,22 @@ Qwen-Image-2.1 into a 6-step LoRA. It improves visual quality without a separate
|
|
| 90 |
prompt-upsampling call. This is our first attempt: it may not reproduce exactly
|
| 91 |
what the actual prompt upsampler would do, and we will continue improving it.
|
| 92 |
|
| 93 |
-
|
| 94 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
|
| 96 |
## Prompts, editing, and resolution
|
| 97 |
|
|
|
|
|
|
|
| 98 |
For **basic**, use the same prompts and reference images you would use with the
|
| 99 |
base model. For **prompt-upsampled**, start with your original prompt; the adapter
|
| 100 |
is intended to capture the benefit of prompt upsampling during generation.
|
|
@@ -102,35 +119,33 @@ Detailed prompts still let you specify the subject, setting, lighting, and style
|
|
| 102 |
|
| 103 |
The adapters were trained at **1K resolution only**, with a mix of simple and
|
| 104 |
upsampled prompts, text-to-image generation, and single- and multi-image editing
|
| 105 |
-
with **up to 3 reference images**.
|
| 106 |
-
|
| 107 |
-
images are outside their training coverage, and quality may vary.
|
| 108 |
|
| 109 |
-
|
| 110 |
-
|
|
|
|
|
|
|
| 111 |
|
| 112 |
## Benchmarking
|
| 113 |
|
| 114 |
-
|
| 115 |
-
The 6-step adapter and separate prompt-upsampling measurements are reported
|
| 116 |
-
separately below.
|
| 117 |
-
|
| 118 |
|
| 119 |
<table>
|
| 120 |
<tr>
|
| 121 |
<td width="50%">
|
| 122 |
-
<img src="assets/Pruna-Q-Image-2.1-t2i-1024.png"
|
| 123 |
</td>
|
| 124 |
<td width="50%">
|
| 125 |
-
<img src="assets/Pruna-Q-Image-2.1-t2i-2048.png"
|
| 126 |
</td>
|
| 127 |
</tr>
|
| 128 |
<tr>
|
| 129 |
<td width="50%">
|
| 130 |
-
<img src="assets/Pruna-Q-Image-2.1-i2i-1024.png"
|
| 131 |
</td>
|
| 132 |
<td width="50%">
|
| 133 |
-
<img src="assets/Pruna-Q-Image-2.1-i2i-2048.png"
|
| 134 |
</td>
|
| 135 |
</tr>
|
| 136 |
</table>
|
|
@@ -138,12 +153,12 @@ separately below.
|
|
| 138 |
Text-to-image with the official Qwen model-card example prompt, BF16, batch size 1,
|
| 139 |
on one NVIDIA H100 80GB. Median of 3 requests after one warmup per configuration.
|
| 140 |
Includes prompt encoding, denoising, and decoding; excludes PNG saving, model
|
| 141 |
-
loading, and warmup. Base: 40 steps with KV cache on;
|
| 142 |
-
KV cache off, as configured for this benchmark. The examples below
|
|
|
|
| 143 |
no CFG, compilation, or CPU offload. These timings do not imply equal image quality.
|
| 144 |
|
| 145 |
-
### 6-step LoRA and prompt upsampling
|
| 146 |
-
|
| 147 |
|
| 148 |
<table>
|
| 149 |
<tr>
|
|
@@ -166,15 +181,15 @@ processing, denoising, and VAE decoding to a PIL image. It excludes model and
|
|
| 166 |
adapter loading, warmup, fixture preparation, file saving, and network transport.
|
| 167 |
No CFG, compilation, CPU offload, or diffusion KV caching.
|
| 168 |
|
| 169 |
-
Prompt upsampling is measured separately using
|
| 170 |
-
|
| 171 |
-
and
|
| 172 |
-
with their task-specific sampling settings and thinking enabled. Timing includes
|
| 173 |
-
request preparation from in-memory images, image and text processing, text
|
| 174 |
-
generation, and response parsing. It does not include image generation. The
|
| 175 |
-
upsampler uses vLLM in eager mode, without prefix or image-processor caching.
|
| 176 |
-
Each request receives fresh image identifiers, so vision encoding is included
|
| 177 |
-
on repeated requests too. Autoregressive KV caching remains enabled; it is
|
| 178 |
separate from diffusion KV caching.
|
| 179 |
|
| 180 |
Each editing case uses a fixed original prompt and 1, 2, or 3 reference images,
|
|
@@ -182,7 +197,7 @@ square-cropped and resized to the requested resolution before timing.
|
|
| 182 |
Text-to-image uses the official Qwen model-card example prompt. Latency depends
|
| 183 |
on the prompt, image
|
| 184 |
content, and generated rewrite length. These cases are not a controlled
|
| 185 |
-
comparison with the
|
| 186 |
measure an end-to-end prompt-upsampling-plus-generation request.
|
| 187 |
|
| 188 |
Software and prompt-upsampling settings
|
|
@@ -191,7 +206,7 @@ Generation uses PyTorch 2.11.0 with CUDA 12.8, Transformers 5.20.1, PEFT 0.21.2,
|
|
| 191 |
and Diffusers commit `6256aa7666cedd47443adc8f82da9a10e110b09c`, the same Diffusers
|
| 192 |
revision used in the quickstart. Prompt upsampling uses vLLM 0.19.1, PyTorch
|
| 193 |
2.10.0, and Transformers 5.19.0. These runtime versions are recorded for the new
|
| 194 |
-
measurements; the
|
| 195 |
|
| 196 |
Both prompt enhancers use their checkpoint's system prompt, thinking enabled,
|
| 197 |
`temperature=1.0`, `top_p=0.95`, `top_k=20`, and `min_p=0.0`. Text-to-image uses
|
|
@@ -209,6 +224,42 @@ pip install 'torch>=2.4.0' 'transformers>=5.17' accelerate peft pillow
|
|
| 209 |
pip install git+https://github.com/huggingface/diffusers@6256aa7666cedd47443adc8f82da9a10e110b09c
|
| 210 |
```
|
| 211 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 212 |
```python
|
| 213 |
import torch
|
| 214 |
from PIL import Image
|
|
@@ -241,7 +292,6 @@ pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
|
|
| 241 |
```
|
| 242 |
|
| 243 |
|
| 244 |
-
|
| 245 |
Both variants load on **`Qwen/Qwen-Image-2.1`** using **`QwenImage21Pipeline`**.
|
| 246 |
Choose one LoRA with `VARIANT`; the prompt-upsampled variant runs directly on your
|
| 247 |
original prompt without a separate prompt-enhancer model. For a local export,
|
|
@@ -252,35 +302,12 @@ Both files have **rank 64** across **232 linear targets**, including `proj_out`.
|
|
| 252 |
Training used alpha 128; the exported B matrices include the `128 / 64 = 2`
|
| 253 |
scale. Load at **LoRA strength 1.0** with the standard pipeline loader.
|
| 254 |
|
| 255 |
-
|
| 256 |
-
|
| 257 |
-
To use an original adapter, replace the `STEPS`, `WEIGHT_NAME`, and `SIGMAS`
|
| 258 |
-
settings above with the following and reset the scheduler as shown. The v0.1
|
| 259 |
-
sigmas are already shifted, so dynamic shifting must be disabled for them.
|
| 260 |
-
|
| 261 |
-
```python
|
| 262 |
-
STEPS = 8 # 8 for higher quality, 5 for higher speed
|
| 263 |
-
WEIGHT_NAME = f"p_q_image_2.1_{STEPS}step_v0.1.safetensors"
|
| 264 |
-
SIGMAS = {
|
| 265 |
-
5: [1.0, 0.94, 6 / 7, 2 / 3, 0.4],
|
| 266 |
-
8: [1.0, 14 / 15, 6 / 7, 10 / 13, 2 / 3, 6 / 11, 0.4, 2 / 9],
|
| 267 |
-
}[STEPS]
|
| 268 |
-
|
| 269 |
-
pipe.unload_lora_weights()
|
| 270 |
-
pipe.load_lora_weights("PrunaAI/Pruna-Q-Image-2.1", weight_name=WEIGHT_NAME)
|
| 271 |
-
pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
|
| 272 |
-
pipe.scheduler.config,
|
| 273 |
-
use_dynamic_shifting=False,
|
| 274 |
-
shift=1.0,
|
| 275 |
-
shift_terminal=None,
|
| 276 |
-
)
|
| 277 |
-
```
|
| 278 |
-
|
| 279 |
-
|
| 280 |
|
| 281 |
## For Text-to-Image
|
| 282 |
|
| 283 |
-
|
| 284 |
|
| 285 |
```python
|
| 286 |
image = pipe(
|
|
@@ -291,73 +318,73 @@ image = pipe(
|
|
| 291 |
),
|
| 292 |
width=1024,
|
| 293 |
height=1024,
|
| 294 |
-
output_resolution=1024,
|
| 295 |
generator=torch.Generator("cuda").manual_seed(42),
|
| 296 |
num_inference_steps=STEPS,
|
| 297 |
sigmas=SIGMAS,
|
| 298 |
true_cfg_scale=1.0,
|
| 299 |
-
use_kv_cache=
|
| 300 |
).images[0]
|
| 301 |
```
|
| 302 |
|
| 303 |
-
|
| 304 |
-
|
| 305 |
### For Image Editing
|
| 306 |
|
| 307 |
```python
|
| 308 |
image = pipe(
|
| 309 |
prompt="Change the background to a sunset beach",
|
| 310 |
image=Image.open("input.png").convert("RGB"),
|
| 311 |
-
output_resolution=1024,
|
| 312 |
generator=torch.Generator("cuda").manual_seed(42),
|
| 313 |
num_inference_steps=STEPS,
|
| 314 |
sigmas=SIGMAS,
|
| 315 |
true_cfg_scale=1.0,
|
| 316 |
-
use_kv_cache=
|
| 317 |
).images[0]
|
| 318 |
```
|
| 319 |
|
|
|
|
| 320 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 321 |
|
| 322 |
-
|
| 323 |
|
| 324 |
-
- **Use the sigma schedule
|
| 325 |
-
|
| 326 |
-
|
| 327 |
-
|
| 328 |
-
-
|
| 329 |
-
- v0.1, 5-step: `1 β 0.94 β 6/7 β 2/3 β 0.4 β 0`.
|
| 330 |
-
These v0.1 schedules are already shifted; use dynamic shifting off and `shift=1.0`.
|
| 331 |
-
- The scheduler appends terminal zero for all variants. Pass each schedule
|
| 332 |
-
without that final zero.
|
| 333 |
- **No CFG.** Keep `true_cfg_scale=1.0` and do not pass a negative prompt.
|
| 334 |
- **Keep the LoRA strength at 1.0.**
|
| 335 |
-
- **
|
| 336 |
-
|
| 337 |
-
|
| 338 |
-
|
| 339 |
-
|
| 340 |
|
| 341 |
## Limitations
|
| 342 |
|
|
|
|
|
|
|
| 343 |
- The prompt-upsampled adapter is a first attempt. It can improve visual quality,
|
| 344 |
-
but may not exactly reproduce the output of the actual prompt upsampler.
|
| 345 |
- Basic is intended to preserve the base model's behavior in fewer steps;
|
| 346 |
-
distillation does not guarantee identical outputs.
|
| 347 |
-
-
|
| 348 |
-
|
| 349 |
-
- Use the step count and sigma schedule that match your adapter. Other step
|
| 350 |
-
counts, schedules, or CFG values are not supported.
|
| 351 |
|
|
|
|
| 352 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 353 |
|
| 354 |
## What this is not
|
| 355 |
|
| 356 |
- Not a standalone model. It needs the `Qwen/Qwen-Image-2.1` base weights.
|
| 357 |
-
- Not
|
| 358 |
-
- Not a finished release. Expect
|
| 359 |
-
|
| 360 |
-
|
| 361 |
|
| 362 |
## License
|
| 363 |
|
|
@@ -376,4 +403,32 @@ Hangzhou Tongyi Laboratory Technology Co., Ltd. All Rights Reserved.
|
|
| 376 |
- Compress your own models with [Pruna](https://github.com/PrunaAI/pruna) and give us a βοΈ for more efficiency!
|
| 377 |
- Want to use our optimized image models right away? Check [P-Image-Ideogram](https://www.pruna.ai/p-image-ideogram) and [P-Image-Edit](https://www.pruna.ai/p-image-edit).
|
| 378 |
|
| 379 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
library_name: diffusers
|
| 6 |
pipeline_tag: text-to-image
|
| 7 |
tags:
|
| 8 |
+
- qwen
|
| 9 |
+
- image-generation
|
| 10 |
+
- image-editing
|
| 11 |
+
- rgba
|
| 12 |
+
- lora
|
| 13 |
+
- distilled
|
| 14 |
+
- few-step
|
| 15 |
---
|
|
|
|
| 16 |
<!-- header start -->
|
| 17 |
<!-- 200823 -->
|
| 18 |
<p align="center">
|
|
|
|
| 41 |
<img src="https://img.shields.io/badge/Performance%20Models-Try%20them%20now-9334E9?style=plastic&logo=data:image/svg%2bxml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI0MTEiIGhlaWdodD0iNDc3IiBmaWxsPSJub25lIj48cGF0aCBmaWxsPSIjZmZmIiBkPSJNMjMuNzAzIDMwMC41MzJjLTEuODQyLS42NzQtNC4xNDItMS4wNTctNS43MDMtMi4wMjktMTIuNTU4LTcuODQ2LTguODM3LTI1LjE3MiA1LjI1NS0yOC4zNjFhLjY2LjY2IDAgMCAwIC4zNDgtLjIwMS42LjYgMCAwIDAgLjE1Ni0uMzZxNi40OS04MS45ODYgNjguNTc3LTEzNC42MjlhLjM0LjM0IDAgMCAwIC4wOTItLjEyNS4zMy4zMyAwIDAgMC0uMDI2LS4zLjMzLjMzIDAgMCAwLS4xMTItLjEwOGMtNC4xMTUtMi40OTctOC42NC00Ljc1LTEyLjM1My03LjQ0M2ExOTAgMTkwIDAgMCAxLTE4Ljc5NC0xNS41MjJjLTIuOTM2LTIuNzY4LTMuNzAzLTUuODkxLjU4LTcuNzdhLjUuNSAwIDAgMCAuMjgtLjYwOEM1MyA3NS45NSA0OC40MTcgNDguNTk5IDQ1LjUxOCAxOS42NXEtLjM5My0zLjg1Mi0uMjcxLTYuMzJhLjc1Ljc1IDAgMCAxIC4xMzQtLjQuNy43IDAgMCAxIC4zMjQtLjI1NXEyLjEwNC0uODQyIDUuODQ0LTEuNDMxYzU4LjExNC05LjEzNiAxMjcuMDA5IDguNDI1IDE0NC4zMDcgNzIuNjQ1cS4xMDMuMzY0LjE3OCAwIDUuMDk2LTI0LjA5NyA0LjYzOC00OC4xOTMtLjA0Ny0yLjc4NyAxLjg3OS00Ljc4OCAxMC40MDctMTAuOCAyNC43OTgtOC42ODdjNy40NzEgMS4wOTUgNy43NDIgNC42MzggNy4yMDkgMTEuMWEzMzQgMzM0IDAgMCAxLTExLjg4NCA2NC43MDYuMzg4LjM4OCAwIDAgMCAuMTgyLjQ0NC40LjQgMCAwIDAgLjE2NC4wNTFjOTIuNDIxIDguNDUzIDE1OC44NzUgNzkuNDg5IDE2NS4wODQgMTcxLjI3NGExIDEgMCAwIDAgLjEyOS40MjQuODMuODMgMCAwIDAgLjI5MS4yOTZjMy43NSAyLjE3IDYuMiAyLjIxNiA4Ljg1NSA2LjU5MiA1LjA2OCA4LjMzMiAxLjQ1IDIwLjQzMS04LjU5MyAyMi40N2ExLjcxIDEuNzEgMCAwIDAtMS4zNjUgMS40NjhxLTkuMTczIDc1LjM4My02NC43NjIgMTIzLjM3MWMtMjguNjIyIDI0LjcwNC02NS43NDQgMzkuNzQ5LTEwMy40NjQgNDIuMzMtMTAwLjM1OSA2Ljg2My0xODQuNDg2LTY1LjYyMi0xOTUuMDA1LTE2NS41N2EuOC44IDAgMCAwLS4xNTQtLjM5NS43NC43NCAwIDAgMC0uMzMzLS4yNU0xNTEuMzM4IDk3LjY0M2wxOS4wNjYgMTkuMzc1YS41LjUgMCAwIDAgLjE1LjEuNS41IDAgMCAwIC4xNzYuMDMxLjQzLjQzIDAgMCAwIC4zMTktLjE1cTEwLjg0Ny0xMi40NjUgNy40NDMtMjguMTU1Yy00LjUzNS0yMC44Ni0xOC41OTgtMzcuNjY0LTM3LjI3MS00OC4zODktMjEuMDY3LTEyLjY4OS00Ny44NjUtMTUuOTA1LTcyLjI4LTE0Ljg0cS0uNjE3LjAzLS4xNC40MjJsNjMuNDcyIDUzLjMyNmEuNC40IDAgMCAwIC4xNDQuMDcyLjM0LjM0IDAgMCAwIC4zMDEtLjA2OC4zLjMgMCAwIDAgLjA4OC0uMTI2IDQ3IDQ3IDAgMCAwIDMuMjgyLTEzLjA4MXEuMDIzLS4yMzMuMTI0LS40MzN0LjI2OC0uMzM0YzIuOTU1LTIuNDc4IDQuMTE1LjQ1OCA0LjY5NCAyLjg1MnEyLjMyIDkuNTM3LjE4NyAxOS4yNDRhMS40OCAxLjQ4IDAgMCAwIC40NCAxLjQwM3ptLTYuNTI2IDkuMjk1LTkuMDI0LTEwLjI3NmEuNy43IDAgMCAwLS4zNDMtLjIwNy44Ni44NiAwIDAgMC0uNDMzIDBxLTExLjY4IDMuMDIxLTIzLjExNC0uODIyYy01LjgxNi0zLjg1Mi0uNzU4LTUuNTE3IDIuNTktNS44MTZxNS45MTgtLjUzMyAxMS43MzUtMi41MDZhLjMuMyAwIDAgMCAuMTI3LS4wOC4zLjMgMCAwIDAgLjA3My0uMTMzLjM0LjM0IDAgMCAwIC4wMDEtLjE1NC4zNC4zNCAwIDAgMC0uMDctLjEzOGMtMTkuNDU5LTIxLjE1LTQwLjc2LTQwLjI0NS02Mi4yNzUtNTkuMjY0cS0uMzU2LS4zMTgtLjMxOC4xNWEzMDggMzA4IDAgMCAwIDguMzg3IDUxLjgwMmM0LjU5MiAxOC4zMDggMTMuNzU1IDM0LjE1NyAzMS44NDggNDEuODcycTI2LjI0NyAxMS4xODIgNTMuNzI5IDIuODMzYS41ODMuNTgzIDAgMCAwIC4yOTktLjg4OCAyMzUgMjM1IDAgMCAwLTEzLjIxMi0xNi4zNzNtNDUuMTI1IDQ5Ljc0NXEtMTIuNTk1IDEuMzE4LTIzLjQ0MS01LjU4M2MtMy4wNzctNC44OTkgMi4wNjYtOC41OTMgNS40NDItMTEuMTgzYS41ODQuNTg0IDAgMCAwIC4xODctLjcwMSA4LjQ2IDguNDYgMCAwIDAtMy40ODgtMy44NjIuODEuODEgMCAwIDAtLjgyMy4wMDljLTE2LjY5MSA5Ljk3Ny0zNC40ODUgMTAuNzgyLTUzLjE4NiA2Ljk2N3EtMS41Ny0uMzItMi44NTIuNjQ1LTYxLjQ1MSA0Ni4zNy02OS42NyAxMjMuOTEzLS42ODMgNi40MzMtMS44MDYgOC44MjdjLTIuOTQ1IDYuMzAyLTcuNzcgNy44NDUtMTQuMjIyIDkuNDcycS0uNDQ5LjExMyAwIC4yMzRjOS4yNzYgMi4zMSAxNC42OCA2LjE0MyAxNi4wNzQgMTYuNDAxIDguNDYyIDYyLjQ5IDQ4Ljk0IDExNy44NjQgMTEwLjM4MyAxMzguMzQxYTEwNCAxMDQgMCAwIDAgMTEuMDA2IDMuNTA3YzkyLjY0NSAyMy45MDkgMTgxLjIzMi0zMy42NjIgMjAyLjgwNC0xMjUuMDU0cTEuNjA5LTYuNzkgMy40NS0xOS4wODVjMS4xMDQtNy4zOTYgNS42OTUtMTIuMTE4IDEzLjE0Ny0xMy41NjguNDEyLS4wODQgMS45MDgtLjc1Ny4yODEtLjk5MS04LjA1MS0xLjE0MS0xMi4yNC02LjM4Ni0xMi45ODgtMTQuMTI4cS0xLjA3NS0xMS4yNS0yLjUxNi0xOS4wMzhjLTExLjc3Mi02My44OTItNTYuMzgzLTExNC41NzItMTE5LjI4NS0xMzEuNDZxLTE1LjEyLTMuODgtMzAuOTg3LTUuMjQ1YS4zODYuMzg2IDAgMCAwLS40MDIuMjUyIDE2OC42IDE2OC42IDAgMCAxLTExLjM1MiAyNC44NTRjLTQuMTMzIDcuNDA2LTguMTYzIDE1LjQ5NC0xNS43NTYgMTYuNDc2Ii8+PGNpcmNsZSBjeD0iMTQwLjA2NSIgY3k9IjI3My4wMyIgcj0iNTAuNjE4IiBzdHJva2U9IiNmZmYiIHN0cm9rZS13aWR0aD0iOS4yMDMiLz48cGF0aCBmaWxsPSIjZmZmIiBkPSJNMTQ2Ljk2NyAyNDIuMDkyYzE1LjY3MiAwIDI4LjM3NiAxMy45NjMgMjguMzc2IDMxLjE4OHMtMTIuNzA0IDMxLjE4OS0yOC4zNzYgMzEuMTg5LTI4LjM3Ny0xMy45NjQtMjguMzc3LTMxLjE4OWMwLTIuNjI1LjI5Ny01LjE3NC44NTItNy42MSAxLjYwNyAzLjcyNyA1LjMxMyA2LjMzOCA5LjYyOSA2LjMzOCA1Ljc4OSAwIDEwLjQ4MS00LjY5MyAxMC40ODItMTAuNDgycy00LjY5My0xMC40ODEtMTAuNDgyLTEwLjQ4MWMtLjc2MSAwLTEuNTA0LjA4My0yLjIxOS4yMzcgNS4xMzktNS42NzYgMTIuMjUzLTkuMTkgMjAuMTE1LTkuMTkiLz48Y2lyY2xlIGN4PSIyNjkuOTMyIiBjeT0iMjczLjAzIiByPSI1MC42MTgiIHN0cm9rZT0iI2ZmZiIgc3Ryb2tlLXdpZHRoPSI5LjIwMyIvPjxwYXRoIGZpbGw9IiNmZmYiIGQ9Ik0yNzYuODM0IDI0Mi4wOTJjMTUuNjcyIDAgMjguMzc2IDEzLjk2MyAyOC4zNzYgMzEuMTg4cy0xMi43MDQgMzEuMTg5LTI4LjM3NiAzMS4xODktMjguMzc3LTEzLjk2NC0yOC4zNzctMzEuMTg5YzAtMi42MjUuMjk3LTUuMTc0Ljg1My03LjYxIDEuNjA2IDMuNzI3IDUuMzEyIDYuMzM4IDkuNjI4IDYuMzM4IDUuNzg5IDAgMTAuNDgyLTQuNjkzIDEwLjQ4Mi0xMC40ODJzLTQuNjkzLTEwLjQ4MS0xMC40ODItMTAuNDgxYy0uNzYxIDAtMS41MDQuMDgzLTIuMjE5LjIzNyA1LjEzOS01LjY3NiAxMi4yNTQtOS4xOSAyMC4xMTUtOS4xOSIvPjxwYXRoIHN0cm9rZT0iI2ZmZiIgc3Ryb2tlLXdpZHRoPSI5LjIwMyIgZD0iTTE3Ny4xMzUgMzQ1LjMyOGMuODk5LS4yNCAyLjgwMS0uMTk3IDYuMzA0LjQ2NCA2LjMxMSAxLjE5IDE2LjU4NSA0LjIyMSAyNi4xMjIgNC4yMjEgOS40NTIgMCAxNy4xMTgtMS45MjYgMjEuMzgzLTIuNjAzLjc3LS4xMjMgMS4zODUtLjE5OSAxLjg3OC0uMjM3LS4zNCAxMy44NDMtMTIuODkzIDI3LjcxLTI3Ljg2OCAyNy43MS0xNS4xNzIgMC0yNy44NzYtMTMuODM1LTI3Ljg3Ni0yOC44MTEgMC0uMzU3LjAyOS0uNTk2LjA1Ny0uNzQ0WiIvPjwvc3ZnPg==" alt="Performance Models">
|
| 42 |
</a>
|
| 43 |
</p>
|
| 44 |
+
<div align="center">
|
| 45 |
|
| 46 |
+
<h1 style="color: #9334E9;">β‘ Pruna-Q-Image-2.1</h1>
|
|
|
|
|
|
|
| 47 |
|
| 48 |
+
<h2>Few-step LoRA adapters for Qwen-Image-2.1</h2>
|
| 49 |
|
| 50 |
+
<h3>
|
| 51 |
+
<span style="color: #9334E9;">6 steps (v0.2), 5 or 8 steps (v0.1)</span>
|
| 52 |
+
Β·
|
| 53 |
+
<span style="color: #9334E9;">Up to 6.3x faster</span>
|
| 54 |
+
Β·
|
| 55 |
+
<span style="color: #9334E9;">No CFG</span>
|
| 56 |
+
Β·
|
| 57 |
+
Text-to-image and image editing
|
| 58 |
+
</h3>
|
| 59 |
|
| 60 |
+
</div>
|
| 61 |
|
| 62 |
+
**Pruna-Q-Image-2.1 is a set of LoRA adapters that let
|
| 63 |
+
[`Qwen/Qwen-Image-2.1`](https://huggingface.co/Qwen/Qwen-Image-2.1) generate
|
| 64 |
+
and edit images in only 6 steps (v0.2), or 5 or 8 steps (v0.1). The adapters load on top of the base
|
|
|
|
|
|
|
| 65 |
model, so the pipeline, text encoder, and VAE stay unchanged. Training is based
|
| 66 |
on DMD. Improved using Qwen.**
|
| 67 |
|
| 68 |
> **v0.2: two new 6-step adapters.** A basic distillation for faithful generation
|
| 69 |
> and editing, and a first attempt at distilling prompt upsampling into the LoRA.
|
| 70 |
+
> The v0.1 adapters remain available.
|
|
|
|
| 71 |
|
| 72 |
+
> **v0.1: first release, work in progress.** Pruna-Q-Image-2.1 does not yet match the
|
| 73 |
+
> visual quality of the base model. We are still improving the distillation and
|
| 74 |
+
> will update this repository as new versions become available.
|
| 75 |
|
| 76 |
## Variants
|
| 77 |
|
| 78 |
+
### v0.2
|
| 79 |
+
|
| 80 |
Pick **basic** for faithful base-model behavior, or **prompt-upsampled** for
|
| 81 |
improved visual quality from a short prompt.
|
| 82 |
|
| 83 |
+
| File | Steps | Trade-off |
|
| 84 |
+
|---|:---:|---|
|
| 85 |
+
| `p_q_image_2.1_basic_6_steps_v0.2.safetensors` | 6 | **Pure few-step distillation.** Recommended for preserving inputs when editing. |
|
| 86 |
+
| `p_q_image_2.1_upsampled_6_steps_v0.2.safetensors` | 6 | **Improved visual quality.** A first attempt at distilling prompt upsampling into the adapter. |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
|
| 88 |
**Basic** is a pure 6-step distillation of the base model, without any additional
|
| 89 |
post-training. It is intended as a drop-in replacement for the base model in
|
|
|
|
| 96 |
prompt-upsampling call. This is our first attempt: it may not reproduce exactly
|
| 97 |
what the actual prompt upsampler would do, and we will continue improving it.
|
| 98 |
|
| 99 |
+
### v0.1
|
| 100 |
+
|
| 101 |
+
Both adapters are v0.1. Pick one based on whether you need **quality** or **speed**.
|
| 102 |
+
|
| 103 |
+
| File | Steps | Trade-off |
|
| 104 |
+
|---|:---:|---|
|
| 105 |
+
| `p_q_image_2.1_8step_v0.1.safetensors` | 8 | **Higher quality.** Recommended as the default. |
|
| 106 |
+
| `p_q_image_2.1_5step_v0.1.safetensors` | 5 | **Higher speed**, with noticeably lower visual quality. |
|
| 107 |
+
|
| 108 |
+
Each adapter is trained for its own sigma schedule. Load only one at a time.
|
| 109 |
+
Do not reuse the v0.1 schedules for the v0.2 adapters.
|
| 110 |
|
| 111 |
## Prompts, editing, and resolution
|
| 112 |
|
| 113 |
+
For **v0.2**:
|
| 114 |
+
|
| 115 |
For **basic**, use the same prompts and reference images you would use with the
|
| 116 |
base model. For **prompt-upsampled**, start with your original prompt; the adapter
|
| 117 |
is intended to capture the benefit of prompt upsampling during generation.
|
|
|
|
| 119 |
|
| 120 |
The adapters were trained at **1K resolution only**, with a mix of simple and
|
| 121 |
upsampled prompts, text-to-image generation, and single- and multi-image editing
|
| 122 |
+
with **up to 3 reference images**. Prompt upsampling is optional; detailed prompts
|
| 123 |
+
usually give better results.
|
|
|
|
| 124 |
|
| 125 |
+
Start at **1024 Γ 1024** and use at most **3 reference images** for editing.
|
| 126 |
+
Higher resolutions (including 2K) and more reference images may work, but are
|
| 127 |
+
outside the training coverage, and quality may vary. The 2K timings below show
|
| 128 |
+
inference speed, not a guarantee of quality at that resolution.
|
| 129 |
|
| 130 |
## Benchmarking
|
| 131 |
|
| 132 |
+
Use a table; Hugging Face can otherwise render the `<img>` elements as separate blocks.
|
|
|
|
|
|
|
|
|
|
| 133 |
|
| 134 |
<table>
|
| 135 |
<tr>
|
| 136 |
<td width="50%">
|
| 137 |
+
<img src="assets/Pruna-Q-Image-2.1-t2i-1024.png" width="100%">
|
| 138 |
</td>
|
| 139 |
<td width="50%">
|
| 140 |
+
<img src="assets/Pruna-Q-Image-2.1-t2i-2048.png" width="100%">
|
| 141 |
</td>
|
| 142 |
</tr>
|
| 143 |
<tr>
|
| 144 |
<td width="50%">
|
| 145 |
+
<img src="assets/Pruna-Q-Image-2.1-i2i-1024.png" width="100%">
|
| 146 |
</td>
|
| 147 |
<td width="50%">
|
| 148 |
+
<img src="assets/Pruna-Q-Image-2.1-i2i-2048.png" width="100%">
|
| 149 |
</td>
|
| 150 |
</tr>
|
| 151 |
</table>
|
|
|
|
| 153 |
Text-to-image with the official Qwen model-card example prompt, BF16, batch size 1,
|
| 154 |
on one NVIDIA H100 80GB. Median of 3 requests after one warmup per configuration.
|
| 155 |
Includes prompt encoding, denoising, and decoding; excludes PNG saving, model
|
| 156 |
+
loading, and warmup. Base: 40 steps with KV cache on; Pruna-Q-Image-2.1: 5 or 8 steps with
|
| 157 |
+
KV cache off, as configured for this benchmark. The examples below enable KV caching for
|
| 158 |
+
Pruna-Q-Image-2.1; the chart has not been remeasured with that setting. LoRAs were unmerged;
|
| 159 |
no CFG, compilation, or CPU offload. These timings do not imply equal image quality.
|
| 160 |
|
| 161 |
+
### v0.2: 6-step LoRA and prompt upsampling
|
|
|
|
| 162 |
|
| 163 |
<table>
|
| 164 |
<tr>
|
|
|
|
| 181 |
adapter loading, warmup, fixture preparation, file saving, and network transport.
|
| 182 |
No CFG, compilation, CPU offload, or diffusion KV caching.
|
| 183 |
|
| 184 |
+
Prompt upsampling is measured separately using
|
| 185 |
+
[`Qwen-Image-2.1-PE-T2I`](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I)
|
| 186 |
+
and [`Qwen-Image-2.1-PE-I2I`](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-I2I),
|
| 187 |
+
with their task-specific sampling settings and thinking enabled. Timing includes
|
| 188 |
+
request preparation from in-memory images, image and text processing, text
|
| 189 |
+
generation, and response parsing. It does not include image generation. The
|
| 190 |
+
upsampler uses vLLM in eager mode, without prefix or image-processor caching.
|
| 191 |
+
Each request receives fresh image identifiers, so vision encoding is included
|
| 192 |
+
on repeated requests too. Autoregressive KV caching remains enabled; it is
|
| 193 |
separate from diffusion KV caching.
|
| 194 |
|
| 195 |
Each editing case uses a fixed original prompt and 1, 2, or 3 reference images,
|
|
|
|
| 197 |
Text-to-image uses the official Qwen model-card example prompt. Latency depends
|
| 198 |
on the prompt, image
|
| 199 |
content, and generated rewrite length. These cases are not a controlled
|
| 200 |
+
comparison with the v0.1 chart inputs. Adding the two medians does not
|
| 201 |
measure an end-to-end prompt-upsampling-plus-generation request.
|
| 202 |
|
| 203 |
Software and prompt-upsampling settings
|
|
|
|
| 206 |
and Diffusers commit `6256aa7666cedd47443adc8f82da9a10e110b09c`, the same Diffusers
|
| 207 |
revision used in the quickstart. Prompt upsampling uses vLLM 0.19.1, PyTorch
|
| 208 |
2.10.0, and Transformers 5.19.0. These runtime versions are recorded for the new
|
| 209 |
+
measurements; the v0.1 charts have not been rerun in this environment.
|
| 210 |
|
| 211 |
Both prompt enhancers use their checkpoint's system prompt, thinking enabled,
|
| 212 |
`temperature=1.0`, `top_p=0.95`, `top_k=20`, and `min_p=0.0`. Text-to-image uses
|
|
|
|
| 224 |
pip install git+https://github.com/huggingface/diffusers@6256aa7666cedd47443adc8f82da9a10e110b09c
|
| 225 |
```
|
| 226 |
|
| 227 |
+
### v0.1
|
| 228 |
+
|
| 229 |
+
```python
|
| 230 |
+
import torch
|
| 231 |
+
from PIL import Image
|
| 232 |
+
from diffusers import FlowMatchEulerDiscreteScheduler, QwenImage21Pipeline
|
| 233 |
+
|
| 234 |
+
STEPS = 8 # 8 for higher quality, 5 for higher speed
|
| 235 |
+
|
| 236 |
+
# The terminal sigma 0 is appended by the scheduler.
|
| 237 |
+
SIGMAS = {
|
| 238 |
+
5: [1.0, 0.94, 6 / 7, 2 / 3, 0.4],
|
| 239 |
+
8: [1.0, 14 / 15, 6 / 7, 10 / 13, 2 / 3, 6 / 11, 0.4, 2 / 9],
|
| 240 |
+
}[STEPS]
|
| 241 |
+
|
| 242 |
+
pipe = QwenImage21Pipeline.from_pretrained(
|
| 243 |
+
"Qwen/Qwen-Image-2.1", torch_dtype=torch.bfloat16
|
| 244 |
+
).to("cuda")
|
| 245 |
+
pipe.load_lora_weights(
|
| 246 |
+
"PrunaAI/Pruna-Q-Image-2.1",
|
| 247 |
+
weight_name=f"p_q_image_2.1_{STEPS}step_v0.1.safetensors",
|
| 248 |
+
)
|
| 249 |
+
|
| 250 |
+
# Use the sigmas exactly as given: no extra shifting.
|
| 251 |
+
pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
|
| 252 |
+
pipe.scheduler.config,
|
| 253 |
+
use_dynamic_shifting=False,
|
| 254 |
+
shift=1.0,
|
| 255 |
+
shift_terminal=None,
|
| 256 |
+
)
|
| 257 |
+
```
|
| 258 |
+
|
| 259 |
+
### v0.2
|
| 260 |
+
|
| 261 |
+
Use this setup instead of the v0.1 setup above for either new adapter.
|
| 262 |
+
|
| 263 |
```python
|
| 264 |
import torch
|
| 265 |
from PIL import Image
|
|
|
|
| 292 |
```
|
| 293 |
|
| 294 |
|
|
|
|
| 295 |
Both variants load on **`Qwen/Qwen-Image-2.1`** using **`QwenImage21Pipeline`**.
|
| 296 |
Choose one LoRA with `VARIANT`; the prompt-upsampled variant runs directly on your
|
| 297 |
original prompt without a separate prompt-enhancer model. For a local export,
|
|
|
|
| 302 |
Training used alpha 128; the exported B matrices include the `128 / 64 = 2`
|
| 303 |
scale. Load at **LoRA strength 1.0** with the standard pipeline loader.
|
| 304 |
|
| 305 |
+
For v0.2 benchmark settings, pass `output_resolution=1024` and
|
| 306 |
+
`use_kv_cache=False` in the generation and editing calls below.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 307 |
|
| 308 |
## For Text-to-Image
|
| 309 |
|
| 310 |
+
Longer, more descriptive prompts give better results.
|
| 311 |
|
| 312 |
```python
|
| 313 |
image = pipe(
|
|
|
|
| 318 |
),
|
| 319 |
width=1024,
|
| 320 |
height=1024,
|
|
|
|
| 321 |
generator=torch.Generator("cuda").manual_seed(42),
|
| 322 |
num_inference_steps=STEPS,
|
| 323 |
sigmas=SIGMAS,
|
| 324 |
true_cfg_scale=1.0,
|
| 325 |
+
use_kv_cache=True,
|
| 326 |
).images[0]
|
| 327 |
```
|
| 328 |
|
|
|
|
|
|
|
| 329 |
### For Image Editing
|
| 330 |
|
| 331 |
```python
|
| 332 |
image = pipe(
|
| 333 |
prompt="Change the background to a sunset beach",
|
| 334 |
image=Image.open("input.png").convert("RGB"),
|
|
|
|
| 335 |
generator=torch.Generator("cuda").manual_seed(42),
|
| 336 |
num_inference_steps=STEPS,
|
| 337 |
sigmas=SIGMAS,
|
| 338 |
true_cfg_scale=1.0,
|
| 339 |
+
use_kv_cache=True,
|
| 340 |
).images[0]
|
| 341 |
```
|
| 342 |
|
| 343 |
+
### Recommended settings
|
| 344 |
|
| 345 |
+
For **v0.2**, use the unshifted six-step sigmas
|
| 346 |
+
`[1.0, 0.925, 0.85, 0.625, 0.375, 0.125]` with dynamic shifting enabled
|
| 347 |
+
and `shift_terminal=None`, as shown in the v0.2 setup. The scheduler appends
|
| 348 |
+
terminal zero. Keep `true_cfg_scale=1.0` and LoRA strength 1.0.
|
| 349 |
|
| 350 |
+
The **v0.1** settings are:
|
| 351 |
|
| 352 |
+
- **Use the sigma schedule that matches your adapter**, and keep the scheduler
|
| 353 |
+
at `shift=1.0` with dynamic shifting off so the sigmas are not shifted twice.
|
| 354 |
+
- 8-step: `1 β 14/15 β 6/7 β 10/13 β 2/3 β 6/11 β 0.4 β 2/9 β 0`
|
| 355 |
+
(shift 2, computed as `Ο = 2t / (1 + t)` on evenly spaced `t`)
|
| 356 |
+
- 5-step: `1 β 0.94 β 6/7 β 2/3 β 0.4 β 0`
|
|
|
|
|
|
|
|
|
|
|
|
|
| 357 |
- **No CFG.** Keep `true_cfg_scale=1.0` and do not pass a negative prompt.
|
| 358 |
- **Keep the LoRA strength at 1.0.**
|
| 359 |
+
- **Write detailed prompts for text-to-image.** Longer prompts that describe
|
| 360 |
+
the subject, setting, lighting, and style work noticeably better than short
|
| 361 |
+
ones.
|
|
|
|
|
|
|
| 362 |
|
| 363 |
## Limitations
|
| 364 |
|
| 365 |
+
### v0.2
|
| 366 |
+
|
| 367 |
- The prompt-upsampled adapter is a first attempt. It can improve visual quality,
|
| 368 |
+
but may not exactly reproduce the output of the actual prompt upsampler.
|
| 369 |
- Basic is intended to preserve the base model's behavior in fewer steps;
|
| 370 |
+
distillation does not guarantee identical outputs.
|
| 371 |
+
- Use the step count and sigma schedule that match your v0.2 adapter. Other step
|
| 372 |
+
counts, schedules, or CFG values are not supported.
|
|
|
|
|
|
|
| 373 |
|
| 374 |
+
### v0.1
|
| 375 |
|
| 376 |
+
- This is a first version. Quality is below that of the base model.
|
| 377 |
+
- The 5-step adapter is faster than the 8-step one, but its images are
|
| 378 |
+
visibly worse.
|
| 379 |
+
- Short or vague text-to-image prompts give weaker results.
|
| 380 |
+
- The adapters are only intended for the settings above. Other step counts,
|
| 381 |
+
schedules, or CFG values are not supported.
|
| 382 |
|
| 383 |
## What this is not
|
| 384 |
|
| 385 |
- Not a standalone model. It needs the `Qwen/Qwen-Image-2.1` base weights.
|
| 386 |
+
- Not a replacement for the base model when you need its full quality.
|
| 387 |
+
- Not a finished release. Expect the weights to change in future versions.
|
|
|
|
|
|
|
| 388 |
|
| 389 |
## License
|
| 390 |
|
|
|
|
| 403 |
- Compress your own models with [Pruna](https://github.com/PrunaAI/pruna) and give us a βοΈ for more efficiency!
|
| 404 |
- Want to use our optimized image models right away? Check [P-Image-Ideogram](https://www.pruna.ai/p-image-ideogram) and [P-Image-Edit](https://www.pruna.ai/p-image-edit).
|
| 405 |
|
| 406 |
+
<style>
|
| 407 |
+
.model-button {
|
| 408 |
+
display: inline-flex;
|
| 409 |
+
flex-direction: row;
|
| 410 |
+
justify-content: center;
|
| 411 |
+
align-items: center;
|
| 412 |
+
gap: 8px;
|
| 413 |
+
|
| 414 |
+
padding: 8px 20px;
|
| 415 |
+
border: none;
|
| 416 |
+
border-radius: 8px;
|
| 417 |
+
|
| 418 |
+
background: #9334e9;
|
| 419 |
+
color: #ffffff;
|
| 420 |
+
|
| 421 |
+
font-size: 14px;
|
| 422 |
+
font-weight: 400;
|
| 423 |
+
line-height: 1;
|
| 424 |
+
text-decoration: none;
|
| 425 |
+
white-space: nowrap;
|
| 426 |
+
cursor: pointer;
|
| 427 |
+
|
| 428 |
+
box-sizing: border-box;
|
| 429 |
+
overflow: visible;
|
| 430 |
+
opacity: 1;
|
| 431 |
+
}
|
| 432 |
+
</style>
|
| 433 |
+
|
| 434 |
+
<a href="https://dashboard.pruna.ai/login?utm_source=huggingface&utm_medium=model_card&utm_campaign=hf_traffic" class="model-button">Try our models</a>
|