nifleisch commited on
Commit
c531c8c
·
verified ·
1 Parent(s): 7f55350

Enable KV cache and inline inference settings in examples

Browse files
Files changed (1) hide show
  1. README.md +10 -12
README.md CHANGED
@@ -88,7 +88,8 @@ Text-to-image with the official Qwen model-card example prompt, BF16, batch size
88
  on one NVIDIA H100 80GB. Median of 3 requests after one warmup per configuration.
89
  Includes prompt encoding, denoising and decoding; excludes PNG saving, model
90
  loading and warmup. Base: 40 steps with KV cache on; P-Turbo: 5 or 8 steps with
91
- KV cache off, following the respective model-card settings. LoRAs were unmerged;
 
92
  no CFG, compilation or CPU offload. These timings do not imply equal image quality.
93
 
94
  ## Quickstart
@@ -128,13 +129,6 @@ pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
128
  shift=1.0,
129
  shift_terminal=None,
130
  )
131
-
132
- settings = dict(
133
- num_inference_steps=STEPS,
134
- sigmas=SIGMAS,
135
- true_cfg_scale=1.0,
136
- use_kv_cache=False,
137
- )
138
  ```
139
 
140
  ## For Text-to-Image
@@ -151,9 +145,11 @@ image = pipe(
151
  width=1024,
152
  height=1024,
153
  generator=torch.Generator("cuda").manual_seed(42),
154
- **settings,
 
 
 
155
  ).images[0]
156
- image.save("t2i_example.png")
157
  ```
158
 
159
  ### For Image Editing
@@ -163,9 +159,11 @@ image = pipe(
163
  prompt="Change the background to a sunset beach",
164
  image=Image.open("input.png").convert("RGB"),
165
  generator=torch.Generator("cuda").manual_seed(42),
166
- **settings,
 
 
 
167
  ).images[0]
168
- image.save("edit_example.png")
169
  ```
170
 
171
  ## Recommended settings
 
88
  on one NVIDIA H100 80GB. Median of 3 requests after one warmup per configuration.
89
  Includes prompt encoding, denoising and decoding; excludes PNG saving, model
90
  loading and warmup. Base: 40 steps with KV cache on; P-Turbo: 5 or 8 steps with
91
+ KV cache off, as configured for this benchmark. The examples below enable KV caching for
92
+ P-Turbo; the chart has not been remeasured with that setting. LoRAs were unmerged;
93
  no CFG, compilation or CPU offload. These timings do not imply equal image quality.
94
 
95
  ## Quickstart
 
129
  shift=1.0,
130
  shift_terminal=None,
131
  )
 
 
 
 
 
 
 
132
  ```
133
 
134
  ## For Text-to-Image
 
145
  width=1024,
146
  height=1024,
147
  generator=torch.Generator("cuda").manual_seed(42),
148
+ num_inference_steps=STEPS,
149
+ sigmas=SIGMAS,
150
+ true_cfg_scale=1.0,
151
+ use_kv_cache=True,
152
  ).images[0]
 
153
  ```
154
 
155
  ### For Image Editing
 
159
  prompt="Change the background to a sunset beach",
160
  image=Image.open("input.png").convert("RGB"),
161
  generator=torch.Generator("cuda").manual_seed(42),
162
+ num_inference_steps=STEPS,
163
+ sigmas=SIGMAS,
164
+ true_cfg_scale=1.0,
165
+ use_kv_cache=True,
166
  ).images[0]
 
167
  ```
168
 
169
  ## Recommended settings