Buckets:

download
raw
25.4 kB
import"../chunks/DsnmJJEf.js";import{i as f,h as I,C as k,H as a,a as l,E as G,s as B}from"../chunks/X8cAGukj.js";import{p as g,o as Z,s,f as C,a as u,b as A,d as b,n as W}from"../chunks/CQ2f15a1.js";const _='{"title":"Pipeline callbacks","local":"pipeline-callbacks","sections":[{"title":"Early stopping","local":"early-stopping","sections":[],"depth":2},{"title":"Display intermediate images","local":"display-intermediate-images","sections":[],"depth":2},{"title":"Modular pipelines","local":"modular-pipelines","sections":[],"depth":2}],"depth":1}';var X=b('<meta name="hf:doc:metadata"/>'),N=b('<p></p> <!> <!> <p>A callback runs at the end of a denoising step and can change pipeline state or tensors for later steps. Use it to adjust attributes or tensor variables for new behavior without rewriting the pipeline.</p> <p>These callbacks work with classic <a href="/docs/diffusers/pr_15212/en/api/pipelines/overview#diffusers.DiffusionPipeline">DiffusionPipeline</a> loops out of the box. They also work with <a href="../modular_diffusers/overview">Modular Diffusers</a> pipelines once a block that runs them is inserted into the denoising loop; see <a href="#modular-pipelines">Modular pipelines</a>.</p> <p>Diffusers provides several callbacks in the pipeline <a href="../api/pipelines/overview#diffusers.callbacks.PipelineCallback">overview</a>.</p> <p>To enable a callback, configure when the callback is executed after a certain number of denoising steps with one of the following arguments.</p> <ul><li><code>cutoff_step_ratio</code> specifies when a callback is activated as a percentage of the total denoising steps. Use when the cutoff should scale with <code>num_inference_steps</code> (for example, drop CFG after 40% of run).</li> <li><code>cutoff_step_index</code> specifies the exact step number a callback is activated. Use when you care about an absolute step (for example, step <code>10</code> on a fixed 25-step schedule).</li></ul> <p>The example below uses <code>cutoff_step_ratio=0.4</code>, which means the callback is activated once denoising reaches 40% of the total inference steps. <a href="/docs/diffusers/pr_15212/en/api/pipelines/overview#diffusers.callbacks.SDXLCFGCutoffCallback">SDXLCFGCutoffCallback</a> disables classifier-free guidance (CFG) after a certain number of steps, which can help save compute without significantly affecting performance.</p> <p>Define a callback with one of the <code>cutoff</code> arguments and pass it to the <code>callback_on_step_end</code> parameter in the pipeline.</p> <!> <p>Official callbacks set their own tensor inputs. For a custom function, pass <code>callback_on_step_end_tensor_inputs</code> as in <a href="#display-intermediate-images">Display intermediate images</a>.</p> <p>If you want to add a new official callback, feel free to open a <a href="https://github.com/huggingface/diffusers/issues/new/choose" rel="nofollow">feature request</a> or <a href="https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#how-to-open-a-pr" rel="nofollow">submit a PR</a>. Otherwise, you can also create your own callback as shown below.</p> <!> <p>Early stopping is useful if you aren’t happy with the intermediate results during generation. This callback sets a hardcoded stop point by setting the <code>_interrupt</code> attribute to <code>True</code>, which makes the denoising loop skip the remaining steps.</p> <!> <!> <p>Visualizing intermediate images is useful for progress monitoring. The preview below is SDXL-only. It maps SDXL latents to RGB with a linear transform for a quick look during denoising. Those weights do not transfer to other models. For Qwen-Image and similar checkpoints, decode with the model VAE instead of this helper.</p> <p><a href="https://huggingface.co/blog/TimothyAlexisVass/explaining-the-sdxl-latent-space" rel="nofollow">Convert</a> Stable Diffusion XL latents (4 channels) to RGB tensors (3 channels).</p> <!> <p>Extract the latents and convert the first image in the batch to RGB. Save the image as a PNG file with the step number.</p> <!> <p>Use <code>callback_on_step_end_tensor_inputs</code> to choose which tensors the callback receives, which in this case, are the latents.</p> <!> <!> <p>Modular pipelines don’t carry the <code>callback_on_step_end</code> and <code>interrupt</code> mechanism. It will come with first-class streaming support instead: <a href="https://github.com/huggingface/diffusers/pull/14159" rel="nofollow">huggingface/diffusers#14159</a>, not merged yet, adds <code>pipe.stream()</code>, which yields the pipeline state at every denoising step and ends the run when you stop iterating. For now, to reuse your existing callback functions, insert a block into the denoising loop: it receives the loop’s <code>BlockState</code> with the step index and timestep, and can read or edit <code>latents</code>.</p> <blockquote class="tip"><p>Edit the blocks <em>before</em> <a href="/docs/diffusers/pr_15212/en/api/modular_diffusers/pipeline_blocks#diffusers.ModularPipelineBlocks.init_pipeline">init_pipeline()</a>: <code>pipe.blocks</code> returns a copy, so changing it after the pipeline exists has no effect.</p></blockquote> <p>The block below takes the callback as a pipeline input and calls it with the same signature as above: the pipeline, the step index, the timestep and a dict of tensors, whose returned values are written back. <code>walk_denoise_loops</code> finds every denoising loop in the blocks, whichever workflow it belongs to, so the same snippet works for any modular pipeline.</p> <!> <p><code>callback_on_step_end</code> is now an input of the pipeline, and a callback written for a <code>DiffusionPipeline</code>, such as <code>decode_tensors</code> above, works as is:</p> <!> <p><a href="#early-stopping">Early stopping</a> works differently: a block cannot skip the loop it runs in, so setting <code>_interrupt</code> has no effect. Raise from the callback instead: the exception propagates out of <code>pipe(...)</code> (no image is decoded), and the pipeline stays usable afterwards.</p> <p>This works for every modular pipeline except the inner loop of nested loops (autoregressive video models such as Helios and Wan-Animate-2 denoise each chunk inside a block of the outer loop). There, insert the block into the outer loop instead: it runs once per chunk and receives the chunk index <code>k</code> in place of <code>i</code> and <code>t</code>.</p> <!> <p></p>',1);function E(T,m){g(m,!1),Z(()=>{new URLSearchParams(window.location.search).get("fw")}),f();var e=N();I("apt59n",h=>{var j=X();B(j,"content",_),u(h,j)});var n=s(C(e),2);k(n,{containerStyle:"float: inline-end; margin-inline-start: 10px; display: inline-flex; position: relative; z-index: 10;"});var t=s(n,2);a(t,{title:"Pipeline callbacks",local:"pipeline-callbacks",headingTag:"h1"});var p=s(t,16);l(p,{code:"aW1wb3J0JTIwdG9yY2glMEFmcm9tJTIwZGlmZnVzZXJzJTIwaW1wb3J0JTIwRFBNU29sdmVyTXVsdGlzdGVwU2NoZWR1bGVyJTJDJTIwU3RhYmxlRGlmZnVzaW9uWExQaXBlbGluZSUwQWZyb20lMjBkaWZmdXNlcnMuY2FsbGJhY2tzJTIwaW1wb3J0JTIwU0RYTENGR0N1dG9mZkNhbGxiYWNrJTBBJTBBY2FsbGJhY2slMjAlM0QlMjBTRFhMQ0ZHQ3V0b2ZmQ2FsbGJhY2soY3V0b2ZmX3N0ZXBfcmF0aW8lM0QwLjQpJTBBJTIzJTIwaWYlMjB1c2luZyUyMGN1dG9mZl9zdGVwX2luZGV4JTBBJTIzJTIwY2FsbGJhY2slMjAlM0QlMjBTRFhMQ0ZHQ3V0b2ZmQ2FsbGJhY2soY3V0b2ZmX3N0ZXBfcmF0aW8lM0ROb25lJTJDJTIwY3V0b2ZmX3N0ZXBfaW5kZXglM0QxMCklMEElMEFwaXBlbGluZSUyMCUzRCUyMFN0YWJsZURpZmZ1c2lvblhMUGlwZWxpbmUuZnJvbV9wcmV0cmFpbmVkKCUwQSUyMCUyMCUyMCUyMCUyMnN0YWJpbGl0eWFpJTJGc3RhYmxlLWRpZmZ1c2lvbi14bC1iYXNlLTEuMCUyMiUyQyUwQSUyMCUyMCUyMCUyMGR0eXBlJTNEdG9yY2guZmxvYXQxNiUyQyUwQSUyMCUyMCUyMCUyMGRldmljZV9tYXAlM0QlMjJjdWRhJTIyJTIwJTIwJTIzJTIwb3IlMjAlMjJtcHMlMjIlMkMlMjAlMjJ4cHUlMjIlMkMlMjAlMjJjcHUlMjIlMEEpJTBBcGlwZWxpbmUuc2NoZWR1bGVyJTIwJTNEJTIwRFBNU29sdmVyTXVsdGlzdGVwU2NoZWR1bGVyLmZyb21fY29uZmlnKHBpcGVsaW5lLnNjaGVkdWxlci5jb25maWclMkMlMjB1c2Vfa2FycmFzX3NpZ21hcyUzRFRydWUpJTBBJTBBcHJvbXB0JTIwJTNEJTIwJTIyYSUyMHNwb3J0cyUyMGNhciUyMG9uJTIwdGhlJTIwcm9hZCUyQyUyMGJlc3QlMjBxdWFsaXR5JTJDJTIwaGlnaCUyMHF1YWxpdHklMkMlMjBoaWdoJTIwZGV0YWlsJTJDJTIwOGslMjByZXNvbHV0aW9uJTIyJTBBb3V0cHV0JTIwJTNEJTIwcGlwZWxpbmUoJTBBJTIwJTIwJTIwJTIwcHJvbXB0JTNEcHJvbXB0JTJDJTBBJTIwJTIwJTIwJTIwbmVnYXRpdmVfcHJvbXB0JTNEJTIyJTIyJTJDJTBBJTIwJTIwJTIwJTIwZ3VpZGFuY2Vfc2NhbGUlM0Q2LjUlMkMlMEElMjAlMjAlMjAlMjBudW1faW5mZXJlbmNlX3N0ZXBzJTNEMjUlMkMlMEElMjAlMjAlMjAlMjBjYWxsYmFja19vbl9zdGVwX2VuZCUzRGNhbGxiYWNrJTJDJTBBKQ==",highlighted:`<span class="hljs-keyword">import</span> torch
<span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> DPMSolverMultistepScheduler, StableDiffusionXLPipeline
<span class="hljs-keyword">from</span> diffusers.callbacks <span class="hljs-keyword">import</span> SDXLCFGCutoffCallback
callback = SDXLCFGCutoffCallback(cutoff_step_ratio=<span class="hljs-number">0.4</span>)
<span class="hljs-comment"># if using cutoff_step_index</span>
<span class="hljs-comment"># callback = SDXLCFGCutoffCallback(cutoff_step_ratio=None, cutoff_step_index=10)</span>
pipeline = StableDiffusionXLPipeline.from_pretrained(
<span class="hljs-string">&quot;stabilityai/stable-diffusion-xl-base-1.0&quot;</span>,
dtype=torch.float16,
device_map=<span class="hljs-string">&quot;cuda&quot;</span> <span class="hljs-comment"># or &quot;mps&quot;, &quot;xpu&quot;, &quot;cpu&quot;</span>
)
pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config, use_karras_sigmas=<span class="hljs-literal">True</span>)
prompt = <span class="hljs-string">&quot;a sports car on the road, best quality, high quality, high detail, 8k resolution&quot;</span>
output = pipeline(
prompt=prompt,
negative_prompt=<span class="hljs-string">&quot;&quot;</span>,
guidance_scale=<span class="hljs-number">6.5</span>,
num_inference_steps=<span class="hljs-number">25</span>,
callback_on_step_end=callback,
)`,lang:"py",wrap:!1});var o=s(p,6);a(o,{title:"Early stopping",local:"early-stopping",headingTag:"h2"});var i=s(o,4);l(i,{code:"aW1wb3J0JTIwdG9yY2glMEFmcm9tJTIwZGlmZnVzZXJzJTIwaW1wb3J0JTIwRGlmZnVzaW9uUGlwZWxpbmUlMEElMEFkZWYlMjBpbnRlcnJ1cHRfY2FsbGJhY2socGlwZWxpbmUlMkMlMjBpJTJDJTIwdCUyQyUyMGNhbGxiYWNrX2t3YXJncyklM0ElMEElMjAlMjAlMjAlMjBzdG9wX2lkeCUyMCUzRCUyMDEwJTBBJTIwJTIwJTIwJTIwaWYlMjBpJTIwJTNEJTNEJTIwc3RvcF9pZHglM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBwaXBlbGluZS5faW50ZXJydXB0JTIwJTNEJTIwVHJ1ZSUwQSUyMCUyMCUyMCUyMHJldHVybiUyMGNhbGxiYWNrX2t3YXJncyUwQSUwQXBpcGVsaW5lJTIwJTNEJTIwRGlmZnVzaW9uUGlwZWxpbmUuZnJvbV9wcmV0cmFpbmVkKCUwQSUyMCUyMCUyMCUyMCUyMlF3ZW4lMkZRd2VuLUltYWdlJTIyJTJDJTBBJTIwJTIwJTIwJTIwZHR5cGUlM0R0b3JjaC5iZmxvYXQxNiUyQyUwQSUyMCUyMCUyMCUyMGRldmljZV9tYXAlM0QlMjJjdWRhJTIyJTJDJTIwJTIwJTIzJTIwb3IlMjAlMjJtcHMlMjIlMkMlMjAlMjJ4cHUlMjIlMkMlMjAlMjJjcHUlMjIlMEEpJTBBcGlwZWxpbmUoJTBBJTIwJTIwJTIwJTIwcHJvbXB0JTNEJTIyQSUyMHBob3RvJTIwb2YlMjBhJTIwY2F0JTIyJTJDJTBBJTIwJTIwJTIwJTIwbnVtX2luZmVyZW5jZV9zdGVwcyUzRDUwJTJDJTBBJTIwJTIwJTIwJTIwY2FsbGJhY2tfb25fc3RlcF9lbmQlM0RpbnRlcnJ1cHRfY2FsbGJhY2slMkMlMEEp",highlighted:`<span class="hljs-keyword">import</span> torch
<span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> DiffusionPipeline
<span class="hljs-keyword">def</span> <span class="hljs-title function_">interrupt_callback</span>(<span class="hljs-params">pipeline, i, t, callback_kwargs</span>):
stop_idx = <span class="hljs-number">10</span>
<span class="hljs-keyword">if</span> i == stop_idx:
pipeline._interrupt = <span class="hljs-literal">True</span>
<span class="hljs-keyword">return</span> callback_kwargs
pipeline = DiffusionPipeline.from_pretrained(
<span class="hljs-string">&quot;Qwen/Qwen-Image&quot;</span>,
dtype=torch.bfloat16,
device_map=<span class="hljs-string">&quot;cuda&quot;</span>, <span class="hljs-comment"># or &quot;mps&quot;, &quot;xpu&quot;, &quot;cpu&quot;</span>
)
pipeline(
prompt=<span class="hljs-string">&quot;A photo of a cat&quot;</span>,
num_inference_steps=<span class="hljs-number">50</span>,
callback_on_step_end=interrupt_callback,
)`,lang:"py",wrap:!1});var c=s(i,2);a(c,{title:"Display intermediate images",local:"display-intermediate-images",headingTag:"h2"});var M=s(c,6);l(M,{code:"aW1wb3J0JTIwdG9yY2glMEFmcm9tJTIwUElMJTIwaW1wb3J0JTIwSW1hZ2UlMEFmcm9tJTIwZGlmZnVzZXJzJTIwaW1wb3J0JTIwQXV0b1BpcGVsaW5lRm9yVGV4dDJJbWFnZSUwQSUwQWRlZiUyMGxhdGVudHNfdG9fcmdiKGxhdGVudHMpJTNBJTBBJTIwJTIwJTIwJTIwd2VpZ2h0cyUyMCUzRCUyMCglMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAoNjAlMkMlMjAtNjAlMkMlMjAyNSUyQyUyMC03MCklMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAoNjAlMkMlMjAlMjAtNSUyQyUyMDE1JTJDJTIwLTUwKSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCg2MCUyQyUyMCUyMDEwJTJDJTIwLTUlMkMlMjAtMzUpJTJDJTBBJTIwJTIwJTIwJTIwKSUwQSUwQSUyMCUyMCUyMCUyMHdlaWdodHNfdGVuc29yJTIwJTNEJTIwdG9yY2gudCh0b3JjaC50ZW5zb3Iod2VpZ2h0cyUyQyUyMGR0eXBlJTNEbGF0ZW50cy5kdHlwZSkudG8obGF0ZW50cy5kZXZpY2UpKSUwQSUyMCUyMCUyMCUyMGJpYXNlc190ZW5zb3IlMjAlM0QlMjB0b3JjaC50ZW5zb3IoKDE1MCUyQyUyMDE0MCUyQyUyMDEzMCklMkMlMjBkdHlwZSUzRGxhdGVudHMuZHR5cGUpLnRvKGxhdGVudHMuZGV2aWNlKSUwQSUyMCUyMCUyMCUyMHJnYl90ZW5zb3IlMjAlM0QlMjB0b3JjaC5laW5zdW0oJTIyLi4ubHh5JTJDbHIlMjAtJTNFJTIwLi4ucnh5JTIyJTJDJTIwbGF0ZW50cyUyQyUyMHdlaWdodHNfdGVuc29yKSUyMCUyQiUyMGJpYXNlc190ZW5zb3IudW5zcXVlZXplKC0xKS51bnNxdWVlemUoLTEpJTBBJTIwJTIwJTIwJTIwaW1hZ2VfYXJyYXklMjAlM0QlMjByZ2JfdGVuc29yLmNsYW1wKDAlMkMlMjAyNTUpLmJ5dGUoKS5jcHUoKS5udW1weSgpLnRyYW5zcG9zZSgxJTJDJTIwMiUyQyUyMDApJTBBJTBBJTIwJTIwJTIwJTIwcmV0dXJuJTIwSW1hZ2UuZnJvbWFycmF5KGltYWdlX2FycmF5KQ==",highlighted:`<span class="hljs-keyword">import</span> torch
<span class="hljs-keyword">from</span> PIL <span class="hljs-keyword">import</span> Image
<span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> AutoPipelineForText2Image
<span class="hljs-keyword">def</span> <span class="hljs-title function_">latents_to_rgb</span>(<span class="hljs-params">latents</span>):
weights = (
(<span class="hljs-number">60</span>, -<span class="hljs-number">60</span>, <span class="hljs-number">25</span>, -<span class="hljs-number">70</span>),
(<span class="hljs-number">60</span>, -<span class="hljs-number">5</span>, <span class="hljs-number">15</span>, -<span class="hljs-number">50</span>),
(<span class="hljs-number">60</span>, <span class="hljs-number">10</span>, -<span class="hljs-number">5</span>, -<span class="hljs-number">35</span>),
)
weights_tensor = torch.t(torch.tensor(weights, dtype=latents.dtype).to(latents.device))
biases_tensor = torch.tensor((<span class="hljs-number">150</span>, <span class="hljs-number">140</span>, <span class="hljs-number">130</span>), dtype=latents.dtype).to(latents.device)
rgb_tensor = torch.einsum(<span class="hljs-string">&quot;...lxy,lr -&gt; ...rxy&quot;</span>, latents, weights_tensor) + biases_tensor.unsqueeze(-<span class="hljs-number">1</span>).unsqueeze(-<span class="hljs-number">1</span>)
image_array = rgb_tensor.clamp(<span class="hljs-number">0</span>, <span class="hljs-number">255</span>).byte().cpu().numpy().transpose(<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">0</span>)
<span class="hljs-keyword">return</span> Image.fromarray(image_array)`,lang:"py",wrap:!1});var r=s(M,4);l(r,{code:"ZGVmJTIwZGVjb2RlX3RlbnNvcnMocGlwZSUyQyUyMHN0ZXAlMkMlMjB0aW1lc3RlcCUyQyUyMGNhbGxiYWNrX2t3YXJncyklM0ElMEElMjAlMjAlMjAlMjBsYXRlbnRzJTIwJTNEJTIwY2FsbGJhY2tfa3dhcmdzJTVCJTIybGF0ZW50cyUyMiU1RCUwQSUwQSUyMCUyMCUyMCUyMGltYWdlJTIwJTNEJTIwbGF0ZW50c190b19yZ2IobGF0ZW50cyU1QjAlNUQpJTBBJTIwJTIwJTIwJTIwaW1hZ2Uuc2F2ZShmJTIyJTdCc3RlcCU3RC5wbmclMjIpJTBBJTBBJTIwJTIwJTIwJTIwcmV0dXJuJTIwY2FsbGJhY2tfa3dhcmdz",highlighted:`<span class="hljs-keyword">def</span> <span class="hljs-title function_">decode_tensors</span>(<span class="hljs-params">pipe, step, timestep, callback_kwargs</span>):
latents = callback_kwargs[<span class="hljs-string">&quot;latents&quot;</span>]
image = latents_to_rgb(latents[<span class="hljs-number">0</span>])
image.save(<span class="hljs-string">f&quot;<span class="hljs-subst">{step}</span>.png&quot;</span>)
<span class="hljs-keyword">return</span> callback_kwargs`,lang:"py",wrap:!1});var d=s(r,4);l(d,{code:"cGlwZWxpbmUlMjAlM0QlMjBBdXRvUGlwZWxpbmVGb3JUZXh0MkltYWdlLmZyb21fcHJldHJhaW5lZCglMEElMjAlMjAlMjAlMjAlMjJzdGFiaWxpdHlhaSUyRnN0YWJsZS1kaWZmdXNpb24teGwtYmFzZS0xLjAlMjIlMkMlMEElMjAlMjAlMjAlMjBkdHlwZSUzRHRvcmNoLmZsb2F0MTYlMkMlMEElMjAlMjAlMjAlMjBkZXZpY2VfbWFwJTNEJTIyY3VkYSUyMiUyMCUyMCUyMyUyMG9yJTIwJTIybXBzJTIyJTJDJTIwJTIyeHB1JTIyJTJDJTIwJTIyY3B1JTIyJTBBKSUwQSUwQWltYWdlJTIwJTNEJTIwcGlwZWxpbmUoJTBBJTIwJTIwJTIwJTIwcHJvbXB0JTNEJTIyQSUyMGNyb2lzc2FudCUyMHNoYXBlZCUyMGxpa2UlMjBhJTIwY3V0ZSUyMGJlYXIuJTIyJTJDJTBBJTIwJTIwJTIwJTIwbmVnYXRpdmVfcHJvbXB0JTNEJTIyRGVmb3JtZWQlMkMlMjB1Z2x5JTJDJTIwYmFkJTIwYW5hdG9teSUyMiUyQyUwQSUyMCUyMCUyMCUyMGNhbGxiYWNrX29uX3N0ZXBfZW5kJTNEZGVjb2RlX3RlbnNvcnMlMkMlMEElMjAlMjAlMjAlMjBjYWxsYmFja19vbl9zdGVwX2VuZF90ZW5zb3JfaW5wdXRzJTNEJTVCJTIybGF0ZW50cyUyMiU1RCUyQyUwQSkuaW1hZ2VzJTVCMCU1RA==",highlighted:`pipeline = AutoPipelineForText2Image.from_pretrained(
<span class="hljs-string">&quot;stabilityai/stable-diffusion-xl-base-1.0&quot;</span>,
dtype=torch.float16,
device_map=<span class="hljs-string">&quot;cuda&quot;</span> <span class="hljs-comment"># or &quot;mps&quot;, &quot;xpu&quot;, &quot;cpu&quot;</span>
)
image = pipeline(
prompt=<span class="hljs-string">&quot;A croissant shaped like a cute bear.&quot;</span>,
negative_prompt=<span class="hljs-string">&quot;Deformed, ugly, bad anatomy&quot;</span>,
callback_on_step_end=decode_tensors,
callback_on_step_end_tensor_inputs=[<span class="hljs-string">&quot;latents&quot;</span>],
).images[<span class="hljs-number">0</span>]`,lang:"py",wrap:!1});var J=s(d,2);a(J,{title:"Modular pipelines",local:"modular-pipelines",headingTag:"h2"});var y=s(J,8);l(y,{code:"ZnJvbSUyMHR5cGluZyUyMGltcG9ydCUyMENhbGxhYmxlJTBBJTBBaW1wb3J0JTIwdG9yY2glMEFmcm9tJTIwZGlmZnVzZXJzJTIwaW1wb3J0JTIwTW9kdWxhclBpcGVsaW5lJTBBZnJvbSUyMGRpZmZ1c2Vycy5tb2R1bGFyX3BpcGVsaW5lcyUyMGltcG9ydCUyMElucHV0UGFyYW0lMkMlMjBMb29wU2VxdWVudGlhbFBpcGVsaW5lQmxvY2tzJTJDJTIwTW9kdWxhclBpcGVsaW5lQmxvY2tzJTBBJTBBJTBBY2xhc3MlMjBDYWxsYmFja09uU3RlcEVuZChNb2R1bGFyUGlwZWxpbmVCbG9ja3MpJTNBJTBBJTIwJTIwJTIwJTIwJTQwcHJvcGVydHklMEElMjAlMjAlMjAlMjBkZWYlMjBpbnB1dHMoc2VsZiklM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjByZXR1cm4lMjAlNUIlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBJbnB1dFBhcmFtKCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMmNhbGxiYWNrX29uX3N0ZXBfZW5kJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwdHlwZV9oaW50JTNEQ2FsbGFibGUlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBkZXNjcmlwdGlvbiUzRCUyMkNhbGxlZCUyMGFmdGVyJTIwZWFjaCUyMHNjaGVkdWxlciUyMHN0ZXAlMjBhcyUyMCU2MGNhbGxiYWNrKHBpcGUlMkMlMjBpJTJDJTIwdCUyQyUyMCU3QidsYXRlbnRzJyUzQSUyMC4uLiU3RCklNjAlM0IlMjB0aGUlMjBkaWN0JTIwaXQlMjByZXR1cm5zJTIwaXMlMjB3cml0dGVuJTIwYmFjay4lMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjApJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwSW5wdXRQYXJhbSglMjJsYXRlbnRzJTIyJTJDJTIwcmVxdWlyZWQlM0RUcnVlKSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCU1RCUwQSUwQSUyMCUyMCUyMCUyMGRlZiUyMF9fY2FsbF9fKHNlbGYlMkMlMjBjb21wb25lbnRzJTJDJTIwYmxvY2tfc3RhdGUlMkMlMjBpJTJDJTIwdCklM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBpZiUyMGJsb2NrX3N0YXRlLmNhbGxiYWNrX29uX3N0ZXBfZW5kJTIwaXMlMjBub3QlMjBOb25lJTNBJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwb3V0cHV0cyUyMCUzRCUyMGJsb2NrX3N0YXRlLmNhbGxiYWNrX29uX3N0ZXBfZW5kKGNvbXBvbmVudHMlMkMlMjBpJTJDJTIwdCUyQyUyMCU3QiUyMmxhdGVudHMlMjIlM0ElMjBibG9ja19zdGF0ZS5sYXRlbnRzJTdEKSUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMGJsb2NrX3N0YXRlLmxhdGVudHMlMjAlM0QlMjBvdXRwdXRzLmdldCglMjJsYXRlbnRzJTIyJTJDJTIwYmxvY2tfc3RhdGUubGF0ZW50cyklMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjByZXR1cm4lMjBjb21wb25lbnRzJTJDJTIwYmxvY2tfc3RhdGUlMEElMEElMEFkZWYlMjB3YWxrX2Rlbm9pc2VfbG9vcHMoYmxvY2spJTNBJTBBJTIwJTIwJTIwJTIwJTIyJTIyJTIyRXZlcnklMjBkZW5vaXNpbmclMjBsb29wJTIwaW4lMjB0aGUlMjBibG9ja3MlMkMlMjB3aGljaGV2ZXIlMjB3b3JrZmxvdyUyMGl0JTIwYmVsb25ncyUyMHRvLiUyMiUyMiUyMiUwQSUyMCUyMCUyMCUyMGlmJTIwaXNpbnN0YW5jZShibG9jayUyQyUyMExvb3BTZXF1ZW50aWFsUGlwZWxpbmVCbG9ja3MpJTIwYW5kJTIwJTIyZGVub2lzZXIlMjIlMjBpbiUyMGJsb2NrLnN1Yl9ibG9ja3MlM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjB5aWVsZCUyMGJsb2NrJTBBJTIwJTIwJTIwJTIwZm9yJTIwc3ViJTIwaW4lMjBnZXRhdHRyKGJsb2NrJTJDJTIwJTIyc3ViX2Jsb2NrcyUyMiUyQyUyMCU3QiU3RCkudmFsdWVzKCklM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjB5aWVsZCUyMGZyb20lMjB3YWxrX2Rlbm9pc2VfbG9vcHMoc3ViKSUwQSUwQSUwQXJlcG9faWQlMjAlM0QlMjAlMjJzdGFiaWxpdHlhaSUyRnN0YWJsZS1kaWZmdXNpb24teGwtYmFzZS0xLjAlMjIlMEFibG9ja3MlMjAlM0QlMjBNb2R1bGFyUGlwZWxpbmUuZnJvbV9wcmV0cmFpbmVkKHJlcG9faWQpLmJsb2NrcyUwQWZvciUyMGxvb3AlMjBpbiUyMHdhbGtfZGVub2lzZV9sb29wcyhibG9ja3MpJTNBJTBBJTIwJTIwJTIwJTIwbG9vcC5zdWJfYmxvY2tzJTVCJTIyY2FsbGJhY2slMjIlNUQlMjAlM0QlMjBDYWxsYmFja09uU3RlcEVuZCgpJTIwJTIwJTIzJTIwYXBwZW5kZWQlMjBhZnRlciUyMHRoZSUyMGxhc3QlMjBzdGVwJTIwb2YlMjB0aGUlMjBsb29wJTBBcGlwZSUyMCUzRCUyMGJsb2Nrcy5pbml0X3BpcGVsaW5lKHJlcG9faWQpJTBBcGlwZS5sb2FkX2NvbXBvbmVudHMoZHR5cGUlM0R0b3JjaC5mbG9hdDE2KSUwQXBpcGUudG8oJTIyY3VkYSUyMik=",highlighted:`<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> <span class="hljs-type">Callable</span>
<span class="hljs-keyword">import</span> torch
<span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> ModularPipeline
<span class="hljs-keyword">from</span> diffusers.modular_pipelines <span class="hljs-keyword">import</span> InputParam, LoopSequentialPipelineBlocks, ModularPipelineBlocks
<span class="hljs-keyword">class</span> <span class="hljs-title class_">CallbackOnStepEnd</span>(<span class="hljs-title class_ inherited__">ModularPipelineBlocks</span>):
<span class="hljs-meta"> @property</span>
<span class="hljs-keyword">def</span> <span class="hljs-title function_">inputs</span>(<span class="hljs-params">self</span>):
<span class="hljs-keyword">return</span> [
InputParam(
<span class="hljs-string">&quot;callback_on_step_end&quot;</span>,
type_hint=<span class="hljs-type">Callable</span>,
description=<span class="hljs-string">&quot;Called after each scheduler step as \`callback(pipe, i, t, {&#x27;latents&#x27;: ...})\`; the dict it returns is written back.&quot;</span>,
),
InputParam(<span class="hljs-string">&quot;latents&quot;</span>, required=<span class="hljs-literal">True</span>),
]
<span class="hljs-keyword">def</span> <span class="hljs-title function_">__call__</span>(<span class="hljs-params">self, components, block_state, i, t</span>):
<span class="hljs-keyword">if</span> block_state.callback_on_step_end <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-literal">None</span>:
outputs = block_state.callback_on_step_end(components, i, t, {<span class="hljs-string">&quot;latents&quot;</span>: block_state.latents})
block_state.latents = outputs.get(<span class="hljs-string">&quot;latents&quot;</span>, block_state.latents)
<span class="hljs-keyword">return</span> components, block_state
<span class="hljs-keyword">def</span> <span class="hljs-title function_">walk_denoise_loops</span>(<span class="hljs-params">block</span>):
<span class="hljs-string">&quot;&quot;&quot;Every denoising loop in the blocks, whichever workflow it belongs to.&quot;&quot;&quot;</span>
<span class="hljs-keyword">if</span> <span class="hljs-built_in">isinstance</span>(block, LoopSequentialPipelineBlocks) <span class="hljs-keyword">and</span> <span class="hljs-string">&quot;denoiser&quot;</span> <span class="hljs-keyword">in</span> block.sub_blocks:
<span class="hljs-keyword">yield</span> block
<span class="hljs-keyword">for</span> sub <span class="hljs-keyword">in</span> <span class="hljs-built_in">getattr</span>(block, <span class="hljs-string">&quot;sub_blocks&quot;</span>, {}).values():
<span class="hljs-keyword">yield</span> <span class="hljs-keyword">from</span> walk_denoise_loops(sub)
repo_id = <span class="hljs-string">&quot;stabilityai/stable-diffusion-xl-base-1.0&quot;</span>
blocks = ModularPipeline.from_pretrained(repo_id).blocks
<span class="hljs-keyword">for</span> loop <span class="hljs-keyword">in</span> walk_denoise_loops(blocks):
loop.sub_blocks[<span class="hljs-string">&quot;callback&quot;</span>] = CallbackOnStepEnd() <span class="hljs-comment"># appended after the last step of the loop</span>
pipe = blocks.init_pipeline(repo_id)
pipe.load_components(dtype=torch.float16)
pipe.to(<span class="hljs-string">&quot;cuda&quot;</span>)`,lang:"py",wrap:!1});var w=s(y,4);l(w,{code:"aW1hZ2UlMjAlM0QlMjBwaXBlKHByb21wdCUzRCUyMkElMjBjcm9pc3NhbnQlMjBzaGFwZWQlMjBsaWtlJTIwYSUyMGN1dGUlMjBiZWFyLiUyMiUyQyUyMGNhbGxiYWNrX29uX3N0ZXBfZW5kJTNEZGVjb2RlX3RlbnNvcnMlMkMlMjBvdXRwdXQlM0QlMjJpbWFnZXMlMjIpJTVCMCU1RA==",highlighted:'image = pipe(prompt=<span class="hljs-string">&quot;A croissant shaped like a cute bear.&quot;</span>, callback_on_step_end=decode_tensors, output=<span class="hljs-string">&quot;images&quot;</span>)[<span class="hljs-number">0</span>]',lang:"py",wrap:!1});var U=s(w,6);G(U,{source:"https://github.com/huggingface/diffusers/blob/main/docs/source/en/using-diffusers/callback.md"}),W(2),u(T,e),A()}export{E as component};

Xet Storage Details

Size:
25.4 kB
·
Xet hash:
9ab8c85b89d188fd66ec36687f0014218e13784256f0b586f5b83de0af029618

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.