Buckets:
| import"../chunks/DsnmJJEf.js";import{i as f,h as I,C as k,H as a,a as l,E as G,s as B}from"../chunks/BLbO27j4.js";import{p as g,o as Z,s,f as C,a as u,b as A,d as b,n as W}from"../chunks/jDjavuwI.js";const _='{"title":"Pipeline callbacks","local":"pipeline-callbacks","sections":[{"title":"Early stopping","local":"early-stopping","sections":[],"depth":2},{"title":"Display intermediate images","local":"display-intermediate-images","sections":[],"depth":2},{"title":"Modular pipelines","local":"modular-pipelines","sections":[],"depth":2}],"depth":1}';var X=b('<meta name="hf:doc:metadata"/>'),N=b('<p></p> <!> <!> <p>A callback runs at the end of a denoising step and can change pipeline state or tensors for later steps. Use it to adjust attributes or tensor variables for new behavior without rewriting the pipeline.</p> <p>These callbacks work with classic <a href="/docs/diffusers/pr_14521/en/api/pipelines/overview#diffusers.DiffusionPipeline">DiffusionPipeline</a> loops out of the box. They also work with <a href="../modular_diffusers/overview">Modular Diffusers</a> pipelines once a block that runs them is inserted into the denoising loop; see <a href="#modular-pipelines">Modular pipelines</a>.</p> <p>Diffusers provides several callbacks in the pipeline <a href="../api/pipelines/overview#diffusers.callbacks.PipelineCallback">overview</a>.</p> <p>To enable a callback, configure when the callback is executed after a certain number of denoising steps with one of the following arguments.</p> <ul><li><code>cutoff_step_ratio</code> specifies when a callback is activated as a percentage of the total denoising steps. Use when the cutoff should scale with <code>num_inference_steps</code> (for example, drop CFG after 40% of run).</li> <li><code>cutoff_step_index</code> specifies the exact step number a callback is activated. Use when you care about an absolute step (for example, step <code>10</code> on a fixed 25-step schedule).</li></ul> <p>The example below uses <code>cutoff_step_ratio=0.4</code>, which means the callback is activated once denoising reaches 40% of the total inference steps. <a href="/docs/diffusers/pr_14521/en/api/pipelines/overview#diffusers.callbacks.SDXLCFGCutoffCallback">SDXLCFGCutoffCallback</a> disables classifier-free guidance (CFG) after a certain number of steps, which can help save compute without significantly affecting performance.</p> <p>Define a callback with one of the <code>cutoff</code> arguments and pass it to the <code>callback_on_step_end</code> parameter in the pipeline.</p> <!> <p>Official callbacks set their own tensor inputs. For a custom function, pass <code>callback_on_step_end_tensor_inputs</code> as in <a href="#display-intermediate-images">Display intermediate images</a>.</p> <p>If you want to add a new official callback, feel free to open a <a href="https://github.com/huggingface/diffusers/issues/new/choose" rel="nofollow">feature request</a> or <a href="https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#how-to-open-a-pr" rel="nofollow">submit a PR</a>. Otherwise, you can also create your own callback as shown below.</p> <!> <p>Early stopping is useful if you aren’t happy with the intermediate results during generation. This callback sets a hardcoded stop point by setting the <code>_interrupt</code> attribute to <code>True</code>, which makes the denoising loop skip the remaining steps.</p> <!> <!> <p>Visualizing intermediate images is useful for progress monitoring. The preview below is SDXL-only. It maps SDXL latents to RGB with a linear transform for a quick look during denoising. Those weights do not transfer to other models. For Qwen-Image and similar checkpoints, decode with the model VAE instead of this helper.</p> <p><a href="https://huggingface.co/blog/TimothyAlexisVass/explaining-the-sdxl-latent-space" rel="nofollow">Convert</a> Stable Diffusion XL latents (4 channels) to RGB tensors (3 channels).</p> <!> <p>Extract the latents and convert the first image in the batch to RGB. Save the image as a PNG file with the step number.</p> <!> <p>Use <code>callback_on_step_end_tensor_inputs</code> to choose which tensors the callback receives, which in this case, are the latents.</p> <!> <!> <p>Modular pipelines don’t carry the <code>callback_on_step_end</code> and <code>interrupt</code> mechanism. It will come with first-class streaming support instead: <a href="https://github.com/huggingface/diffusers/pull/14159" rel="nofollow">huggingface/diffusers#14159</a>, not merged yet, adds <code>pipe.stream()</code>, which yields the pipeline state at every denoising step and ends the run when you stop iterating. For now, to reuse your existing callback functions, insert a block into the denoising loop: it receives the loop’s <code>BlockState</code> with the step index and timestep, and can read or edit <code>latents</code>.</p> <blockquote class="tip"><p>Edit the blocks <em>before</em> <a href="/docs/diffusers/pr_14521/en/api/modular_diffusers/pipeline_blocks#diffusers.ModularPipelineBlocks.init_pipeline">init_pipeline()</a>: <code>pipe.blocks</code> returns a copy, so changing it after the pipeline exists has no effect.</p></blockquote> <p>The block below takes the callback as a pipeline input and calls it with the same signature as above: the pipeline, the step index, the timestep and a dict of tensors, whose returned values are written back. <code>walk_denoise_loops</code> finds every denoising loop in the blocks, whichever workflow it belongs to, so the same snippet works for any modular pipeline.</p> <!> <p><code>callback_on_step_end</code> is now an input of the pipeline, and a callback written for a <code>DiffusionPipeline</code>, such as <code>decode_tensors</code> above, works as is:</p> <!> <p><a href="#early-stopping">Early stopping</a> works differently: a block cannot skip the loop it runs in, so setting <code>_interrupt</code> has no effect. Raise from the callback instead: the exception propagates out of <code>pipe(...)</code> (no image is decoded), and the pipeline stays usable afterwards.</p> <p>This works for every modular pipeline except the inner loop of nested loops (autoregressive video models such as Helios and Wan-Animate-2 denoise each chunk inside a block of the outer loop). There, insert the block into the outer loop instead: it runs once per chunk and receives the chunk index <code>k</code> in place of <code>i</code> and <code>t</code>.</p> <!> <p></p>',1);function E(T,m){g(m,!1),Z(()=>{new URLSearchParams(window.location.search).get("fw")}),f();var e=N();I("apt59n",h=>{var j=X();B(j,"content",_),u(h,j)});var n=s(C(e),2);k(n,{containerStyle:"float: right; margin-left: 10px; display: inline-flex; position: relative; z-index: 10;"});var t=s(n,2);a(t,{title:"Pipeline callbacks",local:"pipeline-callbacks",headingTag:"h1"});var p=s(t,16);l(p,{code:"aW1wb3J0JTIwdG9yY2glMEFmcm9tJTIwZGlmZnVzZXJzJTIwaW1wb3J0JTIwRFBNU29sdmVyTXVsdGlzdGVwU2NoZWR1bGVyJTJDJTIwU3RhYmxlRGlmZnVzaW9uWExQaXBlbGluZSUwQWZyb20lMjBkaWZmdXNlcnMuY2FsbGJhY2tzJTIwaW1wb3J0JTIwU0RYTENGR0N1dG9mZkNhbGxiYWNrJTBBJTBBY2FsbGJhY2slMjAlM0QlMjBTRFhMQ0ZHQ3V0b2ZmQ2FsbGJhY2soY3V0b2ZmX3N0ZXBfcmF0aW8lM0QwLjQpJTBBJTIzJTIwaWYlMjB1c2luZyUyMGN1dG9mZl9zdGVwX2luZGV4JTBBJTIzJTIwY2FsbGJhY2slMjAlM0QlMjBTRFhMQ0ZHQ3V0b2ZmQ2FsbGJhY2soY3V0b2ZmX3N0ZXBfcmF0aW8lM0ROb25lJTJDJTIwY3V0b2ZmX3N0ZXBfaW5kZXglM0QxMCklMEElMEFwaXBlbGluZSUyMCUzRCUyMFN0YWJsZURpZmZ1c2lvblhMUGlwZWxpbmUuZnJvbV9wcmV0cmFpbmVkKCUwQSUyMCUyMCUyMCUyMCUyMnN0YWJpbGl0eWFpJTJGc3RhYmxlLWRpZmZ1c2lvbi14bC1iYXNlLTEuMCUyMiUyQyUwQSUyMCUyMCUyMCUyMGR0eXBlJTNEdG9yY2guZmxvYXQxNiUyQyUwQSUyMCUyMCUyMCUyMGRldmljZV9tYXAlM0QlMjJjdWRhJTIyJTIwJTIwJTIzJTIwb3IlMjAlMjJtcHMlMjIlMkMlMjAlMjJ4cHUlMjIlMkMlMjAlMjJjcHUlMjIlMEEpJTBBcGlwZWxpbmUuc2NoZWR1bGVyJTIwJTNEJTIwRFBNU29sdmVyTXVsdGlzdGVwU2NoZWR1bGVyLmZyb21fY29uZmlnKHBpcGVsaW5lLnNjaGVkdWxlci5jb25maWclMkMlMjB1c2Vfa2FycmFzX3NpZ21hcyUzRFRydWUpJTBBJTBBcHJvbXB0JTIwJTNEJTIwJTIyYSUyMHNwb3J0cyUyMGNhciUyMG9uJTIwdGhlJTIwcm9hZCUyQyUyMGJlc3QlMjBxdWFsaXR5JTJDJTIwaGlnaCUyMHF1YWxpdHklMkMlMjBoaWdoJTIwZGV0YWlsJTJDJTIwOGslMjByZXNvbHV0aW9uJTIyJTBBb3V0cHV0JTIwJTNEJTIwcGlwZWxpbmUoJTBBJTIwJTIwJTIwJTIwcHJvbXB0JTNEcHJvbXB0JTJDJTBBJTIwJTIwJTIwJTIwbmVnYXRpdmVfcHJvbXB0JTNEJTIyJTIyJTJDJTBBJTIwJTIwJTIwJTIwZ3VpZGFuY2Vfc2NhbGUlM0Q2LjUlMkMlMEElMjAlMjAlMjAlMjBudW1faW5mZXJlbmNlX3N0ZXBzJTNEMjUlMkMlMEElMjAlMjAlMjAlMjBjYWxsYmFja19vbl9zdGVwX2VuZCUzRGNhbGxiYWNrJTJDJTBBKQ==",highlighted:`<span class="hljs-keyword">import</span> torch | |
| <span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> DPMSolverMultistepScheduler, StableDiffusionXLPipeline | |
| <span class="hljs-keyword">from</span> diffusers.callbacks <span class="hljs-keyword">import</span> SDXLCFGCutoffCallback | |
| callback = SDXLCFGCutoffCallback(cutoff_step_ratio=<span class="hljs-number">0.4</span>) | |
| <span class="hljs-comment"># if using cutoff_step_index</span> | |
| <span class="hljs-comment"># callback = SDXLCFGCutoffCallback(cutoff_step_ratio=None, cutoff_step_index=10)</span> | |
| pipeline = StableDiffusionXLPipeline.from_pretrained( | |
| <span class="hljs-string">"stabilityai/stable-diffusion-xl-base-1.0"</span>, | |
| dtype=torch.float16, | |
| device_map=<span class="hljs-string">"cuda"</span> <span class="hljs-comment"># or "mps", "xpu", "cpu"</span> | |
| ) | |
| pipeline.scheduler = DPMSolverMultistepScheduler.from_config(pipeline.scheduler.config, use_karras_sigmas=<span class="hljs-literal">True</span>) | |
| prompt = <span class="hljs-string">"a sports car on the road, best quality, high quality, high detail, 8k resolution"</span> | |
| output = pipeline( | |
| prompt=prompt, | |
| negative_prompt=<span class="hljs-string">""</span>, | |
| guidance_scale=<span class="hljs-number">6.5</span>, | |
| num_inference_steps=<span class="hljs-number">25</span>, | |
| callback_on_step_end=callback, | |
| )`,lang:"py",wrap:!1});var o=s(p,6);a(o,{title:"Early stopping",local:"early-stopping",headingTag:"h2"});var i=s(o,4);l(i,{code:"aW1wb3J0JTIwdG9yY2glMEFmcm9tJTIwZGlmZnVzZXJzJTIwaW1wb3J0JTIwRGlmZnVzaW9uUGlwZWxpbmUlMEElMEFkZWYlMjBpbnRlcnJ1cHRfY2FsbGJhY2socGlwZWxpbmUlMkMlMjBpJTJDJTIwdCUyQyUyMGNhbGxiYWNrX2t3YXJncyklM0ElMEElMjAlMjAlMjAlMjBzdG9wX2lkeCUyMCUzRCUyMDEwJTBBJTIwJTIwJTIwJTIwaWYlMjBpJTIwJTNEJTNEJTIwc3RvcF9pZHglM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBwaXBlbGluZS5faW50ZXJydXB0JTIwJTNEJTIwVHJ1ZSUwQSUyMCUyMCUyMCUyMHJldHVybiUyMGNhbGxiYWNrX2t3YXJncyUwQSUwQXBpcGVsaW5lJTIwJTNEJTIwRGlmZnVzaW9uUGlwZWxpbmUuZnJvbV9wcmV0cmFpbmVkKCUwQSUyMCUyMCUyMCUyMCUyMlF3ZW4lMkZRd2VuLUltYWdlJTIyJTJDJTBBJTIwJTIwJTIwJTIwZHR5cGUlM0R0b3JjaC5iZmxvYXQxNiUyQyUwQSUyMCUyMCUyMCUyMGRldmljZV9tYXAlM0QlMjJjdWRhJTIyJTJDJTIwJTIwJTIzJTIwb3IlMjAlMjJtcHMlMjIlMkMlMjAlMjJ4cHUlMjIlMkMlMjAlMjJjcHUlMjIlMEEpJTBBcGlwZWxpbmUoJTBBJTIwJTIwJTIwJTIwcHJvbXB0JTNEJTIyQSUyMHBob3RvJTIwb2YlMjBhJTIwY2F0JTIyJTJDJTBBJTIwJTIwJTIwJTIwbnVtX2luZmVyZW5jZV9zdGVwcyUzRDUwJTJDJTBBJTIwJTIwJTIwJTIwY2FsbGJhY2tfb25fc3RlcF9lbmQlM0RpbnRlcnJ1cHRfY2FsbGJhY2slMkMlMEEp",highlighted:`<span class="hljs-keyword">import</span> torch | |
| <span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> DiffusionPipeline | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">interrupt_callback</span>(<span class="hljs-params">pipeline, i, t, callback_kwargs</span>): | |
| stop_idx = <span class="hljs-number">10</span> | |
| <span class="hljs-keyword">if</span> i == stop_idx: | |
| pipeline._interrupt = <span class="hljs-literal">True</span> | |
| <span class="hljs-keyword">return</span> callback_kwargs | |
| pipeline = DiffusionPipeline.from_pretrained( | |
| <span class="hljs-string">"Qwen/Qwen-Image"</span>, | |
| dtype=torch.bfloat16, | |
| device_map=<span class="hljs-string">"cuda"</span>, <span class="hljs-comment"># or "mps", "xpu", "cpu"</span> | |
| ) | |
| pipeline( | |
| prompt=<span class="hljs-string">"A photo of a cat"</span>, | |
| num_inference_steps=<span class="hljs-number">50</span>, | |
| callback_on_step_end=interrupt_callback, | |
| )`,lang:"py",wrap:!1});var c=s(i,2);a(c,{title:"Display intermediate images",local:"display-intermediate-images",headingTag:"h2"});var M=s(c,6);l(M,{code:"aW1wb3J0JTIwdG9yY2glMEFmcm9tJTIwUElMJTIwaW1wb3J0JTIwSW1hZ2UlMEFmcm9tJTIwZGlmZnVzZXJzJTIwaW1wb3J0JTIwQXV0b1BpcGVsaW5lRm9yVGV4dDJJbWFnZSUwQSUwQWRlZiUyMGxhdGVudHNfdG9fcmdiKGxhdGVudHMpJTNBJTBBJTIwJTIwJTIwJTIwd2VpZ2h0cyUyMCUzRCUyMCglMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAoNjAlMkMlMjAtNjAlMkMlMjAyNSUyQyUyMC03MCklMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAoNjAlMkMlMjAlMjAtNSUyQyUyMDE1JTJDJTIwLTUwKSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCg2MCUyQyUyMCUyMDEwJTJDJTIwLTUlMkMlMjAtMzUpJTJDJTBBJTIwJTIwJTIwJTIwKSUwQSUwQSUyMCUyMCUyMCUyMHdlaWdodHNfdGVuc29yJTIwJTNEJTIwdG9yY2gudCh0b3JjaC50ZW5zb3Iod2VpZ2h0cyUyQyUyMGR0eXBlJTNEbGF0ZW50cy5kdHlwZSkudG8obGF0ZW50cy5kZXZpY2UpKSUwQSUyMCUyMCUyMCUyMGJpYXNlc190ZW5zb3IlMjAlM0QlMjB0b3JjaC50ZW5zb3IoKDE1MCUyQyUyMDE0MCUyQyUyMDEzMCklMkMlMjBkdHlwZSUzRGxhdGVudHMuZHR5cGUpLnRvKGxhdGVudHMuZGV2aWNlKSUwQSUyMCUyMCUyMCUyMHJnYl90ZW5zb3IlMjAlM0QlMjB0b3JjaC5laW5zdW0oJTIyLi4ubHh5JTJDbHIlMjAtJTNFJTIwLi4ucnh5JTIyJTJDJTIwbGF0ZW50cyUyQyUyMHdlaWdodHNfdGVuc29yKSUyMCUyQiUyMGJpYXNlc190ZW5zb3IudW5zcXVlZXplKC0xKS51bnNxdWVlemUoLTEpJTBBJTIwJTIwJTIwJTIwaW1hZ2VfYXJyYXklMjAlM0QlMjByZ2JfdGVuc29yLmNsYW1wKDAlMkMlMjAyNTUpLmJ5dGUoKS5jcHUoKS5udW1weSgpLnRyYW5zcG9zZSgxJTJDJTIwMiUyQyUyMDApJTBBJTBBJTIwJTIwJTIwJTIwcmV0dXJuJTIwSW1hZ2UuZnJvbWFycmF5KGltYWdlX2FycmF5KQ==",highlighted:`<span class="hljs-keyword">import</span> torch | |
| <span class="hljs-keyword">from</span> PIL <span class="hljs-keyword">import</span> Image | |
| <span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> AutoPipelineForText2Image | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">latents_to_rgb</span>(<span class="hljs-params">latents</span>): | |
| weights = ( | |
| (<span class="hljs-number">60</span>, -<span class="hljs-number">60</span>, <span class="hljs-number">25</span>, -<span class="hljs-number">70</span>), | |
| (<span class="hljs-number">60</span>, -<span class="hljs-number">5</span>, <span class="hljs-number">15</span>, -<span class="hljs-number">50</span>), | |
| (<span class="hljs-number">60</span>, <span class="hljs-number">10</span>, -<span class="hljs-number">5</span>, -<span class="hljs-number">35</span>), | |
| ) | |
| weights_tensor = torch.t(torch.tensor(weights, dtype=latents.dtype).to(latents.device)) | |
| biases_tensor = torch.tensor((<span class="hljs-number">150</span>, <span class="hljs-number">140</span>, <span class="hljs-number">130</span>), dtype=latents.dtype).to(latents.device) | |
| rgb_tensor = torch.einsum(<span class="hljs-string">"...lxy,lr -> ...rxy"</span>, latents, weights_tensor) + biases_tensor.unsqueeze(-<span class="hljs-number">1</span>).unsqueeze(-<span class="hljs-number">1</span>) | |
| image_array = rgb_tensor.clamp(<span class="hljs-number">0</span>, <span class="hljs-number">255</span>).byte().cpu().numpy().transpose(<span class="hljs-number">1</span>, <span class="hljs-number">2</span>, <span class="hljs-number">0</span>) | |
| <span class="hljs-keyword">return</span> Image.fromarray(image_array)`,lang:"py",wrap:!1});var r=s(M,4);l(r,{code:"ZGVmJTIwZGVjb2RlX3RlbnNvcnMocGlwZSUyQyUyMHN0ZXAlMkMlMjB0aW1lc3RlcCUyQyUyMGNhbGxiYWNrX2t3YXJncyklM0ElMEElMjAlMjAlMjAlMjBsYXRlbnRzJTIwJTNEJTIwY2FsbGJhY2tfa3dhcmdzJTVCJTIybGF0ZW50cyUyMiU1RCUwQSUwQSUyMCUyMCUyMCUyMGltYWdlJTIwJTNEJTIwbGF0ZW50c190b19yZ2IobGF0ZW50cyU1QjAlNUQpJTBBJTIwJTIwJTIwJTIwaW1hZ2Uuc2F2ZShmJTIyJTdCc3RlcCU3RC5wbmclMjIpJTBBJTBBJTIwJTIwJTIwJTIwcmV0dXJuJTIwY2FsbGJhY2tfa3dhcmdz",highlighted:`<span class="hljs-keyword">def</span> <span class="hljs-title function_">decode_tensors</span>(<span class="hljs-params">pipe, step, timestep, callback_kwargs</span>): | |
| latents = callback_kwargs[<span class="hljs-string">"latents"</span>] | |
| image = latents_to_rgb(latents[<span class="hljs-number">0</span>]) | |
| image.save(<span class="hljs-string">f"<span class="hljs-subst">{step}</span>.png"</span>) | |
| <span class="hljs-keyword">return</span> callback_kwargs`,lang:"py",wrap:!1});var d=s(r,4);l(d,{code:"cGlwZWxpbmUlMjAlM0QlMjBBdXRvUGlwZWxpbmVGb3JUZXh0MkltYWdlLmZyb21fcHJldHJhaW5lZCglMEElMjAlMjAlMjAlMjAlMjJzdGFiaWxpdHlhaSUyRnN0YWJsZS1kaWZmdXNpb24teGwtYmFzZS0xLjAlMjIlMkMlMEElMjAlMjAlMjAlMjBkdHlwZSUzRHRvcmNoLmZsb2F0MTYlMkMlMEElMjAlMjAlMjAlMjBkZXZpY2VfbWFwJTNEJTIyY3VkYSUyMiUyMCUyMCUyMyUyMG9yJTIwJTIybXBzJTIyJTJDJTIwJTIyeHB1JTIyJTJDJTIwJTIyY3B1JTIyJTBBKSUwQSUwQWltYWdlJTIwJTNEJTIwcGlwZWxpbmUoJTBBJTIwJTIwJTIwJTIwcHJvbXB0JTNEJTIyQSUyMGNyb2lzc2FudCUyMHNoYXBlZCUyMGxpa2UlMjBhJTIwY3V0ZSUyMGJlYXIuJTIyJTJDJTBBJTIwJTIwJTIwJTIwbmVnYXRpdmVfcHJvbXB0JTNEJTIyRGVmb3JtZWQlMkMlMjB1Z2x5JTJDJTIwYmFkJTIwYW5hdG9teSUyMiUyQyUwQSUyMCUyMCUyMCUyMGNhbGxiYWNrX29uX3N0ZXBfZW5kJTNEZGVjb2RlX3RlbnNvcnMlMkMlMEElMjAlMjAlMjAlMjBjYWxsYmFja19vbl9zdGVwX2VuZF90ZW5zb3JfaW5wdXRzJTNEJTVCJTIybGF0ZW50cyUyMiU1RCUyQyUwQSkuaW1hZ2VzJTVCMCU1RA==",highlighted:`pipeline = AutoPipelineForText2Image.from_pretrained( | |
| <span class="hljs-string">"stabilityai/stable-diffusion-xl-base-1.0"</span>, | |
| dtype=torch.float16, | |
| device_map=<span class="hljs-string">"cuda"</span> <span class="hljs-comment"># or "mps", "xpu", "cpu"</span> | |
| ) | |
| image = pipeline( | |
| prompt=<span class="hljs-string">"A croissant shaped like a cute bear."</span>, | |
| negative_prompt=<span class="hljs-string">"Deformed, ugly, bad anatomy"</span>, | |
| callback_on_step_end=decode_tensors, | |
| callback_on_step_end_tensor_inputs=[<span class="hljs-string">"latents"</span>], | |
| ).images[<span class="hljs-number">0</span>]`,lang:"py",wrap:!1});var J=s(d,2);a(J,{title:"Modular pipelines",local:"modular-pipelines",headingTag:"h2"});var y=s(J,8);l(y,{code:"ZnJvbSUyMHR5cGluZyUyMGltcG9ydCUyMENhbGxhYmxlJTBBJTBBaW1wb3J0JTIwdG9yY2glMEFmcm9tJTIwZGlmZnVzZXJzJTIwaW1wb3J0JTIwTW9kdWxhclBpcGVsaW5lJTBBZnJvbSUyMGRpZmZ1c2Vycy5tb2R1bGFyX3BpcGVsaW5lcyUyMGltcG9ydCUyMElucHV0UGFyYW0lMkMlMjBMb29wU2VxdWVudGlhbFBpcGVsaW5lQmxvY2tzJTJDJTIwTW9kdWxhclBpcGVsaW5lQmxvY2tzJTBBJTBBJTBBY2xhc3MlMjBDYWxsYmFja09uU3RlcEVuZChNb2R1bGFyUGlwZWxpbmVCbG9ja3MpJTNBJTBBJTIwJTIwJTIwJTIwJTQwcHJvcGVydHklMEElMjAlMjAlMjAlMjBkZWYlMjBpbnB1dHMoc2VsZiklM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjByZXR1cm4lMjAlNUIlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBJbnB1dFBhcmFtKCUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMmNhbGxiYWNrX29uX3N0ZXBfZW5kJTIyJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwdHlwZV9oaW50JTNEQ2FsbGFibGUlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBkZXNjcmlwdGlvbiUzRCUyMkNhbGxlZCUyMGFmdGVyJTIwZWFjaCUyMHNjaGVkdWxlciUyMHN0ZXAlMjBhcyUyMCU2MGNhbGxiYWNrKHBpcGUlMkMlMjBpJTJDJTIwdCUyQyUyMCU3QidsYXRlbnRzJyUzQSUyMC4uLiU3RCklNjAlM0IlMjB0aGUlMjBkaWN0JTIwaXQlMjByZXR1cm5zJTIwaXMlMjB3cml0dGVuJTIwYmFjay4lMjIlMkMlMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjApJTJDJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwSW5wdXRQYXJhbSglMjJsYXRlbnRzJTIyJTJDJTIwcmVxdWlyZWQlM0RUcnVlKSUyQyUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCU1RCUwQSUwQSUyMCUyMCUyMCUyMGRlZiUyMF9fY2FsbF9fKHNlbGYlMkMlMjBjb21wb25lbnRzJTJDJTIwYmxvY2tfc3RhdGUlMkMlMjBpJTJDJTIwdCklM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjBpZiUyMGJsb2NrX3N0YXRlLmNhbGxiYWNrX29uX3N0ZXBfZW5kJTIwaXMlMjBub3QlMjBOb25lJTNBJTBBJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwJTIwb3V0cHV0cyUyMCUzRCUyMGJsb2NrX3N0YXRlLmNhbGxiYWNrX29uX3N0ZXBfZW5kKGNvbXBvbmVudHMlMkMlMjBpJTJDJTIwdCUyQyUyMCU3QiUyMmxhdGVudHMlMjIlM0ElMjBibG9ja19zdGF0ZS5sYXRlbnRzJTdEKSUwQSUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMCUyMGJsb2NrX3N0YXRlLmxhdGVudHMlMjAlM0QlMjBvdXRwdXRzLmdldCglMjJsYXRlbnRzJTIyJTJDJTIwYmxvY2tfc3RhdGUubGF0ZW50cyklMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjByZXR1cm4lMjBjb21wb25lbnRzJTJDJTIwYmxvY2tfc3RhdGUlMEElMEElMEFkZWYlMjB3YWxrX2Rlbm9pc2VfbG9vcHMoYmxvY2spJTNBJTBBJTIwJTIwJTIwJTIwJTIyJTIyJTIyRXZlcnklMjBkZW5vaXNpbmclMjBsb29wJTIwaW4lMjB0aGUlMjBibG9ja3MlMkMlMjB3aGljaGV2ZXIlMjB3b3JrZmxvdyUyMGl0JTIwYmVsb25ncyUyMHRvLiUyMiUyMiUyMiUwQSUyMCUyMCUyMCUyMGlmJTIwaXNpbnN0YW5jZShibG9jayUyQyUyMExvb3BTZXF1ZW50aWFsUGlwZWxpbmVCbG9ja3MpJTIwYW5kJTIwJTIyZGVub2lzZXIlMjIlMjBpbiUyMGJsb2NrLnN1Yl9ibG9ja3MlM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjB5aWVsZCUyMGJsb2NrJTBBJTIwJTIwJTIwJTIwZm9yJTIwc3ViJTIwaW4lMjBnZXRhdHRyKGJsb2NrJTJDJTIwJTIyc3ViX2Jsb2NrcyUyMiUyQyUyMCU3QiU3RCkudmFsdWVzKCklM0ElMEElMjAlMjAlMjAlMjAlMjAlMjAlMjAlMjB5aWVsZCUyMGZyb20lMjB3YWxrX2Rlbm9pc2VfbG9vcHMoc3ViKSUwQSUwQSUwQXJlcG9faWQlMjAlM0QlMjAlMjJzdGFiaWxpdHlhaSUyRnN0YWJsZS1kaWZmdXNpb24teGwtYmFzZS0xLjAlMjIlMEFibG9ja3MlMjAlM0QlMjBNb2R1bGFyUGlwZWxpbmUuZnJvbV9wcmV0cmFpbmVkKHJlcG9faWQpLmJsb2NrcyUwQWZvciUyMGxvb3AlMjBpbiUyMHdhbGtfZGVub2lzZV9sb29wcyhibG9ja3MpJTNBJTBBJTIwJTIwJTIwJTIwbG9vcC5zdWJfYmxvY2tzJTVCJTIyY2FsbGJhY2slMjIlNUQlMjAlM0QlMjBDYWxsYmFja09uU3RlcEVuZCgpJTIwJTIwJTIzJTIwYXBwZW5kZWQlMjBhZnRlciUyMHRoZSUyMGxhc3QlMjBzdGVwJTIwb2YlMjB0aGUlMjBsb29wJTBBcGlwZSUyMCUzRCUyMGJsb2Nrcy5pbml0X3BpcGVsaW5lKHJlcG9faWQpJTBBcGlwZS5sb2FkX2NvbXBvbmVudHMoZHR5cGUlM0R0b3JjaC5mbG9hdDE2KSUwQXBpcGUudG8oJTIyY3VkYSUyMik=",highlighted:`<span class="hljs-keyword">from</span> typing <span class="hljs-keyword">import</span> <span class="hljs-type">Callable</span> | |
| <span class="hljs-keyword">import</span> torch | |
| <span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> ModularPipeline | |
| <span class="hljs-keyword">from</span> diffusers.modular_pipelines <span class="hljs-keyword">import</span> InputParam, LoopSequentialPipelineBlocks, ModularPipelineBlocks | |
| <span class="hljs-keyword">class</span> <span class="hljs-title class_">CallbackOnStepEnd</span>(<span class="hljs-title class_ inherited__">ModularPipelineBlocks</span>): | |
| <span class="hljs-meta"> @property</span> | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">inputs</span>(<span class="hljs-params">self</span>): | |
| <span class="hljs-keyword">return</span> [ | |
| InputParam( | |
| <span class="hljs-string">"callback_on_step_end"</span>, | |
| type_hint=<span class="hljs-type">Callable</span>, | |
| description=<span class="hljs-string">"Called after each scheduler step as \`callback(pipe, i, t, {'latents': ...})\`; the dict it returns is written back."</span>, | |
| ), | |
| InputParam(<span class="hljs-string">"latents"</span>, required=<span class="hljs-literal">True</span>), | |
| ] | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">__call__</span>(<span class="hljs-params">self, components, block_state, i, t</span>): | |
| <span class="hljs-keyword">if</span> block_state.callback_on_step_end <span class="hljs-keyword">is</span> <span class="hljs-keyword">not</span> <span class="hljs-literal">None</span>: | |
| outputs = block_state.callback_on_step_end(components, i, t, {<span class="hljs-string">"latents"</span>: block_state.latents}) | |
| block_state.latents = outputs.get(<span class="hljs-string">"latents"</span>, block_state.latents) | |
| <span class="hljs-keyword">return</span> components, block_state | |
| <span class="hljs-keyword">def</span> <span class="hljs-title function_">walk_denoise_loops</span>(<span class="hljs-params">block</span>): | |
| <span class="hljs-string">"""Every denoising loop in the blocks, whichever workflow it belongs to."""</span> | |
| <span class="hljs-keyword">if</span> <span class="hljs-built_in">isinstance</span>(block, LoopSequentialPipelineBlocks) <span class="hljs-keyword">and</span> <span class="hljs-string">"denoiser"</span> <span class="hljs-keyword">in</span> block.sub_blocks: | |
| <span class="hljs-keyword">yield</span> block | |
| <span class="hljs-keyword">for</span> sub <span class="hljs-keyword">in</span> <span class="hljs-built_in">getattr</span>(block, <span class="hljs-string">"sub_blocks"</span>, {}).values(): | |
| <span class="hljs-keyword">yield</span> <span class="hljs-keyword">from</span> walk_denoise_loops(sub) | |
| repo_id = <span class="hljs-string">"stabilityai/stable-diffusion-xl-base-1.0"</span> | |
| blocks = ModularPipeline.from_pretrained(repo_id).blocks | |
| <span class="hljs-keyword">for</span> loop <span class="hljs-keyword">in</span> walk_denoise_loops(blocks): | |
| loop.sub_blocks[<span class="hljs-string">"callback"</span>] = CallbackOnStepEnd() <span class="hljs-comment"># appended after the last step of the loop</span> | |
| pipe = blocks.init_pipeline(repo_id) | |
| pipe.load_components(dtype=torch.float16) | |
| pipe.to(<span class="hljs-string">"cuda"</span>)`,lang:"py",wrap:!1});var w=s(y,4);l(w,{code:"aW1hZ2UlMjAlM0QlMjBwaXBlKHByb21wdCUzRCUyMkElMjBjcm9pc3NhbnQlMjBzaGFwZWQlMjBsaWtlJTIwYSUyMGN1dGUlMjBiZWFyLiUyMiUyQyUyMGNhbGxiYWNrX29uX3N0ZXBfZW5kJTNEZGVjb2RlX3RlbnNvcnMlMkMlMjBvdXRwdXQlM0QlMjJpbWFnZXMlMjIpJTVCMCU1RA==",highlighted:'image = pipe(prompt=<span class="hljs-string">"A croissant shaped like a cute bear."</span>, callback_on_step_end=decode_tensors, output=<span class="hljs-string">"images"</span>)[<span class="hljs-number">0</span>]',lang:"py",wrap:!1});var U=s(w,6);G(U,{source:"https://github.com/huggingface/diffusers/blob/main/docs/source/en/using-diffusers/callback.md"}),W(2),u(T,e),A()}export{E as component}; | |
Xet Storage Details
- Size:
- 25.3 kB
- Xet hash:
- 3572002c806f7fb625d41f04ca65d267eaed53172c02b7f6d8519e637d65568e
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.