Buckets:

hf-doc-build
/

doc

Files
xet
hf-doc-build/doc / diffusers /main /es /api /pipelines /kandinsky_s2v.html
mishig
1 day ago
download
raw
49.6 kB
<meta charset="utf-8" /><meta name="hf:doc:metadata" content="{&quot;title&quot;:&quot;Kandinsky S2V 1.0&quot;,&quot;local&quot;:&quot;kandinsky-s2v-10&quot;,&quot;sections&quot;:[{&quot;title&quot;:&quot;Puntos de control&quot;,&quot;local&quot;:&quot;checkpoints&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;Uso&quot;,&quot;local&quot;:&quot;usage&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;Notas&quot;,&quot;local&quot;:&quot;notes&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;KandinskyS2VModularPipeline&quot;,&quot;local&quot;:&quot;diffusers.KandinskyS2VModularPipeline&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;KandinskyS2VAutoBlocks&quot;,&quot;local&quot;:&quot;diffusers.KandinskyS2VAutoBlocks&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;KandinskyS2VDistilledModularPipeline&quot;,&quot;local&quot;:&quot;diffusers.KandinskyS2VDistilledModularPipeline&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;KandinskyS2VDistilledAutoBlocks&quot;,&quot;local&quot;:&quot;diffusers.KandinskyS2VDistilledAutoBlocks&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2}],&quot;depth&quot;:1}"/>
		<link href="/docs/diffusers/main/en/_app/immutable/entry/start.D3LnKZ4R.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/chunks/BF9UNEM9.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/chunks/DPnyVmXs.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/entry/app.C0rBoMxR.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/chunks/DSGUT_OW.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/chunks/C045MWWx.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/chunks/DnVIJfC5.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/chunks/DsnmJJEf.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/nodes/0.DTZsP_3B.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/chunks/IaDvMxLQ.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/nodes/197.CNZ1bmB6.js" rel="modulepreload">
		<link href="/docs/diffusers/main/en/_app/immutable/chunks/bKdwVYpU.js" rel="modulepreload">
		<!--1jqdiis--><meta name="hf:doc:metadata" content="{&quot;title&quot;:&quot;Kandinsky S2V 1.0&quot;,&quot;local&quot;:&quot;kandinsky-s2v-10&quot;,&quot;sections&quot;:[{&quot;title&quot;:&quot;Checkpoints&quot;,&quot;local&quot;:&quot;checkpoints&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;Usage&quot;,&quot;local&quot;:&quot;usage&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;Notes&quot;,&quot;local&quot;:&quot;notes&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;KandinskyS2VModularPipeline&quot;,&quot;local&quot;:&quot;diffusers.KandinskyS2VModularPipeline&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;KandinskyS2VAutoBlocks&quot;,&quot;local&quot;:&quot;diffusers.KandinskyS2VAutoBlocks&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;KandinskyS2VDistilledModularPipeline&quot;,&quot;local&quot;:&quot;diffusers.KandinskyS2VDistilledModularPipeline&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2},{&quot;title&quot;:&quot;KandinskyS2VDistilledAutoBlocks&quot;,&quot;local&quot;:&quot;diffusers.KandinskyS2VDistilledAutoBlocks&quot;,&quot;sections&quot;:[],&quot;depth&quot;:2}],&quot;depth&quot;:1}"/><!---->
		<link href="/docs/diffusers/main/en/_app/immutable/assets/0.tn0RQdqM.css" rel="modulepreload"> <!--[--><!--[0--><!--[--><!--[0--><!--[--><p></p> <div class="items-center shrink-0 min-w-[100px] max-sm:min-w-[50px] justify-end ml-auto flex" style="float: inline-end; margin-inline-start: 10px; display: inline-flex; position: relative; z-index: 10;"><div dir="ltr" class="inline-flex rounded-md max-sm:rounded-sm"><button class="inline-flex items-center gap-1 h-7 max-sm:h-7 px-2 max-sm:px-1.5 text-sm font-medium text-gray-800 border border-r-0 rounded-l-md max-sm:rounded-l-sm border-gray-200 bg-white hover:shadow-inner dark:border-gray-850 dark:bg-gray-950 dark:text-gray-200 dark:hover:bg-gray-800" aria-live="polite"><span class="inline-flex items-center justify-center rounded-md p-0.5 max-sm:p-0 hover:text-gray-800 dark:hover:text-gray-200"><svg class="sm:size-3.5 size-3" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg><!----></span> <span dir="auto">Copy page</span></button> <button class="inline-flex items-center justify-center w-6 max-sm:w-5 h-7 max-sm:h-7 disabled:pointer-events-none text-sm text-gray-500 hover:text-gray-700 dark:hover:text-white rounded-r-md max-sm:rounded-r-sm border border-l transition border-gray-200 bg-white hover:shadow-inner dark:border-gray-850 dark:bg-gray-950 dark:text-gray-200 dark:hover:bg-gray-800" aria-haspopup="menu" aria-expanded="false" aria-label="Open copy menu"><svg class="transition-transform text-gray-400 overflow-visible sm:size-3.5 size-3 rotate-0" width="1em" height="1em" viewBox="0 0 12 7" fill="none" xmlns="http://www.w3.org/2000/svg"><path d="M1 1L6 6L11 1" stroke="currentColor"></path></svg><!----></button></div> <!--[-1--><!--]--></div><!----> <!--[0--><h1 class="relative group"><a id="kandinsky-s2v-10" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#kandinsky-s2v-10"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->Kandinsky S2V 1.0<!----></span></h1><!--]--><!----> <p>Kandinsky S2V 1.0 genera un video de avatar a partir de un fotograma de referencia y una pista de audio. La persona en el fotograma se anima para que coincida con el habla: se anima la figura completa, no solo la cabeza. El habla es una condición, no una salida: el modelo no genera audio, y la banda sonora del resultado es la pista que se proporciona.</p> <p>Ambas condiciones pasan por <a href="https://huggingface.co/Qwen/Qwen2.5-Omni-7B" rel="nofollow">Qwen2.5-Omni-7B</a>:
sus estados ocultos codifican el prompt y su torre de audio codifica el habla, que
entra en el transformer a través de adaptadores de atención cruzada alineados cuadro a cuadro, de modo
que cada cuadro latente atiende solo al habla que cubre. Un clip se genera en
fragmentos de 5 s (125 cuadros a 25 fps), cada uno con memoria FramePack de los
anteriores, para que las pistas más largas mantengan la coherencia en los límites entre fragmentos.</p> <p>Kandinsky S2V 1.0 se integra únicamente como <a href="../../modular_diffusers/overview">pipeline modular</a>; no existe una clase estándar <code>DiffusionPipeline</code> para él.</p> <!--[1--><h2 class="relative group"><a id="checkpoints" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#checkpoints"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->Puntos de control<!----></span></h2><!--]--><!----> <table><thead><tr><th>Checkpoint</th><th>Transformer</th><th>Bloques</th><th>Programador</th><th>Pasos</th><th>Orientación</th></tr></thead><tbody><tr><td><a href="https://huggingface.co/kandinskylab/Kandinsky-S2V-1.0-Lite-Diffusers" rel="nofollow">Lite</a></td><td>2B</td><td><a href="/docs/diffusers/main/es/api/pipelines/kandinsky_s2v#diffusers.KandinskyS2VAutoBlocks">KandinskyS2VAutoBlocks</a></td><td><a href="/docs/diffusers/main/es/api/schedulers/flow_match_euler_discrete#diffusers.FlowMatchEulerDiscreteScheduler">FlowMatchEulerDiscreteScheduler</a></td><td>50</td><td>3.0</td></tr><tr><td><a href="https://huggingface.co/kandinskylab/Kandinsky-S2V-1.0-Pro-Diffusers" rel="nofollow">Pro</a></td><td>21B</td><td><a href="/docs/diffusers/main/es/api/pipelines/kandinsky_s2v#diffusers.KandinskyS2VAutoBlocks">KandinskyS2VAutoBlocks</a></td><td><a href="/docs/diffusers/main/es/api/schedulers/flow_match_euler_discrete#diffusers.FlowMatchEulerDiscreteScheduler">FlowMatchEulerDiscreteScheduler</a></td><td>50</td><td>3.0</td></tr><tr><td><a href="https://huggingface.co/kandinskylab/Kandinsky-S2V-1.0-Lite-distill-Diffusers" rel="nofollow">Destilado Lite</a></td><td>2B</td><td><a href="/docs/diffusers/main/es/api/pipelines/kandinsky_s2v#diffusers.KandinskyS2VDistilledAutoBlocks">KandinskyS2VDistilledAutoBlocks</a></td><td><a href="/docs/diffusers/main/es/api/schedulers/piflow#diffusers.PiflowScheduler">PiflowScheduler</a></td><td>16</td><td>off</td></tr><tr><td><a href="https://huggingface.co/kandinskylab/Kandinsky-S2V-1.0-Pro-distill-Diffusers" rel="nofollow">Pro destilado</a></td><td>21B</td><td><a href="/docs/diffusers/main/es/api/pipelines/kandinsky_s2v#diffusers.KandinskyS2VDistilledAutoBlocks">KandinskyS2VDistilledAutoBlocks</a></td><td><a href="/docs/diffusers/main/es/api/schedulers/piflow#diffusers.PiflowScheduler">PiflowScheduler</a></td><td>16</td><td>off</td></tr></tbody></table> <p>Los puntos de control destilados configuran <code>PiflowScheduler</code> con <code>shift=10.0</code> y <code>n_grid=5</code>, no con sus valores predeterminados.</p> <p><code>ModularPipeline.from_pretrained</code> selecciona los bloques del checkpoint. Los pasos
por defecto son los valores anteriores. La guía sin clasificador es el componente <code>guider</code>,
no un argumento de llamada: los checkpoints base lo incluyen en 3.0. Los destilados no
tienen guider ni <code>negative_prompt</code>, porque la guía ya forma parte del estudiante.</p> <!--[1--><h2 class="relative group"><a id="usage" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#usage"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->Uso<!----></span></h2><!--]--><!----> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5   text-gray-600  " title="code excerpt" type="button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg><!----> <div class=" absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0 "><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent;"></div> Copiado</div><!----></button><!----></div> <pre class="language-py "><!----><span class="hljs-keyword">import</span> librosa
<span class="hljs-keyword">import</span> torch
<span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> ModularPipeline
<span class="hljs-keyword">from</span> diffusers.utils <span class="hljs-keyword">import</span> encode_video, load_image

pipe = ModularPipeline.from_pretrained(<span class="hljs-string">&quot;kandinskylab/Kandinsky-S2V-1.0-Lite-distill-Diffusers&quot;</span>)
pipe.load_components(dtype=torch.bfloat16)
pipe.to(<span class="hljs-string">&quot;cuda&quot;</span>)

image = load_image(<span class="hljs-string">&quot;reference.png&quot;</span>)
audio, _ = librosa.load(<span class="hljs-string">&quot;speech.wav&quot;</span>, sr=<span class="hljs-number">16000</span>)  <span class="hljs-comment"># a mono waveform at 16 kHz, for the model</span>
speech, sample_rate = librosa.load(<span class="hljs-string">&quot;speech.wav&quot;</span>, sr=<span class="hljs-literal">None</span>)  <span class="hljs-comment"># the original track, added to the output video</span>

video = pipe(image=image, audio=audio, output=<span class="hljs-string">&quot;videos&quot;</span>)[<span class="hljs-number">0</span>]
encode_video(
    video,
    fps=<span class="hljs-number">25</span>,
    output_path=<span class="hljs-string">&quot;avatar.mp4&quot;</span>,
    audio=torch.from_numpy(speech)[<span class="hljs-literal">None</span>],
    audio_sample_rate=sample_rate,
)<!----></pre></div><!----> <p>Los checkpoints de Pro no caben en una GPU de 80 GB junto al codificador de voz y texto de 7B cuando todos los componentes están en el dispositivo. Descárguelos a través de un <code>ComponentsManager</code>:</p> <div class="code-block relative "><div class="absolute top-2.5 right-4"><button class="inline-flex items-center relative text-sm focus:text-green-500 cursor-pointer focus:outline-none transition duration-200 ease-in-out opacity-0 mx-0.5   text-gray-600  " title="code excerpt" type="button"><svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M28,10V28H10V10H28m0-2H10a2,2,0,0,0-2,2V28a2,2,0,0,0,2,2H28a2,2,0,0,0,2-2V10a2,2,0,0,0-2-2Z" transform="translate(0)"></path><path d="M4,18H2V4A2,2,0,0,1,4,2H18V4H4Z" transform="translate(0)"></path><rect fill="none" width="32" height="32"></rect></svg><!----> <div class=" absolute pointer-events-none transition-opacity bg-black text-white py-1 px-2 leading-tight rounded font-normal shadow left-1/2 top-full transform -translate-x-1/2 translate-y-2 opacity-0 "><div class="absolute bottom-full left-1/2 transform -translate-x-1/2 w-0 h-0 border-black border-4 border-t-0" style="border-left-color: transparent; border-right-color: transparent;"></div> Copiado</div><!----></button><!----></div> <pre class="language-py "><!----><span class="hljs-keyword">import</span> librosa
<span class="hljs-keyword">import</span> torch
<span class="hljs-keyword">from</span> diffusers <span class="hljs-keyword">import</span> ComponentsManager, ModularPipeline
<span class="hljs-keyword">from</span> diffusers.utils <span class="hljs-keyword">import</span> encode_video, load_image

manager = ComponentsManager()
pipe = ModularPipeline.from_pretrained(<span class="hljs-string">&quot;kandinskylab/Kandinsky-S2V-1.0-Pro-Diffusers&quot;</span>, components_manager=manager)
pipe.load_components(dtype=torch.bfloat16)
manager.enable_auto_cpu_offload(device=<span class="hljs-string">&quot;cuda&quot;</span>, memory_reserve_margin=<span class="hljs-string">&quot;40GiB&quot;</span>)

image = load_image(<span class="hljs-string">&quot;reference.png&quot;</span>)
audio, _ = librosa.load(<span class="hljs-string">&quot;speech.wav&quot;</span>, sr=<span class="hljs-number">16000</span>)
speech, sample_rate = librosa.load(<span class="hljs-string">&quot;speech.wav&quot;</span>, sr=<span class="hljs-literal">None</span>)

video = pipe(image=image, audio=audio, output=<span class="hljs-string">&quot;videos&quot;</span>)[<span class="hljs-number">0</span>]
encode_video(
    video,
    fps=<span class="hljs-number">25</span>,
    output_path=<span class="hljs-string">&quot;avatar.mp4&quot;</span>,
    audio=torch.from_numpy(speech)[<span class="hljs-literal">None</span>],
    audio_sample_rate=sample_rate,
)<!----></pre></div><!----> <p>El margen es la memoria que se mantiene libre además de los pesos cuando un modelo se mueve a la GPU.
Debe contener la decodificación VAE de un fragmento, aproximadamente 33 GiB a 512x768: con los 3 GB
predeterminados, el transformer permanece en la GPU durante la decodificación, lo que agota la memoria. Con
40 GiB, el pico es de unos 50 GiB en una GPU de 80 GB. Habilite el offload después de <code>load_components</code>,
para que se aplique a cada componente.</p> <!--[1--><h2 class="relative group"><a id="notes" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#notes"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->Notas<!----></span></h2><!--]--><!----> <ul><li><code>image</code> es una imagen PIL y <code>audio</code> una forma de onda mono flotante en [-1, 1] a 16 kHz, como tensor o matriz con forma <code>(samples,)</code>; la pipeline no lee archivos.</li> <li>Los bloques de nivel superior (<code>text_encoder</code>, <code>vae_encoder</code>, <code>denoise</code>, <code>decode</code>) también se ejecutan de forma independiente;
pasa el mismo <code>num_latent_frames</code> y <code>fps</code> a cada uno cuando los ejecutes por separado.</li> <li>El número de fragmentos sigue la duración de la pista de audio a menos que se pase <code>num_chunks</code>. De todos modos, el video se recorta a la pista, por lo que los fragmentos que exceden su final se calculan y se descartan, con una advertencia.</li> <li>Con la guía, el paso incondicional sustituye el prompt negativo <em>y</em> silencia el habla, por lo que la guía es conjunta sobre el texto y el habla.</li></ul> <!--[1--><h2 class="relative group"><a id="diffusers.KandinskyS2VModularPipeline" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#diffusers.KandinskyS2VModularPipeline"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->KandinskyS2VModularPipeline<!----></span></h2><!--]--><!----> <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"><div><span class="group flex space-x-1.5 items-center text-gray-800 bg-gradient-to-r rounded-tr-lg -mt-4 -ml-4 pt-3 px-2.5" id="diffusers.KandinskyS2VModularPipeline"><!----><h3 class="!m-0"><span class="flex-1 break-all md:text-lg bg-gradient-to-r px-2.5 py-1.5 rounded-xl from-indigo-50/70 to-white dark:from-gray-900 dark:to-gray-950 dark:text-indigo-300 text-indigo-700"><svg class="mr-1.5 text-indigo-500 inline-block -mt-0.5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" focusable="false" role="img" width=".8em" height=".8em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 24 24"><path class="uim-quaternary" d="M20.23 7.24L12 12L3.77 7.24a1.98 1.98 0 0 1 .7-.71L11 2.76c.62-.35 1.38-.35 2 0l6.53 3.77c.29.173.531.418.7.71z" opacity=".25" fill="currentColor"></path><path class="uim-tertiary" d="M12 12v9.5a2.09 2.09 0 0 1-.91-.21L4.5 17.48a2.003 2.003 0 0 1-1-1.73v-7.5a2.06 2.06 0 0 1 .27-1.01L12 12z" opacity=".5" fill="currentColor"></path><path class="uim-primary" d="M20.5 8.25v7.5a2.003 2.003 0 0 1-1 1.73l-6.62 3.82c-.275.13-.576.198-.88.2V12l8.23-4.76c.175.308.268.656.27 1.01z" fill="currentColor"></path></svg><span class="font-light">clase</span> <span class="font-medium">diffusers.</span><span class="font-semibold">KandinskyS2VModularPipeline</span></span></h3><!----> <a id="diffusers.KandinskyS2VModularPipeline" class="header-link invisible with-hover:group-hover:visible pr-2" href="#diffusers.KandinskyS2VModularPipeline"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></a> <!--[0--><a class="!ml-auto !text-gray-400 !no-underline text-sm flex items-center" href="https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/kandinsky_s2v/modular_pipeline.py#L91" target="_blank"><span>&lt;</span> <span class="hidden md:block mx-0.5 hover:!underline">source</span> <span>></span></a><!--]--></span> <!--[0--><p class="font-mono text-xs md:text-sm !leading-relaxed !my-6"><span>(</span> <!--[--><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">blocks<span class="opacity-60">: diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">pretrained_model_name_or_path<span class="opacity-60">: str | os.PathLike | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">components_manager<span class="opacity-60">: diffusers.modular_pipelines.components_manager.ComponentsManager | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">collection<span class="opacity-60">: str | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">workflow<span class="opacity-60">: str | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">modular_config_dict<span class="opacity-60">: dict[str, typing.Any] | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">config_dict<span class="opacity-60">: dict[str, typing.Any] | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">**kwargs<span class="opacity-60"></span></span></span><!--]--> <span>)</span> <!--[-1--><!--]--></p><!--]--> <div class="!mb-10 relative docstring-details "><!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--></div></div><!----> <p>Un ModularPipeline para la generación de video a partir de voz de Kandinsky S2V 1.0.</p></div> <!--[1--><h2 class="relative group"><a id="diffusers.KandinskyS2VAutoBlocks" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#diffusers.KandinskyS2VAutoBlocks"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->KandinskyS2VAutoBlocks<!----></span></h2><!--]--><!----> <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"><div><span class="group flex space-x-1.5 items-center text-gray-800 bg-gradient-to-r rounded-tr-lg -mt-4 -ml-4 pt-3 px-2.5" id="diffusers.KandinskyS2VAutoBlocks"><!----><h3 class="!m-0"><span class="flex-1 break-all md:text-lg bg-gradient-to-r px-2.5 py-1.5 rounded-xl from-indigo-50/70 to-white dark:from-gray-900 dark:to-gray-950 dark:text-indigo-300 text-indigo-700"><svg class="mr-1.5 text-indigo-500 inline-block -mt-0.5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" focusable="false" role="img" width=".8em" height=".8em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 24 24"><path class="uim-quaternary" d="M20.23 7.24L12 12L3.77 7.24a1.98 1.98 0 0 1 .7-.71L11 2.76c.62-.35 1.38-.35 2 0l6.53 3.77c.29.173.531.418.7.71z" opacity=".25" fill="currentColor"></path><path class="uim-tertiary" d="M12 12v9.5a2.09 2.09 0 0 1-.91-.21L4.5 17.48a2.003 2.003 0 0 1-1-1.73v-7.5a2.06 2.06 0 0 1 .27-1.01L12 12z" opacity=".5" fill="currentColor"></path><path class="uim-primary" d="M20.5 8.25v7.5a2.003 2.003 0 0 1-1 1.73l-6.62 3.82c-.275.13-.576.198-.88.2V12l8.23-4.76c.175.308.268.656.27 1.01z" fill="currentColor"></path></svg><span class="font-light">clase</span> <span class="font-medium">diffusers.</span><span class="font-semibold">KandinskyS2VAutoBlocks</span></span></h3><!----> <a id="diffusers.KandinskyS2VAutoBlocks" class="header-link invisible with-hover:group-hover:visible pr-2" href="#diffusers.KandinskyS2VAutoBlocks"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></a> <!--[0--><a class="!ml-auto !text-gray-400 !no-underline text-sm flex items-center" href="https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/kandinsky_s2v/modular_blocks_kandinsky_s2v.py#L108" target="_blank"><span>&lt;</span> <span class="hidden md:block mx-0.5 hover:!underline">source</span> <span>></span></a><!--]--></span> <!--[0--><p class="font-mono text-xs md:text-sm !leading-relaxed !my-6"><span>(</span> <!--[--><!--]--> <span>)</span> <!--[-1--><!--]--></p><!--]--> <div class="!mb-10 relative docstring-details "><!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--></div></div><!----> <p>Pipeline modular para la generación de video a partir de voz con Kandinsky S2V 1.0: un video de avatar a partir de un fotograma de referencia y una pista de voz, generado por fragmentos.</p> <p>Componentes:
text_encoder (<code>Qwen2_5OmniThinkerForConditionalGeneration</code>) tokenizer (<code>Qwen2_5OmniProcessor</code>) text_encoder_2
(<code>CLIPTextModel</code>) tokenizer_2 (<code>CLIPTokenizer</code>) vae (<code>AutoencoderKLHunyuanVideo</code>) video_processor
(<code>VideoProcessor</code>) transformer (<code>KandinskyS2VTransformer3DModel</code>) scheduler
(<code>FlowMatchEulerDiscreteScheduler</code>) guider (<code>ClassifierFreeGuidance</code>)</p> <p>Entradas:
prompt (<code>str</code>, <em>opcional</em>, predeterminado: The video presents a stable medium close-up of a subject facing the
camera directly. The subject exhibits natural, lifelike body dynamics: the head and shoulders move fluidly
with the rhythm of speech. The lighting is even, maintaining a balanced natural exposure to clearly define
facial features without washout. Throughout the sequence, the subject engages in continuous speech marked by
precise, high-fidelity articulation. The lips move in a rhythmic, natural pattern: alternating between brief
firm closures and relaxed openings. During these openings, the tips of the upper teeth are momentarily and
realistically visible, exhibiting natural enamel texture. The upper lip lifts subtly over the teeth during
enunciation, while the jaw performs minimal vertical oscillations synchronized with the speech cadence.
Facial micro-expressions are active yet restrained; eyebrows rise and lower slightly with minute asymmetry.
The eyes maintain a steady, engaged gaze forward, punctuated by smooth, deliberate blinks. ):
El prompt para guiar la generación.
negative_prompt (<code>str</code>, <em>opcional</em>):
El prompt para el paso incondicional de la guía. Predeterminado a <code>DEFAULT_NEGATIVE_PROMPT</code> cuando la
guía lo necesita.
audio (<code>Tensor | ndarray</code>):
La pista de voz que impulsa la generación: una forma de onda mono flotante en [-1, 1] a 16000 Hz, con forma
(samples,).
num_latent_frames (<code>int</code>, <em>opcional</em>, predeterminado: 32):
Marcos latentes por bloque; 32 da 125 marcos (5 s a 25 fps).
fps (<code>int</code>, <em>opcional</em>, predeterminado: 25):
Tasa de fotogramas a la que se calcula la longitud del bloque.
num_chunks (<code>int</code>, <em>opcional</em>):
Número de bloques a generar. Predeterminado a los necesarios para cubrir la pista de voz; el video se recorta
a la pista de todos modos.
image (<code>Image</code>):
El marco de referencia para animar.
height (<code>int</code>, <em>opcional</em>):
La altura en píxeles del video generado. Predeterminado al cubo de 512 más cercano al marco de referencia.
width (<code>int</code>, <em>opcional</em>):
El ancho en píxeles del video generado, elegido junto con <code>height</code>.
generator (<code>Generator</code>, <em>opcional</em>):
Generador de Torch para generación determinista.
num_inference_steps (<code>int</code>, <em>opcional</em>, predeterminado: 50):
El número de pasos de desruido.
scale_factor (<code>tuple</code>, <em>opcional</em>, predeterminado: (1.0, 2.0, 2.0)):
Escala RoPE por eje para el cubo de resolución.
output_type (<code>str</code>, <em>opcional</em>, predeterminado: pil):
Formato de salida: ‘pil’, ‘np’, ‘pt’.</p> <p>Salidas:
videos (<code>list</code>):
Los videos generados.</p></div> <!--[1--><h2 class="relative group"><a id="diffusers.KandinskyS2VDistilledModularPipeline" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#diffusers.KandinskyS2VDistilledModularPipeline"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->KandinskyS2VDistilledModularPipeline<!----></span></h2><!--]--><!----> <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"><div><span class="group flex space-x-1.5 items-center text-gray-800 bg-gradient-to-r rounded-tr-lg -mt-4 -ml-4 pt-3 px-2.5" id="diffusers.KandinskyS2VDistilledModularPipeline"><!----><h3 class="!m-0"><span class="flex-1 break-all md:text-lg bg-gradient-to-r px-2.5 py-1.5 rounded-xl from-indigo-50/70 to-white dark:from-gray-900 dark:to-gray-950 dark:text-indigo-300 text-indigo-700"><svg class="mr-1.5 text-indigo-500 inline-block -mt-0.5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" focusable="false" role="img" width=".8em" height=".8em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 24 24"><path class="uim-quaternary" d="M20.23 7.24L12 12L3.77 7.24a1.98 1.98 0 0 1 .7-.71L11 2.76c.62-.35 1.38-.35 2 0l6.53 3.77c.29.173.531.418.7.71z" opacity=".25" fill="currentColor"></path><path class="uim-tertiary" d="M12 12v9.5a2.09 2.09 0 0 1-.91-.21L4.5 17.48a2.003 2.003 0 0 1-1-1.73v-7.5a2.06 2.06 0 0 1 .27-1.01L12 12z" opacity=".5" fill="currentColor"></path><path class="uim-primary" d="M20.5 8.25v7.5a2.003 2.003 0 0 1-1 1.73l-6.62 3.82c-.275.13-.576.198-.88.2V12l8.23-4.76c.175.308.268.656.27 1.01z" fill="currentColor"></path></svg><span class="font-light">clase</span> <span class="font-medium">diffusers.</span><span class="font-semibold">KandinskyS2VDistilledModularPipeline</span></span></h3><!----> <a id="diffusers.KandinskyS2VDistilledModularPipeline" class="header-link invisible with-hover:group-hover:visible pr-2" href="#diffusers.KandinskyS2VDistilledModularPipeline"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></a> <!--[0--><a class="!ml-auto !text-gray-400 !no-underline text-sm flex items-center" href="https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/kandinsky_s2v/modular_pipeline.py#L109" target="_blank"><span>&lt;</span> <span class="hidden md:block mx-0.5 hover:!underline">source</span> <span>></span></a><!--]--></span> <!--[0--><p class="font-mono text-xs md:text-sm !leading-relaxed !my-6"><span>(</span> <!--[--><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">blocks<span class="opacity-60">: diffusers.modular_pipelines.modular_pipeline.ModularPipelineBlocks | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">pretrained_model_name_or_path<span class="opacity-60">: str | os.PathLike | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">components_manager<span class="opacity-60">: diffusers.modular_pipelines.components_manager.ComponentsManager | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">collection<span class="opacity-60">: str | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">workflow<span class="opacity-60">: str | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">modular_config_dict<span class="opacity-60">: dict[str, typing.Any] | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">config_dict<span class="opacity-60">: dict[str, typing.Any] | None = None</span></span></span><span class="comma cursor-default"><span class="rounded hover:bg-black hover:text-white dark:hover:bg-white dark:hover:text-black">**kwargs<span class="opacity-60"></span></span></span><!--]--> <span>)</span> <!--[-1--><!--]--></p><!--]--> <div class="!mb-10 relative docstring-details "><!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--></div></div><!----> <p>Un ModularPipeline para los puntos de control destilados de pi-Flow de Kandinsky S2V 1.0.</p> <p>La guía se destila en el estudiante, por lo que sus bloques no tienen guía ni prompt negativo, y se ejecuta una sola pasada de adelante por segmento.</p></div> <!--[1--><h2 class="relative group"><a id="diffusers.KandinskyS2VDistilledAutoBlocks" class="header-link block pr-1.5 text-lg no-hover:hidden with-hover:absolute with-hover:p-1.5 with-hover:opacity-0 with-hover:group-hover:opacity-100 with-hover:right-full" href="#diffusers.KandinskyS2VDistilledAutoBlocks"><span><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></span></a> <span><!---->KandinskyS2VDistilledAutoBlocks<!----></span></h2><!--]--><!----> <div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8"><div><span class="group flex space-x-1.5 items-center text-gray-800 bg-gradient-to-r rounded-tr-lg -mt-4 -ml-4 pt-3 px-2.5" id="diffusers.KandinskyS2VDistilledAutoBlocks"><!----><h3 class="!m-0"><span class="flex-1 break-all md:text-lg bg-gradient-to-r px-2.5 py-1.5 rounded-xl from-indigo-50/70 to-white dark:from-gray-900 dark:to-gray-950 dark:text-indigo-300 text-indigo-700"><svg class="mr-1.5 text-indigo-500 inline-block -mt-0.5" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" focusable="false" role="img" width=".8em" height=".8em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 24 24"><path class="uim-quaternary" d="M20.23 7.24L12 12L3.77 7.24a1.98 1.98 0 0 1 .7-.71L11 2.76c.62-.35 1.38-.35 2 0l6.53 3.77c.29.173.531.418.7.71z" opacity=".25" fill="currentColor"></path><path class="uim-tertiary" d="M12 12v9.5a2.09 2.09 0 0 1-.91-.21L4.5 17.48a2.003 2.003 0 0 1-1-1.73v-7.5a2.06 2.06 0 0 1 .27-1.01L12 12z" opacity=".5" fill="currentColor"></path><path class="uim-primary" d="M20.5 8.25v7.5a2.003 2.003 0 0 1-1 1.73l-6.62 3.82c-.275.13-.576.198-.88.2V12l8.23-4.76c.175.308.268.656.27 1.01z" fill="currentColor"></path></svg><span class="font-light">clase</span> <span class="font-medium">diffusers.</span><span class="font-semibold">KandinskyS2VDistilledAutoBlocks</span></span></h3><!----> <a id="diffusers.KandinskyS2VDistilledAutoBlocks" class="header-link invisible with-hover:group-hover:visible pr-2" href="#diffusers.KandinskyS2VDistilledAutoBlocks"><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 256"><path d="M167.594 88.393a8.001 8.001 0 0 1 0 11.314l-67.882 67.882a8 8 0 1 1-11.314-11.315l67.882-67.881a8.003 8.003 0 0 1 11.314 0zm-28.287 84.86l-28.284 28.284a40 40 0 0 1-56.567-56.567l28.284-28.284a8 8 0 0 0-11.315-11.315l-28.284 28.284a56 56 0 0 0 79.196 79.197l28.285-28.285a8 8 0 1 0-11.315-11.314zM212.852 43.14a56.002 56.002 0 0 0-79.196 0l-28.284 28.284a8 8 0 1 0 11.314 11.314l28.284-28.284a40 40 0 0 1 56.568 56.567l-28.285 28.285a8 8 0 0 0 11.315 11.314l28.284-28.284a56.065 56.065 0 0 0 0-79.196z" fill="currentColor"></path></svg><!----></a> <!--[0--><a class="!ml-auto !text-gray-400 !no-underline text-sm flex items-center" href="https://github.com/huggingface/diffusers/blob/main/src/diffusers/modular_pipelines/kandinsky_s2v/modular_blocks_kandinsky_s2v_distilled.py#L103" target="_blank"><span>&lt;</span> <span class="hidden md:block mx-0.5 hover:!underline">source</span> <span>></span></a><!--]--></span> <!--[0--><p class="font-mono text-xs md:text-sm !leading-relaxed !my-6"><span>(</span> <!--[--><!--]--> <span>)</span> <!--[-1--><!--]--></p><!--]--> <div class="!mb-10 relative docstring-details "><!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--> <!--[-1--><!--]--></div></div><!----> <p>Pipeline modular para la generación de video a partir de voz con los puntos de control destilados de pi-Flow Kandinsky S2V 1.0: 16 segmentos por bloque y sin guía, lo cual se destila en el estudiante.</p> <p>Componentes:
text_encoder (<code>Qwen2_5OmniThinkerForConditionalGeneration</code>) tokenizer (<code>Qwen2_5OmniProcessor</code>) text_encoder_2
(<code>CLIPTextModel</code>) tokenizer_2 (<code>CLIPTokenizer</code>) vae (<code>AutoencoderKLHunyuanVideo</code>) video_processor
(<code>VideoProcessor</code>) transformer (<code>KandinskyS2VTransformer3DModel</code>) scheduler (<code>PiflowScheduler</code>)</p> <p>Entradas:
prompt (<code>str</code>, <em>opcional</em>, predeterminado: The video presents a stable medium close-up of a subject facing the
camera directly. The subject exhibits natural, lifelike body dynamics: the head and shoulders move fluidly
with the rhythm of speech. The lighting is even, maintaining a balanced natural exposure to clearly define
facial features without washout. Throughout the sequence, the subject engages in continuous speech marked by
precise, high-fidelity articulation. The lips move in a rhythmic, natural pattern: alternating between brief
firm closures and relaxed openings. During these openings, the tips of the upper teeth are momentarily and
realistically visible, exhibiting natural enamel texture. The upper lip lifts subtly over the teeth during
enunciation, while the jaw performs minimal vertical oscillations synchronized with the speech cadence.
Facial micro-expressions are active yet restrained; eyebrows rise and lower slightly with minute asymmetry.
The eyes maintain a steady, engaged gaze forward, punctuated by smooth, deliberate blinks. ):
El prompt para guiar la generación.
audio (<code>Tensor | ndarray</code>):
La pista de voz que impulsa la generación: una forma de onda mono flotante en [-1, 1] a 16000 Hz, con forma
(samples,).
num_latent_frames (<code>int</code>, <em>opcional</em>, predeterminado: 32):
Marcos latentes por bloque; 32 da 125 marcos (5 s a 25 fps).
fps (<code>int</code>, <em>opcional</em>, predeterminado: 25):
Tasa de fotogramas a la que se calcula la longitud del bloque.
num_chunks (<code>int</code>, <em>opcional</em>):
Número de bloques a generar. El valor predeterminado es el necesario para cubrir la pista de voz; el video se recorta a la pista de todos modos.
image (<code>Image</code>):
El fotograma de referencia para animar.
height (<code>int</code>, <em>opcional</em>):
La altura en píxeles del video generado. El valor predeterminado es el cubo de 512 más cercano al fotograma de referencia.
width (<code>int</code>, <em>opcional</em>):
El ancho en píxeles del video generado, elegido junto con <code>height</code>.
generator (<code>Generator</code>, <em>opcional</em>):
Generador de Torch para una generación determinista.
num_inference_steps (<code>int</code>, <em>opcional</em>, predeterminado: 16):
El número de pasos de desruido.
scale_factor (<code>tuple</code>, <em>opcional</em>, predeterminado: (1.0, 2.0, 2.0)):
Escala RoPE por eje para el cubo de resolución.
output_type (<code>str</code>, <em>opcional</em>, predeterminado: pil):
Formato de salida: ‘pil’, ‘np’, ‘pt’.</p> <p>Salidas:
videos (<code>list</code>):
Los videos generados.</p></div> <a class="!text-gray-400 !no-underline text-sm flex items-center not-prose mt-4" href="https://github.com/huggingface/diffusers/blob/main/docs/source/en/api/pipelines/kandinsky_s2v.md" target="_blank"><svg class="mr-1" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" fill="currentColor" focusable="false" role="img" width="1em" height="1em" preserveAspectRatio="xMidYMid meet" viewBox="0 0 32 32"><path d="M31,16l-7,7l-1.41-1.41L28.17,16l-5.58-5.59L24,9l7,7z"></path><path d="M1,16l7-7l1.41,1.41L3.83,16l5.58,5.59L8,23l-7-7z"></path><path d="M12.419,25.484L17.639,6.552l1.932,0.518L14.351,26.002z"></path></svg><!----> <span><span class="underline">Update</span> on GitHub</span></a><!----> <p></p><!--]--><!----><!--]--><!--]--><!--]--> <!--[-1--><!--]--><!--]-->
			
			<script>
				{
					__sveltekit_g37pnb = {
						base: "/docs/diffusers/main/en",
						assets: "/docs/diffusers/main/en"
					};
					const element = document.currentScript.parentElement;
					Promise.all([
						import("/docs/diffusers/main/en/_app/immutable/entry/start.D3LnKZ4R.js"),
						import("/docs/diffusers/main/en/_app/immutable/entry/app.C0rBoMxR.js")
					]).then(([kit, app]) => {
						kit.start(app, element, {
							node_ids: [0, 197],
							data: [null,null],
							form: null,
							error: null
						});
					});
				}
			</script>
		
Xet Storage Details

Size:: 49.6 kB
Xet hash:: 062d0fd800496febd8cdf8e1ec1816654e9c7ff6c6395f973a1adf6c1c6454ed
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.