Buckets:
| # HunyuanDiT2DControlNetModel | |
| HunyuanDiT2DControlNetModel es una implementación de ControlNet para [Hunyuan-DiT](https://huggingface.co/papers/2405.08748). | |
| ControlNet se introdujo en [Adding Conditional Control to Text-to-Image Diffusion Models](https://huggingface.co/papers/2302.05543) por Lvmin Zhang, Anyi Rao y Maneesh Agrawala. | |
| Con un modelo de ControlNet, puedes proporcionar una imagen de control adicional para condicionar y controlar la generación de Hunyuan-DiT. Por ejemplo, si proporcionas un mapa de profundidad, el modelo de ControlNet genera una imagen que preservará la información espacial del mapa de profundidad. Es una forma más flexible y precisa de controlar el proceso de generación de imágenes. | |
| El resumen del artículo es: | |
| *Presentamos ControlNet, una arquitectura de red neuronal para añadir controles de condicionamiento espacial a grandes modelos de difusión de texto a imagen preentrenados. ControlNet bloquea los grandes modelos de difusión listos para producción y reutiliza sus capas de codificación profundas y robustas, preentrenadas con miles de millones de imágenes, como una base sólida para aprender un conjunto diverso de controles condicionales. La arquitectura neuronal se conecta mediante "convoluciones cero" (capas de convolución inicializadas en cero) que hacen crecer progresivamente los parámetros desde cero y garantizan que ningún ruido perjudicial pueda afectar el ajuste fino. Probamos varios controles de condicionamiento, por ejemplo, bordes, profundidad, segmentación, pose humana, etc., con Stable Diffusion, utilizando una o varias condiciones, con o sin indicaciones. Mostramos que el entrenamiento de ControlNets es robusto con conjuntos de datos pequeños (<50k) y grandes (>1m). Resultados extensos muestran que ControlNet puede facilitar aplicaciones más amplias para controlar los modelos de difusión de imágenes.* | |
| Este código está implementado por el equipo Tencent Hunyuan. Puedes encontrar puntos de control preentrenados para los ControlNets de Hunyuan-DiT en [Tencent Hunyuan](https://huggingface.co/Tencent-Hunyuan). | |
| ## Ejemplo de carga de HunyuanDiT2DControlNetModel | |
| ```py | |
| from diffusers import HunyuanDiT2DControlNetModel | |
| import torch | |
| controlnet = HunyuanDiT2DControlNetModel.from_pretrained("Tencent-Hunyuan/HunyuanDiT-v1.1-ControlNet-Diffusers-Pose", dtype=torch.float16) | |
| ``` | |
| ## HunyuanDiT2DControlNetModel[[diffusers.HunyuanDiT2DControlNetModel]] | |
| #### diffusers.HunyuanDiT2DControlNetModel[[diffusers.HunyuanDiT2DControlNetModel]] | |
| ```python | |
| diffusers.HunyuanDiT2DControlNetModel(conditioning_channels: int = 3, num_attention_heads: int = 16, attention_head_dim: int = 88, in_channels: int | None = None, patch_size: int | None = None, activation_fn: str = 'gelu-approximate', sample_size = 32, hidden_size = 1152, transformer_num_layers: int = 40, mlp_ratio: float = 4.0, cross_attention_dim: int = 1024, cross_attention_dim_t5: int = 2048, pooled_projection_dim: int = 1024, text_len: int = 77, text_len_t5: int = 256, use_style_cond_and_image_meta_size: bool = True) | |
| ``` | |
| [Fuente](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/controlnets/controlnet_hunyuan.py#L40) | |
| #### forward[[diffusers.HunyuanDiT2DControlNetModel.forward]] | |
| ```python | |
| forward(hidden_states, timestep, controlnet_cond: Tensor, conditioning_scale: float = 1.0, encoder_hidden_states = None, text_embedding_mask = None, encoder_hidden_states_t5 = None, text_embedding_mask_t5 = None, image_meta_size = None, style = None, image_rotary_emb = None, return_dict = True) | |
| ``` | |
| [Fuente](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/controlnets/controlnet_hunyuan.py#L215) | |
| **Parámetros:** | |
| hidden_states (`torch.Tensor` de forma `(batch size, dim, height, width)`) : El tensor de entrada. | |
| timestep ( `torch.LongTensor`, *opcional*) : Se utiliza para indicar el paso de desruido. | |
| controlnet_cond ( `torch.Tensor` ) : La entrada de condicionamiento para ControlNet. | |
| conditioning_scale ( `float` ) : Indica la escala de condicionamiento. | |
| encoder_hidden_states ( `torch.Tensor` de forma `(batch size, sequence len, embed dims)`, *opcional*) : Incrustaciones condicionales para la capa de atención cruzada. Esta es la salida de `BertModel`. | |
| text_embedding_mask : torch.Tensor Una máscara de atención de forma `(batch, key_tokens)` se aplica a `encoder_hidden_states`. Esta es la salida de `BertModel`. | |
| encoder_hidden_states_t5 ( `torch.Tensor` de forma `(batch size, sequence len, embed dims)`, *opcional*) : Incrustaciones condicionales para la capa de atención cruzada. Esta es la salida del codificador de texto T5. | |
| text_embedding_mask_t5 : torch.Tensor Una máscara de atención de forma `(batch, key_tokens)` se aplica a `encoder_hidden_states`. Esta es la salida del codificador de texto de T5. | |
| image_meta_size (torch.Tensor) : incrustación condicional que indica los tamaños de imagen | |
| style : torch.Tensor: Embedding condicional que indica el estilo | |
| image_rotary_emb (`torch.Tensor`) : Las incrustaciones rotacionales de imagen que se aplican a los tensores de consulta y clave durante el cálculo de la atención. | |
| return_dict : bool Si se debe devolver un diccionario. | |
| **Devuelve:** | |
| Si `return_dict` es True, se devuelve un `HunyuanControlNetOutput`; en caso contrario, se devuelve una `tuple` cuyo primer elemento es la lista de muestras de bloques de ControlNet. | |
| El método forward de [HunyuanDiT2DControlNetModel](/docs/diffusers/main/es/api/models/controlnet_hunyuandit#diffusers.HunyuanDiT2DControlNetModel). | |
| #### set_attn_processor[[diffusers.HunyuanDiT2DControlNetModel.set_attn_processor]] | |
| ```python | |
| set_attn_processor(processor: diffusers.models.attention_processor.AttnProcessor | diffusers.models.attention_processor.CustomDiffusionAttnProcessor | diffusers.models.attention_processor.AttnAddedKVProcessor | diffusers.models.attention_processor.AttnAddedKVProcessor2_0 | diffusers.models.attention_processor.JointAttnProcessor2_0 | diffusers.models.attention_processor.PAGJointAttnProcessor2_0 | diffusers.models.attention_processor.PAGCFGJointAttnProcessor2_0 | diffusers.models.attention_processor.FusedJointAttnProcessor2_0 | diffusers.models.attention_processor.AllegroAttnProcessor2_0 | diffusers.models.attention_processor.AuraFlowAttnProcessor2_0 | diffusers.models.attention_processor.FusedAuraFlowAttnProcessor2_0 | diffusers.models.attention_processor.FluxAttnProcessor2_0 | diffusers.models.attention_processor.FluxAttnProcessor2_0_NPU | diffusers.models.attention_processor.FusedFluxAttnProcessor2_0 | diffusers.models.attention_processor.FusedFluxAttnProcessor2_0_NPU | diffusers.models.attention_processor.CogVideoXAttnProcessor2_0 | diffusers.models.attention_processor.FusedCogVideoXAttnProcessor2_0 | diffusers.models.attention_processor.XFormersAttnAddedKVProcessor | diffusers.models.attention_processor.XFormersAttnProcessor | diffusers.models.attention_processor.XLAFlashAttnProcessor2_0 | diffusers.models.attention_processor.AttnProcessorNPU | diffusers.models.attention_processor.AttnProcessor2_0 | diffusers.models.attention_processor.MochiVaeAttnProcessor2_0 | diffusers.models.attention_processor.MochiAttnProcessor2_0 | diffusers.models.attention_processor.StableAudioAttnProcessor2_0 | diffusers.models.attention_processor.HunyuanAttnProcessor2_0 | diffusers.models.attention_processor.FusedHunyuanAttnProcessor2_0 | diffusers.models.attention_processor.PAGHunyuanAttnProcessor2_0 | diffusers.models.attention_processor.PAGCFGHunyuanAttnProcessor2_0 | diffusers.models.attention_processor.LuminaAttnProcessor2_0 | diffusers.models.attention_processor.FusedAttnProcessor2_0 | diffusers.models.attention_processor.CustomDiffusionXFormersAttnProcessor | diffusers.models.attention_processor.CustomDiffusionAttnProcessor2_0 | diffusers.models.attention_processor.SlicedAttnProcessor | diffusers.models.attention_processor.SlicedAttnAddedKVProcessor | diffusers.models.attention_processor.SanaLinearAttnProcessor2_0 | diffusers.models.attention_processor.PAGCFGSanaLinearAttnProcessor2_0 | diffusers.models.attention_processor.PAGIdentitySanaLinearAttnProcessor2_0 | diffusers.models.attention_processor.SanaMultiscaleLinearAttention | diffusers.models.attention_processor.SanaMultiscaleAttnProcessor2_0 | diffusers.models.attention_processor.SanaMultiscaleAttentionProjection | diffusers.models.attention_processor.IPAdapterAttnProcessor | diffusers.models.attention_processor.IPAdapterAttnProcessor2_0 | diffusers.models.attention_processor.IPAdapterXFormersAttnProcessor | diffusers.models.attention_processor.SD3IPAdapterJointAttnProcessor2_0 | diffusers.models.attention_processor.PAGIdentitySelfAttnProcessor2_0 | diffusers.models.attention_processor.PAGCFGIdentitySelfAttnProcessor2_0 | diffusers.models.attention_processor.LoRAAttnProcessor | diffusers.models.attention_processor.LoRAAttnProcessor2_0 | diffusers.models.attention_processor.LoRAXFormersAttnProcessor | diffusers.models.attention_processor.LoRAAttnAddedKVProcessor | dict[str, diffusers.models.attention_processor.AttnProcessor | diffusers.models.attention_processor.CustomDiffusionAttnProcessor | diffusers.models.attention_processor.AttnAddedKVProcessor | diffusers.models.attention_processor.AttnAddedKVProcessor2_0 | diffusers.models.attention_processor.JointAttnProcessor2_0 | diffusers.models.attention_processor.PAGJointAttnProcessor2_0 | diffusers.models.attention_processor.PAGCFGJointAttnProcessor2_0 | diffusers.models.attention_processor.FusedJointAttnProcessor2_0 | diffusers.models.attention_processor.AllegroAttnProcessor2_0 | diffusers.models.attention_processor.AuraFlowAttnProcessor2_0 | diffusers.models.attention_processor.FusedAuraFlowAttnProcessor2_0 | diffusers.models.attention_processor.FluxAttnProcessor2_0 | diffusers.models.attention_processor.FluxAttnProcessor2_0_NPU | diffusers.models.attention_processor.FusedFluxAttnProcessor2_0 | diffusers.models.attention_processor.FusedFluxAttnProcessor2_0_NPU | diffusers.models.attention_processor.CogVideoXAttnProcessor2_0 | diffusers.models.attention_processor.FusedCogVideoXAttnProcessor2_0 | diffusers.models.attention_processor.XFormersAttnAddedKVProcessor | diffusers.models.attention_processor.XFormersAttnProcessor | diffusers.models.attention_processor.XLAFlashAttnProcessor2_0 | diffusers.models.attention_processor.AttnProcessorNPU | diffusers.models.attention_processor.AttnProcessor2_0 | diffusers.models.attention_processor.MochiVaeAttnProcessor2_0 | diffusers.models.attention_processor.MochiAttnProcessor2_0 | diffusers.models.attention_processor.StableAudioAttnProcessor2_0 | diffusers.models.attention_processor.HunyuanAttnProcessor2_0 | diffusers.models.attention_processor.FusedHunyuanAttnProcessor2_0 | diffusers.models.attention_processor.PAGHunyuanAttnProcessor2_0 | diffusers.models.attention_processor.PAGCFGHunyuanAttnProcessor2_0 | diffusers.models.attention_processor.LuminaAttnProcessor2_0 | diffusers.models.attention_processor.FusedAttnProcessor2_0 | diffusers.models.attention_processor.CustomDiffusionXFormersAttnProcessor | diffusers.models.attention_processor.CustomDiffusionAttnProcessor2_0 | diffusers.models.attention_processor.SlicedAttnProcessor | diffusers.models.attention_processor.SlicedAttnAddedKVProcessor | diffusers.models.attention_processor.SanaLinearAttnProcessor2_0 | diffusers.models.attention_processor.PAGCFGSanaLinearAttnProcessor2_0 | diffusers.models.attention_processor.PAGIdentitySanaLinearAttnProcessor2_0 | diffusers.models.attention_processor.SanaMultiscaleLinearAttention | diffusers.models.attention_processor.SanaMultiscaleAttnProcessor2_0 | diffusers.models.attention_processor.SanaMultiscaleAttentionProjection | diffusers.models.attention_processor.IPAdapterAttnProcessor | diffusers.models.attention_processor.IPAdapterAttnProcessor2_0 | diffusers.models.attention_processor.IPAdapterXFormersAttnProcessor | diffusers.models.attention_processor.SD3IPAdapterJointAttnProcessor2_0 | diffusers.models.attention_processor.PAGIdentitySelfAttnProcessor2_0 | diffusers.models.attention_processor.PAGCFGIdentitySelfAttnProcessor2_0 | diffusers.models.attention_processor.LoRAAttnProcessor | diffusers.models.attention_processor.LoRAAttnProcessor2_0 | diffusers.models.attention_processor.LoRAXFormersAttnProcessor | diffusers.models.attention_processor.LoRAAttnAddedKVProcessor]) | |
| ``` | |
| [Fuente](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/controlnets/controlnet_hunyuan.py#L141) | |
| **Parámetros:** | |
| processor (`dict` de `AttentionProcessor` o solo `AttentionProcessor`) : La clase de procesador instanciada o un diccionario de clases de procesador que se establecerá como el procesador para **todos** las capas de `Attention`. Si `processor` es un diccionario, la clave debe definir la ruta al procesador de atención cruzada correspondiente. Esto se recomienda encarecidamente al establecer procesadores de atención entrenables. | |
| Establece el procesador de atención que se utilizará para calcular la atención. | |
Xet Storage Details
- Size:
- 13 kB
- Xet hash:
- 06a5ec4b25c2cd43904ef1459731b3c7d2e026bf4f2f84c3f19ecc32fe787580
·
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.