# ZImageTransformer2DModel

用于 [Z-Image](https://huggingface.co/Tongyi-MAI/Z-Image-Turbo) 图像类数据的 Transformer 模型。

## ZImageTransformer2DModel[[diffusers.ZImageTransformer2DModel]]

#### diffusers.ZImageTransformer2DModel[[diffusers.ZImageTransformer2DModel]]

```python
diffusers.ZImageTransformer2DModel(all_patch_size = (2,), all_f_patch_size = (1,), in_channels = 16, dim = 3840, n_layers = 30, n_refiner_layers = 2, n_heads = 30, n_kv_heads = 30, norm_eps = 1e-05, qk_norm = True, cap_feat_dim = 2560, siglip_feat_dim = None, rope_theta = 256.0, t_scale = 1000.0, axes_dims = [32, 48, 48], axes_lens = [1024, 512, 512])
```

[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/transformers/transformer_z_image.py#L361)

#### forward[[diffusers.ZImageTransformer2DModel.forward]]

```python
forward(x: list[torch.Tensor, list[list[torch.Tensor]]], t, cap_feats: list[torch.Tensor, list[list[torch.Tensor]]], return_dict: bool = True, controlnet_block_samples: dict[int, torch.Tensor] | None = None, siglip_feats: list[list[torch.Tensor]] | None = None, image_noise_mask: list[list[int]] | None = None, patch_size: int = 2, f_patch_size: int = 1, joint_attention_kwargs: dict[str, typing.Any] | None = None)
```

[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/transformers/transformer_z_image.py#L896)

**参数：**

x (`list` of `torch.Tensor` or nested `list` of `torch.Tensor`) : 输入潜变量。在标准模式下为扁平列表，在 omni 模式下为嵌套列表。

t (`torch.Tensor`)：用于指示去噪步骤。

cap_feats (`list` of `torch.Tensor` or nested `list` of `torch.Tensor`) : 要使用的条件字幕嵌入（从输入条件（例如提示词）计算得出的嵌入）。

return_dict (`bool`，*可选*，默认为 `True`)：是否返回 `~models.transformer_2d.Transformer2DModelOutput` 而不是普通元组。

controlnet_block_samples (`dict`，键为 `int`，值为 `torch.Tensor`，*可选*)：从块索引到张量的映射。如果指定，这些张量将被添加到 Transformer 块的残差中。

siglip_feats (`list` of `list` of `torch.Tensor`, *可选*) : 用作额外条件输入的可选 SigLIP 图像特征。

image_noise_mask (`list` of `list` of `int`, *可选*) : 在 omni 模式下，用于指示每张图像中噪声 token 与干净 token 的逐图像噪声掩码。

patch_size (`int`, *可选*，默认为 2)：用于将输入潜变量分块的空间块大小。

f_patch_size (`int`，*可选*，默认为 1)：用于对输入潜变量进行分块的时间维度块大小。

joint_attention_kwargs (`dict`, *可选*) : 一个 kwargs 字典。仅使用 `scale` 条目，用于缩放 LoRA 层。

**返回值：**

如果 `return_dict` 为 True，则返回 `~models.transformer_2d.Transformer2DModelOutput`，否则返回一个 `tuple`，其中第一个元素是样本张量。

[ZImageTransformer2DModel](/docs/diffusers/v0.43.0/zh/api/models/z_image_transformer2d#diffusers.ZImageTransformer2DModel) 的前向方法。

流程：patchify -> t_embed -> x_embed -> x_refine -> cap_embed -> cap_refine -> [siglip_embed -> siglip_refine] -> build_unified -> main_layers -> final_layer -> unpatchify

#### patchify_and_embed[[diffusers.ZImageTransformer2DModel.patchify_and_embed]]

```python
patchify_and_embed(all_image: list[torch.Tensor], all_cap_feats: list[torch.Tensor], patch_size: int, f_patch_size: int)
```

[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/transformers/transformer_z_image.py#L590)

基础模式下的 Patchify：每个批次项对应单张图像。

#### patchify_and_embed_omni[[diffusers.ZImageTransformer2DModel.patchify_and_embed_omni]]

```python
patchify_and_embed_omni(all_x: list[list[torch.Tensor]], all_cap_feats: list[list[torch.Tensor]], all_siglip_feats: list[list[torch.Tensor]], patch_size: int, f_patch_size: int, images_noise_mask: list[list[int]])
```

[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/transformers/transformer_z_image.py#L627)

全模态模式下的 Patchify：每个批次项包含多张图像及噪声掩码。