Buckets:

hf-doc-build
/

doc

Files

xet

hf-doc-build/doc / diffusers /v0.43.0 /zh /api /models /consistency_decoder_vae.md

mishig

about 3 hours ago

preview code

download

raw

6.38 kB

# 一致性解码器

Consistency decoder 可用于解码 [StableDiffusionPipeline](/docs/diffusers/v0.43.0/zh/api/pipelines/stable_diffusion/text2img#diffusers.StableDiffusionPipeline) 中来自去噪 UNet 的潜变量。该解码器在 [DALL-E 3 技术报告](https://openai.com/dall-e-3) 中首次提出。

原始代码库可在 [openai/consistencydecoder](https://github.com/openai/consistencydecoder) 中找到。

> [!WARNING]
> 目前，推理仅支持 2 次迭代。

该 pipeline 的完成离不开来自 [此 issue](https://github.com/openai/consistencydecoder/issues/1) 的 [madebyollin](https://github.com/madebyollin) 和 [mrsteyk](https://github.com/mrsteyk) 的帮助。

## ConsistencyDecoderVAE[[diffusers.ConsistencyDecoderVAE]]

#### diffusers.ConsistencyDecoderVAE[[diffusers.ConsistencyDecoderVAE]]

```python
diffusers.ConsistencyDecoderVAE(scaling_factor: float = 0.18215, latent_channels: int = 4, sample_size: int = 32, encoder_act_fn: str = 'silu', encoder_block_out_channels: tuple[int, ...] = (128, 256, 512, 512), encoder_double_z: bool = True, encoder_down_block_types: tuple[str, ...] = ('DownEncoderBlock2D', 'DownEncoderBlock2D', 'DownEncoderBlock2D', 'DownEncoderBlock2D'), encoder_in_channels: int = 3, encoder_layers_per_block: int = 2, encoder_norm_num_groups: int = 32, encoder_out_channels: int = 4, decoder_add_attention: bool = False, decoder_block_out_channels: tuple[int, ...] = (320, 640, 1024, 1024), decoder_down_block_types: tuple[str, ...] = ('ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D'), decoder_downsample_padding: int = 1, decoder_in_channels: int = 7, decoder_layers_per_block: int = 3, decoder_norm_eps: float = 1e-05, decoder_norm_num_groups: int = 32, decoder_num_train_timesteps: int = 1024, decoder_out_channels: int = 6, decoder_resnet_time_scale_shift: str = 'scale_shift', decoder_time_embedding_type: str = 'learned', decoder_up_block_types: tuple[str, ...] = ('ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D'))
```

[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L51)

与 DALL-E 3 配合使用的 consistency decoder。

示例：
```py
>>> import torch
>>> from diffusers import StableDiffusionPipeline, ConsistencyDecoderVAE
>>> vae = ConsistencyDecoderVAE.from_pretrained("openai/consistency-decoder", torch_dtype=torch.float16)
>>> pipe = StableDiffusionPipeline.from_pretrained(
...     "stable-diffusion-v1-5/stable-diffusion-v1-5", vae=vae, torch_dtype=torch.float16
... ).to("cuda")
>>> image = pipe("horse", generator=torch.manual_seed(0)).images[0]
>>> image
```

#### decode[[diffusers.ConsistencyDecoderVAE.decode]]

```python
decode(z: Tensor, generator: typing.Optional[torch.Generator] = None, return_dict: bool = True, num_inference_steps: int = 2)
```

[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L220)

**参数：**

z (torch.Tensor) ：输入的潜在向量。

generator (torch.Generator | None) : 随机数生成器。默认值为 None。

return_dict (bool) : 是否以字典形式返回输出。默认为 True。
num_inference_steps (int) ：推理步数。默认值为 2。
**返回：** DecoderOutput | tuple[torch.Tensor]
解码后的输出。
使用一致性解码器 VAE 模型解码输入潜在向量 `z`。
#### encode[[diffusers.ConsistencyDecoderVAE.encode]]
```python
encode(x: Tensor, return_dict: bool = True)
```
[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L185)
**参数：**
x (`torch.Tensor`) : 输入图像批次。
return_dict (`bool`，*可选*，默认为 `True`)：是否返回 `ConsistencyDecoderVAEOutput` 而非普通元组。
**返回值：**
编码图像的潜在表示。如果 `return_dict` 为 True，则返回 `ConsistencyDecoderVAEOutput`，否则返回普通的 `tuple`。
将一批图像编码为潜在表示。
#### forward[[diffusers.ConsistencyDecoderVAE.forward]]
```python
forward(sample: Tensor, sample_posterior: bool = False, return_dict: bool = True, generator: typing.Optional[torch.Generator] = None)
```
[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L336)
**参数：**
sample (`torch.Tensor`) : 输入样本。
sample_posterior (`bool`，*可选*，默认为 `False`)：是否从后验分布中采样。
return_dict (`bool`，*可选*，默认为 `True`)：是否返回 `DecoderOutput` 而不是普通元组。
generator (`torch.Generator`，*可选*，默认为 `None`)：用于采样的生成器。
**返回：** `DecoderOutput` 或 `tuple`
If return_dict is True, a `DecoderOutput` is returned, otherwise a plain `tuple` is returned.
#### set_default_attn_processor[[diffusers.ConsistencyDecoderVAE.set_default_attn_processor]]
```python
set_default_attn_processor()
```
[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L170)
禁用自定义注意力处理器并设置默认的注意力实现。
#### tiled_encode[[diffusers.ConsistencyDecoderVAE.tiled_encode]]
```python
tiled_encode(x: Tensor, return_dict: bool = True)
```
[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L281)
**参数：**
x (`torch.Tensor`) : 输入图像批次。
return_dict (`bool`，*可选*，默认为 `True`)：是否返回 `ConsistencyDecoderVAEOutput` 而非普通元组。

**返回：** `ConsistencyDecoderVAEOutput` 或 `tuple`

If return_dict is True, a `ConsistencyDecoderVAEOutput` is returned, otherwise a plain `tuple` is returned.
使用分块编码器对一批图像进行编码。
启用此选项后，VAE 会将输入张量拆分为多个图块，分步计算编码。这有助于保持内存占用恒定，而不受图像尺寸的影响。由于每个图块使用不同的编码器，分块编码的最终结果与非分块编码有所不同。为避免分块伪影，图块之间会重叠并混合，以形成平滑的输出。输出中可能仍会出现图块大小的变化，但这些变化应会明显减弱。

Xet Storage Details

Size:: 6.38 kB
Xet hash:: 739a5c6fda5d070982698b50cca1335948becce15caafa200d99a317328630db

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# 一致性解码器

	Consistency decoder 可用于解码 [StableDiffusionPipeline](/docs/diffusers/v0.43.0/zh/api/pipelines/stable_diffusion/text2img#diffusers.StableDiffusionPipeline) 中来自去噪 UNet 的潜变量。该解码器在 [DALL-E 3 技术报告](https://openai.com/dall-e-3) 中首次提出。

	原始代码库可在 [openai/consistencydecoder](https://github.com/openai/consistencydecoder) 中找到。

	> [!WARNING]
	> 目前，推理仅支持 2 次迭代。

	该 pipeline 的完成离不开来自 [此 issue](https://github.com/openai/consistencydecoder/issues/1) 的 [madebyollin](https://github.com/madebyollin) 和 [mrsteyk](https://github.com/mrsteyk) 的帮助。

	## ConsistencyDecoderVAE[[diffusers.ConsistencyDecoderVAE]]

	#### diffusers.ConsistencyDecoderVAE[[diffusers.ConsistencyDecoderVAE]]

	```python
	diffusers.ConsistencyDecoderVAE(scaling_factor: float = 0.18215, latent_channels: int = 4, sample_size: int = 32, encoder_act_fn: str = 'silu', encoder_block_out_channels: tuple[int, ...] = (128, 256, 512, 512), encoder_double_z: bool = True, encoder_down_block_types: tuple[str, ...] = ('DownEncoderBlock2D', 'DownEncoderBlock2D', 'DownEncoderBlock2D', 'DownEncoderBlock2D'), encoder_in_channels: int = 3, encoder_layers_per_block: int = 2, encoder_norm_num_groups: int = 32, encoder_out_channels: int = 4, decoder_add_attention: bool = False, decoder_block_out_channels: tuple[int, ...] = (320, 640, 1024, 1024), decoder_down_block_types: tuple[str, ...] = ('ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D', 'ResnetDownsampleBlock2D'), decoder_downsample_padding: int = 1, decoder_in_channels: int = 7, decoder_layers_per_block: int = 3, decoder_norm_eps: float = 1e-05, decoder_norm_num_groups: int = 32, decoder_num_train_timesteps: int = 1024, decoder_out_channels: int = 6, decoder_resnet_time_scale_shift: str = 'scale_shift', decoder_time_embedding_type: str = 'learned', decoder_up_block_types: tuple[str, ...] = ('ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D', 'ResnetUpsampleBlock2D'))
	```

	[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L51)

	与 DALL-E 3 配合使用的 consistency decoder。

	示例：
	```py
	>>> import torch
	>>> from diffusers import StableDiffusionPipeline, ConsistencyDecoderVAE

	>>> vae = ConsistencyDecoderVAE.from_pretrained("openai/consistency-decoder", torch_dtype=torch.float16)
	>>> pipe = StableDiffusionPipeline.from_pretrained(
	... "stable-diffusion-v1-5/stable-diffusion-v1-5", vae=vae, torch_dtype=torch.float16
	... ).to("cuda")

	>>> image = pipe("horse", generator=torch.manual_seed(0)).images[0]
	>>> image
	```

	#### decode[[diffusers.ConsistencyDecoderVAE.decode]]

	```python
	decode(z: Tensor, generator: typing.Optional[torch.Generator] = None, return_dict: bool = True, num_inference_steps: int = 2)
	```

	[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L220)

	参数：

	z (torch.Tensor) ：输入的潜在向量。

	generator (torch.Generator \| None) : 随机数生成器。默认值为 None。

	return_dict (bool) : 是否以字典形式返回输出。默认为 True。

	num_inference_steps (int) ：推理步数。默认值为 2。

	返回： DecoderOutput \| tuple[torch.Tensor]

	解码后的输出。

	使用一致性解码器 VAE 模型解码输入潜在向量 `z`。

	#### encode[[diffusers.ConsistencyDecoderVAE.encode]]

	```python
	encode(x: Tensor, return_dict: bool = True)
	```

	[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L185)

	参数：

	x (`torch.Tensor`) : 输入图像批次。

	return_dict (`bool`，可选，默认为 `True`)：是否返回 `ConsistencyDecoderVAEOutput` 而非普通元组。

	返回值：

	编码图像的潜在表示。如果 `return_dict` 为 True，则返回 `ConsistencyDecoderVAEOutput`，否则返回普通的 `tuple`。

	将一批图像编码为潜在表示。

	#### forward[[diffusers.ConsistencyDecoderVAE.forward]]

	```python
	forward(sample: Tensor, sample_posterior: bool = False, return_dict: bool = True, generator: typing.Optional[torch.Generator] = None)
	```

	[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L336)

	参数：

	sample (`torch.Tensor`) : 输入样本。

	sample_posterior (`bool`，可选，默认为 `False`)：是否从后验分布中采样。

	return_dict (`bool`，可选，默认为 `True`)：是否返回 `DecoderOutput` 而不是普通元组。

	generator (`torch.Generator`，可选，默认为 `None`)：用于采样的生成器。

	返回： `DecoderOutput` 或 `tuple`

	If return_dict is True, a `DecoderOutput` is returned, otherwise a plain `tuple` is returned.

	#### set_default_attn_processor[[diffusers.ConsistencyDecoderVAE.set_default_attn_processor]]

	```python
	set_default_attn_processor()
	```

	[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L170)

	禁用自定义注意力处理器并设置默认的注意力实现。

	#### tiled_encode[[diffusers.ConsistencyDecoderVAE.tiled_encode]]

	```python
	tiled_encode(x: Tensor, return_dict: bool = True)
	```

	[源代码](https://github.com/huggingface/diffusers/blob/v0.43.0/src/diffusers/models/autoencoders/consistency_decoder_vae.py#L281)

	参数：

	x (`torch.Tensor`) : 输入图像批次。

	return_dict (`bool`，可选，默认为 `True`)：是否返回 `ConsistencyDecoderVAEOutput` 而非普通元组。

	返回： `ConsistencyDecoderVAEOutput` 或 `tuple`

	If return_dict is True, a `ConsistencyDecoderVAEOutput` is returned, otherwise a plain `tuple` is returned.

	使用分块编码器对一批图像进行编码。

	启用此选项后，VAE 会将输入张量拆分为多个图块，分步计算编码。这有助于保持内存占用恒定，而不受图像尺寸的影响。由于每个图块使用不同的编码器，分块编码的最终结果与非分块编码有所不同。为避免分块伪影，图块之间会重叠并混合，以形成平滑的输出。输出中可能仍会出现图块大小的变化，但这些变化应会明显减弱。