Buckets:

hf-doc-build
/

doc

Files

xet

hf-doc-build/doc / diffusers /v0.43.0 /zh /using-diffusers /controlnet.md

mishig

2 days ago

preview code

download

raw

11.4 kB

# ControlNet

[ControlNet](https://huggingface.co/papers/2302.05543) 通过结构控制图像（边缘、深度或姿态）来引导预训练的扩散模型，同时保留文本提示。它冻结基础模型，并添加一个并行网络，其残差用于引导去噪器，从而无需微调即可实现布局控制。

加载适用于所需控制类型（例如 canny）的 ControlNet，然后将其作为 `controlnet=` 参数传递给 [from_pretrained()](/docs/diffusers/v0.43.0/zh/api/pipelines/overview#diffusers.DiffusionPipeline.from_pretrained)。使用 `controlnet_conditioning_scale` 来设置控制对生成过程的引导强度。

> [!TIP]
> ControlNet 可用于许多模型，例如 [Flux](../api/pipelines/controlnet_flux)、[Hunyuan-DiT](../api/pipelines/controlnet_hunyuandit)、[Stable Diffusion 3](../api/pipelines/controlnet_sd3) 等。本指南中的示例使用 Flux 和 Stable Diffusion XL。对于旧版 Stable Diffusion 系列检查点上的 T2I-Adapter，请参阅 [Legacy adapters](./legacy_adapters#t2i-adapter)。对于新的可控生成任务，建议优先使用 ControlNet。

下方的选项卡分别展示了用于文生图、图生图和图像修复的 ControlNet。

使用 [opencv-python](https://github.com/opencv/opencv-python) 生成 canny 图像。

```py
import cv2
import numpy as np
from PIL import Image
from diffusers.utils import load_image
original_image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/non-enhanced-prompt.png"
)
image = np.array(original_image)
low_threshold = 100
high_threshold = 200
image = cv2.Canny(image, low_threshold, high_threshold)
image = image[:, :, None]
image = np.concatenate([image, image, image], axis=2)
canny_image = Image.fromarray(image)
```

将 Canny 图像传入 pipeline。使用 `controlnet_conditioning_scale` 参数来确定分配给控制信号的权重。

```py
import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetPipeline, FluxControlNetModel
controlnet = FluxControlNetModel.from_pretrained(
    "InstantX/FLUX.1-dev-Controlnet-Canny", dtype=torch.bfloat16
)
pipeline = FluxControlNetPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev", controlnet=controlnet, dtype=torch.bfloat16
).to("cuda")  # or "mps", "xpu", "cpu"
prompt = """
A photorealistic overhead image of a cat reclining sideways in a flamingo pool floatie holding a margarita. 
The cat is floating leisurely in the pool and completely relaxed and happy.
"""
pipeline(
    prompt, 
    control_image=canny_image,
    controlnet_conditioning_scale=0.5,
    num_inference_steps=50, 
    guidance_scale=3.5,
).images[0]
```

  
    
    原始图像
  
  
    
    canny 图像
  
  
    
    生成的图像
  
使用 Transformers 中的深度估计流水线生成深度图。

```py
import torch
import numpy as np
from PIL import Image
from transformers import DPTImageProcessor, DPTForDepthEstimation
from diffusers import ControlNetModel, StableDiffusionXLControlNetImg2ImgPipeline, AutoencoderKL
from diffusers.utils import load_image
device = "cuda"  # or "mps", "xpu", "cpu"
depth_estimator = DPTForDepthEstimation.from_pretrained("Intel/dpt-hybrid-midas").to(device)
feature_extractor = DPTImageProcessor.from_pretrained("Intel/dpt-hybrid-midas")
def get_depth_map(image):
    image = feature_extractor(images=image, return_tensors="pt").pixel_values.to(device)
    with torch.no_grad(), torch.autocast(device):
        depth_map = depth_estimator(image).predicted_depth
    depth_map = torch.nn.functional.interpolate(
        depth_map.unsqueeze(1),
        size=(1024, 1024),
        mode="bicubic",
        align_corners=False,
    )
    depth_min = torch.amin(depth_map, dim=[1, 2, 3], keepdim=True)
    depth_max = torch.amax(depth_map, dim=[1, 2, 3], keepdim=True)
    depth_map = (depth_map - depth_min) / (depth_max - depth_min)
    image = torch.cat([depth_map] * 3, dim=1)
    image = image.permute(0, 2, 3, 1).cpu().numpy()[0]
    image = Image.fromarray((image * 255.0).clip(0, 255).astype(np.uint8))
    return image
image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/non-enhanced-prompt.png"
).resize((1024, 1024))
depth_image = get_depth_map(image)
```

将深度图传入 pipeline。使用 `controlnet_conditioning_scale` 参数来确定分配给该控制条件的权重。

```py
controlnet = ControlNetModel.from_pretrained(
    "diffusers/controlnet-depth-sdxl-1.0-small",
    dtype=torch.float16,
)
vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16)
pipeline = StableDiffusionXLControlNetImg2ImgPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    controlnet=controlnet,
    vae=vae,
    dtype=torch.float16,
).to(device)
prompt = """
A photorealistic overhead image of a cat reclining sideways in a flamingo pool floatie holding a margarita. 
The cat is floating leisurely in the pool and completely relaxed and happy.
"""
controlnet_conditioning_scale = 0.5
pipeline(
    prompt,
    image=image,
    control_image=depth_image,
    controlnet_conditioning_scale=controlnet_conditioning_scale,
    strength=0.99,
    num_inference_steps=100,
).images[0]
```

  
    
    原始图像
  
  
    
    深度图
  
   
    
    生成的图像
  
生成一张标记需要替换像素的掩码图像，然后将其作为 `mask_image` 与控制图像一起传入。

```py
import cv2
import torch
import numpy as np
from PIL import Image
from diffusers.utils import load_image
from diffusers import StableDiffusionXLControlNetInpaintPipeline, ControlNetModel
init_image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/non-enhanced-prompt.png"
)
init_image = init_image.resize((1024, 1024))
mask_image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat_mask.png"
)
mask_image = mask_image.resize((1024, 1024))
def make_canny_condition(image):
    image = np.array(image)
    image = cv2.Canny(image, 100, 200)
    image = image[:, :, None]
    image = np.concatenate([image, image, image], axis=2)
    image = Image.fromarray(image)
    return image
control_image = make_canny_condition(init_image)
```

将掩码和控制图像传入管道。使用 `controlnet_conditioning_scale` 参数来确定分配给控制信号的权重。

```py
controlnet = ControlNetModel.from_pretrained(
    "diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16
)
pipeline = StableDiffusionXLControlNetInpaintPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline(
    "a cute and fluffy bunny rabbit",
    num_inference_steps=100,
    strength=0.99,
    controlnet_conditioning_scale=0.5,
    image=init_image,
    mask_image=mask_image,
    control_image=control_image,
).images[0]
```

  
    
    原始图像
  
  
    
    掩码图像
  
   
    
    生成的图像
  
## Multi-ControlNet

通过传入列表来组合多个 ControlNet（例如 canny + depth）。在可能的情况下，对重叠区域进行遮罩，并调整每个 `controlnet_conditioning_scale`。使用 OpenCV Canny（与文生图相同）构建 `canny_image`，使用 DPT（与图生图相同）构建 `depth_image`。在调用 pipeline 之前，将每个控制图像调整至 pipeline 的尺寸。

```py
import cv2
import torch
import numpy as np
from PIL import Image
from transformers import DPTImageProcessor, DPTForDepthEstimation
from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel, AutoencoderKL
from diffusers.utils import load_image
original_image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/non-enhanced-prompt.png"
)
image = np.array(original_image)
canny_image = Image.fromarray(
    np.concatenate([cv2.Canny(image, 100, 200)[:, :, None]] * 3, axis=2)
)
device = "cuda"  # or "mps", "xpu", "cpu"
depth_estimator = DPTForDepthEstimation.from_pretrained("Intel/dpt-hybrid-midas").to(device)
feature_extractor = DPTImageProcessor.from_pretrained("Intel/dpt-hybrid-midas")
def get_depth_map(image):
    image = feature_extractor(images=image, return_tensors="pt").pixel_values.to(device)
    with torch.no_grad(), torch.autocast(device):
        depth_map = depth_estimator(image).predicted_depth
    depth_map = torch.nn.functional.interpolate(
        depth_map.unsqueeze(1),
        size=(1024, 1024),
        mode="bicubic",
        align_corners=False,
    )
    depth_min = torch.amin(depth_map, dim=[1, 2, 3], keepdim=True)
    depth_max = torch.amax(depth_map, dim=[1, 2, 3], keepdim=True)
    depth_map = (depth_map - depth_min) / (depth_max - depth_min)
    image = torch.cat([depth_map] * 3, dim=1)
    image = image.permute(0, 2, 3, 1).cpu().numpy()[0]
    image = Image.fromarray((image * 255.0).clip(0, 255).astype(np.uint8))
    return image
depth_image = get_depth_map(original_image.resize((1024, 1024)))
controlnets = [
    ControlNetModel.from_pretrained(
        "diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16
    ),
    ControlNetModel.from_pretrained(
        "diffusers/controlnet-depth-sdxl-1.0-small", dtype=torch.float16
    ),
]
vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16)
pipeline = StableDiffusionXLControlNetPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnets, vae=vae, dtype=torch.float16
).to(device)
prompt = """
a relaxed rabbit sitting on a striped towel next to a pool with a tropical drink nearby, 
bright sunny day, vacation scene, 35mm photograph, film, professional, 4k, highly detailed
"""
negative_prompt = "lowres, bad anatomy, worst quality, low quality, deformed, ugly"
images = [canny_image.resize((1024, 1024)), depth_image.resize((1024, 1024))]
pipeline(
    prompt,
    negative_prompt=negative_prompt,
    image=images,
    num_inference_steps=100,
    controlnet_conditioning_scale=[0.5, 0.5],
).images[0]
```

  
    
    canny 图像
  
  
    
    深度图
  
   
    
    生成的图像
  
## 猜测模式

[Guess mode](https://github.com/lllyasviel/ControlNet/discussions/188) 允许在缺少（或不需要）文本提示的情况下，仅凭控制信号生成图像。它会根据块深度对 ControlNet 残差进行缩放，从早期下采样块中约 `0.1` 的权重，逐渐提升至中间块处的 `1.0`。这样既能防止早期层过度约束，又能让更深层的块继续承载结构信息。

```py
import torch
from diffusers.utils import load_image
from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel
controlnet = ControlNetModel.from_pretrained(
  "diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16
)
pipeline = StableDiffusionXLControlNetPipeline.from_pretrained(
  "stabilityai/stable-diffusion-xl-base-1.0",
  controlnet=controlnet,
  dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
canny_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/canny-cat.png")
pipeline(
  "",
  image=canny_image,
  guess_mode=True
).images[0]
```

  
    
    canny 图像
  
  
    
    生成的图像
  

Xet Storage Details

Size:: 11.4 kB
Xet hash:: c7361f23f098cff21f41c39fa7becb3433ec581d98a6c1ae4ee4336f30b241dd

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.

	# ControlNet

	[ControlNet](https://huggingface.co/papers/2302.05543) 通过结构控制图像（边缘、深度或姿态）来引导预训练的扩散模型，同时保留文本提示。它冻结基础模型，并添加一个并行网络，其残差用于引导去噪器，从而无需微调即可实现布局控制。

	加载适用于所需控制类型（例如 canny）的 ControlNet，然后将其作为 `controlnet=` 参数传递给 [from_pretrained()](/docs/diffusers/v0.43.0/zh/api/pipelines/overview#diffusers.DiffusionPipeline.from_pretrained)。使用 `controlnet_conditioning_scale` 来设置控制对生成过程的引导强度。

	> [!TIP]
	> ControlNet 可用于许多模型，例如 [Flux](../api/pipelines/controlnet_flux)、[Hunyuan-DiT](../api/pipelines/controlnet_hunyuandit)、[Stable Diffusion 3](../api/pipelines/controlnet_sd3) 等。本指南中的示例使用 Flux 和 Stable Diffusion XL。对于旧版 Stable Diffusion 系列检查点上的 T2I-Adapter，请参阅 [Legacy adapters](./legacy_adapters#t2i-adapter)。对于新的可控生成任务，建议优先使用 ControlNet。

	下方的选项卡分别展示了用于文生图、图生图和图像修复的 ControlNet。

	使用 [opencv-python](https://github.com/opencv/opencv-python) 生成 canny 图像。

	```py
	import cv2
	import numpy as np
	from PIL import Image
	from diffusers.utils import load_image

	original_image = load_image(
	"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/non-enhanced-prompt.png"
	)

	image = np.array(original_image)

	low_threshold = 100
	high_threshold = 200

	image = cv2.Canny(image, low_threshold, high_threshold)
	image = image[:, :, None]
	image = np.concatenate([image, image, image], axis=2)
	canny_image = Image.fromarray(image)
	```

	将 Canny 图像传入 pipeline。使用 `controlnet_conditioning_scale` 参数来确定分配给控制信号的权重。

	```py
	import torch
	from diffusers.utils import load_image
	from diffusers import FluxControlNetPipeline, FluxControlNetModel

	controlnet = FluxControlNetModel.from_pretrained(
	"InstantX/FLUX.1-dev-Controlnet-Canny", dtype=torch.bfloat16
	)
	pipeline = FluxControlNetPipeline.from_pretrained(
	"black-forest-labs/FLUX.1-dev", controlnet=controlnet, dtype=torch.bfloat16
	).to("cuda") # or "mps", "xpu", "cpu"

	prompt = """
	A photorealistic overhead image of a cat reclining sideways in a flamingo pool floatie holding a margarita.
	The cat is floating leisurely in the pool and completely relaxed and happy.
	"""

	pipeline(
	prompt,
	control_image=canny_image,
	controlnet_conditioning_scale=0.5,
	num_inference_steps=50,
	guidance_scale=3.5,
	).images[0]
	```



	原始图像



	canny 图像



	生成的图像


	使用 Transformers 中的深度估计流水线生成深度图。

	```py
	import torch
	import numpy as np
	from PIL import Image
	from transformers import DPTImageProcessor, DPTForDepthEstimation
	from diffusers import ControlNetModel, StableDiffusionXLControlNetImg2ImgPipeline, AutoencoderKL
	from diffusers.utils import load_image

	device = "cuda" # or "mps", "xpu", "cpu"
	depth_estimator = DPTForDepthEstimation.from_pretrained("Intel/dpt-hybrid-midas").to(device)
	feature_extractor = DPTImageProcessor.from_pretrained("Intel/dpt-hybrid-midas")

	def get_depth_map(image):
	image = feature_extractor(images=image, return_tensors="pt").pixel_values.to(device)
	with torch.no_grad(), torch.autocast(device):
	depth_map = depth_estimator(image).predicted_depth

	depth_map = torch.nn.functional.interpolate(
	depth_map.unsqueeze(1),
	size=(1024, 1024),
	mode="bicubic",
	align_corners=False,
	)
	depth_min = torch.amin(depth_map, dim=[1, 2, 3], keepdim=True)
	depth_max = torch.amax(depth_map, dim=[1, 2, 3], keepdim=True)
	depth_map = (depth_map - depth_min) / (depth_max - depth_min)
	image = torch.cat([depth_map] * 3, dim=1)
	image = image.permute(0, 2, 3, 1).cpu().numpy()[0]
	image = Image.fromarray((image * 255.0).clip(0, 255).astype(np.uint8))
	return image

	image = load_image(
	"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/non-enhanced-prompt.png"
	).resize((1024, 1024))
	depth_image = get_depth_map(image)
	```

	将深度图传入 pipeline。使用 `controlnet_conditioning_scale` 参数来确定分配给该控制条件的权重。

	```py
	controlnet = ControlNetModel.from_pretrained(
	"diffusers/controlnet-depth-sdxl-1.0-small",
	dtype=torch.float16,
	)
	vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16)
	pipeline = StableDiffusionXLControlNetImg2ImgPipeline.from_pretrained(
	"stabilityai/stable-diffusion-xl-base-1.0",
	controlnet=controlnet,
	vae=vae,
	dtype=torch.float16,
	).to(device)

	prompt = """
	A photorealistic overhead image of a cat reclining sideways in a flamingo pool floatie holding a margarita.
	The cat is floating leisurely in the pool and completely relaxed and happy.
	"""
	controlnet_conditioning_scale = 0.5
	pipeline(
	prompt,
	image=image,
	control_image=depth_image,
	controlnet_conditioning_scale=controlnet_conditioning_scale,
	strength=0.99,
	num_inference_steps=100,
	).images[0]
	```



	原始图像



	深度图



	生成的图像


	生成一张标记需要替换像素的掩码图像，然后将其作为 `mask_image` 与控制图像一起传入。

	```py
	import cv2
	import torch
	import numpy as np
	from PIL import Image
	from diffusers.utils import load_image
	from diffusers import StableDiffusionXLControlNetInpaintPipeline, ControlNetModel

	init_image = load_image(
	"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/non-enhanced-prompt.png"
	)
	init_image = init_image.resize((1024, 1024))
	mask_image = load_image(
	"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat_mask.png"
	)
	mask_image = mask_image.resize((1024, 1024))

	def make_canny_condition(image):
	image = np.array(image)
	image = cv2.Canny(image, 100, 200)
	image = image[:, :, None]
	image = np.concatenate([image, image, image], axis=2)
	image = Image.fromarray(image)
	return image

	control_image = make_canny_condition(init_image)
	```

	将掩码和控制图像传入管道。使用 `controlnet_conditioning_scale` 参数来确定分配给控制信号的权重。

	```py
	controlnet = ControlNetModel.from_pretrained(
	"diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16
	)
	pipeline = StableDiffusionXLControlNetInpaintPipeline.from_pretrained(
	"stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnet, dtype=torch.float16
	).to("cuda") # or "mps", "xpu", "cpu"
	pipeline(
	"a cute and fluffy bunny rabbit",
	num_inference_steps=100,
	strength=0.99,
	controlnet_conditioning_scale=0.5,
	image=init_image,
	mask_image=mask_image,
	control_image=control_image,
	).images[0]
	```



	原始图像



	掩码图像



	生成的图像


	## Multi-ControlNet

	通过传入列表来组合多个 ControlNet（例如 canny + depth）。在可能的情况下，对重叠区域进行遮罩，并调整每个 `controlnet_conditioning_scale`。使用 OpenCV Canny（与文生图相同）构建 `canny_image`，使用 DPT（与图生图相同）构建 `depth_image`。在调用 pipeline 之前，将每个控制图像调整至 pipeline 的尺寸。

	```py
	import cv2
	import torch
	import numpy as np
	from PIL import Image
	from transformers import DPTImageProcessor, DPTForDepthEstimation
	from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel, AutoencoderKL
	from diffusers.utils import load_image

	original_image = load_image(
	"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/non-enhanced-prompt.png"
	)
	image = np.array(original_image)
	canny_image = Image.fromarray(
	np.concatenate([cv2.Canny(image, 100, 200)[:, :, None]] * 3, axis=2)
	)

	device = "cuda" # or "mps", "xpu", "cpu"
	depth_estimator = DPTForDepthEstimation.from_pretrained("Intel/dpt-hybrid-midas").to(device)
	feature_extractor = DPTImageProcessor.from_pretrained("Intel/dpt-hybrid-midas")

	def get_depth_map(image):
	image = feature_extractor(images=image, return_tensors="pt").pixel_values.to(device)
	with torch.no_grad(), torch.autocast(device):
	depth_map = depth_estimator(image).predicted_depth

	depth_map = torch.nn.functional.interpolate(
	depth_map.unsqueeze(1),
	size=(1024, 1024),
	mode="bicubic",
	align_corners=False,
	)
	depth_min = torch.amin(depth_map, dim=[1, 2, 3], keepdim=True)
	depth_max = torch.amax(depth_map, dim=[1, 2, 3], keepdim=True)
	depth_map = (depth_map - depth_min) / (depth_max - depth_min)
	image = torch.cat([depth_map] * 3, dim=1)
	image = image.permute(0, 2, 3, 1).cpu().numpy()[0]
	image = Image.fromarray((image * 255.0).clip(0, 255).astype(np.uint8))
	return image

	depth_image = get_depth_map(original_image.resize((1024, 1024)))

	controlnets = [
	ControlNetModel.from_pretrained(
	"diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16
	),
	ControlNetModel.from_pretrained(
	"diffusers/controlnet-depth-sdxl-1.0-small", dtype=torch.float16
	),
	]

	vae = AutoencoderKL.from_pretrained("madebyollin/sdxl-vae-fp16-fix", dtype=torch.float16)
	pipeline = StableDiffusionXLControlNetPipeline.from_pretrained(
	"stabilityai/stable-diffusion-xl-base-1.0", controlnet=controlnets, vae=vae, dtype=torch.float16
	).to(device)

	prompt = """
	a relaxed rabbit sitting on a striped towel next to a pool with a tropical drink nearby,
	bright sunny day, vacation scene, 35mm photograph, film, professional, 4k, highly detailed
	"""
	negative_prompt = "lowres, bad anatomy, worst quality, low quality, deformed, ugly"

	images = [canny_image.resize((1024, 1024)), depth_image.resize((1024, 1024))]

	pipeline(
	prompt,
	negative_prompt=negative_prompt,
	image=images,
	num_inference_steps=100,
	controlnet_conditioning_scale=[0.5, 0.5],
	).images[0]
	```



	canny 图像



	深度图



	生成的图像


	## 猜测模式

	[Guess mode](https://github.com/lllyasviel/ControlNet/discussions/188) 允许在缺少（或不需要）文本提示的情况下，仅凭控制信号生成图像。它会根据块深度对 ControlNet 残差进行缩放，从早期下采样块中约 `0.1` 的权重，逐渐提升至中间块处的 `1.0`。这样既能防止早期层过度约束，又能让更深层的块继续承载结构信息。

	```py
	import torch
	from diffusers.utils import load_image
	from diffusers import StableDiffusionXLControlNetPipeline, ControlNetModel

	controlnet = ControlNetModel.from_pretrained(
	"diffusers/controlnet-canny-sdxl-1.0", dtype=torch.float16
	)
	pipeline = StableDiffusionXLControlNetPipeline.from_pretrained(
	"stabilityai/stable-diffusion-xl-base-1.0",
	controlnet=controlnet,
	dtype=torch.float16
	).to("cuda") # or "mps", "xpu", "cpu"

	canny_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/canny-cat.png")
	pipeline(
	"",
	image=canny_image,
	guess_mode=True
	).images[0]
	```



	canny 图像



	生成的图像