Buckets:

hf-doc-build
/

doc

Files
xet
hf-doc-build/doc / diffusers /main /id /using-diffusers /ip_adapter.md
mishig
8 days ago
preview code
download
raw
23.3 kB
# IP-Adapter

[IP-Adapter](https://huggingface.co/papers/2308.06721) mengarahkan model difusi yang telah dilatih sebelumnya menggunakan gambar referensi sambil tetap mempertahankan prompt teks. Metode ini membekukan model dasar dan menambahkan seperangkat kecil lapisan cross-attention gambar, yang membuatnya praktis untuk mencocokkan subjek atau gaya dari foto tanpa perlu fine-tuning.

```text
text prompt                         reference image
    |                                     |
 text encoder                        image encoder
    |                                     |
    v                                     v
 text cross-attn  (frozen)          IP-Adapter cross-attn
    \                                     /
     \                                   /
      +-------->  denoiser  <-----------+
                  (frozen base)
```

Checkpoint IP-Adapter biasanya berukuran ~100MB karena menyimpan bobot adapter, bukan model lengkap. Muat pipeline dasar terlebih dahulu, kemudian muat adapter dengan [load_ip_adapter()](/docs/diffusers/main/id/api/loaders/ip_adapter#diffusers.loaders.IPAdapterMixin.load_ip_adapter).

> [!TIP]
> IP-Adapters tersedia untuk banyak model seperti [Flux](../api/pipelines/flux#ip-adapter) dan [Stable Diffusion 3](../api/pipelines/stable_diffusion/stable_diffusion_3), serta lainnya. Contoh dalam panduan ini menggunakan Stable Diffusion dan Stable Diffusion XL.

Gunakan [set_ip_adapter_scale()](/docs/diffusers/main/id/api/loaders/ip_adapter#diffusers.loaders.IPAdapterMixin.set_ip_adapter_scale) untuk mengontrol seberapa kuat IP-Adapter mengarahkan generasi. `1.0` menerapkan adapter dengan kekuatan penuh; `0.5` biasanya menyeimbangkan prompt teks dan gambar.

Contoh di bawah ini menunjukkan IP-Adapter pada tugas-tugas umum.

```py
import torch
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
pipeline = AutoPipelineForText2Image.from_pretrained(
  "stabilityai/stable-diffusion-xl-base-1.0",
  dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  weight_name="ip-adapter_sdxl.bin"
)
pipeline.set_ip_adapter_scale(0.8)
```

Lewatkan gambar ke `ip_adapter_image` bersama dengan prompt teks untuk menghasilkan gambar.

```py
image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_adapter_diner.png")
pipeline(
    prompt="a polar bear sitting in a chair drinking a milkshake",
    ip_adapter_image=image,
    negative_prompt="deformed, ugly, wrong proportion, low res, bad anatomy, worst quality, low quality",
).images[0]
```

  
    
    Gambar IP-Adapter
  
  
    
    gambar yang dihasilkan
  
```py
import torch
from diffusers import AutoPipelineForImage2Image
from diffusers.utils import load_image
pipeline = AutoPipelineForImage2Image.from_pretrained(
  "stabilityai/stable-diffusion-xl-base-1.0",
  dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  weight_name="ip-adapter_sdxl.bin"
)
pipeline.set_ip_adapter_scale(0.8)
image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_adapter_bear_1.png")
ip_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_adapter_gummy.png")
pipeline(
    prompt="best quality, high quality",
    image=image,
    ip_adapter_image=ip_image,
    strength=0.5,
).images[0]
```

  
    
    gambar input
  
  
    
    Gambar IP-Adapter
  
  
    
    gambar yang dihasilkan
  
```py
import torch
from diffusers import AutoPipelineForInpainting
from diffusers.utils import load_image
pipeline = AutoPipelineForInpainting.from_pretrained(
  "stabilityai/stable-diffusion-xl-base-1.0",
  dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  weight_name="ip-adapter_sdxl.bin"
)
pipeline.set_ip_adapter_scale(0.6)
mask_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_adapter_mask.png")
image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_adapter_bear_1.png")
ip_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_adapter_gummy.png")
pipeline(
    prompt="a cute gummy bear waving",
    image=image,
    mask_image=mask_image,
    ip_adapter_image=ip_image,
).images[0]
```

  
    
    gambar input
  
  
    
    Gambar IP-Adapter
  
  
    
    gambar yang dihasilkan
  
Metode [enable_model_cpu_offload()](/docs/diffusers/main/id/api/pipelines/overview#diffusers.DiffusionPipeline.enable_model_cpu_offload) berguna untuk mengurangi memori dan sebaiknya diaktifkan setelah IP-Adapter dimuat. Jika tidak, image encoder dari IP-Adapter juga akan di-offload ke CPU dan mengembalikan error.

```py
import torch
from diffusers import AnimateDiffPipeline, DDIMScheduler, MotionAdapter
from diffusers.utils import export_to_gif
from diffusers.utils import load_image
adapter = MotionAdapter.from_pretrained(
  "guoyww/animatediff-motion-adapter-v1-5-2",
  dtype=torch.float16
)
pipeline = AnimateDiffPipeline.from_pretrained(
  "emilianJR/epiCRealism",
  motion_adapter=adapter,
  dtype=torch.float16
)
scheduler = DDIMScheduler.from_pretrained(
    "emilianJR/epiCRealism",
    subfolder="scheduler",
    clip_sample=False,
    timestep_spacing="linspace",
    beta_schedule="linear",
    steps_offset=1,
)
pipeline.scheduler = scheduler
pipeline.vae.enable_slicing()
pipeline.load_ip_adapter("h94/IP-Adapter", subfolder="models", weight_name="ip-adapter_sd15.bin")
pipeline.enable_model_cpu_offload()
ip_adapter_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_adapter_inpaint.png")
pipeline(
    prompt="A cute gummy bear waving",
    negative_prompt="bad quality, worse quality, low resolution",
    ip_adapter_image=ip_adapter_image,
    num_frames=16,
    guidance_scale=7.5,
    num_inference_steps=50,
).frames[0]
```

  
    
    Gambar IP-Adapter
  
  
    
    video yang dihasilkan
  
## Varian checkpoint

Muat Plus ketika detail dari gambar referensi paling penting. Muat FaceID ketika Anda memerlukan embedding identitas InsightFace alih-alih embedding gambar CLIP.

```py
import torch
from transformers import CLIPVisionModelWithProjection
from diffusers import AutoPipelineForText2Image
image_encoder = CLIPVisionModelWithProjection.from_pretrained(
    "h94/IP-Adapter",
    subfolder="models/image_encoder",
    dtype=torch.float16
)
pipeline = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    image_encoder=image_encoder,
    dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  weight_name="ip-adapter-plus_sdxl_vit-h.safetensors"
)
```

Checkpoint FaceID menggunakan face embeddings dari [InsightFace](https://github.com/deepinsight/insightface) alih-alih image embeddings CLIP. Gunakan [DDIMScheduler](/docs/diffusers/main/id/api/schedulers/ddim#diffusers.DDIMScheduler) atau [EulerDiscreteScheduler](/docs/diffusers/main/id/api/schedulers/euler#diffusers.EulerDiscreteScheduler) untuk model FaceID. Ekstrak face embeddings dan lewati sebagai daftar tensor ke `ip_adapter_image_embeds`.

```py
# pip install insightface
import cv2
import numpy as np
import torch
from diffusers import StableDiffusionPipeline, DDIMScheduler
from diffusers.utils import load_image
from insightface.app import FaceAnalysis
pipeline = StableDiffusionPipeline.from_pretrained(
    "stable-diffusion-v1-5/stable-diffusion-v1-5",
    dtype=torch.float16,
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.scheduler = DDIMScheduler.from_config(pipeline.scheduler.config)
pipeline.load_ip_adapter(
  "h94/IP-Adapter-FaceID",
  subfolder=None,
  weight_name="ip-adapter-faceid_sd15.bin",
  image_encoder_folder=None
)
pipeline.set_ip_adapter_scale(0.6)
image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_mask_girl1.png")
ref_images_embeds = []
app = FaceAnalysis(name="buffalo_l", providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))
image = cv2.cvtColor(np.asarray(image), cv2.COLOR_RGB2BGR)
faces = app.get(image)
image = torch.from_numpy(faces[0].normed_embedding)
ref_images_embeds.append(image.unsqueeze(0))
ref_images_embeds = torch.stack(ref_images_embeds, dim=0).unsqueeze(0)
neg_ref_images_embeds = torch.zeros_like(ref_images_embeds)
id_embeds = torch.cat([neg_ref_images_embeds, ref_images_embeds]).to(dtype=torch.float16, device="cuda")
pipeline(
    prompt="A photo of a girl",
    ip_adapter_image_embeds=[id_embeds],
    negative_prompt="monochrome, lowres, bad anatomy, worst quality, low quality",
).images[0]
```

Untuk FaceID Plus, muat [CLIPVisionModelWithProjection](https://huggingface.co/docs/transformers/main/en/model_doc/clip#transformers.CLIPVisionModelWithProjection) sebagai encoder gambar.

```py
import torch
from transformers import CLIPVisionModelWithProjection
from diffusers import AutoPipelineForText2Image
image_encoder = CLIPVisionModelWithProjection.from_pretrained(
    "laion/CLIP-ViT-H-14-laion2B-s32B-b79K",
    dtype=torch.float16,
)
pipeline = AutoPipelineForText2Image.from_pretrained(
    "stable-diffusion-v1-5/stable-diffusion-v1-5",
    image_encoder=image_encoder,
    dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter-FaceID",
  subfolder=None,
  weight_name="ip-adapter-faceid-plus_sd15.bin"
)
```

FaceID Plus dan Plus v2 juga memerlukan embedding gambar CLIP di atas embeds InsightFace. Setelah Anda menyiapkan embedding wajah, encode gambar referensi dan tetapkan embeds CLIP ke lapisan proyeksi gambar.

```py
import torch
from diffusers.utils import load_image
ip_adapter_images = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_mask_girl1.png")
num_images = 1
clip_embeds = pipeline.prepare_ip_adapter_image_embeds(
    ip_adapter_image=[ip_adapter_images],
    ip_adapter_image_embeds=None,
    device=torch.device("cuda"),
    num_images_per_prompt=num_images,
    do_classifier_free_guidance=True,
)[0]
pipeline.unet.encoder_hid_proj.image_projection_layers[0].clip_embeds = clip_embeds.to(dtype=torch.float16)
# set to True if using IP-Adapter FaceID Plus v2
pipeline.unet.encoder_hid_proj.image_projection_layers[0].shortcut = False
```

## Embedding gambar

`prepare_ip_adapter_image_embeds()` mengodekan gambar IP-Adapter menjadi embeddings yang dapat Anda simpan dan gunakan kembali. Hitung sebelumnya sekali alih-alih memuat dan mengodekan gambar yang sama setiap kali Anda menjalankan pipeline.

```py
import torch
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
pipeline = AutoPipelineForText2Image.from_pretrained(
  "stabilityai/stable-diffusion-xl-base-1.0",
  dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  weight_name="ip-adapter_sdxl.bin"
)
image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_adapter_bear_1.png")
image_embeds = pipeline.prepare_ip_adapter_image_embeds(
    ip_adapter_image=image,
    ip_adapter_image_embeds=None,
    device="cuda",
    num_images_per_prompt=1,
    do_classifier_free_guidance=True,
)
torch.save(image_embeds, "image_embeds.ipadpt")
```

Muat ulang embedding gambar dengan meneruskannya ke parameter `ip_adapter_image_embeds`. Atur `image_encoder_folder` ke `None` karena Anda tidak lagi memerlukan image encoder untuk menghasilkan embedding gambar.

> [!TIP]
> Anda juga dapat memuat embedding gambar dari sumber lain seperti ComfyUI.

```py
pipeline = AutoPipelineForText2Image.from_pretrained(
  "stabilityai/stable-diffusion-xl-base-1.0",
  dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  image_encoder_folder=None,
  weight_name="ip-adapter_sdxl.bin"
)
pipeline.set_ip_adapter_scale(0.8)
image_embeds = torch.load("image_embeds.ipadpt")
pipeline(
    prompt="a polar bear sitting in a chair drinking a milkshake",
    ip_adapter_image_embeds=image_embeds,
    negative_prompt="deformed, ugly, wrong proportion, low res, bad anatomy, worst quality, low quality",
    num_inference_steps=100,
).images[0]
```

## Masking

Masking biner memungkinkan penugasan gambar IP-Adapter ke area tertentu pada gambar output, sehingga berguna untuk menyusun beberapa gambar IP-Adapter. Setiap gambar IP-Adapter memerlukan mask biner.

Muat [IPAdapterMaskProcessor](/docs/diffusers/main/id/api/loaders/ip_adapter#diffusers.IPAdapterMaskProcessor) untuk memproses prapengolahan mask gambar. Untuk hasil terbaik, berikan `height` dan `width` keluaran untuk memastikan mask dengan rasio aspek yang berbeda berukuran dengan tepat. Jika mask input sudah sesuai dengan rasio aspek gambar yang dihasilkan, Anda tidak perlu mengatur `height` dan `width`.

```py
import torch
from diffusers import AutoPipelineForText2Image
from diffusers.image_processor import IPAdapterMaskProcessor
from diffusers.utils import load_image
pipeline = AutoPipelineForText2Image.from_pretrained(
  "stabilityai/stable-diffusion-xl-base-1.0",
  dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
mask1 = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_mask_mask1.png")
mask2 = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_mask_mask2.png")
processor = IPAdapterMaskProcessor()
masks = processor.preprocess([mask1, mask2], height=1024, width=1024)
```

  
    
    mask 1
  
  
    
    mask 2
  
Sediakan gambar IP-Adapter dan skala mereka sebagai daftar. Teruskan mask yang telah dipreproses ke `cross_attention_kwargs` di pipeline.

```py
face_image1 = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_mask_girl1.png")
face_image2 = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/ip_mask_girl2.png")
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  weight_name=["ip-adapter-plus-face_sdxl_vit-h.safetensors"]
)
pipeline.set_ip_adapter_scale([[0.7, 0.7]])
ip_images = [[face_image1, face_image2]]
masks = [masks.reshape(1, masks.shape[0], masks.shape[2], masks.shape[3])]
pipeline(
  prompt="2 girls",
  ip_adapter_image=ip_images,
  negative_prompt="monochrome, lowres, bad anatomy, worst quality, low quality",
  cross_attention_kwargs={"ip_adapter_masks": masks}
).images[0]
```

  
    
      
      Gambar IP-Adapter 1
    
    
      
      IP-Adapter gambar 2
    
  
  
    
      
      dihasilkan dengan mask
    
    
      
      dihasilkan tanpa mask
    
  
## Resep

Gabungkan IP-Adapter dengan adapter atau pipeline lain, seperti beberapa adapter, ControlNet, InstantStyle, atau LCM few-step, ketika satu gambar referensi tidak cukup.
### Beberapa IP-Adapter

Gabungkan beberapa IP-Adapter untuk menghasilkan gambar dengan gaya yang lebih beragam. Misalnya, Anda dapat menggunakan IP-Adapter Face untuk menghasilkan wajah dan karakter yang konsisten, serta IP-Adapter Plus untuk menghasilkan wajah tersebut dalam gaya tertentu.

Muat encoder gambar dengan [CLIPVisionModelWithProjection](https://huggingface.co/docs/transformers/main/en/model_doc/clip#transformers.CLIPVisionModelWithProjection).

```py
import torch
from diffusers import AutoPipelineForText2Image, DDIMScheduler
from transformers import CLIPVisionModelWithProjection
from diffusers.utils import load_image
image_encoder = CLIPVisionModelWithProjection.from_pretrained(
    "h94/IP-Adapter",
    subfolder="models/image_encoder",
    dtype=torch.float16,
)
```

Muat model dasar, penjadwal, dan IP-Adapter berikut.

- [ip-adapter-plus_sdxl_vit-h](https://huggingface.co/h94/IP-Adapter#ip-adapter-for-sdxl-10) menggunakan patch embeddings dan image encoder ViT-H
- [ip-adapter-plus-face_sdxl_vit-h](https://huggingface.co/h94/IP-Adapter#ip-adapter-for-sdxl-10) menggunakan patch embeddings dan image encoder ViT-H, tetapi dikondisikan pada gambar wajah yang dipotong

```py
pipeline = AutoPipelineForText2Image.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    dtype=torch.float16,
    image_encoder=image_encoder,
)
pipeline.scheduler = DDIMScheduler.from_config(pipeline.scheduler.config)
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  weight_name=["ip-adapter-plus_sdxl_vit-h.safetensors", "ip-adapter-plus-face_sdxl_vit-h.safetensors"]
)
pipeline.set_ip_adapter_scale([0.7, 0.3])
# enable_model_cpu_offload to reduce memory usage
pipeline.enable_model_cpu_offload()
```

Muat sebuah gambar dan folder yang berisi gambar-gambar dengan gaya tertentu untuk diterapkan.

```py
face_image = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/women_input.png")
style_folder = "https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/style_ziggy"
style_images = [load_image(f"{style_folder}/img{i}.png") for i in range(10)]
```

  
    
    gambar wajah
  
  
    
    gaya gambar
  
Lewatkan gambar gaya dan wajah sebagai daftar ke `ip_adapter_image`.

```py
generator = torch.Generator(device="cpu").manual_seed(0)
pipeline(
    prompt="wonderwoman",
    ip_adapter_image=[style_images, face_image],
    negative_prompt="monochrome, lowres, bad anatomy, worst quality, low quality",
).images[0]
```

  
    
    gambar yang dihasilkan
  
### Kontrol struktural

Untuk kontrol struktural, kombinasikan IP-Adapter dengan [ControlNet](../api/pipelines/controlnet) yang dikondisikan pada peta kedalaman, peta tepi, estimasi pose, dan lainnya.

Contoh di bawah ini memuat checkpoint [ControlNetModel](/docs/diffusers/main/id/api/models/controlnet#diffusers.ControlNetModel) yang dikondisikan pada peta kedalaman dan menggabungkannya dengan IP-Adapter.

```py
import torch
from diffusers.utils import load_image
from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
controlnet = ControlNetModel.from_pretrained(
  "lllyasviel/control_v11f1p_sd15_depth",
  dtype=torch.float16
)
pipeline = StableDiffusionControlNetPipeline.from_pretrained(
    "stable-diffusion-v1-5/stable-diffusion-v1-5",
    controlnet=controlnet,
    dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="models",
  weight_name="ip-adapter_sd15.bin"
)
```

Lewatkan peta kedalaman dan gambar IP-Adapter ke pipeline.

```py
depth_map = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/depth.png")
ip_adapter_image = load_image("https://huggingface.co/datasets/YiYiXu/testing-images/resolve/main/statue.png")
pipeline(
  prompt="best quality, high quality",
  image=depth_map,
  ip_adapter_image=ip_adapter_image,
  negative_prompt="monochrome, lowres, bad anatomy, worst quality, low quality",
).images[0]
```

  
    
    Gambar IP-Adapter
  
  
    
    peta kedalaman
  
  
    
    gambar yang dihasilkan
  
### Kontrol gaya dan tata letak

Untuk kontrol gaya dan tata letak, kombinasikan IP-Adapter dengan [InstantStyle](https://huggingface.co/papers/2404.02733). InstantStyle memisahkan *gaya* (warna, tekstur, kesan keseluruhan) dari *konten* dan menerapkan gaya hanya pada blok yang spesifik untuk gaya, sehingga area konten tetap utuh. Hal ini memberikan konsistensi gaya yang lebih kuat dan kontrol tata letak yang lebih jelas.

Aktifkan IP-Adapter hanya pada lapisan tertentu dengan [set_ip_adapter_scale()](/docs/diffusers/main/id/api/loaders/ip_adapter#diffusers.loaders.IPAdapterMixin.set_ip_adapter_scale). Contoh di bawah ini mengaktifkannya pada `block_2` (layout) bagian down dan `block_0` (style) bagian up dari model.

```py
import torch
from diffusers import AutoPipelineForText2Image
from diffusers.utils import load_image
pipeline = AutoPipelineForText2Image.from_pretrained(
  "stabilityai/stable-diffusion-xl-base-1.0",
  dtype=torch.float16
).to("cuda")  # or "mps", "xpu", "cpu"
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="sdxl_models",
  weight_name="ip-adapter_sdxl.bin"
)
scale = {
    "down": {"block_2": [0.0, 1.0]},
    "up": {"block_0": [0.0, 1.0, 0.0]},
}
pipeline.set_ip_adapter_scale(scale)
```

Muat gambar gaya dan hasilkan gambar.

```py
style_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg")
pipeline(
    prompt="a cat, masterpiece, best quality, high quality",
    ip_adapter_image=style_image,
    negative_prompt="text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry",
    guidance_scale=5,
).images[0]
```

  
    
    style image
  
  
    
    gambar yang dihasilkan
  
Gambar-gambar di bawah ini membandingkan aktivasi gaya-only InstantStyle (hanya `block_0`) dengan mengaktifkan IP-Adapter di semua lapisan. Pengaturan semua lapisan biasanya mengikuti prompt gambar lebih kuat dan dapat mengurangi keragaman. Pilih skala gaya-only jika Anda menginginkan perilaku pelestarian tata letak dari InstantStyle.

> [!TIP]
> Anda tidak perlu menentukan semua lapisan dalam kamus `scale`. Lapisan yang tidak disertakan diatur ke 0, yang berarti IP-Adapter dinonaktifkan.

```py
scale = {
    "up": {"block_0": [0.0, 1.0, 0.0]},
}
pipeline.set_ip_adapter_scale(scale)
pipeline(
    prompt="a cat, masterpiece, best quality, high quality",
    ip_adapter_image=style_image,
    negative_prompt="text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry",
    guidance_scale=5,
).images[0]
```

  
    
    hanya-gaya (blok_0 atas)
  
  
    
    semua lapisan
  
### Generasi instan

Gabungkan IP-Adapter dengan [LCM](../api/pipelines/latent_consistency_models) LoRA untuk generasi beberapa langkah.

```py
import torch
from diffusers import DiffusionPipeline, LCMScheduler
from diffusers.utils import load_image
pipeline = DiffusionPipeline.from_pretrained(
  "sd-dreambooth-library/herge-style",
  dtype=torch.float16
)
pipeline.load_ip_adapter(
  "h94/IP-Adapter",
  subfolder="models",
  weight_name="ip-adapter_sd15.bin"
)
pipeline.load_lora_weights("latent-consistency/lcm-lora-sdv1-5")
pipeline.scheduler = LCMScheduler.from_config(pipeline.scheduler.config)
pipeline.enable_model_cpu_offload()
pipeline.set_ip_adapter_scale(0.4)
ip_adapter_image = load_image("https://user-images.githubusercontent.com/24734142/266492875-2d50d223-8475-44f0-a7c6-08b51cb53572.png")
pipeline(
    prompt="herge_style woman in armor, best quality, high quality",
    ip_adapter_image=ip_adapter_image,
    num_inference_steps=4,
    guidance_scale=1,
).images[0]
```

  
    
    gambar yang dihasilkan
  
Xet Storage Details

Size:: 23.3 kB
Xet hash:: 258cc6bafb8c7c082253be2fce7887f2d3a22a8b473440651f0ccdf4fb788b4f
Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.