Text-to-Image
Diffusers
Safetensors
StableDiffusionXLInpaintPipeline
stable-diffusion-xl
stable-diffusion-xl-diffusers
inpainting
Instructions to use diffusers/stable-diffusion-xl-1.0-inpainting-0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use diffusers/stable-diffusion-xl-1.0-inpainting-0.1 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import AutoPipelineForInpainting from diffusers.utils import load_image # switch to "mps" for apple devices pipe = AutoPipelineForInpainting.from_pretrained("diffusers/stable-diffusion-xl-1.0-inpainting-0.1", dtype=torch.float16, device_map="cuda") img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png" mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png" image = load_image(img_url).resize((1024, 1024)) mask_image = load_image(mask_url).resize((1024, 1024)) prompt = "a tiger sitting on a park bench" generator = torch.Generator(device="cuda").manual_seed(0) image = pipe( prompt=prompt, image=image, mask_image=mask_image, guidance_scale=8.0, num_inference_steps=20, # steps between 15 and 30 work well for us strength=0.99, # make sure to use `strength` below 1.0 generator=generator, ).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
fint tuning
#22
by cahodk - opened
Hi, I am trying to fine tune this model on my own token, but I'm not really having any success. I am basically doing this:
https://github.com/huggingface/diffusers/blob/main/examples/research_projects/dreambooth_inpaint/README.md (this first example)
But it fails with different kinds of errors. Currently this:
File "/usr/local/lib/python3.10/dist-packages/diffusers/models/unets/unet_2d_condition.py", line 973, in get_aug_embed
if "text_embeds" not in added_cond_kwargs:
TypeError: argument of type 'NoneType' is not iterable
Is this the wrong approach? Can I even take a diffuser model and fine tune it like this?