WAI14-VPred
Collection
4 items • Updated
Image-to-Image
Diffusers
Safetensors
English
StableDiffusionXLInpaintPipeline
anime
art
inpaint
inpainting
stable-diffusion
stable-diffusion-xl
pip install -U diffusers transformers accelerate
import torch
from diffusers import AutoPipelineForInpainting
from diffusers.utils import load_image
# switch to "mps" for apple devices
pipe = AutoPipelineForInpainting.from_pretrained("ShinoharaHare/Waifu-Inpaint-XL", dtype=torch.float16, device_map="cuda")
img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png"
mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png"
image = load_image(img_url).resize((1024, 1024))
mask_image = load_image(mask_url).resize((1024, 1024))
prompt = "a tiger sitting on a park bench"
generator = torch.Generator(device="cuda").manual_seed(0)
image = pipe(
prompt=prompt,
image=image,
mask_image=mask_image,
guidance_scale=8.0,
num_inference_steps=20, # steps between 15 and 30 work well for us
strength=0.99, # make sure to use `strength` below 1.0
generator=generator,
).images[0]Waifu-Inpaint-XL
Waifu-Inpaint-XL is an SDXL-based model specifically designed for inpainting anime-themed images.
| Task | Model |
|---|---|
| Generation | WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction |
| Inpainting | Waifu-Inpaint-XL |
| Colorizing | Waifu-Colorize-XL |
Overview
Waifu-Inpaint-XL is initialized from WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction, with its input channels expanded from 4 to 9. Among these, 1 additional channel is dedicated to the mask, and 4 extra channels are used for masked latents.
Model Details
- Developed by: ShinoharaHare
- Model type: Diffusion based text-to-image/image-to-image generative model for inpainting
- Language(s) (NLP): English
- License: CreativeML Open RAIL++-M
- Finetuned from: WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction
🧨 Diffusers
import torch
from diffusers import StableDiffusionXLInpaintPipeline
from diffusers.utils import load_image
pipeline = StableDiffusionXLInpaintPipeline.from_pretrained(
'ShinoharaHare/Waifu-Inpaint-XL',
torch_dtype=torch.half
)
pipeline.to('cuda')
image = load_image('https://cdn-uploads.huggingface.co/production/uploads/630ed69a31970d1cd4fd575d/tPo5oPJQpxWamM-tGIYqj.png')
mask_image = load_image('https://cdn-uploads.huggingface.co/production/uploads/630ed69a31970d1cd4fd575d/QpmzmgROUM0eP53Cxx2Ih.png', lambda x: x.convert('L'))
inpainted_image = pipeline(
prompt='blue eyes, holding red spider lily in hand',
image=image,
mask_image=mask_image,
num_inference_steps=28,
guidance_scale=5.0,
height=image.height,
width=image.width,
generator=torch.Generator(pipeline.device).manual_seed(5)
).images[0]
inpainted_image.show()
ComfyUI
- Downloads last month
- 625
Model tree for ShinoharaHare/Waifu-Inpaint-XL
Base model
KBlueLeaf/kohaku-xl-beta5
# Gated model: Login with a HF token with gated access permission hf auth login