WAI14-VPred
Collection
4 items • Updated
Image-to-Image
Diffusers
Safetensors
English
StableDiffusionXLInpaintPipeline
anime
art
inpaint
inpainting
stable-diffusion
stable-diffusion-xl
pip install -U diffusers transformers accelerate
import torch
from diffusers import AutoPipelineForInpainting
from diffusers.utils import load_image
# switch to "mps" for apple devices
pipe = AutoPipelineForInpainting.from_pretrained("ShinoharaHare/Waifu-Colorize-XL", dtype=torch.float16, device_map="cuda")
img_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo.png"
mask_url = "https://raw.githubusercontent.com/CompVis/latent-diffusion/main/data/inpainting_examples/overture-creations-5sI6fQgYIuo_mask.png"
image = load_image(img_url).resize((1024, 1024))
mask_image = load_image(mask_url).resize((1024, 1024))
prompt = "a tiger sitting on a park bench"
generator = torch.Generator(device="cuda").manual_seed(0)
image = pipe(
prompt=prompt,
image=image,
mask_image=mask_image,
guidance_scale=8.0,
num_inference_steps=20, # steps between 15 and 30 work well for us
strength=0.99, # make sure to use `strength` below 1.0
generator=generator,
).images[0]Waifu-Colorize-XL
Waifu-Colorize-XL is an SDXL-based model specifically designed for colorizing anime-themed images.
| Task | Model |
|---|---|
| Generation | WAI-NSFW-illustrious-SDXL-V14.0-V-Prediction |
| Inpainting | Waifu-Inpaint-XL |
| Colorizing | Waifu-Colorize-XL |
Overview
Waifu-Colorize-XL is further trained from Waifu-Inpaint-XL, enabling it to colorize grayscale or binarized images.
Model Details
- Developed by: ShinoharaHare
- Model type: Diffusion based image-to-image generative model for colorizing
- Language(s) (NLP): English
- License: CreativeML Open RAIL++-M
- Finetuned from: Waifu-Inpaint-XL
🧨 Diffusers
import torch
from diffusers import StableDiffusionXLInpaintPipeline
from diffusers.utils import load_image
pipeline = StableDiffusionXLInpaintPipeline.from_pretrained(
'ShinoharaHare/Waifu-Colorize-XL',
torch_dtype=torch.half
)
pipeline.to('cuda')
grayscale_image = load_image('https://cdn-uploads.huggingface.co/production/uploads/630ed69a31970d1cd4fd575d/ZusF3zcH-auhPgSxY0LKs.png')
prompt = '' # Write your own prompt if you want.
masked_image_latents = pipeline.vae.encode(
pipeline.image_processor.preprocess(grayscale_image).to(pipeline.device, pipeline.dtype)
)[0].sample() * pipeline.vae.config.scaling_factor
colorized_image = pipeline(
prompt=prompt,
image=grayscale_image,
mask_image=torch.ones((1, grayscale_image.height, grayscale_image.width)),
masked_image_latents=masked_image_latents,
num_inference_steps=28,
strength=1.0,
guidance_scale=5.0 if prompt else 0.0,
height=grayscale_image.height,
width=grayscale_image.width
).images[0]
colorized_image.show()
ComfyUI
This model requires a custom node to function correctly.
Please install InpaintModelConditioningAdvanced first.
- Downloads last month
- 19
Model tree for ShinoharaHare/Waifu-Colorize-XL
Base model
KBlueLeaf/kohaku-xl-beta5 Finetuned
ShinoharaHare/Waifu-Inpaint-XL
# Gated model: Login with a HF token with gated access permission hf auth login