Text-to-Image
Diffusers
Safetensors
StableDiffusionPipeline
stable-diffusion
stable-diffusion-diffusers
textual_inversion
diffusers-training
Instructions to use aiwithi/Story_Generation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use aiwithi/Story_Generation with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.bfloat16, device_map="cuda") pipe.load_textual_inversion("aiwithi/Story_Generation") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
|
Download diffusers/examples/textual_inversion/README_sdxl.md from aiwithi/Story_Generation: direct link, hf CLI and curl.
- Browser
- Download file 1.43 kB
-
https://huggingface.co/aiwithi/Story_Generation/resolve/main/diffusers/examples/textual_inversion/README_sdxl.md
- Command line
-
hf download hf://aiwithi/Story_Generation/diffusers/examples/textual_inversion/README_sdxl.md
-
curl -L -o README_sdxl.md https://huggingface.co/aiwithi/Story_Generation/resolve/main/diffusers/examples/textual_inversion/README_sdxl.md
1.43 kB
Textual Inversion fine-tuning example for SDXL
export MODEL_NAME="stabilityai/stable-diffusion-xl-base-1.0"
export DATA_DIR="./cat"
accelerate launch textual_inversion_sdxl.py \
--pretrained_model_name_or_path=$MODEL_NAME \
--train_data_dir=$DATA_DIR \
--learnable_property="object" \
--placeholder_token="<cat-toy>" \
--initializer_token="toy" \
--mixed_precision="bf16" \
--resolution=768 \
--train_batch_size=1 \
--gradient_accumulation_steps=4 \
--max_train_steps=500 \
--learning_rate=5.0e-04 \
--scale_lr \
--lr_scheduler="constant" \
--lr_warmup_steps=0 \
--save_as_full_pipeline \
--output_dir="./textual_inversion_cat_sdxl"
Training of both text encoders is supported.
Inference Example
Once you have trained a model using above command, the inference can be done simply using the StableDiffusionXLPipeline.
Make sure to include the placeholder_token in your prompt.
from diffusers import StableDiffusionXLPipeline
import torch
model_id = "./textual_inversion_cat_sdxl"
pipe = StableDiffusionXLPipeline.from_pretrained(model_id,torch_dtype=torch.float16).to("cuda")
prompt = "A <cat-toy> backpack"
image = pipe(prompt, num_inference_steps=50, guidance_scale=7.5).images[0]
image.save("cat-backpack.png")
image = pipe(prompt="", prompt_2=prompt, num_inference_steps=50, guidance_scale=7.5).images[0]
image.save("cat-backpack-prompt_2.png")