Text-to-Image
Diffusers
Safetensors
StableDiffusionPipeline
stable-diffusion
stable-diffusion-diffusers
textual_inversion
diffusers-training
Instructions to use aiwithi/Story_Generation with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use aiwithi/Story_Generation with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", dtype=torch.bfloat16, device_map="cuda") pipe.load_textual_inversion("aiwithi/Story_Generation") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
|
Download diffusers/examples/textual_inversion/README_sdxl.md from aiwithi/Story_Generation: direct link, hf CLI and curl.
- Browser
- Download file 1.43 kB
-
https://huggingface.co/aiwithi/Story_Generation/resolve/main/diffusers/examples/textual_inversion/README_sdxl.md
- Command line
-
hf download hf://aiwithi/Story_Generation/diffusers/examples/textual_inversion/README_sdxl.md
-
curl -L -o README_sdxl.md https://huggingface.co/aiwithi/Story_Generation/resolve/main/diffusers/examples/textual_inversion/README_sdxl.md
1.43 kB
| ## Textual Inversion fine-tuning example for SDXL | |
| ```sh | |
| export MODEL_NAME="stabilityai/stable-diffusion-xl-base-1.0" | |
| export DATA_DIR="./cat" | |
| accelerate launch textual_inversion_sdxl.py \ | |
| --pretrained_model_name_or_path=$MODEL_NAME \ | |
| --train_data_dir=$DATA_DIR \ | |
| --learnable_property="object" \ | |
| --placeholder_token="<cat-toy>" \ | |
| --initializer_token="toy" \ | |
| --mixed_precision="bf16" \ | |
| --resolution=768 \ | |
| --train_batch_size=1 \ | |
| --gradient_accumulation_steps=4 \ | |
| --max_train_steps=500 \ | |
| --learning_rate=5.0e-04 \ | |
| --scale_lr \ | |
| --lr_scheduler="constant" \ | |
| --lr_warmup_steps=0 \ | |
| --save_as_full_pipeline \ | |
| --output_dir="./textual_inversion_cat_sdxl" | |
| ``` | |
| Training of both text encoders is supported. | |
| ### Inference Example | |
| Once you have trained a model using above command, the inference can be done simply using the `StableDiffusionXLPipeline`. | |
| Make sure to include the `placeholder_token` in your prompt. | |
| ```python | |
| from diffusers import StableDiffusionXLPipeline | |
| import torch | |
| model_id = "./textual_inversion_cat_sdxl" | |
| pipe = StableDiffusionXLPipeline.from_pretrained(model_id,torch_dtype=torch.float16).to("cuda") | |
| prompt = "A <cat-toy> backpack" | |
| image = pipe(prompt, num_inference_steps=50, guidance_scale=7.5).images[0] | |
| image.save("cat-backpack.png") | |
| image = pipe(prompt="", prompt_2=prompt, num_inference_steps=50, guidance_scale=7.5).images[0] | |
| image.save("cat-backpack-prompt_2.png") | |
| ``` | |