Name folders after the FSCG-8 categories; add the model card

README.md

@@ -0,0 +1,77 @@
+---
+library_name: diffusers
+license: creativeml-openrail-m
+base_model: stable-diffusion-v1-5/stable-diffusion-v1-5
+pipeline_tag: text-to-image
+tags:
+- stable-diffusion
+- text-to-image
+- procreate
+datasets:
+- Jacklu0831/few-shot-creative-generation-8
+---
+
+# ProCreate checkpoints
+
+Eight Stable Diffusion 1.5 UNets, each fine-tuned for 2,000 steps on one category of [FSCG-8](https://huggingface.co/datasets/Jacklu0831/few-shot-creative-generation-8) (50 caption-image pairs per category), released with **ProCreate, Don't Reproduce! Propulsive Energy Diffusion for Creative Generation** (ECCV 2024).
+
+ProCreate is a sampling method that pushes generated images away from the reference images, so a model fine-tuned on a few examples produces novel samples in the same style instead of replicating its training data.
+
+[Paper](https://arxiv.org/abs/2408.02226) · [Project page](https://agenticlearning.ai/procreate/) · [Code](https://github.com/agentic-learning-ai-lab/procreate-diffusion) · [Dataset](https://huggingface.co/datasets/Jacklu0831/few-shot-creative-generation-8)
+
+## Checkpoints
+
+One folder per FSCG-8 category, each holding a fine-tuned UNet (`UNet2DConditionModel`, diffusers format):
+
+| Folder | Fine-tuned on | Sample prompt |
+| --- | --- | --- |
+| `amedeo_modigliani` | Amedeo Modigliani paintings | a Amedeo Modigliani painting of a boy in a suit and hat |
+| `apple` | Apple product designs | an Apple VR headset |
+| `burberry` | Burberry designs | a Burberry stuffed bear |
+| `frank_gehry` | Frank Gehry architecture | a twisting tall apartment building, designed by Frank Gehry |
+| `nouns` | Nouns characters | a mountain Nouns character |
+| `one_piece` | One Piece characters | a One Piece man in a purple robe |
+| `pokemon` | Pokemon | a jellyfish Pokemon |
+| `rococo` | Rococo style art and furniture | a Rococo style bed |
+
+## Usage
+
+Load a category's UNet into a Stable Diffusion 1.5 pipeline:
+
+```python
+import torch
+from diffusers import StableDiffusionPipeline, UNet2DConditionModel
+
+pipe = StableDiffusionPipeline.from_pretrained(
+    "stable-diffusion-v1-5/stable-diffusion-v1-5", torch_dtype=torch.float16
+).to("cuda")
+pipe.unet = UNet2DConditionModel.from_pretrained(
+    "Jacklu0831/procreate-diffusion", subfolder="apple", torch_dtype=torch.float16
+).to("cuda")
+image = pipe("an Apple VR headset").images[0]
+```
+
+This samples from the fine-tuned model directly. To sample with ProCreate, use `src/inference.py` in the [code repository](https://github.com/agentic-learning-ai-lab/procreate-diffusion), which downloads the checkpoint automatically:
+
+```bash
+python src/inference.py \
+    --dataset_dir few-shot-creative-generation-8/apple \
+    --unet_ckpt_dir Jacklu0831/procreate-diffusion \
+    --unet_subfolder apple \
+    --prompt "an Apple VR headset"
+```
+
+## License
+
+The weights are a derivative of Stable Diffusion 1.5 and are released under the [CreativeML Open RAIL-M license](https://huggingface.co/spaces/CompVis/stable-diffusion-license).
+
+## Citation
+
+```bibtex
+@inproceedings{lu2024procreate,
+  title     = {ProCreate, Don't Reproduce! Propulsive Energy Diffusion for Creative Generation},
+  author    = {Lu, Jack and Teehan, Ryan and Ren, Mengye},
+  booktitle = {European Conference on Computer Vision (ECCV)},
+  year      = {2024}
+}
+```