README.md

---
title: DiffusionDB SD 1.5 LoRA
emoji: 🦉
colorFrom: indigo
colorTo: gray
sdk: gradio
sdk_version: 6.25.0
python_version: '3.12'
app_file: app.py
pinned: false
license: creativeml-openrail-m
short_description: SD 1.5 style LoRA with a live before/after slider
tags:
- text-to-image
- stable-diffusion
- lora
- diffusers
- gradio
models:
- whosouravsharma/diffusiondb-sd15-lora
datasets:
- whosouravsharma/text-to-image-diffusiondb-2M
---

# DiffusionDB SD 1.5 LoRA

Generate images with Stable Diffusion 1.5 plus a rank-32 **style LoRA**
fine-tuned on 13,598 prompt–image pairs from
[DiffusionDB](https://huggingface.co/datasets/poloclub/diffusiondb). The
adapter gives SD 1.5 a **more saturated, higher-contrast, illustrative look**,
and it follows prompts just as well as the base model.

![SD 1.5 base vs. with the LoRA, same prompt and seed](https://huggingface.co/whosouravsharma/diffusiondb-sd15-lora/resolve/main/images/pair-006.jpg)

**Set LoRA strength to 0** to render plain SD 1.5 at the same seed. At
strength 0 the output is pixel-identical to the base model, so the slider is
a true before/after comparison.

> This Space records visitor IP addresses. See [Privacy](#privacy).

This Space is the **frontend** of a two-Space system. Image generation runs
on a GPU in the
[`diffusiondb-sd15-lora-inference`](https://huggingface.co/spaces/whosouravsharma/diffusiondb-sd15-lora-inference)
Space.

## How it works

1. You enter a prompt and, optionally, a negative prompt, step count,
   guidance, LoRA strength and seed.
2. This Space connects to the inference Space with `gradio_client`. If the
   inference Space is asleep, that connection wakes it (see
   [Cold starts](#cold-starts)).
3. The inference Space loads SD 1.5 (fp16) and the LoRA adapter from the
   Hub, applies the requested LoRA strength, and generates one 512×512 image
   with the DPM-Solver++ scheduler.
4. The image and the seed used come back here and are shown.

```
Browser ──▶ UI Space (this, Gradio) ──gradio_client /generate──▶ Inference Space (T4 GPU)
                                                                   │  SD 1.5 fp16
                                                                   │  + LoRA checkpoint-4240
                                                                   │  DPM-Solver++ scheduler
            ◀──────────────── image + seed ────────────────────────┘
```

## Components

| Component | Role |
|---|---|
| **This Space** | Gradio interface. It runs no model code, holds no weights and doesn't install torch |
| [Inference Space](https://huggingface.co/spaces/whosouravsharma/diffusiondb-sd15-lora-inference) | Loads the pipeline and serves one `/generate` endpoint. Runs on a T4 GPU |
| [Model repo](https://huggingface.co/whosouravsharma/diffusiondb-sd15-lora) | LoRA checkpoints, training and evaluation code, and the full model card |
| [Dataset repo](https://huggingface.co/datasets/whosouravsharma/text-to-image-diffusiondb-2M) | The cleaned training data, the cached latents, and the 50 held-out evaluation prompts |

Keeping the interface separate means this Space doesn't need a GPU. Only the
inference Space costs GPU time.

## Model

| | |
|---|---|
| Base | [`stable-diffusion-v1-5/stable-diffusion-v1-5`](https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5) |
| Adapter | LoRA rank 32 on the UNet attention layers, `checkpoint-4240` (the final step) |
| Training | 10 epochs on 13,598 images at 512×512, with the text encoder frozen, in 2.7 hours on one A10G |
| Defaults here | 25 steps, guidance 7.5, LoRA strength 1.0 (adjustable from 0 to 1.5) |

The example prompts in the UI are real prompts from the held-out evaluation
set. The model never trained on them.

## Evaluation at a glance

The model was evaluated on 506 held-out prompts, against the real DiffusionDB
images for those prompts:

| | KID ×10³ ↓ | CLIP score |
|---|---|---|
| SD 1.5 base | 0.14 ± 0.32 | 28.94 |
| + LoRA `checkpoint-4240` | 2.88 ± 0.61 | 28.98 |

- **Prompt-following is unchanged.** The two CLIP scores are within noise of
  each other.
- **The LoRA adds its own style; it doesn't reproduce DiffusionDB's.**
  DiffusionDB is itself SD 1.x output, and plain SD 1.5 already matches it
  (KID ≈ 0). The adapter moves outputs away from it, toward the saturated,
  high-contrast look in the gallery.
- **No extra risk measured.** The safety-checker flag rate is the same as
  base (2.6%), and a nearest-neighbour check finds no copying of training
  images.

The [model card](https://huggingface.co/whosouravsharma/diffusiondb-sd15-lora)
has the full protocol, the loss curve, the training progression, failure cases
and every figure.

## Intended use

**Use it for** giving SD 1.5 images a punchier, illustrative finish, and for
seeing what a LoRA fine-tune changes compared with its base model.

**Don't use it for** muted or pastel palettes (the adapter overrides them),
resolutions other than 512×512, or photorealistic images of real people.

## API

The inference Space can be called directly:

```python
from gradio_client import Client

client = Client("whosouravsharma/diffusiondb-sd15-lora-inference")
image_path, seed = client.predict(
    "a steampunk owl inside a glass jar, intricate detail",  # prompt
    "",      # negative prompt
    25,      # steps (at most 50)
    7.5,     # guidance
    1.0,     # LoRA strength; 0 = plain SD 1.5
    -1,      # seed; -1 = random
    api_name="/generate",
)
```

## Cold starts

The inference Space goes to sleep after 5 minutes without requests. Waking
it means scheduling a GPU, starting the container and loading about 2 GB of
weights, so the first image after a wake is slow (the app estimates about
40 seconds). Later images take a few seconds.

This Space treats that as a wait, not an error. It retries the connection
every 5 seconds for up to 10 minutes and shows **Starting the server…**
with the time elapsed. It shows an error only when waiting can't help: when
the inference Space is `PAUSED`, or in `RUNTIME_ERROR`, `BUILD_ERROR` or
`CONFIG_ERROR`, it has to be restarted from its Settings page.

## Privacy

For each generation, this Space records:

- the visitor's **IP address**, from the first entry of `x-forwarded-for`
- browser **language**, **user agent** and Gradio session ID
- the generation settings and the seed (never the image)

With LangSmith tracing on, which it is in this deployment, this is stored as
run metadata in LangSmith. It is also written to the Space's container logs,
which only the owner can see. An IP address counts as personal data under
GDPR and UK GDPR, and under several US state laws.

| Variable | Default | Effect |
|---|---|---|
| `LANGSMITH_LOG_IP` | `on` | `off` stops recording the raw IP address |
| `LANGSMITH_TRACING` | unset | `true` sends the data above to LangSmith |

## Setup (for forks / redeploys)

| Name | Type | Purpose |
|---|---|---|
| `HF_TOKEN` | secret | **Required.** Used to connect to the inference Space. Without it, every generation stops with an error |
| `BACKEND_SPACE` | variable | Optional. Defaults to `whosouravsharma/diffusiondb-sd15-lora-inference` |
| `LANGSMITH_API_KEY` | secret | Optional. Only needed with tracing |
| `LANGSMITH_TRACING`, `LANGSMITH_PROJECT` | variable | Optional. Turn LangSmith tracing on and name the project |

With tracing on, the UI and the inference Space appear in one trace, not
two. The UI sends `langsmith-trace` and `baggage` headers with its request,
so the backend's `sd15_lora_generate` run is recorded as a child of the
UI's `ui_generate` run.

## Limitations

- **The style is fixed:** saturation and contrast go up on every prompt,
  including ones that ask for muted colours.
- **Some detailed scenes get simplified** or lose their composition.
- Output is fixed at 512×512, the resolution the adapter was trained at.
- The training images were themselves made by SD 1.x, so the adapter
  reproduces that model's artifacts.
- **The safety checker is off** in the inference pipeline, so images are
  not screened. The training data was NSFW-filtered, but that filter is not
  a guarantee.
- The first image after the inference Space has been idle is slow (see
  [Cold starts](#cold-starts)).

## License

CreativeML OpenRAIL-M, the same license as the Stable Diffusion 1.5 base
model. Training data comes from
[DiffusionDB](https://huggingface.co/datasets/poloclub/diffusiondb) (CC0 1.0;
Wang et al., 2022, [arXiv:2210.14896](https://arxiv.org/abs/2210.14896)).