multimodalart's picture
multimodalart HF Staff
Upload folder using huggingface_hub
bad58f8 verified
|
Raw History Blame Contribute Delete
2.2 kB
---
title: RADS — Steering Away from Memorization in Text-to-Image Diffusion
emoji: 🧭
colorFrom: green
colorTo: purple
sdk: gradio
sdk_version: 6.32.0
app_file: app.py
python_version: "3.12"
startup_duration_timeout: 30m
short_description: Steer SD v1.4 away from memorization at inference time
---
# RADS: Reachability-Constrained Reinforcement Learning for Diffusion Steering
Demo for the paper **"Steering Away from Memorization: Reachability-Constrained
Reinforcement Learning for Text-to-Image Diffusion"** ([HF paper page](https://huggingface.co/papers/2603.00140),
[project page](https://s-karnik.github.io/rads-memorization-project-page)).
## What this demo does
RADS is an **inference-time** steering controller for text-to-image diffusion.
A Soft Actor-Critic policy — trained offline against a reachability-based
safety critic — perturbs the prompt's caption embedding in a learned 64-d
latent space at every denoising step, keeping generation inside a safe
(Non-Memorizing) region of the diffusion trajectory. The diffusion backbone
(Stable Diffusion v1.4, DDIM, T=50, g=7.5 — the paper's evaluation setting)
is **never modified**.
For the same prompt and the **same initial latent**, this demo generates:
- **Left:** the plain SD v1.4 baseline (steering strength 0)
- **Right:** the RADS-steered image (steering strength 1, the paper default)
Both images come from an identical seed, so every visible difference is caused
by steering alone. Example prompts are memorization-prone prompts from the
paper's evaluation sets (Webster et al. 2023 and MemBench).
## Checkpoints
SAC agent checkpoints (encoder + actor, training seeds 41–45) and the CLIP
caption-embedding autoencoder are loaded from
[`S-Karnik/rads-model-checkpoints`](https://huggingface.co/S-Karnik/rads-model-checkpoints).
The official implementation lives at
[`S-Karnik/rads-paper-repo`](https://github.com/S-Karnik/rads-paper-repo) (MIT).
## Citation
```bibtex
@inproceedings{karnik2026rads,
title = {Steering Away from Memorization: Reachability-Constrained
Reinforcement Learning for Text-to-Image Diffusion},
author = {Karnik, Shreyas and others},
year = {2026}
}
```