Instructions to use torpedo2003/SAD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use torpedo2003/SAD with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("torpedo2003/SAD", dtype=torch.bfloat16, device_map="cuda") prompt = "Turn this cat into a dog" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") image = pipe(image=input_image, prompt=prompt).images[0] - Notebooks
- Google Colab
- Kaggle
File size: 3,555 Bytes
2d23e77 e51dc43 2d23e77 6eb7beb 2d23e77 6eb7beb 2d23e77 be909b8 6eb7beb 3bae55e 6eb7beb 2d23e77 6eb7beb 2d23e77 6eb7beb 2d23e77 6eb7beb 2d23e77 6eb7beb 2d23e77 6eb7beb 2d23e77 6eb7beb 2d23e77 0106207 2d23e77 0106207 3bae55e 2d23e77 e51dc43 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
license: mit
library_name: diffusers
pipeline_tag: image-to-image
tags:
- image-generation
- diffusion
- vae
- decoder
---
# Sample-Aligned Decoders for Latent Generative Models
[Yu Lei](https://torpedo2003.github.io/)<sup>1,2</sup>, Yi Li<sup>2</sup>, Hongcheng Gao<sup>3</sup>, Yipeng Wei<sup>2</sup>, Liang Li<sup>1</sup>, Li Su<sup>2</sup>, Qingming Huang<sup>2</sup>
<sup>1</sup>Institute of Computing Technology, Chinese Academy of Sciences<br>
<sup>2</sup>University of Chinese Academy of Sciences <sup>3</sup>Tsinghua University
[Paper](https://arxiv.org/abs/2611.22458) 路 [Project page](https://torpedo2003.github.io/SAD/) 路 [Code](https://github.com/torpedo2003/SAD)
These are drop-in decoders for latent generative models. Each file holds only the decoder (`decoder.*`, `post_quant_conv.*`) of
the original autoencoder. It was fine-tuned with SAD on latents from one generator. The encoder, the latent space and the
generator stay unchanged, and so does the inference cost.
## Usage
```bash
pip install git+https://github.com/torpedo2003/SAD
```
```python
from diffusers import DiTPipeline
from sad import load_sad_decoder
pipe = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256").to("cuda")
load_sad_decoder(pipe.vae, "dit-xl2-256/sad-gan") # swaps the decoder in place
images = pipe(class_labels=[279, 388], num_inference_steps=250, guidance_scale=1.5).images
```
You can also load the weights without our package:
```python
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
pipe.vae.load_state_dict(load_file(hf_hub_download("torpedo2003/SAD", "dit-xl2-256/sad-gan.safetensors")), strict=False)
```
The VA-VAE and E2E-VAE decoders are not diffusers models. Load them into the autoencoder classes of the
[LightningDiT](https://github.com/hustvl/LightningDiT) and [REPA-E](https://github.com/End2End-Diffusion/REPA-E) code bases,
using the same `load_sad_decoder` call.
## Files
Each decoder is trained on latents from its own generator and sampler, so use the one that matches your pipeline.
| File | Generator | Tokenizer | Sampler |
|---|---|---|---|
| `dit-xl2-256/sad-gan.safetensors` | DiT-XL/2 | SD-VAE (ft-EMA) | official 250-step DDPM, cfg 1.5 |
| `dit-xl2-256/sad-mmd.safetensors` | DiT-XL/2 | SD-VAE (ft-EMA) | official 250-step DDPM, cfg 1.5 |
| `dit-xl2-256-dpm50/sad-gan.safetensors` | DiT-XL/2 | SD-VAE (ft-EMA) | 50-step DPM-Solver++, cfg 1.5 |
| `dit-xl2-256-dpm50/sad-mmd.safetensors` | DiT-XL/2 | SD-VAE (ft-EMA) | 50-step DPM-Solver++, cfg 1.5 |
| `sit-xl2-repa-256/sad-gan.safetensors` | SiT-XL/2 + REPA | SD-VAE (ft-EMA) | SDE 250 steps |
| `lightningdit-xl-vavae-256/sad-gan.safetensors` | LightningDiT-XL | VA-VAE | official recipe |
| `repa-e-e2evae-256/sad-gan.safetensors` | REPA-E (SiT-XL/1) | E2E-VAE | SDE 250 steps |
| `pixart-alpha-256/sad-gan.safetensors` | PixArt-伪 256px | SD-VAE (ft-EMA) | 20 steps, guidance 4.5 |
Every decoder was trained for 2,000 steps, about one GPU-hour. Training used the GAN discrepancy in DINOv2 space
(位 = 0.05 on ImageNet, 0.01 for PixArt-伪) or MMD (位 = 1), plus the L1 encoder anchor.
To train a decoder for your own generator, see the [training guide](https://github.com/torpedo2003/SAD/tree/main/training).
## Citation
```bibtex
@article{lei2026sad,
title = {Sample-Aligned Decoders for Latent Generative Models},
author = {Lei, Yu and Li, Yi and Gao, Hongcheng and Wei, Yipeng and Li, Liang and Su, Li and Huang, Qingming},
journal = {arXiv preprint arXiv:2611.22458},
year = {2026}
}
``` |