Instructions to use torpedo2003/SAD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use torpedo2003/SAD with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("torpedo2003/SAD", dtype=torch.bfloat16, device_map="cuda") prompt = "Turn this cat into a dog" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png") image = pipe(image=input_image, prompt=prompt).images[0] - Notebooks
- Google Colab
- Kaggle
|
Download README.md from torpedo2003/SAD: direct link, hf CLI and curl.
- Browser
- Download file 3.56 kB
-
https://huggingface.co/torpedo2003/SAD/resolve/main/README.md
- Command line
-
hf download hf://torpedo2003/SAD/README.md
-
curl -L -o README.md https://huggingface.co/torpedo2003/SAD/resolve/main/README.md
3.56 kB
| license: mit | |
| library_name: diffusers | |
| pipeline_tag: image-to-image | |
| tags: | |
| - image-generation | |
| - diffusion | |
| - vae | |
| - decoder | |
| # Sample-Aligned Decoders for Latent Generative Models | |
| [Yu Lei](https://torpedo2003.github.io/)<sup>1,2</sup>, Yi Li<sup>2</sup>, Hongcheng Gao<sup>3</sup>, Yipeng Wei<sup>2</sup>, Liang Li<sup>1</sup>, Li Su<sup>2</sup>, Qingming Huang<sup>2</sup> | |
| <sup>1</sup>Institute of Computing Technology, Chinese Academy of Sciences<br> | |
| <sup>2</sup>University of Chinese Academy of Sciences <sup>3</sup>Tsinghua University | |
| [Paper](https://arxiv.org/abs/2611.22458) 路 [Project page](https://torpedo2003.github.io/SAD/) 路 [Code](https://github.com/torpedo2003/SAD) | |
| These are drop-in decoders for latent generative models. Each file holds only the decoder (`decoder.*`, `post_quant_conv.*`) of | |
| the original autoencoder. It was fine-tuned with SAD on latents from one generator. The encoder, the latent space and the | |
| generator stay unchanged, and so does the inference cost. | |
| ## Usage | |
| ```bash | |
| pip install git+https://github.com/torpedo2003/SAD | |
| ``` | |
| ```python | |
| from diffusers import DiTPipeline | |
| from sad import load_sad_decoder | |
| pipe = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256").to("cuda") | |
| load_sad_decoder(pipe.vae, "dit-xl2-256/sad-gan") # swaps the decoder in place | |
| images = pipe(class_labels=[279, 388], num_inference_steps=250, guidance_scale=1.5).images | |
| ``` | |
| You can also load the weights without our package: | |
| ```python | |
| from huggingface_hub import hf_hub_download | |
| from safetensors.torch import load_file | |
| pipe.vae.load_state_dict(load_file(hf_hub_download("torpedo2003/SAD", "dit-xl2-256/sad-gan.safetensors")), strict=False) | |
| ``` | |
| The VA-VAE and E2E-VAE decoders are not diffusers models. Load them into the autoencoder classes of the | |
| [LightningDiT](https://github.com/hustvl/LightningDiT) and [REPA-E](https://github.com/End2End-Diffusion/REPA-E) code bases, | |
| using the same `load_sad_decoder` call. | |
| ## Files | |
| Each decoder is trained on latents from its own generator and sampler, so use the one that matches your pipeline. | |
| | File | Generator | Tokenizer | Sampler | | |
| |---|---|---|---| | |
| | `dit-xl2-256/sad-gan.safetensors` | DiT-XL/2 | SD-VAE (ft-EMA) | official 250-step DDPM, cfg 1.5 | | |
| | `dit-xl2-256/sad-mmd.safetensors` | DiT-XL/2 | SD-VAE (ft-EMA) | official 250-step DDPM, cfg 1.5 | | |
| | `dit-xl2-256-dpm50/sad-gan.safetensors` | DiT-XL/2 | SD-VAE (ft-EMA) | 50-step DPM-Solver++, cfg 1.5 | | |
| | `dit-xl2-256-dpm50/sad-mmd.safetensors` | DiT-XL/2 | SD-VAE (ft-EMA) | 50-step DPM-Solver++, cfg 1.5 | | |
| | `sit-xl2-repa-256/sad-gan.safetensors` | SiT-XL/2 + REPA | SD-VAE (ft-EMA) | SDE 250 steps | | |
| | `lightningdit-xl-vavae-256/sad-gan.safetensors` | LightningDiT-XL | VA-VAE | official recipe | | |
| | `repa-e-e2evae-256/sad-gan.safetensors` | REPA-E (SiT-XL/1) | E2E-VAE | SDE 250 steps | | |
| | `pixart-alpha-256/sad-gan.safetensors` | PixArt-伪 256px | SD-VAE (ft-EMA) | 20 steps, guidance 4.5 | | |
| Every decoder was trained for 2,000 steps, about one GPU-hour. Training used the GAN discrepancy in DINOv2 space | |
| (位 = 0.05 on ImageNet, 0.01 for PixArt-伪) or MMD (位 = 1), plus the L1 encoder anchor. | |
| To train a decoder for your own generator, see the [training guide](https://github.com/torpedo2003/SAD/tree/main/training). | |
| ## Citation | |
| ```bibtex | |
| @article{lei2026sad, | |
| title = {Sample-Aligned Decoders for Latent Generative Models}, | |
| author = {Lei, Yu and Li, Yi and Gao, Hongcheng and Wei, Yipeng and Li, Liang and Su, Li and Huang, Qingming}, | |
| journal = {arXiv preprint arXiv:2611.22458}, | |
| year = {2026} | |
| } | |
| ``` |