add README.md (hf_hub_upload via CloudFront edge)
Browse files
README.md
CHANGED
|
@@ -9,7 +9,7 @@ tags:
|
|
| 9 |
- pytorch
|
| 10 |
---
|
| 11 |
|
| 12 |
-
<h1><img src="assets/logo.svg" width="104" align="left" hspace="12" alt="LDM is an AE: z_t -> DiT-D (decoding) -> x (image domain) -> DiT-E (encoding) -> z">LDM-is-AE: Latent Diffusion Model is an
|
| 13 |
|
| 14 |
[](LICENSE)
|
| 15 |
[](https://pytorch.org)
|
|
@@ -34,7 +34,7 @@ Latent diffusion is normally a two-stage pipeline: train a
|
|
| 34 |
VAE, then train a diffusion model in its latent space -- and inherit the VAE's bias. **LDM-is-AE removes
|
| 35 |
the pipeline.** We split the DiT backbone into **DiT-D** (the first 30 blocks) and **DiT-E** (the last 2 blocks) and
|
| 36 |
supervise the intermediate feature `F` in the image domain at every timestep. The backbone's hidden
|
| 37 |
-
`decode->encode` path then becomes an explicit **
|
| 38 |
stage without external VAE.
|
| 39 |
|
| 40 |
<p align="center"><img src="assets/fig1_framework.png" width="100%" alt="LDM-is-AE: the DiT backbone is split into DiT-E / DiT-D and the intermediate feature F is aligned with PixelUnshuffle(x) in the image domain at every timestep."></p>
|
|
@@ -42,7 +42,7 @@ stage without external VAE.
|
|
| 42 |
<sub><b>Figure 1.</b> <b>(a)</b> the DiT backbone performs <i>latent → feature → latent</i>;
|
| 43 |
<b>(b)</b> image-space supervision aligns the intermediate feature with the image domain at every timestep;
|
| 44 |
<b>(c)</b> at the zero-noise timestep (<code>t=1</code>) an explicit <i>latent → image → latent</i> path
|
| 45 |
-
makes the backbone an
|
| 46 |
|
| 47 |
---
|
| 48 |
|
|
@@ -206,7 +206,7 @@ Paper: **arXiv:2609.37080** — https://arxiv.org/abs/2609.37080
|
|
| 206 |
|
| 207 |
```bibtex
|
| 208 |
@inproceedings{ldm_is_ae_2026,
|
| 209 |
-
title = {LDM-is-AE: Latent Diffusion Model is an
|
| 210 |
author = {Zhang, Zhengqiang and Sun, Lingchen and Wu, Rongyuan and Yi, Qiaosi and
|
| 211 |
Kong, Xiangtao and Xiao, Chaodong and Zhang, Lei},
|
| 212 |
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
|
|
|
|
| 9 |
- pytorch
|
| 10 |
---
|
| 11 |
|
| 12 |
+
<h1><img src="assets/logo.svg" width="104" align="left" hspace="12" alt="LDM is an AE: z_t -> DiT-D (decoding) -> x (image domain) -> DiT-E (encoding) -> z">LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation (NeurIPS, 2026)</h1>
|
| 13 |
|
| 14 |
[](LICENSE)
|
| 15 |
[](https://pytorch.org)
|
|
|
|
| 34 |
VAE, then train a diffusion model in its latent space -- and inherit the VAE's bias. **LDM-is-AE removes
|
| 35 |
the pipeline.** We split the DiT backbone into **DiT-D** (the first 30 blocks) and **DiT-E** (the last 2 blocks) and
|
| 36 |
supervise the intermediate feature `F` in the image domain at every timestep. The backbone's hidden
|
| 37 |
+
`decode->encode` path then becomes an explicit **auto-encoder**, trained end-to-end in a single
|
| 38 |
stage without external VAE.
|
| 39 |
|
| 40 |
<p align="center"><img src="assets/fig1_framework.png" width="100%" alt="LDM-is-AE: the DiT backbone is split into DiT-E / DiT-D and the intermediate feature F is aligned with PixelUnshuffle(x) in the image domain at every timestep."></p>
|
|
|
|
| 42 |
<sub><b>Figure 1.</b> <b>(a)</b> the DiT backbone performs <i>latent → feature → latent</i>;
|
| 43 |
<b>(b)</b> image-space supervision aligns the intermediate feature with the image domain at every timestep;
|
| 44 |
<b>(c)</b> at the zero-noise timestep (<code>t=1</code>) an explicit <i>latent → image → latent</i> path
|
| 45 |
+
makes the backbone an auto-encoder, which in turn enables <i>image → latent → image</i>.</sub>
|
| 46 |
|
| 47 |
---
|
| 48 |
|
|
|
|
| 206 |
|
| 207 |
```bibtex
|
| 208 |
@inproceedings{ldm_is_ae_2026,
|
| 209 |
+
title = {LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation},
|
| 210 |
author = {Zhang, Zhengqiang and Sun, Lingchen and Wu, Rongyuan and Yi, Qiaosi and
|
| 211 |
Kong, Xiangtao and Xiao, Chaodong and Zhang, Lei},
|
| 212 |
booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
|