xtudbxk commited on
Commit
b3e09d9
·
verified ·
1 Parent(s): cc04f46

add README.md (hf_hub_upload via CloudFront edge)

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -9,7 +9,7 @@ tags:
9
  - pytorch
10
  ---
11
 
12
- <h1><img src="assets/logo.svg" width="104" align="left" hspace="12" alt="LDM is an AE: z_t -> DiT-D (decoding) -> x (image domain) -> DiT-E (encoding) -> z">LDM-is-AE: Latent Diffusion Model is an Intrinsic Auto-Encoder for End-to-End Image Generation (NIPS, 2026)</h1>
13
 
14
  [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
15
  [![Framework](https://img.shields.io/badge/Framework-PyTorch-ee4c2c.svg)](https://pytorch.org)
@@ -34,7 +34,7 @@ Latent diffusion is normally a two-stage pipeline: train a
34
  VAE, then train a diffusion model in its latent space -- and inherit the VAE's bias. **LDM-is-AE removes
35
  the pipeline.** We split the DiT backbone into **DiT-D** (the first 30 blocks) and **DiT-E** (the last 2 blocks) and
36
  supervise the intermediate feature `F` in the image domain at every timestep. The backbone's hidden
37
- `decode->encode` path then becomes an explicit **intrinsic auto-encoder**, trained end-to-end in a single
38
  stage without external VAE.
39
 
40
  <p align="center"><img src="assets/fig1_framework.png" width="100%" alt="LDM-is-AE: the DiT backbone is split into DiT-E / DiT-D and the intermediate feature F is aligned with PixelUnshuffle(x) in the image domain at every timestep."></p>
@@ -42,7 +42,7 @@ stage without external VAE.
42
  <sub><b>Figure 1.</b> <b>(a)</b> the DiT backbone performs <i>latent &rarr; feature &rarr; latent</i>;
43
  <b>(b)</b> image-space supervision aligns the intermediate feature with the image domain at every timestep;
44
  <b>(c)</b> at the zero-noise timestep (<code>t=1</code>) an explicit <i>latent &rarr; image &rarr; latent</i> path
45
- makes the backbone an intrinsic auto-encoder, which in turn enables <i>image &rarr; latent &rarr; image</i>.</sub>
46
 
47
  ---
48
 
@@ -206,7 +206,7 @@ Paper: **arXiv:2609.37080** &mdash; https://arxiv.org/abs/2609.37080
206
 
207
  ```bibtex
208
  @inproceedings{ldm_is_ae_2026,
209
- title = {LDM-is-AE: Latent Diffusion Model is an Intrinsic Auto-Encoder for End-to-End Image Generation},
210
  author = {Zhang, Zhengqiang and Sun, Lingchen and Wu, Rongyuan and Yi, Qiaosi and
211
  Kong, Xiangtao and Xiao, Chaodong and Zhang, Lei},
212
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
 
9
  - pytorch
10
  ---
11
 
12
+ <h1><img src="assets/logo.svg" width="104" align="left" hspace="12" alt="LDM is an AE: z_t -> DiT-D (decoding) -> x (image domain) -> DiT-E (encoding) -> z">LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation (NeurIPS, 2026)</h1>
13
 
14
  [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
15
  [![Framework](https://img.shields.io/badge/Framework-PyTorch-ee4c2c.svg)](https://pytorch.org)
 
34
  VAE, then train a diffusion model in its latent space -- and inherit the VAE's bias. **LDM-is-AE removes
35
  the pipeline.** We split the DiT backbone into **DiT-D** (the first 30 blocks) and **DiT-E** (the last 2 blocks) and
36
  supervise the intermediate feature `F` in the image domain at every timestep. The backbone's hidden
37
+ `decode->encode` path then becomes an explicit **auto-encoder**, trained end-to-end in a single
38
  stage without external VAE.
39
 
40
  <p align="center"><img src="assets/fig1_framework.png" width="100%" alt="LDM-is-AE: the DiT backbone is split into DiT-E / DiT-D and the intermediate feature F is aligned with PixelUnshuffle(x) in the image domain at every timestep."></p>
 
42
  <sub><b>Figure 1.</b> <b>(a)</b> the DiT backbone performs <i>latent &rarr; feature &rarr; latent</i>;
43
  <b>(b)</b> image-space supervision aligns the intermediate feature with the image domain at every timestep;
44
  <b>(c)</b> at the zero-noise timestep (<code>t=1</code>) an explicit <i>latent &rarr; image &rarr; latent</i> path
45
+ makes the backbone an auto-encoder, which in turn enables <i>image &rarr; latent &rarr; image</i>.</sub>
46
 
47
  ---
48
 
 
206
 
207
  ```bibtex
208
  @inproceedings{ldm_is_ae_2026,
209
+ title = {LDM-is-AE: Latent Diffusion Model is an Auto-Encoder for End-to-End Image Generation},
210
  author = {Zhang, Zhengqiang and Sun, Lingchen and Wu, Rongyuan and Yi, Qiaosi and
211
  Kong, Xiangtao and Xiao, Chaodong and Zhang, Lei},
212
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},