RoSeDiT: Role-Separated Diffusion Transformers

Checkpoints of Role-Separated Diffusion Transformers: Global Rule Isolation and Deterministic Sampling Manifold Compression (Zhiguang Liu, Yi Shang; University of Missouri). Paper: https://arxiv.org/abs/2611.19009. Code: https://github.com/lz7fd/RoSeDiT

RoSeDiT is a diffusion transformer architecture that isolates the global rule from the image tokens. Twelve controller tokens maintain an evolving representation of the global condition, a listening mechanism lets adaLN compute its modulation from their states, and a structural pass restricts each image token to its 3x3 neighborhood and the controller tokens. RoSeDiT maintains the same level of parameter count as SiT by reusing the dense attention weights. Deterministic Sampling Manifold Compression (DSMC) is a two-stage training schedule for the continued training of diffusion models.

Files

Class-conditional ImageNet 256x256 models in the latent space of the Stable Diffusion VAE. FID is computed on 50,000 samples from the EMA weights with a 250-step Euler-Maruyama SDE sampler and no classifier-free guidance (ADM evaluation suite).

File Model Steps FID Params --model Flags Size
RoSeDiT-S-2_400K.pt RoSeDiT-S/2 400K 51.42 33.0M SiT-BR-B1W-DG-S/2 --encoder-depth=4 0.12 GB
RoSeDiT-B-2_400K.pt RoSeDiT-B/2 400K 27.18 130.6M SiT-BR-B1W-DG-B/2 --encoder-depth=4 0.49 GB
RoSeDiT-L-2_400K.pt RoSeDiT-L/2 400K 15.34 458.2M SiT-BR-B1W-DG-L/2 --encoder-depth=8 1.71 GB
RoSeDiT-XL-2_400K.pt RoSeDiT-XL/2 400K 12.63 675.3M SiT-BR-B1W-DG-XL/2 --encoder-depth=8 --qk-norm 2.52 GB
RoSeDiT-XL-2_1400K.pt RoSeDiT-XL/2 1.4M 7.98 675.3M SiT-BR-B1W-DG-XL/2 --encoder-depth=8 --qk-norm 2.52 GB
RoSeDiT-XL-2_2400K.pt RoSeDiT-XL/2 2.4M 7.61 675.3M SiT-BR-B1W-DG-XL/2 --encoder-depth=8 --qk-norm 2.52 GB
RoSeDiT-XL-2_DSMC_2400K.pt RoSeDiT-XL/2 + DSMC 2.4M 5.84 675.3M SiT-BR-B1W-DG-XL/2 --encoder-depth=8 --qk-norm 2.52 GB
RoSeDiT-B-2_REPA_400K.pt RoSeDiT-B/2 + REPA 400K 21.01 130.6M + 7.3M projector SiT-BR-B1W-DG-B/2 --encoder-depth=4 --projector-embed-dims=768 0.51 GB

Each file is a PyTorch dictionary with the EMA weights under the key ema and the settings under config. The files hold no optimizer state: they are for sampling, not for resuming training. MD5SUMS lists the checksums.

Use

git clone https://github.com/lz7fd/RoSeDiT && cd RoSeDiT
pip install -r requirements.txt
hf download lz7fdmu/RoSeDiT --local-dir checkpoints

torchrun --nnodes=1 --nproc_per_node=1 generate.py \
  --model "SiT-BR-B1W-DG-XL/2" --qk-norm \
  --num-fid-samples 50000 \
  --ckpt checkpoints/RoSeDiT-XL-2_DSMC_2400K.pt \
  --path-type=linear --encoder-depth=8 --projector-embed-dims=None \
  --per-proc-batch-size=64 --mode=sde --num-steps=250 --cfg-scale=1.0 --global-seed=0

scripts/sample_50k.sh <file> in the code repository picks the flags of each file. RoSeDiT-XL/2 is trained with RMS normalization of the attention queries and keys, so --qk-norm is needed to load it. The 50,000-sample archives behind RoSeDiT-XL/2 at 1.4M steps (FID 7.98) and RoSeDiT-XL/2 + DSMC at 2.4M steps (FID 5.84) are at https://huggingface.co/datasets/lz7fdmu/RoSeDiT_samples

Citation

@article{liu2026rosedit,
  title   = {Role-Separated Diffusion Transformers: Global Rule Isolation and Deterministic Sampling Manifold Compression},
  author  = {Liu, Zhiguang and Shang, Yi},
  journal = {arXiv preprint arXiv:2611.19009},
  year    = {2026}
}

License

MIT. The code is built on REPA.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train lz7fdmu/RoSeDiT

Paper for lz7fdmu/RoSeDiT