import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("SuhaoYu1020/latent-arith-mmdit", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]Trained MMDiT checkpoints (final snapshot of each run), exported 2026-10-17
Each folder = runs/<run>/ of the project: args.json (all launch flags), final/meta.json (step, sigma shift, latent id,
stats file hash) and final/dit/ (diffusers save_pretrained of mmdit_model.MMDiT, bf16, 561M params, 1.1 GB).
Load: MMDiT.from_pretrained("<run>/final/dit"); render + score with
render_eval_sq_dit.py --ckpt <run>/final --out_dir OUT --n_test 1000 --steps 24 (see the code package README, section 4).
All runs: batch 64, lr 2e-4, 12 x 1152, flow matching; data root in args.json.
| run | latent | task / data | sigma schedule | budget | role in the write-up |
|---|---|---|---|---|---|
| mm_s40_vae, mm_s40_siglip2 | VAE / SigLIP2 | one-to-one standard | shift 40 | to 95% test (watcher STOP) | baselines, "shift 15/40 learns" |
| mm_s1_vae, mm_s1_siglip2 | VAE / SigLIP2 | one-to-one standard | shift 1 | 120 epochs | "shift 1 never learns" (VAE 67%, SigLIP2 7%) |
| mm_fx8_vae_s42, mm_fx8_siglip2_s42 | VAE / SigLIP2 | one-to-one standard | fixed sigma 0.8 | 30 epochs | the copiers (dose response, counterfactual canvas, E2 init) |
| mm_fx1.0_vae_s42, mm_fx1.0_siglip2_s42 | VAE / SigLIP2 | one-to-one standard | fixed sigma 1.0 | to 95% | the sigma = 1 regressors |
| mm_om_s15_siglip2 | SigLIP2 | hard one-to-many | shift 15 | 10 epochs (11,200 steps) | curriculum arm "shift 15 throughout" (acc 97 / fid 41 / both 34) |
| mm_om_s15to1_75_siglip2 | SigLIP2 | hard one-to-many | shift 15 -> 1 at 75% | 10 epochs | best switch arm (92 / 54 / 47) |
| mm_om_reg_siglip2 | SigLIP2 | hard one-to-many | fixed sigma 1.0 | 10 epochs | "knows every answer, draws none" (100 / 0 / 0) |
| mm_om_s1_siglip2 | SigLIP2 | hard one-to-many | shift 1 | 10 epochs | "low noise only" (0 / 15 / 0) |
| mm_om_s15_vae | VAE | hard one-to-many | shift 15 | 10 epochs | VAE mirror image (- / 40 / 7) |
| mm_om_s15to1_50_vae | VAE | hard one-to-many | shift 15 -> 1 at 50% | 10 epochs | best VAE arm (- / 66 / 16) |
| mm_s40_siglip2_anshard_long | SigLIP2 | hard one-to-many | shift 40 | 35 epochs (40,000 steps) | long run, 78% OCR at 24 steps |
Switch arms were initialised from the parent run's snapshot at the switch step (--init_ckpt, fresh optimizer, warmup 100);
the parent snapshots themselves are not included (only finals).
Code: https://github.com/SuhaoYu1020/Semantic-Image-Release · Data: https://huggingface.co/datasets/SuhaoYu1020/arith-sq256
- Downloads last month
- -