File size: 2,769 Bytes
d115884 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
pipeline_tag: text-to-video
tags:
- video-generation
- autoregressive-video
- diffusion-distillation
- wan2.1
---
<h1 align="center">DistillAlign: Coordinating Mode Covering and Mode Seeking in Autoregressive Video Distillation</h1>
<p align="center"><b>Anonymous release for ICLR 2027 submission</b></p>
**DistillAlign** aligns and balances the **mode-covering** and **mode-seeking**
objectives of the multi-stage video distillation pipeline — using only a
**1.3B DMD teacher**, it already surpasses baselines refined with a **14B DMD
teacher**.
## Checkpoints
All released generators are **Wan2.1-1.3B students**; the size in each name
refers to the **teacher** used during training.
| Model | Checkpoint | Description |
|---|---|---|
| Initializer (1.3B teacher) | [distillalign_init_1p3b_teacher.pt](https://huggingface.co/DistillAlign-Anonymous/DistillAlign/blob/main/checkpoints/distillalign_init_1p3b_teacher.pt) | Pre-DMD initializer, trained toward a Wan2.1-T2V-1.3B teacher |
| Initializer (14B teacher) | [distillalign_init_14b_teacher.pt](https://huggingface.co/DistillAlign-Anonymous/DistillAlign/blob/main/checkpoints/distillalign_init_14b_teacher.pt) | Pre-DMD initializer, trained toward a Wan2.1-T2V-14B teacher |
| Distilled (1.3B teacher) | [distillalign_distill_1p3b_teacher.pt](https://huggingface.co/DistillAlign-Anonymous/DistillAlign/blob/main/checkpoints/distillalign_distill_1p3b_teacher.pt) | Final joint-distilled generator, Wan2.1-T2V-1.3B DMD teacher |
| Distilled (14B teacher) | [distillalign_distill_14b_teacher.pt](https://huggingface.co/DistillAlign-Anonymous/DistillAlign/blob/main/checkpoints/distillalign_distill_14b_teacher.pt) | Final joint-distilled generator, Wan2.1-T2V-14B DMD teacher |
Each checkpoint stores a plain `{"generator": state_dict}` and loads directly
with the inference and evaluation code in the supplementary material.
## Teacher Reference Caches
Ready-made teacher reference features for the teacher-normalized
distribution evaluation. Pass the `.npz` file to `--teacher-features` and
only the student side needs to be generated.
| Cache | Download | Description |
|---|---|---|
| Wan2.1-1.3B teacher reference features | [wan2.1_t2v_1.3b_reference_vjepa2.npz](https://huggingface.co/DistillAlign-Anonymous/DistillAlign/blob/main/teacher_caches/wan2.1_t2v_1.3b_reference_vjepa2.npz) | 256 x 2560 V-JEPA2 features, ready for `--teacher-features` |
| Wan2.1-14B teacher reference features | [wan2.1_t2v_14b_reference_vjepa2.npz](https://huggingface.co/DistillAlign-Anonymous/DistillAlign/blob/main/teacher_caches/wan2.1_t2v_14b_reference_vjepa2.npz) | 256 x 2560 V-JEPA2 features, ready for `--teacher-features` |
See `teacher_caches/README.md` for the exact sampling and extraction protocol.
|