Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
[Zhuoran Zhao](https://alicezrzhao.github.io/)
1,2 · [Shengju Qian](http://thesouthfrog.com/about.me/)
3 · [Tongtong Liang](https://tongtongliang.github.io/)
4 · [Xianghao Kong](https://refkxh.github.io/)
2 · [Songchun Zhang](https://franklinz233.github.io/)
2 · [Junchao Huang](https://junchao-cs.github.io/)
5 · [Guian Fang](https://enderfga.cn/)
6 · [Xin Wang](https://scholar.google.com/citations?user=2Z1GJ50AAAAJ&hl=en)
3 · [Pan Hui](https://panhui.people.ust.hk/index.html)
1,2 · [Anyi Rao](https://anyirao.com/)
2
1HKUST(GZ) ·
2HKUST ·
3LIGHTSPEED ·
4UCSD ·
5CUHK(SZ) ·
6NUS
## 📖 Overview
Self-rollout DMD enables few-step autoregressive video generation, but its mode-seeking reverse-KL objective may concentrate the student rollout distribution on a narrow set of teacher modes. Moreover, the DMD objective is evaluated only on the completed rollout output and does not directly regularize each intermediate transition, allowing errors to accumulate through subsequent denoising steps and later chunks. Together, these limitations reduce visual quality and realism.
**Mask Forcing** introduces a Dual-Noise Masking Rollout strategy that injects cleaner tokens into noisy rollout inputs through random masks. The resulting perturbations diversify student rollout trajectories to cover more teacher modes, while cleaner tokens guide the denoising of noisier tokens to improve intermediate predictions and reduce error accumulation.
Mask Forcing requires no real-video supervision or additional post-training stages, introduces no additional network forward passes, and leaves inference unchanged.
