mbao827/seeing_the_unseen

Seeing the Unseen: Unified Visible–Invisible Motion for Physically Consistent Video Generation

NeurIPS 2026

📄 Paper (arXiv) · 🌐 Project Page · 💻 Code

Mingyang Bao¹, Fangda Ye¹, Xihao Chen¹, Bobo Li¹, Jason Li², Shengqiong Wu^3,✉

¹National University of Singapore · ²Nanyang Technological University · ³University of Oxford · ✉ Corresponding author

Overview

A learned particle simulator (PGDS) predicts how the scene in a single image will move. A visible–invisible motion field (VIMF) hands that motion to a video diffusion model — including the surfaces that only come into view after objects move.

• Learned physics — PGDS reaches 95.92% average strict Acc-V over seven Physion++ test scenarios with 0 manually specified physical parameters.
• Visible + emergent motion — guidance is split into a reference-visible and a future-emergent tracking video, injected through zero-initialized latent channels (48 → 144) of Wan2.2-TI2V-5B.
• State-of-the-art physical consistency — highest Semantic Adherence on all seven scenarios and best Physical Commonsense on six of seven; +44.0% overall human preference over Wan2.2-5B.

Model Weights

Path	Size	Description
dual_motion_wan2.2_ti2v_5b/transformer/	~10 GB	VIMF-conditioned video transformer (diffusers Wan2_2Transformer3DModel, 144 input channels), fine-tuned from Wan2.2-TI2V-5B
pgds/pgds.pth	~9 MB	PGDS learned particle simulator

Only the fine-tuned transformer is released here; the VAE, text encoder and scheduler are loaded from the original Wan-AI/Wan2.2-TI2V-5B checkpoint.

Usage

Please refer to the code repository for installation and inference instructions.

hf download mbao827/seeing_the_unseen --local-dir checkpoints/seeing_the_unseen

Citation

@inproceedings{bao2026seeing,
  title     = {Seeing the Unseen: Unified Visible--Invisible Motion for Physically Consistent Video Generation},
  author    = {Bao, Mingyang and Ye, Fangda and Chen, Xihao and Li, Bobo and Li, Jason and Wu, Shengqiong},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}

Acknowledgements

This model builds on Wan2.2, SAM-3D-Objects and SAM3.