Image-to-Video
PyTorch
Wan2.2
world-model
video-prediction
robotics
robot-manipulation
action-conditioned
few-step
Instructions to use Junjun2333/TwoPass with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Wan2.2
How to use Junjun2333/TwoPass with Wan2.2:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
|
Download README.md from Junjun2333/TwoPass: direct link, hf CLI and curl.
- Browser
- Download file 2.83 kB
-
https://huggingface.co/Junjun2333/TwoPass/resolve/main/README.md
- Command line
-
hf download hf://Junjun2333/TwoPass/README.md
-
curl -L -o README.md https://huggingface.co/Junjun2333/TwoPass/resolve/main/README.md
2.83 kB
| license: apache-2.0 | |
| base_model: Wan-AI/Wan2.2-TI2V-5B | |
| library_name: pytorch | |
| pipeline_tag: image-to-video | |
| tags: | |
| - world-model | |
| - video-prediction | |
| - robotics | |
| - robot-manipulation | |
| - action-conditioned | |
| - few-step | |
| - wan2.2 | |
| datasets: | |
| - fangqi/IRASim | |
| # TwoPass: Draft-Guided Few-Step Video Prediction for Robotic Manipulation | |
| <p align="center"> | |
| <a href="https://arxiv.org/abs/2611.25553"><img src="https://img.shields.io/badge/arXiv-2611.25553-b31b1b.svg" alt="Paper: arXiv 2611.25553"></a> | |
| <a href="https://github.com/PlantPotatoOnMoon/TwoPass"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github" alt="Code: GitHub"></a> | |
| <a href="#citation"><img src="https://img.shields.io/badge/Citation-BibTeX-2563eb" alt="Citation / BibTeX"></a> | |
| </p> | |
| Checkpoints of **TwoPass**, an action-conditioned video prediction model for | |
| robotic manipulation fine-tuned from | |
| [Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B). Given one | |
| observed frame and the future robot actions, it predicts the following frames | |
| in as few as four network calls. | |
| | File | Dataset | Size | | |
| |---|---|---| | |
| | [`bridge/twopass_bridge.pt`](bridge/twopass_bridge.pt) | Bridge | 20.2 GB | | |
| | [`languagetable/twopass_languagetable.pt`](languagetable/twopass_languagetable.pt) | Language Table | 20.2 GB | | |
| Each file is a PyTorch checkpoint holding the full model weights (`model`, fp32) | |
| and the training step (`step`). | |
| ## Usage | |
| Install the code from [GitHub](https://github.com/PlantPotatoOnMoon/TwoPass), | |
| then download the checkpoints together with the Wan2.2 base weights: | |
| ```bash | |
| hf download Junjun2333/TwoPass --local-dir ./checkpoints | |
| hf download Wan-AI/Wan2.2-TI2V-5B --local-dir ./Wan2.2-TI2V-5B | |
| ``` | |
| Evaluate on the Bridge test set with 4 network calls per clip (use | |
| `configs/languagetable.json` and the Language Table checkpoint for the other | |
| dataset; data preparation is described in the GitHub README): | |
| ```bash | |
| python -m twopass.evaluate --config configs/bridge.json --nfe 4 \ | |
| --checkpoint ./checkpoints/bridge/twopass_bridge.pt --wan_dir ./Wan2.2-TI2V-5B \ | |
| --data_root $DATA_ROOT --i3d_path /path/to/i3d_torchscript.pt --out_dir ./eval/bridge | |
| python -m twopass.aggregate --config configs/bridge.json \ | |
| --data_root $DATA_ROOT --eval_dir ./eval/bridge | |
| ``` | |
| ## Citation | |
| ```bibtex | |
| @article{liu2026twopass, | |
| title = {TwoPass: Draft-Guided Few-Step Video Prediction for Robotic Manipulation}, | |
| author = {Liu, Yijun and Ge, Shijia and Feng, Tongtong and Wang, Xin and Meng, Yuan and Liu, Yuwei and Wang, Zhi and Zhu, Wenwu}, | |
| journal = {arXiv preprint arXiv:2611.25553}, | |
| year = {2026}, | |
| eprint = {2611.25553}, | |
| archivePrefix = {arXiv}, | |
| primaryClass = {cs.CV}, | |
| url = {https://arxiv.org/abs/2611.25553} | |
| } | |
| ``` | |
| ## License | |
| Apache License 2.0, following Wan2.2. | |