Download README.md from comp-visual-planning/cvp-models: direct link, hf CLI and curl.
- Browser
- Download file 3.42 kB
-
https://huggingface.co/comp-visual-planning/cvp-models/resolve/main/README.md
- Command line
-
hf download hf://comp-visual-planning/cvp-models/README.md
-
curl -L -o README.md https://huggingface.co/comp-visual-planning/cvp-models/resolve/main/README.md
license: other
license_name: mit-and-nvidia-open-model-license
license_link: LICENSE.md
library_name: pytorch
tags:
- diffusion
- planning
- compositional-generation
- robotics
- video-generation
Compositional Visual Planning: pretrained models
Pretrained models of Compositional Visual Planning via Inference-Time Diffusion Scaling (project page, code) for the four SAPIEN tasks of the paper: cube, drawer, puzzle and tool. Each task has a video diffusion planner over Cosmos image latents, an inverse dynamics model that predicts the action between two consecutive latent frames, and the action range used to normalize the actions.
Files
| File | Description | Size |
|---|---|---|
cube/diffusion.ckpt |
DiT planner (16 blocks, width 1024, 16 heads) after 100k gradient steps | 2.5 GB |
drawer/diffusion.ckpt |
DiT planner (16 blocks, width 1024, 16 heads) after 460k gradient steps | 2.5 GB |
puzzle/diffusion.ckpt |
DiT planner (24 blocks, width 1024, 16 heads) after 60k gradient steps | 3.7 GB |
tool/diffusion.ckpt |
DiT planner (12 blocks, width 768, 12 heads) after 220k gradient steps | 1.0 GB |
<task>/invdyn.ckpt |
state dict of the inverse dynamics model (embed_dim 384), after 60k gradient steps (40k for tool) | 1.3 MB |
<task>/actions_minmax.json |
per-dimension min and max of the 8-dim actions | < 1 KB |
cosmos/Cosmos-Tokenizer-CI16x16/ |
encoder.jit and decoder.jit of NVIDIA's Cosmos-0.1-Tokenizer-CI16x16, unmodified, with NOTICE and a copy of the license |
163 MB |
All planners are x0-predicting DDPMs with 500 cosine diffusion steps. They work on clips of 16 frames, and each frame
is the (16, 16, 16) Cosmos latent of a 256x256 image, scaled by 0.5. Each planner file holds the online and the EMA weights,
{"model": ..., "model_ema": ...}.
Usage
The code downloads the files of a task when they are missing locally. To download them by hand, run from the root of the code repository:
pip install -U huggingface_hub
hf download comp-visual-planning/cvp-models --local-dir ckpts # everything
hf download comp-visual-planning/cvp-models --include "cube/*" --include "cosmos/*" --local-dir ckpts # one task
Then run python eval.py env=cube method=cvp (ours) or
python eval.py env=cube method=gsc (DiffCollage / Generative Skill Chaining baseline).
License
The planners, inverse dynamics models and action ranges are released under the MIT license. The files in
cosmos/Cosmos-Tokenizer-CI16x16/ are licensed by NVIDIA Corporation under the
NVIDIA Open Model License.
Citation
@misc{zhang2026compositionalvisualplanninginferencetime,
title={Compositional Visual Planning via Inference-Time Diffusion Scaling},
author={Yixin Zhang and Yunhao Luo and Utkarsh Aashu Mishra and Woo Chul Shin and Yongxin Chen and Danfei Xu},
year={2026},
eprint={2603.02646},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2603.02646},
}