--- base_model: - nvidia/Cosmos-Predict2-2B-Video2World license: other license_name: nvidia-base-model-license license_link: https://huggingface.co/nvidia/Cosmos-Predict2-2B-Video2World pipeline_tag: robotics tags: - robotics - world-action-model - video-diffusion - franka --- # MetaWAM-RealWorld-Predict2-2B [**Paper**](https://arxiv.org/abs/2611.14365) | [**Code**](https://github.com/showlab/MetaWAM) | *From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models* (CoRL 2026) MetaWAM is a World Action Model fine-tuned from [Cosmos-Predict2-2B-Video2World](https://huggingface.co/nvidia/Cosmos-Predict2-2B-Video2World). Robot signals (task type, proprioception, action chunks, future proprioception and values) are packed into fixed coefficient slots of disjoint channel bands and lifted into compact *meta-frames* with a 2D inverse DCT. Slot-aware masks then train policy, world-model and value prediction in a single denoising model. This checkpoint is a single policy for a **7-DoF Franka Panda**, trained jointly on three tabletop tasks for 7.2K steps. It sees one wrist camera and two third-person cameras, and predicts 16-step action chunks. The tasks and their instructions are: | Task | Instruction | |---|---| | Sweep | `Sweep the green block into the dustpan` | | Place Cup | `Place the cup on the green coaster` | | Build Block | `Place the orange block on top of the gray block` | The checkpoint is tied to our camera placement and robot setup. Treat it as a reference, or as an initialization for fine-tuning on your own data, rather than as a drop-in policy. ## Files | File | Description | |---|---| | `MetaWAM-RealWorld-Predict2-2B.pt` | Model weights (bf16, consolidated) | | `realworld_dataset_statistics.json` | Action / state normalization statistics of the training data | | `realworld_t5_embeddings.pkl` | T5 embeddings of the three task instructions | | `SHA256SUMS` | Checksums | ## Usage Run the policy server with the inference config `metawam_realworld__inference` from the [MetaWAM repository](https://github.com/showlab/MetaWAM): ```bash hf download Ethanliang99/MetaWAM-RealWorld-Predict2-2B --local-dir checkpoints/MetaWAM-RealWorld-Predict2-2B export FRANKA_CKPT_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/MetaWAM-RealWorld-Predict2-2B.pt export FRANKA_DATASET_STATS_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/realworld_dataset_statistics.json export FRANKA_T5_EMBEDDINGS_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/realworld_t5_embeddings.pkl bash cosmos_policy/experiments/robot/franka/run_deploy_franka.sh ``` See [REALWORLD.md](https://github.com/showlab/MetaWAM/blob/main/REALWORLD.md) for the robot client, data collection and training. ## License These weights are fine-tuned from `nvidia/Cosmos-Predict2-2B-Video2World`; use is subject to the license of the base model. ## Citation ```bibtex @article{liang2026metawam, title = {From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models}, author = {Liang, Guoqiang and Lin, Yiqi and Chen, Yanzhe and Ma, Kevin Yuchen and Shou, Mike Zheng}, journal = {arXiv preprint arXiv:2611.14365}, year = {2026} } ```