MetaWAM-RealWorld-Predict2-2B

Paper | Code | From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models (CoRL 2026)

MetaWAM is a World Action Model fine-tuned from Cosmos-Predict2-2B-Video2World. Robot signals (task type, proprioception, action chunks, future proprioception and values) are packed into fixed coefficient slots of disjoint channel bands and lifted into compact meta-frames with a 2D inverse DCT. Slot-aware masks then train policy, world-model and value prediction in a single denoising model.

This checkpoint is a single policy for a 7-DoF Franka Panda, trained jointly on three tabletop tasks for 7.2K steps. It sees one wrist camera and two third-person cameras, and predicts 16-step action chunks. The tasks and their instructions are:

Task	Instruction
Sweep	Sweep the green block into the dustpan
Place Cup	Place the cup on the green coaster
Build Block	Place the orange block on top of the gray block

The checkpoint is tied to our camera placement and robot setup. Treat it as a reference, or as an initialization for fine-tuning on your own data, rather than as a drop-in policy.

Files

File	Description
MetaWAM-RealWorld-Predict2-2B.pt	Model weights (bf16, consolidated)
realworld_dataset_statistics.json	Action / state normalization statistics of the training data
realworld_t5_embeddings.pkl	T5 embeddings of the three task instructions
SHA256SUMS	Checksums

Usage

Run the policy server with the inference config metawam_realworld__inference from the MetaWAM repository:

hf download Ethanliang99/MetaWAM-RealWorld-Predict2-2B --local-dir checkpoints/MetaWAM-RealWorld-Predict2-2B
export FRANKA_CKPT_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/MetaWAM-RealWorld-Predict2-2B.pt
export FRANKA_DATASET_STATS_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/realworld_dataset_statistics.json
export FRANKA_T5_EMBEDDINGS_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/realworld_t5_embeddings.pkl
bash cosmos_policy/experiments/robot/franka/run_deploy_franka.sh

See REALWORLD.md for the robot client, data collection and training.

License

These weights are fine-tuned from nvidia/Cosmos-Predict2-2B-Video2World; use is subject to the license of the base model.

Citation

@article{liang2026metawam,
  title   = {From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models},
  author  = {Liang, Guoqiang and Lin, Yiqi and Chen, Yanzhe and Ma, Kevin Yuchen and Shou, Mike Zheng},
  journal = {arXiv preprint arXiv:2611.14365},
  year    = {2026}
}