|
Download README.md from Ethanliang99/MetaWAM-RealWorld-Predict2-2B: direct link, hf CLI and curl.
- Browser
- Download file 3.2 kB
-
https://huggingface.co/Ethanliang99/MetaWAM-RealWorld-Predict2-2B/resolve/main/README.md
- Command line
-
hf download hf://Ethanliang99/MetaWAM-RealWorld-Predict2-2B/README.md
-
curl -L -o README.md https://huggingface.co/Ethanliang99/MetaWAM-RealWorld-Predict2-2B/resolve/main/README.md
3.2 kB
| base_model: | |
| - nvidia/Cosmos-Predict2-2B-Video2World | |
| license: other | |
| license_name: nvidia-base-model-license | |
| license_link: https://huggingface.co/nvidia/Cosmos-Predict2-2B-Video2World | |
| pipeline_tag: robotics | |
| tags: | |
| - robotics | |
| - world-action-model | |
| - video-diffusion | |
| - franka | |
| # MetaWAM-RealWorld-Predict2-2B | |
| [**Paper**](https://arxiv.org/abs/2611.14365) | [**Code**](https://github.com/showlab/MetaWAM) | *From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models* (CoRL 2026) | |
| MetaWAM is a World Action Model fine-tuned from [Cosmos-Predict2-2B-Video2World](https://huggingface.co/nvidia/Cosmos-Predict2-2B-Video2World). Robot signals (task type, proprioception, action chunks, future proprioception and values) are packed into fixed coefficient slots of disjoint channel bands and lifted into compact *meta-frames* with a 2D inverse DCT. Slot-aware masks then train policy, world-model and value prediction in a single denoising model. | |
| This checkpoint is a single policy for a **7-DoF Franka Panda**, trained jointly on three tabletop tasks for 7.2K steps. It sees one wrist camera and two third-person cameras, and predicts 16-step action chunks. The tasks and their instructions are: | |
| | Task | Instruction | | |
| |---|---| | |
| | Sweep | `Sweep the green block into the dustpan` | | |
| | Place Cup | `Place the cup on the green coaster` | | |
| | Build Block | `Place the orange block on top of the gray block` | | |
| The checkpoint is tied to our camera placement and robot setup. Treat it as a reference, or as an initialization for fine-tuning on your own data, rather than as a drop-in policy. | |
| ## Files | |
| | File | Description | | |
| |---|---| | |
| | `MetaWAM-RealWorld-Predict2-2B.pt` | Model weights (bf16, consolidated) | | |
| | `realworld_dataset_statistics.json` | Action / state normalization statistics of the training data | | |
| | `realworld_t5_embeddings.pkl` | T5 embeddings of the three task instructions | | |
| | `SHA256SUMS` | Checksums | | |
| ## Usage | |
| Run the policy server with the inference config `metawam_realworld__inference` from the [MetaWAM repository](https://github.com/showlab/MetaWAM): | |
| ```bash | |
| hf download Ethanliang99/MetaWAM-RealWorld-Predict2-2B --local-dir checkpoints/MetaWAM-RealWorld-Predict2-2B | |
| export FRANKA_CKPT_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/MetaWAM-RealWorld-Predict2-2B.pt | |
| export FRANKA_DATASET_STATS_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/realworld_dataset_statistics.json | |
| export FRANKA_T5_EMBEDDINGS_PATH=checkpoints/MetaWAM-RealWorld-Predict2-2B/realworld_t5_embeddings.pkl | |
| bash cosmos_policy/experiments/robot/franka/run_deploy_franka.sh | |
| ``` | |
| See [REALWORLD.md](https://github.com/showlab/MetaWAM/blob/main/REALWORLD.md) for the robot client, data collection and training. | |
| ## License | |
| These weights are fine-tuned from `nvidia/Cosmos-Predict2-2B-Video2World`; use is subject to the license of the base model. | |
| ## Citation | |
| ```bibtex | |
| @article{liang2026metawam, | |
| title = {From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models}, | |
| author = {Liang, Guoqiang and Lin, Yiqi and Chen, Yanzhe and Ma, Kevin Yuchen and Shou, Mike Zheng}, | |
| journal = {arXiv preprint arXiv:2611.14365}, | |
| year = {2026} | |
| } | |
| ``` | |