Describe the Compact Action Expert and its intended use.

db8471f verified 6 days ago

4.03 kB

license: apache-2.0
base_model: Wan-AI/Wan2.2-TI2V-5B
datasets:
  - yuanty/LIBERO-fastwam
library_name: pytorch
tags:
  - robotics
  - world-action-model
  - libero
pipeline_tag: robotics

CAE-FastWAM

A latent world-action policy for the LIBERO benchmark, fine-tuned by Advanced Micro Devices, Inc. from Wan2.2-TI2V-5B.

Code: AMD-AGI/faster-action-models.

Part of the Compact Action Expert (CAE) collection.

Model

The Compact Action Expert combines a 32-dimensional action expert (about 30M parameters) with a suite-conditioned K8 × d16 action VAE. The VAE encodes each 32 × 7 action chunk as 8 latent tokens of width 16, and the action expert denoises those latent chunks.

Files

File	Description
`policy.pt`	Inference weights for the video and action experts
`action_vae.pt`	Suite-conditioned action VAE
`stats/libero_spatial.json`	Normalization statistics for LIBERO-Spatial
`stats/libero_object.json`	Normalization statistics for LIBERO-Object
`stats/libero_goal.json`	Normalization statistics for LIBERO-Goal
`stats/libero_10.json`	Normalization statistics for LIBERO-10

Use the statistics file that matches the suite under evaluation. policy.pt contains inference weights only.

SHA-256 checksums are in ASSET_SHA256SUMS.

Runtime weights

This repository does not include the Wan2.2 VAE, the umT5-XXL text encoder, or the umT5-XXL tokenizer. Download them from their upstream repositories:

huggingface-cli download Wan-AI/Wan2.2-TI2V-5B \
  Wan2.2_VAE.pth models_t5_umt5-xxl-enc-bf16.pth \
  --local-dir checkpoints/Wan-AI/Wan2.2-TI2V-5B
huggingface-cli download Wan-AI/Wan2.1-T2V-1.3B \
  --include "google/umt5-xxl/*" \
  --local-dir checkpoints/Wan-AI/Wan2.1-T2V-1.3B

The full Wan video DiT is not required. policy.pt already contains the fine-tuned video and action experts.

LIBERO simulator assets, BDDL task files, and initial states are also required.

Intended use

Research on world-action models and robot-policy inference efficiency, in simulation.

Limitations

Evaluated only in the LIBERO simulator. Not validated on physical robots.
Not intended for safety-critical or high-stakes applications. Users are responsible for evaluating the policy and adding appropriate safety mechanisms.
Use must also follow the Wan2.2 model card.

License

These weights are licensed under the Apache License 2.0 (LICENSE). See NOTICE.

They are a modified version of Wan2.2-TI2V-5B (Apache-2.0, Wan Team, Alibaba), fine-tuned by Advanced Micro Devices, Inc. The training recipe is FastWAM (MIT). That license covers the training code, not these weights.

Training data: LIBERO (CC BY 4.0), in the LeRobot format published as yuanty/LIBERO-fastwam (CC BY 4.0). This is the dataset used by the FastWAM LIBERO recipe, and it is used as published. No changes were made to the training data. No extra training data was used beyond the FastWAM recipe.

Citing

@misc{CAE-FastWAM,
    title = {CAE-FastWAM: Compact Action Expert for FastWAM},
    url = {https://huggingface.co/amd/CAE-FastWAM},
    author = {Hitesh Kandala, Raghav Goyal, Pratik Prabhanjan Brahma, Emad Barsoum},
    month = {October},
    year = {2026}
}

T. Yuan, Z. Dong, Y. Liu, and H. Zhao, "Fast-WAM: Do World Action Models Need Test-time Future Imagination?" arXiv:2603.16666, 2026.
B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone, "LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning," NeurIPS 2023.