StarVLA/Wan2.2-5B-PI-SIMPLE

StarVLA-WM-Wan-PI SIMPLE 12-Task Joint Policy

This model is a StarVLA StarVLA-WM-Wan-PI (WanPI) policy jointly trained on 12 SIMPLE Unitree G1 humanoid tasks. It uses one shared checkpoint across all tasks; there is no task-specific fine-tuning or task-specific checkpoint selection.

The evaluated artifact is:

checkpoints/steps_100000_pytorch_model.pt

The training job was configured through 120,000 steps. Results in this card refer specifically to the 100,000-step checkpoint, not final_model or the 120,000-step checkpoint.

Model architecture

Item	Value
Framework	StarVLA StarVLA-WM-Wan-PI (WanPI)
Perception backbone	Wan-AI/Wan2.2-TI2V-5B-Diffusers
Text/image encoding	UMT5 text encoder and Wan VAE
Action model	LayerwiseFM flow-matching Action DiT
Wan feature layers	All 30 Wan transformer blocks
Action-DiT blocks	30: 15 layer-wise cross-attention and 15 self-attention blocks
Policy input	One egocentric RGB image, English instruction, 32-D robot state
Policy output	30-step, 36-D whole-body action chunk
Inference denoising steps	4

The image is encoded by the Wan VAE and the instruction by UMT5. Hidden features from all 30 Wan transformer blocks are projected to 1,024 dimensions and paired layer by layer with the Action DiT. Even-indexed Action-DiT blocks cross-attend to the matching Wan features; odd-indexed blocks perform Action-DiT self-attention.

UMT5 conditioning follows the Wan-format sequence convention: the attention mask determines the real text length, the real prefix is retained, and zero vectors pad the sequence back to 512 tokens. The Wan conditioning path does not forward a language/vision attention mask into the Action DiT. The original continuous 32-D robot state is retained and embedded as a separate Action-DiT state token; it is not converted into a language token or a controller latent.

The UMT5 text encoder and Wan VAE were frozen. The Wan transformer remained trainable at the base learning rate, while the Wan feature projector and Action-DiT used their dedicated learning rates.

Policy interface

The verified SIMPLE server contract is:

Field	Contract
Image count/order	One left-head RGB image
Client image	uint8, 224 x 224 x 3
Model image resize	Recorded training size [832, 480] before Wan encoding
Language	Canonical task instruction from the evaluation dataset
State	float32, shape 1 x 32
Normalized action	float32, shape 30 x 36
Action statistics	dataset_statistics.json, key new_embodiment
Evaluation control frequency	50 Hz; this is not a measured model inference rate

The 32-D state contains hand and arm joint values, torso roll/pitch/yaw, and body height. The 36-D action contains hand and arm targets, torso pose, body height, planar velocity commands, yaw velocity, and target yaw. Image, state, action ordering, and normalization must match the StarVLA SIMPLE adapter; the server does not infer or reorder camera views.

Training

The policy was trained with the simple_psi0_all_twelve_tasks LeRobot v2.1 mixture. The evaluated 100k checkpoint comes from the corrected run resumed from 10k model weights; optimizer state was intentionally not restored.

Setting	Value
Evaluated checkpoint	100,000 steps
Configured training length	120,000 steps
Per-device batch size	4
Action horizon	30
State / action dimensions	32 / 36
Action-model learning rate	1e-4
Wan-projector learning rate	1e-4
Wan-transformer base learning rate	1e-5
Scheduler	Cosine decay to 5e-7 with 5,000 warmup steps
Optimizer	AdamW, betas (0.9, 0.95)
Gradient accumulation	1
Gradient clipping	1.0
Checkpoint interval	10,000 steps
Training eval interval	1,000 steps

The joint training mixture contains:

1. G1WholebodyXMovePickTeleop-v0
2. G1WholebodyBendPickMP-v0
3. G1WholebodyHandoverTeleop-v0
4. G1WholebodyLocomotionPickBetweenTablesTeleop-v0
5. G1WholebodyTabletopGraspMP-v0
6. G1WholebodyXMoveBendPickTeleop-v0
7. G1WholebodyCloseDoorTeleop-v0
8. G1WholebodyOpenOvenTeleop-v0
9. G1WholebodyOpenFaucetTeleop-v0
10. G1WholebodyPickAndPlaceAndHugContainerTeleop-v0
11. G1WholebodyPushOfficeChairTeleop-v0
12. G1WholebodyOpenTrashCanTeleop-v0

Evaluation protocol

Each task/level uses the ten fixed official LeRobot evaluation instances, indices 0--9. The evaluator loads each serialized environment configuration and uses that task's canonical language instruction, native maximum episode length, and success criterion. Teleoperation tasks use SIMPLE's decoupled-WBC evaluator with the SONIC execution backend; the two MP tasks use SIMPLE's AMO evaluator.

Levels follow the SIMPLE protocol:

• Level 0: visual appearance and distractor variation.
• Level 1: Level 0 plus lighting variation.
• Level 2: Level 1 plus target-object pose variation.

Verified results

Results are successful episodes out of 10 in Level 0 | Level 1 | Level 2 order. All entries below use the same 100k checkpoint.

Ten-task SIMPLE comparison subset

Task	WanPI-100k
XMovePick	3 | 2 | 0
BendPick	0 | 0 | 0
Handover	5 | 2 | 4
LocomotionPickBetweenTables	8 | 2 | 0
TabletopGrasp	5 | 5 | 1
XMoveBendPick	0 | 0 | 0
CloseDoor	9 | 10 | 10
OpenOven	2 | 2 | 2
OpenFaucet	6 | 0 | 3
PickAndPlaceAndHugContainer	1 | 3 | 5
Level totals	39 | 26 | 25
Overall	90/300 (30.00%)

The official SIMPLE Psi0 reference is 226/300 (75.33%), but it combines ten separately trained task-specific models. WanPI uses one jointly trained model for the full suite. The official table also does not fully establish that all published baselines used the same task-native maximum-step and success-rule profile, so the number is contextual rather than a strictly controlled like-for-like comparison.

Two additional co-training tasks

Task	Level 0	Level 1	Level 2	Total
Push Office Chair	10/10	8/10	9/10	27/30 (90.00%)
Open Trash Can	1/10	1/10	1/10	3/30 (10.00%)
Combined	11/20	9/20	10/20	30/60 (50.00%)

Across all 12 co-training tasks, these verified runs total 120/360 successful episodes (33.33%). This combined value is an internal full-mixture summary, not an official SIMPLE leaderboard metric.

Loading and repository compatibility

This artifact is a complete StarVLA framework checkpoint. It is not a Transformers-native model and not a LoRA adapter. Use it with the StarVLA-WBC WanPI framework and keep the following files together:

README.md
config.yaml
config.full.yaml
dataset_statistics.json
checkpoints/steps_100000_pytorch_model.pt

The registry names StarVLA-WM-Wan-PI and WanPI are both accepted. The Wan 2.2 TI2V base weights must be available at the path configured by framework.world_model.base_wm and framework.qwenvl.base_vlm, or those paths must be overridden for the deployment environment.

Limitations and safety

• The checkpoint has been validated in the SIMPLE simulator only.
• Overall success is uneven across tasks and levels; in particular, the verified BendPick and XMoveBendPick results are zero on the ten-task subset.
• The model assumes the recorded G1 state/action layout, normalization, camera ordering, task language, and controller interface. Silent changes can invalidate behavior even when tensor shapes still match.
• The policy predicts a motion/action command; the external whole-body controller remains responsible for balance, contact constraints, actuator limits, collision handling, and safety.
• Do not connect this checkpoint directly to physical hardware without system identification, joint-limit checks, collision protection, emergency-stop validation, and staged real-world testing.

Download and restore the checkpoint

The 100k checkpoint is published as 8 numbered byte-stream parts, not independent PyTorch checkpoints. Restore the original .pt before loading:

hf download StarVLA/Wan2.2-5B-PI-SIMPLE --local-dir ./Wan2.2-5B-PI-SIMPLE
cd Wan2.2-5B-PI-SIMPLE
python merge_checkpoint.py

Keep all parts in their manifest order. The helper verifies the restored checkpoint's byte count and SHA256. Restoration needs at least 29 GB of additional free space on a filesystem supporting files larger than 4 GB. SHA256SUMS refers to the restored checkpoint.

The configuration uses public base-model identifiers and relative data/output directories. Download the base model from its public repository or override the base-model fields for your deployment. Training topology, machine addresses, account identifiers, and private filesystem paths are omitted from this release.