README.md

---
license: apache-2.0
library_name: starvla
tags:
  - robotics
  - vision-language-action
  - world-model
  - robodojo
  - arxiv:2609.40341
base_model:
  - Wan-AI/Wan2.2-TI2V-5B-Diffusers
  - google/umt5-xxl
---

# Ego4WAM

**What Matters When Scaling Egocentric Human Data for Robot Learning?**

[Paper](https://arxiv.org/abs/2609.40341) ·
[Project page](https://sunzhihao18.github.io/Ego4WAM/)

Ego4WAM studies how human-robot alignment, data duration, task diversity and
available supervision shape the value of egocentric human data for robot
learning using a unified world-action model. This repository contains complete
model bundles for video-pretrained initialization and RoboDojo deployment.
Use the Ego4WAM-enabled source release; the Python import namespace remains
`starVLA`. These checkpoints are not standard Transformers `AutoModel` bundles.

## Models

| Model | Interaction | Intended use |
| --- | --- | --- |
| [Ego4WAM_VideoPretrained](Ego4WAM_VideoPretrained/README.md) | Base | Video-only initialization for Base or Joint RoboDojo Post-training; not a robot-action-posttrained policy |
| [Ego4WAM_VideoPretrained_RoboDojoJoint](Ego4WAM_VideoPretrained_RoboDojoJoint/README.md) | Joint | Video-pretrained, RoboDojo-posttrained policy for inference and evaluation |

The Joint weights are unchanged from the initial root-directory release.
Select a model directory explicitly; there is no default root checkpoint in
this collection. Wan-initialized RoboDojo checkpoints and video-pretrained
RoboDojo Base weights are not included in this release.

## Bundle layout

```text
Ego4WAM/
├── README.md
├── LICENSE
├── .gitattributes
├── Ego4WAM_VideoPretrained/
│   ├── README.md
│   ├── LICENSE
│   ├── inference_config.yaml
│   ├── model.safetensors
│   └── model_assets/
└── Ego4WAM_VideoPretrained_RoboDojoJoint/
    └── the same five-item bundle layout
```

Each checkpoint contains its Video DiT, Action DiT, state projection, VAE and
UMT5 weights. `model_assets/` bundles the VAE/text-encoder configurations and
tokenizer. Additional Wan/VAE/T5 weights do not need to be downloaded when
loading these bundles. RoboDojo normalization and kinematic assets are provided
by the code release, not by an internal experiment manifest.

## Download and load a single model

Use `allow_patterns` to avoid downloading both large checkpoints:

```python
from huggingface_hub import snapshot_download
from starVLA.model.framework.base_framework import baseframework

variant = "Ego4WAM_VideoPretrained_RoboDojoJoint"
root = snapshot_download(
    "HorizonRobotics/Ego4WAM",
    allow_patterns=[f"{variant}/**"],
)
model_dir = f"{root}/{variant}"
model = baseframework.from_pretrained(f"{model_dir}/model.safetensors")
```

This reads the adjacent `inference_config.yaml` and `model_assets/`, constructing
and loading the model with `strict=True`. Set `CKPT` to `model_dir` when using
the RoboDojo policy-server scripts. The model's public configuration exposes
only `framework.name: Ego4WAM` and `framework.interaction_mode: base|joint`.

For video-pretrained initialization, select `Ego4WAM_VideoPretrained` instead.
Set `EGO4WAM_PRETRAINED_DIR` to that downloaded model directory for the Base or
Joint RoboDojo Post-training scripts. See its model card for the Joint
initialization override; do not substitute the already-posttrained Joint policy.

The initial flat release remains available at revision
`262306ba29e9a4581ac5f714b90711498917444f` for existing pinned download commands.
New downloads from `main` should use the model-specific paths above.

## RoboDojo evaluation

The Joint policy uses 32-D state/action representations, a 32-step action
horizon and a fixed 20-step Euler sampler. The server converts predictions to
the 14-D RoboDojo joint-control output. Inputs are head, left-wrist and
right-wrist RGB observations plus state and environment geometry.

Use `examples/simBenchmarks/RoboDojo/eval_files_ego4wam/` and
`docs/RoboDojo.md` in the source release for the pinned RoboDojo version used
in the paper, dependency setup and local launch commands. This release does
not target the current official leaderboard/submission protocol or the newer
synchronized-data retraining. A one-task smoke does not reproduce full-suite
performance, and environment seeds alone do not pin the model RNG stream.

## Integrity

Each `model.safetensors` is 26,913,478,024 bytes, with these SHA256 digests:

| Model | SHA256 |
| --- | --- |
| Ego4WAM_VideoPretrained | `c5ca7480e9fbf504704e2e5bc3d9aba0e415838ef50bc993213e810a18b99f05` |
| Ego4WAM_VideoPretrained_RoboDojoJoint | `ae1b4595e1909e4573ad3214e1c5112f1920a34675340f6adfd4f2e9ec6c12c4` |

## Citation

```bibtex
@article{sun2026ego4wam,
  title={Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?},
  author={Sun, Zhihao and Liu, Liu and Wang, Xinjiang and Jiang, Haoyi and Feng, Wei and Zhang, Huiqiang and Jia, Xiaosong and Su, Zhizhong and Wu, Zuxuan},
  journal={arXiv preprint arXiv:2609.40341},
  year={2026}
}
```

## License and attribution

The released weights and Ego4WAM-authored code additions/modifications are
provided under Apache-2.0. Inherited StarVLA, FastWAM and third-party components
retain their original licenses and copyright notices; see NOTICE and LICENSES
in the source release. Wan2.2, UMT5, datasets, simulator SDKs and benchmark
assets remain subject to their own terms. Each model directory contains its
weights license.