Blame for README.md

---
license: cc-by-nc-4.0
library_name: pytorch
tags:
  - robotics
  - robot-learning
  - latent-action-policy
---

# UniLAP

**A Unified Latent Action Policy for Cross-Embodiment Manipulation**

UniLAP predicts robot action chunks from visual observations, language
instructions, and robot state. This repository provides robot-specific
checkpoints containing the latent action policy and action decoder, together
with the configuration and normalization statistics needed for inference.

[Project and paper information](https://rayyoh.github.io/UniLAP/) ·
[Companion code](https://github.com/RayYoh/UniLAP)

## Checkpoints

| Directory | Robot | Benchmark | Tasks |
| --- | --- | --- | --- |
| `bridge/` | WidowX | SimplerEnv | Cube stacking, carrot placement, spoon placement, eggplant placement |
| `fractal/` | Google Robot | SimplerEnv | Move near, pick up a coke can, open and close drawers |
| `gr1/` | GR1 | RoboCasa | Pick-and-place tasks with a bottle, can, cup, milk, potato, and wine |

The Google Robot checkpoint is shared by the Visual Matching (VM) and
Variant Aggregation (VA) evaluations.

## Files

```text
UniLAP/
├── README.md
├── bridge/
│   ├── config.yaml
│   └── model.ckpt
├── fractal/
│   ├── config.yaml
│   └── model.ckpt
├── gr1/
│   ├── config.yaml
│   └── model.ckpt
└── metadata/
    ├── bridge_stats.json
    ├── fractal_stats.json
    └── gr1_stats.json
```

- `model.ckpt`: latent policy and action decoder weights.
- `config.yaml`: model dimensions, pretrained encoder identifiers, action
  representation, and normalization settings used by the inference loader.
- `metadata/*_stats.json`: action and state normalization statistics.

Keep each checkpoint beside its `config.yaml` and preserve the directory layout.

## Download

Use the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli).
Run these commands from the companion code directory.

Download all checkpoints:

```bash
hf download RayYoh/UniLAP --local-dir checkpoints/UniLAP
```

Or download one robot and its supporting files:

```bash
# WidowX
hf download RayYoh/UniLAP \
  bridge/model.ckpt bridge/config.yaml metadata/bridge_stats.json README.md \
  --local-dir checkpoints/UniLAP

# Google Robot
hf download RayYoh/UniLAP \
  fractal/model.ckpt fractal/config.yaml metadata/fractal_stats.json README.md \
  --local-dir checkpoints/UniLAP

# GR1
hf download RayYoh/UniLAP \
  gr1/model.ckpt gr1/config.yaml metadata/gr1_stats.json README.md \
  --local-dir checkpoints/UniLAP
```

## Inference and evaluation

The companion code release is being prepared at
[RayYoh/UniLAP](https://github.com/RayYoh/UniLAP). The commands below require a
code checkout containing `eval/run_lap_server.py` and `scripts/eval_paper.sh`.
Follow its installation instructions for the `unilap` policy environment and
`eval/envs/README.md` for the `simpler` and `gr1` simulator environments.

Run from the code checkout root:

```bash
conda activate unilap
CHECKPOINT_ROOT="$(pwd)/checkpoints/UniLAP"
```

To start a policy server independently, select `bridge`, `fractal`, or `gr1`:

```bash
ROBOT=bridge
python eval/run_lap_server.py \
  --ckpt_path "$CHECKPOINT_ROOT/$ROBOT/model.ckpt" \
  --stat_path "$CHECKPOINT_ROOT/metadata/${ROBOT}_stats.json" \
  --robot "$ROBOT" \
  --num_inference_steps 4 \
  --device cuda \
  --port 5555
```

The server loads DINOv2 ViT-B/14, SigLIP ViT-B/16
(`vit_base_patch16_siglip_224.v2_webli`), and Flan-T5-large through the companion
code. Their pretrained files must be available locally or downloadable on first use.

For closed-loop evaluation, use the following entrypoints. They start the policy
server and simulator client automatically. For SimplerEnv, set `SIMPLER_PATH` to
your checkout containing the RGB overlay assets.

```bash
export SIMPLER_PATH=/path/to/SimplerEnv

# WidowX
STAT_PATH="$CHECKPOINT_ROOT/metadata/bridge_stats.json" \
  bash scripts/eval_paper.sh bridge "$CHECKPOINT_ROOT/bridge/model.ckpt"

# Google Robot: Visual Matching
STAT_PATH="$CHECKPOINT_ROOT/metadata/fractal_stats.json" \
  bash scripts/eval_paper.sh fractal_vm "$CHECKPOINT_ROOT/fractal/model.ckpt"

# Google Robot: Variant Aggregation
STAT_PATH="$CHECKPOINT_ROOT/metadata/fractal_stats.json" \
  bash scripts/eval_paper.sh fractal_va "$CHECKPOINT_ROOT/fractal/model.ckpt"

# GR1
STAT_PATH="$CHECKPOINT_ROOT/metadata/gr1_stats.json" \
  bash scripts/eval_paper.sh gr1 "$CHECKPOINT_ROOT/gr1/model.ckpt"
```

## Citation

Please cite UniLAP when using these checkpoints in your research. See the
[project page](https://rayyoh.github.io/UniLAP/) for publication updates.

```bibtex
@misc{unilap2026,
  title = {UniLAP: A Unified Latent Action Policy for Cross-Embodiment Manipulation},
  author = {Lei Yao and Qiao Feng and Mutian Tong and Qihao Yun and Lap-Pui Chau and Lingjie Liu},
  year = {2026},
  url = {https://rayyoh.github.io/UniLAP/}
}
```

## License

The UniLAP-authored weights, configurations, and documentation in this repository
are licensed under
[Creative Commons Attribution-NonCommercial 4.0 International](https://creativecommons.org/licenses/by-nc/4.0/legalcode.en).
This permits noncommercial use, sharing, and adaptation with attribution under
the license terms.

The companion code uses the MIT License. Referenced pretrained models, datasets,
and simulators remain subject to their respective original licenses and terms.