| --- |
| license: mit |
| pipeline_tag: robotics |
| tags: |
| - world-action-model |
| - world-model |
| - vision-language-action |
| - scene-flow |
| - robot-manipulation |
| - robotics |
| --- |
| |
| # UNITAS |
|
|
| **UNITAS** is a 3D-native **world action model** for embodied manipulation. |
| This repository provides inference checkpoints. |
|
|
| [Paper](https://arxiv.org/abs/2610.12099) 路 [Project](https://dexforce.com/technical-report/#/unitas-wam) 路 [Code](https://github.com/DexForce/UNITAS) |
|
|
| [RoboTwin](#robotwin) 路 [LIBERO](#libero) 路 [VLABench](#vlabench) 路 [Scene flow](#scene-flow) |
|
|
| ## RoboTwin |
|
|
| Policies for RoboTwin Clean and Randomized evaluation. Use `--domain robotwin`. |
|
|
| | Training | Checkpoint | Download | |
| |---|---|---| |
| | From scratch | [unitas_robotwin_scratch.pt](https://huggingface.co/RobbinWang123/Unitas/blob/main/unitas_robotwin_scratch.pt) | [Download](https://huggingface.co/RobbinWang123/Unitas/resolve/main/unitas_robotwin_scratch.pt) | |
| | Pretrained, then fine-tuned | [unitas_robotwin_finetuned.pt](https://huggingface.co/RobbinWang123/Unitas/blob/main/unitas_robotwin_finetuned.pt) | [Download](https://huggingface.co/RobbinWang123/Unitas/resolve/main/unitas_robotwin_finetuned.pt) | |
|
|
| ## LIBERO |
|
|
| Policies for LIBERO; the fine-tuned model also supports LIBERO-Plus evaluation. Use `--domain libero`. |
|
|
| | Training | Checkpoint | Download | |
| |---|---|---| |
| | From scratch | [unitas_libero_scratch.pt](https://huggingface.co/RobbinWang123/Unitas/blob/main/unitas_libero_scratch.pt) | [Download](https://huggingface.co/RobbinWang123/Unitas/resolve/main/unitas_libero_scratch.pt) | |
| | Pretrained, then fine-tuned | [unitas_libero_finetuned.pt](https://huggingface.co/RobbinWang123/Unitas/blob/main/unitas_libero_finetuned.pt) | [Download](https://huggingface.co/RobbinWang123/Unitas/resolve/main/unitas_libero_finetuned.pt) | |
|
|
| ## VLABench |
|
|
| Policies for VLABench with the five custom evaluation tracks. Use `--domain vlabench`. |
|
|
| | Training | Checkpoint | Download | |
| |---|---|---| |
| | From scratch | [unitas_vlabench_scratch.pt](https://huggingface.co/RobbinWang123/Unitas/blob/main/unitas_vlabench_scratch.pt) | [Download](https://huggingface.co/RobbinWang123/Unitas/resolve/main/unitas_vlabench_scratch.pt) | |
| | Pretrained, then fine-tuned | [unitas_vlabench_finetuned.pt](https://huggingface.co/RobbinWang123/Unitas/blob/main/unitas_vlabench_finetuned.pt) | [Download](https://huggingface.co/RobbinWang123/Unitas/resolve/main/unitas_vlabench_finetuned.pt) | |
|
|
| ## Scene flow |
|
|
| Jointly trained on RoboTwin, LIBERO and VLABench for action and dense scene flow prediction. |
| The model predicts gripper motion once, then uses it to condition scene flow prediction. |
|
|
| | Training | Checkpoint | Download | |
| |---|---|---| |
| | Joint training | [unitas_scene_flow.pt](https://huggingface.co/RobbinWang123/Unitas/blob/main/unitas_scene_flow.pt) | [Download](https://huggingface.co/RobbinWang123/Unitas/resolve/main/unitas_scene_flow.pt) | |
|
|
| ## Usage |
|
|
| Install the inference code from [UNITAS](https://github.com/DexForce/UNITAS) |
| and run the commands below from the repository root. |
| Each `.pt` file contains model tensors and inference configuration (`model` and `args`), |
| without optimizer state or training metadata. |
|
|
| Download Florence-2-large (shared by all UNITAS checkpoints) and the checkpoint you want to use: |
|
|
| ```bash |
| python -m pip install "huggingface_hub[cli]>=0.34,<1" |
| hf download microsoft/Florence-2-large \ |
| --local-dir checkpoints/Florence-2-large --exclude "*.bin" |
| hf download RobbinWang123/Unitas unitas_robotwin_finetuned.pt --local-dir checkpoints |
| ``` |
|
|
| Start a policy server. Set the domain and checkpoint to match the benchmark above; |
| both scratch and fine-tuned models use the same interface. |
|
|
| ```bash |
| python src/unitas/inference/server.py --domain robotwin \ |
| --checkpoint checkpoints/unitas_robotwin_finetuned.pt \ |
| --florence-dir checkpoints/Florence-2-large --port 8000 |
| ``` |
|
|
| The benchmark client connects to `http://127.0.0.1:8000`. |
|
|
| For scene flow, download `unitas_scene_flow.pt` and run a bundled sample from the code checkout: |
|
|
| ```bash |
| python src/unitas/scene_flow/inference.py \ |
| --checkpoint checkpoints/unitas_scene_flow.pt \ |
| --florence-dir checkpoints/Florence-2-large \ |
| --input assets/scene_flow/observations/robotwin_7c59d8eb33_0088.npz \ |
| --output outputs/scene_flow.npz |
| python src/unitas/scene_flow/visualize.py --input outputs/scene_flow.npz --port 8080 |
| ``` |
|
|
| Open `http://127.0.0.1:8080` for the interactive visualization. |
|
|
| ## Citation |
|
|
| If you find UNITAS useful, please cite: |
|
|
| ```bibtex |
| @misc{wang2026unitas3dnativeworldaction, |
| title={UNITAS: A 3D-Native World Action Model for Embodied Manipulation}, |
| author={Ruixiang Wang and Yongyi Su and Wenlve Zhou and Bo Yue and Hengyan Liu and Dekun Lu and Yuxin Tian and Yihan Fang and Zerui Wu and Xing Hu and Jietao Chen and Yong Guo and Ziyan He and Junbin Yuan and Guiliang Liu and Xiaofen Xing and Kui Jia}, |
| year={2026}, |
| eprint={2610.12099}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.RO}, |
| url={https://arxiv.org/abs/2610.12099}, |
| } |
| ``` |
|
|
