| --- |
| language: |
| - en |
| datasets: |
| - MINT-SJTU/Evo-DexBench-dataset |
| tags: |
| - robotics |
| - dexterous-manipulation |
| - diffusion-policy |
| - vision-language-action |
| - jax |
| - tactile |
| - simulation |
| --- |
| |
| # Evo-DexBench Model Access Examples |
|
|
| [](https://arxiv.org/abs/2610.21180) |
| [](https://github.com/MINT-SJTU/Evo-DexBench) |
| [](https://mint-sjtu.github.io/Evo-DexBench.io) |
| [](https://huggingface.co/datasets/MINT-SJTU/Evo-DexBench-dataset) |
| [](https://huggingface.co/datasets/MINT-SJTU/Evo-DexBench_Assets) |
|
|
| Eight inference checkpoint examples for **Evo-DexBench**, developed by |
| **MINT-SJTU**. Use them to connect a policy server to the benchmark's simulator |
| and reproduce the model integration workflow. |
|
|
| ## β¨ Overview |
|
|
| - **Four policies** across eight inference checkpoints. |
| - **Single-arm** checkpoints cover Single2 Core for all four policies. |
| - **Bimanual** checkpoints cover Double3 Core for DP-T, DP-C, and KineDex, |
| and Double2 Core for Ο0.5. |
| - **Separate model and simulator environments** communicate over WebSocket. |
| The Ο0.5 adapter uses OpenPI's WebSocket protocol. |
|
|
| Each checkpoint covers four tasks in one benchmark task family. The links above |
| point to the separate benchmark code, datasets, and simulation assets. |
|
|
| ## π¦ Contents |
|
|
| The eight checkpoints total approximately **30.68 GB**. DP-T, DP-C, and |
| KineDex account for **5.80 GB**. The two Ο0.5 checkpoints account for **24.88 GB**. |
|
|
| ```text |
| Evo-DexBench_model_access_example/ |
| βββ DP-T/ |
| β βββ single2-core.ckpt |
| β βββ double3-core.ckpt |
| βββ DP-C/ |
| β βββ single2-core.ckpt |
| β βββ double3-core.ckpt |
| βββ KineDex/ |
| β βββ single2-core.ckpt |
| β βββ double3-core.ckpt |
| βββ pi05/ |
| β βββ single2-core/ |
| β β βββ params/ |
| β β βββ assets/single2-core/norm_stats.json |
| β βββ double2-core/ |
| β β βββ params/ |
| β β βββ assets/double2-core/norm_stats.json |
| β βββ LICENSE_GEMMA.txt |
| β βββ NOTICE |
| βββ manifest.json |
| βββ README.md |
| ``` |
|
|
| | Policy | Inputs | Single-arm checkpoint | Bimanual checkpoint | |
| | --- | --- | --- | --- | |
| | DP-T | RGB + robot state | Single2 Β· 0.171 GB | Double3 Β· 0.171 GB | |
| | DP-C | RGB + robot state | Single2 Β· 1.177 GB | Double3 Β· 1.185 GB | |
| | KineDex | RGB + robot state + tactile | Single2 Β· 1.544 GB | Double3 Β· 1.552 GB | |
| | Ο0.5 (OpenPI JAX) | RGB + robot state + task instruction | Single2 Β· 12.441 GB | Double2 Β· 12.442 GB | |
|
|
| DP-T (Transformer), DP-C (convolutional U-Net), and KineDex use native policy |
| workspace `.ckpt` files. These exports preserve the original EMA weights, |
| normalizers, model configuration, and policy metadata. The Ο0.5 exports use |
| native OpenPI JAX/Orbax directories with float32 EMA weights in `params/` and |
| the training normalization statistics in `assets/`. Keep both subdirectories |
| together. All eight exports are for inference and omit the optimizer and |
| training state required to resume the original runs. |
|
|
| The [manifest](manifest.json) records checkpoint sizes, SHA-256 checksums, |
| and observation contracts. |
|
|
| ### Supported tasks |
|
|
| | Checkpoint group | Task ID | Task name | |
| | --- | --- | --- | |
| | `single2-core` | `m06_v1` | FaucetHandle | |
| | | `m07_v1` | MouseWheel | |
| | | `m08_v1` | CoffeeKnob | |
| | | `m09_v1` | KettleLidTab | |
| | `double3-core` | `b11_v1` | LaptopClosing | |
| | | `b12_v1` | StaplerPaperMark | |
| | | `b13_v1` | ScissorsOpening | |
| | | `b14_v1` | BoxLidOpening | |
| | `double2-core` (Ο0.5) | `b06_v1` | PenCapping | |
| | | `b07_v1` | CupSleeveFit | |
| | | `b08_v1` | BottleFunnelInsertion | |
| | | `b09_v1` | PlugInsertion | |
|
|
| ### Observation and action contract |
|
|
| All checkpoints use **30 Hz control** with **120 Hz simulation**. DP-T, DP-C, |
| and KineDex use two observation frames and execute eight actions per request |
| by default. |
|
|
| | Group | RGB views, in order | State | DP motion action | KineDex action | |
| | --- | --- | ---: | ---: | ---: | |
| | Single2 | context, interaction, wrist_right | 31 | 30 | 30 motion + 5 force | |
| | Double3 | **interaction**, wrist_left, wrist_right | 62 | 60 | 60 motion + 10 force | |
| |
| RGB views are supplied as `224Γ224` model inputs. KineDex uses full tactile |
| observations with five finger arrays per hand, padded to `29Γ1` each. Its |
| client converts predicted finger forces into bounded joint-control residuals. |
| DP-T and DP-C do not request tactile observations. These examples do not use |
| depth inputs. |
| |
| For Double3, select the interaction camera with the client options shown |
| below. DP uses the original 100-step DDPM inference, and KineDex uses its |
| checkpoint's DDIM settings. |
| |
| Ο0.5 uses one observation frame and the task instruction. It predicts a |
| 50-action chunk and executes ten actions per request by default. |
| |
| | Ο0.5 group | RGB views, in order | State (model) | Action (model) | Prompt token limit | |
| | --- | --- | ---: | ---: | ---: | |
| | Single2 | context, interaction, wrist_right | 31 (32) | 30 (32) | 200 | |
| | Double2 | context, wrist_left, wrist_right | 62 (64) | 60 (64) | 384 | |
|
|
| ## π Quick Start |
|
|
| Run setup and evaluation from an |
| [Evo-DexBench code checkout](https://github.com/MINT-SJTU/Evo-DexBench). |
| DP checkpoints require a code revision containing |
| `examples/policies/diffusion_policy/` and the `policy-dp` setup profile. If these |
| are absent, obtain a DP-enabled release before following the DP commands. |
|
|
| ### 1. Prepare the simulator and assets |
|
|
| ```bash |
| tools/setup_environment.sh --profile sim --name evodex-sim \ |
| --with-dp-client --with-kinedex-client |
| conda activate evodex-sim |
| hf auth login |
| dexbench assets download --asset-dir "$PWD/assets" |
| source assets/activate_evodex_assets.sh |
| ``` |
|
|
| Accept the terms and obtain access to the official |
| [PartNet-Mobility repository](https://huggingface.co/datasets/sapien-sim/PartNetMobility) |
| before configuring assets. See the |
| [asset repository](https://huggingface.co/datasets/MINT-SJTU/Evo-DexBench_Assets) |
| for asset distribution and licensing details. |
|
|
| ### 2. Download checkpoints |
|
|
| Select the policy and task group you want to run. For example, download a |
| single DP-C checkpoint: |
|
|
| ```bash |
| hf download MINT-SJTU/Evo-DexBench_model_access_example DP-C/single2-core.ckpt \ |
| --repo-type model --local-dir checkpoints/model-access-example |
| ``` |
|
|
| For Ο0.5, download the complete Orbax directory along with the license, |
| notice, and manifest: |
|
|
| ```bash |
| hf download MINT-SJTU/Evo-DexBench_model_access_example \ |
| --repo-type model --local-dir checkpoints/model-access-example \ |
| --include 'pi05/single2-core/*' --include 'pi05/LICENSE_GEMMA.txt' \ |
| --include 'pi05/NOTICE' --include 'manifest.json' |
| ``` |
|
|
| Use `pi05/double2-core/*` for the bimanual checkpoint. Each Ο0.5 directory |
| requires about 12.44 GB of disk space. Running `hf download` without a file |
| or include filter downloads all eight checkpoints. Load PyTorch `.ckpt` files |
| from trusted sources because their format uses Python pickle. |
|
|
| ### 3. Run DP-C or DP-T |
|
|
| Create the independent model environment: |
|
|
| ```bash |
| tools/setup_environment.sh --profile policy-dp --name evodex-dp |
| ``` |
|
|
| Terminal 1 β start a DP-C server: |
|
|
| ```bash |
| conda activate evodex-dp |
| python -m dp_policy.server \ |
| --checkpoint checkpoints/model-access-example/DP-C/single2-core.ckpt \ |
| --upstream-root "$PWD/.third_party/DiffusionPolicy-official" \ |
| --host 127.0.0.1 --port 8100 --device cuda:0 |
| ``` |
|
|
| Terminal 2 β run ten diagnostic episodes: |
|
|
| ```bash |
| conda activate evodex-sim |
| source assets/activate_evodex_assets.sh |
| dexbench evaluate --policy dp_c --task-id m09_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8100,"embodiment":"single","execute_steps":8}' \ |
| --log results/dp_c_m09.jsonl |
| ``` |
|
|
| For **DP-T**, serve `DP-T/single2-core.ckpt` and select `--policy dp_t` in the |
| client. Either single-arm checkpoint supports all four Single2 Core tasks. |
|
|
| For **Double3**, restart the server with the matching |
| `double3-core.ckpt`, then use a supported bimanual task and the interaction view: |
|
|
| ```bash |
| dexbench evaluate --policy dp_c --task-id b11_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8100,"embodiment":"dual","execute_steps":8,"camera_roles":["interaction","wrist_left","wrist_right"]}' \ |
| --log results/dp_c_b11.jsonl |
| ``` |
|
|
| For a DP-T Double3 checkpoint, use the same camera contract and `--policy dp_t`. |
|
|
| ### 4. Run KineDex |
|
|
| Create its independent model environment: |
|
|
| ```bash |
| tools/setup_environment.sh --profile policy-kinedex --name evodex-kinedex |
| ``` |
|
|
| Terminal 1 β start the model server: |
|
|
| ```bash |
| conda activate evodex-kinedex |
| python -m kinedex_policy.server \ |
| --checkpoint checkpoints/model-access-example/KineDex/single2-core.ckpt \ |
| --upstream-root "$PWD/.third_party/KineDex-official" \ |
| --host 127.0.0.1 --port 8400 --device cuda:0 |
| ``` |
|
|
| Terminal 2 β evaluate with full tactile inputs: |
|
|
| ```bash |
| conda activate evodex-sim |
| source assets/activate_evodex_assets.sh |
| dexbench evaluate --policy kinedex --task-id m07_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8400,"embodiment":"single","execute_steps":8,"tactile_mode":"full"}' \ |
| --log results/kinedex_m07.jsonl |
| ``` |
|
|
| For **Double3**, restart the server with `KineDex/double3-core.ckpt`, then run: |
|
|
| ```bash |
| dexbench evaluate --policy kinedex --task-id b11_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8400,"embodiment":"dual","execute_steps":8,"tactile_mode":"full","camera_profile":"interaction"}' \ |
| --log results/kinedex_b11.jsonl |
| ``` |
|
|
| ### 5. Run Ο0.5 with the Evo-DexBench adapter |
|
|
| For Ο0.5 simulator inference, use a release of the |
| [official Evo-DexBench repository](https://github.com/MINT-SJTU/Evo-DexBench) |
| that includes `examples/policies/pi05/` and the `policy-pi05` setup profile. |
| The adapter supplies the camera, state, and action mappings for |
| [OpenPI JAX](https://github.com/Physical-Intelligence/openpi). |
|
|
| Install [uv](https://docs.astral.sh/uv/getting-started/installation/) before |
| setting up the model server. Run the setup from the benchmark checkout. |
| It creates a separate Python 3.11 environment in `.third_party/openpi/.venv`: |
|
|
| ```bash |
| tools/setup_environment.sh --profile policy-pi05 |
| ``` |
|
|
| If you already created the simulator environment in step 1, add the Ο0.5 |
| client to it: |
|
|
| ```bash |
| conda activate evodex-sim |
| python -m pip install -e examples/policies/pi05 |
| ``` |
|
|
| For a new Ο0.5 simulator environment, use |
| `tools/setup_environment.sh --profile sim --name evodex-sim --with-pi05-client` |
| and complete the asset setup in step 1. The client package installs the |
| lightweight OpenPI client and requires `websockets>=15`. |
|
|
| Terminal 1 runs the model server from the benchmark checkout: |
|
|
| ```bash |
| export OPENPI_ROOT="$PWD/.third_party/openpi" |
| "$OPENPI_ROOT/.venv/bin/python" examples/policies/pi05/serve.py \ |
| --openpi-root "$OPENPI_ROOT" --embodiment single \ |
| --checkpoint checkpoints/model-access-example/pi05/single2-core --port 8000 |
| ``` |
|
|
| Wait for the server to listen on port 8000 before starting the simulator. |
|
|
| Terminal 2 runs ten simulator episodes: |
|
|
| ```bash |
| conda activate evodex-sim |
| source assets/activate_evodex_assets.sh |
| dexbench evaluate --policy pi05 --task-id m09_v1 \ |
| --episodes 10 --num-envs 1 --obs-mode rgb \ |
| --control-mode arm_pd_ee_delta_pose_hand_pd_joint_pos \ |
| --policy-kwargs '{"host":"127.0.0.1","port":8000,"embodiment":"single","execute_steps":10}' \ |
| --log results/pi05_m09.jsonl |
| ``` |
|
|
| For Double2, set the server's checkpoint path to |
| `checkpoints/model-access-example/pi05/double2-core` and pass `--embodiment dual`. |
| Set `"embodiment":"dual"` in the client options and select one of |
| `b06_v1`β`b09_v1`. The adapter uses the context camera for Double2. |
|
|
| ### Evaluation protocol |
|
|
| The ten-episode commands run diagnostic integration checks. To run the |
| standard 100-episode, four-worker protocol, omit |
| `--episodes` and `--num-envs`. Use a new log filename for every invocation. |
| GPU memory use includes both model inference and simulation rendering; a |
| single-worker check is the starting point for memory-constrained machines. |
|
|
| Training and additional policy options are documented in the code repository's |
| DP and KineDex examples. This model repository does not duplicate their code. |
|
|
| ## βοΈ License |
|
|
| MINT-SJTU provides the DP-T, DP-C, and KineDex checkpoints under **Apache |
| License 2.0**. Upstream implementations, simulation software, datasets, and |
| assets retain their respective licenses. |
|
|
| MINT-SJTU fine-tuned the PaliGemma-based Ο0.5 model on Evo-DexBench |
| demonstrations. The Ο0.5 checkpoints are subject to the Gemma Terms of Use, |
| including the use restrictions in the Gemma Prohibited Use Policy. See the |
| [Gemma Terms of Use](pi05/LICENSE_GEMMA.txt), [redistribution notice](pi05/NOTICE), |
| and [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy). |
|
|
| ## π Acknowledgement |
|
|
| We thank the authors of |
| [Diffusion Policy](https://github.com/real-stanford/diffusion_policy), |
| [KineDex](https://github.com/DinoMini00/KineDex_code), |
| [OpenPI](https://github.com/Physical-Intelligence/openpi), |
| [LeRobot](https://github.com/huggingface/lerobot), |
| [ManiSkill](https://github.com/haosulab/ManiSkill), and |
| [SAPIEN](https://github.com/haosulab/SAPIEN) for their open-source tools. |
| The adapters pin Diffusion Policy to |
| `5ba07ac6661db573af695b419a7947ecb704690f`, KineDex to |
| `9246f77d7b969b181b0f1b600601b9a9c8083011`, and OpenPI to |
| `15a9616a00943ada6c20a0f158e3adb39df2ccac`. |
|
|
| ## π Citation |
|
|
| If you find Evo-DexBench useful in your research, please cite: |
|
|
| ```bibtex |
| @misc{cheng2026evodexbenchbenchmarkinggeneralizationrelated, |
| title={Evo-DexBench: Benchmarking Generalization across Related Tasks for Dexterous Manipulation}, |
| author={Bing Cheng and Yiran Mao and Zewei Ye and Yifan Jin and Xinyi Chen and Ruifeng Tu and Shuhuai Wang and Tao Lin and Zhimu Zhou and Yuxin Du and Fangwei Zhong and Junchi Yan and Bo Zhao}, |
| year={2026}, |
| eprint={2610.21180}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.RO}, |
| url={https://arxiv.org/abs/2610.21180}, |
| } |
| ``` |
|
|
