MINT-SJTU/Evo-DexBench_model_access_example

Evo-DexBench Model Access Examples

Eight inference checkpoint examples for Evo-DexBench, developed by MINT-SJTU. Use them to connect a policy server to the benchmark's simulator and reproduce the model integration workflow.

✨ Overview

• Four policies across eight inference checkpoints.
• Single-arm checkpoints cover Single2 Core for all four policies.
• Bimanual checkpoints cover Double3 Core for DP-T, DP-C, and KineDex, and Double2 Core for π0.5.
• Separate model and simulator environments communicate over WebSocket. The π0.5 adapter uses OpenPI's WebSocket protocol.

Each checkpoint covers four tasks in one benchmark task family. The links above point to the separate benchmark code, datasets, and simulation assets.

📦 Contents

The eight checkpoints total approximately 30.68 GB. DP-T, DP-C, and KineDex account for 5.80 GB. The two π0.5 checkpoints account for 24.88 GB.

Evo-DexBench_model_access_example/
├── DP-T/
│   ├── single2-core.ckpt
│   └── double3-core.ckpt
├── DP-C/
│   ├── single2-core.ckpt
│   └── double3-core.ckpt
├── KineDex/
│   ├── single2-core.ckpt
│   └── double3-core.ckpt
├── pi05/
│   ├── single2-core/
│   │   ├── params/
│   │   └── assets/single2-core/norm_stats.json
│   ├── double2-core/
│   │   ├── params/
│   │   └── assets/double2-core/norm_stats.json
│   ├── LICENSE_GEMMA.txt
│   └── NOTICE
├── manifest.json
└── README.md

Policy	Inputs	Single-arm checkpoint	Bimanual checkpoint
DP-T	RGB + robot state	Single2 · 0.171 GB	Double3 · 0.171 GB
DP-C	RGB + robot state	Single2 · 1.177 GB	Double3 · 1.185 GB
KineDex	RGB + robot state + tactile	Single2 · 1.544 GB	Double3 · 1.552 GB
π0.5 (OpenPI JAX)	RGB + robot state + task instruction	Single2 · 12.441 GB	Double2 · 12.442 GB

DP-T (Transformer), DP-C (convolutional U-Net), and KineDex use native policy workspace .ckpt files. These exports preserve the original EMA weights, normalizers, model configuration, and policy metadata. The π0.5 exports use native OpenPI JAX/Orbax directories with float32 EMA weights in params/ and the training normalization statistics in assets/. Keep both subdirectories together. All eight exports are for inference and omit the optimizer and training state required to resume the original runs.

The manifest records checkpoint sizes, SHA-256 checksums, and observation contracts.

Supported tasks

Checkpoint group	Task ID	Task name
single2-core	m06_v1	FaucetHandle
	m07_v1	MouseWheel
	m08_v1	CoffeeKnob
	m09_v1	KettleLidTab
double3-core	b11_v1	LaptopClosing
	b12_v1	StaplerPaperMark
	b13_v1	ScissorsOpening
	b14_v1	BoxLidOpening
double2-core (π0.5)	b06_v1	PenCapping
	b07_v1	CupSleeveFit
	b08_v1	BottleFunnelInsertion
	b09_v1	PlugInsertion

Observation and action contract

All checkpoints use 30 Hz control with 120 Hz simulation. DP-T, DP-C, and KineDex use two observation frames and execute eight actions per request by default.

Group	RGB views, in order	State	DP motion action	KineDex action
Single2	context, interaction, wrist_right	31	30	30 motion + 5 force
Double3	interaction, wrist_left, wrist_right	62	60	60 motion + 10 force

RGB views are supplied as 224×224 model inputs. KineDex uses full tactile observations with five finger arrays per hand, padded to 29×1 each. Its client converts predicted finger forces into bounded joint-control residuals. DP-T and DP-C do not request tactile observations. These examples do not use depth inputs.

For Double3, select the interaction camera with the client options shown below. DP uses the original 100-step DDPM inference, and KineDex uses its checkpoint's DDIM settings.

π0.5 uses one observation frame and the task instruction. It predicts a 50-action chunk and executes ten actions per request by default.

π0.5 group	RGB views, in order	State (model)	Action (model)	Prompt token limit
Single2	context, interaction, wrist_right	31 (32)	30 (32)	200
Double2	context, wrist_left, wrist_right	62 (64)	60 (64)	384

🚀 Quick Start

Run setup and evaluation from an Evo-DexBench code checkout. DP checkpoints require a code revision containing examples/policies/diffusion_policy/ and the policy-dp setup profile. If these are absent, obtain a DP-enabled release before following the DP commands.

1. Prepare the simulator and assets

tools/setup_environment.sh --profile sim --name evodex-sim \
  --with-dp-client --with-kinedex-client
conda activate evodex-sim
hf auth login
dexbench assets download --asset-dir "$PWD/assets"
source assets/activate_evodex_assets.sh

Accept the terms and obtain access to the official PartNet-Mobility repository before configuring assets. See the asset repository for asset distribution and licensing details.

2. Download checkpoints

Select the policy and task group you want to run. For example, download a single DP-C checkpoint:

hf download MINT-SJTU/Evo-DexBench_model_access_example DP-C/single2-core.ckpt \
  --repo-type model --local-dir checkpoints/model-access-example

For π0.5, download the complete Orbax directory along with the license, notice, and manifest:

hf download MINT-SJTU/Evo-DexBench_model_access_example \
  --repo-type model --local-dir checkpoints/model-access-example \
  --include 'pi05/single2-core/*' --include 'pi05/LICENSE_GEMMA.txt' \
  --include 'pi05/NOTICE' --include 'manifest.json'

Use pi05/double2-core/* for the bimanual checkpoint. Each π0.5 directory requires about 12.44 GB of disk space. Running hf download without a file or include filter downloads all eight checkpoints. Load PyTorch .ckpt files from trusted sources because their format uses Python pickle.

3. Run DP-C or DP-T

Create the independent model environment:

tools/setup_environment.sh --profile policy-dp --name evodex-dp

Terminal 1 — start a DP-C server:

conda activate evodex-dp
python -m dp_policy.server \
  --checkpoint checkpoints/model-access-example/DP-C/single2-core.ckpt \
  --upstream-root "$PWD/.third_party/DiffusionPolicy-official" \
  --host 127.0.0.1 --port 8100 --device cuda:0

Terminal 2 — run ten diagnostic episodes:

conda activate evodex-sim
source assets/activate_evodex_assets.sh
dexbench evaluate --policy dp_c --task-id m09_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --policy-kwargs '{"host":"127.0.0.1","port":8100,"embodiment":"single","execute_steps":8}' \
  --log results/dp_c_m09.jsonl

For DP-T, serve DP-T/single2-core.ckpt and select --policy dp_t in the client. Either single-arm checkpoint supports all four Single2 Core tasks.

For Double3, restart the server with the matching double3-core.ckpt, then use a supported bimanual task and the interaction view:

dexbench evaluate --policy dp_c --task-id b11_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --policy-kwargs '{"host":"127.0.0.1","port":8100,"embodiment":"dual","execute_steps":8,"camera_roles":["interaction","wrist_left","wrist_right"]}' \
  --log results/dp_c_b11.jsonl

For a DP-T Double3 checkpoint, use the same camera contract and --policy dp_t.

4. Run KineDex

Create its independent model environment:

tools/setup_environment.sh --profile policy-kinedex --name evodex-kinedex

Terminal 1 — start the model server:

conda activate evodex-kinedex
python -m kinedex_policy.server \
  --checkpoint checkpoints/model-access-example/KineDex/single2-core.ckpt \
  --upstream-root "$PWD/.third_party/KineDex-official" \
  --host 127.0.0.1 --port 8400 --device cuda:0

Terminal 2 — evaluate with full tactile inputs:

conda activate evodex-sim
source assets/activate_evodex_assets.sh
dexbench evaluate --policy kinedex --task-id m07_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --policy-kwargs '{"host":"127.0.0.1","port":8400,"embodiment":"single","execute_steps":8,"tactile_mode":"full"}' \
  --log results/kinedex_m07.jsonl

For Double3, restart the server with KineDex/double3-core.ckpt, then run:

dexbench evaluate --policy kinedex --task-id b11_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --policy-kwargs '{"host":"127.0.0.1","port":8400,"embodiment":"dual","execute_steps":8,"tactile_mode":"full","camera_profile":"interaction"}' \
  --log results/kinedex_b11.jsonl

5. Run π0.5 with the Evo-DexBench adapter

For π0.5 simulator inference, use a release of the official Evo-DexBench repository that includes examples/policies/pi05/ and the policy-pi05 setup profile. The adapter supplies the camera, state, and action mappings for OpenPI JAX.

Install uv before setting up the model server. Run the setup from the benchmark checkout. It creates a separate Python 3.11 environment in .third_party/openpi/.venv:

tools/setup_environment.sh --profile policy-pi05

If you already created the simulator environment in step 1, add the π0.5 client to it:

conda activate evodex-sim
python -m pip install -e examples/policies/pi05

For a new π0.5 simulator environment, use tools/setup_environment.sh --profile sim --name evodex-sim --with-pi05-client and complete the asset setup in step 1. The client package installs the lightweight OpenPI client and requires websockets>=15.

Terminal 1 runs the model server from the benchmark checkout:

export OPENPI_ROOT="$PWD/.third_party/openpi"
"$OPENPI_ROOT/.venv/bin/python" examples/policies/pi05/serve.py \
  --openpi-root "$OPENPI_ROOT" --embodiment single \
  --checkpoint checkpoints/model-access-example/pi05/single2-core --port 8000

Wait for the server to listen on port 8000 before starting the simulator.

Terminal 2 runs ten simulator episodes:

conda activate evodex-sim
source assets/activate_evodex_assets.sh
dexbench evaluate --policy pi05 --task-id m09_v1 \
  --episodes 10 --num-envs 1 --obs-mode rgb \
  --control-mode arm_pd_ee_delta_pose_hand_pd_joint_pos \
  --policy-kwargs '{"host":"127.0.0.1","port":8000,"embodiment":"single","execute_steps":10}' \
  --log results/pi05_m09.jsonl

For Double2, set the server's checkpoint path to checkpoints/model-access-example/pi05/double2-core and pass --embodiment dual. Set "embodiment":"dual" in the client options and select one of b06_v1–b09_v1. The adapter uses the context camera for Double2.

Evaluation protocol

The ten-episode commands run diagnostic integration checks. To run the standard 100-episode, four-worker protocol, omit --episodes and --num-envs. Use a new log filename for every invocation. GPU memory use includes both model inference and simulation rendering; a single-worker check is the starting point for memory-constrained machines.

Training and additional policy options are documented in the code repository's DP and KineDex examples. This model repository does not duplicate their code.

⚖️ License

MINT-SJTU provides the DP-T, DP-C, and KineDex checkpoints under Apache License 2.0. Upstream implementations, simulation software, datasets, and assets retain their respective licenses.

MINT-SJTU fine-tuned the PaliGemma-based π0.5 model on Evo-DexBench demonstrations. The π0.5 checkpoints are subject to the Gemma Terms of Use, including the use restrictions in the Gemma Prohibited Use Policy. See the Gemma Terms of Use, redistribution notice, and Gemma Prohibited Use Policy.

🙏 Acknowledgement

We thank the authors of Diffusion Policy, KineDex, OpenPI, LeRobot, ManiSkill, and SAPIEN for their open-source tools. The adapters pin Diffusion Policy to 5ba07ac6661db573af695b419a7947ecb704690f, KineDex to 9246f77d7b969b181b0f1b600601b9a9c8083011, and OpenPI to 15a9616a00943ada6c20a0f158e3adb39df2ccac.

📄 Citation

If you find Evo-DexBench useful in your research, please cite:

@misc{cheng2026evodexbenchbenchmarkinggeneralizationrelated,
      title={Evo-DexBench: Benchmarking Generalization across Related Tasks for Dexterous Manipulation},
      author={Bing Cheng and Yiran Mao and Zewei Ye and Yifan Jin and Xinyi Chen and Ruifeng Tu and Shuhuai Wang and Tao Lin and Zhimu Zhou and Yuxin Du and Fangwei Zhong and Junchi Yan and Bo Zhao},
      year={2026},
      eprint={2610.21180},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2610.21180},
}