Document the eight HandsOn checkpoints and upload status
Browse files
README.md
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
tags:
|
| 5 |
+
- robotics
|
| 6 |
+
- dexterous-manipulation
|
| 7 |
+
- handson
|
| 8 |
+
- vision-language-action
|
| 9 |
+
- world-action-model
|
| 10 |
+
datasets:
|
| 11 |
+
- Cyan666/HandsOn-Datasets
|
| 12 |
+
papers:
|
| 13 |
+
- 2610.26765
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# HandsOn: paper multi-task checkpoints
|
| 17 |
+
|
| 18 |
+
The eight multi-task checkpoints selected in **HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation**.
|
| 19 |
+
|
| 20 |
+
**Upload status: in progress.** The release is complete only when `UPLOAD_VERIFIED.json` is present and lists all eight models.
|
| 21 |
+
|
| 22 |
+
[Paper](https://arxiv.org/abs/2610.26765) · [Code and inference adapters](https://github.com/TsinghuaC3I/HandsOn) · [Dataset](https://huggingface.co/datasets/Cyan666/HandsOn-Datasets) · [Model integration guide](https://github.com/TsinghuaC3I/HandsOn/blob/main/docs/paper-models.md)
|
| 23 |
+
|
| 24 |
+
| Model | Optimizer step | Checkpoint path |
|
| 25 |
+
|---|---:|---|
|
| 26 |
+
| π0.5 | 120,000 | `pi05/120000/` |
|
| 27 |
+
| DM0.5 | 86,000 | `dm05/checkpoint-86000/` |
|
| 28 |
+
| HY-Embodied-0.5-VLA | 125,000 | `hy/checkpoint-125000/` |
|
| 29 |
+
| LingBot-VLA | 151,000 | `lingbot/checkpoint-151000/` |
|
| 30 |
+
| OpenWAM | 70,000 | `openwam/step-070000/mp_rank_00_model_states.pt` |
|
| 31 |
+
| FastWAM | 100,000 | `fastwam/step-100000/mp_rank_00_model_states.pt` |
|
| 32 |
+
| Motus | 100,000 | `motus/step-100000/mp_rank_00_model_states.pt` |
|
| 33 |
+
| Spatial Forcing | 90,000 | `spatial/90000/` |
|
| 34 |
+
|
| 35 |
+
These are fixed paper selections, not checkpoints chosen by benchmark success. The single-task DP reference and the prompted GPT-6 controller are outside this weight release.
|
| 36 |
+
|
| 37 |
+
## Download
|
| 38 |
+
|
| 39 |
+
```bash
|
| 40 |
+
pip install -U huggingface_hub
|
| 41 |
+
# One model:
|
| 42 |
+
hf download Cyan666/HandsOn-Checkpoints --include 'pi05/**' CHECKPOINTS.json --local-dir ./handson-checkpoints
|
| 43 |
+
# All eight models:
|
| 44 |
+
hf download Cyan666/HandsOn-Checkpoints --local-dir ./handson-checkpoints
|
| 45 |
+
```
|
| 46 |
+
|
| 47 |
+
The native checkpoint formats and tensor bytes are preserved. Keep each Orbax `params/` tree and each DCP `state/` tree intact, including hidden metadata files. Standalone π0.5 optimizer state and OpenWAM RNG snapshots are omitted; native DCP and PyTorch containers can still contain training state. File sizes, model steps and relative paths are recorded in `CHECKPOINTS.json`.
|
| 48 |
+
|
| 49 |
+
## Inference
|
| 50 |
+
|
| 51 |
+
Use the matching `--model` profile from the [HandsOn model integration guide](https://github.com/TsinghuaC3I/HandsOn/blob/main/docs/paper-models.md). These are native research checkpoints, not `transformers.AutoModel` packages. The release's adapters preserve checkpoint-specific normalization, image preprocessing, denoising and action horizons.
|
| 52 |
+
|
| 53 |
+
π0.5 normalization is at `pi05/120000/assets/training_package_final18k.building/norm_stats.json`; Spatial Forcing normalization is at `spatial/90000/assets/v16_eef52/norm_stats.json`. The other adapters use the normalization assets packaged in the code release.
|
| 54 |
+
|
| 55 |
+
Foundation-model weights, tokenizers, and FastWAM/Motus language caches are supplied separately as documented in the integration guide. Obtain foundation assets from their original publishers and retain their terms. The code provides a language-cache preparation command.
|
| 56 |
+
|
| 57 |
+
Example, after installing the π0.5 inference environment from the code repository:
|
| 58 |
+
|
| 59 |
+
```bash
|
| 60 |
+
python scripts/serve_paper_policy.py --model pi05 \
|
| 61 |
+
--checkpoint /path/to/handson-checkpoints/pi05/120000 \
|
| 62 |
+
--normalization /path/to/handson-checkpoints/pi05/120000/assets/training_package_final18k.building/norm_stats.json \
|
| 63 |
+
--tokenizer /path/to/paligemma_tokenizer.model
|
| 64 |
+
```
|
| 65 |
+
|
| 66 |
+
All profiles output absolute physical EEF52 commands, ordered right arm/hand then left arm/hand, for the shared 30 Hz native evaluator. Loading a checkpoint alone does not reproduce the paper's evaluation protocol or certify a new rollout.
|
| 67 |
+
|
| 68 |
+
## Provenance and terms
|
| 69 |
+
|
| 70 |
+
This repository publishes the selected HandsOn finetuning checkpoints in their original formats. Upstream model and component licenses continue to apply; this repository does not replace or broaden those terms. See the [native source notices](https://github.com/TsinghuaC3I/HandsOn/blob/main/docs/policy-source-notices.md) and the relevant foundation-model cards for component-specific information.
|
| 71 |
+
|
| 72 |
+
The upload verification records file sizes and SHA256 digests. It verifies storage and transfer integrity, not a fresh GPU evaluation or retraining.
|