| --- |
| library_name: pytorch |
| language: |
| - en |
| tags: |
| - arxiv:2610.02784 |
| - robotics |
| - tactile |
| - vision-language-action |
| - UniVTAC |
| --- |
| |
| # SimpleTouch |
|
|
| **Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?** |
|
|
| [Project Page](https://simpletouch-robot.github.io/) · [Paper](https://arxiv.org/pdf/2610.02784) · [arXiv](https://arxiv.org/abs/2610.02784) · [Code](https://github.com/ccyydd/SimpleTouch) |
|
|
| SimpleTouch adds a tactile expert to a pretrained π0.5 policy and learns contact-rich manipulation from task demonstrations, without additional tactile policy pretraining. A frozen T3 encoder represents two RGB tactile images, and the action expert reads the tactile expert's layerwise context. Future-touch prediction provides auxiliary supervision during training and is not run at inference. |
|
|
| ## Checkpoints |
|
|
| This repository provides task-specific **20k-step inference checkpoints** for six UniVTAC tasks. |
|
|
| | Task | Directory | Actions executed per policy query | |
| | --- | --- | ---: | |
| | Lift Bottle | `lift_bottle` | 50 | |
| | Pull-out Key | `pull_out_key` | 50 | |
| | Lift Can | `lift_can` | 50 | |
| | Put Bottle | `put_bottle_in_shelf` | 50 | |
| | Insert Hole | `insert_hole` | 25 | |
| | Insert Tube | `insert_tube` | 25 | |
|
|
| These are the default H50/H25 **execution horizons**, not the number of flow-matching inference steps. Temporal ensembling is off. All six checkpoints use the Fast inference path. |
|
|
| Each task directory contains: |
|
|
| ```text |
| <task>/global_step_20000/ |
| ├── actor/model_state_dict/full_weights.pt |
| ├── config.yaml |
| ├── norm_stats.json |
| └── _SUCCESS |
| ``` |
|
|
| Use the SimpleTouch loader, which reads the configuration and normalization statistics and validates the inference bundle. Optimizer state and distributed training-resume files are not included. |
|
|
| ## Quick Evaluation |
|
|
| Follow the [setup instructions](https://github.com/ccyydd/SimpleTouch/blob/main/README.md) for the policy environment, UniVTAC, T3, and tokenizer first. Run the following commands from the code repository root, with the policy environment active and those paths configured. |
|
|
| **Single GPU:** |
|
|
| ```bash |
| export TASK=insert_hole |
| GPU_ID=0 bash scripts/simpletouch/evaluate_univtac.sh |
| ``` |
|
|
| The launcher automatically downloads and validates the selected checkpoint on first use. Change `TASK` to any directory name in the table above. |
|
|
| **Multiple GPUs:** split the same 100 evaluation episodes across GPUs and merge the results into one directory. |
|
|
| ```bash |
| export TASK=insert_hole |
| SIMPLETOUCH_EVAL_GPUS=0,1,2,3 bash scripts/simpletouch/evaluate_univtac.sh |
| ``` |
|
|
| For a short single-GPU smoke test, prefix the command with `TOTAL_NUM=3`. To use an existing local download, also set `CKPT_PATH=/path/to/insert_hole/global_step_20000`. |
|
|
| Results are saved under `results/UniVTAC/SimpleTac/<task>/<run-id>` in the code repository. Set `SIMPLETOUCH_EVAL_ROOT` to use another output root. |
|
|
| ## Dependencies and Scope |
|
|
| - Obtain the [T3-large Mini encoder](https://huggingface.co/datasets/alanz-mit/FoundationTactile/tree/main/models/t3_large) and PaliGemma tokenizer using the instructions in the code repository. |
| - Install the [UniVTAC](https://github.com/univtac/UniVTAC) simulation environment and assets separately. They are not included in these weight bundles. |
| - These checkpoints are intended for their corresponding UniVTAC tasks, not direct deployment on physical robots. Third-party models, datasets, and dependencies retain their respective licenses and access requirements. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{yang2026simpletouchvisionlanguageactionmodelsmaster, |
| title={SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?}, |
| author={Chen Yang and Linzhe Shi and Changjie Wu and Hang Zhang and Ronghan Chen and Lingjun Zhang and Xu Hu and Mu Xu and Jiansheng Fan and Chen Wang}, |
| year={2026}, |
| eprint={2610.02784}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.RO}, |
| url={https://arxiv.org/abs/2610.02784}, |
| } |
| ``` |
|
|
