laiyingxin commited on
Commit
5f96789
路
verified 路
1 Parent(s): c42c306

Add Touch-R1 model card and NeurIPS citation

Browse files
Files changed (1) hide show
  1. README.md +69 -0
README.md ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-text-to-text
3
+ datasets:
4
+ - laiyingxin/TouchReason
5
+ tags:
6
+ - tactile-reasoning
7
+ - multimodal
8
+ - physical-reasoning
9
+ - reinforcement-learning
10
+ - robotics
11
+ - arxiv:2605.27154
12
+ ---
13
+
14
+ # Touch-R1
15
+
16
+ ### Reinforcing Touch Reasoning in MLLMs
17
+
18
+ [Paper](https://arxiv.org/abs/2605.27154) 路 [Project Page](https://laiyingxin2.github.io/Touch-R1/) 路 [GitHub](https://github.com/laiyingxin2/Touch-R1) 路 [TouchReason Dataset](https://huggingface.co/datasets/laiyingxin/TouchReason)
19
+
20
+ **Touch-R1 brings explicit reasoning to physical contact.** It connects optical tactile observations with vision and language, enabling multimodal models to use contact evidence when judging physical properties and revising misleading visual impressions.
21
+
22
+ > **Release status:** Model card available. Model weights and verified inference instructions are coming soon. The GitHub repository currently hosts the project website; research code is forthcoming.
23
+
24
+ ![Touch-R1 training overview](https://laiyingxin2.github.io/Touch-R1/assets/figures/touch_r1_overview_final.png)
25
+
26
+ ## Model Summary
27
+
28
+ | Item | Description |
29
+ | :--- | :--- |
30
+ | Model type | Tactile reasoning multimodal large language model |
31
+ | Backbone reported in the paper | Qwen2.5-VL-7B |
32
+ | Research data | TouchReason-1M and TouchReason-Bench |
33
+ | Inputs | Optical tactile observations, visual context, and language questions |
34
+ | Outputs | Structured textual reasoning and physical-property judgments |
35
+
36
+ ## Approach
37
+
38
+ The training pipeline combines three stages:
39
+
40
+ 1. **Tactile dynamics pretraining** learns contact-sensitive representations by predicting future tactile tokens.
41
+ 2. **QA supervised fine-tuning** aligns touch, vision, and language with structured reasoning.
42
+ 3. **Tactile-grounded GRPO** combines ordinal-aware accuracy, cross-sensor physical consistency, and structured-format rewards with an input-side tactile grounding objective.
43
+
44
+ Responses organize evidence through `<perceive>`, `<compare>`, and `<conclude>` stages. Counterfactual tactile inputs鈥攔emoved, shuffled, or noise-masked鈥攈elp identify predictions that depend on genuine contact evidence.
45
+
46
+ ## Research Use
47
+
48
+ Touch-R1 is designed for physical-property inference, ordinal comparison, cross-sensor reasoning, and visual鈥搕actile conflict resolution. Its intended interface is tactile/visual question answering with structured textual judgments.
49
+
50
+ The paper reports a **60.1 average score** for Touch-R1-7B on TouchReason-Bench. See the [paper](https://arxiv.org/abs/2605.27154) for the evaluation protocol and complete results.
51
+
52
+ ## Related Resources
53
+
54
+ - **Dataset:** [TouchReason](https://huggingface.co/datasets/laiyingxin/TouchReason)
55
+ - **Project page:** [Method, acquisition videos, and object atlas](https://laiyingxin2.github.io/Touch-R1/)
56
+ - **Project repository:** [Touch-R1](https://github.com/laiyingxin2/Touch-R1)
57
+
58
+ ## Citation
59
+
60
+ ```bibtex
61
+ @inproceedings{lai2026touchr1reinforcingtouchreasoning,
62
+ title = {{Touch-R1}: Reinforcing Touch Reasoning in {MLLMs}},
63
+ author = {Yingxin Lai and Yafei Zhou and Fucai Zhu and Siyu Zhu and Weihao Yuan},
64
+ booktitle = {Advances in Neural Information Processing Systems},
65
+ year = {2026},
66
+ note = {To appear},
67
+ url = {https://arxiv.org/abs/2605.27154}
68
+ }
69
+ ```