README.md

---
license: apache-2.0
---

# FineGrasp: Towards Robust Grasping for Delicate Objects

<div align="center" class="authors">
    <a href="https://scholar.google.com/citations?user=Ke5SamYAAAAJ&hl=en" target="_blank">Yun Du*</a>,
    <a href="https://scholar.google.com/citations?hl=en&user=aB02FDEAAAAJ" target="_blank">Mengao Zhao*</a>,
    <a href="https://wzmsltw.github.io/" target="_blank">Tianwei Lin</a>,
    <a href="" target="_blank">Yiwei Jin</a>,
    <a href="" target="_blank">Chaodong Huang</a>,
    <a href="https://scholar.google.com/citations?user=HQfc8TEAAAAJ&hl=en" target="_blank">Zhizhong Su</a>
</div>

<div align="center" style="line-height: 3;">
  </a>
    <a href="https://horizonrobotics.github.io/robot_lab/finegrasp/index.html" target="_blank" style="margin: 2px;">
    <img alt="Project" src="https://img.shields.io/badge/Project-Website-blue" style="display: inline-block; vertical-align: middle;"/>
  </a>
    <a href="https://github.com/HorizonRobotics/robo_orchard_lab/tree/master/projects/finegrasp_graspnet1b" target="_blank" style="margin: 2px;">
    <img alt="FineGrasp Code" src="https://img.shields.io/badge/FineGrasp_Code-Github-bule" style="display: inline-block; vertical-align: middle;"/>
  </a>
  </a>
    <a href="https://github.com/HorizonRobotics/robo_orchard_grasp_label" target="_blank" style="margin: 2px;">
    <img alt="Grasp Label Code" src="https://img.shields.io/badge/Grasp Label Code-Github-bule" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://arxiv.org/abs/2507.05978" target="_blank" style="margin: 2px;">
    <img alt="Paper" src="https://img.shields.io/badge/Paper-arXiv-red" style="display: inline-block; vertical-align: middle;"/>
  </a>
    <a href="https://huggingface.co/HorizonRobotics/Finegrasp" target="_blank" style="margin: 2px;">
    <img alt="Model" src="https://img.shields.io/badge/Model-HuggingFace-red" style="display: inline-block; vertical-align: middle;"/>
  </a>
  
</div>


## 1. Grasp performance in GraspNet-1Billion dataset.

| Method    | ckpt | Camera    | Seen (AP) | Similar (AP) | Novel (AP) | Average (AP) |
| --------- | ---- | --------- | --------- | ------------ | ---------- | ---------- |
| FineGrasp | finegrasp_pipeline/model.safetensors | Realsense | 71.67     | 62.83        | 27.40      | 53.97 |
| FineGrasp + CD | finegrasp_pipeline/model.safetensors |  Realsense | 73.71 | 64.56 | 28.14 | 55.47 |
| FineGrasp + Simulation Data | finegrasp_pipeline_sim/model.safetensors |  Realsense | 70.21 | 61.98 | 26.18 | 52.79 |


We also provide a grasp-based baseline for [Challenge Cup](https://developer.d-robotics.cc/tiaozhanbei-2025), please refer to this [code](https://github.com/HorizonRobotics/robo_orchard_lab/tree/master/projects/pick_place_agent)
> Notice: finegrasp_pipeline_sim/model.safetensors is trained for [Challenge Cup](https://developer.d-robotics.cc/tiaozhanbei-2025) RoboTwin simulation benchmark.


## 2. Example Infer Code
```Python
import os
import numpy as np
import scipy.io as scio
from PIL import Image
from robo_orchard_lab.models.finegrasp.processor import GraspInput
from huggingface_hub import snapshot_download
from robo_orchard_lab.inference import InferencePipelineMixin

file_path = snapshot_download(
    repo_id="HorizonRobotics/FineGrasp",
    allow_patterns=[
        "finegrasp_pipeline/**",
        "data_example/**"
    ],  
)

loaded_pipeline = InferencePipelineMixin.load(
    os.path.join(file_path, "finegrasp_pipeline")
)

rgb_image_path = os.path.join(file_path, "data_example/0000_rgb.png")
depth_image_path = os.path.join(file_path, "data_example/0000_depth.png")
intrinsic_file = os.path.join(file_path, "data_example/0000.mat")

depth_image = np.array(Image.open(depth_image_path), dtype=np.float32)
rgb_image = np.array(Image.open(rgb_image_path), dtype=np.float32) / 255.0
intrinsic_matrix = scio.loadmat(intrinsic_file)["intrinsic_matrix"]
workspace = [-1, 1, -1, 1, 0.0, 2.0]
depth_scale = 1000.0

input_data = GraspInput(
    rgb_image=rgb_image,
    depth_image=depth_image,
    depth_scale=depth_scale,
    intrinsic_matrix=intrinsic_matrix,
    workspace=workspace,
)

loaded_pipeline.to("cuda")
loaded_pipeline.model.eval()
output = loaded_pipeline(input_data)
print(f"Best grasp pose: {output.grasp_poses[0]}")

```

## Citation
```
@misc{du2025finegrasp,
    title={FineGrasp: Towards Robust Grasping for Delicate Objects}, 
    author={Yun Du and Mengao Zhao and Tianwei Lin and Yiwei Jin and Chaodong Huang and Zhizhong Su},
    year={2025},
    eprint={2507.05978},
    archivePrefix={arXiv},
    primaryClass={cs.RO},
    url={https://arxiv.org/abs/2507.05978}, 
}
```