Instructions to use MikelRubi/act-pick with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use MikelRubi/act-pick with LeRobot:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Card: orchestration, project page, combined datasets
Browse files
README.md
CHANGED
|
@@ -10,6 +10,7 @@ tags:
|
|
| 10 |
- robotekin-hackathon-2026
|
| 11 |
- pick
|
| 12 |
datasets:
|
|
|
|
| 13 |
- MikelRubi/so101-pick-cube_20261023_191626
|
| 14 |
- MikelRubi/so101-pick-cube_20261023_193855
|
| 15 |
- MikelRubi/so101-pick-cube_20261023_195654
|
|
@@ -113,14 +114,17 @@ If there is little robot time, try **v3_super 6K → v3_strict 2.4K → v3_1-2-p
|
|
| 113 |
## The system
|
| 114 |
|
| 115 |
Built at the **Robotekin LeRobot Hackathon 2026** (23–25 October 2026, Robotory, Vitoria-Gasteiz). One SO-101 arm runs
|
| 116 |
-
a small quality-control cell for a metal bracket that may or may not carry a pin:
|
|
|
|
|
|
|
| 117 |
|
| 118 |
```mermaid
|
| 119 |
flowchart LR
|
| 120 |
-
|
|
|
|
| 121 |
pick --> cls{"2 · pin-classifier<br/>front + wrist images"}
|
| 122 |
-
cls -- "
|
| 123 |
-
cls -- "no pin
|
| 124 |
```
|
| 125 |
|
| 126 |
| Stage | Repository | Model |
|
|
@@ -133,10 +137,14 @@ flowchart LR
|
|
| 133 |
Picking is a separate policy because the bracket's geometry makes the right grasp depend on its orientation; the two
|
| 134 |
sorting policies start where the pick policy ends (part held in front of the camera).
|
| 135 |
|
| 136 |
-
**Orchestration.**
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 140 |
|
| 141 |
> **Naming note: `good` / `bad` describe the part, not the quality of a demonstration.**
|
| 142 |
> GOOD = part **with** a pin → green box. BAD = part **without** a pin (defective) → white box.
|
|
@@ -151,8 +159,8 @@ start/end-pose contract between stages is written down in
|
|
| 151 |
|---|---|---|
|
| 152 |
| Arm | SO-101 follower, 6 joints: `shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper` (degrees; gripper 0–100) | dataset `robot_type: so_follower`, recording scripts |
|
| 153 |
| Teleoperation | SO-101 leader arm driving the follower (leader/follower); `action` = leader position | recording scripts |
|
| 154 |
-
| Cameras | Two USB RGB cameras: `front` (fixed
|
| 155 |
-
| Work cell |
|
| 156 |
| Recording PC | Linux, arms on USB serial ports | team notes |
|
| 157 |
|
| 158 |
Not documented yet: camera models and exact mounting positions, workspace dimensions and lighting, servo models,
|
|
|
|
| 10 |
- robotekin-hackathon-2026
|
| 11 |
- pick
|
| 12 |
datasets:
|
| 13 |
+
- MikelRubi/robotekin-so101-pick
|
| 14 |
- MikelRubi/so101-pick-cube_20261023_191626
|
| 15 |
- MikelRubi/so101-pick-cube_20261023_193855
|
| 16 |
- MikelRubi/so101-pick-cube_20261023_195654
|
|
|
|
| 114 |
## The system
|
| 115 |
|
| 116 |
Built at the **Robotekin LeRobot Hackathon 2026** (23–25 October 2026, Robotory, Vitoria-Gasteiz). One SO-101 arm runs
|
| 117 |
+
a small quality-control cell for a metal bracket that may or may not carry a pin. **Project page:**
|
| 118 |
+
[`MikelRubi/robotekin-so101-pin-sorting`](https://huggingface.co/spaces/MikelRubi/robotekin-so101-pin-sorting).
|
| 119 |
+
|
| 120 |
|
| 121 |
```mermaid
|
| 122 |
flowchart LR
|
| 123 |
+
order["Order (voice / text)<br/>intent model → sorting rule"] --> tray["Part lies in the tray"]
|
| 124 |
+
tray --> pick["1 · act-pick<br/>grasp the part according to its orientation,<br/>lift it, hold it up to the camera"]
|
| 125 |
pick --> cls{"2 · pin-classifier<br/>front + wrist images"}
|
| 126 |
+
cls -- "rule: usually pin (GOOD)" --> good["3a · act-up-down-good<br/>carry it to the green box, release"]
|
| 127 |
+
cls -- "rule: usually no pin (BAD)" --> bad["3b · act-up-down-bad<br/>carry it to the white box, release"]
|
| 128 |
```
|
| 129 |
|
| 130 |
| Stage | Repository | Model |
|
|
|
|
| 137 |
Picking is a separate policy because the bracket's geometry makes the right grasp depend on its orientation; the two
|
| 138 |
sorting policies start where the pick policy ends (part held in front of the camera).
|
| 139 |
|
| 140 |
+
**Orchestration.** The team's integrated demo, [`30_full_demo_app.py`](https://github.com/iker-azur/hackathon-robotekin/tree/main/demos/06_imitation_learning), runs the loop on the real follower:
|
| 141 |
+
an order (voice in Spanish, English or Basque, text, or Telegram) is read by the intent model
|
| 142 |
+
[`IkerAzurmendi/bin-intent-gliner2`](https://huggingface.co/IkerAzurmendi/bin-intent-gliner2); the arm waits until a
|
| 143 |
+
part lies still in the tray; the pick policy lifts it to the camera; the classifier averages 1.5 s of frames; the order
|
| 144 |
+
maps PIN / NO PIN to the green or the white box, and that box's policy carries the part there (`policy_routes.json`,
|
| 145 |
+
run through `run_policy.py`). In the usual order GOOD parts go to the green box and BAD parts to the white box. During
|
| 146 |
+
data collection the operator looked at the part and chose which sorting demonstration to record. The start/end-pose
|
| 147 |
+
contract between stages is in [`act-up-down-bad/HANDOFF.md`](https://huggingface.co/MikelRubi/act-up-down-bad/blob/main/HANDOFF.md).
|
| 148 |
|
| 149 |
> **Naming note: `good` / `bad` describe the part, not the quality of a demonstration.**
|
| 150 |
> GOOD = part **with** a pin → green box. BAD = part **without** a pin (defective) → white box.
|
|
|
|
| 159 |
|---|---|---|
|
| 160 |
| Arm | SO-101 follower, 6 joints: `shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper` (degrees; gripper 0–100) | dataset `robot_type: so_follower`, recording scripts |
|
| 161 |
| Teleoperation | SO-101 leader arm driving the follower (leader/follower); `action` = leader position | recording scripts |
|
| 162 |
+
| Cameras | Two USB RGB cameras: `front` (fixed overhead camera looking down at the tray; the part is lifted towards it for inspection) and `wrist` (on the gripper). 640×480 at 30 fps; the first sessions of 23 October used 1280×720 at 10 fps | dataset `meta/info.json` |
|
| 163 |
+
| Work cell | Orange tray with the parts, inspection position in front of the camera, green (big) box and white (small) box | recordings, `tray_detect.py`, `demo_orders.py` |
|
| 164 |
| Recording PC | Linux, arms on USB serial ports | team notes |
|
| 165 |
|
| 166 |
Not documented yet: camera models and exact mounting positions, workspace dimensions and lighting, servo models,
|