MikelRubi commited on
Commit
bb06bbd
·
verified ·
1 Parent(s): 89a898c

Card: orchestration, project page, combined datasets

Browse files
Files changed (1) hide show
  1. README.md +18 -10
README.md CHANGED
@@ -10,6 +10,7 @@ tags:
10
  - robotekin-hackathon-2026
11
  - pick
12
  datasets:
 
13
  - MikelRubi/so101-pick-cube_20261023_191626
14
  - MikelRubi/so101-pick-cube_20261023_193855
15
  - MikelRubi/so101-pick-cube_20261023_195654
@@ -113,14 +114,17 @@ If there is little robot time, try **v3_super 6K → v3_strict 2.4K → v3_1-2-p
113
  ## The system
114
 
115
  Built at the **Robotekin LeRobot Hackathon 2026** (23–25 October 2026, Robotory, Vitoria-Gasteiz). One SO-101 arm runs
116
- a small quality-control cell for a metal bracket that may or may not carry a pin:
 
 
117
 
118
  ```mermaid
119
  flowchart LR
120
- crate["Parts in the crate"] --> pick["1 · act-pick<br/>grasp the part according to its orientation,<br/>lift it, hold it up to the camera"]
 
121
  pick --> cls{"2 · pin-classifier<br/>front + wrist images"}
122
- cls -- "pin → GOOD" --> good["3a · act-up-down-good<br/>carry it to the green box, release"]
123
- cls -- "no pin → BAD" --> bad["3b · act-up-down-bad<br/>carry it to the white box, release"]
124
  ```
125
 
126
  | Stage | Repository | Model |
@@ -133,10 +137,14 @@ flowchart LR
133
  Picking is a separate policy because the bracket's geometry makes the right grasp depend on its orientation; the two
134
  sorting policies start where the pick policy ends (part held in front of the camera).
135
 
136
- **Orchestration.** No script that chains the four stages on the robot has been published. Each policy is run on its
137
- own; during data collection the operator looked at the part and chose which sorting demonstration to record. The
138
- start/end-pose contract between stages is written down in
139
- [`act-up-down-bad/HANDOFF.md`](https://huggingface.co/MikelRubi/act-up-down-bad/blob/main/HANDOFF.md).
 
 
 
 
140
 
141
  > **Naming note: `good` / `bad` describe the part, not the quality of a demonstration.**
142
  > GOOD = part **with** a pin → green box. BAD = part **without** a pin (defective) → white box.
@@ -151,8 +159,8 @@ start/end-pose contract between stages is written down in
151
  |---|---|---|
152
  | Arm | SO-101 follower, 6 joints: `shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper` (degrees; gripper 0–100) | dataset `robot_type: so_follower`, recording scripts |
153
  | Teleoperation | SO-101 leader arm driving the follower (leader/follower); `action` = leader position | recording scripts |
154
- | Cameras | Two USB RGB cameras: `front` (fixed, looking at the work area) and `wrist` (on the gripper). 640×480 at 30 fps; the first sessions of 23 October used 1280×720 at 10 fps | dataset `meta/info.json` |
155
- | Work cell | Crate with the parts, inspection position in front of the camera, green box (GOOD) and white box (BAD) | recordings, team |
156
  | Recording PC | Linux, arms on USB serial ports | team notes |
157
 
158
  Not documented yet: camera models and exact mounting positions, workspace dimensions and lighting, servo models,
 
10
  - robotekin-hackathon-2026
11
  - pick
12
  datasets:
13
+ - MikelRubi/robotekin-so101-pick
14
  - MikelRubi/so101-pick-cube_20261023_191626
15
  - MikelRubi/so101-pick-cube_20261023_193855
16
  - MikelRubi/so101-pick-cube_20261023_195654
 
114
  ## The system
115
 
116
  Built at the **Robotekin LeRobot Hackathon 2026** (23–25 October 2026, Robotory, Vitoria-Gasteiz). One SO-101 arm runs
117
+ a small quality-control cell for a metal bracket that may or may not carry a pin. **Project page:**
118
+ [`MikelRubi/robotekin-so101-pin-sorting`](https://huggingface.co/spaces/MikelRubi/robotekin-so101-pin-sorting).
119
+
120
 
121
  ```mermaid
122
  flowchart LR
123
+ order["Order (voice / text)<br/>intent model → sorting rule"] --> tray["Part lies in the tray"]
124
+ tray --> pick["1 · act-pick<br/>grasp the part according to its orientation,<br/>lift it, hold it up to the camera"]
125
  pick --> cls{"2 · pin-classifier<br/>front + wrist images"}
126
+ cls -- "rule: usually pin (GOOD)" --> good["3a · act-up-down-good<br/>carry it to the green box, release"]
127
+ cls -- "rule: usually no pin (BAD)" --> bad["3b · act-up-down-bad<br/>carry it to the white box, release"]
128
  ```
129
 
130
  | Stage | Repository | Model |
 
137
  Picking is a separate policy because the bracket's geometry makes the right grasp depend on its orientation; the two
138
  sorting policies start where the pick policy ends (part held in front of the camera).
139
 
140
+ **Orchestration.** The team's integrated demo, [`30_full_demo_app.py`](https://github.com/iker-azur/hackathon-robotekin/tree/main/demos/06_imitation_learning), runs the loop on the real follower:
141
+ an order (voice in Spanish, English or Basque, text, or Telegram) is read by the intent model
142
+ [`IkerAzurmendi/bin-intent-gliner2`](https://huggingface.co/IkerAzurmendi/bin-intent-gliner2); the arm waits until a
143
+ part lies still in the tray; the pick policy lifts it to the camera; the classifier averages 1.5 s of frames; the order
144
+ maps PIN / NO PIN to the green or the white box, and that box's policy carries the part there (`policy_routes.json`,
145
+ run through `run_policy.py`). In the usual order GOOD parts go to the green box and BAD parts to the white box. During
146
+ data collection the operator looked at the part and chose which sorting demonstration to record. The start/end-pose
147
+ contract between stages is in [`act-up-down-bad/HANDOFF.md`](https://huggingface.co/MikelRubi/act-up-down-bad/blob/main/HANDOFF.md).
148
 
149
  > **Naming note: `good` / `bad` describe the part, not the quality of a demonstration.**
150
  > GOOD = part **with** a pin → green box. BAD = part **without** a pin (defective) → white box.
 
159
  |---|---|---|
160
  | Arm | SO-101 follower, 6 joints: `shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper` (degrees; gripper 0–100) | dataset `robot_type: so_follower`, recording scripts |
161
  | Teleoperation | SO-101 leader arm driving the follower (leader/follower); `action` = leader position | recording scripts |
162
+ | Cameras | Two USB RGB cameras: `front` (fixed overhead camera looking down at the tray; the part is lifted towards it for inspection) and `wrist` (on the gripper). 640×480 at 30 fps; the first sessions of 23 October used 1280×720 at 10 fps | dataset `meta/info.json` |
163
+ | Work cell | Orange tray with the parts, inspection position in front of the camera, green (big) box and white (small) box | recordings, `tray_detect.py`, `demo_orders.py` |
164
  | Recording PC | Linux, arms on USB serial ports | team notes |
165
 
166
  Not documented yet: camera models and exact mounting positions, workspace dimensions and lighting, servo models,