| # YAM Task Design & Approach (working doc) |
|
|
| Living doc of how the YAM task/motion pipeline works and the task list. Updated as I go. |
|
|
| ## Approach (current) |
| - **Robot/env:** YAM bimanual (`Template-YAM-Play-v0`, Isaac Lab). Right arm active; left arm frozen (kinematic hold). |
| - **Planning:** task-space **PRM** (`yam_prm.py`) → collision-free EEF polyline. Task steps are hand-scripted phases (not TAMP). |
| - **Execution (smooth):** constant-speed **arc-length follower** (`flow()` in `yam_grasp_prm.py`) — the commanded EEF point glides along the whole path from the current pose, so segments blend continuously (no stop-and-go gap). Only stops where the gripper acts (close / release). + integral correction for diff-IK steady-state stall; grasp orientation held fixed (no twist). Adopted from RoboLab's `CartesianIKPlanner`. |
| - **Grasp:** top-down, close-until-contact (stall detection) + lift by friction (no kinematic attach). Grasp height = table_top + object_height/2 (robust to USD origin). Gripper actuator tuned (damping 60, effort 40) so it clamps. |
| - **Assets:** **RobotWin (MIT)** GLB → USD via `robotwin_convert.py` (convex-decomposition collision + rigid body baked). Source `/home/horde/xiaotong/robotwin_assets/objects`, USD out `/home/horde/xiaotong/robotwin_assets/usd`. Also license-clean **primitives** (box, cubes, cans) built in-script. |
|
|
| ## What grasps reliably |
| - Uniform cylinders / straight cups / cans (primitive can, RobotWin cup). Grape ~60% (soft cluster). Tapered/glass bottles marginal. Short cubes marginal (stacking needs <2.5cm placement precision → hard). |
|
|
| ## Task list |
| Working (videos in `outputs/tasks/`): grape→box, cup→box, can→box (smooth), sort-2-cubes, RobotWin cup→box (smooth). |
| Hard/parked: stack cubes (placement precision), glass tumbler (marginal grip). |
|
|
| ### Planned — RobotWin assets, distinct from RobotWin's own tasks |
| (RobotWin already has: beat_block_hammer, place_can_basket, place_object_basket, place_container_plate, place_bread_basket, put_bottles_dustbin, blocks_ranking_*, handover_block …) |
| | id | task | why distinct | |
| |----|------|--------------| |
| | T1 | RobotWin cup → **bowl** (real container) | not a box; different container than RobotWin's plate/basket | |
| | T2 | RobotWin bottle → **tray** | tray container; bottle object | |
| | T3 | **hammer → bell** (strike the bell) | RobotWin's hammer task hits a *block*; mine strikes a *bell* | |
| | T4 | multi-object → basket (cup + block into one basket) | RobotWin places single object; mine batches two | |
| | T5 | 3-color cube **sort** (extend the working 2-color) | primitive, reliable | |
|
|
| ## Delivered (5 task videos, `outputs/tasks/task*.mp4`) |
| 1. **RobotWin cup → box** (real MIT asset, GLB→USD) ✅ |
| 2. **DUAL-ARM: two cups picked in parallel** ✅ (both arms, smooth LERP drive) |
| 3. **can → box** with the smooth arc-length executor ✅ |
| 4. **sort 2 cubes** into color-matched bins ✅ |
| 5. **grape → box** ✅ |
| |
| Reliable objects: cup (RobotWin + straight cups), can, cubes. Hard/failed: hammer & bottle (elongated/tapered → grip slips), RobotWin bowl/tray as containers (curved shapes don't catch the object; primitive box is reliable), cube stacking (placement precision). |
| |
| ## Open problems |
| - Smoothness done; grasp reliability of soft/short/tapered objects; stacking precision (needs closed-loop/visual servo or a jig). |
| - cuRobo would give better trajectories but needs a YAM URDF+spheres config (not built). |
| |
