YAM Task Design & Approach (working doc)

Living doc of how the YAM task/motion pipeline works and the task list. Updated as I go.

Approach (current)

Robot/env: YAM bimanual (Template-YAM-Play-v0, Isaac Lab). Right arm active; left arm frozen (kinematic hold).
Planning: task-space PRM (yam_prm.py) → collision-free EEF polyline. Task steps are hand-scripted phases (not TAMP).
Execution (smooth): constant-speed arc-length follower (flow() in yam_grasp_prm.py) — the commanded EEF point glides along the whole path from the current pose, so segments blend continuously (no stop-and-go gap). Only stops where the gripper acts (close / release). + integral correction for diff-IK steady-state stall; grasp orientation held fixed (no twist). Adopted from RoboLab's CartesianIKPlanner.
Grasp: top-down, close-until-contact (stall detection) + lift by friction (no kinematic attach). Grasp height = table_top + object_height/2 (robust to USD origin). Gripper actuator tuned (damping 60, effort 40) so it clamps.
Assets: RobotWin (MIT) GLB → USD via robotwin_convert.py (convex-decomposition collision + rigid body baked). Source /home/horde/xiaotong/robotwin_assets/objects, USD out /home/horde/xiaotong/robotwin_assets/usd. Also license-clean primitives (box, cubes, cans) built in-script.

What grasps reliably

Uniform cylinders / straight cups / cans (primitive can, RobotWin cup). Grape ~60% (soft cluster). Tapered/glass bottles marginal. Short cubes marginal (stacking needs <2.5cm placement precision → hard).

Task list

Working (videos in outputs/tasks/): grape→box, cup→box, can→box (smooth), sort-2-cubes, RobotWin cup→box (smooth). Hard/parked: stack cubes (placement precision), glass tumbler (marginal grip).

Planned — RobotWin assets, distinct from RobotWin's own tasks

(RobotWin already has: beat_block_hammer, place_can_basket, place_object_basket, place_container_plate, place_bread_basket, put_bottles_dustbin, blocks_ranking_*, handover_block …)

id	task	why distinct
T1	RobotWin cup → bowl (real container)	not a box; different container than RobotWin's plate/basket
T2	RobotWin bottle → tray	tray container; bottle object
T3	hammer → bell (strike the bell)	RobotWin's hammer task hits a block; mine strikes a bell
T4	multi-object → basket (cup + block into one basket)	RobotWin places single object; mine batches two
T5	3-color cube sort (extend the working 2-color)	primitive, reliable

Delivered (5 task videos, `outputs/tasks/task*.mp4`)

RobotWin cup → box (real MIT asset, GLB→USD) ✅
DUAL-ARM: two cups picked in parallel ✅ (both arms, smooth LERP drive)
can → box with the smooth arc-length executor ✅
sort 2 cubes into color-matched bins ✅
grape → box ✅

Reliable objects: cup (RobotWin + straight cups), can, cubes. Hard/failed: hammer & bottle (elongated/tapered → grip slips), RobotWin bowl/tray as containers (curved shapes don't catch the object; primitive box is reliable), cube stacking (placement precision).

Open problems

Smoothness done; grasp reliability of soft/short/tapered objects; stacking precision (needs closed-loop/visual servo or a jig).
cuRobo would give better trajectories but needs a YAM URDF+spheres config (not built).