EE4705 Project 1.3: learned grasp policies (bonus)
Learned right-arm grasp policies for a simulated Unitree G1 (MuJoCo, weld attachment). They are used behind the project's
unchanged grasp skill interface: LearnedGraspSkill in executor/learned_grasp.py of the course repository.
From the post-APPROACH pose the policy runs closed-loop at 10 Hz. It observes the head camera (224×224 RGB), the 7 arm
joint angles and the target position in the robot base frame, and outputs the next 7-D arm joint setpoint. The
executor then attaches once and applies its usual post-condition checks.
| Folder | Policy | Training data | Checkpoint |
|---|---|---|---|
act_bottle_best/ |
ACT (lerobot 0.6.1): chunk 50, 25 executed steps, ResNet-18 | 2,000 scripted stone/cube demos (±10 cm around the nominal pose, any yaw, distractor in 50 %) + 503 scripted bottle demos | step 10k, selected on stone/cube + bottle validation episodes |
diffusion_best/ |
Diffusion Policy (lerobot 0.6.1): horizon 32, 8 executed steps, 2 observation steps, DDIM 10 | 2,000 scripted stone/cube demos (±10 cm), no bottle | step 35k, selected on 20 validation episodes |
Each folder contains the lerobot config.json, model.safetensors, the pre/post-processor files (normalisation
statistics) and train_config.json (the training arguments; seed 1000).
Results (full executor, 30 episodes per cell, same scenes for every policy)
Grasp success = the first GRASP holds the intended object. C1 in-distribution, C2 position shift ±20 cm, C3 bottle, C4 distractor within 3 cm. There were 0 false success claims and 0 undetected failures in every run.
| Grasp | C1 | C2 | C3 (bottle) | C4 | final50, 50 trials, manipulation mode |
|---|---|---|---|---|---|
| Scripted skill | 29/30 | 30/30 | 29/30 | 30/30 | 48/50 |
| ACT, 2k stone/cube demos | 29/30 | 26/30 | 0/30 | 30/30 | 38/50 |
ACT + bottle demos (act_bottle_best) |
29/30 | 27/30 | 21/30 (26/30 tasks) | 30/30 | 47/50 |
Diffusion Policy (diffusion_best) |
29/30 | 27/30 | 0/30 | 30/30 | 38/50 |
The shared C1 miss is one trial that fails before any grasp. Full tables, ablations and the learned PLACE policy
are in docs/bonus/RESULTS.md of the course repository.
Use
From the course repository (branch e2e):
EE4705_GRASP_POLICY=act EE4705_GRASP_CKPT=hf://jiamo0912/ee4705-learned-grasp/act_bottle_best \
python -m eval.runner --mode manipulation --trials eval/trials/student_c_v2
Use EE4705_GRASP_POLICY=diffusion with .../diffusion_best for the Diffusion Policy. hf:// paths are downloaded
with huggingface_hub.snapshot_download into runs/hf/.