Cyclops Reorient — Reward Weight Ablations
Default Reward Parameters
| Index | Name | Default Weight |
|---|---|---|
| 0 | orientation_alignment | 30.0 |
| 1 | hand_pose | 0.2 |
| 2 | action_rate | -0.5 |
| 3 | torque | 0 |
| 4 | tip_slide | -0.3 |
| 5 | cage_escape | -100.0 |
| 6 | finger_collision | -1.0 |
| 7 | hold_escalation | 11.4 |
| 8 | palm_detach | 0.5 |
| 9 | contact_count | 0.0 |
Success threshold: 0.2 rad (~11.5°) | Hold steps: 5 | Goal switch delay: 20 steps
Ablation Runs
Original Study (experiment/without_fingerpad_tip_collision_geom branch)
Adjusted config/cyclops_finger/rsl_rl/ppo.py: entropy_coef 0.001 → 0.01, min_std 0.2 → 0.4, init_std 0.5 → 0.6.
| Run | SLURM Job | orientation_alignment | hand_pose | action_rate | torque | tip_slide | cage_escape | finger_collision | hold_escalation | palm_detach | contact_count | Status |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 01 — default | 29274656 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 02 — action_rate -0.3 | 29275079 | 30.0 | 0.2 | -0.3 | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 03 — orientation_alignment 60.0 | 29275102 | 60.0 | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 04 — hold_escalation 20.0 | 29275119 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | -100.0 | -1.0 | 20.0 | 0.5 | 0.0 | |
| 05 — tip_slide -0.1 | 29275142 | 30.0 | 0.2 | -0.5 | 0 | -0.1 | -100.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 06 — cage_escape -50.0 | 29275352 | 30.0 | 0.2 | -0.5 | 0 | -0.3 | -50.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 07 — cage_escape -20.0 orientation_alignment 100.0 (continued: 29402762, 29528610) | 29274710 | 100.0 | 0.2 | -0.5 | 0 | -0.3 | -20.0 | -1.0 | 11.4 | 0.5 | 0.0 | COMPLETED (final: model_17497.pt) |
| 08 — cage_escape -50.0 orientation_alignment 60.0 | 29275361 | 60.0 | 0.2 | -0.5 | 0 | -0.3 | -50.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 09 — cage_escape -10.0 orientation_alignment 100.0 | 29354503 | 100.0 | 0.2 | -0.5 | 0 | -0.3 | -10.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 10 — cage_escape -5.0 orientation_alignment 100.0 | 29354510 | 100.0 | 0.2 | -0.5 | 0 | -0.3 | -5.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 11 — cage_escape -20.0 orientation_alignment 150.0 (continued: 29555347) | 29354667 | 150.0 | 0.2 | -0.5 | 0 | -0.3 | -20.0 | -1.0 | 11.4 | 0.5 | 0.0 | TIMEOUT (last: model_11950.pt) |
| 15 — cage_escape -10.0 orientation_alignment 150.0 | 29555350 | 150.0 | 0.2 | -0.5 | 0 | -0.3 | -10.0 | -1.0 | 11.4 | 0.5 | 0.0 | TIMEOUT (last: model_4500.pt) |
New Study — Fingertip Contact Target Reward (main branch)
UNDESIRED_OBJECT_CONTACT_BODIES modified to include proximal_link_(center|left|right). touch_site_fingertip changed to a sphere on the sensor surface. Removed proximal_link_(center|left|right) and distal_link_(center|left|right) from DR_CONTACT_GROUP_A_GEOMS. Added base_gripper_col to SELF_COLLISION_GEOMS.
Added Fingertip_Contact_Target_Reward. Replaced Distal_Link with Fingerpad_Sensor in TIP_Collision_Geoms.
| Run | SLURM Job | orientation_alignment | hand_pose | action_rate | torque | tip_slide | cage_escape | finger_collision | hold_escalation | palm_detach | contact_count | Status |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 12 — cage_escape -20.0 orientation_alignment 100.0 | 29374461 | 100.0 | 0.2 | -0.5 | 0 | -0.3 | -20.0 | -1.0 | 11.4 | 0.5 | 0.0 | |
| 13 — cage_escape -10.0 orientation_alignment 100.0 | 29374463 | 100.0 | 0.2 | -0.5 | 0 | -0.3 | -10.0 | -1.0 | 11.4 | 0.5 | 0.0 | learning plateaued |
| 14 — cage_escape -10.0 orientation_alignment 100.0 contact_count 2 | 29374469 | 100.0 | 0.2 | -0.5 | 0 | -0.3 | -10.0 | -1.0 | 11.4 | 0.5 | 2.0 | |
| 16 — cage_escape -20.0 orientation_alignment 150.0 | 29567528 | 150.0 | 0.2 | -0.5 | 0 | -0.3 | -20.0 | -1.0 | 11.4 | 0.5 | 0.0 | FAILED (exit 127: missing -- prefix on CLI overrides, never started) |
New Study — Success Curriculum Scaling with orientation_alignment and cage_escape (experiment/curriculum_reorient branch)
orientation_alignment and cage_escape rewards now scale with success_curriculum (curriculum_min=0.3). CageEscapePenalty supports curriculum_term/curriculum_min without disrupting the cage_drop termination counter. reorient_success_curriculum now uses graduated partial credit (count_threshold=2, was 3) instead of a hard binary bar.
| Run | SLURM Job | orientation_alignment | hand_pose | action_rate | torque | tip_slide | cage_escape | finger_collision | hold_escalation | palm_detach | contact_count | Status |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 17 — cage_escape -20.0 orientation_alignment 100.0 | 29580281 | 100.0 | 0.2 | -0.5 | 0 | -0.3 | -20.0 | -1.0 | 11.4 | 0.5 | 0.0 | RUNNING |
| 18 — cage_escape -20.0 orientation_alignment 150.0 | 29580291 | 150.0 | 0.2 | -0.5 | 0 | -0.3 | -20.0 | -1.0 | 11.4 | 0.5 | 0.0 | RUNNING |
TensorBoard logs and checkpoints for all runs are hosted at huggingface.co/loisliowcsiro/cyclops_runs.