agos_sim2real_geoaux_depthdrop
Diffusion policy for tilted plug-into-socket insertion on the AGOS sim2real hardware (Franka, parallel-jaw fingers, side wrist camera with ZED2i intrinsics at 360 x 640, noisy joint-space wrist-wrench observer, gravity on).
- Inputs: wrist depth (minmax [-0.5, 0] m), 6-D wrist wrench (
wrist_wrench_observer, z-scored), 3D-AGOS (pcsim4 point-cloud estimator image + vector). To = 1. - Output: 10-step action chunk, 9-D EEF-frame deltas (translation / 0.0003 m, rot6d).
- Auxiliary: GeoAux head predicting the relative plug pose (weight 0.1).
- Depth dropout: the normalized depth is zeroed when the plug is within 30 mm of its seated pose (p = 1, linear to 0 at 33 mm), so the contact phase runs on wrench + 3D-AGOS.
- Data: yinongh/tilted_insertion_agos_sim2real_v1, 200 procedural assets (952001-952100, 952501-952600), 0.2 mm clearance, 22 440 trajectories; holdout = last normal + last recovery trajectory of every asset.
- Training: 8 x A100, 64 per GPU, 145 000 steps, AdamW 3e-4 cosine, EMA; code
Insertion-RLcommit5571262c(scripts/train_agos_sim2real_geoaux_depthdrop.sh, modeldepth_wrench_cnn_unet_agos_geoaux).
Files (depth_wrench_cnn_unet_agos_geoaux/20261114_134557/)
checkpoints/last.pt(145k, the evaluated one),checkpoints/best_fixed_noise.pt(25k, lowest holdout fixed-noise MSE)resolved_config.yaml,normalization_stats.{json,npz},metrics.jsonl,summary.json,sensitivity.json,gradient_reachability.json,gate_result.json,git_state.json,plots/,predictions/,dataset_manifest.json
Holdout: sampled-action MSE 0.0073 at 145k; fixed-noise MSE 0.0101 (25k) -> 0.0183 (145k).
Closed-loop evaluation (last.pt, depth dropped by the training rule)
Isaac Gym, sim2real hardware preset, protocol-v10 rollout and success (1000 steps, exec horizon 5, normal < 6 mm, axial < 15 mm, rotation < 0.35 rad modulo symmetry), 3 rounds x 12 envs per task:
| split | success |
|---|---|
| seen (5 trained assets) | 122 / 180 = 67.8 % [60.6, 74.2] |
| unseen (5 new procedural assets) | 129 / 180 = 71.7 % [64.7, 77.7] |
| same 10 shapes at 0.5 mm clearance | 265 / 360 = 73.6 % |
Most failures wedge in the bore with the gripper ~3.8 deg off the socket axis after reaching the mouth still rotating;
zeroing the wrench input leaves closed-loop success unchanged. Details: docs/plan/agos_sim2real_geoaux_depthdrop.md
§5 of the Insertion-RL repository.