agos_sim2real_geoaux_depthdrop

Diffusion policy for tilted plug-into-socket insertion on the AGOS sim2real hardware (Franka, parallel-jaw fingers, side wrist camera with ZED2i intrinsics at 360 x 640, noisy joint-space wrist-wrench observer, gravity on).

  • Inputs: wrist depth (minmax [-0.5, 0] m), 6-D wrist wrench (wrist_wrench_observer, z-scored), 3D-AGOS (pcsim4 point-cloud estimator image + vector). To = 1.
  • Output: 10-step action chunk, 9-D EEF-frame deltas (translation / 0.0003 m, rot6d).
  • Auxiliary: GeoAux head predicting the relative plug pose (weight 0.1).
  • Depth dropout: the normalized depth is zeroed when the plug is within 30 mm of its seated pose (p = 1, linear to 0 at 33 mm), so the contact phase runs on wrench + 3D-AGOS.
  • Data: yinongh/tilted_insertion_agos_sim2real_v1, 200 procedural assets (952001-952100, 952501-952600), 0.2 mm clearance, 22 440 trajectories; holdout = last normal + last recovery trajectory of every asset.
  • Training: 8 x A100, 64 per GPU, 145 000 steps, AdamW 3e-4 cosine, EMA; code Insertion-RL commit 5571262c (scripts/train_agos_sim2real_geoaux_depthdrop.sh, model depth_wrench_cnn_unet_agos_geoaux).

Files (depth_wrench_cnn_unet_agos_geoaux/20261114_134557/)

  • checkpoints/last.pt (145k, the evaluated one), checkpoints/best_fixed_noise.pt (25k, lowest holdout fixed-noise MSE)
  • resolved_config.yaml, normalization_stats.{json,npz}, metrics.jsonl, summary.json, sensitivity.json, gradient_reachability.json, gate_result.json, git_state.json, plots/, predictions/, dataset_manifest.json

Holdout: sampled-action MSE 0.0073 at 145k; fixed-noise MSE 0.0101 (25k) -> 0.0183 (145k).

Closed-loop evaluation (last.pt, depth dropped by the training rule)

Isaac Gym, sim2real hardware preset, protocol-v10 rollout and success (1000 steps, exec horizon 5, normal < 6 mm, axial < 15 mm, rotation < 0.35 rad modulo symmetry), 3 rounds x 12 envs per task:

split success
seen (5 trained assets) 122 / 180 = 67.8 % [60.6, 74.2]
unseen (5 new procedural assets) 129 / 180 = 71.7 % [64.7, 77.7]
same 10 shapes at 0.5 mm clearance 265 / 360 = 73.6 %

Most failures wedge in the bore with the gripper ~3.8 deg off the socket axis after reaching the mouth still rotating; zeroing the wrench input leaves closed-loop success unchanged. Details: docs/plan/agos_sim2real_geoaux_depthdrop.md §5 of the Insertion-RL repository.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Dataset used to train yinongh/agos_sim2real_geoaux_depthdrop