GeoAdapt w/o AGOS (depth + force diffusion policy)
Ablation of the AGOS insertion policy with the AGOS canonical plug/socket images removed. The
policy sees only the wrist depth image and the socket contact force; everything else matches
the AGOS model (agos_canonical_split_unet).
- Task:
AutoMateTaskAGOS, a parallel-jaw Franka with a wrist depth camera inserts a plug into a socket lying flat on the table (Isaac Gym). - Inputs: wrist depth 480x640 (1 frame), socket net contact force (3-D, 1 step).
- Output: 9-D end-effector-frame action chunk, horizon 10 (translation dims 0-2, rotation dims 3-8).
- Architecture: ImageNet ResNet-18 depth trunk (GroupNorm) + force MLP, shared by two diffusion
heads (translation, rotation), each with its own concat fusion and 1-D conditional U-Net. Both
heads see depth + force. DDPM training, 100 timesteps; DDIM 20-step inference. 51.3 M parameters.
See
model_config.yaml.
Training
- Data: 20,630 simulated demonstration trajectories (normal + recovery) on 242 procedurally generated plug/socket assets; 24 trajectories from 12 assets held out for validation.
- 145,000 steps, batch 512 (64 x 8 A100), AdamW lr 3e-4, cosine schedule with 2,000 warmup steps, EMA weights, seed 233. Steps 0-40,000 and 40,000-145,000 were two launches; the second resumed from the step-40,000 checkpoint (model, EMA, optimizer, LR schedule restored; fresh data order and noise RNG).
- Full config:
resolved_config.yaml; per-step log:metrics.jsonl; run summary:summary.json.
Checkpoints
| file | step | holdout fixed-noise MSE | holdout sampled-action MSE |
|---|---|---|---|
checkpoints/last.pt |
145,000 | 0.00996 | 0.0203 |
checkpoints/best_fixed_noise.pt |
85,000 | 0.00853 | 0.0204 |
MSEs are on normalized actions over the held-out windows. These are offline metrics only; no
closed-loop insertion success rate is reported here. Use the EMA weights (ema_state) for
inference; normalization statistics are stored inside each checkpoint and in
normalization_stats.{json,npz}.
Loading
The checkpoints are PyTorch payloads written by the GeoAdapt/AGOS training code
(policy/checkpoint.py: load_checkpoint, build_policy_from_checkpoint(payload, device, use_ema=True)).