H2R box loco-manipulation policies for Unitree G1 (R15, P23)
Residual policies on top of frozen SONIC (NVIDIA GR00T-WholeBodyControl) for walking up to a box, picking it
up, carrying and placing it on a Unitree G1 with rubber hands (R15) or Dex3 hands (P23). Trained with PPO in Isaac Lab
on retargeted OMOMO largebox motions (paired clean / corrupted references) only.
| folder | policy | box input | notes |
|---|---|---|---|
r15/ |
output residual over frozen SONIC, rubber hands | box estimator (deployable: virtual camera + odometry, no simulator truth) | step 15000 |
p23/ |
output residual over frozen SONIC, Dex3 hands | simulator box pose (privileged) | step 15000 |
Each folder: model_step_015000.pt (policy, value and optimizer state; includes the frozen SONIC weights) and the
config.yaml it was trained with. Load with the GR00T-WholeBodyControl fork used for training
(scripts_sonic_render/eval/run_eval.py --checkpoint <folder>/model_step_015000.pt ..., which reads the
config.yaml next to the checkpoint).
Reference results (no steering)
HOI2ROI feasible43 (43 out-of-distribution pick-carry-place cases × 8 episodes), full-size largebox:
| policy | pickup | success (pickup and box ≤ 30 cm from goal) |
|---|---|---|
| R15, box estimator (emulator) | 29.4% | 20.1% |
| R15, real camera pipeline (sim) | 29.7% | 20.9% |
| P23 | 20.3% | 11.3% |
Caveat: the HOI2ROI motions were generated for a box scaled ×0.76 (~25 cm); these numbers use the full-size box. R15 is box-size specific (trained on one box only).
License
Model weights are derived from NVIDIA SONIC (GR00T-WholeBodyControl) and are distributed under the NVIDIA Open Model License. Training motions derive from the OMOMO dataset; no motion data is included here.