H5D joystick locomotion policies (SuperMimic distillation, sim2sim verified)

Three student policies for the H5D humanoid (30 joints, 50 Hz), distilled from a SuperMimic world-model tracker teacher into a joystick-conditioned controller, then verified in a realistic sim2sim proxy (vendor MJCF, explicit 1 kHz PD, torque clamp, noisy IMU, 4 / 20 ms action delay). Each iteration fixed one failure of the previous one.

file what changed score min falls / 90 notes
policy1_p1_clean.onnx clean clip set, stand<->crouch reference switches 0.680 0.639 0 crouch pitch 54 -> 32 deg, up-down falls 3 -> 0; stance still narrows
policy2_p6_near_polish.onnx + canonical stand target, low-lr polish 0.727 0.669 0 best stance / sit; in-place turn weak under low joint damping
policy3_p7_turn_polish_ema_2000.onnx + turn / side-step targets, EMA weights 0.719 0.655 0 every task works in every physics config; start here

Baselines on the same card: 0.579 (3 falls) and 0.533 (10 falls). Full table with per-task numbers in SCORECARD.txt (worst case over 3 scenes x 2 action delays, 15 cases each: stand 60 s, walk, run 2 m/s, low walk, low run, crouch 0.65 m, deep crouch 0.50 m, stand back up, run-stop, pushes, turn 0.8 rad/s, side step 0.4 m/s).

Interface

ONNX inputs obs_hist [B, 10, 102] (last 10 proprioceptive frames) and goal [B, 756], output action [B, 30]; target_q = default_q + action_scale * action, PD at the joints. Joystick fields of goal: goal[750:754] = (vx, vy, wz, pelvis_height), goal[754] = run flag, goal[755] = 1 (joystick mode). Standing height 0.84 m; crouch = height 0.65, deep crouch = 0.50. Drop-in replacement for the policy file of the h5d_sim2sim_v2 controller bundle (same mirror averaging, command slew and heading hold apply).

import onnxruntime as ort, numpy as np
sess = ort.InferenceSession("policy3_p7_turn_polish_ema_2000.onnx")
action = sess.run(None, {"obs_hist": obs_hist.astype(np.float32), "goal": goal.astype(np.float32)})[0]

*_student_*.pt are the PyTorch checkpoints the ONNX files were exported from (pi state dict, command bounds, standing height).

Known limits

  • Low running (1.5 m/s at 0.68 m pelvis) settles at 0.73 m: 79 s of such motion in 71,612 s of training data.
  • Side step reaches 0.24 of 0.40 m/s, in-place turn 0.3-0.5 of 0.8 rad/s.
  • Behaviours such as low walk flip between neighbouring checkpoints; these three were selected by the full scorecard, not taken as the last checkpoint.
  • Verified in a sim2sim proxy, not on hardware. Keep deploy gains as in the bundle's deploy_spec.json; a 0.7x stiffness margin breaks every policy, including the baselines.

Training data

Distilled from a tracker trained on BONES-SEED motion retargeted to H5D with the NVIDIA SOMA Retargeter (BONES-SEED licence, non-commercial). No motion data or robot description files are included here.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading