microduck-arena-lab / docs /SPRINT_GUIDE.md
Kyry10's picture
Microduck Arena Lab: selected policies, reproducible guides and real simulation films
dc9a7f8 verified
|
Raw History Blame Contribute Delete
7.19 kB

Sprint: from a licensed actor to 1.084 s

Our best selected sprint policy completes2 m in a 1.084 s five-seed mean,5/5 finishes. It is D1 launchboost checkpoint 900. The official published entry remains a different checkpoint,2Brace-adapt750 at1.096 s. The0.988 s target was not reached.

Exact selected result

Seed Timed finish
1 1.08 s
2 1.06 s
3 1.10 s
4 1.10 s
5 1.08 s

ONNX SHA256: 25bfd91009f56fcff2fdf8326a56995cba3737759e0e1d681f71ed2c639c9ce4. Training commit:ad8c51e604cd73ceb7c58557905d640c0ee0a419. Local evaluation run:01M4T4N650MJSRNHEZW8M6. Full records are in best-score.json.

The featured clip shows seed 5's1.08 s finish, not the fastest seed 2. Its title reports the1.084 s mean separately. The timed mean implies an effective2/1.084≈1.845 m/s; that is a course-average measure, not the instantaneous peak speed or a hardware claim.

Baselines and the zero-training attempt

Policy Mean Finishes How it was used
Pollen sprint reference 4.892 s 5/5 Benchmark only
Hannes running 1.192 s 5/5 Apache-2.0 warm-start actor
Selected D1 1.084 s 5/5 Our adapted policy

The improvement over the licensed parent is about 9.1%. Hannes' gait already accelerated far beyond the Arena's maximum commanded speed, so we first tried a graph-level command override instead of training. The wrapper writes rawvx before the actor's existing normalizer and retains input/output names,61 observations and14 actions. We tested 1.2,1.6,2.0,2.2,2.4,2.6 and forced lateral/yaw commands to zero. There was no useful speed gain; we retained rawvx2.6 for consistency during training and export.

The wrapper is scripts/override_command.py; the sweep is scripts/sweep_command_override.py in the experiment overlay. This is not input normalizer replacement: the original mean and denominator remain intact.

Race environment

The course is2 m long,0.8 m wide, with0.15 m walls. The robot starts from STAND with±0.01 rad joint noise and±2°yaw. The policy runs for0.5 s before the timer starts. Finish progress is trunk world-x movement from the settle-end origin. Tilt beyond60°held0.3 s falls; fall/timeout scores50 s. Nominal BAM uses7.35 V/drop0.1,3–6substep command delay and a one-control-step joint-velocity observation lag.

Our3 s episodes include the run-up. Actor observations stay61D; the critic receives privileged position/progress, true velocity, time and tilt-hold. We removed pushes, wide mass/friction/voltage randomization and strong sim2real/style terms. Core rewards: uncapped forward trunk velocity, finish bonus, large fall penalty, light yaw/lateral cost and weak action-rate cost. Strong pose/slip/air-time/clearance shaping was removed.

Later code repairs the fall hold so settle steps do not count. The selected D1policy predates that repair; selected historical overlays and current aligned overlays are both provided. Do not confuse current source with the code that originally made a checkpoint.

Actor import and checkpoint lineage

Hannes running ONNX + exact frozen normalizer
  → 2B race adaptation: fresh critic/optimizer, low LR/std
      → checkpoint750:1.096s5/5
          → B3: full PPO resume, lower LR, small fixed std
              → checkpoint800:1.092s5/5
                  → D1: full PPO resume, early reward2.5→5
                      → checkpoint900:1.084s5/5

The import reconstructs the[512,256,128,14] actor MLP and preserves ONNX normalization. Torch/ONNX parity is checked before training. External ONNX weights get a fresh critic and optimizer; our own checkpoint continuations keep actor, critic and Adam moments. The intended LR is set after loading, since optimizer load otherwise restores the old LR.

D1used1024envs,LR3.3333333333e-6,fixedstd0.05,forwardweight5,early-forwardweight5, action-rateweight−0.01 and export/score every 50 iterations. Early reward covers the free settle and first0.3 timed seconds. The exact historical command and configs are retained in records/sprint-2m/selected-run/.

# Historical D1 shape: parent is B3model800, not D1model900.
python scripts/finetune_sprint.py \
  --run-name d1_launchboost \
  --parent-checkpoint "$MICRODUCK_ROOT/best/phase3_b3_polish/checkpoint.pt" \
  --full-resume --num-envs 1024 --iterations 1001 \
  --lr 3.3333333333333333e-6 --std 0.05 --fixed-std \
  --early-forward-weight 5 --save-interval 50 \
  --target-score 0.90 --stage autonomous

Run it from the workspace layout described in LOCAL_WORKSPACE, with the correct checkout on PYTHONPATH. That target-score value was an automatic training stop, not the actual result. Reproducing a training command does not guarantee identical learned parameters; downloading and scoring the selected ONNX reproduces the policy.

Diagnosis: where time was lost

The detailed diagnosis of the nearby B3best800 policy measured, across seeds:

  • settle-end forward speed1.34–1.38 m/s; -100 ms peak forward speed2.04–2.10 m/s;
  • time to95%of peak0.60–0.80 s after settle;
  • roughly 6.7distinct touchdown events/s and0.28 m trunk advance/event;
  • no timed wall contact.

The short rollout's same-foot cycle frequency was3.25–3.34 Hz with0.56–0.58 m stride. These are measured definitions, not interchangeable step/cycle labels. Slow launch and late peak speed motivated D1's early-forward reward, which gained0.008 s over B3. The target2/0.988≈2.024 m/s course average still requires a faster or earlier fast gait.

What did not work

Nearby checkpoints varied sharply:2B500 had only 2/5 finishes, while750 had5/5. Continued safe/speed branches did not reliably retain that peak. A target-speed curriculum and scratch discovery attempts did not yield a selected faster policy in this snapshot.

Later root experiments included stronger early reward, increased exploration, an Arena fall-clock repair, a signed quadratic velocity reward and a larger PPO LR. They were still being evaluated when this documentation snapshot was assembled; the selected policy remained D1 model 900.

We also tried CPU-only ONNX searches around D1:

Single knob Outcome
Leg action-amplitude gain gain1.0 remained best; larger gains often fell
B3→D1 actor interpolation/extrapolation D1alpha1.0 remained best
Raw joint-velocity feedback scaling gain1.0 remained best; most alternatives lost reliability
Final actor leg-bias coordinate search 61 scores, no accepted improvement over1.084 s

This evidence argues against more undirected static trims around the same gait. Change the training incentive or search a different gait, and continue choosing by exact Arena score. Only 5/5 results count as reliable bests.

Score and record

uv run --group workshop arena score path/to/policy.onnx \
  --event sprint-2m --store verification-store

For repeatable media, use tools/render_arena.py with --event sprint-2 m --seed 5 --command .2 --mean-score 1.084. It captures actual Arena state and replays it in a visual-only mesh model. See WEBSITE for CPU rendering and video integration.