jrubiosainz/microduck-game-roller

MICRODUCK GAME roller

HannesVonEssen's microduck-running policy (Apache-2.0) fine-tuned with PPO on a CPU inside the Arena's own simulator, on roller skates: 1.30 s on the official seeds, 200/200 finishes and 0 falls on seeds 1-200. Ignores the command; simulation only.

An entry for the 2 m Roller Sprint of the Microduck Arena. Simulation only — see below.

What it is

A fine-tune of HannesVonEssen/microduck-running (revision 9f45af1f, policy.onnx sha256 007707dd7779…), HannesVonEssen's forward-running policy for the stock MicroDuck on flat ground (iteration 12,195, Apache-2.0), trained with Vottivott/microduck-playground@828d950 on pollen-robotics/microduck_rl@d424a0c. The network has the base's own shape (61 → 512 → 256 → 128 → 14, ELU) and keeps its input normalizer frozen; its weights were trained further with PPO on a CPU, inside the Arena's own runner and physics (MuJoCo 3.10.0, runner version 4), on this event's course:

• chain: HannesVonEssen's microduck-running policy (iteration 12,195) -> 1475 PPO iterations on roller-sprint-2m (run hr1, whose first 25 iterations trained only a fresh critic, up to iteration 499; run hr3, branched from hr1's iteration-499 checkpoint with its critic and optimiser state, with a fall penalty of 50 instead of 3 and a new random seed, whose iterations 500 to 524 trained only the critic); internal forward command 1.4 m/s throughout
• selection: checkpoint it01475, picked among twenty-seven checkpoints of four roller runs (hr1, hr3, ro1 and ro2), all run on the 200 seeds and noise rows of the table below: 1.304 s on the official seeds 1–5 in the trainer's check (after it also finished 20 held-out seeds, 1001–1020, with no fall), the lowest official time of the twenty-seven once the ones that fell are set aside (hr1 it1925 at 1.252 s, which fell on three of seeds 1–200; hr1 it1825 at 1.268 s, which fell on three of seeds 1–200; hr1 it1575 at 1.288 s, which fell on two of seeds 1–200 and fell three times in the noise rows; hr3 it1825 at 1.288 s, which fell on five of seeds 1–200; hr3 it1875 at 1.288 s, which fell on two of seeds 1–200), the lowest score on seeds 1–200 and no fall in any row;
• reward: forward progress while the race clock runs, small penalties for sideways speed, yaw rate, drifting toward the walls and falling; in run hr3 the fall penalty was raised from 3 to 50; in runs hr1 and hr3 an episode also ends when the duck crosses the line, as the Arena's run does, with a one-off bonus equal to the reward of running at 1.5 m/s for every step left before the 3.7 s horizon; 96 parallel copies of the course in runs hr1 and hr3, 64 control steps each per iteration, the Arena's own start perturbation;
• command: a fixed internal command (forward speed 1.4 m/s, zero lateral speed, yaw rate, head and body) is written into the 13 command slots of the observation inside the graph, so the commanded twist is ignored.

No GPU, no new data, no paid compute.

Results

local run, Arena runner	score (non-finish = 40 s)	mean of finishers	finished
seeds 1–5 (the official set)	1.304 s	1.304 s (worst 1.32)	5/5
seeds 1-200	1.327 s	1.327 s (p90 1.38, worst 1.48)	200/200, 0 falls
seeds 1–5 with observation noise σ = 1e-04, 4 runs per seed	1.304–1.336 s (range)	—	20/20
seeds 1–5 with observation noise σ = 1e-03, 4 runs per seed	1.316–1.324 s (range)	—	20/20

The noise rows add Gaussian noise to every observation the policy sees: a stand-in for the float-rounding differences of another machine, which a chaotic gait amplifies.

arena score --event roller-sprint-2m on this file (every speed of its sweep gives the same time, since the command is ignored; the sweep then reports the lowest):

sweep 18 commanded speeds 0.30–2.00 m/s
   0.30 m/s  1.32s
   0.40 m/s  1.32s
   0.50 m/s  1.32s
   0.60 m/s  1.32s
   0.70 m/s  1.32s
   0.80 m/s  1.32s
   0.90 m/s  1.32s
   1.00 m/s  1.32s
   1.10 m/s  1.32s
   1.20 m/s  1.32s
   1.30 m/s  1.32s
   1.40 m/s  1.32s
   1.50 m/s  1.32s
   1.60 m/s  1.32s
   1.70 m/s  1.32s
   1.80 m/s  1.32s
   1.90 m/s  1.32s
   2.00 m/s  1.32s

scored at 0.30 m/s
seed 1: finish   1.32s
seed 2: finish   1.32s
seed 3: finish   1.30s
seed 4: finish   1.28s
seed 5: finish   1.30s

score 1.304s · 5/5 finished · replay from seed 5

Local times are indicative: the Arena's machine can differ by a few hundredths of a second.

History

This repo's previous revision (9b2d884f) was a PPO fine-tune of Pollen's roller-sprint-2m reference (run ro1's iteration-875 checkpoint): 2.152 s on the Arena's board. This revision replaces it with the PPO fine-tune of a different base described above.

Not for a real robot

This policy ignores the commanded twist: it always runs straight ahead on its roller skates at its own pace and never stops by itself. This fine-tune has only ever run in simulation and was trained without any sim-to-real randomisation; its base was trained with velocity pushes, centre-of-mass variation and initial tilt but, by its own card, is not validated on hardware either. Do not load it on a physical Microduck.

Files

• policy.onnx — one input obs [1, 61], one output actions [1, 14], 50 Hz, raw observations (the normalizer is baked in). Its metadata_props are the base's (without run_path) plus microduck_game_* provenance keys.
• manifest.json — schema 2 of the microduck policy manifest, built with Pollen's own microduck_rl publish code.
• LICENSE — the Apache License 2.0, under which the base is published and this fine-tune is distributed.
• NOTICE — where the weights come from and what this repo changed.

Credits

Base policy: HannesVonEssen/microduck-running by HannesVonEssen, under the Apache License 2.0, linked as base_model. Robot, simulator, Arena and publishing tools: Pollen Robotics. This fine-tune is a modified version of that policy, distributed under the same licence: see LICENSE and NOTICE. PPO fine-tune: the MICRODUCK GAME project (jrubiosainz), built with GitHub Copilot.

Resumen (español)

Participante del Sprint sobre patines de 2 m de la Microduck Arena, solo para simulación. Es la política de carrera de HannesVonEssen (Apache-2.0) reentrenada con PPO en CPU dentro del simulador del propio Arena. Ignora la velocidad que se le ordena: no debe cargarse en un robot real.