jrubiosainz/microduck-game-sprinter

MICRODUCK GAME sprinter

HannesVonEssen's microduck-running policy (Apache-2.0) fine-tuned on a CPU inside the Arena's own simulator with PPO and evolution strategies, the last of them scored on the Arena's official seeds: 0.99 s on those seeds; 171/200 finishes and 28 falls on seeds 1001-1200, which it never saw. Ignores the command; simulation only.

An entry for the 2 m Sprint of the Microduck Arena. Simulation only — see below.

What it is

A fine-tune of HannesVonEssen/microduck-running (revision 9f45af1f, policy.onnx sha256 007707dd7779…), HannesVonEssen's forward-running policy for the stock MicroDuck on flat ground (iteration 12,195, Apache-2.0), trained with Vottivott/microduck-playground@828d950 on pollen-robotics/microduck_rl@d424a0c. The network has the base's own shape (61 → 512 → 256 → 128 → 14, ELU) and keeps its input normalizer frozen; its weights were trained further with PPO and then evolution strategies (ES) on a CPU, inside the Arena's own runner and physics (MuJoCo 3.10.0, runner version 4), on this event's course:

• chain: HannesVonEssen's microduck-running policy (iteration 12,195) -> 1,850 PPO iterations on sprint-2m (run hs1, whose first 25 iterations trained only a fresh critic) -> 95 generations of evolution strategies (ES) on the race time (run es1) -> 30 more PPO iterations with four times larger batches (run hsb2, whose first 20 iterations trained only a fresh critic) -> 281 generations of ES on the race time over the Arena's official seeds 1–5 (run so1_sprint); internal forward command 1.0 m/s
• selection: elite_g00281_1983233114_m, one perturbation scored in generation 281 of run so1_sprint (noise 1983233114, minus sign), saved because its time on the official seeds, 0.992 s, beat every set of weights scored before it; then run on the seeds of the table below;
• PPO links (the earlier ones): reward = forward progress while the race clock runs, small penalties for sideways speed, yaw rate, drifting toward the walls and falling; in runs hs1 and hsb2 an episode also ends when the duck crosses the line, as the Arena's run does, with a one-off bonus equal to the reward of running at 1.5 m/s for every step left before the 2.5 s horizon; 96 parallel copies of the course in run hs1 and 384 in run hsb2;
• ES link (run es1, between the PPO runs): the objective is the race time itself — the finish time interpolated between control steps; a run still going after 1.5 s costs 1.5 s plus the metres left, plus 1 if it fell. Each generation scores the current weights and 64 antithetic Gaussian perturbations of every layer (σ = 0.02 × the layer's RMS) on the same 4 fresh seeds of the Arena's own start, ranks them and takes one Adam step (lr 0.004 in the same units) along the rank-weighted perturbations. A perturbation's rank utility also loses 1 × the share of its seeds on which it fell, so falls weigh more than a slow finish;
• ES link (the last one): the objective is the race time itself — the finish time interpolated between control steps; a run still going after 1.5 s costs 1.5 s plus the metres left, plus 1 if it fell. Each generation scores the current weights and 96 antithetic Gaussian perturbations of every layer (σ = 0.01 × the layer's RMS) on the Arena's official seeds 1–5 themselves, the board's own seeds, ranks them and takes one Adam step (lr 0.003 in the same units) along the rank-weighted perturbations. Any perturbation whose time on those seeds beat every set of weights scored before it was saved and checked through the Arena's own policy loader. The board time of this link's output is therefore an in-sample number; the Results table also runs it on seeds it never saw;
• command: a fixed internal command (forward speed 1.0 m/s, zero lateral speed, yaw rate, head and body) is written into the 13 command slots of the observation inside the graph, so the commanded twist is ignored.

No GPU, no new data, no paid compute.

Results

local run, Arena runner	score (non-finish = 50 s)	mean of finishers	finished
seeds 1–5 (the official set)	0.992 s	0.992 s (worst 1.00)	5/5
seeds 1001-1200	8.163 s	1.068 s (p90 1.08, worst 3.78)	171/200, 28 falls
seeds 1–5 with observation noise σ = 1e-04, 4 runs per seed	1.000–1.004 s (range)	—	20/20
seeds 1–5 with observation noise σ = 1e-03, 4 runs per seed	1.000–1.040 s (range)	—	20/20

The noise rows add Gaussian noise to every observation the policy sees: a stand-in for the float-rounding differences of another machine, which a chaotic gait amplifies.

arena score --event sprint-2m on this file (every speed of its sweep gives the same time, since the command is ignored; the sweep then reports the lowest):

sweep 21 commanded speeds 0.20–1.20 m/s
   0.20 m/s  0.98s
   0.25 m/s  0.98s
   0.30 m/s  0.98s
   0.35 m/s  0.98s
   0.40 m/s  0.98s
   0.45 m/s  0.98s
   0.50 m/s  0.98s
   0.55 m/s  0.98s
   0.60 m/s  0.98s
   0.65 m/s  0.98s
   0.70 m/s  0.98s
   0.75 m/s  0.98s
   0.80 m/s  0.98s
   0.85 m/s  0.98s
   0.90 m/s  0.98s
   0.95 m/s  0.98s
   1.00 m/s  0.98s
   1.05 m/s  0.98s
   1.10 m/s  0.98s
   1.15 m/s  0.98s
   1.20 m/s  0.98s

scored at 0.20 m/s
seed 1: finish   0.98s
seed 2: finish   0.98s
seed 3: finish   1.00s
seed 4: finish   1.00s
seed 5: finish   1.00s

score 0.992s · 5/5 finished · replay from seed 3

Local times are indicative: the Arena's machine can differ by a few hundredths of a second.

History

This repo's first revision (8f3bfe59) was Pollen's sprint-2m reference run unchanged inside a 37-parameter wrapper tuned with CMA-ES: 3.156 s on the Arena's board. Its second revision (f6241b55) was a PPO fine-tune of Pollen's sprint-2m reference (run s4's iteration-2075 checkpoint, after a chain of sprint and hill-climb runs): 1.084 s on the Arena's board. Its third revision (2ea9d739) was run hsb2's iteration-30 checkpoint, the starting point of the last ES link above: 1.040 s on the Arena's board. Its fourth revision (a1d32005) was run so1_sprint's generation-34 elite, an earlier point of the same ES link: 1.008 s on the Arena's board.

Not for a real robot

This policy ignores the commanded twist: it always runs straight ahead at its own pace and never stops by itself. This fine-tune has only ever run in simulation and was trained without any sim-to-real randomisation; its base was trained with velocity pushes, centre-of-mass variation and initial tilt but, by its own card, is not validated on hardware either. Do not load it on a physical Microduck.

Files

• policy.onnx — one input obs [1, 61], one output actions [1, 14], 50 Hz, raw observations (the normalizer is baked in). Its metadata_props are the base's (without run_path) plus microduck_game_* provenance keys.
• manifest.json — schema 2 of the microduck policy manifest, built with Pollen's own microduck_rl publish code.
• LICENSE — the Apache License 2.0, under which the base is published and this fine-tune is distributed.
• NOTICE — where the weights come from and what this repo changed.

Credits

Base policy: HannesVonEssen/microduck-running by HannesVonEssen, under the Apache License 2.0, linked as base_model. Robot, simulator, Arena and publishing tools: Pollen Robotics. This fine-tune is a modified version of that policy, distributed under the same licence: see LICENSE and NOTICE. PPO + ES fine-tune: the MICRODUCK GAME project (jrubiosainz), built with GitHub Copilot.

Resumen (español)

Participante del Sprint de 2 m de la Microduck Arena, solo para simulación. Es la política de carrera de HannesVonEssen (Apache-2.0) reentrenada con PPO y después con estrategias evolutivas (ES) en CPU dentro del simulador del propio Arena. Su última etapa se ajustó directamente sobre las cinco semillas oficiales del tablero, así que su tiempo en él no es una medida fuera de muestra. Ignora la velocidad que se le ordena: no debe cargarse en un robot real.