| --- |
| tags: |
| - microduck |
| - robotics |
| - reinforcement-learning |
| - onnx |
| - microduck-slot:walk |
| library_name: microduck |
| pipeline_tag: robotics |
| --- |
| |
| # walk-step |
|
|
| Walk on flat ground with +-1 deg of gear play in every servo, fine-tuned for single steps: walk-flat v2 plus 3,000 iterations (7,000 to 10,000) on a floor that is flat except for one 1.5 m platform 10-25 mm high per 4 m tile (our one-change fork Mjlab-Velocity-Flat-Backlash-Step-MicroDuck), 4096 envs, 64 min on one RTX 5090. It has no terrain scan and meets the step blind. Trained in simulation only. In 20 s takes walking straight ahead over a 25 mm step with no shoves it stayed on its feet in 11 of 15 takes against at most 5 of 15 for walk-flat v2; on the same layout at 1 mm with shoves 10 of 15 against at most 7 of 15. Not yet tested on a real Microduck. |
|
|
| A **perpetual** policy for the [microduck](https://github.com/pollen-robotics/microduck) (61-D observation, 14 actions, 50 Hz). Runs until told otherwise — a gait for the `walk` slot. |
|
|
| ## Run it on a robot |
|
|
| ```bash |
| sudo robotctl policy load walk witcheer/microduck-walk-step |
| ``` |
|
|
| The observation normalizer is baked into `policy.onnx`; feed raw observations. |
| `manifest.json` follows schema 2 of the microduck policy manifest (`docs/policy-manifest.md` in the daemon repo). |
|
|
| ## Training |
|
|
| - **repo**: `pollen-robotics/microduck_rl` |
| - **branch**: `develop` |
| - **commit**: `53b8971b6` |
| - exported from a checkout with uncommitted changes |
|
|
| ## Results in simulation (v1, 2026-10-14) |
|
|
| What changed: one thing, the floor. Pollen's `Mjlab-Velocity-Flat-Backlash-MicroDuck` trains on an infinite flat plane. Our fork `Mjlab-Velocity-Flat-Backlash-Step-MicroDuck` swaps it for a generated floor that is flat except for one raised square per 4 m tile: a 1.5 m platform whose top sits 10 to 25 mm above the floor (drawn per tile), with 2.5 m of flat floor between platforms. Every edge crossing is a single step. Rewards, terminations, pushes, the gear-play robot, the curricula and the observations are the maker's: the policy has no terrain scan and meets the step blind. The fork is a separate task file registered by one import line in `tasks/__init__.py`. The maker's own rough-terrain task also softens terrain contacts and raises solver iterations; this fork does not, to keep one variable. |
|
|
| Training: warm start from [walk-flat v2](https://huggingface.co/witcheer/microduck-walk-flat) (`model_7000`), 3,000 more iterations (7,000 to 10,000), 4096 envs, 64 min on one RTX 5090 (about 1.28 s per iteration), one uncut block. Mean reward over 100-iteration windows: 93.5 at 8,000, 93.5 at 9,000, 95.1 at 10,000 (flat from 8,000); fell_over 0.864, 0.850, 0.808 over the same windows. The velocity tracking error rose from 0.340 in the first window after the resume to 0.468 in the last. |
| |
| Proof takes with `headless_play` in the step task, walk-step against walk-flat v2 under identical settings: 20 s each, standing start on a platform, the walking command fixed straight ahead at 0.3 m/s (play otherwise samples random commands and a take can miss the edge entirely), samples every 0.1 s, every platform forced to 25 mm. A take counts as held when it has no tilt fall and no mid-episode reset (the task's `fell_over` termination fires at 70 degrees of tilt). |
| |
| - 25 mm step, no pushes: 11 of 15 held against at most 5 of 15 for walk-flat v2 (one-sided Fisher exact p = 0.033). walk-flat v2 fell in the edge band (0.65 to 1.15 m from the platform centre) in 9 of the 10 takes it lost. |
| - 25 mm step with play pushes every 0.5 to 1 s: 5 of 15 against 2 of 15 (p = 0.19, not a result on its own). With pushes this often both policies mostly fall, and walk-step still had an edge-band fall in 6 of 15 takes. |
| - Both step conditions pooled: 16 of 30 against at most 7 of 30 (one-sided p = 0.016). |
| - Control, the same layout at 1 mm with pushes: 10 of 15 against at most 7 of 15 (two-sided p = 0.46). No loss without a real step. walk-step holds slightly more here too, not significantly, so part of the gain on the step may be general sturdiness under pushes rather than a step skill. |
| |
| "At most": on a generated floor the task ends an episode when the robot leaves the tile grid, and a single-env take starts in an edge column, so a few takes ended early with the duck upright. That hit walk-flat v2 only (2 takes in the no-push step case, 3 in the control, none for walk-step). Those takes are counted as held for walk-flat v2, which can only flatter it; the counts above are the conservative ones. |
| |
| The clean signal is the no-push step: walk-flat v2 falls at the edge, walk-step mostly walks down it. |
| |
| Not tested: walking up a step (no take of walk-step reached the next platform, 2.5 m away), step heights other than 25 mm and 1 mm, any command other than straight ahead at 0.3 m/s, slopes or stairs, more than 20 s, a second simulator, and a real robot. The higher training tracking error means it may follow speed commands less closely than walk-flat v2; the takes do not measure that. Not yet tested on a real Microduck. |
| |
| Clip and run log: the [microduck-skill-tree dataset](https://huggingface.co/datasets/witcheer/microduck-skill-tree) (folder `level-01e-walk-step`) carries the reward curve, the queue item, the fork file, the side-by-side clip and every take log. Sibling policies: `witcheer/microduck-*`. walk-flat stays the default walk; walk-step is the variant for a floor with a step in it. |
| |
| Simulation only. |
| |
