docs/roller_glide.md

Roller glide — riding one blade

Task: Mjlab-RollerGlide-Flat-MicroDuck (+ V2, V3), built on the official roller-skate velocity env. Goal: a sustained single-blade glide while skating forward (figure-skating style), on a roller module that has two wheels per blade.

Baseline: nobody glides yet

scripts/eval_single_blade.py classifies support every control step from tire–floor contacts (a bout = consecutive single-support steps on the same blade). 10 s straight skating, 5 trials:

policy	push	single-support time	bouts / 10 s	mean bout	max bout	speed
official roller.onnx	0.3	11 %	52	0.022 s	0.06 s	0.61 m/s
official roller.onnx	0.5	17 %	75	0.022 s	0.08 s	—
slalom v2 (ckpt 2000)	0.3	77 %	107	0.072 s	0.14 s	0.28 m/s
slalom v2 (ckpt 2000)	0.5	80 %	105	0.076 s	0.16 s	0.43 m/s

The official policy never leaves double support for more than 80 ms. This is despite its own glide reward (weight 4.0: single support + moving + quiet legs) and single_support (3.0). Two other official terms fight a sustained glide:

• gait_symmetry_penalty (−1.0) accumulates each foot's swing time over the episode and penalises |L − R| / (L + R) — a long glide on one blade is maximally lopsided;
• skating_air_time only pays swings between 0.15 s and 0.45 s — past half a second the lifted foot earns nothing.

Method

• Drop gait_symmetry; widen the skating_air_time window to 4 s. Skating stack otherwise unchanged (wheel_speed 10, glide 4, single_support 3, braking, forward_lean).
• Streak objective (mdp.py, roller-glide section): a per-env accumulator counts consecutive control steps of gated single support on the same blade. Gates: exactly one blade in contact, swing ankle ≥ 2 cm above the stance ankle (a real lift, not a drag), forward speed ≥ 0.2 m/s (a moving glide, never a one-leg stand), upright. Two payouts: glide_hold — per step, min(streak, 1 s)/1 s, weight 2 → 4 → 6 (iterations 0/500/1500), so commitment escalates the pay; glide_frontier — Δ(max streak this episode) × 20, pays each new record once and nothing for camping.
• Reverse-curriculum spawns: 50 % → 35 % → 20 % of episodes are born already gliding — one leg folded (hip −1.0 / knee 1.1 / ankle 0.7 rad, mirrored), trunk leaned 4–10° toward the stance blade, 0.25–0.55 m/s forward. "Keep the glide" gets dense data from iteration 0; "enter the glide" is learned on top (the trick that fixed the roulade's last mile).
• Throttle command 0.1–0.6 (forward-biased), heading held straight.
• Training: 4096 envs, 4000 iterations, one A10G on Hugging Face Jobs (≈ 3 h, ≈ $3). Checkpoints selected by the evaluator above, never by training reward.

Status (2026-09-02): unsolved — three runs, two diagnoses, credits ran out

run	change	outcome
v1 (4000 it, cut at ~3500)	official stack + streak objective (hold 2→6, frontier 20), 50 % born-gliding spawns	training glide terms stayed ≈ 0.04/step against a ≈ 1.9/step skating stack; every checkpoint ≥ 500 falls 10/10 from a standing start. Born-gliding start: does not fall (0/6, vs 6/6 for the official policy) but puts the foot down within 0.1 s and resumes 10 Hz mini-stepping.
v2 (cancelled at ~1100)	glide pay ×2.7, wheel_speed 10 → 4, single_support removed, stance-switch tax 0.5, frontier only beyond 0.3 s	stands still: episode length ~900, wheel_speed ≈ 0.3, glide terms ≈ 0. Cutting the drive plus attempt taxes made "do nothing" the optimum.
v3 (cut at 500 by credits)	v2 pay-outs + wheel_speed back to 10 + switch tax 2.0 + moving-double-support tax 0.5 + spawns that roll (wheels spun at v/r)	moving again (wheel_speed rising, penalties correct); glide terms not yet nonzero at 500 iterations — the hold weight was still on its 4 → 10 ramp. Unjudged.

Two things learned that any next attempt should keep:

1. Born-gliding must mean born rolling. The first spawn gave the base 0.4 m/s with the passive wheels at rest — a braking impulse that trips even the official policy 6/6. Spinning the wheels at v/r (wheel_radius in reset_roller_glide_state, --wheel-radius in the evaluator) removes the artefact; the official policy still falls 6/6 from a rolling single-blade start, so the balance itself is the hard part.
2. Reward mass ordering must be glide ≫ stepping > swizzle > stand, with the drive to move intact. v1 left stepping as good as gliding; v2 removed the drive and got a statue. v3 encodes the full ordering (each non-glide way of moving is taxed) — it is the run to repeat with budget (≈ $3 on an A10G).

Second round (v4–v6): a commanded sequence, and two more root causes

The requested behaviour is a sequence — skate on two feet, lift one foot and keep gliding, put it back down — so from v4 on the glide is a command: body-command slot 0 carries a flag; each episode samples a glide phase [t0, t0 + len) (born-gliding spawns start inside it); glide rewards and taxes are multiplied by the flag, the official stride reward by (1 − flag). Deployment writes the same slot (scripts/eval_single_blade.py --glide-cmd t0,t1, scripts/render_showcase.py glide).

run	change	outcome
v4	commanded phases on v3	smoke only (v3 was still running)
v5	+ spawn from a passively balanced pose (stance hip rolled, partial fold), creep-proof gates (0.05 m/s), forward-progress reward	standing start: no falls but 9 Hz stepping (0.09 s bouts); born-gliding: 6/6 falls; training glide_hold flat at 0.004
v6	+ spawn at the true single-blade height, hold paid from the first counted step, frontier from 0.05 s, dense fold-one-leg shaping	running

Two more findings, both measured on CPU before spending:

3. The spawn height was wrong. reset_base places the trunk at 0.1335–0.1435 m, but with one leg folded the stance tire touches the floor at 0.127–0.130 m — every born-gliding spawn was dropped 0.5–1.5 cm onto one blade. Worse, the roller keyframe used by the CPU tests has the tires 9 mm inside the floor, so the "passively balanced pose" measured in v5 was not what the env got. At the corrected height a pose with the leg folded 80 % and the stance hip rolled 0.22–0.3 rad stays up 2–3 s with no control (the v1 spawn: 0.4 s).
4. No reward gradient at short streaks. glide_hold paid ∝ streak and the frontier only beyond 0.3 s, so the 2–5-step single-blade fragments an exploring policy actually produces earned nothing — in every run the hold term sat at ≈ 0.004. v6 pays from the first counted step and adds a contact-free "fold one leg" shaping term during the commanded phase.

The evaluator (scripts/eval_single_blade.py, --start glide --wheel-radius 0.015 --spawn-z 0.129 --lift-scale 0.8 --stance-roll 0.25) and the Mjlab-RollerGlide{,V2,V3,V4,V5,V6}-Flat-MicroDuck envs and tests are in the repo.

Reproduce

uv run --with pytest pytest tests/test_roller_glide_cfg.py
uv run scripts/eval_single_blade.py --policy policies/roller.onnx --push 0.3,0.5 --seconds 10 --trials 5
uv run train Mjlab-RollerGlideV3-Flat-MicroDuck --env.scene.num-envs 4096 --agent.max_iterations 4000 --hf-jobs --flavor a10g-large
uv run scripts/sweep_hf_checkpoints.py --repo <ckpt repo> --task Mjlab-RollerGlideV3-Flat-MicroDuck --outdir out/glidev3_sweep \
    --eval "uv run scripts/eval_single_blade.py --policy {onnx} --push 0.3,0.5 --seconds 10 --trials 5 --csv {csv}"

v6 interim (checkpoint 1000 of 4000, held-out protocols, push 0.3)

protocol	single-support	bouts / run	bout mean / p90 / max	speed	fell
born gliding (v6 pose, command on 0–6 s)	62 %	2.3	0.16 / 0.42 / 0.74 s	0.12–0.14 m/s	1 / 6
standing start, v6 alone (command 2–6 s)	—	—	—	—	6 / 6 (lunges, down at 0.36 s)
hot-swap: official skater ↔ v6 (glide 2–6 s)	51 %	39	0.10 / 0.23 / 0.28 s	≈ 0	7 / 10

First sub-second single-blade holds from the balanced spawn (v5 never exceeded a few control steps); the two-feet phases are not learned yet at this checkpoint and the policy cannot yet take a rolling hand-off from the official skater. Both are expected to move with the spawn curriculum (born-glide share 0.7 → 0.5 → 0.3 at 1500 / 3000). Evaluator gained --skate-policy (hot-swap by command phase) for the fallback demo.

v7 — spawn the two-feet phase inside the official skater's manifold

The checkpoint sweep of v6 (250–2000, three protocols, n = 6 each) settled the diagnosis:

protocol	best checkpoint	falls	bout max
born gliding (v6 pose, command from t = 0)	2000	3 / 6	0.80 s
hot-swap official → v6 → official (glide 2–6 s)	750 / 1000 / 2000	2–3 / 6	0.26–0.30 s
standing start, v6 alone (command 2–6 s)	none	6 / 6 at every checkpoint	0.02 s

The single-blade sub-skill emerged (born-gliding spawns), the two-feet phase never did: standing spawns pitch over within 0.4 s, stride_two_feet stayed ≈ 0.01, and every training metric stepped down at the 1500 spawn-mix shift (more standing spawns = more immediate falls — the AGENTS.md "metric drops at a stage boundary" signal). Two causes, both measured rather than guessed: the non-glide spawns started from rest, so the policy never had on-policy two-feet skating data to keep; and forward_progress (weight 6) paid a forward lunge right up to the fall.

v7 (Mjlab-RollerGlideV7-Flat-MicroDuck) applies the élan-roulade recipe: the non-glide slice is overlaid with a measured skating state of the released roller policy (scripts/capture_skate_states.py → data/roller_skate_states.npz, 956 states at 0.52–0.79 m/s: joints, joint velocities, base twist, the four wheel spins, trunk roll/pitch ≈ 10° forward lean), blended U(0.7, 1.0) toward rest so partial hand-offs are covered; 15 % of that slice still starts from rest. forward_progress is gated on uprightness. Born-glide share 0.6 → 0.5 → 0.4. The runtime hand-off (official skater → glide policy) is therefore in-distribution by construction, which is also the deployment story.

Spawn-height check before launch (the AGENTS.md "measure target heights off the actual robot" rule, again): with the lowest tire on the floor the released skater's crouched pose puts the trunk at 0.104 m (p5–p95 0.095–0.112) against 0.118 m for HOME, so overlaying its pose at reset_base's HOME height would have dropped the duck 1.4 cm onto rolling wheels. The bank records each pose's floor height (z_need) and the overlay keeps reset_base's drop margin above it. Sanity of the spawn itself (evaluator --start bank): the official policy survives 10/12 bank spawns for 8 s; the two falls happen 0.5–1.3 s in, the actuator-target jolt every reset has (seeding the last-action slot from the bank changed nothing).

v8 — price the pitch on one blade

How do v6's single-blade holds end? Evaluator fall_dir (body-frame projected gravity at the fall; x > 0 is a forward lean), born-gliding protocol, n = 12 per checkpoint:

checkpoint	falls	forward	sideways	fall time	bout max per trial
1000	5 / 12	5	0	1.06–1.20 s	0.52–0.74 s
2000	5 / 12	3	2	0.74–1.30 s	0.38–0.82 s

The bouts cap at 0.6–0.8 s and the duck goes over the front wheel: wheel_speed (10) and forward_progress (6) keep paying for pushing the mass forward, nothing prices the trunk pitch on one blade, and the 32° upright gate only stops paying once the fall is committed. v8 (Mjlab-RollerGlideV8-Flat-MicroDuck) = v7 + glide_pitch (weight 3, flag-gated Gaussian on the forward lean around the released skater's own +0.17 ≈ 10°, std 0.2) + the progress gate tightened to 18°. v7 and v8 train in parallel; the morning pick is by the hot-swap and rolling-start protocols, never by training reward.

v6 final sweep (job completed 23:22 PT, 4000 iterations)

checkpoint	standing start (glide 2–6 s)	born gliding: falls	single support	bouts / 6 s	bout mean	bout max
1000	6 / 6 fell	1 / 6	80 %	31	0.14 s	0.74 s
2000	6 / 6 fell	3 / 6	88 %	20	0.17 s	0.80 s
3250	6 / 6 fell	0 / 6	89 %	41	0.13 s	0.94 s
3500	6 / 6 fell	0 / 6	87 %	42	0.12 s	0.54 s
3999	6 / 6 fell	1 / 6	81 %	54	0.12 s	0.66 s

Every checkpoint falls from a standing start. From the born-gliding spawn the late checkpoints stop falling — but by stepping: 40–54 support switches per 6 s with 0.12 s bouts, i.e. the mini-stepping the stance-switch tax was meant to remove, now cheaper than a hold that ends in a pitch-over. v6 is therefore neither the two-feet phase nor a sustained single-blade glide; its role was diagnostic (spawn height, the missing two-feet data, the forward pitch-over), which v7 and v8 act on.

Training-side check of the same spawn (scripts/validate_skate_spawns.py, the v7 env on CPU with its full DR/noise/delay stack, 24 envs): skate spawns sit 1.4–2.5 cm above the floor (the same drop margin reset_base gives HOME), lean +0.02–0.18, wheels 27–67 rad/s forward; the released roller policy survives 15 / 15 skate-or-rest spawns for 2 s and 0 / 9 born-gliding ones (it never folds a leg). The v7/v8 premise — the two-feet phase starts inside the skater's manifold — holds in the environment that trains it, not just in the evaluator.

v7 / v8: the two-feet phase is learned in 300 iterations — and lost at the next reward step

Both runs (L4, 4096 envs) on wandb, per-window means:

run	window	episode length	falls / window	stride_two_feet	glide_hold (weighted)	entropy
v7	750–1000	758	2.1	0.55	0.67 (weight 10)	7.2
v7	1000–1100	605	5.1	0.42	0.88 (weight 16)	8.6
v7	1100–1500	140–180	25–29	0.05	0.68	9.7
v8	300–400	658	3.5	0.43	0.21 (weight 4)	8.3
v8	400–500	613	4.5	0.42	0.48 (weight 10)	9.6
v8	500–800	160–210	23–26	0.06–0.10	0.43	9.4

With the measured skate-state spawns the two-feet phase appears within 300 iterations (episode length 700+ of a 1000-step episode, 2–4 falls per window against v6's 39) — and both runs collapse on the same trigger, the glide_hold weight step (v8 at 400: 4 → 10; v7 at 1000: 10 → 16). The value loss doubles at the step, entropy climbs, and within 100 iterations the stride term dies. The hold pays from the first counted step (floor 0.5) at 10–16× while nothing prices the fall, so "lift a foot, bank the hold, fall" beats skating on. Pre-collapse v7 checkpoints (500–1000) survive both hand-off protocols — hot-swap 0–1 / 6 falls, rolling start 0 / 6 — but mini-step instead of holding a blade (65–108 support switches per 10 s, 0.06 s bouts). Both runs were cancelled at ~1400 / ~800 iterations; every checkpoint stays in its Hub repo.

v9 (Mjlab-RollerGlideV9-Flat-MicroDuck) = v8 + hold weight 4 → 6 → 8 → 10 (iterations 400 / 1200 / 2400, no step over ×1.5, capped at v7's best regime) + a fall penalty (mjlab is_terminated, −100: a doomed 0.5 s attempt banks ≈ 125–250 of hold at weight 10, the fall must cost more). v9b = v9 with the born-gliding share held at 0.7 for the whole run (v9: 0.6 → 0.5 → 0.4) — the sustained hold has only ever appeared from that spawn. Both launched at 23:37 PT on L4, 4000 iterations each.

Pre-collapse checkpoints on the held-out protocols (push 0.3, glide commanded 2–6 s, n = 6):

checkpoint	standing start	rolling (bank) start	hot-swap official ↔ policy	born gliding	bouts / 10 s	bout max
v7 750	0 / 6 fell	0 / 6	1 / 6	0 / 6	58–105	0.16–0.30 s
v7 1000	0 / 6 fell	0 / 6	0 / 6	0 / 6	57–105	0.16–0.32 s
v8 500	0 / 6 fell	1 / 6	0 / 6	0 / 6	64–114	0.14–0.30 s

These are the first policies in the family that survive every start, including a standing one (v6: 6 / 6 falls at all 16 checkpoints) — they answer a glide command by pumping at ~10 Hz rather than holding a blade. They are the fallback for a two-feet → command → two-feet demo if v9 / v9b do not produce a sustained hold.

v9 at 1500: the collapse is gone, the hold is still missing

v9 checkpoints 250–1500, held-out protocols (push 0.3, glide commanded 2–6 s, n = 6):

protocol	falls (all six checkpoints)	support switches / run	bout max
standing start	0 / 6 at every checkpoint	102–116	0.16–0.38 s
rolling (bank) start	0–1 / 6	92–113	0.16–0.40 s
hot-swap official ↔ v9	0–3 / 6, all after the switch back to the official at 6 s	58–82	0.24–0.44 s
born gliding	0 / 6 at every checkpoint	60–63	0.20–0.34 s

The gentle hold ramp and the priced fall removed the collapse and produced the family's first policy that never falls from a standing start — and it answers the glide command by pumping at ~10 Hz, even from the born-gliding spawn where v6 held 0.8 s. Per-step accounting during a commanded glide: a pump earns the hold floor (0.5 × 6 on about half its steps ≈ 1.5) at almost no risk; a real hold earns 3–6 but risks −100 plus the lost future; the switch tax (2 per switch ≈ 0.4 per step) barely dents the pump. Risk-averse pumping is the argmax.

v10 (Mjlab-RollerGlideV10-Flat-MicroDuck, on v9b) fixes the order rather than the magnitudes: switch tax 2 → 6 (a 10 Hz pump now costs 1.2 per step, more than it earns), hold floor 0.5 → 0.25 (a flicker pays a quarter of a 1 s streak per step), fall −100 → −30 (priced, not prohibitive). Launched 01:05 PT on L4 as the third concurrent run.

v9b at 1500 (born-gliding share held at 0.7) is indistinguishable from v9: 0 / 6 falls from a standing start at every checkpoint, 57–116 support switches per run, bout max ≤ 0.32 s. More practice from the gliding spawn does not by itself produce a hold — the policy puts the foot down immediately and pumps, because that is the safe option under the current prices. The two-feet phase, meanwhile, is solved by every v9 checkpoint from 250 on.

v11 — training wheels for the hold

Every glide run (v6–v10) exits the born-gliding spawn the same way: the lifted foot goes down within a few control steps and the policy pumps, because a put-down is always available and always safe, while the hold is a balance skill it has never practised. v11 removes the exit: a swing-leg lock action term (glide_actions.LockedSwingJointPositionAction) forces the lifted leg's position targets to the folded pose for the first second of every born-gliding episode (window 1.0 → 0.5 → 0.25 → 0 s at iterations 1000 / 2000 / 3000), so "balance on one rolling blade" is on-policy from iteration 0 and the swing_leg_pose reward keeps the policy's own actions aligned with the lock. Non-born episodes, the head and the play/export config are untouched — the deployed policy is a plain joint-position policy. Validated in the environment on CPU (locked envs' swing targets equal the fold, non-born envs untouched, release at the window end). Everything else is v10. v9b (v9's twin) was stopped at 2500 iterations to free the job slot.

v10 at 1500 (switch tax 6, hold floor 0.25, fall −30): no change in kind — 0 / 6 falls from a standing start at every checkpoint, 58–115 support switches per run, bout max ≤ 0.44 s, born gliding exits into pumping at once. The pump is not a marginal-price decision; the hold is simply not in the policy's repertoire and gradient exploration never finds it. That is the case for v11's lock.

v9 final (4000 iterations, completed 02:54 PT)

checkpoint	standing start	rolling (bank) start	hot-swap official ↔ v9	born gliding	bouts / run	bout max
2000	0 / 6	1 / 6	3 / 6 (after the switch back)	0 / 6	63–114	0.16–0.72 s
2750	0 / 6	0 / 6	0 / 6	0 / 6	71–117	0.16–0.26 s
3000–3750	0 / 6	0 / 6	1–5 / 6 (after the switch back)	0 / 6	54–124	0.16–0.46 s
3999	6 / 6	3 / 6	3 / 6	0 / 6	38–79	0.16–0.36 s

v9 is a finished two-feet skater for every start (checkpoint 2750 survives all four protocols), and never holds a blade. Late checkpoints get flaky (2250 and 3999 fall from a standing start) and the official policy increasingly fails to take the state back at the 6 s switch — the hand-off to the official is not something these policies were trained for. Fallback for a two-feet → command → two-feet demo without the hold: out/glidev9_sweep/onnx/ckpt_2750.onnx.

v11 measured (evaluator --lock emulation, born gliding, n = 8): with the swing leg locked for 1 s, v11 checkpoints 1000 and 1250 fall sideways toward the lifted leg at 0.40 s in 8 / 8 trials — deterministically. The v6 spawn pose is balanced only for a policy that can move the folded leg; frozen, it is a falling pendulum, and a v10 checkpoint under the same lock survives the locked second (falls after release, 5 / 8). The lock manufactures falls, not holds; v11 was stopped at ~1600 iterations. Hot-swapping the finished two-feet skater (v9 2750) with v6's best hold checkpoints for the glide phase does not work either: v6 1000 / 2000 / 2250 pitch forward 0.2–0.5 s after the hand-off in 7 / 4 / 3 of 8 trials and hold ≤ 0.36 s when they survive.

v12 / v12b — v6's hold magnitude, paced, on the v10 base

The only sustained holds in the family (0.8 s, v6 1000–2000) came from a 16-weight hold with no fall price; every capped-hold run with a fall price pumps. v12 ramps the hold 4 → 8 → 12 → 16 (iterations 400 / 1200 / 2400, v9's pacing) on the v10 prices (switch tax 6, floor 0.25, fall −30, born share 0.7). v12b is the same with the fall priced only outside the commanded glide phase (is_terminated_when_skating), so a failed hold attempt costs only its lost future while the two-feet phase keeps its price. Both launched at 03:05 PT on L4 next to the still-running v10.

v10 / v12 / v12b outcome (wandb; checkpoint sweeps in out/glide{v10,v12,v12b}_sweep)

run	hold weight at the end	fall price	episode length (last window)	falls / window	glide_frontier (any window)
v10 (4000, completed)	10	−30	859	1.1	0.008
v12 (3500, 5 h timeout)	16	−30	781	1.9	0.009
v12b (3500, 5 h timeout)	16	−30 outside the glide only	168 (collapsed at 2000–2400)	24	0.009

Pricing the fall keeps every run stable up to a 16-weight hold (v12 never collapsed), and removing the price inside the glide phase brings the v7 collapse back (v12b, at hold weight 12) — but the frontier term never moves in any of them: with a fall price the policy does not attempt holds, and without one it attempts and dies. The overnight machine outage (DNS / proxy failures on the Mac) crashed every waiter; the runs themselves completed or hit the 5 h timeout with their checkpoints uploaded, and were re-swept the next evening.

Where this leaves the single-blade glide. The family now contains a finished two-feet skater with a working glide command channel (v9 2750, v10 late checkpoints: never falls from a standing start, follows the flag) and one regime that produced sub-second single-blade holds (v6, 0.8 s, 50 % falls, from a balanced spawn only). A sustained hold has not been reached. The physics is unforgiving: no ankle-roll joint and a 5 cm wheelbase mean a one-blade glide is a bicycle-like balance (steer the blade into the lean) with a 5 cm fore–aft margin, and nothing tried here — spawn manifolds, streak rewards, price orders, a locked leg — created the on-policy successes RL needs to learn it. The next serious attempt should change the problem, not the prices: e.g. a slower target speed with a wider fore–aft margin, a swing foot allowed to skim (a "scooter" glide), or a hardware change (ankle roll / longer blade).

v10 full sweep (250–3999, four protocols, n = 6): no falls from a standing or born-gliding start at any checkpoint but one (3000, 1 / 6), bout max ≤ 0.6 s everywhere, 55–117 support switches per run. Checkpoint 3750 survives all four protocols (0 / 6 each) — a second clean two-feet fallback next to v9 2750 (out/glidev10_sweep/onnx/ckpt_3750.onnx).

v12 / v12b full sweeps (250–3500, four protocols, n = 6): v12 never falls from a standing or born-gliding start at any checkpoint and never holds (bout max ≤ 0.54 s, 58–119 switches per run). v12b matches it up to 2000, then the collapse shows on the held-out protocols too: from 2500 on it falls 6 / 6 from a standing start and 6 / 6 from a rolling start (backward, at 0.6–0.8 s), while still surviving the born-gliding spawn. The held-out numbers agree with the training curves in every version: pricing the fall buys stability and no holds; not pricing it buys attempts and a collapse.