microduck-tricks / docs /roller_glide_plan.md
langli11's picture
Microduck tricks: elan roulade, cone slalom, single-blade lift (policies, docs, clips)
ae7284e verified
|
Raw History Blame Contribute Delete
12.8 kB

Single-blade glide β€” retrospective and the next plan (2026-09-04)

Nine runs (v6–v12b, β‰ˆ $27 of L4 time) did not produce a sustained single-blade glide; the two-feet phase and the glide command channel are solved (v9 2750, v10 3750: 0/6 falls on all four held-out protocols). Full evidence per version: roller_glide.md.

Root causes (measured)

  1. No feasible successes to learn from. Prices (fall βˆ’100 β†’ βˆ’30, switch tax 2 β†’ 6, hold weight 10 β†’ 16) never changed held-out behaviour: with a fall price the policy pumps, without one it attempts and collapses. RL needs on-policy successes; nothing supplied them.
  2. The born-gliding spawn is a state, not a skill. v6 held 0.8 s using the free folded leg as a counterweight; frozen (v11) it falls sideways at 0.40 s, 8/8. scripts/probe_single_blade_balance.py: the passive folded pose rolls 2.5 s before pitching forward; 126 hand-tuned PD gain sets found no rolling one-blade balance (survivors stopped and put the foot down).
  3. A training metric was misread all night. glide_frontier β‰ˆ 0.009 was taken as "no holds"; scripts/audit_glide_gates.py (v6-2000 back inside the v6 env) shows the gates count 70 % of steps and training streaks reach 0.3–0.6 s β€” the value is just per-step normalised. The held-out battery is the only criterion; training holds from the spawn did not transfer to the command.
  4. Physics: no ankle roll, 5 cm inline wheelbase β†’ bicycle-like balance (hip roll for the CoM, hip yaw to steer into the lean) with a 5 cm fore–aft margin.
  5. Process: spawn-height, lock and pacing failures were all measurable on CPU before launch; a network outage killed every waiter; four runs tested the same price hypothesis.

Plan β€” prove feasibility first, then let RL learn it

stage what gate
0 (CPU, 1 day, $0) Model-based expert (hip pitch/ankle for the fore–aft margin, hip roll + hip yaw for lateral balance; trajectory optimisation if PD fails) from the v6 pose; in parallel a scooter definition (swing foot skims 1–2 cm, occasional touches allowed). Output: an expert trajectory bank (joints, velocities, attitude, wheel spins, random phase). β‰₯ 3 s one-blade rolling at β‰₯ 0.2 m/s, 8/10 under perturbation, for either definition β€” otherwise the hardware cannot do a strict single-blade glide: pick the scooter definition or a hardware change, do not train.
1 (1 night, β‰ˆ $4) v10 base (skate-state spawns, gentle hold ramp, fall βˆ’30, switch tax 6) + glide spawns at random phases of the expert trajectories (a true reverse curriculum from feasible states, share 0.7) + a wide-std Gaussian tracking of the expert's stance-leg joints and trunk attitude during the glide phase; four-protocol battery every 250 iterations, selection by the battery only. born protocol hold β‰₯ 1.5 s with ≀ 10 % falls; β‰₯ 1 s holds from the rolling start.
2 (1 night, β‰ˆ $4) Entry/exit with the existing phase-command infrastructure, initialised from stage 1, spawn mix shifting to two-feet starts; slewed internal target for the exit. rolling-start and hot-swap protocols: hold β‰₯ 1.5 s, ≀ 10 % falls, skates on after the exit. Then the video.
3 (fallback) DAgger-style distillation from the expert, then RL fine-tuning; or ship the scooter definition. β€”

Process rules for the next round: CPU spawn-geometry audit + reward-gate audit + 64-env smoke before every launch; one hypothesis per run, taken from held-out measurements; network-tolerant waiters (robust_sweep.sh) with results written to the repo; no weight step above Γ—1.5, stop and roll back on any metric drop at a stage boundary; feasibility before RL.

Stage 0 findings (2026-09-04)

  • Strict single blade (swing foot up): not reachable even for a model-based controller. scripts/expert_single_blade.py (CEM over a linear law on CoM-vs-blade offsets, trunk attitude and rates; stance hip roll/yaw/pitch/ankle + swing hip roll; searched fold strength): best laws survive 0.3–1.2 s, robustness 0/10, three searches (two strict, one scooter-as-single).
  • Why β€” actuator limits, measured (scripts/probe_actuator_limits.py): holding the born pose, the folded swing foot sags to 4–8 mm above the floor and touches on most steps at every commanded fold strength (servo tracking error 0.3–0.5 rad on knee/hip); pulling harder shifts the CoM and tips the duck in 0.4–0.6 s. The stance hip-roll step response has a 0.22 s 63 % rise time (with overshoot) against a β‰ˆ 0.1 s lateral pendulum β€” CoM-shift balance on one blade is outside the XL330 servo's bandwidth on this geometry.
  • Consequence: the whole v6–v12b family was in fact practising a skimming foot (≀ 1 cm clearance); the 1 cm training gate was at the edge of what the hardware can do.
  • Definition B (scooter glide: one leg visibly folded and unloaded, its tire may skim) is being measured next with contact forces (scripts/expert_scooter.py).

The decisive measurement: the born-gliding pose was never single support

scripts/probe_single_blade_geometry.py (contact forces, held pose, rolling at 0.35 m/s):

stance hip-roll (evaluator sign) lean CoM βˆ’ stance blade (lateral) swing-foot load stance load passive survival
+0.25 (v6 and every run since) 7Β° +5.5 cm 66 % 21 % 2.5 s (falls forward)
+0.60 7Β° +8.3 cm 66 % 22 % 4 s+ (standing on the folded foot, 0.03 m/s)
βˆ’0.25 7Β° +0.9 cm 15 % 64 % 0.34 s
βˆ’0.40 7Β° βˆ’0.5 cm 0 % 74 % 0.32 s (a true inverted pendulum)

With the sign the whole family used, the folded foot sits under the body and carries two thirds of the weight β€” the "0.8 s holds" of v6 were standing on the lifted leg's tire. The shift parameter translates the whole robot (feet included) and never moved the CoM over the blade. The training spawn (V6_STANCE_LEFT = {1: +0.25} / V6_STANCE_RIGHT = {10: βˆ’0.25}) has the same sign, so the training never saw a single-blade state either β€” and the actuator-bandwidth conclusion above was drawn on the wrong posture. With the stance hip-roll reversed (βˆ’0.3 to βˆ’0.4) the stance blade is under the CoM, the swing foot is 46 mm up and unloaded: the real balance problem, which the expert search now attacks from a balanced start.

From the corrected spawn the same CEM law (--stance-roll -0.3 --lean 7 --shift 0) rides one blade for 0.8–1.06 s at 0.5–0.8 m/s with 90–98 % single support and the swing tire ≀ 15 % of steps on the floor β€” the first genuine single-blade glides of the whole effort β€” and then falls; the search plateaus at 0.8 s (0/10 reach 3 s). Next oracle: sampling MPC (scripts/mppi_single_blade.py, perfect model, full state, 0.3 s lookahead) at 0.35 and 0.2 m/s. Mjlab-RollerGlideV13-Flat-MicroDuck carries the corrected spawn sign into training (V13_STANCE_LEFT = {1: βˆ’0.3}, V13_STANCE_RIGHT = {10: +0.3}, no root shift).

Stage 0 verdict (2026-09-04, evening)

Every controller that has the full state and the perfect model tops out at the same place:

controller posture true single-blade time notes
v6 RL policy (ckpt 2000), old spawn policy-corrected 0.80 s max, 88 % true single (load-based) falls 3/6
CEM linear law, corrected spawn, 0.35 m/s stance under CoM 0.8–1.06 s lateral CoM drift +12 cm in 0.8 s with the stance hip roll saturated
CEM linear law, 0.6 m/s (steering authority) same 0.5–0.9 s no gain from speed
MPPI (128 samples, 0.4 s horizon) around the best posture same 0.68–0.76 s perfect model, still ≀ 1 s

The load-based metric (eval_single_blade.py, one blade β‰₯ 90 % of the ground force) confirms the v6 holds were real single support once the policy lifted the leg β€” the passive spawn sag was a red herring for the policy, but it was real for every held-pose probe. The binding limit is lateral authority: the stance hip roll can shift the CoM only 1–2 cm, steering at these speeds buys a few degrees, there is no ankle roll, and the trunk+leg pendulum diverges in ~0.3 s. Verdict: a sustained (β‰₯ 3 s) one-blade glide is not reachable on this robot in this simulator. The reachable skill is a ~0.8–1.0 s one-blade glide, entered from skating and exited by placing the foot down before the fall. That is the target proposed for stage 1.

Stage 1 build (2026-09-04): Mjlab-RollerGlideV14-Flat-MicroDuck

Target: lift a foot while skating, ride one blade ~1 s, put it down before the fall, no falls. On v13 (v10's stable base + the corrected spawn sign): the single-support gate of every glide term is now measured by contact load (one blade β‰₯ 90 % of the ground force); a glide_exit reward (weight 10) pays once when a streak β‰₯ 0.6 s ends by the foot coming down upright; commanded glide phases last 0.8–1.5 s; half of the born-gliding envs spawn from the stage-0 expert's mid-glide states (692 true-single states, mirrored for the right-swing side). In-env check (scripts/validate_glide_spawns.py): 16 / 16 born-gliding envs start in true single support on the intended blade and the gate counts all 16. Launch is waiting on HF Jobs credits (the namespace is at zero).

v14 at 1500 (held-out, short command, n = 6–12)

start falls TRUE single-blade bout max note
corrected born pose, command 0–1.2 s 0 / 12 (ckpt 750 and 1500) 0.48–0.70 s (ckpt 750: 2 / 6 trials β‰₯ 0.6 s) holds, then puts the foot down cleanly
rolling start, command 2–3.2 s 0 / 6 at every checkpoint 0.22–0.30 s lifts briefly, does not commit
hot-swap from the official skater 0 / 6 0.20–0.30 s same
standing start 0 / 6 0.26–0.30 s same

The put-down bonus (fixed, β‰₯ 0.6 s) never fired in training (glide_exit = 0.000 in every window): the policy's bouts from a skating start sit at 0.3 s, below the threshold, so the bonus gave no gradient. v15 (launched 15:47 PT, L4 + A10G hedge) pays the put-down in proportion to the bout it ends (min(bout, 1 s)/1 s from 0.2 s, weight 15) and ramps the hold 6 β†’ 9 β†’ 12; v14 keeps running to 4000 for comparison.

v14 final (4000 iterations, A10G, completed 17:40 PT)

From every skating start (rolling, standing, hot-swap) the TRUE single-blade bout stays at 0.18–0.44 s at all 16 checkpoints with 0 falls: the policy lifts briefly and never commits. From the corrected born pose it commits at two checkpoints (2250: 6/6 trials β‰₯ 0.6 s, 0.74 s max; 3999: 5/6, 0.80 s) β€” and falls in 6/6 and 5/6 of those trials. The two behaviours the target needs (commit for ~1 s, then put the foot down) never coexist in v14. v15 (graded put-down bonus) is being swept; v16 (dense entries: first command from 0.3 s, two commands per episode) is built and smoke-tested, waiting for credits.

Interim deliverable (2026-09-04, 18:20 PT)

Seeded takes (renderer --seed, standing start, command at 3.0–4.2 s, side camera): v15 checkpoint 1250 lifts a foot 0.30–0.34 s of true single-blade support in 10 / 10 takes with no falls; v14 2000 with the official skater's hand-off 0.34–0.40 s, 10 / 10 no falls. Saved to the Desktop as microduck_single_leg_glide_v15-1250_take1.mp4 (+ a 0.25Γ— slow-motion cut of the lift). This is the honest current state: a visible lift-and-glide of a third of a second, entered from skating and exited cleanly β€” not yet the ~1 s the physics allows.

v15 final (4000 iterations, A10G, completed 18:35 PT)

start TRUE single-blade bout max (best checkpoints) falls
corrected born pose 0.94 s (3000, 4/6 trials β‰₯ 0.6 s), 1.18 s (3250, 3/6) 1/6, 2/6
rolling (bank) start 0.50–0.54 s (1750, 2000, 2500) 0–2/6
standing start 0.66 s (1750, 1/6; 2/6 falls), 0.44 s (2000, 0 falls) β€”
hot-swap from the official skater 0.24–0.40 s 0–1/6

The graded put-down bonus pushed the born-pose hold to the physical ceiling (β‰ˆ 1 s) and lengthened the lift from skating only marginally (0.3 β†’ 0.5 s). The entry from skating is still the missing skill; each episode offers one attempt at it. v16 (two commands per episode, first from 0.3 s) is the next run once credits allow.

Delivered video (2026-09-04, 19:05 PT)

Seeded takes of v15 checkpoints 1750 / 2000 / 2500 from a standing start (8 seeds each, command at 3.0–4.2 s): the longest clean lift is v15 1750 seed 7 β€” 0.64 s of true single-blade support (lift at 3.96 s), no fall; seeds 2 and 5 give 0.58 s; 2000 tops out at 0.46 s, 2500 at 0.40 s. ~/Desktop/microduck_single_leg_glide.mp4 (+ _slowmo.mp4, 0.25Γ— on 3.6–5.0 s) is that take.