| --- |
| license: mit |
| tags: |
| - reinforcement-learning |
| - locomotion |
| - mujoco |
| - robotics |
| --- |
| |
| # Snow Locomotion — Unitree G1 |
|
|
| A locomotion policy that walks on variable-depth snow, conditioned on an estimated support |
| signal from a sensorized crampon. Companion to the `hackathon-everest` foothold-assurance |
| project, whose estimator error statistics this training loop reproduces. |
|
|
| **Simulation only.** No physical robot, no field-calibrated snow physics, no real sensor data |
| anywhere in the chain. See the claim boundary at the bottom. |
|
|
| ## Which checkpoint do I want? |
|
|
| | directory | what it is | use it for | |
| |---|---|---| |
| | `baseline-v3` | 200M-step flat-ground walker, stock Playground config | **the baseline** — warm start for every fine-tune, and the comparison policy | |
| | `curr-s1` | curriculum stage 1 — snow capped at 112 mm | best snow result so far (reward 15.9, episode length 724) | |
| | `curr-s2` | curriculum stage 2 — snow capped at 175 mm | warm-started from `curr-s1` | |
| | `ft-full` | first fine-tune, brief's original reward weights | best non-curriculum snow policy | |
| | `code/snow_rl` | the package that produced these | reproducing or extending any checkpoint | |
|
|
| ### Ablation arms — controls, not deliverables |
|
|
| `ft-no_map`, `ft-no_support`, `ft-ablation` blank the belief map, the bilateral reserve, and |
| the entire sensor channel respectively. They isolate each component's contribution. |
|
|
| **Caveat:** all three were trained on reward weights that were later measured to walk badly |
| (2% velocity tracking), and came back statistically indistinguishable. Treat them as a |
| method demonstration, not as evidence about the components. |
|
|
| ## Measured results |
|
|
| Against `baseline-v3`, identical terrain, seed and commanded velocity (0.8 m/s): |
|
|
| | terrain | baseline | snow-trained | | |
| |---|---|---|---| |
| | flat (4 mm) | 79% tracking, 5.05 m | 74%, 4.72 m | slight regression | |
| | **shallow (71 mm)** | 26% tracking, 1.34 m | **60%, 3.59 m** | **2.7× distance** | |
| | deep (198 mm) | 2% tracking, 1.11 m | −13%, 0.51 m | regression | |
|
|
| Shallow snow is the result. Deep snow currently regresses — the curriculum stages exist to |
| address that, and the deep row should be re-measured against `curr-s3` when it lands. |
|
|
| Note that **survival time is a misleading metric here**: a policy that stops walking maximises |
| it. Five rounds of reward tuning each survived longer while covering less ground before this |
| was caught. Score on distance and velocity tracking. |
|
|
| ## How it works |
|
|
| Snow is a **force field, not geometry**. The MuJoCo ground is a flat plane representing hard |
| substrate; snow acts through `xfrc_applied` — depth-dependent sinkage plus ploughing drag on |
| the swing foot. A heightfield would model bumpy solid ground, which is the wrong physics for a |
| surface you sink into. |
|
|
| The policy never sees ground truth. It sees a corrupted estimate whose error statistics are |
| measured from the source project's held-out evaluation: a shrinkage slope of 0.677 (weak ground |
| reads stronger than it is), conditional noise, and a **36.7% false-safe rate** — on genuinely |
| weak ground, a confident report of 429 N when the truth is 200 N. Physics reads the truth; the |
| policy pays for acting on the lie. |
|
|
| Observations: 156 for the actor (103 stock + 24 estimator + 24 belief map + 5 reserve), |
| 273 for the critic (adds privileged ground truth, discarded at deployment). Actions: 29 joint |
| targets at 50 Hz. Brax PPO, 512/256/128 networks, stock hyperparameters. |
|
|
| ## Reproducing |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| from brax.training.agents.ppo import checkpoint as ck |
| |
| local = snapshot_download("iteratehack/snow-rl-baseline", allow_patterns="curr-s1/*") |
| params = ck.load(f"{local}/curr-s1/000061276160") # (normaliser, policy, value) |
| ``` |
|
|
| Pinned stack: `jax==0.7.2`, `jaxlib==0.7.2`, `brax==0.14.2`, `playground==0.2.0`, |
| `mujoco==3.12.0`. Later JAX removes `device_put_replicated`, which brax still calls. |
|
|
| ## Claim boundary |
|
|
| - Simulation only; no physical robot, no validated snow physics, no tested hardware. |
| - The estimator error model is reproduced from held-out **simulation** metrics, not measured |
| on real sensors, and inherits every limitation of the synthetic terrain and reduced-order |
| contact model. |
| - The defensible claim is narrow: *a locomotion policy conditioned on an estimated support |
| signal outperforms the identical policy without that channel, on identical terrain* — and |
| as of these checkpoints that holds on shallow snow and not on deep. |
|
|
