SuperMimic G1 tracker with fall recovery (bones_flat_v2_getup_lie, 260k iterations)
Browse files- .gitattributes +8 -0
- README.md +74 -1
- policy.pt +3 -0
- training_state_ckpt_0260000.pt +3 -0
- videos/ckpt_0260000_cartwheel_R_001__A138.mp4 +3 -0
- videos/ckpt_0260000_dance1_subject1_cartwheel1.mp4 +3 -0
- videos/ckpt_0260000_flip_180_R_001__A416.mp4 +3 -0
- videos/ckpt_0260000_flip_360_001__A415.mp4 +3 -0
- videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_prone_a.mp4 +3 -0
- videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_prone_b.mp4 +3 -0
- videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_supine_a.mp4 +3 -0
- videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_supine_b.mp4 +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,11 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
videos/ckpt_0260000_cartwheel_R_001__A138.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
videos/ckpt_0260000_dance1_subject1_cartwheel1.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
videos/ckpt_0260000_flip_180_R_001__A416.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
videos/ckpt_0260000_flip_360_001__A415.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_prone_a.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 41 |
+
videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_prone_b.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 42 |
+
videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_supine_a.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 43 |
+
videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_supine_b.mp4 filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -1,3 +1,76 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: mit-code-bones-seed-data
|
| 4 |
+
library_name: pytorch
|
| 5 |
+
tags:
|
| 6 |
+
- robotics
|
| 7 |
+
- humanoid
|
| 8 |
+
- unitree-g1
|
| 9 |
+
- motion-tracking
|
| 10 |
+
- world-model
|
| 11 |
+
- fall-recovery
|
| 12 |
---
|
| 13 |
+
|
| 14 |
+
# SuperMimic G1 tracker with fall recovery (get-up)
|
| 15 |
+
|
| 16 |
+
A Unitree G1 whole-body motion-tracking policy from an independent reproduction of
|
| 17 |
+
SuperMimic (arXiv:2610.28480), fine-tuned so that it gets up after falling
|
| 18 |
+
(prone, supine or on its side) and resists knock-down pushes.
|
| 19 |
+
Code: [github.com/Mr-Zqr/SuperMimic-Reproduced](https://github.com/Mr-Zqr/SuperMimic-Reproduced) (`main`).
|
| 20 |
+
|
| 21 |
+
基于 SuperMimic 复现的 Unitree G1 全身动作跟踪策略,在此基础上做了"摔倒后起身"微调(设计见仓库 `docs/REPRODUCTION_PLAN.md` 的 D22)。
|
| 22 |
+
|
| 23 |
+
## 结果
|
| 24 |
+
|
| 25 |
+
起身测试:256 个环境,静止站立的 T-pose 参考,全部关闭摔倒终止;"起身"定义为骨盆高度误差 < 0.1 m 且倾角误差 < 0.2,持续 0.5 s,限时 10 s。跟踪评测用固定的 2000 条 BONES 平地子集,每条从第一帧完整播放,无噪声、无域随机化。
|
| 26 |
+
|
| 27 |
+
| 检查点 | 俯卧 / 仰卧 / 侧卧 起身成功率 | 4–6 m/s 推倒率 | 2000 条子集跟踪回报 |
|
| 28 |
+
|---|---|---|---|
|
| 29 |
+
| 跟踪基线 bones_flat_v2,18 万次迭代 | 10% / 0% / — | 91%(18.8 万次迭代时测) | 0.921(2 条失败) |
|
| 30 |
+
| **本模型,26 万次迭代** | **100% / 100% / 100%**(中位 1.3 / 1.3 / 1.5 s) | **0%** | 0.912(6 条失败) |
|
| 31 |
+
|
| 32 |
+
代价:跟踪回报略降(逐 clip 中位 −0.006),主要来自全局根部位置 / 朝向漂移;新增的失败主要是倒立类动作。
|
| 33 |
+
|
| 34 |
+
## 文件
|
| 35 |
+
|
| 36 |
+
| 文件 | 内容 |
|
| 37 |
+
|---|---|
|
| 38 |
+
| `policy.pt` | 策略权重 + 配置(35.6 MB),可直接用仓库 `main` 的评测 / 回放脚本加载 |
|
| 39 |
+
| `training_state_ckpt_0260000.pt` | 完整训练状态(世界模型、critic、优化器、采样器,221 MB),用于续训 |
|
| 40 |
+
| `videos/` | 回放视频:起身(22.8 万次迭代,俯卧 / 仰卧各两段)、侧手翻和 180° / 360° 空翻(26 万次迭代) |
|
| 41 |
+
|
| 42 |
+
## 使用
|
| 43 |
+
|
| 44 |
+
```bash
|
| 45 |
+
git clone https://github.com/Mr-Zqr/SuperMimic-Reproduced.git
|
| 46 |
+
cd SuperMimic-Reproduced
|
| 47 |
+
uv sync
|
| 48 |
+
hf download RayZhao/supermimic_getup policy.pt --local-dir checkpoints
|
| 49 |
+
# 交互回放(需要 NVIDIA GPU;有显示器时用原生 MuJoCo 查看器,否则用浏览器版 viser)
|
| 50 |
+
uv run python scripts/play.py --ckpt checkpoints/policy.pt --npz-dir data/npz/lafan1 --clips fallAndGetUp1_subject1
|
| 51 |
+
# 评测(每条 clip 从第一帧完整播放)
|
| 52 |
+
uv run python scripts/eval_benchmark.py --ckpt checkpoints/policy.pt --npz-dir data/npz/lafan1
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
动作数据不随模型发布,按仓库 README 的"数据"一节准备(LAFAN1 或 BONES-SEED 的 G1 重定向)。
|
| 56 |
+
|
| 57 |
+
注意:`play.py` 和评测保留了训练时的摔倒终止(骨盆高度 / 倾角偏差超过阈值即重置)。想观察倒地后起身,需要关闭终止,见仓库 README 的"摔倒起身"一节。
|
| 58 |
+
|
| 59 |
+
## 训练
|
| 60 |
+
|
| 61 |
+
- 观测:论文 Table A.1 的 93 维本体感知(10 帧历史),加未来 10 帧 G1 参考目标;不含全局位置等状态估计。
|
| 62 |
+
- 基线:bones_flat_v2,在清洗后的 BONES-SEED 平地子集(102,004 条,198 h)上按论文 Table A.5 训练 18 万次迭代,单卡 RTX 4090。
|
| 63 |
+
- 起身微调:从 18 万次迭代续训到 26 万次,学习率恒定 2e-5,分三段:
|
| 64 |
+
1. 18 万 → 19.2 万:30% 的回合关闭摔倒终止(`--env.immune-probability 0.3`);
|
| 65 |
+
2. 19.2 万 → 21.2 万:再加击倒推力(`--env.push-interval-s 2.0 4.0 --env.push-speed 4.0 6.0`);
|
| 66 |
+
3. 21.2 万 → 26 万:再加 10% 从躺姿开始的重置(`--env.lie-probability 0.1`)。
|
| 67 |
+
- 续训 `training_state_ckpt_0260000.pt` 需要带 `per_bin` 重置采样器的代码版本(训练时用了 `--env.gmax-mode per_bin`);该采样器还不在 `main` 上。
|
| 68 |
+
|
| 69 |
+
## 局限
|
| 70 |
+
|
| 71 |
+
- 策略看不到自己的全局位置,长动作(如 LAFAN1 的 2–4 分钟 clip)会逐渐漂离参考。
|
| 72 |
+
- 倒立类动作和超出 G1 关节限位的参考(如跪坐到脚跟)仍会失败。
|
| 73 |
+
|
| 74 |
+
## 许可
|
| 75 |
+
|
| 76 |
+
代码为 MIT 许可(见 GitHub 仓库)。模型在 [BONES-SEED](https://huggingface.co/datasets/bones-studio/seed) 数据上训练,使用请同时遵守 BONES-SEED 的许可条款。
|
policy.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a12bac4f93cf653dbf755ead058b6a1837a2db6363aea4ff51e9c4bc817bfbf7
|
| 3 |
+
size 35581829
|
training_state_ckpt_0260000.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:91c57e9ca93a37aa54dbe1f96abee640ae30e39a04ed3cb93df63bac874d6fe2
|
| 3 |
+
size 221487247
|
videos/ckpt_0260000_cartwheel_R_001__A138.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c9d79856765b3cbf616945adffff660195f9dbd5231d86d24e796a336d854442
|
| 3 |
+
size 1264846
|
videos/ckpt_0260000_dance1_subject1_cartwheel1.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:15f5ec8d5d0c7249cdde6b6b96141cb0ee0517d0a9a7e27526122b489d863a07
|
| 3 |
+
size 1156754
|
videos/ckpt_0260000_flip_180_R_001__A416.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bc2b6df0c3768cb2b9b41bb5028d1f8c3cd0f944b6ce54d1c7b899ea33e01340
|
| 3 |
+
size 1312643
|
videos/ckpt_0260000_flip_360_001__A415.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d8715a2e71e68188d0d5f22a7eaa5cec526ec7d549bf7b5d855ec27b7d1dd822
|
| 3 |
+
size 961486
|
videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_prone_a.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:685f5183878f32844287dde7b63d10c02f7b9fcff04a5e163f8b57ddf29ec5ee
|
| 3 |
+
size 1658725
|
videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_prone_b.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9ca6bfcf4e9c1bebaff404073253f5d5a6dc0ba6d5ca7958bf875e4690660074
|
| 3 |
+
size 1335163
|
videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_supine_a.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:746a13bd6eaaa9d70b6b2ba326c6dd8e9b30163a2811988038543216ca2368f6
|
| 3 |
+
size 1396818
|
videos/getup_bones_flat_v2_getup_lie_ckpt_0228000_supine_b.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7210236a6d834074296365d4711b060c510e87acd1ca8fae4c7466984e70389e
|
| 3 |
+
size 1381635
|