Publish latest step60 Spatial GRPO checkpoint for continued training
Browse filesFull model and optimizer state; Spatial 386/500 (77.2%). Use step60 for training beyond step60. Includes resume60-to80 instructions and a compatible reproduction entry point.
- README.md +23 -4
- SHA256SUMS +18 -4
- latest_policy.json +24 -0
- release_manifest.json +93 -11
- reproduction/README.zh-CN.md +23 -4
- reproduction/portable.py +3 -3
README.md
CHANGED
|
@@ -19,6 +19,24 @@ base_model:
|
|
| 19 |
|
| 20 |
# LIBERO-Spatial 双相机 GRPO 复现
|
| 21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
本发布包对应 π0.5 + Wan 主视角 + Cosmos-WoVR 腕部视角,在另一台 Linux
|
| 23 |
服务器上启动八卡 GRPO。训练使用世界模型生成的观测;成功率是在真实 LIBERO
|
| 24 |
仿真器中,以固定初始状态、10 个任务 × 50 次试验统计的 success-once。
|
|
@@ -30,7 +48,7 @@ base_model:
|
|
| 30 |
- `runtime/`:Python 3.11/3.10 解释器及两个环境实际使用的依赖层。恢复时保持层的
|
| 31 |
顺序,不使用包含重复版本的 `pip freeze` 输出来重建环境。
|
| 32 |
- `weights/pi05/global_step_12000/`:匹配的 few-shot SFT 初始化权重和统计量。
|
| 33 |
-
- `weights/policy/global_step_40/`、`global_step_50/`:可供原生 RLinf 续训的
|
| 34 |
完整策略权重及 DCP 训练状态。`global_step_45/` 只有完整策略权重,可用于评估
|
| 35 |
或模型初始化,不用于恢复优化器状态。
|
| 36 |
- `weights/wan_step1000/`、`weights/cosmos_step2000/`:本轮 GRPO 使用的世界模型
|
|
@@ -42,7 +60,7 @@ base_model:
|
|
| 42 |
- `evidence/`:评估结果、模型配对元数据、环境版本清单与实验谱系。
|
| 43 |
|
| 44 |
参考检查点的真实 Spatial 成功率:40 步 365/500(73.0%),45 步
|
| 45 |
-
370/500(74.0%),50 步 349/500(69.8%)。40 步由前一轮世界模型训练得到;
|
| 46 |
40→60 的本轮续训使用新 Wan 1000 步和新 Cosmos 2000 步。
|
| 47 |
|
| 48 |
## 目标服务器
|
|
@@ -78,7 +96,8 @@ python3 reproduction/portable.py check
|
|
| 78 |
|
| 79 |
## 启动 GRPO
|
| 80 |
|
| 81 |
-
复现
|
|
|
|
| 82 |
|
| 83 |
```bash
|
| 84 |
screen -dmS spatial-grpo bash -c \
|
|
@@ -110,7 +129,7 @@ python3 reproduction/portable.py train --steps 20 --output "$PWD/runs/new20"
|
|
| 110 |
## 评估发布的策略
|
| 111 |
|
| 112 |
```bash
|
| 113 |
-
python3 reproduction/portable.py eval --checkpoint
|
| 114 |
```
|
| 115 |
|
| 116 |
该入口只做真实 LIBERO 评估,不执行 GRPO 更新;沿用八卡启动器,因此仍会加载
|
|
|
|
| 19 |
|
| 20 |
# LIBERO-Spatial 双相机 GRPO 复现
|
| 21 |
|
| 22 |
+
## 最新可续训权重:第 60 步
|
| 23 |
+
|
| 24 |
+
**最新完整策略断点为 `weights/policy/global_step_60/`。第 60 步之后的训练请从这份断点恢复。**
|
| 25 |
+
它包含完整模型权重、优化器状态及 8 个 DCP 分片;真实 LIBERO-Spatial
|
| 26 |
+
10 个任务 × 50 次评估成功率为 **77.2%(386/500)**。
|
| 27 |
+
`latest_policy.json` 记录最新断点、评估结果和续训入口。40、45、50 步用于历史对比。
|
| 28 |
+
|
| 29 |
+
下载并执行下文的 `restore`、`check` 后,从 60 步继续训练 20 步,到总步数 80:
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
screen -dmS spatial-grpo60-to80 bash -c \
|
| 33 |
+
'python3 reproduction/portable.py train --resume 60 --target 80 --output "$PWD/runs/resume60_to80"'
|
| 34 |
+
screen -r spatial-grpo60-to80
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
`--target 80` 表示训练至总步数 80,即新增 20 步;恢复优化器时使用完整断点目录。
|
| 38 |
+
|
| 39 |
+
|
| 40 |
本发布包对应 π0.5 + Wan 主视角 + Cosmos-WoVR 腕部视角,在另一台 Linux
|
| 41 |
服务器上启动八卡 GRPO。训练使用世界模型生成的观测;成功率是在真实 LIBERO
|
| 42 |
仿真器中,以固定初始状态、10 个任务 × 50 次试验统计的 success-once。
|
|
|
|
| 48 |
- `runtime/`:Python 3.11/3.10 解释器及两个环境实际使用的依赖层。恢复时保持层的
|
| 49 |
顺序,不使用包含重复版本的 `pip freeze` 输出来重建环境。
|
| 50 |
- `weights/pi05/global_step_12000/`:匹配的 few-shot SFT 初始化权重和统计量。
|
| 51 |
+
- `weights/policy/global_step_40/`、`global_step_50/`、`global_step_60/`:可供原生 RLinf 续训的
|
| 52 |
完整策略权重及 DCP 训练状态。`global_step_45/` 只有完整策略权重,可用于评估
|
| 53 |
或模型初始化,不用于恢复优化器状态。
|
| 54 |
- `weights/wan_step1000/`、`weights/cosmos_step2000/`:本轮 GRPO 使用的世界模型
|
|
|
|
| 60 |
- `evidence/`:评估结果、模型配对元数据、环境版本清单与实验谱系。
|
| 61 |
|
| 62 |
参考检查点的真实 Spatial 成功率:40 步 365/500(73.0%),45 步
|
| 63 |
+
370/500(74.0%),50 步 349/500(69.8%),最新 60 步 386/500(77.2%)。40 步由前一轮世界模型训练得到;
|
| 64 |
40→60 的本轮续训使用新 Wan 1000 步和新 Cosmos 2000 步。
|
| 65 |
|
| 66 |
## 目标服务器
|
|
|
|
| 96 |
|
| 97 |
## 启动 GRPO
|
| 98 |
|
| 99 |
+
历史复现示例:使用第 40 步完整策略及新世界模型,继续到第 60 步。
|
| 100 |
+
第 60 步之后的续训使用上面的最新 60 步断点。
|
| 101 |
|
| 102 |
```bash
|
| 103 |
screen -dmS spatial-grpo bash -c \
|
|
|
|
| 129 |
## 评估发布的策略
|
| 130 |
|
| 131 |
```bash
|
| 132 |
+
python3 reproduction/portable.py eval --checkpoint 60 --output "$PWD/runs/eval60"
|
| 133 |
```
|
| 134 |
|
| 135 |
该入口只做真实 LIBERO 评估,不执行 GRPO 更新;沿用八卡启动器,因此仍会加载
|
SHA256SUMS
CHANGED
|
@@ -1,4 +1,4 @@
|
|
| 1 |
-
|
| 2 |
738f95dbf4a56003730d8c7ba113dad975a6636a69a8187fc0bc571dcdfdffbe LICENSES/COSMOS-MODEL-CARD.md
|
| 3 |
3bd882474403177ab22e65598d8ab1c1b299804b5bff0019f30c4a01aa405506 LICENSES/COSMOS-POLICY-MODEL-CARD.md
|
| 4 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSES/COSMOS-TOKENIZER-LICENSE.txt
|
|
@@ -10,7 +10,7 @@ b8817d9f58408022ff30bbf1abd39521e7e291c83b33ff04969c45ac46bd6c65 LICENSES/PI05-
|
|
| 10 |
c7c92fee190b604e2d18e27df517aca77871d0c6829d1a58b71521807d3acdb1 LICENSES/README.md
|
| 11 |
4bcf87ecfbbb8e07a01b21415a970c8b53a5283bf6872b657040d3f45c9241f7 LICENSES/WAN-SPATIAL-MODEL-CARD.md
|
| 12 |
9491f7a2676a19460f8e9d3c625dabd53841fa8fe3db92bf6cd73780dbf0348e NOTICE
|
| 13 |
-
|
| 14 |
5b119a98ad7824507ddff3c6c7507ca244261ee416dcfa623876702160c580d3 assets/cosmos/libero_dataset_statistics.json
|
| 15 |
8a03499676c6c196127c577144b9fd09bb02c30f9caf058adbcef09bb99ad8f5 assets/cosmos/libero_t5_embeddings.pkl
|
| 16 |
38071ab59bd94681c686fa51d75a1968f64e470262043be31f7a094e442fd981 assets/cosmos/tokenizer.pth
|
|
@@ -29,12 +29,15 @@ b3fb5c1f5311147b0c53c1881c35e5ef4ec723f06c98bc61e411cafd4871e4b6 evidence/cosmo
|
|
| 29 |
d9c2426cd1209ca943b280c3ed217d410cd571e3085fc3945b1a84befea911a2 evidence/evaluations/global_step_40.json
|
| 30 |
0d1100288aebcc2a22fab2470b97a45c8e7d728e38b36e6566e4021898af324f evidence/evaluations/global_step_45.json
|
| 31 |
54128b9e3c050b89b1582cd6fac382543c037096fb41d13dcad34cfb8c392e09 evidence/evaluations/global_step_50.json
|
|
|
|
|
|
|
| 32 |
fdc3869dc035263b05b2c4ff9b2f6826282e3a6cc3f4cd268f43cd9a329d2159 evidence/rlinf_environment.json
|
| 33 |
b1d0dca8515f82edbd9ffeb1b33f9023e24f23c68a1a644faf23caefedf7c8d1 evidence/world_model_training_manifest.json
|
| 34 |
-
|
|
|
|
| 35 |
f490f560a262e9402230bb0e7f1f965853dc24c4c72b4d6b3941293c40802e0b reproduction/eval_train.py
|
| 36 |
8d0c9116b5b8624aa8dba4f13b76206b7d326a44aa6a9c2b627076c8266b4582 reproduction/launcher.py
|
| 37 |
-
|
| 38 |
936193ae5fa7432637b050654633496a62455573abd0224dcfb8e118aceb8f39 runtime/cosmos_packages_0.tar.zst
|
| 39 |
bbac3dee6879dc41b12082e07400aa6adac52bbf1c4dc8728e53bf8df20ef709 runtime/cosmos_packages_1.tar.zst
|
| 40 |
c4940b5a72db60e1de3eb94534034d4d96be53714acb11044f910c0a8c24f1b2 runtime/python310.tar.zst
|
|
@@ -43,6 +46,7 @@ b40be1f29418f4fc9e813c1e801bd200f62c5acfe12e0a1aa38b945d8ea7705c runtime/python
|
|
| 43 |
a659145d90481d3028e12cc780dead4d8340b64b8e9b692bbec7a3185f359719 runtime/rlinf_packages_0.tar.zst
|
| 44 |
19114ac59362c80778ca751a7ae8635717ffb64d13c81ab0451c420c14b29bc9 runtime/rlinf_packages_1.tar.zst
|
| 45 |
574001e4ec917eaba32464fa65f81a3257b8691f9b41ea351c19ceff8e552175 source/grpo_spatial_source.tar.zst
|
|
|
|
| 46 |
b4a01639bdac0e98073499bbb3a84cd29b2274dba80fa286f0c7ee27bf75b949 weights/cosmos_step2000/checkpoint_manifest.json
|
| 47 |
7110ea589cfb2dabf4c90941cf5fc5c5d23fdf47ab78b7a16b56b5014e7a87bf weights/cosmos_step2000/iter_000002000/model/.metadata
|
| 48 |
01f1eb9e46b123df765f614a18835e86393c3bdf7d6147b7be4453bc551b99e4 weights/cosmos_step2000/iter_000002000/model/__0_0.distcp
|
|
@@ -81,4 +85,14 @@ b5fcade036cff609f4a594c97b0b6d48a0dee7384d1f426b1daa107498ac03f7 weights/policy
|
|
| 81 |
e988b633f0b8d1b80cb055704801bbdd0efbb26e125abc2aa2d5ea3558d600d2 weights/policy/global_step_50/actor/dcp_checkpoint/__6_0.distcp
|
| 82 |
ee625a1335ce36ad687aa57f35d3863e0aae9d4b0bde48240c97540df57ddf7f weights/policy/global_step_50/actor/dcp_checkpoint/__7_0.distcp
|
| 83 |
d38984568e62a9125b0ab9174fd2f0fa5384342db6786b46ecb18b6ab9a7f589 weights/policy/global_step_50/actor/model_state_dict/full_weights.pt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
ed5ae2b151c35b6b509824c829dce196cff1a0ee4d422130a3ad0acf8b45cc47 weights/wan_step1000/model-00001.safetensors
|
|
|
|
| 1 |
+
d78e440fa543d86631f243eddb10420ec0731d526d9a486a87ed14597a33e7bc .gitattributes
|
| 2 |
738f95dbf4a56003730d8c7ba113dad975a6636a69a8187fc0bc571dcdfdffbe LICENSES/COSMOS-MODEL-CARD.md
|
| 3 |
3bd882474403177ab22e65598d8ab1c1b299804b5bff0019f30c4a01aa405506 LICENSES/COSMOS-POLICY-MODEL-CARD.md
|
| 4 |
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSES/COSMOS-TOKENIZER-LICENSE.txt
|
|
|
|
| 10 |
c7c92fee190b604e2d18e27df517aca77871d0c6829d1a58b71521807d3acdb1 LICENSES/README.md
|
| 11 |
4bcf87ecfbbb8e07a01b21415a970c8b53a5283bf6872b657040d3f45c9241f7 LICENSES/WAN-SPATIAL-MODEL-CARD.md
|
| 12 |
9491f7a2676a19460f8e9d3c625dabd53841fa8fe3db92bf6cd73780dbf0348e NOTICE
|
| 13 |
+
a233ff8ba459e8daa3ab72c2c2061c6cb2f9a38f682d9cd0be7840b881b59e0b README.md
|
| 14 |
5b119a98ad7824507ddff3c6c7507ca244261ee416dcfa623876702160c580d3 assets/cosmos/libero_dataset_statistics.json
|
| 15 |
8a03499676c6c196127c577144b9fd09bb02c30f9caf058adbcef09bb99ad8f5 assets/cosmos/libero_t5_embeddings.pkl
|
| 16 |
38071ab59bd94681c686fa51d75a1968f64e470262043be31f7a094e442fd981 assets/cosmos/tokenizer.pth
|
|
|
|
| 29 |
d9c2426cd1209ca943b280c3ed217d410cd571e3085fc3945b1a84befea911a2 evidence/evaluations/global_step_40.json
|
| 30 |
0d1100288aebcc2a22fab2470b97a45c8e7d728e38b36e6566e4021898af324f evidence/evaluations/global_step_45.json
|
| 31 |
54128b9e3c050b89b1582cd6fac382543c037096fb41d13dcad34cfb8c392e09 evidence/evaluations/global_step_50.json
|
| 32 |
+
06b80579aa0beae3bf11400d3734e6b5ac9e19bb9fab8c7603ad0123b03a27c2 evidence/evaluations/global_step_60.json
|
| 33 |
+
cc963ddd48281aece1af1a1ccafde791879340728d117f0d692f1136ee289e59 evidence/policy_step60_resume.json
|
| 34 |
fdc3869dc035263b05b2c4ff9b2f6826282e3a6cc3f4cd268f43cd9a329d2159 evidence/rlinf_environment.json
|
| 35 |
b1d0dca8515f82edbd9ffeb1b33f9023e24f23c68a1a644faf23caefedf7c8d1 evidence/world_model_training_manifest.json
|
| 36 |
+
f436503a0dd25df2dc327a73c54dd776945cf3d952c30875942e72032d99492d latest_policy.json
|
| 37 |
+
1a62781c9104b8dfb0a9d2fd869da6ba0886e6fcf4967dc566090bb4873532c5 reproduction/README.zh-CN.md
|
| 38 |
f490f560a262e9402230bb0e7f1f965853dc24c4c72b4d6b3941293c40802e0b reproduction/eval_train.py
|
| 39 |
8d0c9116b5b8624aa8dba4f13b76206b7d326a44aa6a9c2b627076c8266b4582 reproduction/launcher.py
|
| 40 |
+
b6418fea3feeab7d912f0f84c0a9866cade77c000ec0b574d02fe97295a0230f reproduction/portable.py
|
| 41 |
936193ae5fa7432637b050654633496a62455573abd0224dcfb8e118aceb8f39 runtime/cosmos_packages_0.tar.zst
|
| 42 |
bbac3dee6879dc41b12082e07400aa6adac52bbf1c4dc8728e53bf8df20ef709 runtime/cosmos_packages_1.tar.zst
|
| 43 |
c4940b5a72db60e1de3eb94534034d4d96be53714acb11044f910c0a8c24f1b2 runtime/python310.tar.zst
|
|
|
|
| 46 |
a659145d90481d3028e12cc780dead4d8340b64b8e9b692bbec7a3185f359719 runtime/rlinf_packages_0.tar.zst
|
| 47 |
19114ac59362c80778ca751a7ae8635717ffb64d13c81ab0451c420c14b29bc9 runtime/rlinf_packages_1.tar.zst
|
| 48 |
574001e4ec917eaba32464fa65f81a3257b8691f9b41ea351c19ceff8e552175 source/grpo_spatial_source.tar.zst
|
| 49 |
+
08e1648ba07ce8429bc2faee49fe9a5517fde7d81eae94441e91a07872c91319 source/step60_reproduction_update.tar.zst
|
| 50 |
b4a01639bdac0e98073499bbb3a84cd29b2274dba80fa286f0c7ee27bf75b949 weights/cosmos_step2000/checkpoint_manifest.json
|
| 51 |
7110ea589cfb2dabf4c90941cf5fc5c5d23fdf47ab78b7a16b56b5014e7a87bf weights/cosmos_step2000/iter_000002000/model/.metadata
|
| 52 |
01f1eb9e46b123df765f614a18835e86393c3bdf7d6147b7be4453bc551b99e4 weights/cosmos_step2000/iter_000002000/model/__0_0.distcp
|
|
|
|
| 85 |
e988b633f0b8d1b80cb055704801bbdd0efbb26e125abc2aa2d5ea3558d600d2 weights/policy/global_step_50/actor/dcp_checkpoint/__6_0.distcp
|
| 86 |
ee625a1335ce36ad687aa57f35d3863e0aae9d4b0bde48240c97540df57ddf7f weights/policy/global_step_50/actor/dcp_checkpoint/__7_0.distcp
|
| 87 |
d38984568e62a9125b0ab9174fd2f0fa5384342db6786b46ecb18b6ab9a7f589 weights/policy/global_step_50/actor/model_state_dict/full_weights.pt
|
| 88 |
+
1ed27d7cab01656a7901e9ccad9bd7bfd2cbd9ebb2e66e30b6b0ce35bd021160 weights/policy/global_step_60/actor/dcp_checkpoint/.metadata
|
| 89 |
+
4305cd70f5e7ba12940141920a86139cd8be7802f487686608a2959bbcd9d317 weights/policy/global_step_60/actor/dcp_checkpoint/__0_0.distcp
|
| 90 |
+
6f2534769e8f33a5ac1e211fa2f02ab0f9162db96f304597302a41cec78d811f weights/policy/global_step_60/actor/dcp_checkpoint/__1_0.distcp
|
| 91 |
+
4db813d16ed81cc7e985624f832c55cb5077eea32e9101071874289ef8b84a12 weights/policy/global_step_60/actor/dcp_checkpoint/__2_0.distcp
|
| 92 |
+
2ddce9c502a095cac9a981297d4f39eb2853fbd2558040747bf57150381be382 weights/policy/global_step_60/actor/dcp_checkpoint/__3_0.distcp
|
| 93 |
+
69c393b1dc1c6ebadab2acd9da6f2ccc2faf85de0c440db9002deb60f5b75db0 weights/policy/global_step_60/actor/dcp_checkpoint/__4_0.distcp
|
| 94 |
+
f59e42610adf3a01bcf81e7e753054d42d7243661bb1998c427fbb517b0a4597 weights/policy/global_step_60/actor/dcp_checkpoint/__5_0.distcp
|
| 95 |
+
bb83c397885457716c7347f6a978750c57c38c64eecbd535247ed7e6005e1a4f weights/policy/global_step_60/actor/dcp_checkpoint/__6_0.distcp
|
| 96 |
+
a10bc4924f97525a7eb39c30c9b4437c999524a215006ea49b193deb641ee3b8 weights/policy/global_step_60/actor/dcp_checkpoint/__7_0.distcp
|
| 97 |
+
1160b6e7cb3c135453a68c6491119a061d1ee5b9a7e63afcba25d60be04b7b37 weights/policy/global_step_60/actor/model_state_dict/full_weights.pt
|
| 98 |
ed5ae2b151c35b6b509824c829dce196cff1a0ee4d422130a3ad0acf8b45cc47 weights/wan_step1000/model-00001.safetensors
|
latest_policy.json
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "spatial-grpo-latest-policy-v1",
|
| 3 |
+
"global_step": 60,
|
| 4 |
+
"checkpoint": "weights/policy/global_step_60",
|
| 5 |
+
"full_training_state": true,
|
| 6 |
+
"optimizer_state": true,
|
| 7 |
+
"dcp_shards": 8,
|
| 8 |
+
"parent_policy_step": 55,
|
| 9 |
+
"world_models": {
|
| 10 |
+
"wan_step": 1000,
|
| 11 |
+
"cosmos_step": 2000
|
| 12 |
+
},
|
| 13 |
+
"evaluation": {
|
| 14 |
+
"suite": "libero_spatial",
|
| 15 |
+
"episodes": 500,
|
| 16 |
+
"successes": 386,
|
| 17 |
+
"success_rate": 0.772,
|
| 18 |
+
"protocol": "10 tasks x 50 trials",
|
| 19 |
+
"evidence": "evidence/evaluations/global_step_60.json"
|
| 20 |
+
},
|
| 21 |
+
"completed_utc": "2026-10-25T13:18:28Z",
|
| 22 |
+
"recommended_resume_step": 60,
|
| 23 |
+
"continuation_example": "python3 reproduction/portable.py train --resume 60 --target 80 --output \"$PWD/runs/resume60_to80\""
|
| 24 |
+
}
|
release_manifest.json
CHANGED
|
@@ -7,8 +7,8 @@
|
|
| 7 |
"files": [
|
| 8 |
{
|
| 9 |
"path": ".gitattributes",
|
| 10 |
-
"size":
|
| 11 |
-
"sha256": "
|
| 12 |
},
|
| 13 |
{
|
| 14 |
"path": "LICENSES/COSMOS-MODEL-CARD.md",
|
|
@@ -67,8 +67,8 @@
|
|
| 67 |
},
|
| 68 |
{
|
| 69 |
"path": "README.md",
|
| 70 |
-
"size":
|
| 71 |
-
"sha256": "
|
| 72 |
},
|
| 73 |
{
|
| 74 |
"path": "assets/cosmos/libero_dataset_statistics.json",
|
|
@@ -160,6 +160,16 @@
|
|
| 160 |
"size": 39862,
|
| 161 |
"sha256": "54128b9e3c050b89b1582cd6fac382543c037096fb41d13dcad34cfb8c392e09"
|
| 162 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 163 |
{
|
| 164 |
"path": "evidence/rlinf_environment.json",
|
| 165 |
"size": 22254,
|
|
@@ -170,10 +180,15 @@
|
|
| 170 |
"size": 958295,
|
| 171 |
"sha256": "b1d0dca8515f82edbd9ffeb1b33f9023e24f23c68a1a644faf23caefedf7c8d1"
|
| 172 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 173 |
{
|
| 174 |
"path": "reproduction/README.zh-CN.md",
|
| 175 |
-
"size":
|
| 176 |
-
"sha256": "
|
| 177 |
},
|
| 178 |
{
|
| 179 |
"path": "reproduction/eval_train.py",
|
|
@@ -187,8 +202,8 @@
|
|
| 187 |
},
|
| 188 |
{
|
| 189 |
"path": "reproduction/portable.py",
|
| 190 |
-
"size":
|
| 191 |
-
"sha256": "
|
| 192 |
},
|
| 193 |
{
|
| 194 |
"path": "runtime/cosmos_packages_0.tar.zst",
|
|
@@ -230,6 +245,11 @@
|
|
| 230 |
"size": 290212712,
|
| 231 |
"sha256": "574001e4ec917eaba32464fa65f81a3257b8691f9b41ea351c19ceff8e552175"
|
| 232 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 233 |
{
|
| 234 |
"path": "weights/cosmos_step2000/checkpoint_manifest.json",
|
| 235 |
"size": 1095,
|
|
@@ -420,6 +440,56 @@
|
|
| 420 |
"size": 8526557492,
|
| 421 |
"sha256": "d38984568e62a9125b0ab9174fd2f0fa5384342db6786b46ecb18b6ab9a7f589"
|
| 422 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 423 |
{
|
| 424 |
"path": "weights/wan_step1000/model-00001.safetensors",
|
| 425 |
"size": 10050753552,
|
|
@@ -476,9 +546,14 @@
|
|
| 476 |
"path": "source/grpo_spatial_source.tar.zst",
|
| 477 |
"destination": ".",
|
| 478 |
"sha256": "574001e4ec917eaba32464fa65f81a3257b8691f9b41ea351c19ceff8e552175"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 479 |
}
|
| 480 |
],
|
| 481 |
-
"total_bytes":
|
| 482 |
"reference": {
|
| 483 |
"suite": "libero_spatial",
|
| 484 |
"resume_policy_step": 40,
|
|
@@ -500,7 +575,14 @@
|
|
| 500 |
"50": {
|
| 501 |
"success": 349,
|
| 502 |
"episodes": 500
|
|
|
|
|
|
|
|
|
|
|
|
|
| 503 |
}
|
| 504 |
-
}
|
| 505 |
-
|
|
|
|
|
|
|
|
|
|
| 506 |
}
|
|
|
|
| 7 |
"files": [
|
| 8 |
{
|
| 9 |
"path": ".gitattributes",
|
| 10 |
+
"size": 734,
|
| 11 |
+
"sha256": "d78e440fa543d86631f243eddb10420ec0731d526d9a486a87ed14597a33e7bc"
|
| 12 |
},
|
| 13 |
{
|
| 14 |
"path": "LICENSES/COSMOS-MODEL-CARD.md",
|
|
|
|
| 67 |
},
|
| 68 |
{
|
| 69 |
"path": "README.md",
|
| 70 |
+
"size": 7625,
|
| 71 |
+
"sha256": "a233ff8ba459e8daa3ab72c2c2061c6cb2f9a38f682d9cd0be7840b881b59e0b"
|
| 72 |
},
|
| 73 |
{
|
| 74 |
"path": "assets/cosmos/libero_dataset_statistics.json",
|
|
|
|
| 160 |
"size": 39862,
|
| 161 |
"sha256": "54128b9e3c050b89b1582cd6fac382543c037096fb41d13dcad34cfb8c392e09"
|
| 162 |
},
|
| 163 |
+
{
|
| 164 |
+
"path": "evidence/evaluations/global_step_60.json",
|
| 165 |
+
"size": 39791,
|
| 166 |
+
"sha256": "06b80579aa0beae3bf11400d3734e6b5ac9e19bb9fab8c7603ad0123b03a27c2"
|
| 167 |
+
},
|
| 168 |
+
{
|
| 169 |
+
"path": "evidence/policy_step60_resume.json",
|
| 170 |
+
"size": 453,
|
| 171 |
+
"sha256": "cc963ddd48281aece1af1a1ccafde791879340728d117f0d692f1136ee289e59"
|
| 172 |
+
},
|
| 173 |
{
|
| 174 |
"path": "evidence/rlinf_environment.json",
|
| 175 |
"size": 22254,
|
|
|
|
| 180 |
"size": 958295,
|
| 181 |
"sha256": "b1d0dca8515f82edbd9ffeb1b33f9023e24f23c68a1a644faf23caefedf7c8d1"
|
| 182 |
},
|
| 183 |
+
{
|
| 184 |
+
"path": "latest_policy.json",
|
| 185 |
+
"size": 722,
|
| 186 |
+
"sha256": "f436503a0dd25df2dc327a73c54dd776945cf3d952c30875942e72032d99492d"
|
| 187 |
+
},
|
| 188 |
{
|
| 189 |
"path": "reproduction/README.zh-CN.md",
|
| 190 |
+
"size": 7253,
|
| 191 |
+
"sha256": "1a62781c9104b8dfb0a9d2fd869da6ba0886e6fcf4967dc566090bb4873532c5"
|
| 192 |
},
|
| 193 |
{
|
| 194 |
"path": "reproduction/eval_train.py",
|
|
|
|
| 202 |
},
|
| 203 |
{
|
| 204 |
"path": "reproduction/portable.py",
|
| 205 |
+
"size": 15007,
|
| 206 |
+
"sha256": "b6418fea3feeab7d912f0f84c0a9866cade77c000ec0b574d02fe97295a0230f"
|
| 207 |
},
|
| 208 |
{
|
| 209 |
"path": "runtime/cosmos_packages_0.tar.zst",
|
|
|
|
| 245 |
"size": 290212712,
|
| 246 |
"sha256": "574001e4ec917eaba32464fa65f81a3257b8691f9b41ea351c19ceff8e552175"
|
| 247 |
},
|
| 248 |
+
{
|
| 249 |
+
"path": "source/step60_reproduction_update.tar.zst",
|
| 250 |
+
"size": 13965,
|
| 251 |
+
"sha256": "08e1648ba07ce8429bc2faee49fe9a5517fde7d81eae94441e91a07872c91319"
|
| 252 |
+
},
|
| 253 |
{
|
| 254 |
"path": "weights/cosmos_step2000/checkpoint_manifest.json",
|
| 255 |
"size": 1095,
|
|
|
|
| 440 |
"size": 8526557492,
|
| 441 |
"sha256": "d38984568e62a9125b0ab9174fd2f0fa5384342db6786b46ecb18b6ab9a7f589"
|
| 442 |
},
|
| 443 |
+
{
|
| 444 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/.metadata",
|
| 445 |
+
"size": 1570926,
|
| 446 |
+
"sha256": "1ed27d7cab01656a7901e9ccad9bd7bfd2cbd9ebb2e66e30b6b0ce35bd021160"
|
| 447 |
+
},
|
| 448 |
+
{
|
| 449 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/__0_0.distcp",
|
| 450 |
+
"size": 1272885561,
|
| 451 |
+
"sha256": "4305cd70f5e7ba12940141920a86139cd8be7802f487686608a2959bbcd9d317"
|
| 452 |
+
},
|
| 453 |
+
{
|
| 454 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/__1_0.distcp",
|
| 455 |
+
"size": 1268627329,
|
| 456 |
+
"sha256": "6f2534769e8f33a5ac1e211fa2f02ab0f9162db96f304597302a41cec78d811f"
|
| 457 |
+
},
|
| 458 |
+
{
|
| 459 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/__2_0.distcp",
|
| 460 |
+
"size": 1271441317,
|
| 461 |
+
"sha256": "4db813d16ed81cc7e985624f832c55cb5077eea32e9101071874289ef8b84a12"
|
| 462 |
+
},
|
| 463 |
+
{
|
| 464 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/__3_0.distcp",
|
| 465 |
+
"size": 1265552157,
|
| 466 |
+
"sha256": "2ddce9c502a095cac9a981297d4f39eb2853fbd2558040747bf57150381be382"
|
| 467 |
+
},
|
| 468 |
+
{
|
| 469 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/__4_0.distcp",
|
| 470 |
+
"size": 1267590646,
|
| 471 |
+
"sha256": "69c393b1dc1c6ebadab2acd9da6f2ccc2faf85de0c440db9002deb60f5b75db0"
|
| 472 |
+
},
|
| 473 |
+
{
|
| 474 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/__5_0.distcp",
|
| 475 |
+
"size": 1786989589,
|
| 476 |
+
"sha256": "f59e42610adf3a01bcf81e7e753054d42d7243661bb1998c427fbb517b0a4597"
|
| 477 |
+
},
|
| 478 |
+
{
|
| 479 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/__6_0.distcp",
|
| 480 |
+
"size": 2088971714,
|
| 481 |
+
"sha256": "bb83c397885457716c7347f6a978750c57c38c64eecbd535247ed7e6005e1a4f"
|
| 482 |
+
},
|
| 483 |
+
{
|
| 484 |
+
"path": "weights/policy/global_step_60/actor/dcp_checkpoint/__7_0.distcp",
|
| 485 |
+
"size": 1559511013,
|
| 486 |
+
"sha256": "a10bc4924f97525a7eb39c30c9b4437c999524a215006ea49b193deb641ee3b8"
|
| 487 |
+
},
|
| 488 |
+
{
|
| 489 |
+
"path": "weights/policy/global_step_60/actor/model_state_dict/full_weights.pt",
|
| 490 |
+
"size": 8526557492,
|
| 491 |
+
"sha256": "1160b6e7cb3c135453a68c6491119a061d1ee5b9a7e63afcba25d60be04b7b37"
|
| 492 |
+
},
|
| 493 |
{
|
| 494 |
"path": "weights/wan_step1000/model-00001.safetensors",
|
| 495 |
"size": 10050753552,
|
|
|
|
| 546 |
"path": "source/grpo_spatial_source.tar.zst",
|
| 547 |
"destination": ".",
|
| 548 |
"sha256": "574001e4ec917eaba32464fa65f81a3257b8691f9b41ea351c19ceff8e552175"
|
| 549 |
+
},
|
| 550 |
+
{
|
| 551 |
+
"path": "source/step60_reproduction_update.tar.zst",
|
| 552 |
+
"destination": ".",
|
| 553 |
+
"sha256": "08e1648ba07ce8429bc2faee49fe9a5517fde7d81eae94441e91a07872c91319"
|
| 554 |
}
|
| 555 |
],
|
| 556 |
+
"total_bytes": 126318775207,
|
| 557 |
"reference": {
|
| 558 |
"suite": "libero_spatial",
|
| 559 |
"resume_policy_step": 40,
|
|
|
|
| 575 |
"50": {
|
| 576 |
"success": 349,
|
| 577 |
"episodes": 500
|
| 578 |
+
},
|
| 579 |
+
"60": {
|
| 580 |
+
"success": 386,
|
| 581 |
+
"episodes": 500
|
| 582 |
}
|
| 583 |
+
},
|
| 584 |
+
"latest_policy_step": 60,
|
| 585 |
+
"recommended_resume_step": 60
|
| 586 |
+
},
|
| 587 |
+
"updated_utc": "2026-10-26T08:43:18.347958+00:00"
|
| 588 |
}
|
reproduction/README.zh-CN.md
CHANGED
|
@@ -1,5 +1,23 @@
|
|
| 1 |
# LIBERO-Spatial 双相机 GRPO 复现
|
| 2 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
本发布包对应 π0.5 + Wan 主视角 + Cosmos-WoVR 腕部视角,在另一台 Linux
|
| 4 |
服务器上启动八卡 GRPO。训练使用世界模型生成的观测;成功率是在真实 LIBERO
|
| 5 |
仿真器中,以固定初始状态、10 个任务 × 50 次试验统计的 success-once。
|
|
@@ -11,7 +29,7 @@
|
|
| 11 |
- `runtime/`:Python 3.11/3.10 解释器及两个环境实际使用的依赖层。恢复时保持层的
|
| 12 |
顺序,不使用包含重复版本的 `pip freeze` 输出来重建环境。
|
| 13 |
- `weights/pi05/global_step_12000/`:匹配的 few-shot SFT 初始化权重和统计量。
|
| 14 |
-
- `weights/policy/global_step_40/`、`global_step_50/`:可供原生 RLinf 续训的
|
| 15 |
完整策略权重及 DCP 训练状态。`global_step_45/` 只有完整策略权重,可用于评估
|
| 16 |
或模型初始化,不用于恢复优化器状态。
|
| 17 |
- `weights/wan_step1000/`、`weights/cosmos_step2000/`:本轮 GRPO 使用的世界模型
|
|
@@ -23,7 +41,7 @@
|
|
| 23 |
- `evidence/`:评估结果、模型配对元数据、环境版本清单与实验谱系。
|
| 24 |
|
| 25 |
参考检查点的真实 Spatial 成功率:40 步 365/500(73.0%),45 步
|
| 26 |
-
370/500(74.0%),50 步 349/500(69.8%)。40 步由前一轮世界模型训练得到;
|
| 27 |
40→60 的本轮续训使用新 Wan 1000 步和新 Cosmos 2000 步。
|
| 28 |
|
| 29 |
## 目标服务器
|
|
@@ -59,7 +77,8 @@ python3 reproduction/portable.py check
|
|
| 59 |
|
| 60 |
## 启动 GRPO
|
| 61 |
|
| 62 |
-
复现
|
|
|
|
| 63 |
|
| 64 |
```bash
|
| 65 |
screen -dmS spatial-grpo bash -c \
|
|
@@ -91,7 +110,7 @@ python3 reproduction/portable.py train --steps 20 --output "$PWD/runs/new20"
|
|
| 91 |
## 评估发布的策略
|
| 92 |
|
| 93 |
```bash
|
| 94 |
-
python3 reproduction/portable.py eval --checkpoint
|
| 95 |
```
|
| 96 |
|
| 97 |
该入口只做真实 LIBERO 评估,不执行 GRPO 更新;沿用八卡启动器,因此仍会加载
|
|
|
|
| 1 |
# LIBERO-Spatial 双相机 GRPO 复现
|
| 2 |
|
| 3 |
+
## 最新可续训权重:第 60 步
|
| 4 |
+
|
| 5 |
+
**最新完整策略断点为 `weights/policy/global_step_60/`。第 60 步之后的训练请从这份断点恢复。**
|
| 6 |
+
它包含完整模型权重、优化器状态及 8 个 DCP 分片;真实 LIBERO-Spatial
|
| 7 |
+
10 个任务 × 50 次评估成功率为 **77.2%(386/500)**。
|
| 8 |
+
`latest_policy.json` 记录最新断点、评估结果和续训入口。40、45、50 步用于历史对比。
|
| 9 |
+
|
| 10 |
+
下载并执行下文的 `restore`、`check` 后,从 60 步继续训练 20 步,到总步数 80:
|
| 11 |
+
|
| 12 |
+
```bash
|
| 13 |
+
screen -dmS spatial-grpo60-to80 bash -c \
|
| 14 |
+
'python3 reproduction/portable.py train --resume 60 --target 80 --output "$PWD/runs/resume60_to80"'
|
| 15 |
+
screen -r spatial-grpo60-to80
|
| 16 |
+
```
|
| 17 |
+
|
| 18 |
+
`--target 80` 表示训练至总步数 80,即新增 20 步;恢复优化器时使用完整断点目录。
|
| 19 |
+
|
| 20 |
+
|
| 21 |
本发布包对应 π0.5 + Wan 主视角 + Cosmos-WoVR 腕部视角,在另一台 Linux
|
| 22 |
服务器上启动八卡 GRPO。训练使用世界模型生成的观测;成功率是在真实 LIBERO
|
| 23 |
仿真器中,以固定初始状态、10 个任务 × 50 次试验统计的 success-once。
|
|
|
|
| 29 |
- `runtime/`:Python 3.11/3.10 解释器及两个环境实际使用的依赖层。恢复时保持层的
|
| 30 |
顺序,不使用包含重复版本的 `pip freeze` 输出来重建环境。
|
| 31 |
- `weights/pi05/global_step_12000/`:匹配的 few-shot SFT 初始化权重和统计量。
|
| 32 |
+
- `weights/policy/global_step_40/`、`global_step_50/`、`global_step_60/`:可供原生 RLinf 续训的
|
| 33 |
完整策略权重及 DCP 训练状态。`global_step_45/` 只有完整策略权重,可用于评估
|
| 34 |
或模型初始化,不用于恢复优化器状态。
|
| 35 |
- `weights/wan_step1000/`、`weights/cosmos_step2000/`:本轮 GRPO 使用的世界模型
|
|
|
|
| 41 |
- `evidence/`:评估结果、模型配对元数据、环境版本清单与实验谱系。
|
| 42 |
|
| 43 |
参考检查点的真实 Spatial 成功率:40 步 365/500(73.0%),45 步
|
| 44 |
+
370/500(74.0%),50 步 349/500(69.8%),最新 60 步 386/500(77.2%)。40 步由前一轮世界模型训练得到;
|
| 45 |
40→60 的本轮续训使用新 Wan 1000 步和新 Cosmos 2000 步。
|
| 46 |
|
| 47 |
## 目标服务器
|
|
|
|
| 77 |
|
| 78 |
## 启动 GRPO
|
| 79 |
|
| 80 |
+
历史复现示例:使用第 40 步完整策略及新世界模型,继续到第 60 步。
|
| 81 |
+
第 60 步之后的续训使用上面的最新 60 步断点。
|
| 82 |
|
| 83 |
```bash
|
| 84 |
screen -dmS spatial-grpo bash -c \
|
|
|
|
| 110 |
## 评估发布的策略
|
| 111 |
|
| 112 |
```bash
|
| 113 |
+
python3 reproduction/portable.py eval --checkpoint 60 --output "$PWD/runs/eval60"
|
| 114 |
```
|
| 115 |
|
| 116 |
该入口只做真实 LIBERO 评估,不执行 GRPO 更新;沿用八卡启动器,因此仍会加载
|
reproduction/portable.py
CHANGED
|
@@ -278,7 +278,7 @@ def run(root, args):
|
|
| 278 |
elif args.resume:
|
| 279 |
checkpoint = root / 'weights/policy' / f'global_step_{args.resume}'
|
| 280 |
if not (checkpoint / 'actor/dcp_checkpoint/.metadata').is_file():
|
| 281 |
-
raise ValueError('Resume needs a full DCP checkpoint; use policy step40 or
|
| 282 |
if target <= args.resume:
|
| 283 |
raise ValueError('The target must exceed the resumed step')
|
| 284 |
command += [f'runner.resume_dir={checkpoint}']
|
|
@@ -298,8 +298,8 @@ def main():
|
|
| 298 |
parser.add_argument('command', choices=['restore', 'verify', 'check', 'train', 'eval'])
|
| 299 |
parser.add_argument('--root', type=Path, default=Path(__file__).resolve().parents[1])
|
| 300 |
parser.add_argument('--quick', action='store_true', help='Only check sizes, without hashing payloads')
|
| 301 |
-
parser.add_argument('--resume', type=int, choices=[40, 50])
|
| 302 |
-
parser.add_argument('--checkpoint', type=int, choices=[40, 45, 50], default=
|
| 303 |
parser.add_argument('--steps', type=int, default=20)
|
| 304 |
parser.add_argument('--target', type=int)
|
| 305 |
parser.add_argument('--output', type=Path)
|
|
|
|
| 278 |
elif args.resume:
|
| 279 |
checkpoint = root / 'weights/policy' / f'global_step_{args.resume}'
|
| 280 |
if not (checkpoint / 'actor/dcp_checkpoint/.metadata').is_file():
|
| 281 |
+
raise ValueError('Resume needs a full DCP checkpoint; use policy step40, step50 or step60')
|
| 282 |
if target <= args.resume:
|
| 283 |
raise ValueError('The target must exceed the resumed step')
|
| 284 |
command += [f'runner.resume_dir={checkpoint}']
|
|
|
|
| 298 |
parser.add_argument('command', choices=['restore', 'verify', 'check', 'train', 'eval'])
|
| 299 |
parser.add_argument('--root', type=Path, default=Path(__file__).resolve().parents[1])
|
| 300 |
parser.add_argument('--quick', action='store_true', help='Only check sizes, without hashing payloads')
|
| 301 |
+
parser.add_argument('--resume', type=int, choices=[40, 50, 60])
|
| 302 |
+
parser.add_argument('--checkpoint', type=int, choices=[40, 45, 50, 60], default=60)
|
| 303 |
parser.add_argument('--steps', type=int, default=20)
|
| 304 |
parser.add_argument('--target', type=int)
|
| 305 |
parser.add_argument('--output', type=Path)
|