Upload folder using huggingface_hub
Browse files- README.md +67 -0
- manifest.json +27 -0
- trousers_gnn_depth.ckpt +3 -0
- trousers_gnn_knn.ckpt +3 -0
- trousers_gnn_mesh.ckpt +3 -0
- trousers_pn_depth.ckpt +3 -0
- trousers_pn_mesh.ckpt +3 -0
README.md
ADDED
|
@@ -0,0 +1,67 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- robotics
|
| 5 |
+
- imitation-learning
|
| 6 |
+
- diffusion-policy
|
| 7 |
+
- cloth-manipulation
|
| 8 |
+
---
|
| 9 |
+
|
| 10 |
+
# DP3 floating-gripper garment policies — Hang_Trousers
|
| 11 |
+
|
| 12 |
+
Inference-only checkpoints for five 3D Diffusion Policy variants trained on
|
| 13 |
+
floating-gripper garment-hanging demonstrations in DexGarmentLab
|
| 14 |
+
(`Hang_Trousers_FloatGrip`, 195 demos, 12,516 transitions).
|
| 15 |
+
|
| 16 |
+
Code and full notes: **`floating-gripper-policy`** branch of
|
| 17 |
+
<https://github.com/lei-shu/garmentlab>, under
|
| 18 |
+
`DexGarmentLab/IL_Baselines/Diffusion_Policy_3D/` — see `EXPERIMENTS.md` for how
|
| 19 |
+
these were trained and `SIM_EVAL_HANDOFF.md` for how to run them.
|
| 20 |
+
|
| 21 |
+
## The five variants
|
| 22 |
+
|
| 23 |
+
Each differs only in the observation representation and the point encoder.
|
| 24 |
+
`EE MAE` is held-out end-effector position error against demo actions — **not**
|
| 25 |
+
task success, which has not been measured yet.
|
| 26 |
+
|
| 27 |
+
| file | encoder | observation | edges | epoch | EE MAE |
|
| 28 |
+
|---|---|---|---|---|---|
|
| 29 |
+
| `trousers_pn_depth.ckpt` | PointNet | depth cloud 2048×3 | — | 550 | **6.25 mm** |
|
| 30 |
+
| `trousers_gnn_depth.ckpt` | mesh-GNN | depth cloud | kNN | 200 | 6.85 mm |
|
| 31 |
+
| `trousers_pn_mesh.ckpt` | PointNet | decimated mesh + pothook | — | 900 | 6.94 mm |
|
| 32 |
+
| `trousers_gnn_mesh.ckpt` | mesh-GNN | mesh nodes | **true garment faces** | 300 | 7.24 mm |
|
| 33 |
+
| `trousers_gnn_knn.ckpt` | mesh-GNN | mesh nodes | kNN | 200 | 7.35 mm |
|
| 34 |
+
|
| 35 |
+
All five sit within ~1 mm of each other on a 4-episode validation split, so treat
|
| 36 |
+
them as near-indistinguishable offline. They do beat an observation-free
|
| 37 |
+
"hold position" baseline (48.9 mm) by 8×, so they are genuinely using the
|
| 38 |
+
observation.
|
| 39 |
+
|
| 40 |
+
## Loading
|
| 41 |
+
|
| 42 |
+
```python
|
| 43 |
+
import torch, dill, hydra
|
| 44 |
+
from omegaconf import OmegaConf
|
| 45 |
+
OmegaConf.register_new_resolver("eval", eval, replace=True)
|
| 46 |
+
|
| 47 |
+
p = torch.load("trousers_pn_depth.ckpt", map_location="cpu", pickle_module=dill)
|
| 48 |
+
policy = hydra.utils.instantiate(p["cfg"].policy)
|
| 49 |
+
policy.load_state_dict(p["state_dicts"]["ema_model"])
|
| 50 |
+
policy.cuda().eval()
|
| 51 |
+
|
| 52 |
+
# obs: point_cloud (B, 3, 2048, 3) + agent_pos (B, 3, 8)
|
| 53 |
+
# out: (B, 4, 8) = [left_xyz, right_xyz, grip_L, grip_R], absolute metres
|
| 54 |
+
action = policy.predict_action(obs)["action"]
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
These carry EMA weights only (~0.98 GiB each); the training checkpoints they came
|
| 58 |
+
from were 3.91 GiB with optimizer state. The normalizer travels inside the
|
| 59 |
+
`state_dict`, so actions come back in metres — do not rescale. Gripper bits are
|
| 60 |
+
regressed continuously; threshold at 0.5.
|
| 61 |
+
|
| 62 |
+
`trousers_gnn_mesh` additionally needs `edges`, `n_nodes`, `n_edges` in the
|
| 63 |
+
observation dict (see `SIM_EVAL_HANDOFF.md` §4a — the stock `RobotRunner` drops
|
| 64 |
+
them and it will raise `KeyError: 'edges'`).
|
| 65 |
+
|
| 66 |
+
Regenerate or re-upload with
|
| 67 |
+
`python tools/export_checkpoints.py --out export/ --push <repo>`.
|
manifest.json
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"trousers_pn_depth": {
|
| 3 |
+
"file": "trousers_pn_depth.ckpt",
|
| 4 |
+
"epoch": 550,
|
| 5 |
+
"gib": 0.98
|
| 6 |
+
},
|
| 7 |
+
"trousers_pn_mesh": {
|
| 8 |
+
"file": "trousers_pn_mesh.ckpt",
|
| 9 |
+
"epoch": 900,
|
| 10 |
+
"gib": 0.98
|
| 11 |
+
},
|
| 12 |
+
"trousers_gnn_depth": {
|
| 13 |
+
"file": "trousers_gnn_depth.ckpt",
|
| 14 |
+
"epoch": 200,
|
| 15 |
+
"gib": 0.98
|
| 16 |
+
},
|
| 17 |
+
"trousers_gnn_knn": {
|
| 18 |
+
"file": "trousers_gnn_knn.ckpt",
|
| 19 |
+
"epoch": 200,
|
| 20 |
+
"gib": 0.98
|
| 21 |
+
},
|
| 22 |
+
"trousers_gnn_mesh": {
|
| 23 |
+
"file": "trousers_gnn_mesh.ckpt",
|
| 24 |
+
"epoch": 300,
|
| 25 |
+
"gib": 0.98
|
| 26 |
+
}
|
| 27 |
+
}
|
trousers_gnn_depth.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2d960c79cd3f498a51abbfdd1d7a93bf05d66804269979499b6afbdae5badc45
|
| 3 |
+
size 1050205214
|
trousers_gnn_knn.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:db32cd299db3ad583f98e68e477005eccd71e1efbd180b4b4d1910a7b8843561
|
| 3 |
+
size 1050204694
|
trousers_gnn_mesh.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8228e86ea8937ac0299b9b1333c7e42d519af3cd2fae1ca3123a6ff7e84dd27d
|
| 3 |
+
size 1050204954
|
trousers_pn_depth.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3bacf2432b71f6914c4f2636814f31d0b09fec3e76dd681b5db09697700ebbc7
|
| 3 |
+
size 1049741498
|
trousers_pn_mesh.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1a939d28ab6bca9ca820d092e1601002cae9d5f4fbae81004a8750eca031ab64
|
| 3 |
+
size 1049741306
|