Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
pipeline_tag: robotics
|
| 5 |
+
tags:
|
| 6 |
+
- act
|
| 7 |
+
- action-chunking
|
| 8 |
+
- franka
|
| 9 |
+
- isaac-lab
|
| 10 |
+
- imitation-learning
|
| 11 |
+
datasets:
|
| 12 |
+
- Boyun7/GroundProbe-dataset
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
# GroundProbe ACT Checkpoints
|
| 16 |
+
|
| 17 |
+
Two Action Chunking with Transformers (ACT) policies trained on one cell of
|
| 18 |
+
the [GroundProbe demonstrations](https://huggingface.co/datasets/Boyun7/GroundProbe-dataset):
|
| 19 |
+
`quest_l2_cubes_pilot`, instruction T1 ("Pick the left red block and place it
|
| 20 |
+
in the bin") at the `clean` complexity, simulated Franka Panda in NVIDIA
|
| 21 |
+
Isaac Lab-Arena.
|
| 22 |
+
|
| 23 |
+
ACT has no language input, so these are **not grounding baselines**. They
|
| 24 |
+
exist to show that the demonstrations and the closed-loop evaluation path
|
| 25 |
+
support learning the skill, and they are useful as a reference point when
|
| 26 |
+
bringing up a new policy on the benchmark.
|
| 27 |
+
|
| 28 |
+
Code, scenes and the evaluation harness: <https://github.com/AndersonYu7/Benchmark>
|
| 29 |
+
|
| 30 |
+
## Checkpoints
|
| 31 |
+
|
| 32 |
+
| folder | action encoding | dataset column | qpos input | best epoch |
|
| 33 |
+
|---|---|---|---|---:|
|
| 34 |
+
| `delta_pose/` | delta pose, 7-D | `action` | TCP pose, 7-D | 510 |
|
| 35 |
+
| `joint/` | absolute joint targets, 8-D | `action.joint` | 7 joint positions + total finger opening | 230 |
|
| 36 |
+
|
| 37 |
+
Each folder holds:
|
| 38 |
+
|
| 39 |
+
- `policy_best.ckpt`: the weights with the lowest validation loss
|
| 40 |
+
- `config.json`: architecture settings, the training/validation episode split and their layout seeds
|
| 41 |
+
- `dataset_stats.pkl`: the normalisation statistics the policy was trained with
|
| 42 |
+
|
| 43 |
+
`config.json` names the delta-pose encoding `"action_space": "tcp"`; that is
|
| 44 |
+
the `action` column of the dataset.
|
| 45 |
+
|
| 46 |
+
## Results
|
| 47 |
+
|
| 48 |
+
Closed-loop rollouts under the benchmark success criterion (target inside the
|
| 49 |
+
bin for 30 consecutive control steps), at most 1000 control steps, executing
|
| 50 |
+
the first 25 actions of every 50-action chunk before re-querying the policy.
|
| 51 |
+
|
| 52 |
+
| policy | evaluation layouts | success | Wilson 95% |
|
| 53 |
+
|---|---|---:|---|
|
| 54 |
+
| delta pose | the 20 training-demonstration layouts | 17/20 (85%) | [64%, 95%] |
|
| 55 |
+
| joint | the 20 training-demonstration layouts | 16/20 (80%) | [58%, 92%] |
|
| 56 |
+
| joint | the 5 held-out validation layouts | 2/5 (40%) | [12%, 77%] |
|
| 57 |
+
| joint | 20 layouts outside the collection seed range | 5/20 (25%) | [11%, 47%] |
|
| 58 |
+
|
| 59 |
+
With 50 demonstrations of one cell, the drop on unseen layouts is expected.
|
| 60 |
+
|
| 61 |
+
## Training
|
| 62 |
+
|
| 63 |
+
ResNet-18 backbone, 4 encoder / 7 decoder layers, hidden size 512,
|
| 64 |
+
feed-forward 3200, 8 heads, dropout 0.1, chunk size 50, both cameras
|
| 65 |
+
(`third_camera`, `wrist_camera`) at 240×240, AdamW with learning rate 1e-5
|
| 66 |
+
(backbone 1e-5) and weight decay 1e-4, batch size 8, KL weight 10.
|
| 67 |
+
45 training and 5 validation episodes; validation every 10 epochs; stopped
|
| 68 |
+
after 300 epochs without improvement.
|
| 69 |
+
|
| 70 |
+
## Using them
|
| 71 |
+
|
| 72 |
+
The evaluation harness loads these through `script/policy_adapters.py`
|
| 73 |
+
(`ACTAdapter`), which reads `config.json` and `dataset_stats.pkl` from the
|
| 74 |
+
checkpoint's folder:
|
| 75 |
+
|
| 76 |
+
```bash
|
| 77 |
+
hf download Boyun7/GroundProbe-ACT --local-dir act_ckpt
|
| 78 |
+
|
| 79 |
+
python -m script.eval_policy --policy ACT --scene cubes \
|
| 80 |
+
--ckpt act_ckpt/joint/policy_best.ckpt \
|
| 81 |
+
--eval-episodes 20 --exec-horizon 25 --headless --out eval_out/act_joint \
|
| 82 |
+
--environment l2_spatial_tasks.examples.manipulation.l2_spatial_env:L2SpatialEnv \
|
| 83 |
+
--enable_cameras l2_spatial_env --embodiment franka --enable_cameras True
|
| 84 |
+
```
|
| 85 |
+
|
| 86 |
+
`config.json` records the dataset path as `dataset/quest_l2_cubes_pilot`,
|
| 87 |
+
relative to the repository root, and the harness reads the control rate and
|
| 88 |
+
the joint policy's starting configuration from it. Download the dataset into
|
| 89 |
+
`dataset/` at the repository root before evaluating.
|
| 90 |
+
|
| 91 |
+
## License
|
| 92 |
+
|
| 93 |
+
Apache 2.0, matching the code and the dataset.
|