| --- |
| license: apache-2.0 |
| library_name: pytorch |
| pipeline_tag: robotics |
| tags: |
| - act |
| - action-chunking |
| - franka |
| - isaac-lab |
| - imitation-learning |
| datasets: |
| - Boyun7/GroundProbe-dataset |
| --- |
| |
| # GroundProbe ACT Checkpoints |
|
|
| Two Action Chunking with Transformers (ACT) policies trained on one cell of |
| the [GroundProbe demonstrations](https://huggingface.co/datasets/Boyun7/GroundProbe-dataset): |
| `quest_l2_cubes_pilot`, instruction T1 ("Pick the left red block and place it |
| in the bin") at the `clean` complexity, simulated Franka Panda in NVIDIA |
| Isaac Lab-Arena. |
|
|
| ACT has no language input, so these are **not grounding baselines**. They |
| exist to show that the demonstrations and the closed-loop evaluation path |
| support learning the skill, and they are useful as a reference point when |
| bringing up a new policy on the benchmark. |
|
|
| Code, scenes and the evaluation harness: <https://github.com/AndersonYu7/Benchmark> |
|
|
| ## Checkpoints |
|
|
| | folder | action encoding | dataset column | qpos input | best epoch | |
| |---|---|---|---|---:| |
| | `delta_pose/` | delta pose, 7-D | `action` | TCP pose, 7-D | 510 | |
| | `joint/` | absolute joint targets, 8-D | `action.joint` | 7 joint positions + total finger opening | 230 | |
|
|
| Each folder holds: |
|
|
| - `policy_best.ckpt`: the weights with the lowest validation loss |
| - `config.json`: architecture settings, the training/validation episode split and their layout seeds |
| - `dataset_stats.pkl`: the normalisation statistics the policy was trained with |
|
|
| `config.json` names the delta-pose encoding `"action_space": "tcp"`; that is |
| the `action` column of the dataset. |
|
|
| ## Results |
|
|
| Closed-loop rollouts under the benchmark success criterion (target inside the |
| bin for 30 consecutive control steps), at most 1000 control steps, executing |
| the first 25 actions of every 50-action chunk before re-querying the policy. |
|
|
| | policy | evaluation layouts | success | Wilson 95% | |
| |---|---|---:|---| |
| | delta pose | the 20 training-demonstration layouts | 17/20 (85%) | [64%, 95%] | |
| | joint | the 20 training-demonstration layouts | 16/20 (80%) | [58%, 92%] | |
| | joint | the 5 held-out validation layouts | 2/5 (40%) | [12%, 77%] | |
| | joint | 20 layouts outside the collection seed range | 5/20 (25%) | [11%, 47%] | |
|
|
| With 50 demonstrations of one cell, the drop on unseen layouts is expected. |
|
|
| ## Training |
|
|
| ResNet-18 backbone, 4 encoder / 7 decoder layers, hidden size 512, |
| feed-forward 3200, 8 heads, dropout 0.1, chunk size 50, both cameras |
| (`third_camera`, `wrist_camera`) at 240×240, AdamW with learning rate 1e-5 |
| (backbone 1e-5) and weight decay 1e-4, batch size 8, KL weight 10. |
| 45 training and 5 validation episodes; validation every 10 epochs; stopped |
| after 300 epochs without improvement. |
|
|
| ## Using them |
|
|
| The evaluation harness loads these through `script/policy_adapters.py` |
| (`ACTAdapter`), which reads `config.json` and `dataset_stats.pkl` from the |
| checkpoint's folder: |
|
|
| ```bash |
| hf download Boyun7/GroundProbe-ACT --local-dir act_ckpt |
| |
| python -m script.eval_policy --policy ACT --scene cubes \ |
| --ckpt act_ckpt/joint/policy_best.ckpt \ |
| --eval-episodes 20 --exec-horizon 25 --headless --out eval_out/act_joint \ |
| --environment l2_spatial_tasks.examples.manipulation.l2_spatial_env:L2SpatialEnv \ |
| --enable_cameras l2_spatial_env --embodiment franka --enable_cameras True |
| ``` |
|
|
| `config.json` records the dataset path as `dataset/quest_l2_cubes_pilot`, |
| relative to the repository root, and the harness reads the control rate and |
| the joint policy's starting configuration from it. Download the dataset into |
| `dataset/` at the repository root before evaluating. |
|
|
| ## License |
|
|
| Apache 2.0, matching the code and the dataset. |
|
|
