Robotics
LeRobot
robomimic
diffusion-policy
imitation-learning
variable-admittance
force-control
doosan
Instructions to use aleleanza/diffusion-policy-vac-pipe with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use aleleanza/diffusion-policy-vac-pipe with LeRobot:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Add model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,114 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: robomimic
|
| 4 |
+
pipeline_tag: robotics
|
| 5 |
+
tags:
|
| 6 |
+
- robotics
|
| 7 |
+
- diffusion-policy
|
| 8 |
+
- imitation-learning
|
| 9 |
+
- lerobot
|
| 10 |
+
- variable-admittance
|
| 11 |
+
- force-control
|
| 12 |
+
- doosan
|
| 13 |
+
datasets:
|
| 14 |
+
- aleleanza/vac-pipe-dual-cam
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# Diffusion Policy β VAC Pipe Insertion (input/output ablation)
|
| 18 |
+
|
| 19 |
+
Six **stock robomimic Diffusion Policy** checkpoints for the contact-rich
|
| 20 |
+
**pipe-insertion** task on a Doosan M0609 + Inspire RH56 hand under a **Variable
|
| 21 |
+
Admittance Controller (VAC)**. All trained on
|
| 22 |
+
[`aleleanza/vac-pipe-dual-cam`](https://huggingface.co/datasets/aleleanza/vac-pipe-dual-cam)
|
| 23 |
+
(202 episodes, 145,712 frames @ 50 Hz).
|
| 24 |
+
|
| 25 |
+
This repo is a clean **2 Γ 3 ablation**: two *output* representations (where the policy
|
| 26 |
+
sits relative to the admittance filter) Γ three *input* modalities.
|
| 27 |
+
|
| 28 |
+
## Variants (subfolders)
|
| 29 |
+
|
| 30 |
+
| Subfolder | Output (action) | Inputs (obs) | Action dim | Best val loss | @epoch |
|
| 31 |
+
|---|---|---|---:|---:|---:|
|
| 32 |
+
| `vac_pre_vis` | **pre**: `user_cmd[6]` + `K` + `ΞΆ` + `hand_binary` | vision | 9 | 0.2445 | 67 |
|
| 33 |
+
| `vac_pre_vis_wrench` | pre | vision + wrench | 9 | 0.2583 | 8 |
|
| 34 |
+
| `vac_pre_vis_wrench_state` | pre | vision + wrench + state | 9 | 0.2739 | 11 |
|
| 35 |
+
| `vac_post_vis` | **post**: `vel_cmd[6]` + `hand_binary` | vision | 7 | 0.0580 | 48 |
|
| 36 |
+
| `vac_post_vis_wrench` | post | vision + wrench | 7 | 0.0572 | 48 |
|
| 37 |
+
| `vac_post_vis_wrench_state` | post | vision + wrench + state | 7 | **0.0547** | 48 |
|
| 38 |
+
|
| 39 |
+
- **pre** = predict the operator command *before* admittance, including the compliance
|
| 40 |
+
command itself (`stiffness_cmd` K + damping ΞΆ) β the policy *learns to set compliance*.
|
| 41 |
+
Consumed downstream by the variable-admittance node.
|
| 42 |
+
- **post** = predict the Cartesian velocity the controller executed *after* admittance β
|
| 43 |
+
the policy bypasses admittance and drives velocity directly.
|
| 44 |
+
|
| 45 |
+
Each subfolder contains:
|
| 46 |
+
|
| 47 |
+
```
|
| 48 |
+
<variant>/
|
| 49 |
+
βββ best.pth # lowest validation loss
|
| 50 |
+
βββ last.pth # final epoch
|
| 51 |
+
βββ config.json # full robomimic training config
|
| 52 |
+
βββ action_stats.json # action normalization (min-max) + action_components + hand binarization
|
| 53 |
+
βββ dataset_summary.json # train/valid episode split + frame counts
|
| 54 |
+
```
|
| 55 |
+
|
| 56 |
+
## Architecture & training
|
| 57 |
+
|
| 58 |
+
- **Algorithm:** robomimic Diffusion Policy (DDPM noise-prediction loss; **no** anchor /
|
| 59 |
+
additional loss β stock).
|
| 60 |
+
- **Backbone:** conditional UNet `[128, 256, 512]`; ~89.4β90.0M params.
|
| 61 |
+
- **Horizons:** observation 2, action 4, prediction 8 (frame stack 2, seq length 8).
|
| 62 |
+
- **Image:** D405 wrist stream (`observation.images.camera`) β `front_rgb`, **84Γ84**.
|
| 63 |
+
- **Common:** 300 epochs, batch 16, lr 1e-4, DDPM 50/50 train/infer steps.
|
| 64 |
+
- **Split:** train episodes 0β171, valid 172β201.
|
| 65 |
+
- **Action normalization:** min-max to [β1, 1]; hand head is **binarized** from
|
| 66 |
+
`action.absolute[:, 8:12]` (mean β₯ 0.85 β open). See each `action_stats.json`.
|
| 67 |
+
|
| 68 |
+
### Reading the results
|
| 69 |
+
|
| 70 |
+
The **post** (vel_cmd) variants reach far lower validation loss (~0.055β0.058) than the
|
| 71 |
+
**pre** (user_cmd + K + ΞΆ) variants (~0.24β0.27): predicting executed velocity is an
|
| 72 |
+
easier target than predicting raw operator intent plus compliance. For `pre + wrench`
|
| 73 |
+
and `pre + wrench+state` the best validation arrives very early (epoch 8β11) while
|
| 74 |
+
training loss keeps dropping β a sign of **overfitting on proprioceptive inputs** at this
|
| 75 |
+
model/dataset size. Adding wrench/state did **not** help at this scale.
|
| 76 |
+
|
| 77 |
+
## Inference (ROS 2)
|
| 78 |
+
|
| 79 |
+
Run with the project's `robot_learning` real-time inference nodes (Doosan M0609 + Inspire
|
| 80 |
+
hand). The action contract determines the runner:
|
| 81 |
+
|
| 82 |
+
- **`pre` variants (9D)** β `diffusion_policy_vac_preimg_runner`. Publishes
|
| 83 |
+
`/delta_pose_cmd` + `/predicted_K` + `/predicted_zeta`, consumed by
|
| 84 |
+
`variable_admittance_node` (`variable_K:=true`). Add the Bota driver for `*_wrench*`.
|
| 85 |
+
- **`post` variants (7D)** β velocity runner: streams `vel_cmd` directly via the DSR
|
| 86 |
+
`speedl` interface (no admittance node).
|
| 87 |
+
|
| 88 |
+
```bash
|
| 89 |
+
# pre family (variable-stiffness path)
|
| 90 |
+
ros2 run robot_learning diffusion_policy_vac_preimg_runner \
|
| 91 |
+
--ros-args -p checkpoint:=/path/to/vac_pre_vis_wrench_state/best.pth
|
| 92 |
+
|
| 93 |
+
# post family (direct velocity path)
|
| 94 |
+
ros2 run robot_learning diffusion_policy_fixed_k_runner \
|
| 95 |
+
--ros-args -p checkpoint:=/path/to/vac_post_vis_wrench_state/best.pth -p mode:=vel_cmd
|
| 96 |
+
```
|
| 97 |
+
|
| 98 |
+
The binary hand head publishes to `/inspire_hand/left/cmd`; RGB observations come from
|
| 99 |
+
the compressed camera topic; `*_wrench*` variants subscribe to `/bota_ft_sensor/wrench`
|
| 100 |
+
(tared per trial). `action_stats.json` provides the exact normalization to undo at
|
| 101 |
+
inference.
|
| 102 |
+
|
| 103 |
+
## Intended use & limitations
|
| 104 |
+
|
| 105 |
+
- **Use:** research on force-aware / compliance-predicting imitation learning for
|
| 106 |
+
contact-rich insertion; a baseline ablation for VAC.
|
| 107 |
+
- **Limitations:** single task (`pipe_fixing`), single embodiment (M0609 + RH56), 84Γ84
|
| 108 |
+
vision, binary hand. The `pre` variants overfit at this scale. Not validated for
|
| 109 |
+
safety-critical or autonomous deployment.
|
| 110 |
+
|
| 111 |
+
## Related
|
| 112 |
+
|
| 113 |
+
- **Dataset:** [`aleleanza/vac-pipe-dual-cam`](https://huggingface.co/datasets/aleleanza/vac-pipe-dual-cam)
|
| 114 |
+
- **Collection:** *VAC β Pipe Insertion*
|