---
license: mit
library_name: lerobot
pipeline_tag: robotics
tags:
- LeRobot
- diffusion-policy
- imitation-learning
- delta-robot
- pick-and-place
- real-world
datasets:
- KirkHaskell/delta_robot_bolt_pick_place
---

# Diffusion Policy: delta robot bolt pick-and-place (100 demos, 50k steps), 10/10 on the real robot

A [Diffusion Policy](https://arxiv.org/abs/2303.04137) trained with LeRobot 0.5.0 on all 100
demonstrations of [`KirkHaskell/delta_robot_bolt_pick_place`](https://huggingface.co/datasets/KirkHaskell/delta_robot_bolt_pick_place).
It drives a home-built 3-DOF delta robot with an electromagnet gripper: it picks up a standing bolt
and drops it into a cup.

**Real-robot result: 10/10 successes** (95% Wilson CI 72–100%). It succeeded at all 5 bolt
positions, including one partly hidden by the cup and one outside the demonstrated area, and every
trial needed a single grab attempt. It was the best of the 24 models in the study, and subjectively
the most impressive. Full study: <https://github.com/KirkEHaskell/delta-robot-act-vs-diffusion>

## ⚠️ This is a modified Diffusion Policy: the joint-state input must be zeroed

LeRobot's default Diffusion Policy failed on this robot. Trained on leader–follower teleoperation
data, it learned to copy its own joint state instead of using the cameras, and it never left the
start pose. This model differs from the defaults in four ways:

| Setting | LeRobot default | This model |
|---|---|---|
| Vision backbone | ResNet-18 from scratch, GroupNorm | ResNet-18 ImageNet-pretrained, BatchNorm |
| Augmentation | none | random crop 216×288 |
| Observation frames | 2 | 1 |
| Joint state | used | **zeroed after normalization, in training and at inference** |
| Horizon / executed actions | 16 / 8 | 48 / 24 (1.6 s / 0.8 s at 30 Hz) |

The checkpoint's config cannot express "state blinded." The model still has an `observation.state`
input, but it was trained with that input set to zeros. **At inference, zero
`batch["observation.state"]` after the preprocessor and before `select_action`.** The repo's
`robot/run_policy.py` does this automatically: this folder contains a `BLIND_STATE` marker file
that the script detects.

```python
batch = preprocessor(obs)
batch["observation.state"] = torch.zeros_like(batch["observation.state"])
action = postprocessor(policy.select_action(batch))
```

| | |
|---|---|
| Inputs | 3 cameras (`side`, `top_right`, `top_left`, 240×320); the joint state is present but zeroed |
| Output | 4-D action: 3 absolute joint targets (deg) + magnet (on if > 0.5) |
| Training | 50k steps, cosine LR, batch 32, seed 1000, fp32, one RTX 3090 (~3 h), with `BLIND_STATE=1` via `training/lerobot_train_fixed.py` |
| Inference in the study | DDPM, all 100 denoising steps, RTX 3090: ~0.76 s per 24-action plan. The robot pauses while it plans |
| CPU | DDPM-100 takes ~18 s per plan. DDIM with 10 steps takes ~2 s and matched DDPM-100 within ~1° offline, but was not used for the scored trials |

## Use

```bash
hf download KirkHaskell/diffusion_delta_robot_bolt_pick_place --local-dir models/diffusion
cd robot && python run_policy.py dry --ckpt ../models/diffusion
```

**Limitations:** this model is specific to one robot, camera placement, scene and camera colour
settings (two cameras have a fixed white balance). It is not real-time without a GPU. Undertrained
versions of this configuration tend to stall rather than act.