dit_block_2 / README.md
anschrur's picture
Add model card
819ba2a verified
|
Raw History Blame Contribute Delete
5.11 kB
---
pipeline_tag: robotics
tags:
- imitation-learning
- flow-matching
- diffusion-policy
- so-101
- robot-imitation-glue
---
# dit_block_2 β€” DiT flow-matching policy, cup-into-ring
Single-camera Diffusion-Transformer flow policy for the SO-10x task *pick up the cup, place it in the ring*.
Trained on the 2026-11-15 clean wrist retake of the cup dataset.
**This is not a `from_pretrained` checkpoint.** It is one raw `torch.save` dict written by
`robot_imitation_glue.baseline.train_dit`. lerobot's policy registry cannot open it β€” there is no
`config.json` / `model.safetensors` here. Load it through `DiTAgent` (below).
## File
| file | bytes | md5 |
|---|---|---|
| `policy.ckpt` | 66,705,733 | `669479fdbcfa196c7c251c4bf9e7c549` |
Contents: `config`, `image_size`, `crop_ratio`, `model_state_dict` (16,660,576 params),
`proprio_normalizer_state_dict`, `action_normalizer_state_dict`. The two normalisers ship with the weights β€”
the model is unusable without them, and nothing else (no dataset paths, no credentials) is inside the file.
## Task interface
- **State / action**: 6-D absolute Feetech positions, dataset order
`shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper`. Dims 0-4 are **degrees**; dim 5 is
**% open** (the gripper servo is hard-wired to `RANGE_0_100`). Not one unit β€” do not scale the vector.
- **Vision**: one camera, `observation.images.front`, recorded 480Γ—640 AV1 β†’ resized to 128Γ—128 with a 0.9 centre crop.
- **Chunk**: `prediction_horizon = 48` counts **frames**, and the dataset is **30 fps**, so one chunk is 1.6 s of
motion. A control loop must therefore run at 30 fps; at 10 fps the same chunk stretches to 4.8 s.
## Architecture
| | |
|---|---|
| hidden dim | 256 |
| layers / heads | 4 / 4 |
| camera encoder | resnet18, ImageNet-pretrained, frozen statistics shipped as `encoders.0.mean/std` |
| prediction | flow matching, Euler sampler (10 steps is what the deployment script uses) |
| params | 16,660,576 |
## Training
| | |
|---|---|
| dataset | [`anschrur/record-test_2_20261115_115045`](https://huggingface.co/datasets/anschrur/record-test_2_20261115_115045) β€” 30 episodes, 13,218 frames, 30 fps, 1 task |
| optimiser steps | 41,200 = 200 epochs Γ— floor(13,218 / 64), `drop_last=True` |
| batch / optimiser | 64 / AdamW, lr 1e-4, weight decay 1e-4 |
| schedule | 500-step linear warmup β†’ cosine to 1 % of base (final lr 1e-6), grad-clip 10, seed 42 |
| final train loss | 0.018505 (epoch 200/200) |
| hardware / wall clock | 1Γ— RTX 5090 (imec iLab.t GPULab cluster 9), 1.7 h |
| written | 2026-11-15 13:28, `/project_ghent/dit/outputs/dit_cup_2/policy.ckpt` |
`--save-every 20` rewrites the *same* `policy.ckpt` path, so these are the epoch-200 weights only β€” no intermediate
snapshots and no epoch sweep to plot. The dataset declares no held-out split, so `train_loss` is a training-set
number, not an evaluation result.
## What changed against `anschrur/dit-cup`
Identical architecture and hyperparameters, with the epoch count raised 30 β†’ 200 so the **step** totals match
(40,710 then vs 41,200 now); the dataset is the only variable. `dit-cup` trained on `record-test_2`, whose
`wrist_roll` labels carry a ~170Β° offset in 32 of 200 episodes. The retake removed it, and the difference shows up
in the action normaliser stored inside the file (degrees for dims 0-4, % open for dim 5):
| dim | `dit-cup` mean / std | `dit_block_2` mean / std |
|---|---|---|
| shoulder_pan | βˆ’2.94 / 18.42 | 0.79 / 16.15 |
| shoulder_lift | βˆ’20.40 / 46.84 | 79.98 / 51.82 |
| elbow_flex | 28.94 / 36.92 | βˆ’71.29 / 43.93 |
| wrist_flex | 60.48 / 14.90 | 73.85 / 13.39 |
| **wrist_roll** | **βˆ’144.60 / 69.53** | **4.37 / 20.51** |
| gripper | 8.01 / 13.20 | 6.85 / 11.46 |
`wrist_roll` std falls 69.5 β†’ 20.5 once the offset is gone. The other dims move too because the retake is a
different 30-episode recording, not a relabelled subset of the old one β€” those four joints are not directly
comparable.
## Loading
```python
from robot_imitation_glue.baseline.dit_agent import DiTAgent # plain clone on sys.path is enough
agent = DiTAgent(
checkpoint_path="policy.ckpt",
camera_key_mapping={"front": "observation.images.front"}, # live camera key -> policy key
state_key="observation.state",
device="cpu",
n_sampling_steps=10,
)
```
Closed-loop on an arm β€” the repo is public, so this needs no token:
```bash
python dit_on_robot.py --repo-id anschrur/dit_block_2 --fps 30 --dry-run # arm must NOT move on this one
```
`dit_on_robot.py` (thesis glue, not published here) wires `DiTAgent` to lerobot's `SOFollower` with a wrap-aware
per-step delta guard, because lerobot's own `max_relative_target` clamp mishandles the Β±180Β° `wrist_roll` boundary.
## Not measured
No open-loop action MSE, no evaluation episodes, no success rate, and no head-to-head against `dit-cup` beyond the
normaliser stats above. The wrist_roll label defect in the previous dataset is the reason this run exists; whether
it improves closed-loop behaviour is what the next robot rollout has to answer.