dit_block_2 / README.md
anschrur's picture
Add model card
819ba2a verified
|
Raw History Blame Contribute Delete
5.11 kB
metadata
pipeline_tag: robotics
tags:
  - imitation-learning
  - flow-matching
  - diffusion-policy
  - so-101
  - robot-imitation-glue

dit_block_2 β€” DiT flow-matching policy, cup-into-ring

Single-camera Diffusion-Transformer flow policy for the SO-10x task pick up the cup, place it in the ring. Trained on the 2026-11-15 clean wrist retake of the cup dataset.

This is not a from_pretrained checkpoint. It is one raw torch.save dict written by robot_imitation_glue.baseline.train_dit. lerobot's policy registry cannot open it β€” there is no config.json / model.safetensors here. Load it through DiTAgent (below).

File

file bytes md5
policy.ckpt 66,705,733 669479fdbcfa196c7c251c4bf9e7c549

Contents: config, image_size, crop_ratio, model_state_dict (16,660,576 params), proprio_normalizer_state_dict, action_normalizer_state_dict. The two normalisers ship with the weights β€” the model is unusable without them, and nothing else (no dataset paths, no credentials) is inside the file.

Task interface

  • State / action: 6-D absolute Feetech positions, dataset order shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper. Dims 0-4 are degrees; dim 5 is % open (the gripper servo is hard-wired to RANGE_0_100). Not one unit β€” do not scale the vector.
  • Vision: one camera, observation.images.front, recorded 480Γ—640 AV1 β†’ resized to 128Γ—128 with a 0.9 centre crop.
  • Chunk: prediction_horizon = 48 counts frames, and the dataset is 30 fps, so one chunk is 1.6 s of motion. A control loop must therefore run at 30 fps; at 10 fps the same chunk stretches to 4.8 s.

Architecture

hidden dim 256
layers / heads 4 / 4
camera encoder resnet18, ImageNet-pretrained, frozen statistics shipped as encoders.0.mean/std
prediction flow matching, Euler sampler (10 steps is what the deployment script uses)
params 16,660,576

Training

dataset anschrur/record-test_2_20261115_115045 β€” 30 episodes, 13,218 frames, 30 fps, 1 task
optimiser steps 41,200 = 200 epochs Γ— floor(13,218 / 64), drop_last=True
batch / optimiser 64 / AdamW, lr 1e-4, weight decay 1e-4
schedule 500-step linear warmup β†’ cosine to 1 % of base (final lr 1e-6), grad-clip 10, seed 42
final train loss 0.018505 (epoch 200/200)
hardware / wall clock 1Γ— RTX 5090 (imec iLab.t GPULab cluster 9), 1.7 h
written 2026-11-15 13:28, /project_ghent/dit/outputs/dit_cup_2/policy.ckpt

--save-every 20 rewrites the same policy.ckpt path, so these are the epoch-200 weights only β€” no intermediate snapshots and no epoch sweep to plot. The dataset declares no held-out split, so train_loss is a training-set number, not an evaluation result.

What changed against anschrur/dit-cup

Identical architecture and hyperparameters, with the epoch count raised 30 β†’ 200 so the step totals match (40,710 then vs 41,200 now); the dataset is the only variable. dit-cup trained on record-test_2, whose wrist_roll labels carry a ~170Β° offset in 32 of 200 episodes. The retake removed it, and the difference shows up in the action normaliser stored inside the file (degrees for dims 0-4, % open for dim 5):

dim dit-cup mean / std dit_block_2 mean / std
shoulder_pan βˆ’2.94 / 18.42 0.79 / 16.15
shoulder_lift βˆ’20.40 / 46.84 79.98 / 51.82
elbow_flex 28.94 / 36.92 βˆ’71.29 / 43.93
wrist_flex 60.48 / 14.90 73.85 / 13.39
wrist_roll βˆ’144.60 / 69.53 4.37 / 20.51
gripper 8.01 / 13.20 6.85 / 11.46

wrist_roll std falls 69.5 β†’ 20.5 once the offset is gone. The other dims move too because the retake is a different 30-episode recording, not a relabelled subset of the old one β€” those four joints are not directly comparable.

Loading

from robot_imitation_glue.baseline.dit_agent import DiTAgent  # plain clone on sys.path is enough

agent = DiTAgent(
    checkpoint_path="policy.ckpt",
    camera_key_mapping={"front": "observation.images.front"},  # live camera key -> policy key
    state_key="observation.state",
    device="cpu",
    n_sampling_steps=10,
)

Closed-loop on an arm β€” the repo is public, so this needs no token:

python dit_on_robot.py --repo-id anschrur/dit_block_2 --fps 30 --dry-run   # arm must NOT move on this one

dit_on_robot.py (thesis glue, not published here) wires DiTAgent to lerobot's SOFollower with a wrap-aware per-step delta guard, because lerobot's own max_relative_target clamp mishandles the Β±180Β° wrist_roll boundary.

Not measured

No open-loop action MSE, no evaluation episodes, no success rate, and no head-to-head against dit-cup beyond the normaliser stats above. The wrist_roll label defect in the previous dataset is the reason this run exists; whether it improves closed-loop behaviour is what the next robot rollout has to answer.