|
Download README.md from anschrur/dit_block_3: direct link, hf CLI and curl.
- Browser
- Download file 6.73 kB
-
https://huggingface.co/anschrur/dit_block_3/resolve/main/README.md
- Command line
-
hf download hf://anschrur/dit_block_3/README.md
-
curl -L -o README.md https://huggingface.co/anschrur/dit_block_3/resolve/main/README.md
6.73 kB
| pipeline_tag: robotics | |
| tags: | |
| - imitation-learning | |
| - flow-matching | |
| - diffusion-policy | |
| - so-101 | |
| - robot-imitation-glue | |
| # dit_block_3 β DiT flow-matching policy, cup-into-ring | |
| Single-camera Diffusion-Transformer flow policy for the SO-10x task *pick up the cup, place it in the ring*. | |
| Trained on the same clean wrist retake as `dit_block_2`, grown from 30 to 101 teleoperated episodes, at a | |
| matched gradient-step budget. | |
| **This is not a `from_pretrained` checkpoint.** It is one raw `torch.save` dict written by | |
| `robot_imitation_glue.baseline.train_dit`. lerobot's policy registry cannot open it β there is no | |
| `config.json` / `model.safetensors` here. Load it through `DiTAgent` (below). | |
| ## File | |
| | file | bytes | md5 | | |
| |---|---|---| | |
| | `policy.ckpt` | 66,705,733 | `036a435230f9bff3f1f5e7ce430ac2a6` | | |
| Contents: `config`, `image_size`, `crop_ratio`, `model_state_dict` (16,660,576 params), | |
| `proprio_normalizer_state_dict`, `action_normalizer_state_dict`. The two normalisers ship with the weights β | |
| the model is unusable without them, and nothing else (no dataset paths, no credentials) is inside the file. | |
| Byte size is identical to `dit_block_2`'s because the architecture is unchanged; only the numbers differ. | |
| ## Task interface | |
| - **State / action**: 6-D absolute Feetech positions, dataset order | |
| `shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll, gripper`. Dims 0-4 are **degrees**; dim 5 is | |
| **% open** (the gripper servo is hard-wired to `RANGE_0_100`). Not one unit β do not scale the vector. | |
| - **Vision**: one camera, `observation.images.front`, recorded 480Γ640 AV1 β resized to 128Γ128 with a 0.9 centre crop. | |
| - **Chunk**: `prediction_horizon = 48` counts **frames**, and the dataset is **30 fps**, so one chunk is 1.6 s of | |
| motion. A control loop must therefore run at 30 fps; at 10 fps the same chunk stretches to 4.8 s. | |
| ## Architecture | |
| | | | | |
| |---|---| | |
| | hidden dim | 256 | | |
| | layers / heads | 4 / 4 | | |
| | camera encoder | resnet18, ImageNet-pretrained, frozen statistics shipped as `encoders.0.mean/std` | | |
| | prediction | flow matching, Euler sampler (10 steps is what the deployment script uses) | | |
| | params | 16,660,576 | | |
| Identical to `anschrur/dit_block_2` in every architectural respect, so the two are drop-in substitutes for each | |
| other at the deployment script. | |
| ## Training | |
| | | | | |
| |---|---| | |
| | dataset | [`anschrur/record-test_2_20261115_115045`](https://huggingface.co/datasets/anschrur/record-test_2_20261115_115045) β 101 episodes, 28,498 frames, 30 fps, 1 task | | |
| | optimiser steps | 41,385 = 93 epochs Γ floor(28,498 / 64), `drop_last=True` | | |
| | batch / optimiser | 64 / AdamW, lr 1e-4, weight decay 1e-4 | | |
| | schedule | 500-step linear warmup β cosine to 1 % of base (final lr 1e-6), grad-clip 10, seed 42 | | |
| | final train loss | 0.034032 (epoch 93/93); 0.091574 @ 20, 0.059593 @ 40, 0.043904 @ 60, 0.034347 @ 80 | | |
| | hardware / wall clock | 1Γ RTX 5090 (imec iLab.t GPULab cluster 9), 1 h 46 min | | |
| | written | 2026-11-15 17:04 UTC, `/project_ghent/dit/outputs/dit_cup_3/policy.ckpt` (GPULab job `dit`, `eeb9465c`) | | |
| The epoch count (93) is not inherited β it solves for the step total. Steps = epochs Γ floor(frames / batch), | |
| because the LR schedule is built from `epochs Γ len(dataloader)` and the loader drops the last partial batch. | |
| Holding steps at ~41 k makes this run a like-for-like exchange with `dit_block_2` (41,200 steps) on the same | |
| compute budget; 2.2Γ the frames therefore means each frame is visited 93 rather than 200 times. | |
| `--save-every 20` rewrites the *same* `policy.ckpt` path, so these are the epoch-93 weights only β five writes, | |
| one surviving file, no intermediate snapshots and no epoch sweep to plot. The dataset declares | |
| `splits {train: 0:101}`, so `train_loss` is a training-set number, not an evaluation result. | |
| ## What changed against `anschrur/dit_block_2` | |
| Same dataset directory, same wrist-clean labels, same architecture and hyperparameters. The dataset grew from | |
| 30 episodes / 13,218 frames to 101 episodes / 28,498 frames, and the epoch count dropped 200 β 93 to keep the | |
| step totals matched. Action normaliser (degrees for dims 0-4, % open for dim 5), measured out of the file: | |
| | dim | `dit_block_2` mean / std | `dit_block_3` mean / std | | |
| |---|---|---| | |
| | shoulder_pan | 0.79 / 16.15 | β1.61 / 16.72 | | |
| | shoulder_lift | 79.98 / 51.82 | 86.14 / 45.21 | | |
| | elbow_flex | β71.29 / 43.93 | β73.83 / 36.81 | | |
| | wrist_flex | 73.85 / 13.39 | 70.36 / 14.21 | | |
| | **wrist_roll** | **4.37 / 20.51** | **β4.31 / 19.38** | | |
| | gripper | 6.85 / 11.46 | 8.34 / 12.70 | | |
| `wrist_roll` stays near zero mean with std β 20 β that is the label-fix signature from `dit_block_2` surviving | |
| into the 71 new episodes, and it is the one row here that is a direct quality check rather than a distribution | |
| shift. The other five moved because the added episodes reach more of the workspace; every std is in the same | |
| band, so no joint collapsed or exploded. | |
| **Do not read the loss as a regression.** `dit_block_2` finished at 0.018505 and this at 0.034032, but the two | |
| numbers are over different frame sets at different visit counts: 13,218 frames seen 200Γ versus 28,498 frames | |
| seen 93Γ. The lower figure is the run that memorised its training set harder, not the better policy. The | |
| comparable statement is shape, not level β and the flattening here (0.034347 @ 80 β 0.034032 @ 93) says the | |
| optimiser was near exhausted by epoch 80, so 93 epochs is not leaving much on the table. | |
| ## Loading | |
| ```python | |
| from robot_imitation_glue.baseline.dit_agent import DiTAgent # plain clone on sys.path is enough | |
| agent = DiTAgent( | |
| checkpoint_path="policy.ckpt", | |
| camera_key_mapping={"front": "observation.images.front"}, # live camera key -> policy key | |
| state_key="observation.state", | |
| device="cpu", | |
| n_sampling_steps=10, | |
| ) | |
| ``` | |
| Closed-loop on an arm β the repo is public, so this needs no token: | |
| ```bash | |
| python dit_on_robot.py --repo-id anschrur/dit_block_3 --fps 30 --dry-run # arm must NOT move on this one | |
| ``` | |
| `dit_on_robot.py` (thesis glue, not published here) wires `DiTAgent` to lerobot's `SOFollower` with a wrap-aware | |
| per-step delta guard, because lerobot's own `max_relative_target` clamp mishandles the Β±180Β° `wrist_roll` boundary. | |
| ## Not measured | |
| No open-loop action MSE, no evaluation episodes, no success rate, and no head-to-head against `dit_block_2` | |
| beyond the normaliser stats above. Whether 2.2Γ the demonstrations buys closed-loop behaviour at matched step | |
| count is exactly what the next rollout on the arm has to answer β `dit_block_2` froze on the first real run, and | |
| that failure mode is not yet attributed to data volume, so this checkpoint inherits the question rather than | |
| settling it. | |