model card + handoff
Browse files
README.md
ADDED
|
@@ -0,0 +1,92 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# BEST (round 1) — previous best stacking policy (superseded by doug-act-stack-best2)
|
| 2 |
+
|
| 3 |
+
act_stack_v3_dag60k step 10000: 60k + 10k fine-tune on 50 demos + 72 round-1 DAgger corrections. Grabs all 3 blocks, slow/no release. Use with ensembling 0.01.
|
| 4 |
+
|
| 5 |
+
# Block stacking: BEST2 is the policy to build on (handoff, 2026-11-18)
|
| 6 |
+
|
| 7 |
+
**For Michelle (and her Claude): start every new stacking experiment from BEST2.** It is the best real-arm
|
| 8 |
+
stacking policy we have. Don't retrain from scratch or from the 60k base: round 3 below did that and got
|
| 9 |
+
worse on the arm even though its val loss was the lowest.
|
| 10 |
+
|
| 11 |
+
## What to download
|
| 12 |
+
|
| 13 |
+
| What | Hugging Face | Local (Doug's laptop) |
|
| 14 |
+
|---|---|---|
|
| 15 |
+
| **BEST2 policy** (start here) | model `hrl-imerse/doug-act-stack-best2` | `task2_v3/outputs/BEST2_stack_dag60k_r2_10k/pretrained_model` (read-only) |
|
| 16 |
+
| BEST2's exact training set | dataset `hrl-imerse/doug-stack-v3-best2-train` (154 eps: 72 r1 + 32 r2 corrections, then the 50 demos) | `task2_v3/data/so101_block_stack_v3_dag60k_r2` |
|
| 17 |
+
| Original 50 teleop demos | dataset `hrl-imerse/doug-stack-v3-trimmed` | `task2_v3/data/so101_block_stack_v3_trimmed` |
|
| 18 |
+
| DAgger corrections (raw, with `intervention` column) | datasets `hrl-imerse/doug-stack-dagger-r1`, `-r2`, `-r3` | `task2_v3/data/rollout_dagger_stack60k`, `rollout_dagger_stack_dag60k10k`, `rollout_dagger_stack_r2_10k` |
|
| 19 |
+
| BEST (round 1, previous best) | model `hrl-imerse/doug-act-stack-best1` | `task2_v3/outputs/BEST_stack_dag60k_10k` |
|
| 20 |
+
|
| 21 |
+
Task string: `Stack the orange, green and purple blocks into a tower.` Features: `observation.state` (6),
|
| 22 |
+
`action` (6, absolute joint targets in degrees + gripper 0-100), `observation.images.wrist` (480x640x3, 30 fps).
|
| 23 |
+
Wrist camera only. Stacking order is always red/orange -> green -> purple; the three blocks start in a line of
|
| 24 |
+
three slots right of the tray (far/middle/near) with colours shuffled.
|
| 25 |
+
|
| 26 |
+
## How BEST2 was made
|
| 27 |
+
|
| 28 |
+
1. `act_stack_v3` (ACT, chunk 100, batch 8, image aug on) trained 100k steps on the 50 trimmed demos; **60k**
|
| 29 |
+
was the best real-arm checkpoint (val loss 0.2375).
|
| 30 |
+
2. **DAgger round 1** on 60k: 72 corrections (grasps, wrong block). Fine-tune **from 60k**, 10k steps on
|
| 31 |
+
[72 corrections + 50 demos], each once -> `act_stack_v3_dag60k` 10k = **BEST** (val 0.2275). On the arm: grabbed
|
| 32 |
+
all 3 blocks, stacked green, but held blocks on the stack 20-40 s without opening.
|
| 33 |
+
3. **DAgger round 2** on BEST: 32 short release corrections (open + lift ~0.5-1 s after the block lands).
|
| 34 |
+
Fine-tune **from 60k**, 10k steps on [72 r1 + 32 r2 + 50 demos] -> `act_stack_v3_dag60k_r2` 10k = **BEST2**
|
| 35 |
+
(val 0.2295). On the arm: releases after 0.6-0.8 s like the demos; "almost perfect, but not quite".
|
| 36 |
+
4. DAgger round 3 on BEST2: 24 corrections. Retrained **from 60k** on r1+r2+r3+demos -> val 0.2251 (lowest), but
|
| 37 |
+
**clearly worse on the arm**. Kept only for reference (`doug-stack-dagger-r3`).
|
| 38 |
+
|
| 39 |
+
Fine-tune recipe (task2_v3, ~10 min on an RTX 3500 Ada laptop, ~17 steps/s):
|
| 40 |
+
|
| 41 |
+
```bash
|
| 42 |
+
../sim/.venv/bin/python train_fast.py \
|
| 43 |
+
--policy.path=<BEST2 pretrained_model> \
|
| 44 |
+
--policy.device=cuda --policy.push_to_hub=false \
|
| 45 |
+
--dataset.repo_id=local/<merged> --dataset.root=$PWD/data/<merged> \
|
| 46 |
+
--dataset.eval_split=<5/n_episodes, so the last 5 episodes = held-out demos> --eval_steps=2500 \
|
| 47 |
+
--dataset.image_transforms.enable=true --accelerator.mixed_precision=bf16 \
|
| 48 |
+
--batch_size=8 --steps=5000 --save_freq=1000 --seed=1000 \
|
| 49 |
+
--output_dir=$PWD/outputs/train/<run> --job_name=<run> --trackio.enable=false
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
`train_fast.py` = lerobot-train with `torch.compile` on the ACT model (needs the `__main__` guard, it has one).
|
| 53 |
+
Merge datasets with `lerobot.datasets.dataset_tools.merge_datasets`, after `remove_feature(..., "intervention")`
|
| 54 |
+
on DAgger data; put the 50 demos LAST so the val split is the same 5 held-out demos every time.
|
| 55 |
+
|
| 56 |
+
## Adding sim data to BEST2 (Michelle's plan)
|
| 57 |
+
|
| 58 |
+
- Start from BEST2 (`--policy.path`), not from scratch: fine-tune briefly (2.5-5k steps, save every 1k).
|
| 59 |
+
- Keep all of BEST2's real data in the mix (`doug-stack-v3-best2-train`) so it doesn't forget; add the sim
|
| 60 |
+
episodes on top. Sim frames must match the real format: 640x480 wrist view with the real lens distortion,
|
| 61 |
+
same state/action units and joint order, same task string. Antony's sim (`assignment2/sim`, `--task stack`)
|
| 62 |
+
writes this format.
|
| 63 |
+
- The normalizer stats come from BEST2's checkpoint. If sim data shifts the state/action ranges a lot, check
|
| 64 |
+
`policy_preprocessor_step_3_normalizer_processor.safetensors` before trusting results.
|
| 65 |
+
- **Judge on the real arm, not val loss.** Run against BEST2 under the same conditions.
|
| 66 |
+
|
| 67 |
+
## Running a policy on the arm
|
| 68 |
+
|
| 69 |
+
```bash
|
| 70 |
+
cd assignment2/task2_v3
|
| 71 |
+
../sim/.venv/bin/python go_home.py 5 # ALWAYS first: slow move to the folded rest pose
|
| 72 |
+
EPISODE_S=90 ./3_eval_rollouts.sh <variant> <pretrained_model dir> ens 1
|
| 73 |
+
```
|
| 74 |
+
|
| 75 |
+
- Ensembling (`ens`, coeff 0.01) for every stacking checkpoint. 90 s episode limit.
|
| 76 |
+
- Keys: -> ends and saves, <- discards and redoes. LeRobot's key listener is **global**: typing anywhere (even in a
|
| 77 |
+
chat window) presses its keys and ends/discards trials.
|
| 78 |
+
- Check the wrist camera isn't black first (a blocked lens made several trials run blind).
|
| 79 |
+
|
| 80 |
+
DAgger collection: `./dagger_loop.sh <name> <pretrained_model dir>` (home -> RIGHT ARROW starts a tower ->
|
| 81 |
+
SPACE pause, TAB start/end a correction with the leader arm, ESC end tower -> home; Q quits). Corrections append
|
| 82 |
+
to `data/rollout_dagger_<name>`. Leader calibration id `so_leader`. If a USB glitch kills a tower, the loop goes
|
| 83 |
+
home and waits for RIGHT ARROW again.
|
| 84 |
+
|
| 85 |
+
## Lessons
|
| 86 |
+
|
| 87 |
+
- DAgger rounds 1 and 2 helped; each targeted one failure (block choice/grasp, then release).
|
| 88 |
+
- Round 3 hurt. Retraining from 60k each round adds run-to-run variance. Next time fine-tune from the current
|
| 89 |
+
best (BEST2) for a few thousand steps instead, and keep each round's corrections focused on one failure.
|
| 90 |
+
- Val loss on the 5 held-out demos does not predict arm success here (round 3 had the lowest).
|
| 91 |
+
- Release failures came from the arm stopping on the stack ~2-4 deg short of the commanded elbow angle; the
|
| 92 |
+
policy waited for a state it never reached. Short "open + lift" corrections fixed it.
|