| --- |
| library_name: lerobot |
| license: mit |
| pipeline_tag: robotics |
| datasets: |
| - lerobot/aloha_sim_transfer_cube_human |
| tags: |
| - robotics |
| - lerobot |
| - act |
| - imitation-learning |
| - aloha |
| - simulation |
| model-index: |
| - name: act_aloha_sim_transfer_cube |
| results: |
| - task: |
| type: robotics |
| name: AlohaTransferCube-v0 (gym-aloha) |
| dataset: |
| type: lerobot/aloha_sim_transfer_cube_human |
| name: aloha_sim_transfer_cube_human |
| metrics: |
| - type: success_rate |
| value: 75.5 |
| name: success rate over 200 episodes (95% Wilson CI 69.1-80.9) |
| --- |
| |
| # ACT on ALOHA TransferCube, trained from scratch on an Apple M4 laptop |
|
|
| An [ACT](https://arxiv.org/abs/2304.13705) policy for the `AlohaTransferCube-v0` simulation task, |
| trained with [LeRobot](https://github.com/huggingface/lerobot) 0.7.0 on the |
| [`lerobot/aloha_sim_transfer_cube_human`](https://huggingface.co/datasets/lerobot/aloha_sim_transfer_cube_human) |
| dataset (50 human demonstrations). Training ran on an Apple M4 MacBook Pro (MPS, no NVIDIA GPU) for |
| 80k steps at batch size 8, about 10 hours in total. LeRobot's default ACT hyperparameters, nothing tuned. |
|
|
| Code, evaluation scripts, per-checkpoint results and training curves: |
| [github.com/ChadSaglam/lerobot-sim-transfer-cube](https://github.com/ChadSaglam/lerobot-sim-transfer-cube). |
|
|
| ## Evaluation |
|
|
| Same simulator, same episode seeds (`--seed=1000`), 200 episodes, 95% Wilson intervals from `lerobot-eval`. |
|
|
| | Policy | Training steps | Success | 95% CI | |
| |---|---|---|---| |
| | official [`lerobot/act_aloha_sim_transfer_cube_human`](https://huggingface.co/lerobot/act_aloha_sim_transfer_cube_human) | 80k | 81.0% (162/200) | 75.0-85.8 | |
| | this model (seed 1000) | 80k | 75.5% (151/200) | 69.1-80.9 | |
| | same run at 50k steps | 50k | 64.0% (128/200) | 57.1-70.3 | |
| | second seed (1001) at 50k steps | 50k | 62.0% (124/200) | 55.1-68.4 | |
|
|
| The intervals of this model and the official checkpoint overlap; the two are statistically |
| indistinguishable at this sample size. The 50k checkpoints are clearly below the baseline. |
|
|
| Two things the per-checkpoint evaluations showed (details in the GitHub README): the training loss did |
| not predict task success (two seeds with identical loss curves differed by 34 points in success at 10k |
| steps), and 50-episode evaluations cannot rank neighbouring checkpoints. |
|
|
| ## How to use |
|
|
| ```bash |
| pip install 'lerobot[aloha]==0.7.0' |
| |
| lerobot-eval \ |
| --policy.path=ChadSaglam/act_aloha_sim_transfer_cube \ |
| --env.type=aloha \ |
| --env.task=AlohaTransferCube-v0 \ |
| --eval.n_episodes=200 \ |
| --eval.batch_size=10 \ |
| --seed=1000 \ |
| --policy.device=cuda # mps on Apple Silicon; add --eval.use_async_envs=false on macOS |
| ``` |
|
|
| ## Training |
|
|
| ```bash |
| lerobot-train \ |
| --policy.type=act \ |
| --policy.device=mps \ |
| --policy.push_to_hub=false \ |
| --dataset.repo_id=lerobot/aloha_sim_transfer_cube_human \ |
| --env.type=aloha \ |
| --env.task=AlohaTransferCube-v0 \ |
| --env_eval_freq=0 \ |
| --seed=1000 \ |
| --steps=50000 \ |
| --batch_size=8 \ |
| --save_freq=10000 \ |
| --output_dir=outputs/train/act_transfer_cube_seed1000 |
| # then resumed from the 50k checkpoint with --resume=true --steps=80000 |
| ``` |
|
|
| - Policy: ACT, ResNet-18 backbone (ImageNet init), chunk size 100, 100 action steps, VAE on, KL weight 10, |
| AdamW, learning rate 1e-5, batch size 8. `train_config.json` in this repository has every setting. |
| - Throughput on the M4: 2.2 steps/s. 50k steps in 6 h 22 min, the 30k-step extension in 3 h 47 min. |
| - Checkpoint format: LeRobot 0.7.0 processor pipeline (`policy_preprocessor.json`, `policy_postprocessor.json`). |
|
|
| ## Limitations |
|
|
| - Simulation only; one task; one evaluation seed set. Another `--seed` would move every number by a few points. |
| - Success is the `gym-aloha` definition (cube lifted by the right arm and transferred to the left gripper). |
| - No hyperparameter tuning; LeRobot defaults throughout. |
|
|
