README.md

metadata

library_name: lerobot
license: mit
pipeline_tag: robotics
datasets:
  - lerobot/aloha_sim_transfer_cube_human
tags:
  - robotics
  - lerobot
  - act
  - imitation-learning
  - aloha
  - simulation
model-index:
  - name: act_aloha_sim_transfer_cube
    results:
      - task:
          type: robotics
          name: AlohaTransferCube-v0 (gym-aloha)
        dataset:
          type: lerobot/aloha_sim_transfer_cube_human
          name: aloha_sim_transfer_cube_human
        metrics:
          - type: success_rate
            value: 75.5
            name: success rate over 200 episodes (95% Wilson CI 69.1-80.9)

ACT on ALOHA TransferCube, trained from scratch on an Apple M4 laptop

An ACT policy for the AlohaTransferCube-v0 simulation task, trained with LeRobot 0.7.0 on the lerobot/aloha_sim_transfer_cube_human dataset (50 human demonstrations). Training ran on an Apple M4 MacBook Pro (MPS, no NVIDIA GPU) for 80k steps at batch size 8, about 10 hours in total. LeRobot's default ACT hyperparameters, nothing tuned.

Code, evaluation scripts, per-checkpoint results and training curves: github.com/ChadSaglam/lerobot-sim-transfer-cube.

Evaluation

Same simulator, same episode seeds (--seed=1000), 200 episodes, 95% Wilson intervals from lerobot-eval.

Policy	Training steps	Success	95% CI
official lerobot/act_aloha_sim_transfer_cube_human	80k	81.0% (162/200)	75.0-85.8
this model (seed 1000)	80k	75.5% (151/200)	69.1-80.9
same run at 50k steps	50k	64.0% (128/200)	57.1-70.3
second seed (1001) at 50k steps	50k	62.0% (124/200)	55.1-68.4

The intervals of this model and the official checkpoint overlap; the two are statistically indistinguishable at this sample size. The 50k checkpoints are clearly below the baseline.

Two things the per-checkpoint evaluations showed (details in the GitHub README): the training loss did not predict task success (two seeds with identical loss curves differed by 34 points in success at 10k steps), and 50-episode evaluations cannot rank neighbouring checkpoints.

How to use

pip install 'lerobot[aloha]==0.7.0'

lerobot-eval \
  --policy.path=ChadSaglam/act_aloha_sim_transfer_cube \
  --env.type=aloha \
  --env.task=AlohaTransferCube-v0 \
  --eval.n_episodes=200 \
  --eval.batch_size=10 \
  --seed=1000 \
  --policy.device=cuda          # mps on Apple Silicon; add --eval.use_async_envs=false on macOS

Training

lerobot-train \
  --policy.type=act \
  --policy.device=mps \
  --policy.push_to_hub=false \
  --dataset.repo_id=lerobot/aloha_sim_transfer_cube_human \
  --env.type=aloha \
  --env.task=AlohaTransferCube-v0 \
  --env_eval_freq=0 \
  --seed=1000 \
  --steps=50000 \
  --batch_size=8 \
  --save_freq=10000 \
  --output_dir=outputs/train/act_transfer_cube_seed1000
# then resumed from the 50k checkpoint with --resume=true --steps=80000

• Policy: ACT, ResNet-18 backbone (ImageNet init), chunk size 100, 100 action steps, VAE on, KL weight 10, AdamW, learning rate 1e-5, batch size 8. train_config.json in this repository has every setting.
• Throughput on the M4: 2.2 steps/s. 50k steps in 6 h 22 min, the 30k-step extension in 3 h 47 min.
• Checkpoint format: LeRobot 0.7.0 processor pipeline (policy_preprocessor.json, policy_postprocessor.json).

Limitations

• Simulation only; one task; one evaluation seed set. Another --seed would move every number by a few points.
• Success is the gym-aloha definition (cube lifted by the right arm and transferred to the left gripper).
• No hyperparameter tuning; LeRobot defaults throughout.