shubham4413/so100_act_sim

SO-100/101 ACT Nut-Stacking Policy (Simulation)

This is a LeRobot Action Chunking Transformer (ACT) policy trained in Gazebo to perform the task:

Pick up the red nut and stack it on the other red part.

Demo

The video shows an illustrative rollout of this ACT checkpoint in the Gazebo and RViz simulation environment. It is not a standardized evaluation or a real-robot safety demonstration.

Training data

The policy was trained on 90 selected episodes from the 100-episode so101_combined_improved_100 LeRobot v3 simulation dataset. That complete dataset contains 183,023 frames at 30 FPS. Episode selection used the deterministic seed-42 training split; stationary-frame sampling was capped during training without modifying the source dataset.

Each observation contains:

• a 640×480 left RGB camera image;
• a 640×480 wrist/FPV RGB camera image; and
• six measured joint positions.

The policy predicts six joint-position actions for shoulder pan, shoulder lift, elbow flex, wrist flex, wrist roll, and the gripper.

Training configuration

Property	Value
Policy	Action Chunking Transformer (ACT)
Training steps	100,000
Batch size	8
Training episodes	90 of 100
Action chunk size	50
Actions executed before replanning	10
Vision backbone	ResNet-18
Backbone initialization	ImageNet-1K v1
Transformer width	512
Attention heads	8
Encoder/decoder layers	4 / 1
VAE	Enabled, latent dimension 32
KL weight	10
Optimizer	AdamW
Learning rate	1e-5
Weight decay	1e-4
Training seed	1000
Mixed precision	Disabled

Loading with LeRobot

from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("shubham4413/so100_act_sim")

The observation keys, camera ordering, state/action conventions, and normalization processors must match the training configuration included in this repository.

Evaluation status

No statistically meaningful success rate is reported. The included recording demonstrates one simulation rollout and should not be interpreted as evidence of general robustness.

Limitations and safety

• This checkpoint was trained in simulation and has not been validated as a robust sim-to-real policy.
• Performance outside the recorded object poses, workspace geometry, camera calibration, and dynamics distribution is unknown.
• Contact forces are approximated by the simulator; the policy has no force sensing input.
• The controller predicts position commands and may produce unsafe motion when observation timing, calibration, or joint conventions differ.
• Real-robot experiments require conservative speed and joint limits, command clamping, an emergency stop, collision-aware supervision, and dry-run validation before actuating hardware.

Provenance

The uploaded artifact is the final 100000/pretrained_model checkpoint from act_pretrained_trimmed_so101_combined_improved_100_20260829_000326. Optimizer state and other resume-only training files are intentionally not included.