---
library_name: lerobot
pipeline_tag: robotics
tags:
  - lerobot
  - robotics
  - robot-learning
  - imitation-learning
  - act
  - action-chunking-transformer
  - so100
  - so101
  - gazebo
---

# SO-100/101 ACT Nut-Stacking Policy (Simulation)

This is a LeRobot Action Chunking Transformer (ACT) policy trained in Gazebo
to perform the task:

> Pick up the red nut and stack it on the other red part.

## Demo

The video shows an illustrative rollout of this ACT checkpoint in the Gazebo
and RViz simulation environment. It is not a standardized evaluation or a
real-robot safety demonstration.

<video controls muted playsinline width="100%">
  <source src="https://huggingface.co/shubham4413/so100_act_sim/resolve/main/replay.mp4" type="video/mp4">
  Your browser does not support embedded MP4 video.
</video>

## Training data

The policy was trained on 90 selected episodes from the 100-episode
`so101_combined_improved_100` LeRobot v3 simulation dataset. That complete
dataset contains 183,023 frames at 30 FPS. Episode selection used the
deterministic seed-42 training split; stationary-frame sampling was capped
during training without modifying the source dataset.

Each observation contains:

- a 640×480 left RGB camera image;
- a 640×480 wrist/FPV RGB camera image; and
- six measured joint positions.

The policy predicts six joint-position actions for shoulder pan, shoulder
lift, elbow flex, wrist flex, wrist roll, and the gripper.

## Training configuration

| Property | Value |
|---|---:|
| Policy | Action Chunking Transformer (ACT) |
| Training steps | 100,000 |
| Batch size | 8 |
| Training episodes | 90 of 100 |
| Action chunk size | 50 |
| Actions executed before replanning | 10 |
| Vision backbone | ResNet-18 |
| Backbone initialization | ImageNet-1K v1 |
| Transformer width | 512 |
| Attention heads | 8 |
| Encoder/decoder layers | 4 / 1 |
| VAE | Enabled, latent dimension 32 |
| KL weight | 10 |
| Optimizer | AdamW |
| Learning rate | 1e-5 |
| Weight decay | 1e-4 |
| Training seed | 1000 |
| Mixed precision | Disabled |

## Loading with LeRobot

```python
from lerobot.policies.act.modeling_act import ACTPolicy

policy = ACTPolicy.from_pretrained("shubham4413/so100_act_sim")
```

The observation keys, camera ordering, state/action conventions, and
normalization processors must match the training configuration included in
this repository.

## Evaluation status

No statistically meaningful success rate is reported. The included recording
demonstrates one simulation rollout and should not be interpreted as evidence
of general robustness.

## Limitations and safety

- This checkpoint was trained in simulation and has not been validated as a
  robust sim-to-real policy.
- Performance outside the recorded object poses, workspace geometry, camera
  calibration, and dynamics distribution is unknown.
- Contact forces are approximated by the simulator; the policy has no force
  sensing input.
- The controller predicts position commands and may produce unsafe motion when
  observation timing, calibration, or joint conventions differ.
- Real-robot experiments require conservative speed and joint limits, command
  clamping, an emergency stop, collision-aware supervision, and dry-run
  validation before actuating hardware.

## Provenance

The uploaded artifact is the final `100000/pretrained_model` checkpoint from
`act_pretrained_trimmed_so101_combined_improved_100_20260829_000326`. Optimizer
state and other resume-only training files are intentionally not included.