| --- |
| library_name: lerobot |
| pipeline_tag: robotics |
| tags: |
| - lerobot |
| - robotics |
| - robot-learning |
| - imitation-learning |
| - act |
| - action-chunking-transformer |
| - so100 |
| - so101 |
| - gazebo |
| --- |
| |
| # SO-100/101 ACT Nut-Stacking Policy (Simulation) |
|
|
| This is a LeRobot Action Chunking Transformer (ACT) policy trained in Gazebo |
| to perform the task: |
|
|
| > Pick up the red nut and stack it on the other red part. |
|
|
| ## Demo |
|
|
| The video shows an illustrative rollout of this ACT checkpoint in the Gazebo |
| and RViz simulation environment. It is not a standardized evaluation or a |
| real-robot safety demonstration. |
|
|
| <video controls muted playsinline width="100%"> |
| <source src="https://huggingface.co/shubham4413/so100_act_sim/resolve/main/replay.mp4" type="video/mp4"> |
| Your browser does not support embedded MP4 video. |
| </video> |
|
|
| ## Training data |
|
|
| The policy was trained on 90 selected episodes from the 100-episode |
| `so101_combined_improved_100` LeRobot v3 simulation dataset. That complete |
| dataset contains 183,023 frames at 30 FPS. Episode selection used the |
| deterministic seed-42 training split; stationary-frame sampling was capped |
| during training without modifying the source dataset. |
|
|
| Each observation contains: |
|
|
| - a 640×480 left RGB camera image; |
| - a 640×480 wrist/FPV RGB camera image; and |
| - six measured joint positions. |
|
|
| The policy predicts six joint-position actions for shoulder pan, shoulder |
| lift, elbow flex, wrist flex, wrist roll, and the gripper. |
|
|
| ## Training configuration |
|
|
| | Property | Value | |
| |---|---:| |
| | Policy | Action Chunking Transformer (ACT) | |
| | Training steps | 100,000 | |
| | Batch size | 8 | |
| | Training episodes | 90 of 100 | |
| | Action chunk size | 50 | |
| | Actions executed before replanning | 10 | |
| | Vision backbone | ResNet-18 | |
| | Backbone initialization | ImageNet-1K v1 | |
| | Transformer width | 512 | |
| | Attention heads | 8 | |
| | Encoder/decoder layers | 4 / 1 | |
| | VAE | Enabled, latent dimension 32 | |
| | KL weight | 10 | |
| | Optimizer | AdamW | |
| | Learning rate | 1e-5 | |
| | Weight decay | 1e-4 | |
| | Training seed | 1000 | |
| | Mixed precision | Disabled | |
|
|
| ## Loading with LeRobot |
|
|
| ```python |
| from lerobot.policies.act.modeling_act import ACTPolicy |
| |
| policy = ACTPolicy.from_pretrained("shubham4413/so100_act_sim") |
| ``` |
|
|
| The observation keys, camera ordering, state/action conventions, and |
| normalization processors must match the training configuration included in |
| this repository. |
|
|
| ## Evaluation status |
|
|
| No statistically meaningful success rate is reported. The included recording |
| demonstrates one simulation rollout and should not be interpreted as evidence |
| of general robustness. |
|
|
| ## Limitations and safety |
|
|
| - This checkpoint was trained in simulation and has not been validated as a |
| robust sim-to-real policy. |
| - Performance outside the recorded object poses, workspace geometry, camera |
| calibration, and dynamics distribution is unknown. |
| - Contact forces are approximated by the simulator; the policy has no force |
| sensing input. |
| - The controller predicts position commands and may produce unsafe motion when |
| observation timing, calibration, or joint conventions differ. |
| - Real-robot experiments require conservative speed and joint limits, command |
| clamping, an emergency stop, collision-aware supervision, and dry-run |
| validation before actuating hardware. |
|
|
| ## Provenance |
|
|
| The uploaded artifact is the final `100000/pretrained_model` checkpoint from |
| `act_pretrained_trimmed_so101_combined_improved_100_20260829_000326`. Optimizer |
| state and other resume-only training files are intentionally not included. |
|
|
