CS6501 Lab 3: ACT Pick Doll
An Action Chunking with Transformers (ACT) policy trained in LeRobot 0.6.2
to grasp a doll and lift it clear of a table using an SO-ARM101 Follower.
This repository contains the final step-20,000 checkpoint from
lab3-act-team-20k-01.
Training data
The merged training dataset contains 25 teleoperated episodes and 7,568
frames from five team members. The dataset repository also preserves the
original member recordings under raw/.
- Visual input:
observation.images.front, RGB, 640 x 480, recorded at 30 FPS. - State input:
observation.state, six Follower joint values. - Output: six joint-position actions, with ACT action chunks of length 100.
- No depth input is used.
Training
| Setting | Value |
|---|---|
| Framework | LeRobot 0.6.2 |
| Policy | ACT, ResNet-18 visual backbone |
| Training location | UVA Rivanna, CUDA GPU |
| Completed optimizer steps | 20,000 |
| Batch size | 32 |
| Seed | 100000 |
| Checkpoint interval | 5,000 steps |
| DataLoader workers | 1 |
| Video decoder | PyAV |
Other policy settings are in config.json; the full training configuration
is in train_config.json. The final log reported loss 0.056 and L1 loss
0.053, with KL loss displayed as 0.000 at log precision. Training loss is
not a measurement of real-robot task success.
Files and loading
Download all repository files together, including both processor JSON files
and their normalization-state .safetensors files. The model weights alone
are insufficient to reproduce the preprocessing and output scaling.
Optimizer and random-number-generator state remain in the full Rivanna
checkpoint and are not included in this inference repository.
This model is intended for LeRobot's real-robot rollout interface, with the
camera named front and the Follower's own saved calibration. Keep camera
placement, resolution, focus, and workspace consistent with collection.
The training configuration uses CUDA; choose an available inference device
when deploying (the tested laptop uses CPU).
Verification and limitations
The downloaded archive passed integrity checks. Model weights loaded with strict matching, saved processors loaded on CPU, and synthetic inference returned six finite actions. These checks verify loading, not task success. Real-robot evaluation outcomes and videos are documented separately in the lab report; no success rate is claimed here.
The dataset is small and specific to one task and setup. Generalization to new objects, camera views, or workspaces has not been established. Operate the arm under direct supervision with its base secured and movement area clear. Be ready to stop unexpected motion.
Team
Jinyoon Kim (xna8aw), Austin Song (rtx2nb), Boyang Yan (rhe9cf), Sai Tarra (mmk3rs), and Harris Ilyas (tnb6hv).
- Downloads last month
- 25