SO-101 single-cube pickup
Grasps one randomly placed cube and lifts it for a short hold using the updated calibrated wrist camera. This is a simulation-qualified pickup command, using demonstration-derived motion plus deterministic PPO corrections. It is not a stacking policy.
Results
| Fresh unaided qualification placements | Strict stable pickup | Original pickup criterion |
|---|---|---|
| 100 | 86/100 (86%) | 93/100 (93%) |
Evaluated on 12 November 2026 with the calibrated camera from the datasheet. Every attempt started from the fixed calibrated arm pose with one cube sampled from the datasheet placement distribution. Cubes are 25 mm wide. Attempts lasted at most 15 simulated seconds (300 commands at 20 Hz). No teacher or inverse-kinematics assistance was used during qualification. Seeds: 2820000–2820099.
The strict criterion requires bilateral inside-pad grasp, at least 10 mm clearance under the cube, no other support, and cube speed at most 0.1 m/s continuously for one second. The original criterion requires midpoint world z > 22.5 mm with arm contact continuously for one second. Both are reported separately. The policy passed the user-defined 80/100 strict advancement gate. This finite qualification sample was used for curriculum advancement; it does not establish real-world reliability. Aggregate results · Individual outcomes
Local export-parity replay, wrist view
Videos are illustrative and are not additional qualification samples. The local export replay on seed 2820000 completed a strict pickup in 117 commands. Cloud and local simulation outcomes can differ. Replay record
Inputs
Pass these fields to Policy.predict(state) at 20 Hz:
| Field | Format |
|---|---|
joint_positions |
Six measured follower positions |
previous_joint_targets |
Six targets last applied by the controller |
joint_velocities |
Six measured follower velocities |
images.wrist_cam.pixels |
Full 720 × 1280 × 3 HWC RGB NumPy uint8 frame |
Motor order: shoulder pan, shoulder lift, elbow flex, wrist flex, wrist roll, gripper. Positions use degrees for the first five axes and percent for the gripper; velocities use degrees/s and percentage points/s. The three vectors are concatenated in table order and normalized with frozen training mean and variance, then clipped. The neural actor receives the full RGB frame scaled to [0,1], without an external crop, resize or frame history. The internal orange-pixel feature samples every fourth pixel in each direction.
Orange-pixel location selects an interpolated demonstrated joint trajectory, including a distinct outward grasp for distant cubes. If the cube is hidden at reset, the controller can pitch the wrist, estimate the cube location from the new view and return before grasping. An empty initial grasp can trigger a return and second look using measured jaw deflection. These motions use only images, follower feedback and internal episode memory.
The actor never receives simulator cube poses, contact flags, success flags or teacher commands. Camera geometry used for image-to-table estimation is learned/calibrated configuration, not per-step ground-truth state.
Camera and robot configuration
The package targets the datasheet snapshot 7f4456fa25c4d35b521fa556701afade4c641ba0. It uses the device-specific Innomaker U20CAM-1080p-S1 lens calibration and provisional visual mount fit dated 11 November 2026. Supply the distorted full-frame RGB image in policy orientation, rotated 180 degrees from lens-calibration coordinates. Do not apply another rotation if the upstream runner already supplies policy orientation, and do not undistort or crop it before prediction. Camera profile records the matrix, distortion coefficients and mount-fit limitations.
The required reset pose is [0, -16.802, 3.027, 87.881, -86.664, 50.984] in follower units. The host resets the scene and arm before calling reset; the policy does not reset physical hardware. Robot profile · Follower alignment
Output and use
from policy import Policy
policy = Policy()
policy.reset({}) # once the host has reset the arm and scene
command = policy.predict(state) # call once per 50 ms control update
# command == {"joint_positions": [six absolute follower targets]}
Run from a local copy of this repository with requirements.txt installed. The entry point loads a CPU TorchScript actor and compressed motion/normalization configuration once. It adds bounded PPO corrections to the nominal motion, clips normalized commands and converts them to absolute follower degrees/percent. View/return phases retain their qualified feedback controller. After 300 calls, predict raises an episode-limit error; the host must stop earlier when appropriate and reset before another attempt. It does not return a success signal or execute placement/release.
Only NumPy, SciPy and PyTorch are required for inference. MuJoCo, Stable-Baselines3, cloud access and IK are unnecessary. The helper modules are part of the package: policy.py, controller, motion interpolation, cube selection, actor, motion and frozen normalization, manifest.
Training, verification and limits
The nine-hour curriculum used simulated demonstrations, corrective examples and PPO with 12 concurrent environments. The qualified single-cube policy combines the expanded demonstration library and conservative PPO corrections. Selection preserved the exact frozen policy that passed qualification; the cumulative counter was retained at 1,186,608. Checkpoint and source provenance
Portable inference matched original checkpoint targets within 1e-5 follower units across 900 synthetic feedback updates plus a full 117-command physics replay. Synthetic cases exercise visible, hidden and retry conditions; they are interface checks, not performance trials. The exact maximum errors and phases are in export-check.json. Cropped/wrong-dtype images are rejected.
This policy was qualified with one cube. Use the separate eight-cube policy for the evaluated cluttered task. Placement and taller stacks are outside this model's qualified behavior. Live-arm transfer, Jetson latency and hardware queue acceptance remain untested.
The submission guide specifies full-frame wrist RGB, but its canonical manifest schema and approved runtime ID/digest remain pending. The included manifest is provisional and leaves the runtime fields unset. Public availability is not hardware queue approval. No new license is asserted by this export.
Related: the eight-cube policy · previous camera policy.