so101-reach / README.md
bokalnv's picture
Align policy descriptor filename and simplify model card
f88e850 verified
|
Raw History Blame Contribute Delete
13.1 kB
---
license: other
license_name: nvidia-license
license_link: https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf
library_name: pytorch
tags:
- reinforcement-learning
- robotics
- sim-to-real
- isaac-lab
- ppo
- onnx
- reach
- manipulation
- nvidia
---
# **SO-101 Reach** β€” Delta Fingertip Policy
## Description
SO-101 Reach is a reinforcement-learning policy that moves the end-effector
(fingertip) of a [LeRobot SO-101](https://github.com/huggingface/lerobot)
robot arm to randomly sampled 3-D goal positions using **delta joint
actions**. Each policy output is a relative joint displacement rather than
an absolute target, making the controller inherently smoother and less
sensitive to sim-to-real state offsets.
The policy is trained entirely in simulation using NVIDIA Isaac Lab, from
`GRIP-Reach-Delta-NoVel-v0` β€” GRIP's only reach environment. It observes **20
dimensions** and carries no joint-velocity term.
- **ONNX** (`.onnx`) β€” the deployed artifact,
[`policy_fingertip_delta.onnx`](./policy_fingertip_delta.onnx)
(SHA-256 `25e08afb6bc6cd62e669996303e8cd1f7a0a2aaa98e6b808b544a7d2b51e308f`).
This repository provides the deployed ONNX model, an Isaac Lab I/O descriptor, and
this model card. Training code and the application runtime are not included.
This model is for demonstration purposes and not for production usage.
## Release Date
Hugging Face 08/31/2026 via [https://huggingface.co/nvidia/so101-reach](https://huggingface.co/nvidia/so101-reach)
## License/Terms of Use
[Nvidia License](https://developer.download.nvidia.com/licenses/NVIDIA-OneWay-Noncommercial-License-22Mar2022.pdf)
### Deployment Geography
Global
### Use Case
This model is intended for roboticists and researchers working on
sim-to-real transfer for low-cost manipulators. Primary uses include:
- Sim-to-real end-effector reaching on the SO-101 arm.
- Warm-start or pre-training for downstream manipulation tasks
(e.g. pick-and-place).
- Benchmarking sim-to-real transfer methods for affordable robot arms.
## Model Architecture
**Architecture Type:** Multilayer Perceptron (MLP)
**Number of model parameters:** 46,788
**Network Architecture:** The policy is an actor-critic architecture
trained with PPO (Proximal Policy Optimization) via
[RSL-RL](https://github.com/leggedrobotics/rsl_rl). Only the actor
network is included in the exported files. Observations are
empirically normalised β€” the normaliser is baked into the ONNX graph so raw
observations can be fed directly.
| Component | Details |
| ------------------------- | --------------------------- |
| Actor | MLP `[256, 128, 64]`, ELU |
| Critic (training only) | MLP `[256, 128, 64]`, ELU |
| Observation normalisation | Empirical (baked into graph) |
## Input
**Input Type(s):** Robot state, Goal command
**Input Format(s):** Robot state: Floating Point; Goal command: Floating Point
**Input Parameters:** One-Dimensional (1D) vector of size 20 (`float32`)
**Other Properties Related to Input:**
| Slice | Dim | Description |
| --------- | --- | ----------------------------------------------- |
| `[0:6]` | 6 | Joint positions relative to default |
| `[6:9]` | 3 | End-effector (fingertip) position in root frame |
| `[9:16]` | 7 | Goal command (position + quaternion) |
| `[16:20]` | 4 | Last action (clipped to [-1, 1]) |
The end-effector frame is an offset from the `gripper_link` body:
`[-0.0079, -0.000218, -0.0981]` m.
**No velocity term:** The policy observes no joint velocity at all. A runtime
must not supply one.
## Output
**Output Type(s):** Actions (delta joint displacements)
**Output Format:** Floating point
**Output Parameters:** One-Dimensional (1D) vector of size 4 (`float32`).
Raw ONNX outputs are not bounded to `[-1, 1]`. The deployment runtime must clip
them before applying the action scale and per-tick joint-displacement limit below.
**Other Properties Related to Output:**
Actions are **delta** joint-position offsets for four arm joints:
```text
target = current_pos + clip(action, -1, 1) * scale
```
| Parameter | Value |
| --------------- | ----- |
| `scale` | 0.20 rad |
| training `max_step_size` | 0.10 rad per 50 Hz tick |
| action delay | 1 - 4 ticks, randomised |
The action delay is randomised during training so the policy does not rely on its
command taking effect on the tick it was issued, which is the one thing a real
servo bus β€” the serial link daisy-chaining the arm's Feetech STS-3215 servos to
the controller β€” reliably will not do.
At deployment, apply the same `0.10 rad` clamp at 50 Hz.
| Index | Joint |
| ----- | ---------------- |
| 0 | `shoulder_pan` |
| 1 | `shoulder_lift` |
| 2 | `elbow_flex` |
| 3 | `wrist_flex` |
`wrist_roll` and `gripper` are not actuated by this policy.
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated
systems, in this case NVIDIA Jetson Thor via TensorRT. By leveraging NVIDIA's
hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the
model achieves faster training and inference times compared to CPU-only
solutions.
## Software Integration
**Runtime Engine(s):** ONNX Runtime, TensorRT (via Holoscan)
**Checkpoint Formats:**
| Format | File | Used by |
| ------------ | ------------------ | ------------------------------ |
| ONNX | `policy_fingertip_delta.onnx` (this repository) | ONNX Runtime / TensorRT |
Every training run writes a `policy_contract.json` recording its exact delta
action contract, and exports carry the same file; it is validated before
deployment or sim-to-sim evaluation. Alongside the action semantics, scale and
clamp, the contract records the observation width and whether the policy consumes
a joint velocity, so a runtime expecting a different interface is refused rather
than run.
The [Isaac Lab I/O descriptor](./policy_fingertip_delta.yaml) declares the
observation layout, action scale and joint limits for `policy_fingertip_delta.onnx`
(checksum above). It is not a LEAPP manifest or a standalone controller.
**Deployment supplement (50 Hz):** ONNX opset 18; `obs: float32[1,20]` β†’
`actions: float32[1,4]`. Observation normalization is already embedded.
- Zero observation slots `[4:6]`; hold `wrist_roll` and `gripper` at `0` rad.
- Compute fingertip position in the base frame using the `gripper_link` local
offset `[-0.0079, -0.000218, -0.0981]` m. Goals are base-frame
`[x,y,z,1,0,0,0]`, with quaternion order **WXYZ**.
- Initialize `last_action` to zeros; subsequently use the previous **clipped,
unscaled** policy output.
- Clip raw actions to `[-1,1]`; form `target = measured_pos + 0.20 * action`.
Clamp to descriptor joint limits, **then** to `measured_pos Β± 0.10` rad.
Reject non-finite measured states; replace non-finite policy outputs with zeros
before clipping. Do not inject training's randomized delay at deployment.
**Supported Hardware Microarchitecture Compatibility:**
- x86-64 CPU
- NVIDIA Jetson Thor
**Supported Operating System(s):**
- Linux
The integration of foundation and fine-tuned models into AI systems requires
additional testing using use-case-specific data to ensure safe and effective
deployment. Following the V-model methodology, iterative testing and validation
at both unit and system levels are essential to mitigate risks, meet technical
and functional requirements, and ensure compliance with safety and ethical
standards before deployment.
## Model Version(s)
SO-101 Reach Version 3.0 β€” delta action policy with a 20-D observation, trained
from `GRIP-Reach-Delta-NoVel-v0`.
## Training, Testing, and Evaluation Datasets
### Training Dataset
**Data Collection Method:** Automated (reinforcement learning in
simulation)
**Labeling Method:** Not Applicable
**Data Modality:** Numeric robot state vectors (joint positions, fingertip
position, goal command), generated in simulation
**Non-Audio, Image, Text Training Data Size:** Not Applicable β€” training is
on-policy, so transitions are consumed as they are generated and no dataset is
stored.
**Properties:**
| | |
| -------------------------- | --------------------------------------------- |
| **Simulator** | NVIDIA Isaac Lab / Isaac Sim |
| **Algorithm** | PPO (on-policy, RSL-RL) |
| **Parallel environments** | 4 096 |
| **Max iterations** | 5 000 |
| **Control frequency** | 50 Hz (sim dt = 0.01 s, decimation = 2) |
| **Episode length** | 12 s (600 steps), goals resample every 4 s |
| **Action type** | Delta (`target = current + action * 0.20`) |
| **Observation noise** | Gaussian (joint pos 0.01, EE 0.003) |
| **Seeds** | 3; the best by evaluation error was exported |
| **Domain randomisation** | Actuator gain scaling x[0.7, 1.3] |
| **Initial pose** | Arm joints resampled in +-1.5 rad on reset |
| **Action delay** | 1 - 4 ticks (20 - 80 ms), resampled per reset |
| **Reward** | Dense tanh(d/s) EE-to-goal + penalty terms |
The initial-pose and action-delay randomisations are what make the policy usable
from an arbitrary starting configuration on a bus that does not apply a command on
the tick it was issued. Both are deliberately wide: the reset range covers most of
the joint travel rather than a neighbourhood of the home pose.
**Goal sampling ranges (robot base frame):**
| Axis | Range (m) |
| ------------ | ------------ |
| x (forward) | 0.08 - 0.28 |
| y (lateral) | -0.18 - 0.18 |
| z (vertical) | 0.05 - 0.30 |
### Evaluation Dataset
**Data Collection Method:** Automated (simulation rollouts in
`GRIP-Reach-Delta-NoVel-Play-v0`)
**Labeling Method:** Not Applicable
**Properties:**
| | |
| ------------------------- | ---------------------------------------------- |
| **Parallel environments** | 50 |
| **Episode length** | 4 s, matching the goal resample period so each |
| | episode measures exactly one goal |
| **Observation noise** | Disabled |
| **Domain randomisation** | Disabled (actuator gains not randomised) |
Hardware validation measured goal completion on a 3-D grid at a 0.05 m threshold
on a real SO-101. Settled accuracy below that threshold has not been characterised.
## Inference
**Engines:** ONNX Runtime, TensorRT (via Holoscan)
**Test Hardware:** x86-64 CPU, NVIDIA Jetson Thor
The deployed artifact is ONNX. Observation normalisation is baked into the graph,
so raw observations are fed directly:
```python
import numpy as np
import onnxruntime as ort
session = ort.InferenceSession("policy_fingertip_delta.onnx")
obs = np.zeros((1, 20), dtype=np.float32)
action = session.run(None, {session.get_inputs()[0].name: obs})[0] # shape (1, 4)
joint_delta = np.clip(np.clip(action, -1, 1) * 0.20, -0.10, 0.10)
# target = current_joint_pos + joint_delta
```
Because the target is built from the *measured* position, a non-finite
measurement propagates straight through the clamp into the commanded position. Any
runtime consuming this policy has to reject non-finite joint states rather than
pass them on.
## Model Limitations
This model is not tested or intended for use in mission-critical
applications that require functional safety. Use in such applications
is at the user's own risk.
- Position-only goals β€” no orientation control of the end-effector.
- Trained for a fixed-base tabletop setup with the SO-101 arm only.
- Performance may degrade outside the trained goal ranges or with
significantly different hardware.
- The policy does not control `wrist_roll` or `gripper` joints.
- The observation carries no joint velocity, so the policy cannot distinguish a
stationary arm from a moving one at the same position.
- Reaching is characterised against a 0.05 m threshold; the settled accuracy
achievable below that has not been measured.
## Ethical Considerations
NVIDIA believes Trustworthy AI is a shared responsibility and we have
established policies and practices to enable development for a wide array of AI
applications. Developers should work with their internal model team to ensure
this model meets requirements for the relevant industry and use case and
addresses unforeseen product misuse.
Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns
[here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).