---
language:
  - en
tags:
  - robotics
  - reinforcement-learning
  - unitree-h2
  - table-tennis
  - onnx
---

# Unitree H2: Online CLoSD and Real-Robot Residual RL

This repository contains the deployment and online reinforcement learning system for a Unitree H2 table-tennis robot. A frozen CLoSD motion generator supplies reference motions, a frozen tracking policy controls the body, and an optional TD3 learner improves seven right-arm joint targets using measured ball outcomes.

Source code, the supported `original` deployment models, and H2 robot assets are hosted together in this Hugging Face model repository. Execution takes place on a configured Linux robot workstation. This is a robotics runtime with several cooperating processes; it is not a Transformers pipeline or a hosted Space.

## System overview

```text
OptiTrack ROS -> mocap adapter -> ZMQ -> ball predictor
                                  |        |
                                  |        v
                                  |   planner command -> CLoSD reference
                                  |                         |
H2 LowState -> C++ bridge -> H2BR -> 50 Hz tracking policy (182 -> 31)
                                  |                         |
                                  |                  optional residual (213 -> 7)
                                  |                         |
                                  |                   joint targets -> H2BR
                                  |                                      |
                                  |                         1000 Hz bridge -> H2 LowCmd
                                  v
                     measured ball/racket scoring -> complete episodes
                                  -> GPU TD3 learner -> actor for the next ball
```

H2BR is the shared-memory interface between the Python controller and the C++ bridge. TD3 is the actor/critic algorithm used to learn the residual correction. The base policy and CLoSD generator remain frozen during online learning.

## Included models and scope

| Component | File | Purpose |
| --- | --- | --- |
| Base tracking policy | [Stage-C 10500 ONNX](policies/h2/h2_stageC_model_10500.onnx) | 182 observations to 31 joint actions; normalization is embedded |
| Motion generator | [CLoSD runtime bundle](closd/H2/model_80000.closd-runtime.pt) | Generate the reference motion on CUDA |
| Residual actor | [Residual ONNX](policies/h2/h2_stageC10500_online_ball_000010_residual.onnx) and [JSON metadata](policies/h2/h2_stageC10500_online_ball_000010_residual.json) | 213 inputs to 7 normalized right-arm corrections |
| Robot resources | [H2 MJCF](assets/robot_description/h2/mjcf/h2_pingpong.xml), adjacent meshes, [ready pose](ready_pose/H2/frozen_closd_ready_pose_toeout_lowload.npz) | Simulation, joint mapping, and startup reference |
| Ball model | [Bounce calibration](models/real_ball_hybrid_bounce.json) | Predictor calibration used by this deployment |

The supported public deployment is `--deployment-version original`. Historical `v1`/`v2` source paths remain for reference; their policy weights and upstream trees are not part of this release. Base-policy pretraining, CLoSD training, and historical training runs are outside this package.

The residual ONNX is an inference export from a run with **100 warmup balls and 10 online balls**. Its JSON metadata records `hardware_qualified=false` and historical export comparisons; these do not establish performance or hardware qualification for a new installation. An ONNX residual cannot resume training. Train from scratch or resume your own full training checkpoint.

## Start with offline validation

Use Linux with CUDA PyTorch and the dependencies in [setup](docs/SETUP.md). Run these commands from the downloaded repository root, with no hardware control processes running:

```bash
export H2_ROOT="$PWD"
export H2_PY="$(command -v python3)"
"$H2_PY" scripts/preflight.py
"$H2_PY" run_sim_shm.py \
  --ack-simulation-only H2-MUJOCO-SIMULATION-ONLY \
  --duration-s 9.0
```

Require `PREFLIGHT PASS` and a simulation report with `passed=true`. These commands do not start a Unitree LowCmd publisher. The full `tests/` regression suite is not included in this publication copy; hardware commissioning also requires regression acceptance from a development checkout matching this release. Linux build and runtime validation remain pending.

For hardware, first complete site-specific commissioning, including suspended verification of all 31 motor IDs. The launchers preserve manual **L1** start and latched **R1** stop. A dry-run can preview a training session:

```bash
bash scripts/launch_h2_tmux.sh --dry-run \
  --deployment-version original --rl-mode train
```

The checked-in [configuration](config.yaml) uses 100 valid warmup balls and 200 valid online balls, Actor learning rate `1e-6`, and 1,500 Critic warmup updates. These are experiment settings, not a claimed optimum. RL is disabled unless requested.

## Documentation and code

- [Setup](docs/SETUP.md): Linux dependencies, model files, and the existing ROS/motion-capture environment.
- [C++ bridge](cpp_lowcmd_bridge/README.md): ROS 2 build and shared-memory contract.

The entry points are `run_real_shm.py`, `run_sim_shm.py`, `run_live_sim_shm.py`, and `predictor.py`. Control and learning code lives in `runtime/` and `runtime/rl/`; Linux launchers and inspection tools live in `scripts/`.

Original code is provided under [Apache-2.0](LICENSE). Third-party materials remain subject to their own license terms; the source-code license does not relicense bundled third-party models or robot assets.