BAAI/PingPongRealWorldRL

Unitree H2: Online CLoSD and Real-Robot Residual RL

This repository contains the deployment and online reinforcement learning system for a Unitree H2 table-tennis robot. A frozen CLoSD motion generator supplies reference motions, a frozen tracking policy controls the body, and an optional TD3 learner improves seven right-arm joint targets using measured ball outcomes.

Source code, the supported original deployment models, and H2 robot assets are hosted together in this Hugging Face model repository. Execution takes place on a configured Linux robot workstation. This is a robotics runtime with several cooperating processes; it is not a Transformers pipeline or a hosted Space.

System overview

OptiTrack ROS -> mocap adapter -> ZMQ -> ball predictor
                                  |        |
                                  |        v
                                  |   planner command -> CLoSD reference
                                  |                         |
H2 LowState -> C++ bridge -> H2BR -> 50 Hz tracking policy (182 -> 31)
                                  |                         |
                                  |                  optional residual (213 -> 7)
                                  |                         |
                                  |                   joint targets -> H2BR
                                  |                                      |
                                  |                         1000 Hz bridge -> H2 LowCmd
                                  v
                     measured ball/racket scoring -> complete episodes
                                  -> GPU TD3 learner -> actor for the next ball

H2BR is the shared-memory interface between the Python controller and the C++ bridge. TD3 is the actor/critic algorithm used to learn the residual correction. The base policy and CLoSD generator remain frozen during online learning.

Included models and scope

Component	File	Purpose
Base tracking policy	Stage-C 10500 ONNX	182 observations to 31 joint actions; normalization is embedded
Motion generator	CLoSD runtime bundle	Generate the reference motion on CUDA
Residual actor	Residual ONNX and JSON metadata	213 inputs to 7 normalized right-arm corrections
Robot resources	H2 MJCF, adjacent meshes, ready pose	Simulation, joint mapping, and startup reference
Ball model	Bounce calibration	Predictor calibration used by this deployment

The supported public deployment is --deployment-version original. Historical v1/v2 source paths remain for reference; their policy weights and upstream trees are not part of this release. Base-policy pretraining, CLoSD training, and historical training runs are outside this package.

The residual ONNX is an inference export from a run with 100 warmup balls and 10 online balls. Its JSON metadata records hardware_qualified=false and historical export comparisons; these do not establish performance or hardware qualification for a new installation. An ONNX residual cannot resume training. Train from scratch or resume your own full training checkpoint.

Start with offline validation

Use Linux with CUDA PyTorch and the dependencies in setup. Run these commands from the downloaded repository root, with no hardware control processes running:

export H2_ROOT="$PWD"
export H2_PY="$(command -v python3)"
"$H2_PY" scripts/preflight.py
"$H2_PY" run_sim_shm.py \
  --ack-simulation-only H2-MUJOCO-SIMULATION-ONLY \
  --duration-s 9.0

Require PREFLIGHT PASS and a simulation report with passed=true. These commands do not start a Unitree LowCmd publisher. The full tests/ regression suite is not included in this publication copy; hardware commissioning also requires regression acceptance from a development checkout matching this release. Linux build and runtime validation remain pending.

For hardware, first complete site-specific commissioning, including suspended verification of all 31 motor IDs. The launchers preserve manual L1 start and latched R1 stop. A dry-run can preview a training session:

bash scripts/launch_h2_tmux.sh --dry-run \
  --deployment-version original --rl-mode train

The checked-in configuration uses 100 valid warmup balls and 200 valid online balls, Actor learning rate 1e-6, and 1,500 Critic warmup updates. These are experiment settings, not a claimed optimum. RL is disabled unless requested.

Documentation and code

• Setup: Linux dependencies, model files, and the existing ROS/motion-capture environment.
• C++ bridge: ROS 2 build and shared-memory contract.

The entry points are run_real_shm.py, run_sim_shm.py, run_live_sim_shm.py, and predictor.py. Control and learning code lives in runtime/ and runtime/rl/; Linux launchers and inspection tools live in scripts/.

Original code is provided under Apache-2.0. Third-party materials remain subject to their own license terms; the source-code license does not relicense bundled third-party models or robot assets.