| --- |
| language: |
| - en |
| tags: |
| - robotics |
| - reinforcement-learning |
| - unitree-h2 |
| - table-tennis |
| - onnx |
| --- |
| |
| # Unitree H2: Online CLoSD and Real-Robot Residual RL |
|
|
| This repository contains the deployment and online reinforcement learning system for a Unitree H2 table-tennis robot. A frozen CLoSD motion generator supplies reference motions, a frozen tracking policy controls the body, and an optional TD3 learner improves seven right-arm joint targets using measured ball outcomes. |
|
|
| Source code, the supported `original` deployment models, and H2 robot assets are hosted together in this Hugging Face model repository. Execution takes place on a configured Linux robot workstation. This is a robotics runtime with several cooperating processes; it is not a Transformers pipeline or a hosted Space. |
|
|
| ## System overview |
|
|
| ```text |
| OptiTrack ROS -> mocap adapter -> ZMQ -> ball predictor |
| | | |
| | v |
| | planner command -> CLoSD reference |
| | | |
| H2 LowState -> C++ bridge -> H2BR -> 50 Hz tracking policy (182 -> 31) |
| | | |
| | optional residual (213 -> 7) |
| | | |
| | joint targets -> H2BR |
| | | |
| | 1000 Hz bridge -> H2 LowCmd |
| v |
| measured ball/racket scoring -> complete episodes |
| -> GPU TD3 learner -> actor for the next ball |
| ``` |
|
|
| H2BR is the shared-memory interface between the Python controller and the C++ bridge. TD3 is the actor/critic algorithm used to learn the residual correction. The base policy and CLoSD generator remain frozen during online learning. |
|
|
| ## Included models and scope |
|
|
| | Component | File | Purpose | |
| | --- | --- | --- | |
| | Base tracking policy | [Stage-C 10500 ONNX](policies/h2/h2_stageC_model_10500.onnx) | 182 observations to 31 joint actions; normalization is embedded | |
| | Motion generator | [CLoSD runtime bundle](closd/H2/model_80000.closd-runtime.pt) | Generate the reference motion on CUDA | |
| | Residual actor | [Residual ONNX](policies/h2/h2_stageC10500_online_ball_000010_residual.onnx) and [JSON metadata](policies/h2/h2_stageC10500_online_ball_000010_residual.json) | 213 inputs to 7 normalized right-arm corrections | |
| | Robot resources | [H2 MJCF](assets/robot_description/h2/mjcf/h2_pingpong.xml), adjacent meshes, [ready pose](ready_pose/H2/frozen_closd_ready_pose_toeout_lowload.npz) | Simulation, joint mapping, and startup reference | |
| | Ball model | [Bounce calibration](models/real_ball_hybrid_bounce.json) | Predictor calibration used by this deployment | |
|
|
| The supported public deployment is `--deployment-version original`. Historical `v1`/`v2` source paths remain for reference; their policy weights and upstream trees are not part of this release. Base-policy pretraining, CLoSD training, and historical training runs are outside this package. |
|
|
| The residual ONNX is an inference export from a run with **100 warmup balls and 10 online balls**. Its JSON metadata records `hardware_qualified=false` and historical export comparisons; these do not establish performance or hardware qualification for a new installation. An ONNX residual cannot resume training. Train from scratch or resume your own full training checkpoint. |
|
|
| ## Start with offline validation |
|
|
| Use Linux with CUDA PyTorch and the dependencies in [setup](docs/SETUP.md). Run these commands from the downloaded repository root, with no hardware control processes running: |
|
|
| ```bash |
| export H2_ROOT="$PWD" |
| export H2_PY="$(command -v python3)" |
| "$H2_PY" scripts/preflight.py |
| "$H2_PY" run_sim_shm.py \ |
| --ack-simulation-only H2-MUJOCO-SIMULATION-ONLY \ |
| --duration-s 9.0 |
| ``` |
|
|
| Require `PREFLIGHT PASS` and a simulation report with `passed=true`. These commands do not start a Unitree LowCmd publisher. The full `tests/` regression suite is not included in this publication copy; hardware commissioning also requires regression acceptance from a development checkout matching this release. Linux build and runtime validation remain pending. |
|
|
| For hardware, first complete site-specific commissioning, including suspended verification of all 31 motor IDs. The launchers preserve manual **L1** start and latched **R1** stop. A dry-run can preview a training session: |
|
|
| ```bash |
| bash scripts/launch_h2_tmux.sh --dry-run \ |
| --deployment-version original --rl-mode train |
| ``` |
|
|
| The checked-in [configuration](config.yaml) uses 100 valid warmup balls and 200 valid online balls, Actor learning rate `1e-6`, and 1,500 Critic warmup updates. These are experiment settings, not a claimed optimum. RL is disabled unless requested. |
|
|
| ## Documentation and code |
|
|
| - [Setup](docs/SETUP.md): Linux dependencies, model files, and the existing ROS/motion-capture environment. |
| - [C++ bridge](cpp_lowcmd_bridge/README.md): ROS 2 build and shared-memory contract. |
|
|
| The entry points are `run_real_shm.py`, `run_sim_shm.py`, `run_live_sim_shm.py`, and `predictor.py`. Control and learning code lives in `runtime/` and `runtime/rl/`; Linux launchers and inspection tools live in `scripts/`. |
|
|
| Original code is provided under [Apache-2.0](LICENSE). Third-party materials remain subject to their own license terms; the source-code license does not relicense bundled third-party models or robot assets. |
|
|
