| --- |
| license: apache-2.0 |
| pipeline_tag: robotics |
| library_name: openpi |
| tags: |
| - robotics |
| - openpi |
| - pi0.5 |
| - vision-language-action |
| - reinforcement-learning |
| - bimanual-manipulation |
| - roboeval |
| datasets: |
| - helen9975/roboeval_ee_delta_20hz_scaled |
| --- |
| |
| # RoboEval CoRL 2026 solution: off-policy RL with online inference tuning |
|
|
| This is the policy I submitted to the [RoboEval Challenge at CoRL 2026](https://robo-eval.github.io/challenge.html): |
| one modified π0.5 for eight bimanual tasks with two Franka arms, in simulation. The weights are an average of two |
| checkpoints of the same run, 0.65 × step 48250 + 0.35 × step 58000. |
|
|
| - Code: https://github.com/IliaLarchenko/RoboEvalSolution |
| - Report: https://arxiv.org/abs/2611.11464 |
| - Project page: https://ilialarchenko.com/projects/roboeval2026/ |
|
|
| ## Results |
|
|
| Scored with the official RoboEval Diagnostic Score (RDS), each task with its final inference settings. |
|
|
| | | Episodes per task | RDS | Success | |
| |---|---:|---:|---:| |
| | Official seeds (1000–1049) | 50 | **0.987** | 99.25% | |
| | Held-out random seeds | 500 | **0.990** | 99.60% | |
|
|
| | Task | Official success | Official RDS | Held-out success | Held-out RDS | |
| |---|---:|---:|---:|---:| |
| | LiftPot | 1.000 | 1.000 | 1.000 | 0.999 | |
| | LiftTray | 0.980 | 0.984 | 0.998 | 0.998 | |
| | StackTwoBlocks | 1.000 | 0.977 | 0.996 | 0.979 | |
| | StackSingleBookShelf | 1.000 | 1.000 | 0.998 | 0.998 | |
| | CubeHandover | 1.000 | 1.000 | 1.000 | 0.998 | |
| | PackBox | 1.000 | 0.969 | 1.000 | 0.974 | |
| | PickSingleBookFromTable | 0.960 | 0.967 | 0.988 | 0.988 | |
| | RotateValve | 1.000 | 1.000 | 0.988 | 0.989 | |
|
|
| A 50-episode run varies by about ±0.005 RDS, so the two rows agree within noise. I made every decision on held-out |
| seeds and never tuned on the official ones. |
|
|
| ## Inputs and outputs |
|
|
| - **Inputs:** five RGB cameras at 256×256 (head, front, external, left wrist, right wrist), proprioception (joint |
| positions and velocities, gripper states) and a task id. There is no text prompt. Some tasks hide some of these |
| inputs at inference. |
| - **Actions:** chunks of 30 steps (1.5 s at 20 Hz) of 14-D end-effector actions for both arms: per arm a 3-D position, |
| a 3-D rotation vector and a gripper command, relative to the commanded pose at the chunk start. They convert to the |
| challenge's end-effector delta actions. |
| - **Tasks:** LiftPot, LiftTray, StackTwoBlocks, StackSingleBookShelf, CubeHandover, PackBox, PickSingleBookFromTable |
| and RotateValve, all on the PositionAndOrientation variation. |
|
|
| ## How to run it |
|
|
| The policy runs through the code repository, not through `transformers`. You need a Linux machine with an NVIDIA GPU |
| (driver >= 580 for the default CUDA 13 build; on older drivers use `bash setup.sh --cuda12`). |
|
|
| ```bash |
| git clone --recurse-submodules https://github.com/IliaLarchenko/RoboEvalSolution.git && cd RoboEvalSolution |
| bash setup.sh # .venv (policy) and .venv-sim (simulator) |
| .venv/bin/hf download IliaLarchenko/RoboEvalSolution --local-dir ../checkpoint |
| |
| CUDA_VISIBLE_DEVICES=0 .venv/bin/python scripts/serve_policy.py --config-name pi05_roboeval \ |
| --checkpoint-dir ../checkpoint --num-candidates 6 & |
| MUJOCO_GL=egl .venv-sim/bin/python scripts/rollout.py --server ws://localhost:8765 --out ../runs/eval_x \ |
| --episodes-per-task 50 --workers 20 --best configs/settings.json |
| ``` |
|
|
| `rollout.py` builds the environments like the official evaluator and writes the official scores to `summary.json`. It |
| refuses the official seeds 1000–1049. Stop the server afterwards with `kill %1`. |
|
|
| ## Inference settings |
|
|
| The settings matter a lot and differ per task: the number of flow steps, noise temperature, CFG scale, best-of-N |
| ranked by the Q head, how many actions to execute and carry over, execution speed, gripper threshold and hidden |
| inputs. They are in |
| [`configs/settings.json`](https://github.com/IliaLarchenko/RoboEvalSolution/blob/main/configs/settings.json). |
| Without them you won't reproduce the scores above. |
|
|
| ## Training |
|
|
| The model is π0.5 (a PaliGemma VLM plus a flow-matching action expert) with a value query, a Q head and advantage |
| conditioning, so it is also its own value function. I trained it in an off-policy RL loop with advantages of the exact |
| challenge score (AWR, and advantage conditioning inspired by RECAP). Every rollout was also a Thompson-sampling trial |
| of about 20 inference parameters. Training used about 600 GPU-hours. |
|
|
| Data: |
|
|
| - the organisers' [RoboEval demonstrations](https://huggingface.co/datasets/helen9975/roboeval_ee_delta_20hz_scaled); |
| - my own scripted demonstrations with full reward, generated in simulation from privileged state; |
| - more than 240k of the policy's own rollouts, plus recovery episodes that continue from its failures. |
|
|
| The only external pretraining is the public π0.5 base model. |
|
|
| ## Limitations |
|
|
| - It was trained and evaluated in simulation only and has never run on a real robot. |
| - It is tuned to these eight tasks, this robot, these cameras and this exact score. Don't expect it to work elsewhere |
| without retraining. |
| - Part of the score comes from per-task inference tuning, which is slow and would be hard to justify in a real |
| deployment. |
|
|
| ## License |
|
|
| Code and weights are released under Apache-2.0. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{larchenko2026roboeval, |
| title = {Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge}, |
| author = {Larchenko, Ilia}, |
| year = {2026}, |
| eprint = {2611.11464}, |
| archivePrefix = {arXiv}, |
| primaryClass = {cs.RO}, |
| url = {https://arxiv.org/abs/2611.11464} |
| } |
| ``` |
|
|
| The benchmark, simulator and demonstrations are from [RoboEval](https://robo-eval.github.io) |
| ([Wang et al., 2025](https://arxiv.org/abs/2507.00435)). The base model is π0.5 by |
| [Physical Intelligence](https://github.com/Physical-Intelligence/openpi). |
|
|
