license: apache-2.0
pipeline_tag: robotics
library_name: openpi
tags:
- robotics
- openpi
- pi0.5
- vision-language-action
- reinforcement-learning
- bimanual-manipulation
- roboeval
datasets:
- helen9975/roboeval_ee_delta_20hz_scaled
RoboEval CoRL 2026 solution: off-policy RL with online inference tuning
This is the policy I submitted to the RoboEval Challenge at CoRL 2026: one modified π0.5 for eight bimanual tasks with two Franka arms, in simulation. The weights are an average of two checkpoints of the same run, 0.65 × step 48250 + 0.35 × step 58000.
- Code: https://github.com/IliaLarchenko/RoboEvalSolution
- Report: https://arxiv.org/abs/2611.11464
- Project page: https://ilialarchenko.com/projects/roboeval2026/
Results
Scored with the official RoboEval Diagnostic Score (RDS), each task with its final inference settings.
| Episodes per task | RDS | Success | |
|---|---|---|---|
| Official seeds (1000–1049) | 50 | 0.987 | 99.25% |
| Held-out random seeds | 500 | 0.990 | 99.60% |
| Task | Official success | Official RDS | Held-out success | Held-out RDS |
|---|---|---|---|---|
| LiftPot | 1.000 | 1.000 | 1.000 | 0.999 |
| LiftTray | 0.980 | 0.984 | 0.998 | 0.998 |
| StackTwoBlocks | 1.000 | 0.977 | 0.996 | 0.979 |
| StackSingleBookShelf | 1.000 | 1.000 | 0.998 | 0.998 |
| CubeHandover | 1.000 | 1.000 | 1.000 | 0.998 |
| PackBox | 1.000 | 0.969 | 1.000 | 0.974 |
| PickSingleBookFromTable | 0.960 | 0.967 | 0.988 | 0.988 |
| RotateValve | 1.000 | 1.000 | 0.988 | 0.989 |
A 50-episode run varies by about ±0.005 RDS, so the two rows agree within noise. I made every decision on held-out seeds and never tuned on the official ones.
Inputs and outputs
- Inputs: five RGB cameras at 256×256 (head, front, external, left wrist, right wrist), proprioception (joint positions and velocities, gripper states) and a task id. There is no text prompt. Some tasks hide some of these inputs at inference.
- Actions: chunks of 30 steps (1.5 s at 20 Hz) of 14-D end-effector actions for both arms: per arm a 3-D position, a 3-D rotation vector and a gripper command, relative to the commanded pose at the chunk start. They convert to the challenge's end-effector delta actions.
- Tasks: LiftPot, LiftTray, StackTwoBlocks, StackSingleBookShelf, CubeHandover, PackBox, PickSingleBookFromTable and RotateValve, all on the PositionAndOrientation variation.
How to run it
The policy runs through the code repository, not through transformers. You need a Linux machine with an NVIDIA GPU
(driver >= 580 for the default CUDA 13 build; on older drivers use bash setup.sh --cuda12).
git clone --recurse-submodules https://github.com/IliaLarchenko/RoboEvalSolution.git && cd RoboEvalSolution
bash setup.sh # .venv (policy) and .venv-sim (simulator)
.venv/bin/hf download IliaLarchenko/RoboEvalSolution --local-dir ../checkpoint
CUDA_VISIBLE_DEVICES=0 .venv/bin/python scripts/serve_policy.py --config-name pi05_roboeval \
--checkpoint-dir ../checkpoint --num-candidates 6 &
MUJOCO_GL=egl .venv-sim/bin/python scripts/rollout.py --server ws://localhost:8765 --out ../runs/eval_x \
--episodes-per-task 50 --workers 20 --best configs/settings.json
rollout.py builds the environments like the official evaluator and writes the official scores to summary.json. It
refuses the official seeds 1000–1049. Stop the server afterwards with kill %1.
Inference settings
The settings matter a lot and differ per task: the number of flow steps, noise temperature, CFG scale, best-of-N
ranked by the Q head, how many actions to execute and carry over, execution speed, gripper threshold and hidden
inputs. They are in
configs/settings.json.
Without them you won't reproduce the scores above.
Training
The model is π0.5 (a PaliGemma VLM plus a flow-matching action expert) with a value query, a Q head and advantage conditioning, so it is also its own value function. I trained it in an off-policy RL loop with advantages of the exact challenge score (AWR, and advantage conditioning inspired by RECAP). Every rollout was also a Thompson-sampling trial of about 20 inference parameters. Training used about 600 GPU-hours.
Data:
- the organisers' RoboEval demonstrations;
- my own scripted demonstrations with full reward, generated in simulation from privileged state;
- more than 240k of the policy's own rollouts, plus recovery episodes that continue from its failures.
The only external pretraining is the public π0.5 base model.
Limitations
- It was trained and evaluated in simulation only and has never run on a real robot.
- It is tuned to these eight tasks, this robot, these cameras and this exact score. Don't expect it to work elsewhere without retraining.
- Part of the score comes from per-task inference tuning, which is slow and would be hard to justify in a real deployment.
License
Code and weights are released under Apache-2.0.
Citation
@misc{larchenko2026roboeval,
title = {Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge},
author = {Larchenko, Ilia},
year = {2026},
eprint = {2611.11464},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2611.11464}
}
The benchmark, simulator and demonstrations are from RoboEval (Wang et al., 2025). The base model is π0.5 by Physical Intelligence.