IliaLarchenko/RoboEvalSolution

RoboEval CoRL 2026 solution: off-policy RL with online inference tuning

This is the policy I submitted to the RoboEval Challenge at CoRL 2026: one modified π0.5 for eight bimanual tasks with two Franka arms, in simulation. The weights are an average of two checkpoints of the same run, 0.65 × step 48250 + 0.35 × step 58000.

• Code: https://github.com/IliaLarchenko/RoboEvalSolution
• Report: https://arxiv.org/abs/2611.11464
• Project page: https://ilialarchenko.com/projects/roboeval2026/

Results

Scored with the official RoboEval Diagnostic Score (RDS), each task with its final inference settings.

	Episodes per task	RDS	Success
Official seeds (1000–1049)	50	0.987	99.25%
Held-out random seeds	500	0.990	99.60%

Task	Official success	Official RDS	Held-out success	Held-out RDS
LiftPot	1.000	1.000	1.000	0.999
LiftTray	0.980	0.984	0.998	0.998
StackTwoBlocks	1.000	0.977	0.996	0.979
StackSingleBookShelf	1.000	1.000	0.998	0.998
CubeHandover	1.000	1.000	1.000	0.998
PackBox	1.000	0.969	1.000	0.974
PickSingleBookFromTable	0.960	0.967	0.988	0.988
RotateValve	1.000	1.000	0.988	0.989

A 50-episode run varies by about ±0.005 RDS, so the two rows agree within noise. I made every decision on held-out seeds and never tuned on the official ones.

Inputs and outputs

• Inputs: five RGB cameras at 256×256 (head, front, external, left wrist, right wrist), proprioception (joint positions and velocities, gripper states) and a task id. There is no text prompt. Some tasks hide some of these inputs at inference.
• Actions: chunks of 30 steps (1.5 s at 20 Hz) of 14-D end-effector actions for both arms: per arm a 3-D position, a 3-D rotation vector and a gripper command, relative to the commanded pose at the chunk start. They convert to the challenge's end-effector delta actions.
• Tasks: LiftPot, LiftTray, StackTwoBlocks, StackSingleBookShelf, CubeHandover, PackBox, PickSingleBookFromTable and RotateValve, all on the PositionAndOrientation variation.

How to run it

The policy runs through the code repository, not through transformers. You need a Linux machine with an NVIDIA GPU (driver >= 580 for the default CUDA 13 build; on older drivers use bash setup.sh --cuda12).

git clone --recurse-submodules https://github.com/IliaLarchenko/RoboEvalSolution.git && cd RoboEvalSolution
bash setup.sh    # .venv (policy) and .venv-sim (simulator)
.venv/bin/hf download IliaLarchenko/RoboEvalSolution --local-dir ../checkpoint

CUDA_VISIBLE_DEVICES=0 .venv/bin/python scripts/serve_policy.py --config-name pi05_roboeval \
    --checkpoint-dir ../checkpoint --num-candidates 6 &
MUJOCO_GL=egl .venv-sim/bin/python scripts/rollout.py --server ws://localhost:8765 --out ../runs/eval_x \
    --episodes-per-task 50 --workers 20 --best configs/settings.json

rollout.py builds the environments like the official evaluator and writes the official scores to summary.json. It refuses the official seeds 1000–1049. Stop the server afterwards with kill %1.

Inference settings

The settings matter a lot and differ per task: the number of flow steps, noise temperature, CFG scale, best-of-N ranked by the Q head, how many actions to execute and carry over, execution speed, gripper threshold and hidden inputs. They are in configs/settings.json. Without them you won't reproduce the scores above.

Training

The model is π0.5 (a PaliGemma VLM plus a flow-matching action expert) with a value query, a Q head and advantage conditioning, so it is also its own value function. I trained it in an off-policy RL loop with advantages of the exact challenge score (AWR, and advantage conditioning inspired by RECAP). Every rollout was also a Thompson-sampling trial of about 20 inference parameters. Training used about 600 GPU-hours.

Data:

• the organisers' RoboEval demonstrations;
• my own scripted demonstrations with full reward, generated in simulation from privileged state;
• more than 240k of the policy's own rollouts, plus recovery episodes that continue from its failures.

The only external pretraining is the public π0.5 base model.

Limitations

• It was trained and evaluated in simulation only and has never run on a real robot.
• It is tuned to these eight tasks, this robot, these cameras and this exact score. Don't expect it to work elsewhere without retraining.
• Part of the score comes from per-task inference tuning, which is slow and would be hard to justify in a real deployment.

License

Code and weights are released under Apache-2.0.

Citation

@misc{larchenko2026roboeval,
  title  = {Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge},
  author = {Larchenko, Ilia},
  year   = {2026},
  eprint = {2611.11464},
  archivePrefix = {arXiv},
  primaryClass = {cs.RO},
  url    = {https://arxiv.org/abs/2611.11464}
}

The benchmark, simulator and demonstrations are from RoboEval (Wang et al., 2025). The base model is π0.5 by Physical Intelligence.