pi05_ur5_polo_rtc_test

A pipeline test checkpoint: openpi ฯ€0.5, fully fine-tuned on dual UR5 + Robotiq polo-shirt folding demonstrations, with training-time real-time chunking. It was trained to validate data conversion, training and evaluation end to end. It is not a tuned or robot-validated model (55 demonstrations, 5000 steps).

Contents

Path
params/ JAX / Orbax (OCDBT) parameters, step 4999 (EMA)
assets/ normalization statistics used at inference
eval/ open-loop and real-time-chunking metrics and plots (below)

Robot interface

  • 20 Hz, 14-dim state and action: [left 6 joints, left gripper, right 6 joints, right gripper].
  • Joints in rad. The right arm is in a mirrored "logical" frame: physical = sign * logical + offset with sign = [1, 1, -1, 1, -1, 1], offset = [0, -40ยฐ, 0, 0, 0, 0].
  • Gripper: 0 = open, 1 = closed (Robotiq position / 255).
  • Three cameras: top view and both wrists, 1280x720 padded (aspect kept, black bars) to 224x224.
  • Actions: 50-step chunks (2.5 s). Joints are predicted as deltas to the current state, grippers absolute.
  • Prompt: Fold the t-shirt. (the task string stored with the recordings).

Training

  • Data: yzsun2025/eai_dataset, RAW_DATA/fold_the_polo_extra_procedure_hdf5. Episodes sorted by number; the first 55 for training (35,070 frames), the next 5 held out.
  • Model: Pi0Config(pi05=True, action_horizon=50, rtc_training_max_delay=5), quantile normalization.
  • Initialized from openpi pi05_base; full fine-tune, batch 32, 5000 steps, AdamW, peak LR 5e-5 with 1000 warmup steps (cosine schedule configured for 40k steps, so the LR was still near its peak at the end), EMA 0.999. One A100 80GB, about 5.5 h. Final training loss about 0.0065.
  • Training-time real-time chunking: for each sample a delay d โˆˆ [0, 5] is drawn, the first d actions are given to the model clean (flow time 0) and excluded from the loss. At inference the actions the robot will still execute from the previous chunk can be held fixed while the rest is generated.

Evaluation (step 4999, offline)

Open-loop: the model is queried every 10 frames with recorded observations and compared with the recorded actions (joint and gripper dims, rad / gripper units).

Episodes MAE, first 10 steps MAE, full 50 steps
held out (5) 0.016 0.045
training (3) 0.010 0.014

Real-time chunking, 5 held-out episodes, a new chunk every 10 steps, mean jump at a chunk switch over the 12 joints (rad; an ordinary step is about 0.009):

Mode delay 3 delay 5 A100, in process RTX 4090, websocket round trip (median / p95)
none 0.059 0.061 84 ms 82 / 83 ms
guided (inference-time) 0.051 0.046 138 ms 134 / 138 ms
trained (prefix held fixed) 0.018 0.016 85 ms 84 / 85 ms

Offline replay follows the recording, not the policy, so it measures smoothness at chunk switches, not closed-loop success. The RTX 4090 round trip is client and serve_policy.py on the same machine with three 224x224 uint8 images and the state per request: in trained mode that is 2 control steps at 20 Hz. The first request of each mode compiles for 15โ€“19 s.

Usage

Requires an openpi fork with the pi05_ur5_polo training config and training-time RTC (Pi0Config.rtc_training_max_delay, Pi0.sample_actions_prefix); it does not load in upstream openpi or LeRobot as is.

hf download yzsun2025/pi05_ur5_polo_rtc_test --local-dir checkpoints/pi05_ur5_polo/polo_rtc_test/4999
uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi05_ur5_polo \
    --policy.dir=checkpoints/pi05_ur5_polo/polo_rtc_test/4999

Requests choose the chunking mode per call (rtc_mode, rtc_s, rtc_delay).

Built on openpi pi05_base (Physical Intelligence), which builds on PaliGemma. Use is subject to the terms of those base models.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Dataset used to train yzsun2025/pi05_ur5_polo_rtc_test