pi05-rm65-lora

This repository contains an inference-ready JAX/Orbax checkpoint of OpenPI pi0.5, LoRA-finetuned for an RM65 single-arm pick-and-place task.

The task used during training was:

Pick up the purple-and-white box and place it into the cardboard box.

Important

• This is a full inference checkpoint, not a standalone LoRA adapter. A separate pi05_base download is not required for inference.
• The optimizer and continuation state (train_state) are intentionally not included. This repository is intended for inference, not resuming training.
• The training dataset is not included.
• At publication time the model had passed offline replay checks, but had not yet been validated for autonomous execution on a real robot.
• This model is not a safety controller. Add workspace, velocity, delta-action, gripper, timeout, and emergency-stop safeguards before commanding hardware.

Model and training details

Item	Value
Base model	OpenPI pi05_base
OpenPI model type	pi0.5 flow-matching head
Fine-tuning	LoRA (gemma_2b_lora + gemma_300m_lora)
Training steps	30,000
Published checkpoint	Step 29,999
Batch size	8
Action horizon	16
Internal action dimension	32 (padded)
Returned action dimension	7
Dataset	Local LeRobot v3, 81 episodes, 79,827 frames
Cameras	Front RGB + wrist RGB

The final recorded training loss at step 29,990 was 0.0127. This value is a training diagnostic and is not a real-robot success metric.

Input and output contract

Send the following observation dictionary to the OpenPI policy server:

observation = {
    # uint8 RGB in HWC layout is recommended. The server resizes to 224x224.
    "observation/image": front_rgb,
    "observation/wrist_image": wrist_rgb,

    # [tcp_x, tcp_y, tcp_z, tcp_rx, tcp_ry, tcp_rz, gripper]
    # XYZ is in meters, RPY is in radians, gripper is normalized to about [0, 1].
    "observation/state": state,

    "prompt": "Pick up the purple-and-white box and place it into the cardboard box.",
}

Camera mapping:

Request field	Model image field
observation/image	base_0_rgb
observation/wrist_image	left_wrist_0_rgb
Missing third camera	right_wrist_0_rgb (zeros, masked out)

The returned actions array has shape (16, 7):

[delta_x, delta_y, delta_z, delta_rx, delta_ry, delta_rz, target_gripper]

The first six values are TCP deltas in the same convention used by the training data. The seventh value is an absolute normalized gripper target. Clamp and validate every value in the robot-side safety layer before use.

Installation

The checkpoint uses a small RM65 integration patch on top of OpenPI commit 15a9616a00943ada6c20a0f158e3adb39df2ccac. The exact four patches used to train and serve this checkpoint are included under openpi-patches/.

git clone --recurse-submodules https://github.com/Physical-Intelligence/openpi.git
cd openpi
git checkout 15a9616a00943ada6c20a0f158e3adb39df2ccac
git am /path/to/pi05-rm65-lora/openpi-patches/*.patch

GIT_LFS_SKIP_SMUDGE=1 uv sync
GIT_LFS_SKIP_SMUDGE=1 uv pip install -e .

Download this model:

hf download ireash/pi05-rm65-lora \
  --local-dir /path/to/pi05-rm65-lora

Start the policy server:

OPENPI_DATA_HOME="$HOME/.cache/openpi" \
XLA_PYTHON_CLIENT_MEM_FRACTION=0.85 \
uv run scripts/serve_policy.py \
  --default-prompt="Pick up the purple-and-white box and place it into the cardboard box." \
  --port=8000 \
  policy:checkpoint \
  --policy.config=pi05_rm65_lora \
  --policy.dir=/path/to/pi05-rm65-lora

The PaliGemma tokenizer is downloaded on first use unless it is already in the OpenPI cache.

Offline replay check

An offline check sampled three points from each of all 81 training episodes (243 requests total):

• 243/243 requests returned finite (16, 7) actions.
• Mean loopback inference latency on an RTX 4090 was 0.164 seconds; p95 was 0.188 seconds after compilation.
• Mean action MAE against the recorded training chunks was 0.00546.
• Gripper open/close agreement after thresholding at 0.5 was 99.02%.

These are training-set replay diagnostics, not held-out evaluation results.

Limitations and safety

• The model was tuned for one task, one robot state/action convention, and the camera viewpoints present in the training data.
• Network latency, camera placement, tool/work coordinate frames, gripper direction, and controller timing must match the deployment setup.
• Sampled gripper predictions can slightly exceed [0, 1]; offline replay observed approximately [-0.0143, 1.0118]. Always clip them.
• Begin real-robot evaluation with read-only telemetry, then dry-run logging, then guarded low-speed single-step motion with an operator at the emergency stop. Do not send an entire action chunk open-loop during initial testing.

Provenance

• Upstream project: https://github.com/Physical-Intelligence/openpi
• Upstream base code commit: 15a9616a00943ada6c20a0f158e3adb39df2ccac
• Fine-tuning code commit: e7f752ba8d41aa3f82c31f106de416fb26bdafdf
• License: Apache-2.0