myx160/pi05-pap-rtc

Pi0.5 PAP policy — training-time RTC

This public repository contains a complete JAX inference release: Orbax model parameters, checkpoint-local normalization, the corresponding OpenPI adaptations, an independent inference configuration, a bundled tokenizer, pinned dependencies, and a synthetic offline example.

Select a checkpoint with a fixed revision, such as step-027000. This is an intermediate checkpoint at step 27,000 of a planned 30,000-step full fine-tuning run. Later checkpoints use the same repository and paths, with tags step-027000 through step-030000 as they are published. Only entries marked published in releases.json are available. main is the latest published release; deployments should record the tag and resolved commit.

Download a specific release

Using an existing Hugging Face tooling environment:

from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="myx160/pi05-pap-rtc",
    revision="step-027000",
    local_dir="./pi05-pap-rtc-step027000",
)

This downloads one release, not all historical checkpoints. Keep the params/ OCDBT directory structure intact. This format is loaded by OpenPI/Orbax, not AutoModel.from_pretrained.

Install and run the example

Requirements: Linux x86_64, Python 3.11, a compatible NVIDIA driver for the pinned CUDA 12 wheels, and uv. All environments and caches default to paths under the downloaded snapshot.

cd pi05-pap-rtc-step027000
source runtime/env.sh
uv sync --project runtime --python 3.11 --locked --no-dev
JAX_PLATFORMS=cpu runtime/.venv/bin/python runtime/verify_preprocessing.py
CUDA_VISIBLE_DEVICES=0 runtime/.venv/bin/python runtime/smoke_infer.py --output smoke_output.npz

The dependency index is recorded in runtime/uv.lock; no dependency upgrade is needed. runtime/env.sh makes the vendored modules importable and selects on-demand CUDA allocation without fixed memory fraction or preallocation. Select one visible GPU for inference. The runtime operates locally after download, including tokenization; training data, training-machine paths, the training split file, and access to GCS are not required.

The example is synthetic and performs no robot actuation. Its results are a loading/interface test, not a robot task success evaluation. runtime/validation.json records the shared runtime validation performed at step 3000. Step-27000 checkpoint checks are recorded in checkpoint_info.json; full GPU inference has not been rerun for these weights. Preprocessing reference arrays were generated with the original training workspace; runtime/examples contains the input and reference outputs.

Use the policy

After sourcing runtime/env.sh, import the public entry point:

from pap_runtime import load_policy, load_request

root = "."  # path to this downloaded snapshot
policy = load_policy(root, seed=0)  # default: 10 flow integration steps
obs = load_request("runtime/examples/synthetic_request.npz")
result = policy.infer(obs)
print(result["actions"].shape)         # (24, 16), absolute robot targets
print(result["action_postfix"].shape)  # (12, 16), newly predicted targets

Replace the synthetic observation with the real controller inputs described in INTERFACE.md. The policy requires a committed future 12-step action prefix in raw absolute robot coordinates. It keeps that prefix fixed throughout sampling. The executor appends only the returned postfix at its matching time indices.

WebSocket server and client

The OpenPI msgpack/NumPy server and client are included. After the same installation and environment setup:

CUDA_VISIBLE_DEVICES=0 runtime/.venv/bin/python runtime/serve.py --host 127.0.0.1 --port 8000
# In another shell, source runtime/env.sh first:
runtime/.venv/bin/python runtime/client_example.py --host 127.0.0.1 --port 8000

Set --host to the deployment interface when serving another machine. The server exposes /healthz and returns policy metadata when a client connects. The initial request includes JIT compilation; warm up with an offline request before measuring latency. The client example only saves outputs; robot execution and queue scheduling are handled by the caller.

Model and preprocessing

• Official pi05_base initialization, 3,353,433,872 fully trainable parameters.
• Training: two A100 80GB GPUs, pure data parallelism, global batch 96 / 48 per GPU, activation recomputation, BF16 computation, FP32 master parameters and Adam state, EMA disabled.
• Fixed action horizon 24 = 12 clean prefix + 12 predicted postfix; training loss on postfix only.
• Three images: cam_left_high, cam_left_wrist, cam_right_wrist; resize with padding to 224×224.
• Robot representation: 14 arm joints followed by two grippers; model internal dimension 32.
• Arm actions: delta from the current state internally, restored to absolute targets on output. Gripper actions remain absolute throughout the robot-coordinate transformations.
• Both action grippers share the training-data min/max range [0.000015554427591268905, 5.400000095367432]. This is an empirical command range, not a physical calibration. State normalization is separate. Every release includes its own assets/ statistics.
• The original state and 14-joint quantile ranges are preserved. State is discretized into the pi05 text prompt after normalization; maximum text length is 200 tokens.

Training optimizer: AdamW beta1=0.9, beta2=0.95, epsilon=1e-8, weight decay=1e-10, global gradient clipping 1.0. Learning rate warms up for 1,000 steps to 2.5e-5 and cosine-decays to 2.5e-6 at step 30,000. Training uses 181 episodes / 47,629 frames. Adam state remains in the local training checkpoints and is not part of these inference releases.

Source and evaluation

The runtime dependency closure is vendored in runtime/src/, including all inference-relevant RTC, Gemma, Pi0 and PAP policy changes. runtime/pap_runtime.py is the portable replacement for the training-only pi05_pap_rtc factory. Its configuration is config.json. runtime/provenance.json records the upstream commit, tokenizer hash, and training model source hashes. The export adds a local tokenizer path option and fixes the policy constructor to accept an explicit JAX PRNG key. Model computation and preprocessing formulas are unchanged. Original source license: runtime/OPENPI_LICENSE.

The implementation is based on OpenPI, commit 215abfb217dbac7d5f1273282331b9b1866c0479. Base weights originate from Physical Intelligence pi05_base. Bundled tokenizer provenance is documented in runtime/provenance.json.

At step 3000, mean training loss over steps 2501–3000 was approximately 0.00640. Held-out loss and robot task success have not been measured for this checkpoint. See RELEASING.md for the versioning procedure used for subsequent checkpoints.

Current release: step 27000. The example reference output and runtime validation were produced at step 3000; they document runtime behavior, not expected numerical predictions from step 27000.

Checkpoint 18000 publication

This revision contains the finalized step-18000 weights and their own normalization. The reusable runtime was validated with step 3000. A separate inference/robot evaluation for these new weights has not been run. The bundled reference_output.npz and its JSON report remain a step-3000 runtime baseline, not the expected output of this checkpoint.

Checkpoint 24000 publication

This revision contains the finalized step-24000 weights and their own normalization. The reusable runtime was validated with step 3000. A separate inference/robot evaluation for these new weights has not been run. The bundled reference_output.npz and its JSON report remain a step-3000 runtime baseline, not the expected output of this checkpoint.

Checkpoint 27000 publication

This revision contains the finalized step-27000 weights and their own normalization. The reusable runtime was validated with step 3000. A separate inference/robot evaluation for these new weights has not been run. The bundled reference_output.npz and its JSON report remain a step-3000 runtime baseline, not the expected output of this checkpoint.