## OpenPI $\pi_{0.5}$
### Set-up Envrionment
> ℹ️ The following commands assume the variables eg., `PSI_HOME` are loaded by running `cd /path/to/psi0 && source .env`.
Set up seperate environment for $\pi_{0.5}$:
> ℹ️ We manage the $\Psi_0$ environment and all the baselines through `uv` and they all share the same `src/` code. See [Environment Management](../README.md) for more details.
```
uv venv .venv-openpi --python 3.10
source .venv-openpi/bin/activate
VIRTUAL_ENV=.venv-openpi uv pip install -e .
VIRTUAL_ENV=.venv-openpi uv pip install -e src/openpi/openpi-client
VIRTUAL_ENV=.venv-openpi GIT_LFS_SKIP_SMUDGE=1 uv pip install -r baselines/pi05/requirements-openpi.txt
```
Apply the `transformers` library patches:
> See also the official OpenPI [README.md](https://github.com/Physical-Intelligence/openpi?tab=readme-ov-file#setup)
```
cp -r src/openpi/models_pytorch/transformers_replace/* .venv-openpi/lib/python3.10/site-packages/transformers/
```
### Download Pre-Trained Weights
Download pretrained `pi05_droid` model of `torch` variant:
```
hf download USC-PSI-Lab/psi-model \
--local-dir=$PSI_HOME/cache/checkpoints \
--include="openpi/pi05_droid/*" \
--repo-type=model
```
[Optional] Expand to see how we produced the `pi05_droid` pytorch checkpoint.
Download `pi05-droid` checkpoints
```
python src/openpi/shared/download.py
```
Convet from `jax` to `pytorch` checkpoint
```
uv run examples/convert_jax_model_to_pytorch.py \
--checkpoint_dir=/home/user/.cache/openpi/openpi-assets/checkpoints/pi05_droid \
--config_name=pi05_droid \
--output_path $PSI_HOME/cache/checkpoints/openpi/pi05_droid
```
### Download Psi0 Task Data
Download the task data, for example
```
export task=G1WholebodyBendPick-v1
```
```
hf download songlinwei/hfm simple/$task.zip --local-dir=$PSI_HOME/data --repo-type=dataset
unzip "$PSI_HOME/data/simple/$task.zip" -d "$PSI_HOME/data/simple"
```
Create a new `TrainConfig` for the task in `src/openpi/training/config.py`:
```
vim src/openpi/training/config.py
```
for example
```
# ...
TrainConfig(
name="simple_bend_pick_v1",
project_name="psi",
num_workers=8,
model=pi0_config.Pi0Config(
pi05=True,
action_dim=36,
action_horizon=30,
max_token_len=250,
),
data=LeRobotHFMDataConfig(
repo_id=f"{os.environ['PSI_HOME']}/data/simple/G1WholebodyBendPick-v1",
base_config=DataConfig(prompt_from_task=True),
),
weight_loader=weight_loaders.CheckpointWeightLoader("gs://openpi-assets/checkpoints/pi05_droid/params"),
num_train_steps=40_000,
batch_size=128,
lr_schedule=_optimizer.CosineDecaySchedule(
warmup_steps=1_000,
peak_lr=1e-4,
decay_steps=40_000,
decay_lr=1e-8,
),
pytorch_weight_path=f"{os.environ['PSI_HOME']}/cache/checkpoints/pi05_droid",
policy_metadata={"dataset": "G1WholebodyBendPick-v1"},
checkpoint_base_dir=f".runs/openpi-05"
),
# ...
```
Compute stats for the task using official script (which is slow)
```
python src/openpi/compute_norm_stats.py --config-name simple_bend_pick_v1
```
> Or you can rewrite our precomputed stats to `openpi` format:
> `python src/openpi/rewrite_norm_stats.py --task_path=$PSI_HOME/data/simple/G1WholebodyBendPick-v1`.
> The calculations are slightly different but it's ok.
### Fine-Tune $\pi_{0.5}$
Launch the training script
```
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 torchrun --standalone --nnodes=1 --nproc_per_node=8 src/openpi/train_pytorch.py \
simple_bend_pick_v1 \
--exp_name=simple_bend_pick_v1 \
--save_interval=10000 \
--checkpoint_base_dir=.runs/openpi-05
```
### Eval $\pi_{0.5}$
```
export port=9000
bash scripts/deploy/serve_pi05.sh $task $step $port
```
Open-loop evaluation
```
export PYTORCH_WEIGHT_PATH=/hfm/songlin/we_learn/.cache/checkpoints/pi05_droid
export DATA_HOME=/hfm/songlin/we_learn/.data/real_teleop_g1/lerobot
export task=Pick_bottle_and_turn_and_pour_into_cup
export port=9000
python scripts/train/openpi/eval_openloop.py --port=$port --task=$task
```
[Optional] Expand to see more implementation details.
Below are the gist of how to adpat OpenPI on humaonoid loco-manipulation tasks:
1. change action dim from `32` to `36`
```
# vim src/openpi/models_pytorch/pi0_pytorch.py
self.action_in_proj = nn.Linear(36, action_expert_config.width)
self.action_out_proj = nn.Linear(action_expert_config.width, 36)
```
2. adapt the loading of pretrained `pi05_torch`
```
model_path = os.path.join(config.pytorch_weight_path, "model.safetensors")
# Psi-0: adapt action dim to 36
from safetensors.torch import load_file
state_dict = load_file(model_path)
pad_dim = config.model.action_dim - state_dict["action_in_proj.weight"].shape[1]
if pad_dim > 0:
# eg., torch.Size([1024, 32]) -> torch.Size([1024, 36])
# Replicate the last 4 columns instead of padding with zeros
w = state_dict["action_in_proj.weight"]
to_pad = w[:, -pad_dim:]
state_dict["action_in_proj.weight"] = torch.cat([w, to_pad], dim=1)
b = state_dict["action_out_proj.bias"]
state_dict["action_out_proj.bias"] = torch.cat([b, b[-pad_dim:]], dim=0)
w = state_dict["action_out_proj.weight"]
to_pad = w[-pad_dim:, :]
state_dict["action_out_proj.weight"] = torch.cat([w, to_pad], dim=0)
# https://github.com/Physical-Intelligence/openpi/issues/669
state_dict["paligemma_with_expert.paligemma.model.language_model.embed_tokens.weight"] = \
state_dict["paligemma_with_expert.paligemma.lm_head.weight"]
_model = model.module if isinstance(model, torch.nn.parallel.DistributedDataParallel) else model
missing_keys, unexpected_keys = _model.load_state_dict(state_dict, strict=False)
```
3. create TrainConfig in config.py
```
TrainConfig(
name="Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw",
project_name="hfm",
num_workers=8,
model=pi0_config.Pi0Config(
pi05=True,
action_dim=36,
action_horizon=16,
max_token_len=250,
),
data=LeRobotHFMDataConfig( # FIXME
repo_id= f"{os.environ['DATA_HOME']}/Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw",
base_config=DataConfig(prompt_from_task=True),
),
weight_loader=weight_loaders.CheckpointWeightLoader("gs://openpi-assets/checkpoints/pi05_droid/params"),
num_train_steps=40_000,
batch_size=128,
lr_schedule=_optimizer.CosineDecaySchedule(
warmup_steps=1_000,
peak_lr=1e-4,
decay_steps=40_000,
decay_lr=1e-8,
),
pytorch_weight_path=os.environ["PYTORCH_WEIGHT_PATH"],
policy_metadata={"dataset": "Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw"},
),
```
4. launch training
```
bash scripts/train/openpi/benchmark_pi05_nv_slurm.sh \
Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw
```
5. [Optional] upload model weights
```
#export task=Hold_lunch_bag_with_both_hands_and_squat_to_put_on_the_coffee_table
#export task=Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw
export task=Pull_the_tray_out_of_chips_can_and_throw_the_can_into_trash_bin
export step=40000
hf upload USC-PSI-Lab/psi-model \
.runs/openpi-05/$task/$task/$step/model.safetensors \
benchmarks/openpi-05/$task/$step/model.safetensors \
--repo-type=model
```
6. Serve
Download:
```
export task=Remove_the_cap_turn_on_the_faucet_and_fill_the_bottle_with_water
export step=40000
python scripts/data/download.py \
--repo-id=USC-PSI-Lab/psi-data-models \
--remote-dir=benchmarks/openpi-05/$task/$step \
--repo-type=model \
--local-dir=.runs/openpi-05/$task/$task/$step
```
and Serve:
```
export port=9000
bash scripts/deploy/serve_pi05.sh $task $step $port
```
Open-loop evaluation
```
export PYTORCH_WEIGHT_PATH=/hfm/songlin/we_learn/.cache/checkpoints/pi05_droid
export DATA_HOME=/hfm/songlin/we_learn/.data/real_teleop_g1/lerobot
export task=Pick_bottle_and_turn_and_pour_into_cup
export port=9000
python scripts/train/openpi/eval_openloop.py --port=$port --task=$task
```