Tinghong-Ye22's picture
Add files using upload-large-folder tool
211ae03 verified
|
Raw History Blame Contribute Delete
8.36 kB

OpenPI $\pi_{0.5}$

Set-up Envrionment

ℹ️ The following commands assume the variables eg., PSI_HOME are loaded by running cd /path/to/psi0 && source .env.

Set up seperate environment for $\pi_{0.5}$:

ℹ️ We manage the $\Psi_0$ environment and all the baselines through uv and they all share the same src/ code. See Environment Management for more details.

uv venv .venv-openpi --python 3.10
source .venv-openpi/bin/activate
VIRTUAL_ENV=.venv-openpi uv pip install -e .
VIRTUAL_ENV=.venv-openpi uv pip install -e src/openpi/openpi-client
VIRTUAL_ENV=.venv-openpi GIT_LFS_SKIP_SMUDGE=1 uv pip install -r baselines/pi05/requirements-openpi.txt

Apply the transformers library patches:

See also the official OpenPI README.md

cp -r src/openpi/models_pytorch/transformers_replace/* .venv-openpi/lib/python3.10/site-packages/transformers/

Download Pre-Trained Weights

Download pretrained pi05_droid model of torch variant:

hf download USC-PSI-Lab/psi-model \
    --local-dir=$PSI_HOME/cache/checkpoints \
    --include="openpi/pi05_droid/*" \
    --repo-type=model
[Optional] Expand to see how we produced the `pi05_droid` pytorch checkpoint.
Download `pi05-droid` checkpoints

```
python src/openpi/shared/download.py
```

Convet from `jax` to `pytorch` checkpoint
```
uv run examples/convert_jax_model_to_pytorch.py \
    --checkpoint_dir=/home/user/.cache/openpi/openpi-assets/checkpoints/pi05_droid \
    --config_name=pi05_droid \
    --output_path $PSI_HOME/cache/checkpoints/openpi/pi05_droid
```

Download Psi0 Task Data

Download the task data, for example

export task=G1WholebodyBendPick-v1
hf download songlinwei/hfm simple/$task.zip --local-dir=$PSI_HOME/data --repo-type=dataset
unzip "$PSI_HOME/data/simple/$task.zip" -d "$PSI_HOME/data/simple"

Create a new TrainConfig for the task in src/openpi/training/config.py:

vim src/openpi/training/config.py

for example

    # ...
    TrainConfig(
        name="simple_bend_pick_v1",
        project_name="psi",
        num_workers=8,
        model=pi0_config.Pi0Config(
            pi05=True,
            action_dim=36,
            action_horizon=30,
            max_token_len=250,
        ),
        data=LeRobotHFMDataConfig(
            repo_id=f"{os.environ['PSI_HOME']}/data/simple/G1WholebodyBendPick-v1",
            base_config=DataConfig(prompt_from_task=True),
        ),
        weight_loader=weight_loaders.CheckpointWeightLoader("gs://openpi-assets/checkpoints/pi05_droid/params"),
        num_train_steps=40_000,
        batch_size=128,
        lr_schedule=_optimizer.CosineDecaySchedule(
            warmup_steps=1_000,
            peak_lr=1e-4,
            decay_steps=40_000,
            decay_lr=1e-8,
        ),
        pytorch_weight_path=f"{os.environ['PSI_HOME']}/cache/checkpoints/pi05_droid",
        policy_metadata={"dataset": "G1WholebodyBendPick-v1"},
        checkpoint_base_dir=f".runs/openpi-05"
    ),
    # ...

Compute stats for the task using official script (which is slow)

python src/openpi/compute_norm_stats.py --config-name simple_bend_pick_v1

Or you can rewrite our precomputed stats to openpi format: python src/openpi/rewrite_norm_stats.py --task_path=$PSI_HOME/data/simple/G1WholebodyBendPick-v1. The calculations are slightly different but it's ok.

Fine-Tune $\pi_{0.5}$

Launch the training script

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 torchrun --standalone --nnodes=1 --nproc_per_node=8 src/openpi/train_pytorch.py \
    simple_bend_pick_v1 \
    --exp_name=simple_bend_pick_v1 \
    --save_interval=10000 \
    --checkpoint_base_dir=.runs/openpi-05

Eval $\pi_{0.5}$

export port=9000
bash scripts/deploy/serve_pi05.sh $task $step $port

Open-loop evaluation

export PYTORCH_WEIGHT_PATH=/hfm/songlin/we_learn/.cache/checkpoints/pi05_droid
export DATA_HOME=/hfm/songlin/we_learn/.data/real_teleop_g1/lerobot
export task=Pick_bottle_and_turn_and_pour_into_cup
export port=9000

python scripts/train/openpi/eval_openloop.py --port=$port --task=$task
[Optional] Expand to see more implementation details.

Below are the gist of how to adpat OpenPI on humaonoid loco-manipulation tasks:

  1. change action dim from 32 to 36

    # vim src/openpi/models_pytorch/pi0_pytorch.py
    self.action_in_proj = nn.Linear(36, action_expert_config.width)
    self.action_out_proj = nn.Linear(action_expert_config.width, 36)
    
  2. adapt the loading of pretrained pi05_torch

    model_path = os.path.join(config.pytorch_weight_path, "model.safetensors")
    # Psi-0: adapt action dim to 36
    from safetensors.torch import load_file
    state_dict = load_file(model_path)
    pad_dim = config.model.action_dim - state_dict["action_in_proj.weight"].shape[1]
    if pad_dim > 0:
        # eg., torch.Size([1024, 32]) -> torch.Size([1024, 36])
        # Replicate the last 4 columns instead of padding with zeros
        w = state_dict["action_in_proj.weight"]
        to_pad = w[:, -pad_dim:]
        state_dict["action_in_proj.weight"] = torch.cat([w, to_pad], dim=1)
    
        b = state_dict["action_out_proj.bias"]
        state_dict["action_out_proj.bias"] = torch.cat([b, b[-pad_dim:]], dim=0)
    
        w = state_dict["action_out_proj.weight"]
        to_pad = w[-pad_dim:, :]
        state_dict["action_out_proj.weight"] = torch.cat([w, to_pad], dim=0)
    
    # https://github.com/Physical-Intelligence/openpi/issues/669
    state_dict["paligemma_with_expert.paligemma.model.language_model.embed_tokens.weight"] = \
        state_dict["paligemma_with_expert.paligemma.lm_head.weight"]
    
    _model = model.module if isinstance(model, torch.nn.parallel.DistributedDataParallel) else model
    missing_keys, unexpected_keys = _model.load_state_dict(state_dict, strict=False)
    
  3. create TrainConfig in config.py

    TrainConfig(
        name="Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw",
        project_name="hfm",
        num_workers=8,
        model=pi0_config.Pi0Config(
            pi05=True,
            action_dim=36,
            action_horizon=16,
            max_token_len=250,
        ),
        data=LeRobotHFMDataConfig( # FIXME
            repo_id= f"{os.environ['DATA_HOME']}/Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw",
            base_config=DataConfig(prompt_from_task=True),
        ),
        weight_loader=weight_loaders.CheckpointWeightLoader("gs://openpi-assets/checkpoints/pi05_droid/params"),
        num_train_steps=40_000,
        batch_size=128,
        lr_schedule=_optimizer.CosineDecaySchedule(
            warmup_steps=1_000,
            peak_lr=1e-4,
            decay_steps=40_000,
            decay_lr=1e-8,
        ),
        pytorch_weight_path=os.environ["PYTORCH_WEIGHT_PATH"],
        policy_metadata={"dataset": "Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw"},
    ),
    
  4. launch training

bash scripts/train/openpi/benchmark_pi05_nv_slurm.sh \
    Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw
  1. [Optional] upload model weights
#export task=Hold_lunch_bag_with_both_hands_and_squat_to_put_on_the_coffee_table
#export task=Pick_toys_into_box_and_lift_and_turn_and_put_on_the_chair_new_target_yaw
export task=Pull_the_tray_out_of_chips_can_and_throw_the_can_into_trash_bin
export step=40000
hf upload USC-PSI-Lab/psi-model \
    .runs/openpi-05/$task/$task/$step/model.safetensors \
    benchmarks/openpi-05/$task/$step/model.safetensors \
    --repo-type=model
  1. Serve Download:
export task=Remove_the_cap_turn_on_the_faucet_and_fill_the_bottle_with_water
export step=40000
python scripts/data/download.py \
    --repo-id=USC-PSI-Lab/psi-data-models \
    --remote-dir=benchmarks/openpi-05/$task/$step \
    --repo-type=model \
    --local-dir=.runs/openpi-05/$task/$task/$step

and Serve:

export port=9000
bash scripts/deploy/serve_pi05.sh $task $step $port

Open-loop evaluation

export PYTORCH_WEIGHT_PATH=/hfm/songlin/we_learn/.cache/checkpoints/pi05_droid
export DATA_HOME=/hfm/songlin/we_learn/.data/real_teleop_g1/lerobot
export task=Pick_bottle_and_turn_and_pour_into_cup
export port=9000

python scripts/train/openpi/eval_openloop.py --port=$port --task=$task