PWM_code/RISE_aux/docs/deploy.md

Inference Deployment on Piper

Real-time inference deployment for OpenPi policies on Agilex Piper dual-arm robots.

---

Prerequisites

System Requirements

• OS: Ubuntu 20.04
• ROS: ROS Noetic

Hardware Setup

1. Piper Dual Arms

Configure master + slave arms following the Piper Arm Setup Guide.

2. RealSense Cameras

Install ROS wrapper:

sudo apt-get install ros-noetic-realsense2-camera

Update camera serials in deploy/multi_camera.launch:

<arg name="serial_no_camera1" default="YOUR_CAMERA_1_SERIAL"/>
<arg name="serial_no_camera2" default="YOUR_CAMERA_2_SERIAL"/>
<arg name="serial_no_camera3" default="YOUR_CAMERA_3_SERIAL"/>

Deploy and test:

cp deploy/multi_camera.launch /opt/ros/noetic/share/realsense2_camera/launch/
roslaunch realsense2_camera multi_camera.launch

---

Installation

1. Create Environment

conda create -n deploy python=3.10 -y
conda activate deploy

2. Install Dependencies

cd deploy
pip install -r requirements.txt
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118

For other CUDA versions, see PyTorch Installation Guide

3. Install openpi-client

cd deploy/packages/openpi-client
pip install -e .

---

Running Inference

Start the openpi policy server on your GPU machine first.

Launch in 3 terminals:

# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Robot Arms
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=1 auto_enable:=true

# Terminal 3: Inference
conda activate deploy
python deploy/piper_deploy.py \
  --host 172.16.99.11 \
  --port 8000 \
  --ctrl_type joint \
  --use_temporal_smoothing \
  --chunk_size 50 \
  --lang_embeddings "Pick and sort bricks on the conveyor."

---

Data Collection

Both scripts save data in HDF5 format with synchronized images, joint states, and actions.

Output Structure:

dataset_dir/
├── task_name/
│   ├── episode_0.hdf5
│   ├── episode_1.hdf5
│   └── video/
│       ├── cam_high/
│       ├── cam_left_wrist/
│       └── cam_right_wrist/
└── aloha_mobile/           # for inference data
    ├── aloha_mobile_success/
    └── aloha_mobile_fail/

Teleoperation Collection

Manual control via master-slave setup.

# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Arms (master-slave mode)
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=0 auto_enable:=false

# Terminal 3: Collection
conda activate deploy
cd deploy/data_collection
python collect_data.py \
  --max_timesteps 5000 \
  --export_video \
  --dataset_dir ~/data/my_task \
  --episode_idx 0

Controls: Press Space to stop and save early.

Inference Collection

Record autonomous policy execution. Requires running policy server.

# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Arms (inference mode)
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=1 auto_enable:=true

# Terminal 3: Collection
conda activate deploy
cd deploy/data_collection
python collect_inference_data.py \
  --use_temporal_smoothing \
  --ctrl_type joint \
  --export_video \
  --chunk_size 50 \
  --host 172.16.99.11 \
  --port 8000 \
  --dataset_dir ~/data/rl_task

Workflow:

1. Press Enter to start recording
2. Press s to stop robot
3. Label episode: 1 (success) or 0 (failure)