# Inference Deployment on Piper

Real-time inference deployment for OpenPi policies on Agilex Piper dual-arm robots.

---

## Prerequisites

### System Requirements

- **OS**: Ubuntu 20.04
- **ROS**: ROS Noetic

### Hardware Setup

**1. Piper Dual Arms**

Configure master + slave arms following the [Piper Arm Setup Guide](../deploy/Piper_ros_private-ros-noetic/README.md).

**2. RealSense Cameras**

Install ROS wrapper:
```bash
sudo apt-get install ros-noetic-realsense2-camera
```

Update camera serials in `deploy/multi_camera.launch`:
```xml
<arg name="serial_no_camera1" default="YOUR_CAMERA_1_SERIAL"/>
<arg name="serial_no_camera2" default="YOUR_CAMERA_2_SERIAL"/>
<arg name="serial_no_camera3" default="YOUR_CAMERA_3_SERIAL"/>
```

Deploy and test:
```bash
cp deploy/multi_camera.launch /opt/ros/noetic/share/realsense2_camera/launch/
roslaunch realsense2_camera multi_camera.launch
```

---

## Installation

**1. Create Environment**
```bash
conda create -n deploy python=3.10 -y
conda activate deploy
```

**2. Install Dependencies**
```bash
cd deploy
pip install -r requirements.txt
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118
```
> For other CUDA versions, see [PyTorch Installation Guide](https://pytorch.org/get-started/locally/)

**3. Install openpi-client**
```bash
cd deploy/packages/openpi-client
pip install -e .
```

---

## Running Inference

Start the [openpi policy server](https://github.com/Physical-Intelligence/openpi) on your GPU machine first.

**Launch in 3 terminals:**

```bash
# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Robot Arms
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=1 auto_enable:=true

# Terminal 3: Inference
conda activate deploy
python deploy/piper_deploy.py \
  --host 172.16.99.11 \
  --port 8000 \
  --ctrl_type joint \
  --use_temporal_smoothing \
  --chunk_size 50 \
  --lang_embeddings "Pick and sort bricks on the conveyor."
```

---

## Data Collection

Both scripts save data in HDF5 format with synchronized images, joint states, and actions.

**Output Structure:**
```
dataset_dir/
├── task_name/
│   ├── episode_0.hdf5
│   ├── episode_1.hdf5
│   └── video/
│       ├── cam_high/
│       ├── cam_left_wrist/
│       └── cam_right_wrist/
└── aloha_mobile/           # for inference data
    ├── aloha_mobile_success/
    └── aloha_mobile_fail/
```

### Teleoperation Collection

Manual control via master-slave setup.

```bash
# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Arms (master-slave mode)
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=0 auto_enable:=false

# Terminal 3: Collection
conda activate deploy
cd deploy/data_collection
python collect_data.py \
  --max_timesteps 5000 \
  --export_video \
  --dataset_dir ~/data/my_task \
  --episode_idx 0
```

**Controls:** Press `Space` to stop and save early.

### Inference Collection

Record autonomous policy execution. Requires running policy server.

```bash
# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Arms (inference mode)
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=1 auto_enable:=true

# Terminal 3: Collection
conda activate deploy
cd deploy/data_collection
python collect_inference_data.py \
  --use_temporal_smoothing \
  --ctrl_type joint \
  --export_video \
  --chunk_size 50 \
  --host 172.16.99.11 \
  --port 8000 \
  --dataset_dir ~/data/rl_task
```

**Workflow:**
1. Press `Enter` to start recording
2. Press `s` to stop robot
3. Label episode: `1` (success) or `0` (failure)