Hugging Face's logo

hchen657Naza
/
PWM

Diffusers
Safetensors
Model card Files
xet
 Community
PWM
File size: 3,840 Bytes
23e1f91
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
# Inference Deployment on Piper

Real-time inference deployment for OpenPi policies on Agilex Piper dual-arm robots.

---

## Prerequisites

### System Requirements

- **OS**: Ubuntu 20.04
- **ROS**: ROS Noetic

### Hardware Setup

**1. Piper Dual Arms**

Configure master + slave arms following the [Piper Arm Setup Guide](../deploy/Piper_ros_private-ros-noetic/README.md).

**2. RealSense Cameras**

Install ROS wrapper:
```bash
sudo apt-get install ros-noetic-realsense2-camera
```

Update camera serials in `deploy/multi_camera.launch`:
```xml
<arg name="serial_no_camera1" default="YOUR_CAMERA_1_SERIAL"/>
<arg name="serial_no_camera2" default="YOUR_CAMERA_2_SERIAL"/>
<arg name="serial_no_camera3" default="YOUR_CAMERA_3_SERIAL"/>
```

Deploy and test:
```bash
cp deploy/multi_camera.launch /opt/ros/noetic/share/realsense2_camera/launch/
roslaunch realsense2_camera multi_camera.launch
```

---

## Installation

**1. Create Environment**
```bash
conda create -n deploy python=3.10 -y
conda activate deploy
```

**2. Install Dependencies**
```bash
cd deploy
pip install -r requirements.txt
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118
```
> For other CUDA versions, see [PyTorch Installation Guide](https://pytorch.org/get-started/locally/)

**3. Install openpi-client**
```bash
cd deploy/packages/openpi-client
pip install -e .
```

---

## Running Inference

Start the [openpi policy server](https://github.com/Physical-Intelligence/openpi) on your GPU machine first.

**Launch in 3 terminals:**

```bash
# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Robot Arms
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=1 auto_enable:=true

# Terminal 3: Inference
conda activate deploy
python deploy/piper_deploy.py \
  --host 172.16.99.11 \
  --port 8000 \
  --ctrl_type joint \
  --use_temporal_smoothing \
  --chunk_size 50 \
  --lang_embeddings "Pick and sort bricks on the conveyor."
```

---

## Data Collection

Both scripts save data in HDF5 format with synchronized images, joint states, and actions.

**Output Structure:**
```
dataset_dir/
├── task_name/
│   ├── episode_0.hdf5
│   ├── episode_1.hdf5
│   └── video/
│       ├── cam_high/
│       ├── cam_left_wrist/
│       └── cam_right_wrist/
└── aloha_mobile/           # for inference data
    ├── aloha_mobile_success/
    └── aloha_mobile_fail/
```

### Teleoperation Collection

Manual control via master-slave setup.

```bash
# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Arms (master-slave mode)
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=0 auto_enable:=false

# Terminal 3: Collection
conda activate deploy
cd deploy/data_collection
python collect_data.py \
  --max_timesteps 5000 \
  --export_video \
  --dataset_dir ~/data/my_task \
  --episode_idx 0
```

**Controls:** Press `Space` to stop and save early.

### Inference Collection

Record autonomous policy execution. Requires running policy server.

```bash
# Terminal 1: Cameras
roslaunch realsense2_camera multi_camera.launch

# Terminal 2: Arms (inference mode)
bash deploy/Piper_ros_private-ros-noetic/can_config.sh

roslaunch piper start_ms_piper.launch mode:=1 auto_enable:=true

# Terminal 3: Collection
conda activate deploy
cd deploy/data_collection
python collect_inference_data.py \
  --use_temporal_smoothing \
  --ctrl_type joint \
  --export_video \
  --chunk_size 50 \
  --host 172.16.99.11 \
  --port 8000 \
  --dataset_dir ~/data/rl_task
```

**Workflow:**
1. Press `Enter` to start recording
2. Press `s` to stop robot
3. Label episode: `1` (success) or `0` (failure)