code-Psi0 / examples /simple /README.md
Tinghong-Ye22's picture
Add files using upload-large-folder tool
2d3265e verified
|
Raw History Blame Contribute Delete
1.61 kB
# Evaluate Policies on SIMPLE
pull `SIMPLE`
```
git submodule update --init --recursive
cd third_party/SIMPLE
```
## Install Dokcer version
Setup environment variables
```
cp .env.sample .env;
# then edit the .env file to set the "DATA_DIR" to current directory (e.g., `DATA_DIR=/home/***/projects/psi/third_party/SIMPLE/data`)
```
Build docker image
```
docker compose build isaac-sim
```
## Install UV version (TODO)
```
git submodule update --init --recursive third_party/SIMPLE
```
## Evaluation
copy the training dataset to SIMPLE `data`, which will be mounted into the docker container
```
cp -r $DATA_HOME/G1WholebodyBendPick-v0-psi0 ./data/
```
### Using nix
Evaluate a PSI checkpoint directly from this repo with the SIMPLE library API:
```
source .venv-psi/bin/activate
python examples/simple/simple_eval.py \
--run-dir /path/to/.runs/finetune/... \
--ckpt-step 40000 \
--data-dir /hfm/data/simple/G1WholebodyBendPick-v0-psi0 \
--env-id simple/G1WholebodyBendPick-v0 \
--num-episodes 1 \
--save-video
```
> After evaluation is done, checkout the videos from `third_party/SIMPLE/data/evals/psi0`
> REMINDER: Start server before starting the eval client
### Using Docker
launch the eval client
```
export task=simple/G1WholebodyBendPick-v0
export policy=psi0
export host=localhost
export port=22085
GPUs=1 docker compose run eval $task $policy \
--host=$host \
--port=$port \
--sim-mode=mujoco_isaac \
--headless \
--max-episode-steps=360 \
--data-format=lerobot \
--data-dir=data/G1WholebodyBendPick-v0-psi0 \
--num-episodes=10
```