Tinghong-Ye22's picture
Add files using upload-large-folder tool
55e0599 verified
|
Raw History Blame Contribute Delete
1.61 kB

Evaluate Policies on SIMPLE

pull SIMPLE

git submodule update --init --recursive
cd third_party/SIMPLE

Install Dokcer version

Setup environment variables

cp .env.sample .env; 
# then edit the .env file to set the "DATA_DIR" to current directory (e.g., `DATA_DIR=/home/***/projects/psi/third_party/SIMPLE/data`)

Build docker image

docker compose build isaac-sim

Install UV version (TODO)

git submodule update --init --recursive third_party/SIMPLE

Evaluation

copy the training dataset to SIMPLE data, which will be mounted into the docker container

cp -r $DATA_HOME/G1WholebodyBendPick-v0-psi0 ./data/

Using nix

Evaluate a PSI checkpoint directly from this repo with the SIMPLE library API:

source .venv-psi/bin/activate
python examples/simple/simple_eval.py \
    --run-dir /path/to/.runs/finetune/... \
    --ckpt-step 40000 \
    --data-dir /hfm/data/simple/G1WholebodyBendPick-v0-psi0 \
    --env-id simple/G1WholebodyBendPick-v0 \
    --num-episodes 1 \
    --save-video

After evaluation is done, checkout the videos from third_party/SIMPLE/data/evals/psi0 REMINDER: Start server before starting the eval client

Using Docker

launch the eval client

export task=simple/G1WholebodyBendPick-v0
export policy=psi0
export host=localhost
export port=22085
GPUs=1 docker compose run eval $task $policy \
    --host=$host \
    --port=$port  \
    --sim-mode=mujoco_isaac \
    --headless \
    --max-episode-steps=360 \
    --data-format=lerobot \
    --data-dir=data/G1WholebodyBendPick-v0-psi0 \
    --num-episodes=10