examples/SimplerEnv/README.md

This document provides instructions for reproducing our experimental results with SimplerEnv.

The evaluation process consists of two main parts:

1. Setting up the simpler_env environment and dependencies.
2. Running the evaluation by launching services in both starVLA and simpler_env environments.

We have verified that this workflow runs successfully on both NVIDIA A100 and RTX 4090 GPUs.

📦 1. Environment Setup

To set up the environment, please first follow the official SimplerEnv repository to install the base simpler_env environment.

Afterwards, inside the simpler_env environment, install the following dependencies:

conda activate simpler_env
pip install tyro matplotlib mediapy websockets msgpack
pip install numpy==1.24.4

⚠️ Common Issues When testing SimplerEnv on NVIDIA A100, you may encounter the following error: libvulkan.so.1: cannot open shared object file: No such file or directory You can refer to this link to fix: Installation Guide – Vulkan Section

🔧 Verification Your SimplerEnv

We provide a minimal environment verification script:

python examples/SimplerEnv/test_your_simplerEnv.py

If you see the "✅ Env built successfully" message, it means SimplerEnv is installed correctly and ready to use.

---

🚀 2. Eval SimplerEnv

The evaluation should be run from the repository root using two separate terminals/machines, one for each environment:

• starVLA environment: runs the policy inference server.
• simpler_env environment: runs the simulation eval code.

Step 0. Download offical checkpoint

Available SimplerEnv WidowX checkpoints (see docs/model_zoo.md for the full list):

• Qwen3VL-GR00T-Bridge-RT-1 — 65.3 avg success
• Qwen3VL-PI_v3-Bridge-RT-1 — 69.8 avg success (layer-wise cross-DiT flow-matching head, π₀.₅-style)

Step 1. Start the server (starVLA environment)

In the first terminal, activate the starVLA conda environment and run:

bash examples/SimplerEnv/eval_files/run_policy_server.sh

⚠️ Note: Please ensure that you specify the correct checkpoint path in
run_policy_server.sh

---

Step 2. Start the simulation (simpler_env environment)

In the second terminal, activate the simpler_env conda environment and run:

export MODEL_PATH=.../checkpoints/steps_50000_pytorch_model.pt # for read normonization json and get name to save video under ckpt dir
bash examples/SimplerEnv/start_simpler_env.sh ${MODEL_PATH} 

This script will automatically launch the WidowX Robot evaluation tasks, reproducing the benchmark results reported above.

⚠️ Note: Please ensure that you specify the correct SimplerEnv_PATHin start_simpler_env.sh

⚠️ Common Issues

When run policy server but NotImplementedError:Framework QwenGR00T is not implemented, you may need to python QwenGR00T.py to check your env.

🚀 Training on OXE

Data Preparation

Steps:

1. Download a LeRobot-format OXE dataset

• bridge_orig_lerobot
• fractal20220817_data_lerobot

2. Including modality.json in each *lerobot/meta/modality.json

• bridge modality. Rename as modality.json and put it as bridge_orig_lerobot/meta/modality.json
• fractal modality. Rename as modality.json and put it as fractal20220817_data_lerobot/meta/modality.json

3. Add your dataset path to config.yaml:

   datasets:
     vla_data:
       dataset_py: lerobot_datasets
       data_root_dir: playground/Datasets/OXE_LEROBOT_DATASET  # path to your dataset
       data_mix: bridge_rt_1
   

Check Your Dataoader

We provide a simple way to check your dataloader. Make sure you can load batched data:

python starVLA/dataloader/lerobot_datasets.py --config_yaml examples/SimplerEnv/train_files/starvla_cotrain_oxe.yaml

Framework Preparation

Before running, you need to ensure that your framework can forward and predict_action using a fake data example.

Try the following command:

python starVLA/model/framework/VLM4A/QwenGR00T.py --config_yaml examples/SimplerEnv/train_files/starvla_cotrain_oxe.yaml

Note: You can modify the following code snippet to align with your dataset:

    # Generate a fake sample
    image = Image.fromarray(np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8))
    # Create a sample
    sample = {
        "action": np.random.uniform(-1, 1, size=(16, 7)).astype(np.float16),  # action_chunk, action_dim
        "image": [image, image],  # two views
        "lang": "This is a fake for testing.",
        "state": np.random.uniform(-1, 1, size=(1, 7)).astype(np.float16),  # chunk, state_dim
    }

Training

Once everything is ready, use our provided script to start training:

bash ./examples/SimplerEnv/train_files/run_oxe_train.sh

⚠️ Note: Ensure that the script explicitly uses the validated config path in run_lerobot_datasets.sh. If not already passed, add the --config_yaml argument.