Setup and running

Environment

Isaac Sim 5.0 + Isaac Lab 2.2.0, Python 3.11, an NVIDIA GPU (the PhysX GPU pipeline is required).

Isaac Lab may already exist on the machine inside another project's virtualenv. If so, prefer an overlay venv — a fresh venv whose .pth file points at that interpreter's site-packages — over reinstalling Isaac Sim:

python -m venv .venv
echo "/path/to/existing/isaac/venv/lib/python3.11/site-packages" \
    > .venv/lib/python3.11/site-packages/_isaac_overlay.pth

Isaac Lab's source checkout is also needed for isaaclab_tasks / isaaclab_assets; add it the same way if it is not pip-installed.

Configure

Everything is driven by two environment variables and a relative path. Nothing below hardcodes a machine-specific location:

export ROBOTWIN_USD=./robotwin_usd          # converted USD assets (see Assets)
export ROBOTWIN_SRC=./robotwin_object/objects   # source meshes, only needed to convert
export OMNI_KIT_ACCEPT_EULA=YES

ROBOTWIN_USD defaults are read at import time, so export it before running anything. Unzip assets/robotwin_usd.zip from this repo's HF page to populate it.

Run

python scripts/yam_task.py --list
python scripts/yam_task.py --task grape_box --seed 3 \
    --video outputs/tasks/grape_box.mp4 \
    --kit_args="--/rtx/verifyDriverVersion/enabled=false"

Two flags are commonly required and easy to miss:

OMNI_KIT_ACCEPT_EULA=YES — otherwise Omniverse blocks on an interactive EULA prompt and the run hangs forever with no output at all.
--kit_args="--/rtx/verifyDriverVersion/enabled=false" — needed on drivers whose version string Omniverse misparses (e.g. 535.309.01 read as 535.53 and rejected as too old). The check is wrong, not the driver. Drop this flag if your driver passes the check.

Extras: --no-randomize (nominal poses, for reproducing a failure), --set gripper_effort=70 (override any task parameter), --first-frame out.png (build the scene, save one frame, exit without solving).

Config precedence: class attribute → source/bimanual/yam/tasks/configs/<task>.yaml → --set. Each task ships a YAML mirroring its parameters with every key commented out on purpose — an active key there overrides the Python and keeps overriding it, so a later code change silently does nothing.

Full architecture, task-authoring guide, the asset-gotcha table and the verification workflow: source/bimanual/yam/README.md. Every non-obvious failure and the measurement that settled it: DIAGNOSTICS.md — read this before debugging a new asset.

The one habit that matters

When a grasp or a joint misbehaves, measure the asset offline before touching a parameter. Most of the hard bugs here looked like tuning problems and were not:

# where is the graspable part, and is the mesh origin even on the object?
python -c "
import trimesh, numpy as np
m = trimesh.load('objects/083_brush/visual/base0.glb', force='mesh'); v = m.vertices
tall = int(np.argmax(m.extents)); oth = [a for a in range(3) if a != tall]
lo, hi = v[:,tall].min(), v[:,tall].max()
for i in range(8):                      # cross-section along the long axis
    s = v[(v[:,tall] >= lo+(hi-lo)*i/8) & (v[:,tall] < lo+(hi-lo)*(i+1)/8)]
    print(f'{i/8:.0%}', s[:,oth[0]].ptp(), s[:,oth[1]].ptp(), 'centroid', s[:,oth].mean(0))
"

That one command found the brush's 1 cm handle and the 3.7 cm offset between the mesh and the body origin that object_pos() reports — two separate bugs that both presented as gap 0.0 cm. The same check on the kettle found a 4.7 cm offset. For articulated assets, read the URDF's joint axis and bounding_box.json rather than inferring anything from the USD.