File size: 3,815 Bytes
000b008 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 | # Setup and running
## Environment
Isaac Sim 5.0 + Isaac Lab 2.2.0, Python 3.11, an NVIDIA GPU (the PhysX GPU pipeline is required).
Isaac Lab may already exist on the machine inside another project's virtualenv. If so, prefer an
**overlay venv** — a fresh venv whose `.pth` file points at that interpreter's `site-packages` —
over reinstalling Isaac Sim:
```bash
python -m venv .venv
echo "/path/to/existing/isaac/venv/lib/python3.11/site-packages" \
> .venv/lib/python3.11/site-packages/_isaac_overlay.pth
```
Isaac Lab's *source checkout* is also needed for `isaaclab_tasks` / `isaaclab_assets`; add it the
same way if it is not pip-installed.
### Configure
Everything is driven by two environment variables and a relative path. Nothing below hardcodes a
machine-specific location:
```bash
export ROBOTWIN_USD=./robotwin_usd # converted USD assets (see Assets)
export ROBOTWIN_SRC=./robotwin_object/objects # source meshes, only needed to convert
export OMNI_KIT_ACCEPT_EULA=YES
```
`ROBOTWIN_USD` defaults are read at import time, so export it before running anything. Unzip
`assets/robotwin_usd.zip` from this repo's HF page to populate it.
### Run
```bash
python scripts/yam_task.py --list
python scripts/yam_task.py --task grape_box --seed 3 \
--video outputs/tasks/grape_box.mp4 \
--kit_args="--/rtx/verifyDriverVersion/enabled=false"
```
Two flags are commonly required and easy to miss:
* `OMNI_KIT_ACCEPT_EULA=YES` — otherwise Omniverse blocks on an interactive EULA prompt and the
run hangs forever with no output at all.
* `--kit_args="--/rtx/verifyDriverVersion/enabled=false"` — needed on drivers whose version
string Omniverse misparses (e.g. `535.309.01` read as `535.53` and rejected as too old). The
check is wrong, not the driver. Drop this flag if your driver passes the check.
Extras: `--no-randomize` (nominal poses, for reproducing a failure), `--set gripper_effort=70`
(override any task parameter), `--first-frame out.png` (build the scene, save one frame, exit
without solving).
**Config precedence:** `class attribute → source/bimanual/yam/tasks/configs/<task>.yaml → --set`.
Each task ships a YAML mirroring its parameters *with every key commented out on purpose* — an
active key there overrides the Python and keeps overriding it, so a later code change silently
does nothing.
Full architecture, task-authoring guide, the asset-gotcha table and the verification workflow:
[`source/bimanual/yam/README.md`](source/bimanual/yam/README.md).
Every non-obvious failure and the measurement that settled it:
[`DIAGNOSTICS.md`](DIAGNOSTICS.md) — read this before debugging a new asset.
### The one habit that matters
When a grasp or a joint misbehaves, **measure the asset offline before touching a parameter.**
Most of the hard bugs here looked like tuning problems and were not:
```bash
# where is the graspable part, and is the mesh origin even on the object?
python -c "
import trimesh, numpy as np
m = trimesh.load('objects/083_brush/visual/base0.glb', force='mesh'); v = m.vertices
tall = int(np.argmax(m.extents)); oth = [a for a in range(3) if a != tall]
lo, hi = v[:,tall].min(), v[:,tall].max()
for i in range(8): # cross-section along the long axis
s = v[(v[:,tall] >= lo+(hi-lo)*i/8) & (v[:,tall] < lo+(hi-lo)*(i+1)/8)]
print(f'{i/8:.0%}', s[:,oth[0]].ptp(), s[:,oth[1]].ptp(), 'centroid', s[:,oth].mean(0))
"
```
That one command found the brush's 1 cm handle **and** the 3.7 cm offset between the mesh and the
body origin that `object_pos()` reports — two separate bugs that both presented as `gap 0.0 cm`.
The same check on the kettle found a 4.7 cm offset. For articulated assets, read the URDF's joint
`axis` and `bounding_box.json` rather than inferring anything from the USD.
|