First hour on a new machine
Do these in order. Everything else — how the pipeline works, why the sources are picked the
way they are, runtime numbers — is in README.md; this file is only the setup path.
Assumes you have this repo and nothing else.
0. What you must bring yourself
| size | why it is not here | |
|---|---|---|
WAN weights — Wan2.1-VACE-1.3B, VACE-Annotators |
31 GB | too large to ship; get them from the VACE release or copy from the original machine |
| GPU with ≥ 27 GB free | --offload_model False keeps the DiT resident |
Everything else is in this repo.
1. Unpack the data (~3 GB, 2 min)
cd <repo>/ # the directory holding aug32/, VACE/, datasets_pack/
mkdir -p data && cd data
tar -xf ../datasets_pack/source_episodes.tar # 2.7 GB -> pickplace_target_human/
tar -xzf ../datasets_pack/aug_materials_ext.tar.gz # 121 MB -> aug_materials_ext/
tar -xzf ../datasets_pack/target_meshes.tar.gz # 14 MB -> objaverse/ lightwheel/
cd ..
Check:
data/pickplace_target_human/<5 tasks>/{videos,extras}
data/aug_materials_ext/<4 tasks>/ep<N>/mask_gt_target_object_<camera>.mp4
data/{objaverse,lightwheel}/<category>/<model>/model.xml
2. Link the WAN weights
ln -s /path/to/VACE-home-models VACE/models
ls VACE/models # must show Wan2.1-VACE-1.3B and VACE-Annotators
3. Build two Python environments
A — batch driver. Needs robocasa, robosuite, mujoco, trimesh, pyrender,
opencv-python, numpy. This one also renders the target meshes' reference views, so
trimesh + pyrender are not optional. Versions that work: mujoco 3.3.1, robosuite 1.5.2,
robocasa 1.0.0, trimesh 4.12.1, pyrender 0.1.45, numpy 1.26.4, Python 3.10.
B — WAN inference. Python 3.12, torch 2.5.1+cu124, flash_attn 2.8.3, plus
VACE/requirements.txt. wan must be importable (it is a site-package in the original
environment, installed from the VACE repo).
export VACE_PY_ROBOCASA=/path/to/env-A/bin/python
export VACE_PY_WAN=/path/to/env-B/bin/python
4. Export the data paths
cd <repo>
export VACE_ROBOCASA_ROOT=$PWD/data/pickplace_target_human
export VACE_AUG_MATERIALS=$PWD/data/aug_materials_ext
export VACE_ASSET_ROOTS=$PWD/data/lightwheel:$PWD/data/objaverse
Leave VACE_RENDER_SCRIPT_DIR and VACE_RENDER_PYTHON alone — they default to
aug32/vendor/ and to VACE_PY_ROBOCASA, both already correct.
Put all five exports in a file you can source; every later step needs them.
5. Preflight
./aug32/preflight.sh
Every line must read OK. It checks both interpreters, the vendored renderer and its
imports, the VACE fork and weights, the three data roots, per-task GT-mask coverage, and
free GPU memory. A MISS names the exact path that is wrong.
6. Smoke test before the real batch
One assignment end to end, ~10 min on one GPU, to catch a broken path before a multi-hour run:
mkdir -p /tmp/vace_smoke
$VACE_PY_ROBOCASA aug32/build_aug32_assignments.py \
--out-root /tmp/vace_smoke --per-mesh 1 --targets teapot/teapot_6
head -1 /tmp/vace_smoke/assignments.jsonl > /tmp/vace_smoke/one.jsonl
mv /tmp/vace_smoke/one.jsonl /tmp/vace_smoke/assignments.jsonl
./aug32/run_aug32.sh /tmp/vace_smoke /tmp/vace_smoke_out 0
Expect /tmp/vace_smoke_out/<task>-ep<N>_<src>_teapot_6/ with three camera folders, each
holding one mp4.
7. The real run
./aug32/run_aug32.sh /scratch/vace_aug32 /scratch/output_aug_256 0
8 targets × 32 episodes × 3 cameras ≈ 3,084 WAN jobs, ~44 h on one GPU at the measured
51.4 s/job. Split it: run_aug32.sh ... <gpu> 4 writes four worker manifests, and the four
WAN processes can be moved onto four GPUs (~11 h). Every stage is idempotent, so an
interrupted run resumes where it stopped.
Common failures
| symptom | cause |
|---|---|
FileNotFoundError: Missing GT target-object mask video |
VACE_AUG_MATERIALS not exported, or pointing above aug_materials_ext/ |
ModuleNotFoundError: No module named 'trimesh' during prep-worker |
env A lacks trimesh/pyrender — install them, or point VACE_RENDER_PYTHON at an env that has them |
unrecognized arguments: --batch_manifest |
VACE/ was replaced with a fresh upstream clone; apply aug32/vace_wan_inference_batch.patch |
| WAN job ~82 s instead of ~51 s | --offload_model False got dropped; it is in run_aug32.sh, do not remove it |
| exo cameras come out with the object erased rather than swapped | expected at 256×256 — the target mask is robot0_eye_in_hand ( |
| assignments have fewer same-category sources than expected | the dataset only holds 5 teapot / 3 donut / 2 jar / 2 syrup_bottle episodes. Only MeasuringCup009 can fill 32 from its own category |