doc/generation.md

Scaling up to a dataset

scripts/generate.py runs many randomized episodes per task and keeps the ones that succeeded. Each episode gets a fresh seed, so the scene randomization differs every time; the scripted solver plays it and the task's own success conditions decide whether it counts. That filtering is what makes this a demonstration generator rather than a batch runner.

python scripts/generate.py --episodes 20                       # every registered task
python scripts/generate.py --tasks passing --episodes 100      # only tasks known to work
python scripts/generate.py --tasks grape_box,stack_blocks --episodes 50
python scripts/generate.py --episodes 20 --resume              # skip episodes already done

What it writes

data/
  <task>/ep<seed>.mp4     video — successes only, unless --keep-failures
  <task>/ep<seed>.json    result, per-condition PASS/FAIL, and the episode's actual placements
  manifest.json           running totals and success rate per task

The per-episode JSON records the randomized placements the environment actually used, not the nominal ones, so an episode can be replayed exactly with --seed.

Options worth knowing

flag	why
--tasks passing	reads outputs/tasks/v3_*.log and selects only tasks whose last development run succeeded — spending 100 seeds on a task that cannot complete is wasted GPU time
--start-seed	defaults to 100, deliberately clear of the 0–9 seeds used during development, so generated data never overlaps the episodes used to tune the task
--resume	skips episodes whose JSON already exists; the script is safe to interrupt
--keep-failures	keeps failed videos for debugging instead of deleting them

Cost

One episode is one simulator launch — expect roughly 40–90 s each, so this is long-running by nature. It prints one line per episode and writes manifest.json after every episode, so progress survives an interruption.

Success rates vary by task and are worth checking before committing to a large run: a task that passes at seed 0 may be sensitive to the jitter. The manifest gives you exactly that number.

A caution

The filter is only as honest as the success conditions. Three checks in this suite were caught reporting success for episodes that did nothing (see ../DIAGNOSTICS.md) — a pour whose grasp had failed, and two joints that "moved" under gravity alone. Before generating at scale on a task you have just written, render a few contact sheets and confirm the numeric check agrees with what you see; otherwise you will mass-produce convincing-looking failures.