File size: 2,674 Bytes
000b008 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | # Scaling up to a dataset
`scripts/generate.py` runs many randomized episodes per task and keeps the ones that succeeded.
Each episode gets a fresh seed, so the scene randomization differs every time; the scripted
solver plays it and the task's own success conditions decide whether it counts. That filtering
is what makes this a demonstration generator rather than a batch runner.
```bash
python scripts/generate.py --episodes 20 # every registered task
python scripts/generate.py --tasks passing --episodes 100 # only tasks known to work
python scripts/generate.py --tasks grape_box,stack_blocks --episodes 50
python scripts/generate.py --episodes 20 --resume # skip episodes already done
```
## What it writes
```
data/
<task>/ep<seed>.mp4 video — successes only, unless --keep-failures
<task>/ep<seed>.json result, per-condition PASS/FAIL, and the episode's actual placements
manifest.json running totals and success rate per task
```
The per-episode JSON records the **randomized placements the environment actually used**, not the
nominal ones, so an episode can be replayed exactly with `--seed`.
## Options worth knowing
| flag | why |
|---|---|
| `--tasks passing` | reads `outputs/tasks/v3_*.log` and selects only tasks whose last development run succeeded — spending 100 seeds on a task that cannot complete is wasted GPU time |
| `--start-seed` | defaults to 100, deliberately clear of the 0–9 seeds used during development, so generated data never overlaps the episodes used to tune the task |
| `--resume` | skips episodes whose JSON already exists; the script is safe to interrupt |
| `--keep-failures` | keeps failed videos for debugging instead of deleting them |
## Cost
One episode is one simulator launch — expect roughly 40–90 s each, so this is long-running by
nature. It prints one line per episode and writes `manifest.json` after every episode, so
progress survives an interruption.
Success rates vary by task and are worth checking before committing to a large run: a task that
passes at seed 0 may be sensitive to the jitter. The manifest gives you exactly that number.
## A caution
The filter is only as honest as the success conditions. Three checks in this suite were caught
reporting success for episodes that did nothing (see [`../DIAGNOSTICS.md`](../DIAGNOSTICS.md)) —
a pour whose grasp had failed, and two joints that "moved" under gravity alone. Before generating
at scale on a task you have just written, render a few contact sheets and confirm the numeric
check agrees with what you see; otherwise you will mass-produce convincing-looking failures.
|