# Scaling up to a dataset

`scripts/generate.py` runs many randomized episodes per task and keeps the ones that succeeded.
Each episode gets a fresh seed, so the scene randomization differs every time; the scripted
solver plays it and the task's own success conditions decide whether it counts. That filtering
is what makes this a demonstration generator rather than a batch runner.

```bash
python scripts/generate.py --episodes 20                       # every registered task
python scripts/generate.py --tasks passing --episodes 100      # only tasks known to work
python scripts/generate.py --tasks grape_box,stack_blocks --episodes 50
python scripts/generate.py --episodes 20 --resume              # skip episodes already done
```

## What it writes

```
data/
  <task>/ep<seed>.mp4     video — successes only, unless --keep-failures
  <task>/ep<seed>.json    result, per-condition PASS/FAIL, and the episode's actual placements
  manifest.json           running totals and success rate per task
```

The per-episode JSON records the **randomized placements the environment actually used**, not the
nominal ones, so an episode can be replayed exactly with `--seed`.

## Options worth knowing

| flag | why |
|---|---|
| `--tasks passing` | reads `outputs/tasks/v3_*.log` and selects only tasks whose last development run succeeded — spending 100 seeds on a task that cannot complete is wasted GPU time |
| `--start-seed` | defaults to 100, deliberately clear of the 0–9 seeds used during development, so generated data never overlaps the episodes used to tune the task |
| `--resume` | skips episodes whose JSON already exists; the script is safe to interrupt |
| `--keep-failures` | keeps failed videos for debugging instead of deleting them |

## Cost

One episode is one simulator launch — expect roughly 40–90 s each, so this is long-running by
nature. It prints one line per episode and writes `manifest.json` after every episode, so
progress survives an interruption.

Success rates vary by task and are worth checking before committing to a large run: a task that
passes at seed 0 may be sensitive to the jitter. The manifest gives you exactly that number.

## A caution

The filter is only as honest as the success conditions. Three checks in this suite were caught
reporting success for episodes that did nothing (see [`../DIAGNOSTICS.md`](../DIAGNOSTICS.md)) —
a pour whose grasp had failed, and two joints that "moved" under gravity alone. Before generating
at scale on a task you have just written, render a few contact sheets and confirm the numeric
check agrees with what you see; otherwise you will mass-produce convincing-looking failures.