| # Data for Lecture 6 |
|
|
| `phrases.txt` and `variable_text.txt` were generated deterministically with Python |
| `random.Random(6270)`. Each contains 640 examples. Their vocabulary describes |
| colored shapes and directions; no external dataset is needed. |
|
|
| | Color | Shape | Direction | |
| | --- | --- | --- | |
| | red | circle | left | |
| | blue | square | right | |
| | green | triangle | up | |
| | gold | star | down | |
|
|
| The fixed-length corpus has four tokens per line. The variable-length corpus |
| uses `COLOR SHAPE`, `COLOR SHAPE moves DIRECTION`, or the latter followed by |
| `slowly` or `twice`. The color, shape, and direction remain correlated. This |
| makes invalid independent combinations easy to inspect. |
|
|
| Training uses a seeded shuffle and 80/20 example split. The grammar has a small |
| support, so examples repeat and both partitions contain the same grammar. |
| Training-support fraction measures membership in this small support; it is |
| not a test of open-ended language generalization. |
|
|
| For a custom corpus, supply a UTF-8 text file with one whitespace-tokenized |
| sequence per line. Fixed-length methods require equal token counts. The |
| expanding method accepts variable lengths up to 32. Vocabulary and maximum |
| length are saved in the checkpoint, so sampling does not need the original file. |
|
|
| The continuous dataset is sampled from four equally weighted Gaussians centered |
| at `(−1.5,−1.5)`, `(−1.5,1.5)`, `(1.5,−1.5)`, and `(1.5,1.5)`, with coordinate |
| standard deviation 0.22. The latent example embeds these points on |
| `(x, y, 0.3(x²−y²))` before learning its autoencoder. SSFM uses analytic |
| Ornstein-Uhlenbeck marginals with initial variance one, drift `−x`, and diffusion |
| coefficient 0.7. |
|
|
