| data/heldout_windows_uint16.npy: notice for a changed excerpt of a third-party dataset |
|
|
| Source: TinyStories, by Ronen Eldan and Yuanzhi Li (2023), published on the Hugging Face hub as the dataset roneneldan/TinyStories, |
| file TinyStoriesV2-GPT4-valid.txt (sha256 6874bae9a4c1a4e7edcf0e53b86c17817e9cf881fc75ff2368da457b80c0585d). Its dataset card |
| describes the stories as synthetically generated short stories that use a small vocabulary, this version from GPT-4 generations. |
|
|
| Licence: this file is published under the Community Data License Agreement, Sharing, Version 1.0 (CDLA-Sharing-1.0), unmodified, |
| with no further restriction added. The text of the agreement: https://cdla.dev/sharing-1-0/ |
| Credit and attribution for the data belong to its Data Providers named above. |
|
|
| This file has been changed from the source (notice under Section 3.1(b) of the agreement): |
| - the validation stories were normalized; a few that failed the cleaning (empty, cut, or not plain ASCII after normalizing) were |
| dropped, and 10 more were dropped as near-duplicates of training text; |
| - the rest were tokenized with a 4,096-token vocabulary and joined into one token stream (the tokenizer is not included); |
| - 256 windows of 66 consecutive token ids were taken from that stream at fixed start positions chosen before the run; |
| - the result is stored as a 256 by 66 array of unsigned 16-bit integers in numpy .npy format (load with allow_pickle=False). |
|
|
| These are the held-out positions on which every network in this release was measured. No network trained on them. |
|
|
