Blame for data/NOTICE_data.txt

data/heldout_windows_uint16.npy: notice for a changed excerpt of a third-party dataset

Source: TinyStories, by Ronen Eldan and Yuanzhi Li (2023), published on the Hugging Face hub as the dataset roneneldan/TinyStories,
file TinyStoriesV2-GPT4-valid.txt (sha256 6874bae9a4c1a4e7edcf0e53b86c17817e9cf881fc75ff2368da457b80c0585d). Its dataset card
describes the stories as synthetically generated short stories that use a small vocabulary, this version from GPT-4 generations.

Licence: this file is published under the Community Data License Agreement, Sharing, Version 1.0 (CDLA-Sharing-1.0), unmodified,
with no further restriction added. The text of the agreement: https://cdla.dev/sharing-1-0/
Credit and attribution for the data belong to its Data Providers named above.

This file has been changed from the source (notice under Section 3.1(b) of the agreement):
- the validation stories were normalized; a few that failed the cleaning (empty, cut, or not plain ASCII after normalizing) were
  dropped, and 10 more were dropped as near-duplicates of training text;
- the rest were tokenized with a 4,096-token vocabulary and joined into one token stream (the tokenizer is not included);
- 256 windows of 66 consecutive token ids were taken from that stream at fixed start positions chosen before the run;
- the result is stored as a 256 by 66 array of unsigned 16-bit integers in numpy .npy format (load with allow_pickle=False).

These are the held-out positions on which every network in this release was measured. No network trained on them.