File size: 735 Bytes
c8ba9c6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | # Dataset attribution
Default upstream dataset: **FineWeb-Edu**, Hugging Face / HuggingFaceFW.
Repository: https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu
Configuration: sample-100BT. Language: English. Dataset card lists ODC-BY.
Read the upstream dataset card and license for applicable attribution and
underlying web-content terms. This ZIP distributes no dataset texts.
The immutable source revision is recorded in work/data/manifest.json at preparation.
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale:
https://arxiv.org/abs/2406.17557
The WarpState code and tokenizer are copied from the user's existing project.
No new ownership or license over those user-provided assets is asserted here.
|