yamalies/experiment-v2-data / data /sources_hf.json
yamalies's picture
download
raw
907 Bytes
{
"version": 2,
"name": "training_v2-teutonic-II-110B-finewebedu",
"description": "HF corpus mirror for Teutonic-II-110B FineWeb-Edu (MiMo-V2.5-Pro). Prefer live R2 catalog for full shards; this points at stevendev/corpus after teutonic_dataset_hf_sync.",
"seq_len": 2048,
"dtype": "uint32",
"train_seed": 86,
"eval_seed": 1234,
"eval_total": 2000,
"default_train_sequences": 5000000,
"tokenizer": "XiaomiMiMo/MiMo-V2.5-Pro",
"datasets_manifest_url": "https://pub-fedac496355c4edc9aed57189e6e190f.r2.dev/datasets/manifest.json",
"sources": [
{
"name": "finewebedu",
"weight": 1.0,
"eval_weight": 1.0,
"num_shards": 200,
"eval_num_shards": 80,
"bucket": "stevendev/corpus",
"manifest": "dataset/finewebedu/manifest.json",
"r2_manifest_url": "https://pub-d923bc4e8fcb45f6b703bc750bcf8aa6.r2.dev/finewebedu/manifest.json"
}
]
}

Xet Storage Details

Size:
907 Bytes
·
Xet hash:
b46ca0b3701a733907082a7133165d2d17829a9cd60a6d3ad8710fc9a4ef96d8

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.