Vanessa-1B / pretok_progress.json
darkstudio009's picture
Upload pretok_progress.json with huggingface_hub
1746484 verified
Raw History Blame Contribute Delete
171 Bytes
{"next_shard_idx": 242, "next_file_idx": 33, "docs_done_in_file": 614000, "docs_done": 24172101, "tokens_so_far": 25012001024, "tokenizer_fingerprint": "4e63cc80e8c9e08b"}