| library_name: transformers | |
| datasets: | |
| - HuggingFaceFW/fineweb | |
| # Model Card for Model ID | |
| This is a BPE tokenizer with 10,048 tokens trained on a portion of FineWeb's 10B token sample. |
| library_name: transformers | |
| datasets: | |
| - HuggingFaceFW/fineweb | |
| # Model Card for Model ID | |
| This is a BPE tokenizer with 10,048 tokens trained on a portion of FineWeb's 10B token sample. |