Hugging Face's logo

fromziro
/
DataBench

Safetensors
English
ablations
datasets
1m
llama
custom_tokenizer
Model card Files
xet
 Community
Harley-ml commited on
Commit
167e244
·
verified ·
1 Parent(s): 0d7cc1b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -65,6 +65,6 @@ Rather than training one model, we trained multiple models, each on a different
65
 
66
  # Results
67
 
68
- | Dataset | Tokens | Val Loss ↓ | ARC-Easy ↑ | ARC-Challenge ↑ | HellaSwag ↑ | PIQA ↑ | SWAG ↑ | BLiMP ↑ |
69
  |:-----------------|-------:|-----------:|-----------:|----------------:|------------:|-------:|-------:|--------:|
70
- | [Project Gutenberg](https://huggingface.co/common-pile/project_gutenberg_filtered) | 499,863,368 | 2.9309 | 25.95% | 24.91% | 24.79% | 50.22% | 27.00% | 63.76% |
 
65
 
66
  # Results
67
 
68
+ | Dataset | Tokens | Final Val Loss ↓ | ARC-Easy ↑ | ARC-Challenge ↑ | HellaSwag ↑ | PIQA ↑ | SWAG ↑ | BLiMP ↑ | Avg ↑ |
69
  |:-----------------|-------:|-----------:|-----------:|----------------:|------------:|-------:|-------:|--------:|
70
+ | [FineWeb]() | 500,000,000 | 3.344866 | 27.82% | 22.01% | 26.89% | 52.83% | 30.34% | 65.21% | 37.52% |