HuggingFaceFW/fineweb-edu
Viewer • Updated • 3.5B • 414k • 1.41k
AresXXS1-1M
AresXXS1-1M is a tiny model experiment with a normal llama architecture and a small vocab.
Despite being just 1 million parameters, it still achieves results comparable to 10 million parameter models and larger.
Benchmarks
| Benchmark | AresXXS1-1M | Nanowhale-100M | Ethoric-10M | TBB-3M-v2 |
|---|---|---|---|---|
| PIQA | 59.14 | 51.80 | 59.85 | 56.75 |
| HellaSwag | 25.80 | 26.31 | 28.75 | 27.29 |
| ARC easy | 35.98 | 28.79 | 37.58 | 33.38 |
| ARC challenge | 40.77 | 20.65 | 24.66 | 22.53 |
| Arithmark 3.0 | 42.20 | 26.60 | 41.60 | 45.10 |
| Int Index | 14.27 | 2.51 | 12.94 | 10.39 |