AresXXS1-1M

AresXXS1-1M is a tiny model experiment with a normal llama architecture and a small vocab.

Despite being just 1 million parameters, it still achieves results comparable to 10 million parameter models and larger.

Benchmarks

Benchmark AresXXS1-1M Nanowhale-100M Ethoric-10M TBB-3M-v2
PIQA 59.14 51.80 59.85 56.75
HellaSwag 25.80 26.31 28.75 27.29
ARC easy 35.98 28.79 37.58 33.38
ARC challenge 40.77 20.65 24.66 22.53
Arithmark 3.0 42.20 26.60 41.60 45.10
Int Index 14.27 2.51 12.94 10.39
Downloads last month
52
Safetensors
Model size
1.02M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train ArrowLabs/AresXXS1-1M