Boris-2 / benchmarks_comparison.csv
KlondikeDev's picture
Add benchmark chart, 9-task comparison CSV and LICENSE
6062798 verified
Raw History Blame Contribute Delete
892 Bytes
model,params,hellaswag,arc_easy,arc_challenge,piqa,mmlu_continuation,winogrande,openbookqa,sciq,commonsense_qa,average_9,source
Boris-2,144M,50.27,59.76,32.0,72.85,28.96,52.57,36.6,85.1,20.88,48.78,lm-eval (all 9)
Boris-125M,125M,29.25,40.7,23.81,59.58,25.08,49.25,28.2,58.8,19.49,37.13,lm-eval (all 9)
GPT-X3-150M,149M,46.16,59.64,31.31,71.33,28.87,54.62,34.4,80.3,23.1,47.75,leaderboard (HS/ARC/PIQA) + lm-eval (other 5)
SmolLM2-135M,135M,43.22,58.63,29.69,68.44,29.9,52.96,33.4,78.2,19.49,45.99,leaderboard (HS/ARC/PIQA) + lm-eval (other 5)
KLM-1.6,99M,45.28,61.07,33.53,70.18,29.26,52.33,32.8,89.3,18.92,48.07,leaderboard (HS/ARC/PIQA) + lm-eval (other 5)
BananaMind-2.5-Pro,139M,43.6,62.79,31.14,68.12,29.24,52.57,34.0,85.1,20.39,47.44,lm-eval (all 9)
LFM-2.5-350M-Base,354M,45.16,68.86,41.13,67.63,33.32,55.25,36.2,84.9,41.28,52.64,leaderboard (HS/ARC/PIQA) + lm-eval (other 5)