| --- |
| language: |
| - en |
| license: apache-2.0 |
| datasets: |
| - HuggingFaceFW/fineweb-edu |
| - CEAMFA/rewrite |
| - openbmb/Ultra-FineWeb-L3 |
| --- |
| |
| Experimental deep smol lm trained on 23BT using a tinyllama tokenizer |
|
|
| # Benchmark Results |
|
|
| ## Final Summary |
|
|
| | Benchmark | Primary Metric | Score | Accuracy | Normalized Accuracy | Notes | |
| |---|---|---:|---:|---:|---| |
| | **BananaMind Base Bench 1.1** | Overall Elo | **875.00** | 33.43% | — | Weighted Accuracy: **32.52%** | |
| | **ArithMark 3.0** | acc_norm | **33.10%** | 32.90% | 33.10% | Official AxiomicLabs runner | |
| | **ARC Easy** | acc_norm | **37.21%** | 40.07% | 37.21% | lm-eval 0.4.12 · zero-shot | |
| | **ARC Challenge** | acc_norm | **22.87%** | 17.92% | 22.87% | lm-eval 0.4.12 · zero-shot | |
| | **HellaSwag** | acc_norm | **28.38%** | 27.60% | 28.38% | lm-eval 0.4.12 · zero-shot | |
| | **PIQA** | acc_norm | **60.23%** | 59.79% | 60.23% | lm-eval 0.4.12 · zero-shot | |
| |
| ## Intelligence Index |
| |
| > **INT Index: 10.603** |
| |
| ### Highlights |
| |
| - **Overall Elo:** 875 |
| - **BananaMind Weighted Accuracy:** 32.52% |
| - **Best benchmark:** PIQA — 60.23% normalized accuracy |
| - **ARC Easy:** 37.21% |
| - **ArithMark 3.0:** 33.10% |
| - **HellaSwag:** 28.38% |
| - **ARC Challenge:** 22.87% |
