BananaMind 2 Pro: We've (almost) matched SmolLM2 at 20x fewer tokens... Trained On a 5070 Ti

Today I'm sharing the results of BananaMind 2 Pro, our custom ~139M small language model.

We wanted to see how far we could push this scale while keeping the project practical enough to develop and run on consumer hardware — including an RTX 5070 Ti 16GB.

And the result is pretty interesting:

We trained BananaMind 2 Pro only on 100B tokens.

For comparison:

• BananaMind 2 Pro: 100B tokens
• SmolLM-135M: 600B tokens
• SmolLM2-135M: 2T tokens

That means BananaMind 2 Pro used:

• 6× fewer training tokens than SmolLM-135M
• 20× fewer training tokens than SmolLM2-135M

Despite that difference, we're surprisingly close across a large part of our evaluation suite.

The results

Small correction, SmolLM was trained on 600B tokens, the image is stale.

Across all these benchmarks, BananaMind 2 Pro has retained 96.08% of SmolLM2-135M's benchmark score while using 20× fewer training tokens.

See detailed benchmark retention

Benchmark	Score Retained
ARC Easy	91.05%
ARC Challenge	91.88%
PIQA	98.66%
HellaSwag	98.98%
ArithMark 3	97.45%
ArithMark 2	98.16%
INT Index	92.00%
Code	94.75%
Base Bench	101.79%
Average	96.08%

But benchmarks aren't the whole story

Interestingly, the gap flips when we look at actual user preference votes.

On the community-run SLM Arena, we get shown outputs and vote which model is better.

At the current leaderboard snapshot (18/9/2026, 8:02 AM CEST), BananaMind 2 Pro is ranked #1 with 1163.8 Elo, while SmolLM2-135M is at 1082.1 Elo.

That's an 81 Elo lead for BananaMind 2 Pro.

So while BananaMind 2 Pro retains about 96.08% of SmolLM2's benchmark score, it currently outperforms SmolLM2 in direct user preference voting.

See the SLM Arena comparison

Model	Rank	Elo	Battles	Wins	Losses	Ties	Both Bad
BananaMind-2-Pro	#1	1163.8	51	32	11	1	7
SmolLM2-135M	#6	1082.1	43	22	11	4	6

Elo difference: +81.7 in favor of BananaMind 2 Pro.

Try the arena yourself

SLM Arena

Training data

We've trained BananaMind 2 Pro on a 100B-token mixture built primarily around educational web data, with additional synthetic, mathematical, and code-focused data.

Just over half of the training budget went to FineWeb-Edu, while DCLM contributed another 26.13%.

We used the remeaning ~23.7% to split it between Cosmopedia-v2, FineMath-4+, and NPSet-2 Python Edu, giving the model additional synthetic educational data, mathematics, and Python-focused training.

So the model was trained on a deliberately mixed dataset, with roughly three quarters of the total coming from FineWeb-Edu and DCLM and the remaining quarter providing more specialized data.

20× fewer tokens than SmolLM2

SmolLM2 saw 2 trillion tokens during training. BananaMind 2 Pro saw only 100 billion.

That's just 5% as many training tokens.

Yet the gap is soo small on benchmarks that I wouldnt even notice it.

PIQA

• BananaMind 2 Pro: 67.52%
• SmolLM2-135M: 68.44%
• Difference: 0.92 percentage points

HellaSwag

• BananaMind 2 Pro: 42.78%
• SmolLM2-135M: 43.22%
• Difference: 0.44 points

ArithMark 3

• BananaMind 2 Pro: 38.20%
• SmolLM2-135M: 39.20%
• Difference: 1.00 point

ArithMark 2

• BananaMind 2 Pro: 32.08%
• SmolLM2-135M: 32.68%
• Difference: 0.60 points

That's while we only trained it on 20× fewer training tokens.

It doesn't win everything

To be clear, BananaMind 2 Pro does not beat SmolLM2 across the board.

SmolLM2 remains ahead on ARC Easy, ARC Challenge, PIQA, HellaSwag, ArithMark 3, ArithMark 2, INT Index, and Code.

But by a small margin on most.

Some of those gaps are higher though.

ARC Easy, for example:

• BananaMind 2 Pro: 53.38%
• SmolLM2-135M: 58.63%

INT Index:

• BananaMind 2 Pro: 24.96
• SmolLM2-135M: 27.13

So I think BananaMind 2 Definitly hasnt beaten those models.

Against SmolLM-135M

The original SmolLM-135M comparison is also interesting.

SmolLM-135M was trained on 600B tokens, six times as many as BananaMind 2 Pro.

Despite that, BananaMind 2 Pro comes out ahead on several benchmarks.

HellaSwag

• BananaMind 2 Pro: 42.78%
• SmolLM-135M: 42.70%

ArithMark 3

• BananaMind 2 Pro: 38.20%
• SmolLM-135M: 36.00%

ArithMark 2

• BananaMind 2 Pro: 32.08%
• SmolLM-135M: 28.84%

Base Bench

• BananaMind 2 Pro: 1138
• SmolLM-135M: 1125

BananaMind 2 Pro is 2.20 points ahead on ArithMark 3 and 3.24 points ahead on ArithMark 2, despite using half as 6x less tokens.

Base Bench

One of the more interesting results is Base Bench:

BananaMind 2 Pro produced the highest Base Bench score of the three models in this evaluation.

That obviously doesn't mean it's universally the strongest model. The individual benchmark results show that SmolLM2 still leads on most tasks.

But considering the training budget, it's a result I'm very happy with.

Code still has room to improve

Code is one of the areas where BananaMind 2 Pro still has a noticeable gap.

BananaMind 2 Pro trails SmolLM2 by 78 points and SmolLM by 178 points.

So while we're getting very close on several general reasoning and arithmetic benchmarks, code is still an obvious area to improve in future BananaMind models.

And yes, it was trained on a 5070 TI

The full 100B-token training run was done on a consumer GPU rather than an H100, A100, or other datacenter accelerator.

That's a big part of what makes this project interesting to me. BananaMind is built around the idea that meaningful language-model training and experimentation shouldn't be limited to people with access to massive GPU clusters.

At ~140M parameters, BananaMind 2 Pro is small enough to train, run, benchmark, inspect, and modify on hardware you can actually have in a desktop PC.

What I'm taking away from this

The biggest takeaway isn't that 100B tokens is some magic number.

It's that training-token count alone doesn't determine how good a small language model will be.

Architecture, data quality, data mixture, optimization, tokenizer choices, and training strategy all matter.

SmolLM2 remains extremely strong at this parameter scale, and its 2T-token training run clearly gives it an advantage on several benchmarks.

But seeing BananaMind 2 Pro get this close after only 100B tokens is encouraging.

Especially when the difference is:

100B tokens vs 2T tokens.

1× vs 20×.

There is still plenty to improve.

ARC Easy has a noticeable gap.

INT Index can go higher.

Code needs work.

But BananaMind 2 Pro has reached the point where the question isn't just:

Can a small experimental model learn anything useful?

It's becoming:

How much performance can we extract from every training token?

That's what I want to keep pushing with BananaMind.

---

BananaMind 2 Pro

• Parameters: ~140M
• Training tokens: 100B
• ARC Easy: 53.38%
• ARC Challenge: 27.28%
• PIQA: 67.52%
• HellaSwag: 42.78%
• ArithMark 3: 38.20%
• ArithMark 2: 32.08%
• INT Index: 24.96
• Code: 1407
• Base Bench: 1138

20× fewer training tokens than SmolLM2-135M.

Our next improvements will be completly new architectures.