BananaMind 2 Pico — Chinchilla Test

Freshly initialized 900,002-parameter Pico model trained to the Chinchilla budget of 20 tokens per parameter. This is a controlled short-run comparison, not a continuation of Banaxi-Tech/pico-test.

Field Value
Parameters 900,002
Training tokens 18,000,040
Tokens per parameter 20.000000
Context / vocabulary 4,096 / 384
Architecture 6L, 96D, XSA refresh on layers 4 and 6
Data 70% FineWeb-HQ, 30% Cosmopedia v2
Optimizers Muon 0.07 + AdamW 0.004
Precision / hardware bfloat16 autocast / NVIDIA RTX PRO 6000 Blackwell Server Edition
Template revision 5fa6920cf5ae031f03bd044a4de12564ec453837

Load with trust_remote_code=True.

Downloads last month
-
Safetensors
Model size
937k params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train Banaxi-Tech/test-pico-chinchilla