Spaces:
Running
Running
| title: README | |
| emoji: 🧠 | |
| colorFrom: indigo | |
| colorTo: blue | |
| sdk: static | |
| pinned: false | |
| # TinyBrainBot | |
| Small language models trained from scratch on limited hardware, released with the details of how they were made: the data, the training runs, and the things that went wrong along the way. | |
| **Start here:** [TinyBrainBot-350M-v4-Thinking](https://huggingface.co/nkthebass/tinybrainbot-350m-v4-thinking) thinks through a question before it answers, and replies to small talk directly. A GGUF for llama.cpp, LM Studio and Ollama is included. | |
| **Try it now:** [TinyBrainBot 100M Thinking demo](https://huggingface.co/spaces/nkthebass/tinybrainbot-100m-thinking) runs the 100M thinking model right in your browser, no install needed. | |
| ## Models | |
| | Series | Models | Notes | | |
| |---|---|---| | |
| | [v4](https://huggingface.co/collections/nkthebass/tinybrainbot-v4-6ac14ac6a9a6a6806ba4a707) | [350M Thinking](https://huggingface.co/nkthebass/tinybrainbot-350m-v4-thinking), [100M Thinking](https://huggingface.co/nkthebass/tinybrainbot-100m-v4-thinking), [25M Base](https://huggingface.co/nkthebass/tinybrainbot-25m-v4-base) | Newest. Thinking models that reason inside `<think>` before answering, and a 25M base that beats Pythia-70M on 10 of 13 benchmarks | | |
| | [v3](https://huggingface.co/collections/nkthebass/tinybrainbot-v3-6ac14ac4b73470ecf713992f) | 350M and 100M base, instruct and math | The 100M beats Supra2-100M-Instruct on 6 of 7 benchmarks | | |
| | [v2](https://huggingface.co/collections/nkthebass/tinybrainbot-v2-6ac14ac3659e8a86ec3923cd) | 320M and 303M base, instruct and math | The 320M math model beats GPT-3 175B on 4 to 5 digit arithmetic | | |
| | [v1](https://huggingface.co/collections/nkthebass/tinybrainbot-v1-6ac14ac38f73e742da1998dd) | 303M base and instruct, 216M demo | The first models | | |
| ## Research | |
| **[Do data-mix rankings transfer across scale?](https://huggingface.co/nkthebass/tinybrainbot-pilot-100m-datamix)** A controlled study at 100M parameters: three models that are identical except for their training data, measured on held-out text, seven domain slices and fifteen benchmarks. | |
| **[One recipe, five sizes](https://huggingface.co/nkthebass/tinybrainbot-v4-scaling-500k-25m)** Five models from 500K to 25M parameters trained on exactly the same 11B tokens, so the only difference is size. The 10M averages above Pythia-70M. | |
| All models are currently published from [nkthebass](https://huggingface.co/nkthebass). | |