| --- |
| license: apache-2.0 |
| datasets: |
| - HuggingFaceFW/fineweb |
| - HuggingFaceFW/fineweb-2 |
| - openbmb/UltraData-Code |
| - IFM/Math-Reasoning |
| - openbmb/UltraData-SFT-2605 |
| - openbmb/UltraData-SFT-Agent-2609 |
| language: |
| - en |
| - zh |
| pipeline_tag: text-generation |
| --- |
|  |
|
|
| # Entropy the size of a positron |
| Incredibly small 3 expert Mixture of Experts trained to be decent in coding, math, and generic conversation. I'm still training it :'D |
|
|
| I suggest using [BeeLlama.cpp](https://github.com/Anbeeld/beellama.cpp) for squeezing as much performance as possible out of this thing :] |
|
|
| This model will include an mmproj to try to have basic image support using the frozen embeddings of [wkcn/TinyCLIP-ViT-8M-16-Text-3M-YFCC15M](https://huggingface.co/wkcn/TinyCLIP-ViT-8M-16-Text-3M-YFCC15M). |
