metadata
license: apache-2.0
datasets:
- HuggingFaceFW/fineweb
- HuggingFaceFW/fineweb-2
- openbmb/UltraData-Code
- IFM/Math-Reasoning
- openbmb/UltraData-SFT-2605
- openbmb/UltraData-SFT-Agent-2609
language:
- en
- zh
pipeline_tag: text-generation
Entropy the size of a positron
Incredibly small 3 expert Mixture of Experts trained to be decent in coding, math, and generic conversation. I'm still training it :'D
I suggest using BeeLlama.cpp for squeezing as much performance as possible out of this thing :]
This model will include an mmproj to try to have basic image support using the frozen embeddings of wkcn/TinyCLIP-ViT-8M-16-Text-3M-YFCC15M.