A small Mixtral-style Mixture-of-Experts causal language model for pretraining research on a single consumer GPU.
Michael Lee
mikecovlee
·
AI & ML interests
LLM
Recent Activity
updated a model 2 days ago
mikecovlee/tinymixtral-v2.0-beta updated a model 2 days ago
mikecovlee/tinymixtral-v1.1-1b updated a model 2 days ago
mikecovlee/tinymixtral-v1.1-0.5bOrganizations
models 5
mikecovlee/tinymixtral-v2.0-beta
Text Generation • Updated • 236
mikecovlee/tinymixtral-v1.1-1b
Text Generation • Updated • 295
mikecovlee/tinymixtral-v1.1-0.5b
Text Generation • Updated • 247 • 1
mikecovlee/tinymixtral
Text Generation • 0.5B • Updated • 251
mikecovlee/tinymixtral-v1.0
Text Generation • Updated • 228 • 1
datasets 0
None public yet