Open MoE LLM for coding trained from scratch on free compute. 3.45B total / ~0.5B active, Gated DeltaNet + full attention.
AI & ML interests
Open mixture-of-experts LLMs, code generation, long context, linear attention (Gated DeltaNet), training on free compute (JAX/TPU)
Recent Activity
View all activity