-
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3)
-
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3)
-
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3)
-
Upload full ablation run contents (scripts, checkpoints, and results)
-
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3)
-
initial commit