111 kB

Ctrl+K

Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3)

8e73a21 verified 23 days ago

.DS_Store

6.15 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 23 days ago
.gitattributes

1.52 kB
initial commit 23 days ago
README-2.md

3.26 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 23 days ago
configuration_xpertgpt.py

1.08 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 23 days ago
modeling_xpertgpt.py

19.1 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 23 days ago
train.py

72.6 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 23 days ago
upload_all_hf.py

3.72 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 23 days ago
upload_project_hf.py

3.29 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 23 days ago