Theskywalker07
/

expert_1_64

Model card Files Files and versions

Ctrl+K

Ctrl+K

1 contributor

History: 15 commits

Theskywalker07's picture

Upload full run contents (scripts, checkpoints, and results) (part 13)

593feb9 verified 19 days ago

babylm_eval_repo
Upload full run contents (scripts, checkpoints, and results) (part 13) 19 days ago
checkpoints
Upload full run contents (scripts, checkpoints, and results) (part 13) 19 days ago
results
Upload full run contents (scripts, checkpoints, and results) (part 13) 19 days ago
.DS_Store

6.15 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 19 days ago
.gitattributes

2.33 kB
Upload full run contents (scripts, checkpoints, and results) (part 12) 19 days ago
README-2.md

3.26 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 19 days ago
configuration_xpertgpt.py

1.08 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 19 days ago
modeling_xpertgpt.py

19 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 19 days ago
train.py

72.5 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 19 days ago
training_terminal.log

1.39 MB
Upload full run contents (scripts, checkpoints, and results) (part 13) 19 days ago
upload_all_hf.py

3.7 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 19 days ago
upload_project_hf.py

3.23 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 19 days ago