Theskywalker07
/

expert_2_16_64

Model card Files Files and versions

2.81 GB

Ctrl+K

Ctrl+K

1 contributor

History: 15 commits

Theskywalker07's picture

Upload full run contents (scripts, checkpoints, and results) (part 13)

142495f verified 21 days ago

babylm_eval_repo
Upload full run contents (scripts, checkpoints, and results) (part 13) 21 days ago
checkpoints
Upload full run contents (scripts, checkpoints, and results) (part 13) 21 days ago
results
Upload full run contents (scripts, checkpoints, and results) (part 13) 21 days ago
.DS_Store

6.15 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 22 days ago
.gitattributes

2.33 kB
Upload full run contents (scripts, checkpoints, and results) (part 12) 21 days ago
README-2.md

3.28 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 22 days ago
configuration_xpertgpt.py

1.08 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 22 days ago
modeling_xpertgpt.py

19 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 22 days ago
train.py

72.6 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 22 days ago
training_terminal.log

1.39 MB
Upload full run contents (scripts, checkpoints, and results) (part 13) 21 days ago
upload_all_hf.py

3.71 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 22 days ago
upload_project_hf.py

3.24 kB
Update strict-small architecture files (SwiGLU, sliding window [64, 16, 8, 4], ln3, ln_post_moe, no res3) 22 days ago