Matt-94 commited on
Commit
ebd23bf
·
verified ·
1 Parent(s): 49755e7

Upload pretrained model.pt (FS-SSA-LM-94M, PPL 37.14)

Browse files

Pretrained checkpoint for FS-SSA-GPT (~93.9M parameters):
- Architecture: Softmax-Free Spiking Self-Attention (FS-SSA), K=2 timesteps, signed spikes (+/-)
- Dataset: FineWeb-Edu (~655M tokens seen across 10,000 steps)
- Metrics: Best Val Loss = 3.61467 | Best PPL = 37.14
- Training setup: A100-SXM4-80GB, causal decay ladder, learnable channel gains
- DOI: 10.5281/zenodo.22702448

Files changed (1) hide show
  1. model.pt +3 -0
model.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1ebfec0d8b8830b3915a2a6fab6271b771e965feecd88a093613f8cb97264fc
3
+ size 1799126029