Upload pretrained model.pt (FS-SSA-LM-94M, PPL 37.14)
Browse filesPretrained checkpoint for FS-SSA-GPT (~93.9M parameters):
- Architecture: Softmax-Free Spiking Self-Attention (FS-SSA), K=2 timesteps, signed spikes (+/-)
- Dataset: FineWeb-Edu (~655M tokens seen across 10,000 steps)
- Metrics: Best Val Loss = 3.61467 | Best PPL = 37.14
- Training setup: A100-SXM4-80GB, causal decay ladder, learnable channel gains
- DOI: 10.5281/zenodo.22702448