File size: 5,194 Bytes
cc5c3e9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | ---
tags:
- spiking-neural-network
- snn
- sparse
- brain-inspired
- connectome
- drosophila
- mnist
- fashion-mnist
- image-classification
- awareliquid
pipeline_tag: image-classification
---
# Sparse-SNN — connectome-inspired sparse spiking networks
**Sparse spiking neural networks whose structural masks are derived from the
statistical laws of the real *Drosophila* connectome** — matching dense MLP
accuracy at ~2 orders of magnitude lower energy.
Two trained classifiers: MNIST and Fashion-MNIST. Architecture: `784 → 800 → 10`
with 5% connection density, LIF neurons and surrogate-gradient training
(straight-through estimator with sigmoid gradient).
## Files
| File | Description |
|---|---|
| `sparse_snn_mnist.pt` | MNIST checkpoint — **96.83%** test accuracy (firing rate 7.9%) |
| `sparse_snn_fashion.pt` | Fashion-MNIST checkpoint — **87.07%** test accuracy (firing rate 8.2%) |
| `train_results.json` | Raw metrics from the training runs |
Each checkpoint contains the `state_dict` (weights + mask buffer), the E/I sign
matrix, the full config, and metrics.
## ResNet SNN — CIFAR-10 (2026-09, human-brain-simulation repo)
ResNet-style spiking network (LIF + firing-rate residuals) trained directly on
CIFAR-10, six-experiment series, all recorded:
| Experiment | CIFAR-10 |
|---|---|
| **ResNet SNN v1 (3 residual blocks, 60 ep)** | **84.76%** |
| v1 + 200 ep + Cutout (long training) | **84.89%** |
| Full-depth ResNet-18 (9 LIF layers) | 67.93% (spike death) |
| Wide channels 96/192/384 | 66.96–75.50% (fixed-threshold saturation) |
| PLIF learnable threshold + Cutout | 83.43% |
Honest ceiling: **~85%** for this direct-training recipe family on CIFAR-10
(dense CNN baseline 75.00%; VGG-SNN variant 86.22% in the same repo). 90%+
needs a different route (T≥10 + threshold balancing, or ANN→SNN conversion
calibration). Details: github.com/AwareLiquid/human-brain-simulation.
## Measured results (these checkpoints)
| Task | Dense MLP (documented) | Sparse SNN (this checkpoint) | Gap | Energy saving* |
|---|---|---|---|---|
| MNIST | 98.32% | **96.83%** | 1.5 pts | **~112×** |
| Fashion-MNIST | 87.56% | **87.07%** | 0.5 pts | **~107×** |
\* Energy estimate using the repository's 45nm CMOS model (Horowitz 2014:
MAC = 3.7 pJ, addition = 0.9 pJ) with this checkpoint's measured firing rate —
event-driven spike ops are additions only. This is a **model-based estimate,
not measured on hardware**. The project's documented headline range across runs
is 105–106×.
## What the research found (and disproved)
1. **Disproved**: the MaleCNS connectome topology as a *static* structure
carries **no measurable advantage** over random / structured-sparse graphs
(5 experiments: classification, temporal, robustness, sample-efficiency,
plasticity — all indistinguishable).
2. **Extracted** — the topology's *statistical laws* that DO matter:
sparsity **0.09%**, long-tail degree distribution (scale-free, max/mean ≈ 75),
strong small-worldness (clustering 6.65× random, path length 2.39),
**E/I ratio 60/40**.
3. **Built**: a trainable sparse SNN from these laws → near-lossless accuracy
at ~105–112× energy savings.
## Usage
Requires the model code from the repository (the class is small and
self-contained):
```python
import torch
from sparse_snn import SparseSNN # github.com/AwareLiquid/human-brain-simulation
ckpt = torch.load("sparse_snn_mnist.pt", map_location="cpu", weights_only=False)
cfg = ckpt["config"]
model = SparseSNN(cfg["in_dim"], cfg["hid_dim"], cfg["out_dim"],
ckpt["state_dict"]["mask"], ckpt["sign"],
T=cfg["T"], decay=cfg["decay"], threshold=cfg["threshold"])
model.load_state_dict(ckpt["state_dict"])
model.eval()
# inputs: flattened 28x28 grayscale in [0, 1]
```
Loading was verified to reproduce the saved accuracy bit-exactly
(`acc=0.9683` MNIST / `acc=0.8707` Fashion on reload).
## Honest boundaries
- The accuracy gap vs a dense MLP is small but real (1.5 pts MNIST, 0.5 pts
Fashion) — "near-lossless", not lossless.
- The connectome's *static topology* gave no advantage; only its statistical
laws transferred. Do not attribute the result to "copying the fly brain".
- Energy figures are analytic (45nm CMOS), not measured on neuromorphic
hardware.
- Recurrent spiking training remains an open problem in this line (sMNIST from
scratch reaches only ~70%; conversion methods do better but lose efficiency).
- Masks are generated with a fixed seed (`mask_seed=0`) — rerun the code in the
repository to regenerate the exact structure.
## Related
- Code & full experiment report: [AwareLiquid/human-brain-simulation](https://github.com/AwareLiquid/human-brain-simulation)
- [AwareLiquid/M1](https://github.com/AwareLiquid/M1) — MT-LNN liquid architecture
- [awareliquid.ai](https://awareliquid.ai) — benchmarks and retractions
## Try it
- 🎮 Interactive demo (runs in your browser, ONNX Runtime Web): https://huggingface.co/spaces/AwareLiquid/Sparse-SNN-demo
## License
MIT. MaleCNS connectome data is CC-BY 4.0 (HHMI Janelia / Cambridge / Google
Research, male-cns.janelia.org) — not redistributed here.
|