| --- |
| tags: |
| - spiking-neural-network |
| - snn |
| - sparse |
| - brain-inspired |
| - connectome |
| - drosophila |
| - mnist |
| - fashion-mnist |
| - image-classification |
| - awareliquid |
| pipeline_tag: image-classification |
| --- |
| |
| # Sparse-SNN — connectome-inspired sparse spiking networks |
|
|
| **Sparse spiking neural networks whose structural masks are derived from the |
| statistical laws of the real *Drosophila* connectome** — matching dense MLP |
| accuracy at ~2 orders of magnitude lower energy. |
|
|
| Two trained classifiers: MNIST and Fashion-MNIST. Architecture: `784 → 800 → 10` |
| with 5% connection density, LIF neurons and surrogate-gradient training |
| (straight-through estimator with sigmoid gradient). |
|
|
| ## Files |
|
|
| | File | Description | |
| |---|---| |
| | `sparse_snn_mnist.pt` | MNIST checkpoint — **96.83%** test accuracy (firing rate 7.9%) | |
| | `sparse_snn_fashion.pt` | Fashion-MNIST checkpoint — **87.07%** test accuracy (firing rate 8.2%) | |
| | `train_results.json` | Raw metrics from the training runs | |
|
|
| Each checkpoint contains the `state_dict` (weights + mask buffer), the E/I sign |
| matrix, the full config, and metrics. |
|
|
| ## ResNet SNN — CIFAR-10 (2026-09, human-brain-simulation repo) |
|
|
| ResNet-style spiking network (LIF + firing-rate residuals) trained directly on |
| CIFAR-10, six-experiment series, all recorded: |
|
|
| | Experiment | CIFAR-10 | |
| |---|---| |
| | **ResNet SNN v1 (3 residual blocks, 60 ep)** | **84.76%** | |
| | v1 + 200 ep + Cutout (long training) | **84.89%** | |
| | Full-depth ResNet-18 (9 LIF layers) | 67.93% (spike death) | |
| | Wide channels 96/192/384 | 66.96–75.50% (fixed-threshold saturation) | |
| | PLIF learnable threshold + Cutout | 83.43% | |
|
|
| Honest ceiling: **~85%** for this direct-training recipe family on CIFAR-10 |
| (dense CNN baseline 75.00%; VGG-SNN variant 86.22% in the same repo). 90%+ |
| needs a different route (T≥10 + threshold balancing, or ANN→SNN conversion |
| calibration). Details: github.com/AwareLiquid/human-brain-simulation. |
|
|
| ## Measured results (these checkpoints) |
|
|
| | Task | Dense MLP (documented) | Sparse SNN (this checkpoint) | Gap | Energy saving* | |
| |---|---|---|---|---| |
| | MNIST | 98.32% | **96.83%** | 1.5 pts | **~112×** | |
| | Fashion-MNIST | 87.56% | **87.07%** | 0.5 pts | **~107×** | |
|
|
| \* Energy estimate using the repository's 45nm CMOS model (Horowitz 2014: |
| MAC = 3.7 pJ, addition = 0.9 pJ) with this checkpoint's measured firing rate — |
| event-driven spike ops are additions only. This is a **model-based estimate, |
| not measured on hardware**. The project's documented headline range across runs |
| is 105–106×. |
|
|
| ## What the research found (and disproved) |
|
|
| 1. **Disproved**: the MaleCNS connectome topology as a *static* structure |
| carries **no measurable advantage** over random / structured-sparse graphs |
| (5 experiments: classification, temporal, robustness, sample-efficiency, |
| plasticity — all indistinguishable). |
| 2. **Extracted** — the topology's *statistical laws* that DO matter: |
| sparsity **0.09%**, long-tail degree distribution (scale-free, max/mean ≈ 75), |
| strong small-worldness (clustering 6.65× random, path length 2.39), |
| **E/I ratio 60/40**. |
| 3. **Built**: a trainable sparse SNN from these laws → near-lossless accuracy |
| at ~105–112× energy savings. |
|
|
| ## Usage |
|
|
| Requires the model code from the repository (the class is small and |
| self-contained): |
|
|
| ```python |
| import torch |
| from sparse_snn import SparseSNN # github.com/AwareLiquid/human-brain-simulation |
| |
| ckpt = torch.load("sparse_snn_mnist.pt", map_location="cpu", weights_only=False) |
| cfg = ckpt["config"] |
| model = SparseSNN(cfg["in_dim"], cfg["hid_dim"], cfg["out_dim"], |
| ckpt["state_dict"]["mask"], ckpt["sign"], |
| T=cfg["T"], decay=cfg["decay"], threshold=cfg["threshold"]) |
| model.load_state_dict(ckpt["state_dict"]) |
| model.eval() |
| # inputs: flattened 28x28 grayscale in [0, 1] |
| ``` |
|
|
| Loading was verified to reproduce the saved accuracy bit-exactly |
| (`acc=0.9683` MNIST / `acc=0.8707` Fashion on reload). |
|
|
| ## Honest boundaries |
|
|
| - The accuracy gap vs a dense MLP is small but real (1.5 pts MNIST, 0.5 pts |
| Fashion) — "near-lossless", not lossless. |
| - The connectome's *static topology* gave no advantage; only its statistical |
| laws transferred. Do not attribute the result to "copying the fly brain". |
| - Energy figures are analytic (45nm CMOS), not measured on neuromorphic |
| hardware. |
| - Recurrent spiking training remains an open problem in this line (sMNIST from |
| scratch reaches only ~70%; conversion methods do better but lose efficiency). |
| - Masks are generated with a fixed seed (`mask_seed=0`) — rerun the code in the |
| repository to regenerate the exact structure. |
|
|
| ## Related |
|
|
| - Code & full experiment report: [AwareLiquid/human-brain-simulation](https://github.com/AwareLiquid/human-brain-simulation) |
| - [AwareLiquid/M1](https://github.com/AwareLiquid/M1) — MT-LNN liquid architecture |
| - [awareliquid.ai](https://awareliquid.ai) — benchmarks and retractions |
|
|
|
|
| ## Try it |
|
|
| - 🎮 Interactive demo (runs in your browser, ONNX Runtime Web): https://huggingface.co/spaces/AwareLiquid/Sparse-SNN-demo |
|
|
| ## License |
|
|
| MIT. MaleCNS connectome data is CC-BY 4.0 (HHMI Janelia / Cambridge / Google |
| Research, male-cns.janelia.org) — not redistributed here. |
|
|
