AwareLiquid
/

Sparse-SNN

@@ -1,112 +1,130 @@
----
-tags:
-- spiking-neural-network
-- snn
-- sparse
-- brain-inspired
-- connectome
-- drosophila
-- mnist
-- fashion-mnist
-- image-classification
-- awareliquid
-pipeline_tag: image-classification
----
-# Sparse-SNN — connectome-inspired sparse spiking networks
-**Sparse spiking neural networks whose structural masks are derived from the
-statistical laws of the real *Drosophila* connectome** — matching dense MLP
-accuracy at ~2 orders of magnitude lower energy.
-Two trained classifiers: MNIST and Fashion-MNIST. Architecture: `784 → 800 → 10`
-with 5% connection density, LIF neurons and surrogate-gradient training
-(straight-through estimator with sigmoid gradient).
-## Files
-| File | Description |
-|---|---|
-| `sparse_snn_mnist.pt` | MNIST checkpoint — **96.83%** test accuracy (firing rate 7.9%) |
-| `sparse_snn_fashion.pt` | Fashion-MNIST checkpoint — **87.07%** test accuracy (firing rate 8.2%) |
-| `train_results.json` | Raw metrics from the training runs |
-Each checkpoint contains the `state_dict` (weights + mask buffer), the E/I sign
-matrix, the full config, and metrics.
-## Measured results (these checkpoints)
-| Task | Dense MLP (documented) | Sparse SNN (this checkpoint) | Gap | Energy saving* |
-|---|---|---|---|---|
-| MNIST | 98.32% | **96.83%** | 1.5 pts | **~112×** |
-| Fashion-MNIST | 87.56% | **87.07%** | 0.5 pts | **~107×** |
-\* Energy estimate using the repository's 45nm CMOS model (Horowitz 2014:
-MAC = 3.7 pJ, addition = 0.9 pJ) with this checkpoint's measured firing rate —
-event-driven spike ops are additions only. This is a **model-based estimate,
-not measured on hardware**. The project's documented headline range across runs
-is 105–106×.
-## What the research found (and disproved)
-1. **Disproved**: the MaleCNS connectome topology as a *static* structure
-   carries **no measurable advantage** over random / structured-sparse graphs
-   (5 experiments: classification, temporal, robustness, sample-efficiency,
-   plasticity — all indistinguishable).
-2. **Extracted** — the topology's *statistical laws* that DO matter:
-   sparsity **0.09%**, long-tail degree distribution (scale-free, max/mean ≈ 75),
-   strong small-worldness (clustering 6.65× random, path length 2.39),
-   **E/I ratio 60/40**.
-3. **Built**: a trainable sparse SNN from these laws → near-lossless accuracy
-   at ~105–112× energy savings.
-## Usage
-Requires the model code from the repository (the class is small and
-self-contained):
-```python
-import torch
-from sparse_snn import SparseSNN   # github.com/AwareLiquid/human-brain-simulation
-ckpt = torch.load("sparse_snn_mnist.pt", map_location="cpu", weights_only=False)
-cfg = ckpt["config"]
-model = SparseSNN(cfg["in_dim"], cfg["hid_dim"], cfg["out_dim"],
-                  ckpt["state_dict"]["mask"], ckpt["sign"],
-                  T=cfg["T"], decay=cfg["decay"], threshold=cfg["threshold"])
-model.load_state_dict(ckpt["state_dict"])
-model.eval()
-# inputs: flattened 28x28 grayscale in [0, 1]
-```
-Loading was verified to reproduce the saved accuracy bit-exactly
-(`acc=0.9683` MNIST / `acc=0.8707` Fashion on reload).
-## Honest boundaries
-- The accuracy gap vs a dense MLP is small but real (1.5 pts MNIST, 0.5 pts
-  Fashion) — "near-lossless", not lossless.
-- The connectome's *static topology* gave no advantage; only its statistical
-  laws transferred. Do not attribute the result to "copying the fly brain".
-- Energy figures are analytic (45nm CMOS), not measured on neuromorphic
-  hardware.
-- Recurrent spiking training remains an open problem in this line (sMNIST from
-  scratch reaches only ~70%; conversion methods do better but lose efficiency).
-- Masks are generated with a fixed seed (`mask_seed=0`) — rerun the code in the
-  repository to regenerate the exact structure.
-## Related
-- Code & full experiment report: [AwareLiquid/human-brain-simulation](https://github.com/AwareLiquid/human-brain-simulation)
-- [AwareLiquid/M1](https://github.com/AwareLiquid/M1) — MT-LNN liquid architecture
-- [awareliquid.ai](https://awareliquid.ai) — benchmarks and retractions
-## Try it
-- 🎮 Interactive demo (runs in your browser, ONNX Runtime Web): https://huggingface.co/spaces/AwareLiquid/Sparse-SNN-demo
-## License
-MIT. MaleCNS connectome data is CC-BY 4.0 (HHMI Janelia / Cambridge / Google
-Research, male-cns.janelia.org) — not redistributed here.

+---
+tags:
+- spiking-neural-network
+- snn
+- sparse
+- brain-inspired
+- connectome
+- drosophila
+- mnist
+- fashion-mnist
+- image-classification
+- awareliquid
+pipeline_tag: image-classification
+---
+# Sparse-SNN — connectome-inspired sparse spiking networks
+**Sparse spiking neural networks whose structural masks are derived from the
+statistical laws of the real *Drosophila* connectome** — matching dense MLP
+accuracy at ~2 orders of magnitude lower energy.
+Two trained classifiers: MNIST and Fashion-MNIST. Architecture: `784 → 800 → 10`
+with 5% connection density, LIF neurons and surrogate-gradient training
+(straight-through estimator with sigmoid gradient).
+## Files
+| File | Description |
+|---|---|
+| `sparse_snn_mnist.pt` | MNIST checkpoint — **96.83%** test accuracy (firing rate 7.9%) |
+| `sparse_snn_fashion.pt` | Fashion-MNIST checkpoint — **87.07%** test accuracy (firing rate 8.2%) |
+| `train_results.json` | Raw metrics from the training runs |
+Each checkpoint contains the `state_dict` (weights + mask buffer), the E/I sign
+matrix, the full config, and metrics.
+## ResNet SNN — CIFAR-10 (2026-09, human-brain-simulation repo)
+ResNet-style spiking network (LIF + firing-rate residuals) trained directly on
+CIFAR-10, six-experiment series, all recorded:
+| Experiment | CIFAR-10 |
+|---|---|
+| **ResNet SNN v1 (3 residual blocks, 60 ep)** | **84.76%** |
+| v1 + 200 ep + Cutout (long training) | **84.89%** |
+| Full-depth ResNet-18 (9 LIF layers) | 67.93% (spike death) |
+| Wide channels 96/192/384 | 66.96–75.50% (fixed-threshold saturation) |
+| PLIF learnable threshold + Cutout | 83.43% |
+Honest ceiling: **~85%** for this direct-training recipe family on CIFAR-10
+(dense CNN baseline 75.00%; VGG-SNN variant 86.22% in the same repo). 90%+
+needs a different route (T≥10 + threshold balancing, or ANN→SNN conversion
+calibration). Details: github.com/AwareLiquid/human-brain-simulation.
+## Measured results (these checkpoints)
+| Task | Dense MLP (documented) | Sparse SNN (this checkpoint) | Gap | Energy saving* |
+|---|---|---|---|---|
+| MNIST | 98.32% | **96.83%** | 1.5 pts | **~112×** |
+| Fashion-MNIST | 87.56% | **87.07%** | 0.5 pts | **~107×** |
+\* Energy estimate using the repository's 45nm CMOS model (Horowitz 2014:
+MAC = 3.7 pJ, addition = 0.9 pJ) with this checkpoint's measured firing rate —
+event-driven spike ops are additions only. This is a **model-based estimate,
+not measured on hardware**. The project's documented headline range across runs
+is 105–106×.
+## What the research found (and disproved)
+1. **Disproved**: the MaleCNS connectome topology as a *static* structure
+   carries **no measurable advantage** over random / structured-sparse graphs
+   (5 experiments: classification, temporal, robustness, sample-efficiency,
+   plasticity — all indistinguishable).
+2. **Extracted** — the topology's *statistical laws* that DO matter:
+   sparsity **0.09%**, long-tail degree distribution (scale-free, max/mean ≈ 75),
+   strong small-worldness (clustering 6.65× random, path length 2.39),
+   **E/I ratio 60/40**.
+3. **Built**: a trainable sparse SNN from these laws → near-lossless accuracy
+   at ~105–112× energy savings.
+## Usage
+Requires the model code from the repository (the class is small and
+self-contained):
+```python
+import torch
+from sparse_snn import SparseSNN   # github.com/AwareLiquid/human-brain-simulation
+ckpt = torch.load("sparse_snn_mnist.pt", map_location="cpu", weights_only=False)
+cfg = ckpt["config"]
+model = SparseSNN(cfg["in_dim"], cfg["hid_dim"], cfg["out_dim"],
+                  ckpt["state_dict"]["mask"], ckpt["sign"],
+                  T=cfg["T"], decay=cfg["decay"], threshold=cfg["threshold"])
+model.load_state_dict(ckpt["state_dict"])
+model.eval()
+# inputs: flattened 28x28 grayscale in [0, 1]
+```
+Loading was verified to reproduce the saved accuracy bit-exactly
+(`acc=0.9683` MNIST / `acc=0.8707` Fashion on reload).
+## Honest boundaries
+- The accuracy gap vs a dense MLP is small but real (1.5 pts MNIST, 0.5 pts
+  Fashion) — "near-lossless", not lossless.
+- The connectome's *static topology* gave no advantage; only its statistical
+  laws transferred. Do not attribute the result to "copying the fly brain".
+- Energy figures are analytic (45nm CMOS), not measured on neuromorphic
+  hardware.
+- Recurrent spiking training remains an open problem in this line (sMNIST from
+  scratch reaches only ~70%; conversion methods do better but lose efficiency).
+- Masks are generated with a fixed seed (`mask_seed=0`) — rerun the code in the
+  repository to regenerate the exact structure.
+## Related
+- Code & full experiment report: [AwareLiquid/human-brain-simulation](https://github.com/AwareLiquid/human-brain-simulation)
+- [AwareLiquid/M1](https://github.com/AwareLiquid/M1) — MT-LNN liquid architecture
+- [awareliquid.ai](https://awareliquid.ai) — benchmarks and retractions
+## Try it
+- 🎮 Interactive demo (runs in your browser, ONNX Runtime Web): https://huggingface.co/spaces/AwareLiquid/Sparse-SNN-demo
+## License
+MIT. MaleCNS connectome data is CC-BY 4.0 (HHMI Janelia / Cambridge / Google
+Research, male-cns.janelia.org) — not redistributed here.