---
license: mit
language:
- en
- fr
tags:
- spiking-neural-networks
- one-shot-learning
- associative-memory
- hippocampus
- stdp
- neuromorphic-computing
- sequence-learning
- cpu-only
---

# SPIRA — Spiking Plastic Recurrent Architecture

![SPIRA architecture](paper/figures/fig_architecture.png)

A CPU-only sequence-learning system built **outside the transformer / BPE /
backpropagation paradigm**, from three components:

| Component | What it replaces |
|---|---|
| **EPT** — entropy-patch tokenization | BPE / WordPiece / SentencePiece |
| **Hierarchical spiking cortex** — fast LIF reservoir + slow area, reward-modulated plasticity | Transformer + backprop |
| **Hippocampus** — one-shot associative memory with replay consolidation | Fine-tuning / gradient learning |

**v2** of the prototype: a 16,384-column readout (chosen from a measured coverage
curve), a two-timescale hierarchical cortex, and hippocampal replay that closes the
complementary-systems loop.

## Headline results (2 vCPU, 4 GB RAM, 13.2 MB mixed text+code corpus)

| Proof | Result |
|---|---|
| One-shot learning | **99.3%** exact recall after a single 0.4 s pass over an unseen book |
| **Systems consolidation** | cortex-only recall of that book **0.3% → 46.9%** (64.2% in-vocab, 138×) after replaying 100k hippocampal tokens — the hippocampus itself stays at 99.3% |
| Pattern completion | **99.0%** recall with 1 of 4 context patches corrupted (sparse AM); 52.3% in the deployed full AM (capacity/completion trade) |
| Replay interference | none — held-out accuracy *improves* slightly after replay |
| Zero-shot coverage | 6.3% patch novelty on held-out text+code; 16k readout covers 64% of held-out targets (v1: 38%) |
| Efficiency | ≈9.2 M synaptic MACs/token — ≈**81× fewer ops/token** than GPT-2-small training (v1: 2.4 M, 314×) |
| Generation | ≈2,000 chars/s closed-loop on 2 vCPU; in-browser at ~450-700 chars/s |

Honest negative result (reported in the paper): the cortex does not amortize
generalization to unseen documents the way gradient-trained models do, and the slow
area's token-level fusion self-discounts at this scale — replay consolidates but does
not interpolate. The third iteration (next section) repaired the cortex's readout
representation and narrowed the gap substantially, but at matched vocabulary
(4,096) the SNN alone still trails the patch bigram on held-out text — **2.07% vs
4.94% in-vocab** under a controlled A/B — so the negative result stands, now
quantified head-to-head. See [`paper/spira_paper.pdf`](paper/spira_paper.pdf),
Sections 7–8, and [`V3_REPORT.md`](V3_REPORT.md).

## Third iteration (v3): cortex representation repair

Probes on the shipped v2 checkpoint located the cortex's held-out weakness in the
**representation**, not the learning rule: the reservoir state's effective memory
horizon was ≈1 token (the current patch decodes from the state at 99.94%, the
previous patch at 39.7% vs 33.0% majority chance), and an offline ridge readout
fitted on the same states scored 0.36% — *worse* than the online delta readout's
0.51%. The readout simply never saw the input, and nothing older than one token
survived in its state. v3 fixes exactly that:

1. **Direct input pathway** — a decaying trace (τ = 0.5) of the exact input-neuron
   pattern is concatenated to the readout state, so the readout sees the recent
   input identities directly (as standard reservoir readouts do).
2. **Multi-timescale trace banks** — spike traces at τ = 0.4 / 0.8 / 0.95, the slow
   banks normalized by (1 − τ) so they do not swamp the fast one; readout state
   grows 512 → 2,560 dimensions.
3. **Structured inputs** — each patch activates 6 full-hash neurons **plus** 6
   byte-trigram hash neurons, so similar patches share input neurons and statistical
   strength.

Controlled A/B (identical protocol for both arms: same rebuilt corpus, train
13.27 MB / test 640 KB, vocabulary 4,096, SNN 200k tokens × 2 epochs, AM 50k
contexts, held-out test 20,000 positions, vocabulary coverage 45.24%; bigram
trained on train cols [0:600k] and evaluated on exactly the same positions):

| Held-out test, SNN alone, in-vocab | v2 baseline | v3 | patch bigram @4096 |
|---|---|---|---|
| Test (20k positions) | 0.40% | **2.07%** (×5.2) | **4.94%** |
| Trained region (8k positions) | 3.76% | **13.29%** (×3.5) | — |

**Honest verdict:** the pre-registered success criterion — the SNN alone beats the
bigram — is **not met** (2.07% vs 4.94%). The gap narrowed from ×12.4 to ×2.4, and
the representation diagnosis is confirmed experimentally, but it is not closed.
Free generation is still incoherent in both arms, and at this small budget fusion
*hurts* on held-out (v3 fused 0.13% < SNN alone 2.07%): the AM written on only 50k
contexts drags the score down. Costs: training 189 t/s vs ~530 t/s (2.7× slower),
checkpoint 45.9 MB vs 14.6 MB. Full details: [`V3_REPORT.md`](V3_REPORT.md),
raw numbers in `python/artifacts/metrics_v3_ab.json`, A/B checkpoint in
`python/artifacts/spira_model_v3_ab.npz`.

## Repository layout

```
python/            NumPy-only research prototype
  spira/           the package (hashing, ept, spiking [hierarchy], hippocampus,
                   model [teach/replay/calibrate], baselines, train, evaluate
                   [replay experiment], export_deploy [format v2])
  scripts/         corpus builder, chunked training driver, figure maker,
                   AM rebuild, probes/diagnostics from the research log
  artifacts/       trained checkpoint (72 MB), metrics.json, spike raster,
                   browser-demo screenshots
  README.md        full Python quick start & design notes
paper/             LaTeX source + compiled research paper (spira_paper.pdf)
web/               Next.js browser demo — interactive tokenizer visualization,
                   closed-loop generation, one-shot teach & corrupted recall,
                   spike-raster playback (TS port of the v2 model incl. the
                   hierarchical slow area)
```

The 29 MB browser-inference artifact (`spira_deploy.json`) is kept once, at
`web/public/spira_deploy.json`, where the demo fetches it.

## Quick start — web demo

```bash
cd web
npm install        # or: bun install
npm run dev        # http://localhost:3000
```

No backend needed: the full model (EPT tokenizer + 512-neuron spiking reservoir +
256-neuron slow area + associative memory) runs client-side, compatible with the
Python prototype. (Hippocampal replay is Python-only.)

## Quick start — Python prototype

```bash
cd python
pip install numpy                  # only dependency

python3 scripts/build_corpus.py    # downloads Gutenberg books + uses local
                                   # CPython stdlib -> data/{train,test,oneshot}.bin

# either evaluate the shipped checkpoint directly (resumable by section)...
python3 -m spira.evaluate --model artifacts/spira_model.npz \
    --train data/train.bin --test data/test.bin \
    --oneshot data/oneshot.bin --out artifacts/metrics.json

# ...or retrain from scratch (~55 min on 2 vCPU, chunked)
python3 scripts/train_chunk.py --start 0      --count 150000   # + more chunks to 600k
python3 scripts/train_chunk.py --start 0      --count 150000   # epoch 2 (+ chunks)
python3 scripts/train_chunk.py --start 450000 --count 150000 --raster
python3 scripts/train_chunk.py --am-only
python3 scripts/train_chunk.py --start 450000 --warmup 150000
```

See [`python/README.md`](python/README.md) for the replay/consolidation snippet and
the full design notes (variance-matched fusion, counterfactual trust gain, exact AM
checkpoint round-trips).

## Paper

[`paper/spira_paper.pdf`](paper/spira_paper.pdf) — *SPIRA: Sequence Learning Without
Backpropagation or GPU Clusters* (third iteration), preprint, October 2026. LaTeX
sources and all figures are included.

## License

MIT — see [`LICENSE`](LICENSE).

## Citation

```bibtex
@misc{spira2026,
  title  = {SPIRA: Sequence Learning Without Backpropagation or GPU Clusters},
  author = {SPIRA Project},
  year   = {2026},
  month  = {10},
  note   = {Preprint, third iteration},
  url    = {https://huggingface.co/thefinalboss/SPIRA}
}
```