SPIRA v3: cortex representation repair β direct input pathway, multi-timescale trace banks, structured trigram inputs; controlled A/B (SNN alone 0.40% -> 2.07% in-vocab held-out vs bigram 4.94% at matched vocab 4096); paper, README and docs updated to third iteration
6923c51 verified | license: mit | |
| language: | |
| - en | |
| - fr | |
| tags: | |
| - spiking-neural-networks | |
| - one-shot-learning | |
| - associative-memory | |
| - hippocampus | |
| - stdp | |
| - neuromorphic-computing | |
| - sequence-learning | |
| - cpu-only | |
| # SPIRA β Spiking Plastic Recurrent Architecture | |
|  | |
| A CPU-only sequence-learning system built **outside the transformer / BPE / | |
| backpropagation paradigm**, from three components: | |
| | Component | What it replaces | | |
| |---|---| | |
| | **EPT** β entropy-patch tokenization | BPE / WordPiece / SentencePiece | | |
| | **Hierarchical spiking cortex** β fast LIF reservoir + slow area, reward-modulated plasticity | Transformer + backprop | | |
| | **Hippocampus** β one-shot associative memory with replay consolidation | Fine-tuning / gradient learning | | |
| **v2** of the prototype: a 16,384-column readout (chosen from a measured coverage | |
| curve), a two-timescale hierarchical cortex, and hippocampal replay that closes the | |
| complementary-systems loop. | |
| ## Headline results (2 vCPU, 4 GB RAM, 13.2 MB mixed text+code corpus) | |
| | Proof | Result | | |
| |---|---| | |
| | One-shot learning | **99.3%** exact recall after a single 0.4 s pass over an unseen book | | |
| | **Systems consolidation** | cortex-only recall of that book **0.3% β 46.9%** (64.2% in-vocab, 138Γ) after replaying 100k hippocampal tokens β the hippocampus itself stays at 99.3% | | |
| | Pattern completion | **99.0%** recall with 1 of 4 context patches corrupted (sparse AM); 52.3% in the deployed full AM (capacity/completion trade) | | |
| | Replay interference | none β held-out accuracy *improves* slightly after replay | | |
| | Zero-shot coverage | 6.3% patch novelty on held-out text+code; 16k readout covers 64% of held-out targets (v1: 38%) | | |
| | Efficiency | β9.2 M synaptic MACs/token β β**81Γ fewer ops/token** than GPT-2-small training (v1: 2.4 M, 314Γ) | | |
| | Generation | β2,000 chars/s closed-loop on 2 vCPU; in-browser at ~450-700 chars/s | | |
| Honest negative result (reported in the paper): the cortex does not amortize | |
| generalization to unseen documents the way gradient-trained models do, and the slow | |
| area's token-level fusion self-discounts at this scale β replay consolidates but does | |
| not interpolate. The third iteration (next section) repaired the cortex's readout | |
| representation and narrowed the gap substantially, but at matched vocabulary | |
| (4,096) the SNN alone still trails the patch bigram on held-out text β **2.07% vs | |
| 4.94% in-vocab** under a controlled A/B β so the negative result stands, now | |
| quantified head-to-head. See [`paper/spira_paper.pdf`](paper/spira_paper.pdf), | |
| Sections 7β8, and [`V3_REPORT.md`](V3_REPORT.md). | |
| ## Third iteration (v3): cortex representation repair | |
| Probes on the shipped v2 checkpoint located the cortex's held-out weakness in the | |
| **representation**, not the learning rule: the reservoir state's effective memory | |
| horizon was β1 token (the current patch decodes from the state at 99.94%, the | |
| previous patch at 39.7% vs 33.0% majority chance), and an offline ridge readout | |
| fitted on the same states scored 0.36% β *worse* than the online delta readout's | |
| 0.51%. The readout simply never saw the input, and nothing older than one token | |
| survived in its state. v3 fixes exactly that: | |
| 1. **Direct input pathway** β a decaying trace (Ο = 0.5) of the exact input-neuron | |
| pattern is concatenated to the readout state, so the readout sees the recent | |
| input identities directly (as standard reservoir readouts do). | |
| 2. **Multi-timescale trace banks** β spike traces at Ο = 0.4 / 0.8 / 0.95, the slow | |
| banks normalized by (1 β Ο) so they do not swamp the fast one; readout state | |
| grows 512 β 2,560 dimensions. | |
| 3. **Structured inputs** β each patch activates 6 full-hash neurons **plus** 6 | |
| byte-trigram hash neurons, so similar patches share input neurons and statistical | |
| strength. | |
| Controlled A/B (identical protocol for both arms: same rebuilt corpus, train | |
| 13.27 MB / test 640 KB, vocabulary 4,096, SNN 200k tokens Γ 2 epochs, AM 50k | |
| contexts, held-out test 20,000 positions, vocabulary coverage 45.24%; bigram | |
| trained on train cols [0:600k] and evaluated on exactly the same positions): | |
| | Held-out test, SNN alone, in-vocab | v2 baseline | v3 | patch bigram @4096 | | |
| |---|---|---|---| | |
| | Test (20k positions) | 0.40% | **2.07%** (Γ5.2) | **4.94%** | | |
| | Trained region (8k positions) | 3.76% | **13.29%** (Γ3.5) | β | | |
| **Honest verdict:** the pre-registered success criterion β the SNN alone beats the | |
| bigram β is **not met** (2.07% vs 4.94%). The gap narrowed from Γ12.4 to Γ2.4, and | |
| the representation diagnosis is confirmed experimentally, but it is not closed. | |
| Free generation is still incoherent in both arms, and at this small budget fusion | |
| *hurts* on held-out (v3 fused 0.13% < SNN alone 2.07%): the AM written on only 50k | |
| contexts drags the score down. Costs: training 189 t/s vs ~530 t/s (2.7Γ slower), | |
| checkpoint 45.9 MB vs 14.6 MB. Full details: [`V3_REPORT.md`](V3_REPORT.md), | |
| raw numbers in `python/artifacts/metrics_v3_ab.json`, A/B checkpoint in | |
| `python/artifacts/spira_model_v3_ab.npz`. | |
| ## Repository layout | |
| ``` | |
| python/ NumPy-only research prototype | |
| spira/ the package (hashing, ept, spiking [hierarchy], hippocampus, | |
| model [teach/replay/calibrate], baselines, train, evaluate | |
| [replay experiment], export_deploy [format v2]) | |
| scripts/ corpus builder, chunked training driver, figure maker, | |
| AM rebuild, probes/diagnostics from the research log | |
| artifacts/ trained checkpoint (72 MB), metrics.json, spike raster, | |
| browser-demo screenshots | |
| README.md full Python quick start & design notes | |
| paper/ LaTeX source + compiled research paper (spira_paper.pdf) | |
| web/ Next.js browser demo β interactive tokenizer visualization, | |
| closed-loop generation, one-shot teach & corrupted recall, | |
| spike-raster playback (TS port of the v2 model incl. the | |
| hierarchical slow area) | |
| ``` | |
| The 29 MB browser-inference artifact (`spira_deploy.json`) is kept once, at | |
| `web/public/spira_deploy.json`, where the demo fetches it. | |
| ## Quick start β web demo | |
| ```bash | |
| cd web | |
| npm install # or: bun install | |
| npm run dev # http://localhost:3000 | |
| ``` | |
| No backend needed: the full model (EPT tokenizer + 512-neuron spiking reservoir + | |
| 256-neuron slow area + associative memory) runs client-side, compatible with the | |
| Python prototype. (Hippocampal replay is Python-only.) | |
| ## Quick start β Python prototype | |
| ```bash | |
| cd python | |
| pip install numpy # only dependency | |
| python3 scripts/build_corpus.py # downloads Gutenberg books + uses local | |
| # CPython stdlib -> data/{train,test,oneshot}.bin | |
| # either evaluate the shipped checkpoint directly (resumable by section)... | |
| python3 -m spira.evaluate --model artifacts/spira_model.npz \ | |
| --train data/train.bin --test data/test.bin \ | |
| --oneshot data/oneshot.bin --out artifacts/metrics.json | |
| # ...or retrain from scratch (~55 min on 2 vCPU, chunked) | |
| python3 scripts/train_chunk.py --start 0 --count 150000 # + more chunks to 600k | |
| python3 scripts/train_chunk.py --start 0 --count 150000 # epoch 2 (+ chunks) | |
| python3 scripts/train_chunk.py --start 450000 --count 150000 --raster | |
| python3 scripts/train_chunk.py --am-only | |
| python3 scripts/train_chunk.py --start 450000 --warmup 150000 | |
| ``` | |
| See [`python/README.md`](python/README.md) for the replay/consolidation snippet and | |
| the full design notes (variance-matched fusion, counterfactual trust gain, exact AM | |
| checkpoint round-trips). | |
| ## Paper | |
| [`paper/spira_paper.pdf`](paper/spira_paper.pdf) β *SPIRA: Sequence Learning Without | |
| Backpropagation or GPU Clusters* (third iteration), preprint, October 2026. LaTeX | |
| sources and all figures are included. | |
| ## License | |
| MIT β see [`LICENSE`](LICENSE). | |
| ## Citation | |
| ```bibtex | |
| @misc{spira2026, | |
| title = {SPIRA: Sequence Learning Without Backpropagation or GPU Clusters}, | |
| author = {SPIRA Project}, | |
| year = {2026}, | |
| month = {10}, | |
| note = {Preprint, third iteration}, | |
| url = {https://huggingface.co/thefinalboss/SPIRA} | |
| } | |
| ``` | |