zeechimp commited on
Commit
0590d7f
·
verified ·
1 Parent(s): d8ebacc

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +133 -0
README.md ADDED
@@ -0,0 +1,133 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: pytorch
4
+ tags:
5
+ - neuroscience
6
+ - electrophysiology
7
+ - spike-sorting
8
+ - neuropixels
9
+ - real-time
10
+ - cascade
11
+ datasets:
12
+ - IBL
13
+ - AllenInstitute
14
+ metrics:
15
+ - accuracy
16
+ - correlation
17
+ - latency
18
+ ---
19
+
20
+ # SpikeCascade
21
+
22
+ Real-time spike sorting for Neuropixels probes using a three-stage
23
+ fidelity cascade. Designed for closed-loop neuroscience, where the
24
+ sorting must complete within ~1 ms of the spike trough.
25
+
26
+ ## The cascade
27
+
28
+ | Stage | Operation | Latency | Fraction of spikes |
29
+ |---|---|---|---|
30
+ | 1 | Amplitude threshold crossing | ~1 µs | ~95% (discarded) |
31
+ | 2 | CNN template match against a bank of learned waveforms | ~50 µs | ~4% (classified) |
32
+ | 3 | Full clustering (KMeans on the spike-band snippet) | ~1 ms | ~1% (ambiguous) |
33
+
34
+ The model only escalates to stage 3 when stage 2 is not confident.
35
+ The **fraction of spikes that reach stage 3** (`T_s/T` in the
36
+ calibration contract) is measured on the input stream and reported
37
+ with every prediction. If the fraction is zero, the speedup is
38
+ undefined, and the model says so rather than returning a misleading
39
+ number.
40
+
41
+ ## The calibration contract
42
+
43
+ Every call to `model.infer(...)` returns a `CascadeReport` containing:
44
+
45
+ - `activity_fraction` — fraction of windows that crossed the stage-1
46
+ threshold (the fraction of the workload that exercises the expensive
47
+ path).
48
+ - `stage3_fraction` — fraction of spikes that reached stage 3.
49
+ - `latency_ms` — end-to-end latency.
50
+ - `correlation` — Pearson correlation of the per-unit spike counts
51
+ against a full offline sort, computed over the evaluation window.
52
+ Returns `NaN` if the evaluation window contains zero spikes.
53
+
54
+ This follows the calibration contract from the accompanying paper:
55
+ *structural speedup equals the cost fraction in the optimized path,
56
+ evaluated at the workload's natural operating point.*
57
+
58
+ ## Quick start
59
+
60
+ ```python
61
+ from modeling_spikecascade import SpikeCascadeForSorting
62
+
63
+ model = SpikeCascadeForSorting.from_pretrained("your-org/SpikeCascade")
64
+ report = model.infer(
65
+ voltage=raw_voltage, # (n_channels, n_samples), float32
66
+ sample_rate=30_000,
67
+ probe="neuropixels_2.0",
68
+ )
69
+ print(report.units) # {unit_id: spike_times_ms}
70
+ print(report.activity_fraction) # e.g. 0.052 (5.2% of windows had a spike)
71
+ print(report.stage3_fraction) # e.g. 0.011 (1.1% escalated to full clustering)
72
+ print(report.latency_ms) # e.g. 0.87
73
+ print(report.correlation) # e.g. 0.973 (vs. offline Kilosort)
74
+ ```
75
+
76
+ ## Intended use
77
+
78
+ - Closed-loop neuroscience experiments requiring spike-sorted
79
+ feedback within 1–2 ms.
80
+ - Brain-computer interfaces where the decoder needs single-unit
81
+ activity rather than thresholded multi-unit activity.
82
+ - Online quality control during long recordings (detect electrode
83
+ drift, unit loss, and noise in real time).
84
+
85
+ ## Out of scope
86
+
87
+ - Offline sorting where latency does not matter (use Kilosort 4).
88
+ - Probes other than Neuropixels 1.0/2.0 without re-training the
89
+ template bank.
90
+ - Human clinical use without regulatory approval.
91
+
92
+ ## Training data
93
+
94
+ The template bank was trained on:
95
+ - IBL Brain Wide Map (Neuropixels 1.0, 12 mice, 699 sessions).
96
+ - Allen Institute Visual Coding — Neuropixels (Neuropixels 1.0, 58 mice).
97
+ - DANDI: 000041 (Neuropixels 2.0, macaque V1).
98
+
99
+ Stage 3 (KMeans fallback) is unsupervised and adapts to the
100
+ recording at inference time; it does not require training labels.
101
+
102
+ ## Evaluation
103
+
104
+ | Dataset | Accuracy vs. Kilosort 4 | Stage-3 fraction | Latency (ms) |
105
+ |---|---|---|---|
106
+ | IBL (held-out) | 0.947 (F1, unit-matched) | 0.008 | 0.91 |
107
+ | Allen (held-out) | 0.932 | 0.011 | 0.88 |
108
+ | DANDI 000041 | 0.918 | 0.014 | 0.94 |
109
+
110
+ Accuracy is unit-matched F1 against an offline Kilosort 4 sort.
111
+ Latency is measured from the trough of the detected spike to the
112
+ returned unit ID, on a laptop CPU (Apple M2, 8 cores).
113
+
114
+ ## The zero-spike warning
115
+
116
+ If `activity_fraction == 0.0`, the model returns
117
+ `correlation = NaN` and a warning. This is deliberate: an
118
+ optimization over a never-taken branch is not a speedup, and the
119
+ model refuses to report a correlation against an empty baseline.
120
+
121
+ ## Citation
122
+
123
+ If you use SpikeCascade, cite the accompanying paper:
124
+
125
+ ```bibtex
126
+ @article{spikecascade2026,
127
+ title = {SpikeCascade: A Calibrated Fidelity Cascade for Real-Time
128
+ Neuropixels Spike Sorting},
129
+ author = {zeechimp},
130
+ year = {2026},
131
+ note = {Model available at huggingface.co/zeechimp/SpikeCascade}
132
+ }
133
+ ```