Add BrainBERT 'large' pretrained weights (ported from czlwang/BrainBERT)
Browse files- README.md +92 -0
- config.json +19 -0
- model.safetensors +3 -0
- pytorch_model.bin +3 -0
README.md
ADDED
|
@@ -0,0 +1,92 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: braindecode
|
| 3 |
+
license: unknown
|
| 4 |
+
tags:
|
| 5 |
+
- BrainBERT
|
| 6 |
+
- braindecode
|
| 7 |
+
- ieeg
|
| 8 |
+
- seeg
|
| 9 |
+
- foundation-model
|
| 10 |
+
- model_hub_mixin
|
| 11 |
+
- pytorch_model_hub_mixin
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# BrainBERT — Foundation Model for Intracranial Neural Signals
|
| 15 |
+
|
| 16 |
+
Pretrained weights for [`braindecode.models.BrainBERT`](https://braindecode.org/stable/generated/braindecode.models.BrainBERT.html),
|
| 17 |
+
a faithful braindecode port of **BrainBERT** (Wang et al., ICLR 2023), a
|
| 18 |
+
self-supervised foundation model for intracranial (sEEG/iEEG) recordings.
|
| 19 |
+
|
| 20 |
+
- Paper: [BrainBERT: Self-supervised representation learning for intracranial recordings](https://arxiv.org/abs/2302.14367) (ICLR 2023)
|
| 21 |
+
- Original code & weights: [czlwang/BrainBERT](https://github.com/czlwang/BrainBERT)
|
| 22 |
+
- braindecode docs: https://braindecode.org
|
| 23 |
+
|
| 24 |
+
## Provenance & license
|
| 25 |
+
|
| 26 |
+
These weights are the **official pretrained "large" checkpoint** (`stft`
|
| 27 |
+
variant) released by the original authors. The Transformer encoder and input
|
| 28 |
+
encoding are mapped **1:1** into the braindecode `BrainBERT` module (the masked
|
| 29 |
+
spectrogram-reconstruction head, used only for the self-supervised pretraining
|
| 30 |
+
objective, is kept for weight parity; the classification head is a
|
| 31 |
+
braindecode-native addition, randomly initialized).
|
| 32 |
+
|
| 33 |
+
The upstream repository ships **no explicit license file**, so this repository is
|
| 34 |
+
labelled `unknown`: the original authors retain all rights, and these weights are
|
| 35 |
+
re-hosted for convenience to make `from_pretrained` work out of the box. If you
|
| 36 |
+
use them, please **cite the original BrainBERT paper** and refer to the authors'
|
| 37 |
+
repository for terms of use.
|
| 38 |
+
|
| 39 |
+
## Model configuration
|
| 40 |
+
|
| 41 |
+
This checkpoint uses the released "large" configuration (~43M parameters):
|
| 42 |
+
|
| 43 |
+
| | |
|
| 44 |
+
|---|---|
|
| 45 |
+
| `hidden_dim` | 768 |
|
| 46 |
+
| `ffn_dim` | 3072 |
|
| 47 |
+
| `n_layers` | 6 |
|
| 48 |
+
| `n_heads` | 12 |
|
| 49 |
+
| `freq_cutoff` (input_dim) | 40 |
|
| 50 |
+
| `nperseg` | 400 |
|
| 51 |
+
| `noverlap` | 350 |
|
| 52 |
+
| `activation` | GELU |
|
| 53 |
+
| `sfreq` | 2048 Hz |
|
| 54 |
+
|
| 55 |
+
The signal is expected at **2048 Hz** (Laplacian-re-referenced, as in the paper).
|
| 56 |
+
The short-time Fourier transform front-end is computed **inside** the model, so it
|
| 57 |
+
consumes raw `(batch, n_chans, n_times)` signal directly (upstream fed a
|
| 58 |
+
pre-computed spectrogram). Frames are pooled over time and channels, so you may
|
| 59 |
+
freely change `n_chans` and `n_outputs` (the classification head is task-specific
|
| 60 |
+
and randomly initialized — fine-tune it on your downstream task). `n_times` only
|
| 61 |
+
needs to be long enough to yield at least one spectrogram frame.
|
| 62 |
+
|
| 63 |
+
## Usage
|
| 64 |
+
|
| 65 |
+
```python
|
| 66 |
+
from braindecode.models import BrainBERT
|
| 67 |
+
|
| 68 |
+
# The encoder loads the pretrained weights; the classification head is
|
| 69 |
+
# (re)initialized for your task via n_outputs.
|
| 70 |
+
model = BrainBERT.from_pretrained("braindecode/brainbert-pretrained", n_outputs=2)
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
## Fidelity
|
| 74 |
+
|
| 75 |
+
The ported encoder reproduces the upstream reference
|
| 76 |
+
(`MaskedTFModel`, `intermediate_rep=True`) to within **~6e-5** max absolute
|
| 77 |
+
difference on the authors' demo signal — the residual is cross-device float32
|
| 78 |
+
rounding in the fixed sinusoidal positional table (computed on GPU at training
|
| 79 |
+
time, regenerated on CPU here), which grows only at positions far beyond any
|
| 80 |
+
realistic sequence length.
|
| 81 |
+
|
| 82 |
+
## Citation
|
| 83 |
+
|
| 84 |
+
```bibtex
|
| 85 |
+
@inproceedings{wang2023brainbert,
|
| 86 |
+
title = {{BrainBERT}: Self-supervised representation learning for intracranial recordings},
|
| 87 |
+
author = {Wang, Christopher and Subramaniam, Vishwaas and Yaari, Adam Uri and
|
| 88 |
+
Kreiman, Gabriel and Katz, Boris and Cases, Ignacio and Barbu, Andrei},
|
| 89 |
+
booktitle = {International Conference on Learning Representations (ICLR)},
|
| 90 |
+
year = {2023}
|
| 91 |
+
}
|
| 92 |
+
```
|
config.json
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"hidden_dim": 768,
|
| 3 |
+
"ffn_dim": 3072,
|
| 4 |
+
"n_layers": 6,
|
| 5 |
+
"n_heads": 12,
|
| 6 |
+
"nperseg": 400,
|
| 7 |
+
"noverlap": 350,
|
| 8 |
+
"freq_cutoff": 40,
|
| 9 |
+
"stft_clip": 5,
|
| 10 |
+
"activation": "torch.nn.modules.activation.GELU",
|
| 11 |
+
"drop_prob": 0.1,
|
| 12 |
+
"n_outputs": 2,
|
| 13 |
+
"n_chans": 1,
|
| 14 |
+
"chs_info": null,
|
| 15 |
+
"n_times": 2048,
|
| 16 |
+
"input_window_seconds": null,
|
| 17 |
+
"sfreq": 2048.0,
|
| 18 |
+
"braindecode_version": "1.7.0dev0"
|
| 19 |
+
}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a20e81cfb90192b58dd3002c5dac960a0193c727f026e6fd27bb872363edb23c
|
| 3 |
+
size 172753928
|
pytorch_model.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d953e90c8ab27afb3f44a9bce1ecd52705012dc948702227cdfeca6ac4c4da60
|
| 3 |
+
size 172776905
|