--- license: mit library_name: braindecode tags: - braindecode - pytorch - safetensors - meg - bbl - self-supervised - foundation-model datasets: - CamCAN --- # BBL (pretrained, CamCAN) Weights of the MEG encoder of *The Brain's Bitter Lesson* (Jayalath et al., ICML 2025), pretrained on CamCAN with the band, phase and amplitude pretext tasks, for [`braindecode.models.BBL`](https://braindecode.org/stable/generated/braindecode.models.BBL.html), converted from the authors' checkpoint to braindecode's parameter names. The class is added by [braindecode#1553](https://github.com/braindecode/braindecode/pull/1553) (braindecode 2.1). ```python from braindecode.models import BBL model = BBL.from_pretrained("braindecode/bbl-pretrained", n_outputs=2) features = model(x, return_features=True)["features"] # (batch, 528) ``` The linear head (`final_layer`, `n_outputs=2` in `config.json`) is not part of the release: it is PyTorch's default init under `torch.manual_seed(0)`. Passing `n_outputs` to `from_pretrained` replaces it. Fine-tune or linear-probe before use. ## Training data The data that shaped these weights. The ids are the `datasets:` field of this card, which braindecode reads into `model.card_data.datasets` when you call `from_pretrained` (MOABB class names for MOABB datasets, OpenNeuro / NEMAR accessions, Hub ids otherwise). `nm`/`on` ids are datasets on [NEMAR](https://nemar.org) (`on` = NEMAR's copy of the OpenNeuro dataset with the same number). Rows marked `private:` are data the authors did not release; they are not in `datasets:`. | `datasets:` id | Dataset | Stage | Modality | |---|---|---|---| | `CamCAN` | [Cambridge Centre for Ageing and Neuroscience (Cam-CAN) MEG data, Shafto et al. 2014 / Taylor et al. 2017](https://cam-can.mrc-cbu.cam.ac.uk/dataset/) | pretraining | MEG | - self-supervised band/phase/amplitude pretext tasks; 583 Cam-CAN subject rows trained (train split; rows with non-zero Adam state) - Sources: https://github.com/neural-processing-lab/the-brains-bitter-lesson/blob/3965a2be11677d9c2b9cecb88109a0af6bf19d1c/README.md; checkpoint hyper_parameters (inspected 2026-11-06, meg-ports-20261106/bbl/weights/CHECKPOINT.md and evidence/parity.log); arXiv:2406.04328 Sec. 4 Datasets. ## Intended input - MEG, `(batch, n_chans, 125)`: CamCAN's 306 Elekta `MEG*` channels in FIF order (the projector is indexed by position; inputs with fewer channels are zero-padded to 306, as the reference collator does). `dataset_index` defaults to slot 0 (CamCAN). - Preprocessed as in the authors' `data/preprocess.py` and `data/utils.py`: band-pass 0.1-125 Hz with 50 and 100 Hz notches, resampled to 250 Hz, 0.5 s windows (125 samples); each channel scaled by the median and interquartile range fitted on its recording, clipped to `[-5, 5]`, and the mean of the first 0.5 s subtracted. `forward` only does the zero-padding. - `subject_index` row `d * 1024 + s` for subject `s` of dataset slot `d`. Only CamCAN subject rows are trained; the `trained_subjects` buffer flags those 583 rows and drives the mean-subject fallback used when no `subject_index` is given. ## Source - Paper: D. Jayalath, G. Landau, B. Shillingford, M. Woolrich, O. Parker Jones. *The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning.* ICML 2025. [arXiv:2406.04328](https://arxiv.org/abs/2406.04328) - Code: [neural-processing-lab/the-brains-bitter-lesson @ `3965a2be11677d9c2b9cecb88109a0af6bf19d1c`](https://github.com/neural-processing-lab/the-brains-bitter-lesson/tree/3965a2be11677d9c2b9cecb88109a0af6bf19d1c) (MIT) - Weights: the checkpoint "pretrained with CamCAN" linked from the README of that repository ("Pre-trained checkpoints"), hosted on Google Drive (file id `1cYdKDYoIupjoMfq5cEWgB-W74XhztWEw`), downloaded 2026-11-06: 509,226,678 bytes, sha256 `94a35ca9bb8ca00941ce355d1b5cd97e4984b689209317a4eb18a3560a8e940b`. A PyTorch Lightning 2.5.0.post0 `Pretrainer` checkpoint (original name `camcan_preproc_full_cont-best-epoch=02-val_loss=1.26.ckpt`; epoch 2, global step 60507; `input_dim=306, model_dim=512, subject_embedding_dim=16, projector_dim=1024`, tasks band, phase, amp). ## Conversion `convert.py` (in this repository) was run with braindecode at `bruAristimunha/braindecode@388ac6cb96361c2df3d5b530172ed6662b9e6a65` (branch `feat/meg-bbl`, PR #1553): ``` python convert.py bbl.ckpt ``` It keeps the `backbone.*` tensors, renames them to braindecode's keys (the old-style `weight_g`/`weight_v` weight norm maps to `parametrizations.weight.original0/1`, which computes the same `torch._weight_norm`), fills the `trained_subjects` buffer with the subject rows the optimizer updated (non-zero Adam second moment; the original keeps that set in a Python attribute that is not saved), loads everything into `BBL(n_chans=306, n_outputs=2, n_times=125, sfreq=250.0)` and calls `save_pretrained`. Tensor values are copied unchanged. Dropped (training only): the SSL projector, the band/amplitude/phase pretext classifiers, and the Lightning/optimizer state. | file | sha256 | |---|---| | `model.safetensors` | `3f5022d2564762917d5b9d0df35060b8f0dd4f09128cc9cb076d5fd6aed193c3` | | `pytorch_model.bin` | `7d0da29ad7d4404917bfa70cf6b4b3c692dcbc10501eec9ec100fb5c785679bb` | | `config.json` | `71ba8c6efac12d7595f0723522c090400017a9b41255a8f620847a1313699f7f` | | `convert.py` | `c3388e146017e55aa439dfba995ea8e5a06efe3e4d77acd3f97b98112b950581` | ## Verification `BBL.from_pretrained(, strict=True)` (loaded tensors equal `convert.py` applied to the original checkpoint) against the original `Backbone` with the same weights and inputs, CPU float32, eval mode, 4 windows of 0.5 s: features, logits, loss, input gradients and the gradients of all 55 backbone parameters and of the head are equal (max-abs difference 0.0) with 306 sensors; with 204 sensors (zero-padded to 306) the features are equal. The mean-subject fallback (batch 1) differs by 3.7e-8 in the 16 subject-embedding features (the mean is taken in another order); the encoder part is equal. ## License and provenance `license: mit` follows the repository that distributes the checkpoint. The code of [the-brains-bitter-lesson](https://github.com/neural-processing-lab/the-brains-bitter-lesson) is MIT ([licence](https://github.com/neural-processing-lab/the-brains-bitter-lesson/blob/main/LICENSE), Copyright (c) 2025 Dulhan Jayalath). The weights are distributed by the authors through that repository's README (Google Drive link) without a separate weight licence. They are re-hosted here unmodified in converted form (renamed keys, training-only parts dropped), with attribution to the authors. Authors or rights holders who want this copy changed or removed: please open an issue at https://github.com/braindecode/braindecode/issues and it will be taken down. ## Citation ```bibtex @inproceedings{jayalath2025bbl, title = {{The Brain's Bitter Lesson: Scaling Speech Decoding With Self-Supervised Learning}}, author = {Jayalath, Dulhan and Landau, Gilad and Shillingford, Brendan and Woolrich, Mark and Parker Jones, Oiwi}, booktitle = {Forty-second International Conference on Machine Learning (ICML)}, year = {2025}, organization = {PMLR} } @article{aristimunha2025braindecode, title = {Braindecode: a deep learning library for raw electrophysiological data}, author = {Aristimunha, Bruno and others}, journal = {Zenodo}, year = {2025}, doi = {10.5281/zenodo.17699192}, } ```