Card: list the training data (datasets:, base_model:) and add a Training data section
ef0e61b verified | license: mit | |
| library_name: braindecode | |
| tags: | |
| - braindecode | |
| - pytorch | |
| - safetensors | |
| - meg | |
| - meggpt | |
| - source-space | |
| - foundation-model | |
| datasets: | |
| - CamCAN | |
| # MEG-GPT (pretrained, Cam-CAN) | |
| Weights of MEG-GPT (Huang et al., Imaging Neuroscience 2026) and of the encoder of its | |
| tokenizer, EphysTokenizer, for | |
| [`braindecode.models.MEGGPT`](https://braindecode.org/stable/generated/braindecode.models.MEGGPT.html), | |
| converted from the authors' release to braindecode's parameter names. MEG-GPT is a causal | |
| time-attention GPT trained on tokenised source-space MEG (52 parcels at 250 Hz) from the | |
| Cam-CAN resting-state data. The class is added by | |
| [braindecode#1555](https://github.com/braindecode/braindecode/pull/1555) (braindecode 2.1). | |
| ```python | |
| from braindecode.models import MEGGPT | |
| model = MEGGPT.from_pretrained("braindecode/meggpt-pretrained", n_outputs=4) | |
| features = model(x, return_features=True)["features"] | |
| ``` | |
| The classification head (`final_layer`, `n_outputs=2` in `config.json`) is not part of | |
| the release: it is PyTorch's default init under `torch.manual_seed(0)`. Passing | |
| `n_outputs` to `from_pretrained` replaces it. Fine-tune or linear-probe before use. | |
| <!-- braindecode:training-data --> | |
| ## Training data | |
| The data that shaped these weights. The ids are the `datasets:` field of this card, | |
| which braindecode reads into `model.card_data.datasets` when you call | |
| `from_pretrained` (MOABB class names for MOABB datasets, OpenNeuro / NEMAR | |
| accessions, Hub ids otherwise). `nm`/`on` ids are datasets on [NEMAR](https://nemar.org) | |
| (`on` = NEMAR's copy of the OpenNeuro dataset with the same number). Rows marked | |
| `private:` are data the authors did not release; they are not in `datasets:`. | |
| | `datasets:` id | Dataset | Stage | Modality | | |
| |---|---|---|---| | |
| | `CamCAN` | [Cambridge Centre for Ageing and Neuroscience (Cam-CAN) MEG data, Shafto et al. 2014 / Taylor et al. 2017](https://cam-can.mrc-cbu.cam.ac.uk/dataset/) | pretraining, pretraining | MEG | | |
| - EphysTokenizer: paper trains it on the first 50 of 612 Cam-CAN subjects (eyes-closed rest, 52-parcel source space) | |
| - MEG-GPT next-token pretraining on tokenised Cam-CAN resting-state; released config: 621 Cam-CAN train sessions, 90/10 per-subject window split | |
| - Sources: arXiv:2510.18080v1 Sec. 2.5 Datasets; MEG-GPT@78ff9439 models/meg-gpt/config.yaml; MEG-GPT@78ff9439 examples/camcan/01_train_tokenizer.py. | |
| ## Intended input | |
| `x` is `(batch, 52, n_times)` float, source-space MEG: the 52 parcels of the released | |
| parcellation (Glasser 52) in its order, sign-flip aligned and z-scored per parcel over the | |
| recording (`tokenize_session(standardize=True)` in the reference; Section 2.5.3 of the | |
| paper), at 250 Hz. The model does not normalise its input and does not check `sfreq`. | |
| The window is cut into 100-sample segments; set `on_non_divisible` for windows that are | |
| not a multiple of 100. | |
| ## Source | |
| - Paper: R. Huang, S. Cho, C. Gohil, O. Parker Jones, M. Woolrich. *MEG-GPT: A | |
| transformer-based foundation model for magnetoencephalography data.* Imaging | |
| Neuroscience 4, IMAG.a.1301 (2026). https://doi.org/10.1162/imag.a.1301 | |
| ([arXiv:2510.18080](https://arxiv.org/abs/2510.18080)) | |
| - Code and weights: [OHBA-analysis/MEG-GPT @ `78ff9439febaeb8e838fd670ec589578e635fd0c`](https://github.com/OHBA-analysis/MEG-GPT/tree/78ff9439febaeb8e838fd670ec589578e635fd0c) | |
| (MIT), directory `models/`; tokenizer code: | |
| [OHBA-analysis/EphysTokenizer @ `44cac87caee7ae2521d2b88e7bc71c510b10a023`](https://github.com/OHBA-analysis/EphysTokenizer/tree/44cac87caee7ae2521d2b88e7bc71c510b10a023) (MIT). | |
| | original file (MEG-GPT @ `78ff9439`) | sha256 | | |
| |---|---| | |
| | `models/meg-gpt/model_state.pt` | `10755496172a8c921cd9505884d00c6370bed211e99863471e790f1bbea2eeb9` | | |
| | `models/meg-gpt/config.yaml` | `4434140a4aff5f19124a257cf0425fb9b564b05060c85317f31a8f935ac91069` | | |
| | `models/tokenizer/model_state.pt` | `1d766a574942122ed28d2d08dcd2e33dc53b2b8a30628c100d786516152cf2a7` | | |
| | `models/tokenizer/vocab.pkl` | `ea0273f3efc6f349ae231b00332a511f303e13aadd69e18c3c6b93ea65838959` | | |
| | `models/tokenizer/config.yaml` | `f312ab9a56d3a0f2937afd6a9235e6af87887f808be054103634bcd3367b6241` | | |
| ## Conversion | |
| `convert.py` (in this repository) was run with braindecode at | |
| `bruAristimunha/braindecode@2f253bc1d71b9cba66ad40e90907b7ac12cec343` (branch `feat/meg-meggpt`, PR #1555): | |
| ``` | |
| python convert.py <MEG-GPT>/models <out_dir> | |
| ``` | |
| It renames the keys of the two state dicts, stores the vocabulary | |
| (`vocab.pkl["label_map"]`) as the `tokenizer.vocab` buffer, loads them into | |
| `MEGGPT(n_chans=52, n_outputs=2, sfreq=250)` and calls `save_pretrained`. Tensor values are | |
| copied unchanged. | |
| Dropped: the Cam-CAN session embedding (`input_embedding_layer.extra_embeds.0.*`, 621 | |
| sessions; specific to the pretraining data, and the paper discards it for downstream use, | |
| Section 2.5.4), the next-token `prediction_head`, the attention-mask buffers (rebuilt from | |
| the configuration, checked equal), the tokenizer's signal decoder (`token_basis_layer`) and | |
| its annealing temperature. | |
| | file | sha256 | | |
| |---|---| | |
| | `model.safetensors` | `e009d3f4d81eb6d194f04a72f59cca19140600266ab7d464691f77d388ccd821` | | |
| | `pytorch_model.bin` | `3e5f7116d3d1719fc74925d2f387bb4b089c16c895d8d4af606c406570b175b1` | | |
| | `config.json` | `f022492a654181df38c66f9230b681fa20f521039de83d6c119dfead4f61bc5f` | | |
| | `convert.py` | `792b681d01dd3e3e2a16dbd51f67cfc80da3387094714ed9f9b39cb28e657172` | | |
| ## Verification | |
| `MEGGPT.from_pretrained(<this repo>, strict=True)` against the original code (CPU, float32, | |
| eval mode, same inputs): token ids identical; decoder hidden states, time-averaged | |
| features, logits through the released prediction head and the gradients of 99 tensors in | |
| one backward pass agree with max-abs difference 0.0. | |
| ## License | |
| MIT, as the original release: [MEG-GPT licence](https://github.com/OHBA-analysis/MEG-GPT/blob/main/LICENSE) | |
| (Copyright (c) 2026 OHBA Analysis Group). The weights ship in `models/` of that | |
| MIT-licensed repository, with no separate weight licence. Converted and re-hosted with | |
| attribution to the authors; for corrections or removal, open an issue at | |
| https://github.com/braindecode/braindecode/issues. | |
| ## Citation | |
| ```bibtex | |
| @article{huang2026meggpt, | |
| title = {{MEG-GPT}: A transformer-based foundation model for magnetoencephalography data}, | |
| author = {Huang, Rukuang and Cho, SungJun and Gohil, Chetan and Parker Jones, Oiwi and Woolrich, Mark}, | |
| journal = {Imaging Neuroscience}, | |
| volume = {4}, | |
| pages = {IMAG.a.1301}, | |
| year = {2026}, | |
| doi = {10.1162/imag.a.1301} | |
| } | |
| @article{aristimunha2025braindecode, | |
| title = {Braindecode: a deep learning library for raw electrophysiological data}, | |
| author = {Aristimunha, Bruno and others}, | |
| journal = {Zenodo}, | |
| year = {2025}, | |
| doi = {10.5281/zenodo.17699192}, | |
| } | |
| ``` | |