---
license: mit
library_name: braindecode
tags:
- braindecode
- pytorch
- safetensors
- meg
- meggpt
- source-space
- foundation-model
datasets:
- CamCAN
---

# MEG-GPT (pretrained, Cam-CAN)

Weights of MEG-GPT (Huang et al., Imaging Neuroscience 2026) and of the encoder of its
tokenizer, EphysTokenizer, for
[`braindecode.models.MEGGPT`](https://braindecode.org/stable/generated/braindecode.models.MEGGPT.html),
converted from the authors' release to braindecode's parameter names. MEG-GPT is a causal
time-attention GPT trained on tokenised source-space MEG (52 parcels at 250 Hz) from the
Cam-CAN resting-state data. The class is added by
[braindecode#1555](https://github.com/braindecode/braindecode/pull/1555) (braindecode 2.1).

```python
from braindecode.models import MEGGPT

model = MEGGPT.from_pretrained("braindecode/meggpt-pretrained", n_outputs=4)
features = model(x, return_features=True)["features"]
```

The classification head (`final_layer`, `n_outputs=2` in `config.json`) is not part of
the release: it is PyTorch's default init under `torch.manual_seed(0)`. Passing
`n_outputs` to `from_pretrained` replaces it. Fine-tune or linear-probe before use.

<!-- braindecode:training-data -->
## Training data

The data that shaped these weights. The ids are the `datasets:` field of this card,
which braindecode reads into `model.card_data.datasets` when you call
`from_pretrained` (MOABB class names for MOABB datasets, OpenNeuro / NEMAR
accessions, Hub ids otherwise). `nm`/`on` ids are datasets on [NEMAR](https://nemar.org)
(`on` = NEMAR's copy of the OpenNeuro dataset with the same number). Rows marked
`private:` are data the authors did not release; they are not in `datasets:`.

| `datasets:` id | Dataset | Stage | Modality |
|---|---|---|---|
| `CamCAN` | [Cambridge Centre for Ageing and Neuroscience (Cam-CAN) MEG data, Shafto et al. 2014 / Taylor et al. 2017](https://cam-can.mrc-cbu.cam.ac.uk/dataset/) | pretraining, pretraining | MEG |

- EphysTokenizer: paper trains it on the first 50 of 612 Cam-CAN subjects (eyes-closed rest, 52-parcel source space)
- MEG-GPT next-token pretraining on tokenised Cam-CAN resting-state; released config: 621 Cam-CAN train sessions, 90/10 per-subject window split
- Sources: arXiv:2510.18080v1 Sec. 2.5 Datasets; MEG-GPT@78ff9439 models/meg-gpt/config.yaml; MEG-GPT@78ff9439 examples/camcan/01_train_tokenizer.py.


## Intended input

`x` is `(batch, 52, n_times)` float, source-space MEG: the 52 parcels of the released
parcellation (Glasser 52) in its order, sign-flip aligned and z-scored per parcel over the
recording (`tokenize_session(standardize=True)` in the reference; Section 2.5.3 of the
paper), at 250 Hz. The model does not normalise its input and does not check `sfreq`.
The window is cut into 100-sample segments; set `on_non_divisible` for windows that are
not a multiple of 100.

## Source

- Paper: R. Huang, S. Cho, C. Gohil, O. Parker Jones, M. Woolrich. *MEG-GPT: A
  transformer-based foundation model for magnetoencephalography data.* Imaging
  Neuroscience 4, IMAG.a.1301 (2026). https://doi.org/10.1162/imag.a.1301
  ([arXiv:2510.18080](https://arxiv.org/abs/2510.18080))
- Code and weights: [OHBA-analysis/MEG-GPT @ `78ff9439febaeb8e838fd670ec589578e635fd0c`](https://github.com/OHBA-analysis/MEG-GPT/tree/78ff9439febaeb8e838fd670ec589578e635fd0c)
  (MIT), directory `models/`; tokenizer code:
  [OHBA-analysis/EphysTokenizer @ `44cac87caee7ae2521d2b88e7bc71c510b10a023`](https://github.com/OHBA-analysis/EphysTokenizer/tree/44cac87caee7ae2521d2b88e7bc71c510b10a023) (MIT).

| original file (MEG-GPT @ `78ff9439`) | sha256 |
|---|---|
| `models/meg-gpt/model_state.pt` | `10755496172a8c921cd9505884d00c6370bed211e99863471e790f1bbea2eeb9` |
| `models/meg-gpt/config.yaml` | `4434140a4aff5f19124a257cf0425fb9b564b05060c85317f31a8f935ac91069` |
| `models/tokenizer/model_state.pt` | `1d766a574942122ed28d2d08dcd2e33dc53b2b8a30628c100d786516152cf2a7` |
| `models/tokenizer/vocab.pkl` | `ea0273f3efc6f349ae231b00332a511f303e13aadd69e18c3c6b93ea65838959` |
| `models/tokenizer/config.yaml` | `f312ab9a56d3a0f2937afd6a9235e6af87887f808be054103634bcd3367b6241` |

## Conversion

`convert.py` (in this repository) was run with braindecode at
`bruAristimunha/braindecode@2f253bc1d71b9cba66ad40e90907b7ac12cec343` (branch `feat/meg-meggpt`, PR #1555):

```
python convert.py <MEG-GPT>/models <out_dir>
```

It renames the keys of the two state dicts, stores the vocabulary
(`vocab.pkl["label_map"]`) as the `tokenizer.vocab` buffer, loads them into
`MEGGPT(n_chans=52, n_outputs=2, sfreq=250)` and calls `save_pretrained`. Tensor values are
copied unchanged.

Dropped: the Cam-CAN session embedding (`input_embedding_layer.extra_embeds.0.*`, 621
sessions; specific to the pretraining data, and the paper discards it for downstream use,
Section 2.5.4), the next-token `prediction_head`, the attention-mask buffers (rebuilt from
the configuration, checked equal), the tokenizer's signal decoder (`token_basis_layer`) and
its annealing temperature.

| file | sha256 |
|---|---|
| `model.safetensors` | `e009d3f4d81eb6d194f04a72f59cca19140600266ab7d464691f77d388ccd821` |
| `pytorch_model.bin` | `3e5f7116d3d1719fc74925d2f387bb4b089c16c895d8d4af606c406570b175b1` |
| `config.json` | `f022492a654181df38c66f9230b681fa20f521039de83d6c119dfead4f61bc5f` |
| `convert.py` | `792b681d01dd3e3e2a16dbd51f67cfc80da3387094714ed9f9b39cb28e657172` |

## Verification

`MEGGPT.from_pretrained(<this repo>, strict=True)` against the original code (CPU, float32,
eval mode, same inputs): token ids identical; decoder hidden states, time-averaged
features, logits through the released prediction head and the gradients of 99 tensors in
one backward pass agree with max-abs difference 0.0.

## License

MIT, as the original release: [MEG-GPT licence](https://github.com/OHBA-analysis/MEG-GPT/blob/main/LICENSE)
(Copyright (c) 2026 OHBA Analysis Group). The weights ship in `models/` of that
MIT-licensed repository, with no separate weight licence. Converted and re-hosted with
attribution to the authors; for corrections or removal, open an issue at
https://github.com/braindecode/braindecode/issues.

## Citation

```bibtex
@article{huang2026meggpt,
  title   = {{MEG-GPT}: A transformer-based foundation model for magnetoencephalography data},
  author  = {Huang, Rukuang and Cho, SungJun and Gohil, Chetan and Parker Jones, Oiwi and Woolrich, Mark},
  journal = {Imaging Neuroscience},
  volume  = {4},
  pages   = {IMAG.a.1301},
  year    = {2026},
  doi     = {10.1162/imag.a.1301}
}

@article{aristimunha2025braindecode,
  title   = {Braindecode: a deep learning library for raw electrophysiological data},
  author  = {Aristimunha, Bruno and others},
  journal = {Zenodo},
  year    = {2025},
  doi     = {10.5281/zenodo.17699192},
}
```