Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG
NeurIPS 2026
Brain-OF Giant is the largest scale of the Brain-OF family, designed to learn richer representations from fMRI, EEG, and MEG through cross-modal synergy. ARNESS maps signals with varying channel/ROI counts and temporal resolutions to a shared latent space. Sparse MoE captures shared and modality-specific patterns, while DINT attention filters irrelevant context and integrates local and global dependencies. Masked Temporal-Frequency Modeling pretrains the model to reconstruct masked signals in the time and frequency domains. The published Base, Large, Huge, and Giant models were pretrained on 37 public datasets with 5.87 million samples from 32,278 participants.
## Model specifications | Property | Value | | --- | --- | | Total parameters | 9B | | Active parameters per token | 1.2B | | Backbone layers | 48 | | Patch size | 64 time points | | Checkpoint format | FSDP distributed checkpoint directory | ## Usage After [installing the environment](https://github.com/HanningGuo/BrainOF#installation), replace the checkpoint directory below and run the example from the BrainOF repository root. It extracts a **1,024-dimensional embedding** from a randomly generated signal: Keep `.metadata` and all checkpoint shards in that directory. This example uses PyTorch Distributed Checkpoint to load the full encoder into CPU memory, so sufficient RAM is required. ```python import numpy as np import torch import torch.distributed.checkpoint as dcp from datasets import DataTransform from models.transformer import brainof_giant_9B_A1_2B model = brainof_giant_9B_A1_2B(num_classes=0, layer_scale_type="per_channel") checkpoint = {"model": {"encoder": model.state_dict()}} dcp.load( checkpoint, checkpoint_id="/path/to/brainof_giant_9B_A1_2B", no_dist=True, ) model.load_state_dict(checkpoint["model"]["encoder"]) model.eval() # Generate a random signal: 19 channels, 2,000 time points signal = np.random.default_rng(0).standard_normal((19, 2000)).astype(np.float32) transform = DataTransform(signal_size=signal.shape, patch_size=64, max_seq_length=2048) patches, mask, _ = transform(signal) x = torch.tensor(patches, dtype=torch.float32).unsqueeze(0) attn_mask = torch.tensor(mask, dtype=torch.bool).unsqueeze(0) with torch.inference_mode(): embedding = model(x, attn_mask) print(embedding.shape) # torch.Size([1, 1024]) ``` For real data, replace the random signal with a [preprocessed](https://github.com/HanningGuo/BrainOF#data-preparation) NumPy array of shape `(channels_or_ROIs, time_points)`. `DataTransform` normalizes, patches, and pads it. We recommend using at most **2,048 patch tokens** per sample and the token count must not exceed `max_seq_length` (see [Input Size and Tokenization](https://github.com/HanningGuo/BrainOF#input-size-and-tokenization)). ## Model family | Model | Total parameters | Active / token | Layers | Embedding dim. | | :--- | ---: | ---: | ---: | ---: | | [](https://huggingface.co/han-ning/Brain-OF-Base) | 47.5M | 21.5M | 12 | 256 | | [](https://huggingface.co/han-ning/Brain-OF-Large) | 331M | 150M | 24 | 512 | | [](https://huggingface.co/han-ning/Brain-OF-Huge) | 1.7B | 500M | 36 | 768 | |
| **9B** | **1.2B** | **48** | **1024** |
## License
This repository and the model weights are licensed under the [CC BY-NC 4.0](https://huggingface.co/han-ning/Brain-OF-Giant/blob/main/LICENSE).
## Contact
Brain-OF Giant is currently available through limited access only. A public release is tentatively scheduled for the **first half of 2027**.
To request access before the public release, please email [Hanning Guo](mailto:hanning.guo@rwth-aachen.de) from your **official institutional email address** (e.g., `.edu` or an equivalent university or research institute domain), including a brief description of your **research objectives**.
## Citation
```bibtex
@inproceedings{guo2026brainof,
title = {{Brain-OF}: An Omnifunctional Foundation Model for {fMRI}, {EEG} and {MEG}},
author = {Guo, Hanning and Bi, Hanwen and Abdellatif, Farah and Galbenus, Andrei and Shah, Jon. N. and Morrison, Abigail and Dammers, Jürgen},
booktitle = {Advances in Neural Information Processing Systems},
year = {2026},
url = {https://arxiv.org/abs/2602.23410}
}
```