license: mit
tags:
- fmri
- neuroscience
- tokenizer
- neuroimaging
HBAR: Hierarchical Brain Activity Representation
Pretrained tokenizers from A Hierarchical Tokenization Framework for Voxel-Level fMRI Representation Learning (Karanpartap Singh, Yue Zhao, Mohammad H. Abbasi, Ehsan Adeli).
Paper (arXiv:2611.11081) · Code · Project page
HBAR is a hierarchical, discrete tokenizer for resting-state fMRI. Voxel-level BOLD signals are organized into three scales: sub-parcel voxel chunks, parcels (DiFuMo-1024), and functional networks (Yeo-17, plus one group for parcels with no Yeo assignment). Each scale is quantized with binary spherical quantization (BSQ), and the decoder reconstructs the signal residually from network to parcel to voxel.
Models
| Folder | Paper row | Compression | Parameters |
|---|---|---|---|
hbar52_fixed/ |
HBAR-52 (fixed atlas hierarchy) | 52x | 5.65M |
hbar415_fixed/ |
HBAR-415 (fixed atlas hierarchy) | 415x | 7.31M |
Each folder has config.yaml (identical to configs/fmri/<name>.yaml in the code repository) and
one checkpoint per training seed: seed0.pth, seed1.pth, seed2.pth. These are the
final-epoch (epoch 10) weights of the three seeds reported in the paper. The checkpoints hold only
the model weights ({"model": state_dict}) and load with torch.load(..., weights_only=True).
layout/ holds the shared spatial layout the models were trained on: the 190,468 gray-matter voxel
indices on the MNI152NLin2009cAsym 2 mm grid, voxel-to-parcel assignments, the 12,358 sixteen-voxel
chunks, and the parcel-to-network mapping. It is derived from the atlases only and contains no
subject data. The models expect inputs built with exactly this layout.
Training data
HCP Young Adult resting-state fMRI, one run per subject (the longest left-to-right run), preprocessed
with fMRIPrep to MNI152NLin2009cAsym at 2 mm and z-scored per voxel. The models were trained on the
877 training subjects of the subject-level split in the code repository
(data/subject_splits/hcp_ya_1096_80_10_10_seed0.json); all reported numbers are on the 109 held-out
test subjects. Inputs are windows of 32 TRs. HCP data must be obtained from the
Human Connectome Project under its data use terms.
Usage
git clone https://github.com/su-karanps/HBAR && cd HBAR
pip install -r requirements.txt
hf download karanps/HBAR --local-dir checkpoints
Build your dataset with the code repository's pipeline, reusing the released layout so the voxel set matches the models. This skips the gray-matter mask and layout steps:
python3 data/scripts/build_multiscale_hcpya.py --mode process_shard --root_dir /path/to/derivatives \
--layout_dir checkpoints/layout --output_dir /path/to/build/shard_0000 --num_shards 1 --shard_index 0
python3 data/scripts/merge_multiscale_shards.py --shards_root /path/to/build \
--output_index /path/to/build/merged_index.json
python3 data/scripts/build_uncompressed_index.py --index_path /path/to/build/merged_index.json \
--output_dir /path/to/build
Then set index_path to /path/to/build/merged_index_uncompressed.json and layout_dir to
checkpoints/layout in configs/fmri/data.yaml, data_probe.yaml and data_ar.yaml, and pass a checkpoint to any evaluation script,
for example:
python3 scripts/eval/recon_metrics_per_subject.py --data_config configs/fmri/data.yaml --split test \
--windows_per_subject 16 --seed 0 \
--model hbar52_fixed:checkpoints/hbar52_fixed/config.yaml:checkpoints/hbar52_fixed/seed0.pth \
--output_npz results/recon_test.npz --output_json results/recon_test.json
Token export for downstream use:
torchrun --standalone --nproc_per_node=1 scripts/downstream/precompute_fMRI_tokenization.py \
--model_config checkpoints/hbar52_fixed/config.yaml --checkpoint checkpoints/hbar52_fixed/seed0.pth \
--data_config configs/fmri/data_probe.yaml --out_dir tokens/hbar52_fixed_s0
See the code repository for training, linear probes, the autoregressive readout, and the baselines and ablations.
Citation
@misc{singh2026hbar,
title={A Hierarchical Tokenization Framework for Voxel-Level fMRI Representation Learning},
author={Karanpartap Singh and Yue Zhao and Mohammad H. Abbasi and Ehsan Adeli},
year={2026},
eprint={2611.11081},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2611.11081},
}
License
MIT. Copyright (c) 2026 Karanpartap Singh and the Stanford Translational AI Lab (STAI).