| --- |
| license: mit |
| tags: |
| - fmri |
| - neuroscience |
| - tokenizer |
| - neuroimaging |
| --- |
| |
| # HBAR: Hierarchical Brain Activity Representation |
|
|
| Pretrained tokenizers from **A Hierarchical Tokenization Framework for Voxel-Level fMRI Representation |
| Learning** (Karanpartap Singh, Yue Zhao, Mohammad H. Abbasi, Ehsan Adeli). |
|
|
| [Paper (arXiv:2611.11081)](https://arxiv.org/abs/2611.11081) · |
| [Code](https://github.com/su-karanps/HBAR) · |
| [Project page](https://karanps.com/hbar) |
|
|
| HBAR is a hierarchical, discrete tokenizer for resting-state fMRI. Voxel-level BOLD signals are |
| organized into three scales: sub-parcel voxel chunks, parcels (DiFuMo-1024), and functional networks |
| (Yeo-17, plus one group for parcels with no Yeo assignment). Each scale is quantized with binary |
| spherical quantization (BSQ), and the decoder reconstructs the signal residually from network to |
| parcel to voxel. |
|
|
| ## Models |
|
|
| | Folder | Paper row | Compression | Parameters | |
| |---|---|---|---| |
| | `hbar52_fixed/` | HBAR-52 (fixed atlas hierarchy) | 52x | 5.65M | |
| | `hbar415_fixed/` | HBAR-415 (fixed atlas hierarchy) | 415x | 7.31M | |
|
|
| Each folder has `config.yaml` (identical to `configs/fmri/<name>.yaml` in the code repository) and |
| one checkpoint per training seed: `seed0.pth`, `seed1.pth`, `seed2.pth`. These are the |
| final-epoch (epoch 10) weights of the three seeds reported in the paper. The checkpoints hold only |
| the model weights (`{"model": state_dict}`) and load with `torch.load(..., weights_only=True)`. |
|
|
| `layout/` holds the shared spatial layout the models were trained on: the 190,468 gray-matter voxel |
| indices on the MNI152NLin2009cAsym 2 mm grid, voxel-to-parcel assignments, the 12,358 sixteen-voxel |
| chunks, and the parcel-to-network mapping. It is derived from the atlases only and contains no |
| subject data. The models expect inputs built with exactly this layout. |
|
|
| ## Training data |
|
|
| HCP Young Adult resting-state fMRI, one run per subject (the longest left-to-right run), preprocessed |
| with fMRIPrep to MNI152NLin2009cAsym at 2 mm and z-scored per voxel. The models were trained on the |
| 877 training subjects of the subject-level split in the code repository |
| (`data/subject_splits/hcp_ya_1096_80_10_10_seed0.json`); all reported numbers are on the 109 held-out |
| test subjects. Inputs are windows of 32 TRs. HCP data must be obtained from the |
| [Human Connectome Project](https://www.humanconnectome.org/) under its data use terms. |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/su-karanps/HBAR && cd HBAR |
| pip install -r requirements.txt |
| hf download karanps/HBAR --local-dir checkpoints |
| ``` |
|
|
| Build your dataset with the code repository's pipeline, reusing the released layout so the voxel set |
| matches the models. This skips the gray-matter mask and layout steps: |
|
|
| ```bash |
| python3 data/scripts/build_multiscale_hcpya.py --mode process_shard --root_dir /path/to/derivatives \ |
| --layout_dir checkpoints/layout --output_dir /path/to/build/shard_0000 --num_shards 1 --shard_index 0 |
| python3 data/scripts/merge_multiscale_shards.py --shards_root /path/to/build \ |
| --output_index /path/to/build/merged_index.json |
| python3 data/scripts/build_uncompressed_index.py --index_path /path/to/build/merged_index.json \ |
| --output_dir /path/to/build |
| ``` |
|
|
| Then set `index_path` to `/path/to/build/merged_index_uncompressed.json` and `layout_dir` to |
| `checkpoints/layout` in `configs/fmri/data.yaml`, `data_probe.yaml` and `data_ar.yaml`, and pass a checkpoint to any evaluation script, |
| for example: |
|
|
| ```bash |
| python3 scripts/eval/recon_metrics_per_subject.py --data_config configs/fmri/data.yaml --split test \ |
| --windows_per_subject 16 --seed 0 \ |
| --model hbar52_fixed:checkpoints/hbar52_fixed/config.yaml:checkpoints/hbar52_fixed/seed0.pth \ |
| --output_npz results/recon_test.npz --output_json results/recon_test.json |
| ``` |
|
|
| Token export for downstream use: |
|
|
| ```bash |
| torchrun --standalone --nproc_per_node=1 scripts/downstream/precompute_fMRI_tokenization.py \ |
| --model_config checkpoints/hbar52_fixed/config.yaml --checkpoint checkpoints/hbar52_fixed/seed0.pth \ |
| --data_config configs/fmri/data_probe.yaml --out_dir tokens/hbar52_fixed_s0 |
| ``` |
|
|
| See the [code repository](https://github.com/su-karanps/HBAR) for training, linear probes, the |
| autoregressive readout, and the baselines and ablations. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{singh2026hbar, |
| title={A Hierarchical Tokenization Framework for Voxel-Level fMRI Representation Learning}, |
| author={Karanpartap Singh and Yue Zhao and Mohammad H. Abbasi and Ehsan Adeli}, |
| year={2026}, |
| eprint={2611.11081}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.LG}, |
| url={https://arxiv.org/abs/2611.11081}, |
| } |
| ``` |
|
|
| ## License |
|
|
| MIT. Copyright (c) 2026 Karanpartap Singh and the Stanford Translational AI Lab (STAI). |
|
|
