ProjectionHeads

Projection heads used by NeuroVLM to align brain-map autoencoder latents and SPECTER text latents.

Files

  • proj_head_image_infonce.safetensors
  • proj_head_text_infonce.safetensors
  • proj_head_text_mse.safetensors
  • config.json

Load through neurovlm.models.load_model("proj_head_image_infonce"), load_model("proj_head_text_infonce"), or load_model("proj_head_text_mse").

Revision 2026-10

Weights of the revised NeuroVLM paper. What changed:

  • Deterministic training. Every stage runs with seeded data loaders and deterministic kernels. Two runs from scratch gave byte-identical weights on the same GPU model, driver and library versions.
  • MLP projector. The brain-to-text projectors (stages 13 and 14) are residual MLPs (28.3 M parameters) instead of Q-Formers. A package names its projector in config.json["projector"].
  • Stage-14 recipe. Each training text is projected onto the canonical bank of its own level, every training map is re-binarized per batch, and title tokens weigh 5 in the loss. Five seeds were trained and the one with the lowest validation loss (seed 3) is packaged.
  • Calibration. Text-to-brain maps are multiplied by one global factor per model, fitted on the validation split and clipped at 1 (map_calibration.json, head 19.16 and adapter 6.13).
  • Title decoding. After a [NETWORK] or [REGION] tag the decoder writes one of the canonical names in title_vocabulary.json (48 network and 83 region titles), so it never invents a title.

The three heads are retrained (stage 09). proj_head_text_mse is the text-to-brain head (v2 loss), and its calibration factor is head in map_calibration.json.

file size sha256
proj_head_image_infonce.safetensors 1.2 MB 917f14964043ce9f021990cb758d7dacf463ed40a31c6f9ac990e53fe31647c9
proj_head_text_infonce.safetensors 2.4 MB 7a079ac851c627be8fb8804ad23ec227a03fbfe35548b5cc284069f7069ff5fa
proj_head_text_mse.safetensors 2.4 MB ed00cccbf66f5082464c3cd0d1ab0643a7f6ba21ea66ce0617eabfef13c615c0
map_calibration.json 0.9 kB 6f9daa11efb35446e32ad5604c36bdeae19ea3bc305cd5c3f289aea28154be83

Provenance: scripts/outputs/runs/final of NeuroVLM (scripts/pipeline/train_pipeline.py, finished 2026-10-10T19:56:25), training code at git 51bedf4 with 131 modified files, inputs pinned by scripts/config/hf_manifest.json (sha256 49f3e3a9825b). scripts/README.md records the later runs from scratch that gave the same bytes.

Earlier weights: revision bcb7e8ce6664d364359d3614eb82dca4eb9a72d7.

Downloads last month
17
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support