NeuroQformer
NeuroQformer contains the QFormer weights used by NeuroVLM brain-map-to-text generation.
Files
model.safetensors: tensor weights for the QFormer wrapper.config.json: architecture config and canonical semantic metadata.
The weights include the QFormer, frozen image projection head, and frozen canonical semantic projection banks. The canonical bank tensors are registered as buffers, so they are part of the model state and never trainable.
Load with:
from neurovlm.retrieval_resources import _load_neuro_qformer
model = _load_neuro_qformer(device="cpu", projection_temp=0.05, canonical_basis="network")
Set projection_temp=None to bypass canonical projection.
Revision 2026-10
Weights of the revised NeuroVLM paper. What changed:
- Deterministic training. Every stage runs with seeded data loaders and deterministic kernels. Two runs from scratch gave byte-identical weights on the same GPU model, driver and library versions.
- MLP projector. The brain-to-text projectors (stages 13 and 14) are residual MLPs (28.3 M parameters) instead of Q-Formers. A package names its projector in
config.json["projector"]. - Stage-14 recipe. Each training text is projected onto the canonical bank of its own level, every training map is re-binarized per batch, and title tokens weigh 5 in the loss. Five seeds were trained and the one with the lowest validation loss (seed 3) is packaged.
- Calibration. Text-to-brain maps are multiplied by one global factor per model, fitted on the validation split and clipped at 1 (
map_calibration.json,head19.16 andadapter6.13). - Title decoding. After a [NETWORK] or [REGION] tag the decoder writes one of the canonical names in
title_vocabulary.json(48 network and 83 region titles), so it never invents a title.
model.safetensors and config.json hold the stage-14 MLP projector with the frozen image head and the canonical banks, which are also in canonical_semantic_banks.pt. title_vocabulary.json lists the titles the decoder may write. Loading needs neurovlm with ResidualMLPProjector (October 2026, NeuroQFormer.from_state_dict_payload reads config.json["projector"]). load_projector in NeuroVLM's scripts/models/_common_models.py reads the package too.
| file | size | sha256 |
|---|---|---|
model.safetensors |
118.0 MB | eaf5ee494e96e40f9c75f2b9bd4aed0edcbe4589b8fafe9d0365402a5515a5a9 |
config.json |
1.5 kB | 6e366e8e6fd5b254daaf13e1d75f0e805790f1fdcd988f82c71e37d4ceb99fcf |
canonical_semantic_banks.pt |
3.4 MB | f28293a0595afab0a137ba8cc96b8023b5562e7b464f7071a3d5230da7f20771 |
title_vocabulary.json |
41.3 kB | 31acb78f7f5ef304cd6e69d037d4f91af069883859b2919b2c5c1a79056bdf85 |
Provenance: scripts/outputs/runs/final of NeuroVLM (scripts/pipeline/train_pipeline.py, finished 2026-10-10T19:56:25), training code at git 51bedf4 with 131 modified files, inputs pinned by scripts/config/hf_manifest.json (sha256 49f3e3a9825b). scripts/README.md records the later runs from scratch that gave the same bytes.
Earlier weights: revision 29b45da49a50edb4a130c5134d600e3aad66c668.
- Downloads last month
- 27