Revision 2026-10: final projection heads, map calibration (git 51bedf4, hf_manifest 49f3e3a9825b, 2026-10-10)
Browse filesRun: scripts/outputs/runs/final (scripts/pipeline/train_pipeline.py), finished 2026-10-10T19:56:25.
Training code: git 51bedf4ed6c6b2a0868f8f5fcb5bdef2e330e322 with 131 modified files (runs/final/provenance.json).
Inputs: scripts/config/hf_manifest.json sha256 49f3e3a9825b9754f921b12e75575dcd75e015f15faf9b0b94748acaa370b78b.
Published by scripts/pipeline/push_hf_release.py at git e31d9e5182bf059a7078cfc6944b10974dce5a89.
proj_head_image_infonce.safetensors 917f14964043ce9f021990cb758d7dacf463ed40a31c6f9ac990e53fe31647c9
proj_head_text_infonce.safetensors 7a079ac851c627be8fb8804ad23ec227a03fbfe35548b5cc284069f7069ff5fa
proj_head_text_mse.safetensors ed00cccbf66f5082464c3cd0d1ab0643a7f6ba21ea66ce0617eabfef13c615c0
map_calibration.json 6f9daa11efb35446e32ad5604c36bdeae19ea3bc305cd5c3f289aea28154be83
- README.md +23 -0
- map_calibration.json +25 -0
- proj_head_image_infonce.safetensors +1 -1
- proj_head_text_infonce.safetensors +1 -1
- proj_head_text_mse.safetensors +1 -1
|
@@ -19,3 +19,26 @@ Projection heads used by NeuroVLM to align brain-map autoencoder latents and SPE
|
|
| 19 |
|
| 20 |
Load through `neurovlm.models.load_model("proj_head_image_infonce")`,
|
| 21 |
`load_model("proj_head_text_infonce")`, or `load_model("proj_head_text_mse")`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
|
| 20 |
Load through `neurovlm.models.load_model("proj_head_image_infonce")`,
|
| 21 |
`load_model("proj_head_text_infonce")`, or `load_model("proj_head_text_mse")`.
|
| 22 |
+
|
| 23 |
+
## Revision 2026-10
|
| 24 |
+
|
| 25 |
+
Weights of the revised NeuroVLM paper. What changed:
|
| 26 |
+
|
| 27 |
+
- **Deterministic training.** Every stage runs with seeded data loaders and deterministic kernels. Two runs from scratch gave byte-identical weights on the same GPU model, driver and library versions.
|
| 28 |
+
- **MLP projector.** The brain-to-text projectors (stages 13 and 14) are residual MLPs (28.3 M parameters) instead of Q-Formers. A package names its projector in `config.json["projector"]`.
|
| 29 |
+
- **Stage-14 recipe.** Each training text is projected onto the canonical bank of its own level, every training map is re-binarized per batch, and title tokens weigh 5 in the loss. Five seeds were trained and the one with the lowest validation loss (seed 3) is packaged.
|
| 30 |
+
- **Calibration.** Text-to-brain maps are multiplied by one global factor per model, fitted on the validation split and clipped at 1 (`map_calibration.json`, `head` 19.16 and `adapter` 6.13).
|
| 31 |
+
- **Title decoding.** After a [NETWORK] or [REGION] tag the decoder writes one of the canonical names in `title_vocabulary.json` (48 network and 83 region titles), so it never invents a title.
|
| 32 |
+
|
| 33 |
+
The three heads are retrained (stage 09). `proj_head_text_mse` is the text-to-brain head (v2 loss), and its calibration factor is `head` in `map_calibration.json`.
|
| 34 |
+
|
| 35 |
+
| file | size | sha256 |
|
| 36 |
+
|---|---|---|
|
| 37 |
+
| `proj_head_image_infonce.safetensors` | 1.2 MB | `917f14964043ce9f021990cb758d7dacf463ed40a31c6f9ac990e53fe31647c9` |
|
| 38 |
+
| `proj_head_text_infonce.safetensors` | 2.4 MB | `7a079ac851c627be8fb8804ad23ec227a03fbfe35548b5cc284069f7069ff5fa` |
|
| 39 |
+
| `proj_head_text_mse.safetensors` | 2.4 MB | `ed00cccbf66f5082464c3cd0d1ab0643a7f6ba21ea66ce0617eabfef13c615c0` |
|
| 40 |
+
| `map_calibration.json` | 0.9 kB | `6f9daa11efb35446e32ad5604c36bdeae19ea3bc305cd5c3f289aea28154be83` |
|
| 41 |
+
|
| 42 |
+
Provenance: `scripts/outputs/runs/final` of NeuroVLM (`scripts/pipeline/train_pipeline.py`, finished 2026-10-10T19:56:25), training code at git `51bedf4` with 131 modified files, inputs pinned by `scripts/config/hf_manifest.json` (sha256 `49f3e3a9825b`). `scripts/README.md` records the later runs from scratch that gave the same bytes.
|
| 43 |
+
|
| 44 |
+
Earlier weights: revision `bcb7e8ce6664d364359d3614eb82dca4eb9a72d7`.
|
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"rule": "global: maps * factor, clipped at 1",
|
| 3 |
+
"head": 19.164856467116945,
|
| 4 |
+
"head_peak_quantiles": {
|
| 5 |
+
"0.01": 0.0026107050012797116,
|
| 6 |
+
"0.5": 0.05217884108424187,
|
| 7 |
+
"0.99": 0.42548597455024767
|
| 8 |
+
},
|
| 9 |
+
"adapter": 6.125958446658484,
|
| 10 |
+
"adapter_peak_quantiles": {
|
| 11 |
+
"0.01": 0.02615324817597866,
|
| 12 |
+
"0.5": 0.1632397621870041,
|
| 13 |
+
"0.99": 0.6896851539611819
|
| 14 |
+
},
|
| 15 |
+
"source": {
|
| 16 |
+
"models": {
|
| 17 |
+
"autoencoder": "autoencoder.pt:60b6173ab357153a457503e95cecc478230a3f3d3d1f8b2dc5243aeb44e2c0f7",
|
| 18 |
+
"proj_head_text_mse": "proj_head_text_mse.pt:48e33efe7d3bae69bbd82b69b3131720ee0728a2693f8a8cb995bd9ad1121a2e",
|
| 19 |
+
"neuro_adapter": "adapter_anat_full.pt:299bf85267cc9ed7147388f1a458a1477e4aff395a92c5a61eb8880360a34353"
|
| 20 |
+
},
|
| 21 |
+
"split": "pmids_split.pt",
|
| 22 |
+
"n_val": 2961,
|
| 23 |
+
"factor": "median over validation maps of 1 / map peak"
|
| 24 |
+
}
|
| 25 |
+
}
|
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1183048
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:917f14964043ce9f021990cb758d7dacf463ed40a31c6f9ac990e53fe31647c9
|
| 3 |
size 1183048
|
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 2363216
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7a079ac851c627be8fb8804ad23ec227a03fbfe35548b5cc284069f7069ff5fa
|
| 3 |
size 2363216
|
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 2363216
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ed00cccbf66f5082464c3cd0d1ab0643a7f6ba21ea66ce0617eabfef13c615c0
|
| 3 |
size 2363216
|