rphammonds commited on
Commit
b92b1cf
·
verified ·
1 Parent(s): bcb7e8c

Revision 2026-10: final projection heads, map calibration (git 51bedf4, hf_manifest 49f3e3a9825b, 2026-10-10)

Browse files

Run: scripts/outputs/runs/final (scripts/pipeline/train_pipeline.py), finished 2026-10-10T19:56:25.
Training code: git 51bedf4ed6c6b2a0868f8f5fcb5bdef2e330e322 with 131 modified files (runs/final/provenance.json).
Inputs: scripts/config/hf_manifest.json sha256 49f3e3a9825b9754f921b12e75575dcd75e015f15faf9b0b94748acaa370b78b.
Published by scripts/pipeline/push_hf_release.py at git e31d9e5182bf059a7078cfc6944b10974dce5a89.

proj_head_image_infonce.safetensors 917f14964043ce9f021990cb758d7dacf463ed40a31c6f9ac990e53fe31647c9
proj_head_text_infonce.safetensors 7a079ac851c627be8fb8804ad23ec227a03fbfe35548b5cc284069f7069ff5fa
proj_head_text_mse.safetensors ed00cccbf66f5082464c3cd0d1ab0643a7f6ba21ea66ce0617eabfef13c615c0
map_calibration.json 6f9daa11efb35446e32ad5604c36bdeae19ea3bc305cd5c3f289aea28154be83

README.md CHANGED
@@ -19,3 +19,26 @@ Projection heads used by NeuroVLM to align brain-map autoencoder latents and SPE
19
 
20
  Load through `neurovlm.models.load_model("proj_head_image_infonce")`,
21
  `load_model("proj_head_text_infonce")`, or `load_model("proj_head_text_mse")`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  Load through `neurovlm.models.load_model("proj_head_image_infonce")`,
21
  `load_model("proj_head_text_infonce")`, or `load_model("proj_head_text_mse")`.
22
+
23
+ ## Revision 2026-10
24
+
25
+ Weights of the revised NeuroVLM paper. What changed:
26
+
27
+ - **Deterministic training.** Every stage runs with seeded data loaders and deterministic kernels. Two runs from scratch gave byte-identical weights on the same GPU model, driver and library versions.
28
+ - **MLP projector.** The brain-to-text projectors (stages 13 and 14) are residual MLPs (28.3 M parameters) instead of Q-Formers. A package names its projector in `config.json["projector"]`.
29
+ - **Stage-14 recipe.** Each training text is projected onto the canonical bank of its own level, every training map is re-binarized per batch, and title tokens weigh 5 in the loss. Five seeds were trained and the one with the lowest validation loss (seed 3) is packaged.
30
+ - **Calibration.** Text-to-brain maps are multiplied by one global factor per model, fitted on the validation split and clipped at 1 (`map_calibration.json`, `head` 19.16 and `adapter` 6.13).
31
+ - **Title decoding.** After a [NETWORK] or [REGION] tag the decoder writes one of the canonical names in `title_vocabulary.json` (48 network and 83 region titles), so it never invents a title.
32
+
33
+ The three heads are retrained (stage 09). `proj_head_text_mse` is the text-to-brain head (v2 loss), and its calibration factor is `head` in `map_calibration.json`.
34
+
35
+ | file | size | sha256 |
36
+ |---|---|---|
37
+ | `proj_head_image_infonce.safetensors` | 1.2 MB | `917f14964043ce9f021990cb758d7dacf463ed40a31c6f9ac990e53fe31647c9` |
38
+ | `proj_head_text_infonce.safetensors` | 2.4 MB | `7a079ac851c627be8fb8804ad23ec227a03fbfe35548b5cc284069f7069ff5fa` |
39
+ | `proj_head_text_mse.safetensors` | 2.4 MB | `ed00cccbf66f5082464c3cd0d1ab0643a7f6ba21ea66ce0617eabfef13c615c0` |
40
+ | `map_calibration.json` | 0.9 kB | `6f9daa11efb35446e32ad5604c36bdeae19ea3bc305cd5c3f289aea28154be83` |
41
+
42
+ Provenance: `scripts/outputs/runs/final` of NeuroVLM (`scripts/pipeline/train_pipeline.py`, finished 2026-10-10T19:56:25), training code at git `51bedf4` with 131 modified files, inputs pinned by `scripts/config/hf_manifest.json` (sha256 `49f3e3a9825b`). `scripts/README.md` records the later runs from scratch that gave the same bytes.
43
+
44
+ Earlier weights: revision `bcb7e8ce6664d364359d3614eb82dca4eb9a72d7`.
map_calibration.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "rule": "global: maps * factor, clipped at 1",
3
+ "head": 19.164856467116945,
4
+ "head_peak_quantiles": {
5
+ "0.01": 0.0026107050012797116,
6
+ "0.5": 0.05217884108424187,
7
+ "0.99": 0.42548597455024767
8
+ },
9
+ "adapter": 6.125958446658484,
10
+ "adapter_peak_quantiles": {
11
+ "0.01": 0.02615324817597866,
12
+ "0.5": 0.1632397621870041,
13
+ "0.99": 0.6896851539611819
14
+ },
15
+ "source": {
16
+ "models": {
17
+ "autoencoder": "autoencoder.pt:60b6173ab357153a457503e95cecc478230a3f3d3d1f8b2dc5243aeb44e2c0f7",
18
+ "proj_head_text_mse": "proj_head_text_mse.pt:48e33efe7d3bae69bbd82b69b3131720ee0728a2693f8a8cb995bd9ad1121a2e",
19
+ "neuro_adapter": "adapter_anat_full.pt:299bf85267cc9ed7147388f1a458a1477e4aff395a92c5a61eb8880360a34353"
20
+ },
21
+ "split": "pmids_split.pt",
22
+ "n_val": 2961,
23
+ "factor": "median over validation maps of 1 / map peak"
24
+ }
25
+ }
proj_head_image_infonce.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cb293e185e3736ac7917f576e727fe8f8f4bf2a8666b9d1bd39eac5d85f4ac13
3
  size 1183048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:917f14964043ce9f021990cb758d7dacf463ed40a31c6f9ac990e53fe31647c9
3
  size 1183048
proj_head_text_infonce.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1198c6b21ef236137485ee3ab14f8f679fca9783f387bd6a12a416f97e2c360f
3
  size 2363216
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a079ac851c627be8fb8804ad23ec227a03fbfe35548b5cc284069f7069ff5fa
3
  size 2363216
proj_head_text_mse.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b023b8c5a646f4ce526b25b5c23062136c707a5a359790b88001526ab2e98409
3
  size 2363216
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed00cccbf66f5082464c3cd0d1ab0643a7f6ba21ea66ce0617eabfef13c615c0
3
  size 2363216