shawn24 commited on
Commit
5e07ca4
·
verified ·
1 Parent(s): a8c1b3b

Replace incorrect risk weights with five BU S1 KD plus S2 benchmark checkpoints

Browse files
risk_checkpoints/{Mammo-FM_risk_stage1_distillation_fold1.pth-best.tar → BU_S1_KD_S2/fold1/Mammo-FM_BU_S1_KD_S2_fold1.pth-best.tar} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bd16aa6dba9f1812d52947a325df92cd8aace440ddec2fee21cbd8357fcdf41a
3
  size 184831762
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f57730d674716c5870e5f0678c82c2a21cdff37eea518059f09809032830653e
3
  size 184831762
risk_checkpoints/{Mammo-FM_risk_stage2_finetuning_fold1.pth-best.tar → BU_S1_KD_S2/fold2/Mammo-FM_BU_S1_KD_S2_fold2.pth-best.tar} RENAMED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:397098ff49c914dba52a56c9c3ec530fe4d1637170a0a536d5e84c0dc95bc6cf
3
  size 184831762
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cac0884effa796de7fc32ac98b630301755d1d72a12a52addab24853a2c618a9
3
  size 184831762
risk_checkpoints/BU_S1_KD_S2/fold3/Mammo-FM_BU_S1_KD_S2_fold3.pth-best.tar ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f6b1cf5845c708a530ef8f5f98ecf2db87b54217f74b430e9b068ec99a40d1d6
3
+ size 184831762
risk_checkpoints/BU_S1_KD_S2/fold4/Mammo-FM_BU_S1_KD_S2_fold4.pth-best.tar ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b1bf8769f3d6ee86253e36a473047396fb2e9d34470c440b8c5c4421dcbb9742
3
+ size 184831762
risk_checkpoints/BU_S1_KD_S2/fold5/Mammo-FM_BU_S1_KD_S2_fold5.pth-best.tar ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0df00131c5a91ccd2e9d464bdbfb561f9604d589e38e61fa2903de5c50b8490f
3
+ size 184831762
risk_checkpoints/BU_S1_KD_S2/manifest.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "schema_version": 1,
3
+ "model": "Mammo-FM BU S1 KD + S2",
4
+ "training_variant": "from_stage1",
5
+ "description": "Five final Stage-2 risk models initialized from Stage-1 knowledge distillation. Each model is evaluated only on its matching held-out BU patient fold.",
6
+ "outputs": "Five raw independent sigmoid probabilities; uncalibrated.",
7
+ "metric_aggregation": "Unweighted mean across five held-out folds; AUPRC uses average precision and year-specific eligibility masks.",
8
+ "files": [
9
+ {
10
+ "fold": 1,
11
+ "path": "risk_checkpoints/BU_S1_KD_S2/fold1/Mammo-FM_BU_S1_KD_S2_fold1.pth-best.tar",
12
+ "sha256": "f57730d674716c5870e5f0678c82c2a21cdff37eea518059f09809032830653e",
13
+ "bytes": 184831762,
14
+ "stored_epoch": 2
15
+ },
16
+ {
17
+ "fold": 2,
18
+ "path": "risk_checkpoints/BU_S1_KD_S2/fold2/Mammo-FM_BU_S1_KD_S2_fold2.pth-best.tar",
19
+ "sha256": "cac0884effa796de7fc32ac98b630301755d1d72a12a52addab24853a2c618a9",
20
+ "bytes": 184831762,
21
+ "stored_epoch": 4
22
+ },
23
+ {
24
+ "fold": 3,
25
+ "path": "risk_checkpoints/BU_S1_KD_S2/fold3/Mammo-FM_BU_S1_KD_S2_fold3.pth-best.tar",
26
+ "sha256": "f6b1cf5845c708a530ef8f5f98ecf2db87b54217f74b430e9b068ec99a40d1d6",
27
+ "bytes": 184831762,
28
+ "stored_epoch": 6
29
+ },
30
+ {
31
+ "fold": 4,
32
+ "path": "risk_checkpoints/BU_S1_KD_S2/fold4/Mammo-FM_BU_S1_KD_S2_fold4.pth-best.tar",
33
+ "sha256": "b1bf8769f3d6ee86253e36a473047396fb2e9d34470c440b8c5c4421dcbb9742",
34
+ "bytes": 184831762,
35
+ "stored_epoch": 6
36
+ },
37
+ {
38
+ "fold": 5,
39
+ "path": "risk_checkpoints/BU_S1_KD_S2/fold5/Mammo-FM_BU_S1_KD_S2_fold5.pth-best.tar",
40
+ "sha256": "0df00131c5a91ccd2e9d464bdbfb561f9604d589e38e61fa2903de5c50b8490f",
41
+ "bytes": 184831762,
42
+ "stored_epoch": 5
43
+ }
44
+ ],
45
+ "validation_status": "Historical benchmark checkpoints identified and file hashes verified; fresh port inference validation is separate and pending."
46
+ }
risk_checkpoints/README.md CHANGED
@@ -1,47 +1,96 @@
1
- # Mammo-FM risk checkpoints
2
-
3
- The historical risk artifacts are released at
4
- [`batmanLab/Mammo-FM/risk_checkpoints`](https://huggingface.co/batmanLab/Mammo-FM/tree/main/risk_checkpoints).
5
-
6
- | File | Original training stage | Payload epoch | Bytes | SHA-256 |
7
- | --- | --- | ---: | ---: | --- |
8
- | `Mammo-FM_risk_stage1_distillation_fold1.pth-best.tar` | MIRAI probability distillation | 17 | 184,831,762 | `bd16aa6dba9f1812d52947a325df92cd8aace440ddec2fee21cbd8357fcdf41a` |
9
- | `Mammo-FM_risk_stage2_finetuning_fold1.pth-best.tar` | supervised five-year risk training | 8 | 184,831,762 | `397098ff49c914dba52a56c9c3ec530fe4d1637170a0a536d5e84c0dc95bc6cf` |
10
-
11
- These files are uploaded byte-for-byte under descriptive release names. Their
12
- payloads contain `model`, `optimizer`, `scheduler`, and `epoch`. They do not
13
- contain explicit `stage`, `fold`, validation loss, class weights, teacher
14
- provenance, or base-model configuration. “fold1” is retained from the original
15
- filename rather than inferred from checkpoint metadata.
16
-
17
- The Stage 2 training run has separate local completion evidence at epoch 20
18
- (SHA-256 `5238eff89cae22106f4c35f23434ba448190a68232a1214ceaedce19c5268cab`);
19
- that latest-state file is not part of this best-checkpoint release. No
20
- corresponding Stage 1 completion artifact was located, so Stage 1 is described
21
- only as the historical best-validation checkpoint at embedded epoch 17.
22
-
23
- The historical model state contains 1,763 entries because its wrapper
24
- registered the same EfficientNet image encoder at both `image_encoder.*` and
25
- `aggregator.image_encoder.*`. The two namespaces are duplicates within each
26
- file. `mayo_mammofm_risk.model.compatible_model_state` verifies tensor equality
27
- and removes only the duplicate namespace, leaving 911 keys for strict loading.
28
-
29
- The original Mayo launchers explicitly identify these exact filenames as the
30
- Stage 1 and Stage 2 Mayo models. The payloads themselves do not contain enough
31
- metadata to independently re-establish that lineage, so the original path,
32
- filename, payload inventory, and hashes are retained as provenance. Between
33
- the two checkpoints, all image-encoder parameter tensors remain identical;
34
- only BatchNorm running means, variances, and counters differ. This matches the
35
- historical wrapper leaving frozen BatchNorm modules in training mode. The
36
- cleaned runner corrects that behavior and therefore may not reproduce the
37
- historical numbers bit-for-bit.
38
-
39
- The historical optimizer state is not resumed by the cleaned runner. Only
40
- model weights are loaded. A compatible Mammo-FM base checkpoint is also
41
- required to construct the image-encoder configuration before loading either
42
- file. Strict loading was verified with the public
43
- [`Mammo-FM_BatmanlabTrained_CLIP.tar`](https://huggingface.co/batmanLab/Mammo-FM/blob/main/Mammo-FM_BatmanlabTrained_CLIP.tar)
44
- checkpoint: the normalized Stage 2 state matched all 911 model keys.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
  The repository's model-weight license applies. The weights are for
47
- non-commercial academic research and must not be used for clinical decisions.
 
 
1
+ # Mammo-FM risk checkpoints: BU S1 KD + S2
2
+
3
+ This release contains five BU fold-specific **final Stage 2 risk models
4
+ initialized through Stage 1 MIRAI knowledge distillation**. It is the
5
+ historical `all_years/aug_n/from_stage1` model family, not Stage 1 alone and
6
+ not the S2-only comparison variant. These checkpoints replace the previously
7
+ released Mayo-labelled Stage 1/Stage 2 pair, which was not the intended BU
8
+ benchmark release.
9
+
10
+ ## Download and provenance
11
+
12
+ All five files contain 184,831,762 bytes. Stored epoch values are reproduced
13
+ without changing their indexing convention. The
14
+ [release manifest](https://huggingface.co/batmanLab/Mammo-FM/blob/main/risk_checkpoints/BU_S1_KD_S2/manifest.json)
15
+ contains the same checkpoint identities and hashes.
16
+
17
+ | BU fold | Final S1 KD + S2 checkpoint | Stored epoch | SHA-256 |
18
+ | --- | --- | ---: | --- |
19
+ | 1 | [Download fold 1](https://huggingface.co/batmanLab/Mammo-FM/resolve/main/risk_checkpoints/BU_S1_KD_S2/fold1/Mammo-FM_BU_S1_KD_S2_fold1.pth-best.tar) | 2 | `f57730d674716c5870e5f0678c82c2a21cdff37eea518059f09809032830653e` |
20
+ | 2 | [Download fold 2](https://huggingface.co/batmanLab/Mammo-FM/resolve/main/risk_checkpoints/BU_S1_KD_S2/fold2/Mammo-FM_BU_S1_KD_S2_fold2.pth-best.tar) | 4 | `cac0884effa796de7fc32ac98b630301755d1d72a12a52addab24853a2c618a9` |
21
+ | 3 | [Download fold 3](https://huggingface.co/batmanLab/Mammo-FM/resolve/main/risk_checkpoints/BU_S1_KD_S2/fold3/Mammo-FM_BU_S1_KD_S2_fold3.pth-best.tar) | 6 | `f6b1cf5845c708a530ef8f5f98ecf2db87b54217f74b430e9b068ec99a40d1d6` |
22
+ | 4 | [Download fold 4](https://huggingface.co/batmanLab/Mammo-FM/resolve/main/risk_checkpoints/BU_S1_KD_S2/fold4/Mammo-FM_BU_S1_KD_S2_fold4.pth-best.tar) | 6 | `b1bf8769f3d6ee86253e36a473047396fb2e9d34470c440b8c5c4421dcbb9742` |
23
+ | 5 | [Download fold 5](https://huggingface.co/batmanLab/Mammo-FM/resolve/main/risk_checkpoints/BU_S1_KD_S2/fold5/Mammo-FM_BU_S1_KD_S2_fold5.pth-best.tar) | 5 | `0df00131c5a91ccd2e9d464bdbfb561f9604d589e38e61fa2903de5c50b8490f` |
24
+
25
+ ```bash
26
+ hf download batmanLab/Mammo-FM \
27
+ --include 'risk_checkpoints/BU_S1_KD_S2/*' \
28
+ --local-dir /path/to/checkpoints
29
+ ```
30
+
31
+ The weights are released byte-for-byte from the historical best-validation
32
+ checkpoints. Their payloads contain `model`, `optimizer`, `scheduler`, and
33
+ `epoch`; fold and training-stage lineage come from the corresponding original
34
+ experiment artifacts. No images, reports, patient-level tables, predictions,
35
+ feature caches, or API credentials are included. Store downloaded weights
36
+ and all experiment outputs outside the source repository.
37
+
38
+ ## BU held-out evaluation
39
+
40
+ Checkpoint `fold{k}` must be evaluated only on source rows with
41
+ `split_fold_{k}=test_fold_{k}`. The five patient-disjoint test folds cover
42
+ 13,694 examinations from 9,900 patients. They are not an ensemble; do not
43
+ apply one fold's model to the entire cohort or average the five models'
44
+ predictions for this benchmark.
45
+
46
+ Preserve the historical BU PNG preprocessing and the four-view order
47
+ `LMLO -> LCC -> RMLO -> RCC`, including its view/side metadata. A generic
48
+ Mayo-style DICOM loader is a different input pipeline and is not a drop-in
49
+ replacement for this benchmark.
50
+
51
+ The model returns raw, uncalibrated independent sigmoid probabilities for
52
+ years 1 through 5. AUROC and AUPRC (average precision) are evaluated within
53
+ each fold using horizon-specific follow-up masks and reported as unweighted
54
+ five-fold means. C-index is reported as the mean and between-fold standard
55
+ deviation. Do not replace these with pooled metrics, calibrated risks, or
56
+ monotonic/cumulative post-processing.
57
+
58
+ | Horizon | Eligible examinations | Positive examinations |
59
+ | --- | ---: | ---: |
60
+ | 1 year | 13,694 | 283 |
61
+ | 2 years | 12,724 | 495 |
62
+ | 3 years | 11,878 | 700 |
63
+ | 4 years | 11,113 | 888 |
64
+ | 5 years | 10,369 | 1,044 |
65
+
66
+ ## Model loading
67
+
68
+ The historical model state registers the same encoder under
69
+ `image_encoder.*` and `aggregator.image_encoder.*`. The Mammo-FM compatibility
70
+ loader verifies exact equality of these duplicate tensors before removing
71
+ only the duplicate namespace. The complete remaining state must then be
72
+ loaded strictly.
73
+
74
+ Download the compatible base checkpoint for architecture construction:
75
+
76
+ ```bash
77
+ hf download batmanLab/Mammo-FM Mammo-FM_BatmanlabTrained_CLIP.tar \
78
+ --local-dir /path/to/checkpoints
79
+ ```
80
+
81
+ The base checkpoint supplies the encoder architecture configuration. Loading
82
+ the full risk checkpoint replaces the encoder and all risk-model tensors;
83
+ the final predictions do not retain the base encoder's initialization.
84
+ Optimizer and scheduler states are not used for inference. Load serialized
85
+ training checkpoints only from a trusted source.
86
+
87
+ ## Validation status and use restrictions
88
+
89
+ Historical checkpoint identity and ported-code numerical parity are separate
90
+ checks. Availability of these weights does not establish that the port has
91
+ reproduced the historical BU held-out predictions or metrics. The fold-matched
92
+ comparison must complete before such a claim is made.
93
 
94
  The repository's model-weight license applies. The weights are for
95
+ non-commercial academic research only and must not be used for clinical
96
+ decisions.