TamAko783/anomalies-diffusion-andi-60k

ANDi-style normative diffusion model for brain-MRI anomaly localisation (60k steps)

Research use only. Not a medical device. Not validated for clinical use. Must not guide patient care.

Developed by Experimental Machines. This is the best model of a small study on whether diffusion models can find anomalies in brain MRI without lesion labels, cheaply, and with an honest false-positive guarantee (healthy-only conformal calibration and bias audit). Companion repositories: anomalies-diffusion-andi (same model at 20k steps), anomalies-diffusion-dae-baseline (denoising-autoencoder baseline), anomalies-diffusion-calibration (conformal calibration and bias audit).

Model details

• Type: pixel-space denoising diffusion model trained on healthy 2D axial T2 slices; anomaly map = aggregated difference between the model's reverse-step posterior mean and the one implied by the input, over low-noise timesteps (Aggregated Normative Diffusion).
• Origin: independent re-implementation of the idea in Frotscher et al., arXiv:2312.01904. The reference repository has no licence, so no upstream code was copied. Differences from the paper: smaller network, 128x128 slices, plain Gaussian noise (no pyramid noise), far fewer steps, scoring on every 10th timestep. Numbers here are not comparable to the published ones.
• Architecture: U-Net, base width 32, channel multipliers (1, 2, 2, 3, 4), 2 res-blocks per level, self-attention at 16x16 and 8x8, dropout 0.1 (levels at 32x32 and below), 6.43 M parameters; v-prediction; shifted-cosine log-SNR schedule.
• Training: AdamW, lr 1e-4 constant, batch 32, EMA 0.9995, bf16 autocast, 60,000 steps (continued from a 20,000-step run: network, EMA, optimiser and data-order state restored; dropout RNG re-seeded, so not bit-identical to one uninterrupted run).
• Weights: best_model.safetensors (25.8 MB, EMA weights, step 60000).
• Code: the uadcal package used to train and score this model is not published yet; weights are not loadable without it. The architecture above is specified for re-implementation, and code is available from the authors on request.

Intended use

Research on unsupervised anomaly detection, conformal calibration, and cost/benefit studies of diffusion detectors on T2 brain MRI that has been preprocessed like the training data. Suitable as a reproducible baseline.

Out-of-scope use

Diagnosis, triage, screening, or any clinical decision. Other sequences (T1, FLAIR), other body parts, children, or scans preprocessed differently. Do not treat a flagged region as a lesion or an unflagged scan as healthy.

Training data

319 healthy-subject T2 scans from IXI (Guy's, Hammersmith, IOP sites; subject-level, hashed split; no calibration or test subject was used for training). Preprocessing: N4 bias correction, HD-BET skull-stripping, affine registration to SRI24, histogram-mode intensity normalisation, central axial brain slices at 128x128. IXI: CC BY-SA 3.0, https://brain-development.org/ixi-dataset/.

Evaluation

Held-out lesion data: 20 multiple-sclerosis T2 scans from MSLUB (Lesjak et al., Neuroinformatics 2017; CC-BY); 10 further MSLUB scans were used only to choose the checkpoint and threshold. Mean over scans, [95% bootstrap CI over subjects]:

Metric	ANDi 60k	ANDi 20k	DAE baseline	Intensity baseline
Voxel AP	0.090 [0.060, 0.123]	0.084 [0.057, 0.114]	0.079 [0.036, 0.132]	0.022
Dice (validation-tuned threshold)	0.125 [0.088, 0.163]	0.120 [0.084, 0.157]	0.107	0.048
Voxel AUROC	0.707	0.701	n/a	0.655
Lesion-component recall	23.4%	22.4%	8.4%	42.7%
Seconds per volume (L4 GPU)	4.86	4.84	0.23	0.02

• More training helps a little. Validation AP at 10k...60k steps: 0.070, 0.081, 0.086, 0.087, 0.089, 0.092 (no plateau). Paired 60k minus 20k on the test scans: AP +0.0056 [+0.0021, +0.0102], Dice +0.0051 [+0.0008, +0.0110], better on 16 and 15 of 20 scans.
• Not shown better than the cheap baseline. Paired 60k minus DAE: AP +0.011 [-0.036, +0.056], Dice +0.018 [-0.032, +0.067] (indistinguishable from zero); scan-level AUROC +0.107 [+0.067, +0.146]. The diffusion model costs about 21x more per volume.

Healthy-only conformal calibration (done on the 20k model; not re-run for 60k)

Calibrated on 173 healthy IXI scans, tested on 86 other healthy scans: realised false-positive rate 0.047 (alpha 0.05) and 0.128 (alpha 0.10), both inside the 99% validity band. With a healthy-calibrated threshold the detector is nearly silent on MSLUB (Dice 0.012 at alpha 0.05) [likely dataset shift; untested]. See the calibration repository.

Bias, risks and limitations

• Dataset shift dominates. Training/calibration data (IXI) and lesion data (MSLUB) come from different scanners and populations; scan- and voxel-level scores mix domain shift with lesions.
• Small, narrow evaluation: 20 test scans, one disease (multiple sclerosis), one lesion cohort; checkpoint selection on 10 scans is noisy; results were seen before the calibration audit, so nothing here is pre-registered confirmatory evidence.
• Fairness not established. IXI has three sites (one with 7 test scans) and coarse age/sex metadata. The audit found no confirmed group disparity; one nominal association (age 40-59 vs <40, ANDi at alpha 0.10, p = 0.009) is exploratory and uncorrected for multiplicity. Absence of evidence at this sample size is not evidence of fairness.
• Only healthy adult volunteers were used for training; behaviour on paediatric, elderly-pathological or atypical anatomy is unknown.
• No MRI data is included in these repositories.

How to cite / contact

Experimental Machines, 2026. Healthy-only conformal calibration and bias audit of unsupervised brain-MRI anomaly detectors. Please cite IXI, MSLUB and Frotscher et al. (arXiv:2312.01904) as well. Questions: open a discussion on this repository.

Files

best_model.safetensors, report.pdf, comparison_report.pdf (60k vs DAE), summary.json, comparison.json, per_scan.json, dae_summary.json.

Project results, time and cost (all experiments)

All runs use Runpod: one Pod at a time, one GPU per Pod (an NVIDIA L4, or an equivalent fallback GPU when the L4 was unavailable), controlled from a laptop. "Billed" amounts are from Runpod's billing records for the specific Pods; "est." amounts are wall-clock time x list price where the Pod could not be itemised.

Time and cost

#	Stage	Date	Pod time	Cost (USD)	Basis
1	DAE baseline: preprocess, train 20k steps, benchmark, upload	Nov 1	~42 min	~0.41	est.
2	ANDi 20k steps: train, benchmark, paired comparison with DAE	Nov 1	~1 h 50 min	~1.08	est.
3	Healthy-only calibration audit (DAE, ANDi-20k)	Nov 1	~1 h 11 min	0.68	billed
4	ANDi continued to 60k steps, benchmark	Nov 1-2	~3 h 16 min	1.94	billed
5	qmr-v1 study, first Pod (hit its 4.5 h runtime cap during evaluation)	Nov 2	4 h 30 min	3.26	billed
6	qmr-v1 two resume attempts that exited at once (stale finished-job marker)	Nov 2	~6 min	0.15	billed
7	qmr-v1 resume under a fresh marker (completed)	Nov 2	~51 min	0.57	billed
8	qmr-v2: float32 audit, training replay, pAUC-vs-BCE fix study	Nov 3	~2 h 58 min	1.61	billed
	Total itemised (rows 1-8)		~15.4 h	~9.70	8.21 billed + 1.49 est.

Not itemised and therefore not in the total: several short code-sync Pods (a few cents each), earlier development Pods run before the one-Pod-at-a-time rule was added (some were duplicates), and storage on the shared network volume (Runpod bills the whole volume, which is shared with other uses; its share for this project was not measured). Cost of a single scan at inference on an L4 at list price: ANDi about 4.9 s per volume ($0.0008); DAE 0.23 s ($0.00004).

Results

All lesion results use 20 held-out MS scans (MSLUB, a different scanner vendor from the IXI training data) and are exploratory after the first look.

Experiment	Headline result	Strength
DAE baseline (not a diffusion model)	Voxel AP 0.079, Dice 0.107, scan AUROC 0.600, 0.23 s per volume	benchmark
ANDi diffusion model, 20k steps	AP 0.084, Dice 0.120, AUROC 0.701, 4.84 s per volume	benchmark
ANDi, 60k steps	AP 0.090, Dice 0.125, AUROC 0.707; +0.0056 AP vs 20k (CI +0.002 to +0.010); vs DAE AP +0.011 (CI -0.036 to +0.056, not significant)	small gain; ~21x the DAE's cost
Healthy-only conformal calibration (guarantee)	False-positive rate inside the expected band on 86 healthy test scans for every detector tested	solid (healthy data only)
Group bias audit (site, age, sex)	No confirmed group disparity; one uncorrected nominal signal (ANDi, age 40-59 vs <40)	underpowered
Healthy-calibrated threshold on MS lesions	Dice 0.012 (ANDi-20k) and 0.032 (DAE) in the audit; ~0.02 for raw scores of every detector in qmr-v1	consistent across runs; likely scanner shift (untested)
qmr-v1: calibration-aligned objective (QMR)	Failed: Dice 0.000 at the healthy-calibrated threshold; worse than its BCE ablation (-0.0007)	clear
qmr-v1: per-voxel normalisation (CSS)	DAE Dice 0.023 -> 0.052 (+0.029, CI +0.013 to +0.048); no detectable effect for ANDi	one clear positive, n = 20
qmr-v1: synthetic lesions for model selection	Not a substitute for real labels (Dice 0.03-0.05 lower); selection signal nearly flat	exploratory
qmr-v2: float32 audit	Float16 sigmoid ceiling explains the exact zero (thresholds became finite) but not the poor detection (Dice ~0.002)	clear
qmr-v2: training replay	Learned threshold trailed the healthy tail while both saturated (one seed)	supports the failure mechanism in that run
qmr-v2: pre-specified fix (partial-AUC)	Failed: collapsed to a constant score; MSLUB Dice 0.0001 vs 0.021 for BCE on logits	clear
qmr-v2: BCE on float32 logits	Dice 0.029 and 0.014 (mean 0.021; DAE 0.023); original seed re-scored gave 0.0015	exploratory; seed spread as large as the effect