medical
ct
File size: 4,609 Bytes
c6c96c7
 
 
 
 
 
 
 
 
 
 
 
 
 
de0e9d5
 
c6c96c7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
de0e9d5
 
 
 
 
 
 
 
 
 
 
c6c96c7
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
---
license: cc-by-nc-sa-4.0
datasets:
- ibrahimhamamci/CT-RATE
base_model:
- rmaguado/VoxelFM
- Qwen/Qwen3.5-9B
tags:
- medical
- ct
---

# LightAlign CT-RATE weights

Paper: [What Matters for Cross-Modal Alignment](https://arxiv.org/abs/2611.20000) (arXiv:2611.20000).

Linear map and report-side finding classifiers for 18 CT-RATE findings, from the paper "What Matters for Cross-Modal Alignment".

Code: [github.com/sypsyp97/lightalign](https://github.com/sypsyp97/lightalign) · Project page: [sypsyp97.github.io/lightalign](https://sypsyp97.github.io/lightalign/)

The image encoder is frozen VoxelFM ViT-B 3D (`rmaguado/VoxelFM`, revision `7a58beab42c0ba213f40b9dbcaae43d3488416dd`; code commit `fba41a34c3e4dcfebbe902be5e2b2ce26db27d52`). The feature is the CLS token after block 11, after the final layer norm.

The text encoder is frozen Qwen3.5-9B (`Qwen/Qwen3.5-9B`, revision `c202236235762e1c871ad0ccb60c8ee5ba337b9a`). The feature is the mean over tokens of the middle hidden state (`len(hidden_states) // 2`). The vision tower is not used. The public commit is pinned here. The command that produced the original text arrays did not pass a revision.

The map is ridge regression from image features to text features, fit on all 5,994 paired training patients (seed 0). Each modality is L2-normalized, centered by the mean of those unit vectors on the full training pool, and L2-normalized again. Ridge uses `RidgeCV` with alphas `logspace(-3, 3, 13)` and `fit_intercept=False`. There is no intercept in this file.

Each finding then has a `StandardScaler` and a logistic regression fit on the centered report embeddings and the report-derived label. A new scan is mapped into that space. `TC` is the decision function. `P` is the cosine with the positive prompt embedding minus the cosine with the negative prompt embedding. The prompt strings are `IMPRESSION:\n{label} is present.` and `IMPRESSION:\nNo {label.lower()}.`.

Files: `lightalign-ctrate-n5994-seed0.npz` and `lightalign-ctrate-n5994-seed0.json`.

## Intended use

Research on chest CT representation and zero-shot finding readout.

Not for clinical use. Not a medical device. Do not use the scores to diagnose, triage, or treat a patient.

## Training data

CT-RATE (`ibrahimhamamci/CT-RATE`), gated, license CC BY-NC-SA 4.0. Training rows are the hashed 30% of `train_fixed` reconstructions `*_a_1` after the dataset non-chest list is removed (5,994 patients: 2,024 `sel`, 1,945 `train`, 2,025 `test`). The map uses every row. Labels are the 18 report-derived abnormality columns shipped with CT-RATE. They are not a new manual annotation.

These weights inherit CC BY-NC-SA 4.0.

## How to use

Score one NIfTI volume with the `lightalign` package:

```bash
lightalign-predict --ct volume.nii.gz --weights yipengsun/lightalign-ctrate --voxelfm-code /path/to/VoxelFM
```

Prediction needs a CUDA GPU, the VoxelFM code checkout (`--voxelfm-code` or `VOXELFM_CODE`), the VoxelFM weights (downloaded from `rmaguado/VoxelFM` by default), and this file. It does not need Qwen3.5-9B. The same package rebuilds this file from CT-RATE embeddings. See the package README for the case rules, the pair counts, and the chest-unit table.

## Limitations

- Research only. The scores are not calibrated probabilities. This file does not contain an operating point. Weighted F1 and accuracy from the reproduction script use a threshold chosen on the evaluated volumes.
- Eighteen findings, taken from report text in one public dataset. A finding the report does not name is a negative label in that table. That is not a radiologist saying the finding is absent on the scan.
- The reader expects a 3D chest CT in NIfTI. Voxels are read from `dataobj` with no rescale slope or intercept. The CT-RATE fixed release already stores corrected Hounsfield units. A file that still stores unscaled values in `dataobj` will be wrong.
- The map is linear. It was fit on 5,994 pairs from CT-RATE. Other hospitals, scanners, and reconstructions were not part of the fit.
- A different Qwen3.5-9B snapshot would not match these coefficients.
- Attention uses fused scaled dot-product attention. Each process checks it once against the reference attention on a reduced volume, and stops if the cosine is below 0.9999.

## Citation

```bibtex
@article{sun2026whatmatters,
  title   = {What Matters for Cross-Modal Alignment},
  author  = {Sun, Yipeng and Wang, Jiajun and Mei, Siyuan and Xia, Yan and Bayer, Siming and Christlein, Vincent and Maier, Andreas},
  journal = {arXiv preprint arXiv:2611.20000},
  year    = {2026}
}
```

## License

cc-by-nc-sa-4.0