VoxTell fine-tuned for report-sentence grounding in chest CT (final model)
Part of the master's thesis From Scan to Report: Detecting, Segmenting and Reporting Findings in Chest CT (code: https://github.com/Switchharsh/Switchharsh-From_Scan_to_Report_Master_Thesis; thesis: https://github.com/Switchharsh/Switchharsh-From_Scan_to_Report_Master_Thesis/tree/main/thesis).
Given a chest CT and one sentence from its radiology report (for example "Ground glass opacity in the posterobasal segment of the right lower lobe"), the model segments the finding the sentence describes. It is VoxTell v1.1 (Rokuss et al., CVPR 2026) fine-tuned on the ReXGroundingCT training split. This is the final segmentation model of the thesis.
Files
voxtell_thesis_final/
plans.json # identical to VoxTell v1.1
fold_0/checkpoint_final.pth # {"network_weights": ...}, same keys and architecture as VoxTell v1.1
The checkpoint holds only the network weights (optimizer state removed); they are bit-identical to the evaluated model. The text encoder (Qwen3-Embedding-4B) was frozen and is not included; VoxTell loads it, or its precomputed embedding bank, as usual.
Training
- Data: ReXGroundingCT training split (2,992 scans), native voxel spacing, volumes cropped to the non-zero region and z-score normalised.
- All VoxTell weights trained (image encoder, decoder, text projection and the prompt decoder); text encoder frozen.
- One in three examples is an absent-prompt negative (a sentence about a finding not in the scan, with an empty target). Other crops are centred within 48 voxels of a foreground voxel with probability 0.85, random otherwise. No augmentation (sentences name the side, so no flipping).
- Loss: binary cross-entropy + soft Dice, deep supervision at five scales (weights 1, 1/2, 1/4, 1/8, 1/16, normalised); every voxel is supervised.
- AdamW, learning rate 1.5e-5, weight decay 1e-4, cosine decay to 1e-6, 30 epochs, effective batch 3 (3 GPUs), bfloat16, gradient clipping 1.0.
- Weights are the uniform average of epochs 20-29.
Results (ReXGroundingCT validation, 200 cases, organisers' evaluator)
| model | sliding-window step | Dice per case | hit rate | instance F1 |
|---|---|---|---|---|
| VoxTell v1.1, zero-shot | 0.5 | 0.2023 | 0.467 | |
| this model | 0.15 | 0.3007 | 0.643 | 0.205 |
| this model | 0.5 | 0.2874 | 0.638 | 0.190 |
Hit rate = share of findings with Dice of at least 0.1. Use a sliding-window step of 0.15 to reproduce the thesis numbers (+0.013 Dice over step 0.5 for this model). All numbers are on the validation split; test labels are not public.
Usage
Use it like the released VoxTell model directory, e.g. with the VoxTell CLI (pip install git+https://github.com/MIC-DKFZ/VoxTell.git):
huggingface-cli download suitch/scan2report-voxtell-grounding --local-dir scan2report-voxtell-grounding
voxtell-predict -i scan.nii.gz -o out -m scan2report-voxtell-grounding/voxtell_thesis_final \
-p "Ground glass opacity in the posterobasal segment of the right lower lobe"
The VoxTell CLI uses its default sliding-window step. The thesis predictions (step 0.15) were made with rexgrounding predict --adapter voxtell --tile-step-size 0.15
from the code repository (ReXGrounding/scripts/run_val_full2swa.slurm).
Limitations
- Research model, not for clinical use.
- Trained and evaluated on CT-RATE scans (one hospital population) with ReXGroundingCT masks; training masks mark at most three instances per finding.
- Weak on diffuse findings such as emphysema, and on findings without a clear boundary; mean Dice of about 0.30 means many masks are only partly correct.
License and citation
CC BY-NC-SA 4.0, inherited from VoxTell, ReXGroundingCT and CT-RATE. Non-commercial use only. Please cite VoxTell and ReXGroundingCT when you use this model:
@inproceedings{rokuss2026voxtell,
title={Voxtell: Free-text promptable universal 3d medical image segmentation},
author={Rokuss, Maximilian and Langenberg, Moritz and Kirchhoff, Yannick and Isensee, Fabian and Hamm, Benjamin and Ulrich, Constantin and Regnery, Sebastian and Bauer, Lukas and Katsigiannopulos, Efthimios and Norajitra, Tobias and Maier-Hein, Klaus},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={37538--37557},
year={2026}
}
Model tree for suitch/scan2report-voxtell-grounding
Base model
mrokuss/VoxTell