CXAS models: chest X-ray anatomy segmentation
Weights for CXAS (pip install cxas), which
segments the anatomy of frontal and lateral chest radiographs and writes DICOM SEG and SR objects. CXAS downloads
these files on first use and caches them in ~/.cxas/weights; the CXAS Docker image contains them already.
Research use only. Not a medical device; not for clinical decisions.
Licence: MIT.
Contents
| Path | Model | Classes | Encoder |
|---|---|---|---|
M2F_anatomy_v2/frontal/ |
Mask2Former, ViT-CoMer pyramid, 200 queries | 187 | DINOv3 ViT-S/16 (frozen) |
M2F_anatomy_v2/lateral/ |
Mask2Former, ViT-CoMer pyramid, 200 queries | 177 | RAD-DINO ViT-B/14 |
UNet_ResNet50_default.pth |
the original CXAS UNet (CXAS <= 0.0.20) | 159 | ResNet-50 |
Each M2F_anatomy_v2/<view>/ bundle holds model.safetensors (decoder weights), backbone/ (the encoder in
Hugging Face format), config.yaml (training configuration), categories.json (label space), adapter.json
(label corrections applied after inference) and meta.json (source run and checkpoint hash). SHA256SUMS lists a
checksum for every file; CXAS verifies them after download.
Usage
from cxas import CXAS
model = CXAS(model_name="M2F_anatomy_v2", gpus="0") # or gpus="cpu"
pred = model.process_file("image.dcm") # view from DICOM ViewPosition, else detected
cxas_idc -i /data/dicom -o /data/out # per radiograph: DICOM SEG + landmark SR + result.json
The 159 classes of the original CXAS label space come out in segmentation_preds; the frontal model's further
classes (regions below the hemidiaphragms, image quadrants, humeri, head, lung subdivisions) in
extra_segmentation.
Training data
- Frontal (187 classes): started from models trained on PAXRay++ (masks of 177 structures projected from CT), then extended class-incrementally on MIMIC-CXR frontal images to 187 classes (head, humeri, regions below the hemidiaphragms, four image quadrants). The MIMIC-CXR labels are model-generated pseudo-labels, not manual annotations. The label adapter corrects side names that are swapped in that training data.
- Lateral (177 classes): PAXRay++ only, including its lateral projections. PAXRay++ labels lateral ribs per level without a side, and its lateral "upper zone lung" and "lung base" are swapped; the adapter swaps them back and derives the whole lung and the lobes from the zones.
- Image quadrants: the four quadrant classes tile the whole radiograph at the midline and mid-thorax; they are image quadrants, not abdominal quadrants. Sides are the patient's.
Evaluation (PAXRay++ test split, synthetic)
Deployment settings: shortest edge 512 px, best-scoring instance per class above 0.05. Point estimates.
| View | Images | Model | Mask AP | Dice of the output mask |
|---|---|---|---|---|
| Frontal | 735 | frontal (187) | 42.4 | 0.719 |
| Lateral | 740 | lateral (177) | 45.0 | 0.746 |
PAXRay++ is synthetic and in-domain for the PAXRay-trained models; it under-rates the frontal model, which was adapted to real radiographs. On real chest radiographs from the Imaging Data Commons the frontal model is the most robust of the candidates we compared (it alone segments a radiograph stored rotated by 90 degrees), but no manual ground truth exists there.
Limitations
- Laterals: no left/right rib classes (PAXRay++ has none on laterals); anterior ribs 10-11 are weak (Dice ~0.4).
- Classes nest (e.g.
lungcontainsleft lungcontainsleft lung base), so masks overlap by design. - Masks are predicted at stride 4 of a 512 px input and upsampled once to native resolution.
Citation
@inproceedings{Seibold_2022_BMVC,
author = {Seibold, Constantin Marc and Reiss, Simon and Sarfraz, M. Saquib and Fink, Matthias A. and Mayer, Victoria and Sellner, Jan and Kim, Moon Sung and Maier-Hein, Klaus H. and Kleesiek, Jens and Stiefelhagen, Rainer},
title = {Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding},
booktitle = {33rd British Machine Vision Conference (BMVC)},
year = {2022}
}
@article{seibold2023accurate,
title = {Accurate fine-grained segmentation of human anatomy in radiographs via volumetric pseudo-labeling},
author = {Seibold, Constantin and Jaus, Alexander and Fink, Matthias A and Kim, Moon and Reiss, Simon and Herrmann, Ken and Kleesiek, Jens and Stiefelhagen, Rainer},
journal = {arXiv preprint arXiv:2306.03934},
year = {2023}
}