CXAS models: chest X-ray anatomy segmentation

Weights for CXAS (pip install cxas), which segments the anatomy of frontal and lateral chest radiographs and writes DICOM SEG and SR objects. CXAS downloads these files on first use and caches them in ~/.cxas/weights; the CXAS Docker image contains them already.

Research use only. Not a medical device; not for clinical decisions.

Licence: MIT.

Contents

Path Model Classes Encoder
M2F_anatomy_v2/frontal/ Mask2Former, ViT-CoMer pyramid, 200 queries 187 DINOv3 ViT-S/16 (frozen)
M2F_anatomy_v2/lateral/ Mask2Former, ViT-CoMer pyramid, 200 queries 177 RAD-DINO ViT-B/14
UNet_ResNet50_default.pth the original CXAS UNet (CXAS <= 0.0.20) 159 ResNet-50

Each M2F_anatomy_v2/<view>/ bundle holds model.safetensors (decoder weights), backbone/ (the encoder in Hugging Face format), config.yaml (training configuration), categories.json (label space), adapter.json (label corrections applied after inference) and meta.json (source run and checkpoint hash). SHA256SUMS lists a checksum for every file; CXAS verifies them after download.

Usage

from cxas import CXAS
model = CXAS(model_name="M2F_anatomy_v2", gpus="0")   # or gpus="cpu"
pred = model.process_file("image.dcm")                # view from DICOM ViewPosition, else detected
cxas_idc -i /data/dicom -o /data/out    # per radiograph: DICOM SEG + landmark SR + result.json

The 159 classes of the original CXAS label space come out in segmentation_preds; the frontal model's further classes (regions below the hemidiaphragms, image quadrants, humeri, head, lung subdivisions) in extra_segmentation.

Training data

  • Frontal (187 classes): started from models trained on PAXRay++ (masks of 177 structures projected from CT), then extended class-incrementally on MIMIC-CXR frontal images to 187 classes (head, humeri, regions below the hemidiaphragms, four image quadrants). The MIMIC-CXR labels are model-generated pseudo-labels, not manual annotations. The label adapter corrects side names that are swapped in that training data.
  • Lateral (177 classes): PAXRay++ only, including its lateral projections. PAXRay++ labels lateral ribs per level without a side, and its lateral "upper zone lung" and "lung base" are swapped; the adapter swaps them back and derives the whole lung and the lobes from the zones.
  • Image quadrants: the four quadrant classes tile the whole radiograph at the midline and mid-thorax; they are image quadrants, not abdominal quadrants. Sides are the patient's.

Evaluation (PAXRay++ test split, synthetic)

Deployment settings: shortest edge 512 px, best-scoring instance per class above 0.05. Point estimates.

View Images Model Mask AP Dice of the output mask
Frontal 735 frontal (187) 42.4 0.719
Lateral 740 lateral (177) 45.0 0.746

PAXRay++ is synthetic and in-domain for the PAXRay-trained models; it under-rates the frontal model, which was adapted to real radiographs. On real chest radiographs from the Imaging Data Commons the frontal model is the most robust of the candidates we compared (it alone segments a radiograph stored rotated by 90 degrees), but no manual ground truth exists there.

Limitations

  • Laterals: no left/right rib classes (PAXRay++ has none on laterals); anterior ribs 10-11 are weak (Dice ~0.4).
  • Classes nest (e.g. lung contains left lung contains left lung base), so masks overlap by design.
  • Masks are predicted at stride 4 of a 512 px input and upsampled once to native resolution.

Citation

@inproceedings{Seibold_2022_BMVC,
  author    = {Seibold, Constantin Marc and Reiss, Simon and Sarfraz, M. Saquib and Fink, Matthias A. and Mayer, Victoria and Sellner, Jan and Kim, Moon Sung and Maier-Hein, Klaus H. and Kleesiek, Jens and Stiefelhagen, Rainer},
  title     = {Detailed Annotations of Chest X-Rays via CT Projection for Report Understanding},
  booktitle = {33rd British Machine Vision Conference (BMVC)},
  year      = {2022}
}
@article{seibold2023accurate,
  title   = {Accurate fine-grained segmentation of human anatomy in radiographs via volumetric pseudo-labeling},
  author  = {Seibold, Constantin and Jaus, Alexander and Fink, Matthias A and Kim, Moon and Reiss, Simon and Herrmann, Ken and Kleesiek, Jens and Stiefelhagen, Rainer},
  journal = {arXiv preprint arXiv:2306.03934},
  year    = {2023}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Paper for cmseibold/cxas-models