---
language:
- en
library_name: CAPTOR
tags:
- medical
- radiology
- computed-tomography
- vision-language
- contrastive-learning
- 3d
- pytorch
pipeline_tag: image-feature-extraction
---

# CAPTOR

**CAPTOR** (Chest Abdomen Pelvis foundaTion model for Oncologic Radiology) is a CT vision-language model. It is trained CLIP-style to align 3D CT regions (thorax, abdomen, pelvis, ...) with the corresponding parts of radiology reports.

- **Image encoder:** 3D-inflated ResNet-152 (I3ResNet)
- **Text encoder:** Clinical-Longformer
- **Architecture:** based on [Merlin](https://github.com/StanfordMIMI/Merlin)
- **Code:** https://github.com/Guerbet-AI/CAPTOR
- **Paper:** [CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT](https://arxiv.org/abs/2610.14226)

## Checkpoints

Each checkpoint is in its own subfolder. Only the one you request is downloaded.

`CAPTOR_30K` : ~30K CT studies from our proprietray dataset 

`CAPTOR_100K` : ~100K CT studies from our proprietray dataset and public datasets : Merlin, CT-RATE, INSPECT, BIMCV-R 

## Quick start

```bash
pip install git+https://github.com/Guerbet-AI/CAPTOR.git
```

```python
from captor import MerlinArchitecture

model = MerlinArchitecture.from_pretrained("Guerbet-AI/CAPTOR", subfolder="CAPTOR_30K")
print(model)
```

Weights, config and tokenizer are fetched automatically. No ResNet-152 or Clinical-Longformer initialization weights are needed.

The model is returned in eval mode. Call `model.train()` to fine-tune.

### Image embeddings only

```python
model = MerlinArchitecture.from_pretrained(
    "Guerbet-AI/CAPTOR", subfolder="CAPTOR_30K", ImageEmbedding=True
)
```

### Offline use

On a machine with internet:

```python
from huggingface_hub import snapshot_download

snapshot_download("Guerbet-AI/CAPTOR", local_dir="./captor_weights",
                  allow_patterns=["tokenizer/*", "CAPTOR_30K/*"])
```

Copy the folder to the offline machine, then:

```python
model = MerlinArchitecture.from_pretrained("./captor_weights", subfolder="CAPTOR_30K")
```

## Input preprocessing

The model expects CT volumes cropped around an anatomical region. Segmentation with TotalSegmentator, crop-center extraction and the dataloaders are part of the pipeline in the [GitHub repository](https://github.com/Guerbet-AI/CAPTOR), which also documents the expected data format. Loading the model alone does not preprocess raw NIfTI files.

## License and acknowledgements

CAPTOR builds on Merlin, Clinical-Longformer, torchvision's ResNet-152 and TotalSegmentator. Please respect their licenses.

## Citation

```bibtex
@misc{jeanmactoux2026captorvisionlanguagefoundationmodel,
      title={CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT}, 
      author={Séléna Jean-Mactoux and Sophie Loizillon and Ghina Jardali and Amandine Crombé and Nathalie Lassau and Alexandre Bône},
      year={2026},
      eprint={2610.14226},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.14226}, 
}
```