--- license: cc-by-nc-sa-4.0 library_name: pytorch pipeline_tag: zero-shot-classification tags: - 3D CT - computed tomography - vision-language - medical imaging - radiology datasets: - ibrahimhamamci/CT-RATE --- # ORGANIC-3D encoder (`enc4_organic3d`) A 3D CT image–text encoder that scores any sentence against a CT volume and follows the image evidence for each finding: its score drops when the finding's organ is erased, stays when another organ is erased, and prefers the sentence that names the correct side. Paper: [*ORGANIC-3D: Grounding 3D CT Vision Language Models in Organ-Level Image–Text Coupling*](https://arxiv.org/abs/2611.23529). · Code: https://github.com/lzl199704/organic3d · Benchmark: https://huggingface.co/datasets/zllhhhh/organic3d-benchmark · VLM: https://huggingface.co/zllhhhh/organic3d-vlm **Research use only. Not for clinical use.** ## Usage ```bash pip install -e # the `organic` package and its requirements python organic3d.py encoder --ct scan.nii.gz --region chest --totalseg \ --sentence "There is a nodule in the right upper lobe." ``` The script downloads this repository, resamples the CT to 1 mm RAS, centres a 384 mm crop with TotalSegmentator (or an estimate without it) and prints the score of every finding of the region (Δf = S(V, "There is f.") − S(V, "No f.")) and of each `--sentence`. From Python: `from organic3d import Encoder`. ## Model | | | |---|---| | Image tower | COLIPRI 3D ViT (Primus-M), 192³ input at 2 mm, fine-tuned | | Anatomy tokens | 144 learned queries (16 organ groups × 8, left and right lung and kidney separate, + 16 global), 2 cross-attention layers | | Score | each sentence is routed to the tokens of the organs it names and pools them with text-conditioned weights; cosine to the CXR-BERT sentence embedding / temperature | | Files | `model.safetensors` (272 M parameters, fp32), `config.json`, `organic3d.py` | ## Training Public data only: the CT-RATE and Merlin training splits (3 : 2). Contrastive loss, classification loss and the four ORGANIC losses (organ-erasure with a control organ, mirror, matched-pair, anatomy-attention), 12,000 steps on 4 × H100. Config: `configs/encoder/enc4_organic3d.yaml` in the code repository. ## Results (paper Table 1, %) | | CT-RATE | RAD-ChestCT | Merlin | |---|---|---|---| | MLC-AUROC | 87.7 | 79.7 | 87.7 | | Matched-pair AUROC (MP) | 82.5 | 78.2 | 86.5 | | Organ-erasure ΔAUROC (OE) | 28.5 | 18.4 | 21.3 | | Laterality accuracy (Side) | 76.9 | – | 63.8 | Lung lobe erasure hit rate 74.7 (CT-RATE); lesion AUROC 79.5 and lesion-erasure hit rate 74.9 (ReXGroundingCT); zero-shot tumour detection 75.0 and lesion-erasure hit rate 70.7 (AbdomenAtlas 3.0). ## Limitations Trained on chest CT from one centre (CT-RATE) and abdominal CT from one centre (Merlin) with report-derived labels; performance elsewhere is unknown. Laterality is far from perfect (Side 76.9 / 63.8). Licence terms of COLIPRI, CT-RATE and Merlin apply. ## Citation ```bibtex @article{liu2026organic3d, title = {ORGANIC3D: Grounding 3D CT Vision Language Models in Organ Level Image Text Coupling}, author = {Liu, Zelong and Ahmed, Hamza and Lee, Wynn Wingyi and Torjman, Mickael and Dimitrakakis, Alexander and Lu, Jiahua and Deyer, Timothy and Taouli, Bachir and Fayad, Zahi A. and Mei, Xueyan}, journal = {arXiv preprint arXiv:2611.23529}, year = {2026} } ```