|
Download README.md from zllhhhh/organic3d-encoder: direct link, hf CLI and curl.
- Browser
- Download file 3.45 kB
-
https://huggingface.co/zllhhhh/organic3d-encoder/resolve/main/README.md
- Command line
-
hf download hf://zllhhhh/organic3d-encoder/README.md
-
curl -L -o README.md https://huggingface.co/zllhhhh/organic3d-encoder/resolve/main/README.md
3.45 kB
| license: cc-by-nc-sa-4.0 | |
| library_name: pytorch | |
| pipeline_tag: zero-shot-classification | |
| tags: | |
| - 3D CT | |
| - computed tomography | |
| - vision-language | |
| - medical imaging | |
| - radiology | |
| datasets: | |
| - ibrahimhamamci/CT-RATE | |
| # ORGANIC-3D encoder (`enc4_organic3d`) | |
| A 3D CT image–text encoder that scores any sentence against a CT volume and follows the image evidence for each | |
| finding: its score drops when the finding's organ is erased, stays when another organ is erased, and prefers the | |
| sentence that names the correct side. Paper: [*ORGANIC-3D: Grounding 3D CT Vision Language Models in Organ-Level Image–Text Coupling*](https://arxiv.org/abs/2611.23529). | |
| · Code: https://github.com/lzl199704/organic3d | |
| · Benchmark: https://huggingface.co/datasets/zllhhhh/organic3d-benchmark | |
| · VLM: https://huggingface.co/zllhhhh/organic3d-vlm | |
| **Research use only. Not for clinical use.** | |
| ## Usage | |
| ```bash | |
| pip install -e <organic3d repository> # the `organic` package and its requirements | |
| python organic3d.py encoder --ct scan.nii.gz --region chest --totalseg \ | |
| --sentence "There is a nodule in the right upper lobe." | |
| ``` | |
| The script downloads this repository, resamples the CT to 1 mm RAS, centres a 384 mm crop with TotalSegmentator (or an | |
| estimate without it) and prints the score of every finding of the region (Δf = S(V, "There is f.") − S(V, "No f.")) | |
| and of each `--sentence`. From Python: `from organic3d import Encoder`. | |
| ## Model | |
| | | | | |
| |---|---| | |
| | Image tower | COLIPRI 3D ViT (Primus-M), 192³ input at 2 mm, fine-tuned | | |
| | Anatomy tokens | 144 learned queries (16 organ groups × 8, left and right lung and kidney separate, + 16 global), 2 cross-attention layers | | |
| | Score | each sentence is routed to the tokens of the organs it names and pools them with text-conditioned weights; cosine to the CXR-BERT sentence embedding / temperature | | |
| | Files | `model.safetensors` (272 M parameters, fp32), `config.json`, `organic3d.py` | | |
| ## Training | |
| Public data only: the CT-RATE and Merlin training splits (3 : 2). Contrastive loss, classification loss and the four | |
| ORGANIC losses (organ-erasure with a control organ, mirror, matched-pair, anatomy-attention), 12,000 steps on 4 × H100. | |
| Config: `configs/encoder/enc4_organic3d.yaml` in the code repository. | |
| ## Results (paper Table 1, %) | |
| | | CT-RATE | RAD-ChestCT | Merlin | | |
| |---|---|---|---| | |
| | MLC-AUROC | 87.7 | 79.7 | 87.7 | | |
| | Matched-pair AUROC (MP) | 82.5 | 78.2 | 86.5 | | |
| | Organ-erasure ΔAUROC (OE) | 28.5 | 18.4 | 21.3 | | |
| | Laterality accuracy (Side) | 76.9 | – | 63.8 | | |
| Lung lobe erasure hit rate 74.7 (CT-RATE); lesion AUROC 79.5 and lesion-erasure hit rate 74.9 (ReXGroundingCT); | |
| zero-shot tumour detection 75.0 and lesion-erasure hit rate 70.7 (AbdomenAtlas 3.0). | |
| ## Limitations | |
| Trained on chest CT from one centre (CT-RATE) and abdominal CT from one centre (Merlin) with report-derived labels; | |
| performance elsewhere is unknown. Laterality is far from perfect (Side 76.9 / 63.8). Licence terms of COLIPRI, CT-RATE | |
| and Merlin apply. | |
| ## Citation | |
| ```bibtex | |
| @article{liu2026organic3d, | |
| title = {ORGANIC3D: Grounding 3D CT Vision Language Models in Organ Level Image Text Coupling}, | |
| author = {Liu, Zelong and Ahmed, Hamza and Lee, Wynn Wingyi and Torjman, Mickael and Dimitrakakis, Alexander and | |
| Lu, Jiahua and Deyer, Timothy and Taouli, Bachir and Fayad, Zahi A. and Mei, Xueyan}, | |
| journal = {arXiv preprint arXiv:2611.23529}, | |
| year = {2026} | |
| } | |
| ``` | |