CXRFE — Chest X-ray Fact Encoder

CXRFE is a radiology fact encoder for chest X-ray report text. It embeds factual statements (and short report phrases) into a 128-dimensional projected embedding space for retrieval, ranking, NLI-style comparison, and fact-level evaluation metrics.

It is part of the two-stage Extracting and Encoding framework from Findings of ACL 2024:

1. Fact extraction — pamessina/T5FactExtractor
2. Fact encoding — this model (pamessina/CXRFE)

Paper: Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation

Model details

Architecture	CXR-BERT (CXRBertModel) with a projection head
Initialized from	microsoft/BiomedVLP-CXR-BERT-specialized
Hidden size	768
Projected embedding size	128 (projection_size)
Intended inputs	Short radiology facts / sentences (typically after fact extraction)
License	Apache 2.0

Note: This public checkpoint is trained with slightly more NLI data than the single best CXRFE variant reported in the paper. Additional paper-matched variants may be released later.

How to use

Requires trust_remote_code=True (custom CXR-BERT code from the BioViL / CXR-BERT family).

Projected embeddings (recommended)

This is the representation used by CXRFEScore (get_projected_text_embeddings):

import torch
from transformers import AutoModel, AutoTokenizer

device = "cuda" if torch.cuda.is_available() else "cpu"
model_id = "pamessina/CXRFE"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModel.from_pretrained(model_id, trust_remote_code=True).to(device)
model.eval()

texts = [
    "small right pleural effusion",
    "normal heart size",
]

inputs = tokenizer(
    texts,
    add_special_tokens=True,
    padding="longest",
    return_tensors="pt",
)
input_ids = inputs["input_ids"].to(device)
attention_mask = inputs["attention_mask"].to(device)

with torch.no_grad():
    embeddings = model.get_projected_text_embeddings(
        input_ids=input_ids,
        attention_mask=attention_mask,
    )

print(embeddings.shape)  # (batch_size, 128)

Easiest path: CXRFEScore

If you want fact extraction + encoding + report-pair scoring in one API:

pip install cxrfescore
# optional heatmaps:
pip install "cxrfescore[viz]"

from cxrfescore import CXRFEScore

metric = CXRFEScore(device="cuda")  # default encoder: pamessina/CXRFE
result = metric(
    ["There is a small right pleural effusion. The heart size is normal."],
    ["Small right pleural effusion. Normal heart size."],
)
print(result["mean_similarity"])

Demo notebook: CXR-Fact-Encoder / notebooks/cxrfescore_demo.ipynb

Related resources

• Paper hub: https://github.com/PabloMessina/CXR-Fact-Encoder
• Metric package: https://github.com/PabloMessina/CXRFEScore · PyPI
• Companion fact extractor: https://huggingface.co/pamessina/T5FactExtractor
• ACL Anthology: https://aclanthology.org/2024.findings-acl.236/
• arXiv: https://arxiv.org/abs/2407.01948

Citation

If you use CXRFE, please cite:

@inproceedings{messina-etal-2024-extracting,
    title = "Extracting and Encoding: Leveraging Large Language Models and Medical Knowledge to Enhance Radiological Text Representation",
    author = "Messina, Pablo  and
      Vidal, Rene  and
      Parra, Denis  and
      Soto, Alvaro  and
      Araujo, Vladimir",
    booktitle = "Findings of the Association for Computational Linguistics: ACL 2024",
    month = aug,
    year = "2024",
    address = "Bangkok, Thailand",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2024.findings-acl.236/",
    doi = "10.18653/v1/2024.findings-acl.236",
    pages = "3955--3986"
}