---
license: apache-2.0
library_name: pytorch
pipeline_tag: image-classification
tags:
- medical-imaging
- x-ray
- fracture-detection
- contrastive
- clm
- research-prototype
datasets:
- FracAtlas
---

# Second Eye: CLM image state head for fracture detection (FracAtlas, C-RADIOv2-H)

> **Research prototype, not a medical device.** Not validated for clinical use. Do not use it to
> diagnose or treat anyone.

A small trainable image **state head** for [CLM (Contrastive Language Models)](https://github.com/Contrastive-LM/CLM).
Frozen `nvidia/C-RADIOv2-H` image features (the vision tower of NVIDIA Nemotron Nano 2 VL) go
through this head into CLM's 512-d action space. There they are scored by `scale · cos` against
CLM's frozen **action tower** (Qwen3-8B last-token pooling + the released
[CLM-v0.1-8B](https://huggingface.co/Contrastive-LM/CLM-v0.1-8B) action head) embedding the
answer options:

- Question: *"Is there a bone fracture in this X-ray?"*
- Options: *"Yes, there is a fracture."* / *"No, there is no fracture."*

Code, training and evaluation: [https://github.com/HarvinderBhullar/second-eye/tree/hackathon](https://github.com/HarvinderBhullar/second-eye/tree/hackathon) (see `docs/VISION.md`).

## Files

| file | |
|---|---|
| `model.safetensors` | head weights (incl. train-set input standardisation) and `options`: the 2 frozen option vectors it was trained against |
| `config.json` | architecture (3840 → 1024 → 512, GELU), encoder, tile size, pooling, question and options |
| `calibration.json` | Platt scaling (`a`, `b`) and two thresholds on calibrated p(fracture), all fit on the validation split |

## Usage

```bash
git clone -b hackathon https://github.com/HarvinderBhullar/second-eye && cd second-eye
scripts/vision/step1_env.sh
python -m clm.vision.serve --model harvinderbhullar/second-eye --port 8701
curl -s -X POST localhost:8701/predict -F image=@xray.jpg
```

Input: one `512×512` RGB tile per image (greyscale X-rays converted to
RGB, bicubic resize), RADIO summary vector, L2-normalised. Calibrated probabilities and the
decision (`p ≥ 0.0675`, the val operating point with sensitivity ≥ 0.90) are
valid only for the two trained options. Reworded options keep the ranking but not the calibration.

## Evaluation

FracAtlas after integrity cleaning (truncated JPEGs, uniform bottom fills and byte duplicates
removed): 3,938 images, 70/15/15 split stratified on fracture, seed 42. Every choice was made on
validation. The test split (591 images, 104 fractured) was evaluated
once. AUROC CI: 1,000-sample stratified bootstrap.

| method | test AUROC [95% CI] | sens / spec @ val ≥0.90-sens threshold | ECE raw → Platt |
|---|---:|---:|---:|
| CLM head · radio_summary | 0.925 [0.891, 0.953] | 0.913 / 0.723 | 0.062 → 0.020 |
| Linear probe · radio_summary | 0.893 [0.850, 0.931] | 0.885 / 0.612 | 0.085 → 0.041 |
| Resolution only (leakage check) | 0.616 [0.577, 0.656] | 0.933 / 0.269 | 0.323 → 0.025 |
| (a) random option vectors · radio_summary | 0.913 [0.878, 0.945] | 0.875 / 0.733 | 0.075 → 0.027 |

**Limitations.**
- With two fixed options, random option vectors do about as well as the CLM option embeddings
  (see the last row). The head is effectively a classifier, and its value is the CLM interface,
  not accuracy.
- The head is ahead of a linear probe in point estimate, but the CIs overlap.
- It is trained on one public dataset from a limited set of sources, with no patient IDs and no
  external validation.
- Image size alone reaches about 0.62 AUROC on this data (a body-region confounder).

## Requirements and licences

- These head weights: Apache-2.0.
- Using them requires [`nvidia/C-RADIOv2-H`](https://huggingface.co/nvidia/C-RADIOv2-H) (NVIDIA,
  under its own licence) for the features, and Qwen3-8B plus the CLM action head only for custom
  options.
- Training data: **FracAtlas**, Abedeen et al., *Scientific Data* (2023),
  [doi:10.6084/m9.figshare.22363012](https://doi.org/10.6084/m9.figshare.22363012), CC BY 4.0.
- Built on [Contrastive-LM/CLM](https://github.com/Contrastive-LM/CLM) (Apache-2.0) at the Nebius × NVIDIA hackathon,
  Berlin, October 2026.