Second Eye: FracAtlas state head (C-RADIOv2-H)

README.md

@@ -0,0 +1,87 @@
+---
+license: apache-2.0
+library_name: pytorch
+pipeline_tag: image-classification
+tags:
+- medical-imaging
+- x-ray
+- fracture-detection
+- contrastive
+- clm
+- research-prototype
+datasets:
+- FracAtlas
+---
+
+# Second Eye: CLM image state head for fracture detection (FracAtlas, C-RADIOv2-H)
+
+> **Research prototype, not a medical device.** Not validated for clinical use. Do not use it to
+> diagnose or treat anyone.
+
+A small trainable image **state head** for [CLM (Contrastive Language Models)](https://github.com/Contrastive-LM/CLM).
+Frozen `nvidia/C-RADIOv2-H` image features (the vision tower of NVIDIA Nemotron Nano 2 VL) go
+through this head into CLM's 512-d action space. There they are scored by `scale · cos` against
+CLM's frozen **action tower** (Qwen3-8B last-token pooling + the released
+[CLM-v0.1-8B](https://huggingface.co/Contrastive-LM/CLM-v0.1-8B) action head) embedding the
+answer options:
+
+- Question: *"Is there a bone fracture in this X-ray?"*
+- Options: *"Yes, there is a fracture."* / *"No, there is no fracture."*
+
+Code, training and evaluation: [https://github.com/HarvinderBhullar/second-eye/tree/hackathon](https://github.com/HarvinderBhullar/second-eye/tree/hackathon) (see `docs/VISION.md`).
+
+## Files
+
+| file | |
+|---|---|
+| `model.safetensors` | head weights (incl. train-set input standardisation) and `options`: the 2 frozen option vectors it was trained against |
+| `config.json` | architecture (3840 → 1024 → 512, GELU), encoder, tile size, pooling, question and options |
+| `calibration.json` | Platt scaling (`a`, `b`) and two thresholds on calibrated p(fracture), all fit on the validation split |
+
+## Usage
+
+```bash
+git clone -b hackathon https://github.com/HarvinderBhullar/second-eye && cd second-eye
+scripts/vision/step1_env.sh
+python -m clm.vision.serve --model harvinderbhullar/second-eye --port 8701
+curl -s -X POST localhost:8701/predict -F image=@xray.jpg
+```
+
+Input: one `512×512` RGB tile per image (greyscale X-rays converted to
+RGB, bicubic resize), RADIO summary vector, L2-normalised. Calibrated probabilities and the
+decision (`p ≥ 0.0675`, the val operating point with sensitivity ≥ 0.90) are
+valid only for the two trained options. Reworded options keep the ranking but not the calibration.
+
+## Evaluation
+
+FracAtlas after integrity cleaning (truncated JPEGs, uniform bottom fills and byte duplicates
+removed): 3,938 images, 70/15/15 split stratified on fracture, seed 42. Every choice was made on
+validation. The test split (591 images, 104 fractured) was evaluated
+once. AUROC CI: 1,000-sample stratified bootstrap.
+
+| method | test AUROC [95% CI] | sens / spec @ val ≥0.90-sens threshold | ECE raw → Platt |
+|---|---:|---:|---:|
+| CLM head · radio_summary | 0.925 [0.891, 0.953] | 0.913 / 0.723 | 0.062 → 0.020 |
+| Linear probe · radio_summary | 0.893 [0.850, 0.931] | 0.885 / 0.612 | 0.085 → 0.041 |
+| Resolution only (leakage check) | 0.616 [0.577, 0.656] | 0.933 / 0.269 | 0.323 → 0.025 |
+| (a) random option vectors · radio_summary | 0.913 [0.878, 0.945] | 0.875 / 0.733 | 0.075 → 0.027 |
+
+**Limitations.**
+- With two fixed options, random option vectors do about as well as the CLM option embeddings
+  (see the last row). The head is effectively a classifier, and its value is the CLM interface,
+  not accuracy.
+- The head is ahead of a linear probe in point estimate, but the CIs overlap.
+- It is trained on one public dataset from a limited set of sources, with no patient IDs and no
+  external validation.
+- Image size alone reaches about 0.62 AUROC on this data (a body-region confounder).
+
+## Requirements and licences
+
+- These head weights: Apache-2.0.
+- Using them requires [`nvidia/C-RADIOv2-H`](https://huggingface.co/nvidia/C-RADIOv2-H) (NVIDIA,
+  under its own licence) for the features, and Qwen3-8B plus the CLM action head only for custom
+  options.
+- Training data: **FracAtlas**, Abedeen et al., *Scientific Data* (2023),
+  [doi:10.6084/m9.figshare.22363012](https://doi.org/10.6084/m9.figshare.22363012), CC BY 4.0.
+- Built on [Contrastive-LM/CLM](https://github.com/Contrastive-LM/CLM) (Apache-2.0) at the Nebius × NVIDIA hackathon,
+  Berlin, October 2026.

calibration.json

@@ -0,0 +1,9 @@
+{
+  "variant": "radio_summary",
+  "platt": {
+    "a": 0.48017218708992004,
+    "b": -0.5284084677696228
+  },
+  "threshold_youden": 0.3394530117511749,
+  "threshold_sens90": 0.0674779936671257
+}

config.json

@@ -0,0 +1,19 @@
+{
+  "in_dim": 3840,
+  "out_dim": 512,
+  "hidden": 1024,
+  "variant": "radio_summary",
+  "encoder": "nvidia/C-RADIOv2-H",
+  "tile": 512,
+  "pooling": "RADIO summary vector",
+  "question": "Is there a bone fracture in this X-ray?",
+  "options": [
+    "Yes, there is a fracture.",
+    "No, there is no fracture."
+  ],
+  "positive_option": 0,
+  "action_tower": {
+    "encoder": "Qwen/Qwen3-8B (last-token pooling)",
+    "head": "Contrastive-LM/CLM-v0.1-8B/CLM_v0.1-8B.pt"
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a48d1787a88cc6bcda5935a95dc974b0cae55ac15284d451f055b2fe713b85f2
+size 17867364