Spaces:
Sleeping
Sleeping
Commit ·
a185e27
1
Parent(s): 0e91bec
Initial deployment — Angio AI
Browse files- README.md +52 -9
- app.py +1598 -0
- packages.txt +7 -0
- requirements.txt +28 -0
README.md
CHANGED
|
@@ -1,15 +1,58 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: gradio
|
| 7 |
-
sdk_version:
|
| 8 |
-
python_version: '3.13'
|
| 9 |
app_file: app.py
|
| 10 |
pinned: false
|
| 11 |
-
license:
|
| 12 |
-
short_description: Automation of Angiography Segment and Stenosis Detection
|
| 13 |
---
|
| 14 |
|
| 15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: Angio AI
|
| 3 |
+
emoji: 🫀
|
| 4 |
+
colorFrom: red
|
| 5 |
+
colorTo: blue
|
| 6 |
sdk: gradio
|
| 7 |
+
sdk_version: 4.31.5
|
|
|
|
| 8 |
app_file: app.py
|
| 9 |
pinned: false
|
| 10 |
+
license: mit
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
+
# Angio AI — Coronary Angiography Analysis System
|
| 14 |
+
|
| 15 |
+
**MS Data Science Thesis · Information Technology University Lahore**
|
| 16 |
+
**Supervisor: Dr. Arif Mehmood**
|
| 17 |
+
|
| 18 |
+
---
|
| 19 |
+
|
| 20 |
+
## What it does
|
| 21 |
+
|
| 22 |
+
Upload a coronary X-ray angiography (XCA) video and the pipeline automatically:
|
| 23 |
+
|
| 24 |
+
| Step | Module | Output |
|
| 25 |
+
|------|--------|--------|
|
| 26 |
+
| 1 | Keyframe extraction | Best diagnostic frame (contrast × sharpness score) |
|
| 27 |
+
| 2 | Mask2Former | Stenosis detection — bounding boxes + instance masks |
|
| 28 |
+
| 3a | ResUNet | Binary vessel mask (Dice 0.8015 on ARCADE) |
|
| 29 |
+
| 3b | YOLOv8m-seg | 26-class coronary anatomy segmentation |
|
| 30 |
+
| 4 | FFR Pipeline v4 | Quantitative Flow Ratio (QFR) estimation |
|
| 31 |
+
| 5 | SYNTAX Score | Lesion complexity scoring |
|
| 32 |
+
|
| 33 |
+
---
|
| 34 |
+
|
| 35 |
+
## Models
|
| 36 |
+
|
| 37 |
+
All checkpoints are stored in [`MuhammadAdil63/angio-ai-checkpoints`](https://huggingface.co/MuhammadAdil63/angio-ai-checkpoints) and downloaded automatically on first run.
|
| 38 |
+
|
| 39 |
+
| File | Architecture | Task | Dice / mAP |
|
| 40 |
+
|------|-------------|------|-----------|
|
| 41 |
+
| `mask2former_best.pth` | Mask2Former Swin-Base | Stenosis detection | — |
|
| 42 |
+
| `binary_best.pth` | ResUNet (16→256 ch) | Binary vessel seg | 0.8015 |
|
| 43 |
+
| `best.pt` | YOLOv8m-seg nc=26 | 26-class anatomy seg | — |
|
| 44 |
+
|
| 45 |
+
Dataset: [ARCADE Challenge](https://arcade.grand-challenge.org/) (syntax + stenosis splits)
|
| 46 |
+
|
| 47 |
+
---
|
| 48 |
+
|
| 49 |
+
## Notes
|
| 50 |
+
|
| 51 |
+
- **Hardware**: CPU-only (HF free tier) — inference takes ~30–60 seconds per video
|
| 52 |
+
- **Input**: MP4 / AVI coronary angiography video, ideally ≥5 seconds
|
| 53 |
+
- **Scale**: Default 3.75 px/mm (ARCADE hardware). Adjust slider for non-ARCADE data.
|
| 54 |
+
- **FFR formula**: QFR = 1 − (0.33·DS + 0.60·DS²) — Tu et al. JACC 2016 / FAVOR II
|
| 55 |
+
|
| 56 |
+
---
|
| 57 |
+
|
| 58 |
+
*This tool is for research purposes only and is not intended for clinical use.*
|
app.py
ADDED
|
@@ -0,0 +1,1598 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ── angio_ai_app.py ─────────────────────────────────────────────────────────────────────────────
|
| 2 |
+
# Angio AI — Coronary Angiography Analysis System
|
| 3 |
+
# Muhammad Adil · MS Data Science · ITU Lahore
|
| 4 |
+
# Supervisor: Dr. Arif Mehmood
|
| 5 |
+
#
|
| 6 |
+
# ── DEPLOYMENT — HuggingFace Spaces ──────────────────────────────────────────
|
| 7 |
+
# Checkpoints are loaded from HF Hub: MuhammadAdil63/angio-ai-checkpoints
|
| 8 |
+
# Set HF_TOKEN as a Space Secret (Settings → Secrets) for private repo access.
|
| 9 |
+
# Files expected in that repo:
|
| 10 |
+
# mask2former_best.pth (Mask2Former Swin-Base fine-tuned)
|
| 11 |
+
# binary_best.pth (ResUNet binary vessel segmentation)
|
| 12 |
+
# best.pt (YOLOv8m-seg 26-class ARCADE syntax)
|
| 13 |
+
#
|
| 14 |
+
# ── DEPENDENCIES ────────────────────────────────────────────────────────────────────────────
|
| 15 |
+
# pip install gradio torch torchvision opencv-python-headless
|
| 16 |
+
# scikit-image scipy matplotlib transformers
|
| 17 |
+
# ultralytics (for YOLOv8m-seg inference)
|
| 18 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 19 |
+
|
| 20 |
+
import os
|
| 21 |
+
import sys
|
| 22 |
+
import json
|
| 23 |
+
import base64
|
| 24 |
+
import tempfile
|
| 25 |
+
import warnings
|
| 26 |
+
warnings.filterwarnings("ignore")
|
| 27 |
+
|
| 28 |
+
import cv2
|
| 29 |
+
import numpy as np
|
| 30 |
+
import matplotlib
|
| 31 |
+
matplotlib.use("Agg")
|
| 32 |
+
import matplotlib.pyplot as plt
|
| 33 |
+
import matplotlib.patches as mpatches
|
| 34 |
+
from pathlib import Path
|
| 35 |
+
|
| 36 |
+
import gradio as gr
|
| 37 |
+
import torch
|
| 38 |
+
import huggingface_hub # noqa: F401 (ensure installed; used in _hf_ckpt)
|
| 39 |
+
|
| 40 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 41 |
+
# CONFIG — edit these paths before running
|
| 42 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 43 |
+
|
| 44 |
+
# ── Checkpoint download from HuggingFace Hub ─────────────────────────────────
|
| 45 |
+
# Checkpoints live in: MuhammadAdil63/angio-ai-checkpoints (private repo)
|
| 46 |
+
# huggingface_hub reads HF_TOKEN from environment (set as a Space secret).
|
| 47 |
+
def _hf_ckpt(filename: str) -> str:
|
| 48 |
+
"""Download checkpoint from HF Hub on first call, return local cache path."""
|
| 49 |
+
from huggingface_hub import hf_hub_download
|
| 50 |
+
return hf_hub_download(
|
| 51 |
+
repo_id = "MuhammadAdil63/angio-ai-checkpoints",
|
| 52 |
+
filename = filename,
|
| 53 |
+
repo_type= "model",
|
| 54 |
+
token = os.environ.get("HF_TOKEN"), # set in Space Secrets
|
| 55 |
+
)
|
| 56 |
+
|
| 57 |
+
CONFIG = {
|
| 58 |
+
# ── Checkpoint paths (resolved lazily via HF Hub) ─────────────────────────
|
| 59 |
+
# These callables are invoked once inside each model's _load_*() function.
|
| 60 |
+
"MASK2FORMER_CKPT" : lambda: _hf_ckpt("mask2former_best.pth"),
|
| 61 |
+
"RESUNET_CKPT" : lambda: _hf_ckpt("binary_best.pth"),
|
| 62 |
+
"YOLO_CKPT" : lambda: _hf_ckpt("best.pt"),
|
| 63 |
+
|
| 64 |
+
# ── YOLO inference params ─────────────────────────────────────────────────
|
| 65 |
+
"YOLO_CONF" : 0.25,
|
| 66 |
+
"YOLO_IOU" : 0.70,
|
| 67 |
+
"YOLO_IMGSZ" : 512,
|
| 68 |
+
|
| 69 |
+
# ── FFR pipeline scale (ARCADE hardcoded) ────────────────────────────────
|
| 70 |
+
"PX_PER_MM" : 3.75,
|
| 71 |
+
|
| 72 |
+
# ── Device — CPU on HF free tier ─────────────────────────────────────────
|
| 73 |
+
"DEVICE" : "cuda:0" if torch.cuda.is_available() else "cpu",
|
| 74 |
+
}
|
| 75 |
+
|
| 76 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 77 |
+
# ITU LOGO — encode to base64 for embedding in HTML
|
| 78 |
+
# Place ITULog.png in the same directory as this script.
|
| 79 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 80 |
+
|
| 81 |
+
def _encode_logo(path: str) -> str:
|
| 82 |
+
try:
|
| 83 |
+
with open(path, "rb") as f:
|
| 84 |
+
return "data:image/png;base64," + base64.b64encode(f.read()).decode()
|
| 85 |
+
except FileNotFoundError:
|
| 86 |
+
return ""
|
| 87 |
+
|
| 88 |
+
LOGO_ITU = _encode_logo(str(Path(__file__).parent / "ITULog.png"))
|
| 89 |
+
|
| 90 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 91 |
+
# CSS THEME — white / bluish, minimalist, medical
|
| 92 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 93 |
+
|
| 94 |
+
CSS = """
|
| 95 |
+
/* ── Base ── */
|
| 96 |
+
:root {
|
| 97 |
+
--primary: #1a6fa8;
|
| 98 |
+
--primary-lt: #e8f3fb;
|
| 99 |
+
--primary-mid: #4a9fd4;
|
| 100 |
+
--accent: #0d9e6e; /* Angio AI green */
|
| 101 |
+
--bg: #f7fafd;
|
| 102 |
+
--surface: #ffffff;
|
| 103 |
+
--border: #cce0f0;
|
| 104 |
+
--text: #1a2533;
|
| 105 |
+
--text-sec: #5a7a96;
|
| 106 |
+
--danger: #d64040;
|
| 107 |
+
--warn: #d98b00;
|
| 108 |
+
--radius: 10px;
|
| 109 |
+
--shadow: 0 2px 12px rgba(26,111,168,0.10);
|
| 110 |
+
}
|
| 111 |
+
|
| 112 |
+
body, .gradio-container {
|
| 113 |
+
background: var(--bg) !important;
|
| 114 |
+
font-family: 'Segoe UI', system-ui, sans-serif !important;
|
| 115 |
+
color: var(--text) !important;
|
| 116 |
+
}
|
| 117 |
+
|
| 118 |
+
/* ── Header bar ── */
|
| 119 |
+
#header {
|
| 120 |
+
background: var(--surface);
|
| 121 |
+
border-bottom: 2px solid var(--border);
|
| 122 |
+
padding: 14px 28px;
|
| 123 |
+
display: flex;
|
| 124 |
+
align-items: center;
|
| 125 |
+
justify-content: space-between;
|
| 126 |
+
box-shadow: var(--shadow);
|
| 127 |
+
margin-bottom: 0 !important;
|
| 128 |
+
}
|
| 129 |
+
#logo-itu img { height: 48px; }
|
| 130 |
+
#logo-angio {
|
| 131 |
+
font-size: 26px;
|
| 132 |
+
font-weight: 700;
|
| 133 |
+
color: var(--accent);
|
| 134 |
+
letter-spacing: 1px;
|
| 135 |
+
}
|
| 136 |
+
#logo-angio span {
|
| 137 |
+
font-size: 13px;
|
| 138 |
+
color: var(--text-sec);
|
| 139 |
+
font-weight: 400;
|
| 140 |
+
display: block;
|
| 141 |
+
letter-spacing: 0;
|
| 142 |
+
}
|
| 143 |
+
|
| 144 |
+
/* ── Tab bar ── */
|
| 145 |
+
.tab-nav button {
|
| 146 |
+
color: var(--text-sec) !important;
|
| 147 |
+
border-bottom: 3px solid transparent !important;
|
| 148 |
+
font-size: 15px !important;
|
| 149 |
+
font-weight: 500 !important;
|
| 150 |
+
padding: 10px 24px !important;
|
| 151 |
+
background: transparent !important;
|
| 152 |
+
}
|
| 153 |
+
.tab-nav button.selected {
|
| 154 |
+
color: var(--primary) !important;
|
| 155 |
+
border-bottom-color: var(--primary) !important;
|
| 156 |
+
}
|
| 157 |
+
|
| 158 |
+
/* ── Cards ── */
|
| 159 |
+
.card {
|
| 160 |
+
background: var(--surface);
|
| 161 |
+
border: 1px solid var(--border);
|
| 162 |
+
border-radius: var(--radius);
|
| 163 |
+
padding: 22px;
|
| 164 |
+
box-shadow: var(--shadow);
|
| 165 |
+
margin-bottom: 16px;
|
| 166 |
+
}
|
| 167 |
+
.card-title {
|
| 168 |
+
font-size: 14px;
|
| 169 |
+
font-weight: 600;
|
| 170 |
+
color: var(--primary);
|
| 171 |
+
text-transform: uppercase;
|
| 172 |
+
letter-spacing: 0.5px;
|
| 173 |
+
margin-bottom: 12px;
|
| 174 |
+
padding-bottom: 8px;
|
| 175 |
+
border-bottom: 1px solid var(--primary-lt);
|
| 176 |
+
}
|
| 177 |
+
|
| 178 |
+
/* ── Upload zone ── */
|
| 179 |
+
#upload-zone {
|
| 180 |
+
border: 2px dashed var(--primary-mid) !important;
|
| 181 |
+
border-radius: var(--radius) !important;
|
| 182 |
+
background: var(--primary-lt) !important;
|
| 183 |
+
min-height: 200px !important;
|
| 184 |
+
}
|
| 185 |
+
|
| 186 |
+
/* ── Sliders ── */
|
| 187 |
+
input[type=range] { accent-color: var(--primary); }
|
| 188 |
+
.gradio-slider label { color: var(--text-sec) !important; font-size: 13px !important; }
|
| 189 |
+
|
| 190 |
+
/* ── Buttons ── */
|
| 191 |
+
#btn-analyse {
|
| 192 |
+
background: var(--primary) !important;
|
| 193 |
+
color: white !important;
|
| 194 |
+
border: none !important;
|
| 195 |
+
border-radius: var(--radius) !important;
|
| 196 |
+
font-size: 15px !important;
|
| 197 |
+
font-weight: 600 !important;
|
| 198 |
+
padding: 12px 32px !important;
|
| 199 |
+
cursor: pointer !important;
|
| 200 |
+
width: 100% !important;
|
| 201 |
+
transition: background 0.2s !important;
|
| 202 |
+
}
|
| 203 |
+
#btn-analyse:hover { background: #155d90 !important; }
|
| 204 |
+
#btn-reset {
|
| 205 |
+
background: transparent !important;
|
| 206 |
+
color: var(--text-sec) !important;
|
| 207 |
+
border: 1px solid var(--border) !important;
|
| 208 |
+
border-radius: var(--radius) !important;
|
| 209 |
+
font-size: 13px !important;
|
| 210 |
+
padding: 8px 20px !important;
|
| 211 |
+
cursor: pointer !important;
|
| 212 |
+
width: 100% !important;
|
| 213 |
+
}
|
| 214 |
+
|
| 215 |
+
/* ── Metric badges ── */
|
| 216 |
+
.metric-row {
|
| 217 |
+
display: flex;
|
| 218 |
+
gap: 12px;
|
| 219 |
+
flex-wrap: wrap;
|
| 220 |
+
margin-bottom: 12px;
|
| 221 |
+
}
|
| 222 |
+
.metric-badge {
|
| 223 |
+
flex: 1;
|
| 224 |
+
min-width: 110px;
|
| 225 |
+
background: var(--primary-lt);
|
| 226 |
+
border: 1px solid var(--border);
|
| 227 |
+
border-radius: 8px;
|
| 228 |
+
padding: 10px 14px;
|
| 229 |
+
text-align: center;
|
| 230 |
+
}
|
| 231 |
+
.metric-badge .val {
|
| 232 |
+
font-size: 22px;
|
| 233 |
+
font-weight: 700;
|
| 234 |
+
color: var(--primary);
|
| 235 |
+
}
|
| 236 |
+
.metric-badge .val.ischemic { color: var(--danger); }
|
| 237 |
+
.metric-badge .val.borderline { color: var(--warn); }
|
| 238 |
+
.metric-badge .val.ok { color: var(--accent); }
|
| 239 |
+
.metric-badge .lbl {
|
| 240 |
+
font-size: 11px;
|
| 241 |
+
color: var(--text-sec);
|
| 242 |
+
text-transform: uppercase;
|
| 243 |
+
letter-spacing: 0.4px;
|
| 244 |
+
}
|
| 245 |
+
|
| 246 |
+
/* ── Result images ── */
|
| 247 |
+
#result-images img {
|
| 248 |
+
border-radius: var(--radius);
|
| 249 |
+
border: 1px solid var(--border);
|
| 250 |
+
}
|
| 251 |
+
|
| 252 |
+
/* ── Status ── */
|
| 253 |
+
#status-box {
|
| 254 |
+
padding: 10px 16px;
|
| 255 |
+
border-radius: 8px;
|
| 256 |
+
font-size: 13px;
|
| 257 |
+
font-weight: 500;
|
| 258 |
+
border: 1px solid var(--border);
|
| 259 |
+
background: var(--primary-lt);
|
| 260 |
+
color: var(--primary);
|
| 261 |
+
}
|
| 262 |
+
|
| 263 |
+
/* ── Footer ── */
|
| 264 |
+
#footer {
|
| 265 |
+
text-align: center;
|
| 266 |
+
font-size: 12px;
|
| 267 |
+
color: var(--text-sec);
|
| 268 |
+
padding: 20px;
|
| 269 |
+
border-top: 1px solid var(--border);
|
| 270 |
+
margin-top: 24px;
|
| 271 |
+
}
|
| 272 |
+
"""
|
| 273 |
+
|
| 274 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 275 |
+
# HEADER HTML
|
| 276 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 277 |
+
|
| 278 |
+
def _header_html() -> str:
|
| 279 |
+
itu_tag = (f'<img src="{LOGO_ITU}" alt="ITU Logo" style="height:48px">'
|
| 280 |
+
if LOGO_ITU else
|
| 281 |
+
'<span style="font-weight:700;color:#1a6fa8;font-size:18px">ITU</span>')
|
| 282 |
+
return f"""
|
| 283 |
+
<div id="header">
|
| 284 |
+
<div id="logo-itu">{itu_tag}</div>
|
| 285 |
+
<div id="logo-angio">
|
| 286 |
+
Angio AI
|
| 287 |
+
<span>Coronary Analysis System · ITU Lahore</span>
|
| 288 |
+
</div>
|
| 289 |
+
</div>"""
|
| 290 |
+
|
| 291 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 292 |
+
# MODULE 1 — KEYFRAME EXTRACTION (from app.py)
|
| 293 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 294 |
+
|
| 295 |
+
def extract_best_frame(video_path: str, n: int = 1) -> tuple:
|
| 296 |
+
"""
|
| 297 |
+
Extract the single best frame from the video.
|
| 298 |
+
Returns (frame_rgb_512, frame_index, score).
|
| 299 |
+
"""
|
| 300 |
+
cap = cv2.VideoCapture(video_path)
|
| 301 |
+
total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
|
| 302 |
+
if total == 0:
|
| 303 |
+
cap.release()
|
| 304 |
+
raise ValueError("Could not read video — check format/codec.")
|
| 305 |
+
|
| 306 |
+
step = max(1, total // min(120, total))
|
| 307 |
+
scores = []
|
| 308 |
+
fidx = 0
|
| 309 |
+
|
| 310 |
+
while cap.isOpened():
|
| 311 |
+
ret, frame = cap.read()
|
| 312 |
+
if not ret:
|
| 313 |
+
break
|
| 314 |
+
if fidx % step == 0:
|
| 315 |
+
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
|
| 316 |
+
h, w = gray.shape
|
| 317 |
+
roi = gray[int(h*0.2):int(h*0.8), int(w*0.2):int(w*0.8)]
|
| 318 |
+
contrast = float(roi.mean())
|
| 319 |
+
sharpness = float(cv2.Laplacian(roi, cv2.CV_64F).var())
|
| 320 |
+
score = contrast * 0.6 + min(sharpness, 5000) / 5000 * 100 * 0.4
|
| 321 |
+
scores.append((score, fidx, frame.copy()))
|
| 322 |
+
fidx += 1
|
| 323 |
+
cap.release()
|
| 324 |
+
|
| 325 |
+
if not scores:
|
| 326 |
+
raise ValueError("No frames could be sampled from video.")
|
| 327 |
+
|
| 328 |
+
scores.sort(key=lambda x: x[0], reverse=True)
|
| 329 |
+
best_score, best_idx, best_frame = scores[0]
|
| 330 |
+
|
| 331 |
+
rgb = cv2.cvtColor(best_frame, cv2.COLOR_BGR2RGB)
|
| 332 |
+
rgb = cv2.resize(rgb, (512, 512))
|
| 333 |
+
return rgb, best_idx, best_score, total
|
| 334 |
+
|
| 335 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 336 |
+
# MODULE 2 — MASK2FORMER INFERENCE
|
| 337 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 338 |
+
|
| 339 |
+
_m2f_model = None
|
| 340 |
+
_m2f_proc = None
|
| 341 |
+
|
| 342 |
+
def _load_mask2former():
|
| 343 |
+
global _m2f_model, _m2f_proc
|
| 344 |
+
if _m2f_model is not None:
|
| 345 |
+
return
|
| 346 |
+
from transformers import (AutoImageProcessor,
|
| 347 |
+
Mask2FormerForUniversalSegmentation)
|
| 348 |
+
ckpt = CONFIG["MASK2FORMER_CKPT"]() # resolves via HF Hub download
|
| 349 |
+
if not os.path.exists(ckpt):
|
| 350 |
+
raise FileNotFoundError(f"Mask2Former checkpoint not found:\n{ckpt}")
|
| 351 |
+
|
| 352 |
+
_m2f_proc = AutoImageProcessor.from_pretrained(
|
| 353 |
+
"facebook/mask2former-swin-base-coco-instance")
|
| 354 |
+
_m2f_model = Mask2FormerForUniversalSegmentation.from_pretrained(
|
| 355 |
+
"facebook/mask2former-swin-base-coco-instance",
|
| 356 |
+
num_labels=2, ignore_mismatched_sizes=True)
|
| 357 |
+
|
| 358 |
+
state = torch.load(ckpt, map_location="cpu")
|
| 359 |
+
weights = state.get("model_state", state)
|
| 360 |
+
_m2f_model.load_state_dict(weights, strict=False)
|
| 361 |
+
_m2f_model.to(CONFIG["DEVICE"]).eval()
|
| 362 |
+
|
| 363 |
+
|
| 364 |
+
def run_mask2former(frame_rgb: np.ndarray, conf_thr: float) -> list:
|
| 365 |
+
"""
|
| 366 |
+
Returns list of dicts: [{bbox:[x1,y1,x2,y2], score:float, mask:np.ndarray}, ...]
|
| 367 |
+
"""
|
| 368 |
+
_load_mask2former()
|
| 369 |
+
from PIL import Image
|
| 370 |
+
pil = Image.fromarray(frame_rgb)
|
| 371 |
+
inputs = _m2f_proc(images=pil, return_tensors="pt")
|
| 372 |
+
inputs = {k: v.to(CONFIG["DEVICE"]) for k, v in inputs.items()}
|
| 373 |
+
|
| 374 |
+
with torch.no_grad():
|
| 375 |
+
outputs = _m2f_model(**inputs)
|
| 376 |
+
|
| 377 |
+
results = _m2f_proc.post_process_instance_segmentation(
|
| 378 |
+
outputs, target_sizes=[(512, 512)])[0]
|
| 379 |
+
|
| 380 |
+
detections = []
|
| 381 |
+
for seg_info in results["segments_info"]:
|
| 382 |
+
score = float(seg_info["score"])
|
| 383 |
+
if score < conf_thr:
|
| 384 |
+
continue
|
| 385 |
+
mask = (results["segmentation"] == seg_info["id"]).cpu().numpy().astype(np.uint8)
|
| 386 |
+
ys, xs = np.where(mask)
|
| 387 |
+
if len(ys) == 0:
|
| 388 |
+
continue
|
| 389 |
+
x1, y1, x2, y2 = int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())
|
| 390 |
+
detections.append({"bbox": [x1, y1, x2, y2], "score": score, "mask": mask})
|
| 391 |
+
|
| 392 |
+
return detections
|
| 393 |
+
|
| 394 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 395 |
+
# MODULE 3 — RESUNET BINARY SEGMENTATION
|
| 396 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 397 |
+
|
| 398 |
+
_resunet_model = None
|
| 399 |
+
|
| 400 |
+
# ── ResUNet architecture — exactly matching binary_best.pth ──────────────────
|
| 401 |
+
#
|
| 402 |
+
# Decoded from checkpoint key names via dump_keys.py:
|
| 403 |
+
#
|
| 404 |
+
# enc0 Conv2d(1, 16, 3, 3, bias=True) plain entry conv
|
| 405 |
+
# down1..4 ConvTranspose2d strided ×2 downsampler 16→32→64→128→256
|
| 406 |
+
# res1..4 _ResBlock(ch) encoder residual blocks ch=16,32,64,128
|
| 407 |
+
# bot.0..2 _ResBlock(256) × 3 bottleneck
|
| 408 |
+
# up3..0 ConvTranspose2d ×2 upsampler 256→128→64→32→16
|
| 409 |
+
# res12..15 _ResBlock(ch) decoder residual blocks ch=128,64,32,16
|
| 410 |
+
# head Sequential[Conv3×3, ReLU, BN, output head
|
| 411 |
+
# Conv3×3, ReLU, BN, Conv1×1]
|
| 412 |
+
#
|
| 413 |
+
# _ResBlock structure (from key pattern res*.bn_in, res*.units.N.*, res*.bn_out):
|
| 414 |
+
# bn_in → BN
|
| 415 |
+
# units → ModuleList of N sub-units, each: [Conv3×3, ReLU, Conv3×3, BN]
|
| 416 |
+
# bn_out → BN
|
| 417 |
+
# forward: x → bn_in → for each unit: residual(x + unit(x)) → bn_out
|
| 418 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 419 |
+
|
| 420 |
+
class _ResBlock(torch.nn.Module):
|
| 421 |
+
"""
|
| 422 |
+
Matches checkpoint pattern:
|
| 423 |
+
res*.bn_in.*
|
| 424 |
+
res*.units.0.0.weight (Conv3×3)
|
| 425 |
+
res*.units.0.0.bias
|
| 426 |
+
res*.units.0.2.weight (Conv3×3)
|
| 427 |
+
res*.units.0.2.bias
|
| 428 |
+
res*.units.0.3.* (BN)
|
| 429 |
+
... (3 units total)
|
| 430 |
+
res*.bn_out.*
|
| 431 |
+
Sequential index mapping inside each unit:
|
| 432 |
+
0 = Conv2d
|
| 433 |
+
1 = ReLU (no params — not in state dict)
|
| 434 |
+
2 = Conv2d
|
| 435 |
+
3 = BatchNorm2d
|
| 436 |
+
"""
|
| 437 |
+
def __init__(self, ch, n_units=3):
|
| 438 |
+
super().__init__()
|
| 439 |
+
self.bn_in = torch.nn.BatchNorm2d(ch)
|
| 440 |
+
self.units = torch.nn.ModuleList([
|
| 441 |
+
torch.nn.Sequential(
|
| 442 |
+
torch.nn.Conv2d(ch, ch, 3, padding=1, bias=True),
|
| 443 |
+
torch.nn.ReLU(inplace=True),
|
| 444 |
+
torch.nn.Conv2d(ch, ch, 3, padding=1, bias=True),
|
| 445 |
+
torch.nn.BatchNorm2d(ch),
|
| 446 |
+
)
|
| 447 |
+
for _ in range(n_units)
|
| 448 |
+
])
|
| 449 |
+
self.bn_out = torch.nn.BatchNorm2d(ch)
|
| 450 |
+
|
| 451 |
+
def forward(self, x):
|
| 452 |
+
x = self.bn_in(x)
|
| 453 |
+
for unit in self.units:
|
| 454 |
+
x = x + unit(x) # no extra ReLU on residual sum
|
| 455 |
+
return self.bn_out(x)
|
| 456 |
+
|
| 457 |
+
|
| 458 |
+
class ResUNet(torch.nn.Module):
|
| 459 |
+
"""
|
| 460 |
+
Exact architecture matching binary_best.pth (Dice = 0.8015).
|
| 461 |
+
Key name mapping:
|
| 462 |
+
enc0 → self.enc0 (plain Conv2d)
|
| 463 |
+
down1..4 → self.down1..down4 (ConvTranspose2d strided downsampler)
|
| 464 |
+
res1..4 → self.res1..res4 (encoder ResBlocks)
|
| 465 |
+
bot.0..2 → self.bot[0..2] (bottleneck ResBlocks)
|
| 466 |
+
up3..0 → self.up3..up0 (ConvTranspose2d upsampler)
|
| 467 |
+
res12..15 → self.res12..res15 (decoder ResBlocks)
|
| 468 |
+
head → self.head (Sequential output head)
|
| 469 |
+
"""
|
| 470 |
+
def __init__(self):
|
| 471 |
+
super().__init__()
|
| 472 |
+
# ── Entry conv ────────────────────────────────────────────────────────
|
| 473 |
+
self.enc0 = torch.nn.Conv2d(1, 16, 3, padding=1, bias=True)
|
| 474 |
+
|
| 475 |
+
# ── Encoder downsamples — strided Conv2d (not ConvTranspose2d) ─────────
|
| 476 |
+
# Checkpoint shape (out, in, kH, kW): down1=(32,16,2,2) → Conv2d(16→32)
|
| 477 |
+
# stride=2 kernel=2 halves spatial dimensions while increasing channels
|
| 478 |
+
self.down1 = torch.nn.Conv2d(16, 32, 2, stride=2)
|
| 479 |
+
self.down2 = torch.nn.Conv2d(32, 64, 2, stride=2)
|
| 480 |
+
self.down3 = torch.nn.Conv2d(64, 128, 2, stride=2)
|
| 481 |
+
self.down4 = torch.nn.Conv2d(128, 256, 2, stride=2)
|
| 482 |
+
|
| 483 |
+
# ── Encoder residual blocks ───────────────────────���───────────────────
|
| 484 |
+
self.res1 = _ResBlock(16)
|
| 485 |
+
self.res2 = _ResBlock(32)
|
| 486 |
+
self.res3 = _ResBlock(64)
|
| 487 |
+
self.res4 = _ResBlock(128)
|
| 488 |
+
|
| 489 |
+
# ── Bottleneck ────────────────────────────────────────────────────────
|
| 490 |
+
self.bot = torch.nn.ModuleList([
|
| 491 |
+
_ResBlock(256),
|
| 492 |
+
_ResBlock(256),
|
| 493 |
+
_ResBlock(256),
|
| 494 |
+
])
|
| 495 |
+
|
| 496 |
+
# ── Decoder upsamplers ────────────────────────────────────────────────
|
| 497 |
+
# Shape from keys: up3=(256,128,2,2) bias=(128,)
|
| 498 |
+
# up2=(128,64,2,2) bias=(64,)
|
| 499 |
+
# up1=(64,32,2,2) bias=(32,)
|
| 500 |
+
# up0=(32,16,2,2) bias=(16,)
|
| 501 |
+
self.up3 = torch.nn.ConvTranspose2d(256, 128, 2, stride=2)
|
| 502 |
+
self.up2 = torch.nn.ConvTranspose2d(128, 64, 2, stride=2)
|
| 503 |
+
self.up1 = torch.nn.ConvTranspose2d(64, 32, 2, stride=2)
|
| 504 |
+
self.up0 = torch.nn.ConvTranspose2d(32, 16, 2, stride=2)
|
| 505 |
+
|
| 506 |
+
# ── Decoder residual blocks ───────────────────────────────────────────
|
| 507 |
+
self.res12 = _ResBlock(128)
|
| 508 |
+
self.res13 = _ResBlock(64)
|
| 509 |
+
self.res14 = _ResBlock(32)
|
| 510 |
+
self.res15 = _ResBlock(16)
|
| 511 |
+
|
| 512 |
+
# ── Output head ───────────────────────────────────────────────────────
|
| 513 |
+
# From keys: head.0=Conv(16,16,3,3), head.2=BN(16), head.3=Conv(16,16,3,3),
|
| 514 |
+
# head.5=BN(16), head.6=Conv(2,16,1,1)
|
| 515 |
+
# Index 1 and 4 are ReLU (no state dict entries)
|
| 516 |
+
self.head = torch.nn.Sequential(
|
| 517 |
+
torch.nn.Conv2d(16, 16, 3, padding=1, bias=True), # 0
|
| 518 |
+
torch.nn.ReLU(inplace=True), # 1
|
| 519 |
+
torch.nn.BatchNorm2d(16), # 2
|
| 520 |
+
torch.nn.Conv2d(16, 16, 3, padding=1, bias=True), # 3
|
| 521 |
+
torch.nn.ReLU(inplace=True), # 4
|
| 522 |
+
torch.nn.BatchNorm2d(16), # 5
|
| 523 |
+
torch.nn.Conv2d(16, 2, 1, bias=True), # 6
|
| 524 |
+
)
|
| 525 |
+
|
| 526 |
+
def forward(self, x):
|
| 527 |
+
# ── Encoder ──────────────────────────────────────────────────────────
|
| 528 |
+
# Entry conv → ResBlock → down. Skip taken AFTER ResBlock.
|
| 529 |
+
x = self.enc0(x) # (B,16,512,512)
|
| 530 |
+
r1 = self.res1(x); x = self.down1(r1) # skip r1, down → (B,32,256,256)
|
| 531 |
+
r2 = self.res2(x); x = self.down2(r2) # skip r2, down → (B,64,128,128)
|
| 532 |
+
r3 = self.res3(x); x = self.down3(r3) # skip r3, down → (B,128,64,64)
|
| 533 |
+
r4 = self.res4(x); x = self.down4(r4) # skip r4, down → (B,256,32,32)
|
| 534 |
+
|
| 535 |
+
# ── Bottleneck ───────────────────────────────────────────────────────
|
| 536 |
+
for blk in self.bot:
|
| 537 |
+
x = blk(x)
|
| 538 |
+
|
| 539 |
+
# ── Decoder — ResBlock first, then add skip ───────────────────────────
|
| 540 |
+
x = self.res12(self.up3(x)) + r4 # (B,128,64,64)
|
| 541 |
+
x = self.res13(self.up2(x)) + r3 # (B,64,128,128)
|
| 542 |
+
x = self.res14(self.up1(x)) + r2 # (B,32,256,256)
|
| 543 |
+
x = self.res15(self.up0(x)) + r1 # (B,16,512,512)
|
| 544 |
+
|
| 545 |
+
return self.head(x)
|
| 546 |
+
|
| 547 |
+
|
| 548 |
+
def _load_resunet():
|
| 549 |
+
global _resunet_model
|
| 550 |
+
if _resunet_model is not None:
|
| 551 |
+
return
|
| 552 |
+
ckpt = CONFIG["RESUNET_CKPT"]() # resolves via HF Hub download
|
| 553 |
+
if not os.path.exists(ckpt):
|
| 554 |
+
raise FileNotFoundError(f"ResUNet checkpoint not found:\n{ckpt}")
|
| 555 |
+
_resunet_model = ResUNet()
|
| 556 |
+
state = torch.load(ckpt, map_location="cpu")
|
| 557 |
+
weights = state.get("model_state", state)
|
| 558 |
+
|
| 559 |
+
# Try strict first; if it fails print exact mismatches and retry strict=False
|
| 560 |
+
try:
|
| 561 |
+
_resunet_model.load_state_dict(weights, strict=True)
|
| 562 |
+
print(f"[ResUNet] ✓ Perfect load (strict=True) — "
|
| 563 |
+
f"epoch={state.get('epoch','?')} "
|
| 564 |
+
f"dice={state.get('best_val_dice',0):.4f}")
|
| 565 |
+
except RuntimeError as e:
|
| 566 |
+
print(f"[ResUNet] strict=True failed:\n {e}")
|
| 567 |
+
result = _resunet_model.load_state_dict(weights, strict=False)
|
| 568 |
+
print(f"[ResUNet] strict=False — "
|
| 569 |
+
f"missing={len(result.missing_keys)} "
|
| 570 |
+
f"unexpected={len(result.unexpected_keys)}")
|
| 571 |
+
for k in result.missing_keys[:10]:
|
| 572 |
+
print(f" MISSING {k}")
|
| 573 |
+
for k in result.unexpected_keys[:10]:
|
| 574 |
+
print(f" UNEXPECTED {k}")
|
| 575 |
+
|
| 576 |
+
_resunet_model.to(CONFIG["DEVICE"]).eval()
|
| 577 |
+
print(f"[ResUNet] Loaded (eval mode) — "
|
| 578 |
+
f"epoch={state.get('epoch','?')} "
|
| 579 |
+
f"dice={state.get('best_val_dice',0):.4f}")
|
| 580 |
+
|
| 581 |
+
|
| 582 |
+
def _preprocess_resunet(rgb: np.ndarray) -> torch.Tensor:
|
| 583 |
+
"""ARCADE preprocessing: invert → top-hat → subtract → CLAHE → normalise."""
|
| 584 |
+
gray = cv2.cvtColor(rgb, cv2.COLOR_RGB2GRAY).astype(np.uint8)
|
| 585 |
+
inv = cv2.bitwise_not(gray)
|
| 586 |
+
se = cv2.getStructuringElement(cv2.MORPH_RECT, (50, 50))
|
| 587 |
+
th = cv2.morphologyEx(inv, cv2.MORPH_TOPHAT, se)
|
| 588 |
+
sub = cv2.subtract(gray, th)
|
| 589 |
+
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
|
| 590 |
+
proc = clahe.apply(sub).astype(np.float32) / 255.0
|
| 591 |
+
t = torch.from_numpy(proc).unsqueeze(0).unsqueeze(0) # (1,1,512,512)
|
| 592 |
+
return t
|
| 593 |
+
|
| 594 |
+
|
| 595 |
+
def run_resunet(frame_rgb: np.ndarray):
|
| 596 |
+
"""
|
| 597 |
+
Returns:
|
| 598 |
+
binary_mask : (512,512) uint8, values 0/1
|
| 599 |
+
binary_overlay: (512,512,3) uint8 RGB — green vessel overlay on original
|
| 600 |
+
Architecture fix applied: skip connections are now res(up(x)) + skip
|
| 601 |
+
(not res(up(x) + skip)), and ResBlock has no extra ReLU on residual sum.
|
| 602 |
+
Model runs in eval() mode — BN running stats are now correct.
|
| 603 |
+
"""
|
| 604 |
+
_load_resunet()
|
| 605 |
+
_resunet_model.eval()
|
| 606 |
+
t = _preprocess_resunet(frame_rgb).to(CONFIG["DEVICE"])
|
| 607 |
+
with torch.no_grad():
|
| 608 |
+
logits = _resunet_model(t)
|
| 609 |
+
pred = logits.argmax(dim=1).squeeze().cpu().numpy().astype(np.uint8)
|
| 610 |
+
vessel_px = int((pred == 1).sum())
|
| 611 |
+
print(f"[ResUNet] vessel px={vessel_px} ({100*(pred==1).mean():.1f}%)")
|
| 612 |
+
|
| 613 |
+
# Build green overlay for display in Results tab
|
| 614 |
+
overlay = frame_rgb.copy().astype(np.float32)
|
| 615 |
+
green = np.array([0, 255, 80], dtype=np.float32)
|
| 616 |
+
mask3d = pred == 1
|
| 617 |
+
overlay[mask3d] = overlay[mask3d] * 0.35 + green * 0.65
|
| 618 |
+
overlay = overlay.astype(np.uint8)
|
| 619 |
+
# Draw contours for sharper vessel edges
|
| 620 |
+
contours, _ = cv2.findContours(pred, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
|
| 621 |
+
cv2.drawContours(overlay, contours, -1, (0, 255, 80), 1)
|
| 622 |
+
# Coverage label
|
| 623 |
+
cov = 100 * vessel_px / pred.size
|
| 624 |
+
cv2.putText(overlay, f"Vessel coverage: {cov:.1f}%",
|
| 625 |
+
(8, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 80), 1, cv2.LINE_AA)
|
| 626 |
+
|
| 627 |
+
return pred, overlay
|
| 628 |
+
|
| 629 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 630 |
+
# MODULE 4 — YOLOv8m-seg INFERENCE (replaces nnUNet)
|
| 631 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 632 |
+
#
|
| 633 |
+
# Checkpoint : CONFIG["YOLO_CKPT"] -> best.pt from YOLOv8m-seg training
|
| 634 |
+
# Model : YOLOv8m-seg, nc=26, trained on ARCADE syntax dataset
|
| 635 |
+
#
|
| 636 |
+
# Input : frame_rgb -- (512,512,3) uint8 RGB
|
| 637 |
+
# seg_conf -- confidence threshold from UI slider
|
| 638 |
+
# Output: seg_map -- (512,512) uint8 label map, values 0..25
|
| 639 |
+
# 0 = background (no detection)
|
| 640 |
+
# 1..25 = ARCADE class index
|
| 641 |
+
#
|
| 642 |
+
# Output format is identical to the old run_nnunet() so that
|
| 643 |
+
# render_nnunet_overlay() and compute_syntax_score() work unchanged.
|
| 644 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 645 |
+
|
| 646 |
+
# ARCADE 26-class name list (index = YOLO cls_id + 1)
|
| 647 |
+
_YOLO_CLASS_NAMES = [
|
| 648 |
+
"background", # 0 (background, not predicted by YOLO)
|
| 649 |
+
"LMCA", # 1
|
| 650 |
+
"LAD prox", # 2
|
| 651 |
+
"LAD mid", # 3
|
| 652 |
+
"LAD dist", # 4
|
| 653 |
+
"D1", # 5
|
| 654 |
+
"D2", # 6
|
| 655 |
+
"D3", # 7
|
| 656 |
+
"Septal", # 8
|
| 657 |
+
"LAD var", # 9
|
| 658 |
+
"LCX prox", # 10
|
| 659 |
+
"LCX mid", # 11
|
| 660 |
+
"LCX dist", # 12
|
| 661 |
+
"OM1", # 13
|
| 662 |
+
"OM2", # 14
|
| 663 |
+
"OM3", # 15
|
| 664 |
+
"LCX var", # 16
|
| 665 |
+
"RCA prox", # 17
|
| 666 |
+
"RCA mid", # 18
|
| 667 |
+
"RCA dist", # 19
|
| 668 |
+
"PDA", # 20
|
| 669 |
+
"PLV", # 21
|
| 670 |
+
"AM", # 22
|
| 671 |
+
"Conus", # 23
|
| 672 |
+
"SAN", # 24
|
| 673 |
+
"Other", # 25
|
| 674 |
+
]
|
| 675 |
+
|
| 676 |
+
_yolo_model = None # singleton -- loaded once on first call, reused every frame
|
| 677 |
+
|
| 678 |
+
|
| 679 |
+
def _load_yolo():
|
| 680 |
+
"""Lazy-load YOLOv8m-seg checkpoint into module-level singleton."""
|
| 681 |
+
global _yolo_model
|
| 682 |
+
if _yolo_model is not None:
|
| 683 |
+
return
|
| 684 |
+
try:
|
| 685 |
+
from ultralytics import YOLO
|
| 686 |
+
except ImportError:
|
| 687 |
+
raise ImportError(
|
| 688 |
+
"ultralytics not installed. Run: pip install ultralytics"
|
| 689 |
+
)
|
| 690 |
+
ckpt = CONFIG["YOLO_CKPT"]() # resolves via HF Hub download
|
| 691 |
+
if not os.path.exists(ckpt):
|
| 692 |
+
raise FileNotFoundError(
|
| 693 |
+
f"YOLOv8 checkpoint not found:\n{ckpt}\n"
|
| 694 |
+
f"Check HF_TOKEN secret and repo MuhammadAdil63/angio-ai-checkpoints"
|
| 695 |
+
)
|
| 696 |
+
print(f"[YOLO] Loading checkpoint: {ckpt}")
|
| 697 |
+
_yolo_model = YOLO(ckpt)
|
| 698 |
+
print(f"[YOLO] Model loaded OK")
|
| 699 |
+
|
| 700 |
+
|
| 701 |
+
def run_yolo_seg(frame_rgb: np.ndarray, seg_conf: float) -> np.ndarray:
|
| 702 |
+
"""
|
| 703 |
+
Run YOLOv8m-seg on a single RGB frame.
|
| 704 |
+
Returns seg_map (512x512 uint8), label values 0..25.
|
| 705 |
+
|
| 706 |
+
Label map composition:
|
| 707 |
+
- Initialised to 0 (background).
|
| 708 |
+
- Instances sorted ascending by confidence so higher-conf masks
|
| 709 |
+
overwrite lower-conf ones at overlapping pixels.
|
| 710 |
+
- YOLO cls_id is 0-based (0..24 for nc=25 visible classes).
|
| 711 |
+
We store label_val = cls_id + 1 so 0 stays background,
|
| 712 |
+
matching ARCADE_COLOURS / ARCADE_LABELS which use keys 1..25.
|
| 713 |
+
"""
|
| 714 |
+
_load_yolo()
|
| 715 |
+
|
| 716 |
+
device = CONFIG["DEVICE"]
|
| 717 |
+
conf = float(seg_conf) if seg_conf else CONFIG["YOLO_CONF"]
|
| 718 |
+
iou = CONFIG["YOLO_IOU"]
|
| 719 |
+
imgsz = CONFIG["YOLO_IMGSZ"]
|
| 720 |
+
|
| 721 |
+
# YOLO expects BGR
|
| 722 |
+
frame_bgr = cv2.cvtColor(frame_rgb, cv2.COLOR_RGB2BGR)
|
| 723 |
+
|
| 724 |
+
try:
|
| 725 |
+
results = _yolo_model.predict(
|
| 726 |
+
source = frame_bgr,
|
| 727 |
+
conf = conf,
|
| 728 |
+
iou = iou,
|
| 729 |
+
imgsz = imgsz,
|
| 730 |
+
device = device,
|
| 731 |
+
retina_masks = False,
|
| 732 |
+
verbose = False,
|
| 733 |
+
)
|
| 734 |
+
except Exception as e:
|
| 735 |
+
print(f"[YOLO] predict() failed: {e}")
|
| 736 |
+
return np.zeros((512, 512), dtype=np.uint8)
|
| 737 |
+
|
| 738 |
+
result = results[0]
|
| 739 |
+
h, w = frame_rgb.shape[:2]
|
| 740 |
+
seg_map = np.zeros((h, w), dtype=np.uint8)
|
| 741 |
+
|
| 742 |
+
if result.masks is None or result.boxes is None:
|
| 743 |
+
print("[YOLO] No masks returned -- check conf threshold or checkpoint.")
|
| 744 |
+
return seg_map
|
| 745 |
+
|
| 746 |
+
masks = result.masks.data.cpu().numpy() # (N, Hm, Wm) float32
|
| 747 |
+
cls_ids = result.boxes.cls.cpu().int().numpy() # (N,) 0-based
|
| 748 |
+
confs = result.boxes.conf.cpu().numpy() # (N,)
|
| 749 |
+
|
| 750 |
+
# Ascending confidence sort so higher-conf masks win at overlaps
|
| 751 |
+
order = np.argsort(confs)
|
| 752 |
+
|
| 753 |
+
n_written = 0
|
| 754 |
+
for i in order:
|
| 755 |
+
cls_id = int(cls_ids[i])
|
| 756 |
+
mask_resized = cv2.resize(
|
| 757 |
+
masks[i], (w, h), interpolation=cv2.INTER_LINEAR
|
| 758 |
+
)
|
| 759 |
+
binary = mask_resized > 0.5
|
| 760 |
+
label_val = min(cls_id + 1, 25) # clamp to valid 1-25 range
|
| 761 |
+
seg_map[binary] = label_val
|
| 762 |
+
n_written += int(binary.sum())
|
| 763 |
+
|
| 764 |
+
detected_cls = [_YOLO_CLASS_NAMES[min(int(c) + 1, 25)] for c in cls_ids]
|
| 765 |
+
print(f"[YOLO] Detections: {len(cls_ids)} | classes: {detected_cls} | "
|
| 766 |
+
f"vessel px: {n_written} ({100 * n_written / (h * w):.1f}%)")
|
| 767 |
+
print(f"[YOLO] seg_map unique labels: {np.unique(seg_map)}")
|
| 768 |
+
return seg_map
|
| 769 |
+
|
| 770 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 771 |
+
# MODULE 5 — FFR PIPELINE (inline, from ffr_pipeline_v4.py)
|
| 772 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 773 |
+
|
| 774 |
+
from skimage.morphology import skeletonize, remove_small_objects, disk
|
| 775 |
+
from skimage.measure import label as sk_label
|
| 776 |
+
from scipy.ndimage import distance_transform_edt
|
| 777 |
+
from scipy.signal import argrelmin, savgol_filter
|
| 778 |
+
from skimage.filters import threshold_otsu
|
| 779 |
+
|
| 780 |
+
FFR_CFG = {
|
| 781 |
+
"close_radius" : 3,
|
| 782 |
+
"open_radius" : 1,
|
| 783 |
+
"min_vessel_px" : 100,
|
| 784 |
+
"stenosis_overlap_thr": 0.05,
|
| 785 |
+
# Lowered from 0.20 → 0.05 for video frames which have sparser masks
|
| 786 |
+
# than training PNGs — prevents all detections being unlocalized
|
| 787 |
+
"min_overlap_for_ffr": 0.05,
|
| 788 |
+
"min_ds_fraction" : 0.10,
|
| 789 |
+
"min_branch_len" : 15,
|
| 790 |
+
"use_dominant_branch": True,
|
| 791 |
+
"min_coverage_warn" : 0.50,
|
| 792 |
+
"sample_step" : 3,
|
| 793 |
+
"perp_half_len" : 40, # reduced from 50 to avoid hitting background
|
| 794 |
+
"smooth_window" : 11,
|
| 795 |
+
"smooth_poly" : 2,
|
| 796 |
+
"reference_pct" : 75,
|
| 797 |
+
# Physiological cap: coronary vessels are 1.5–5mm in diameter
|
| 798 |
+
# If auto-computed ref exceeds this, clamp it
|
| 799 |
+
"ref_diam_max_mm" : 5.0,
|
| 800 |
+
"ref_diam_min_mm" : 1.0,
|
| 801 |
+
"px_per_mm" : 3.75,
|
| 802 |
+
}
|
| 803 |
+
|
| 804 |
+
def _refine_mask(img_gray, binary_mask):
|
| 805 |
+
masked = img_gray[binary_mask == 1]
|
| 806 |
+
if len(masked) == 0:
|
| 807 |
+
return binary_mask.copy()
|
| 808 |
+
thr = threshold_otsu(masked)
|
| 809 |
+
return ((binary_mask == 1) & (img_gray < thr)).astype(np.uint8)
|
| 810 |
+
|
| 811 |
+
def _morph_clean(mask):
|
| 812 |
+
m = mask.copy()
|
| 813 |
+
k_c = disk(FFR_CFG["close_radius"]).astype(np.uint8)
|
| 814 |
+
k_o = disk(FFR_CFG["open_radius"]).astype(np.uint8)
|
| 815 |
+
m = cv2.morphologyEx(m, cv2.MORPH_CLOSE, k_c)
|
| 816 |
+
m = cv2.morphologyEx(m, cv2.MORPH_OPEN, k_o)
|
| 817 |
+
return remove_small_objects(m.astype(bool),
|
| 818 |
+
min_size=FFR_CFG["min_vessel_px"]).astype(np.uint8)
|
| 819 |
+
|
| 820 |
+
def _find_branch_pts(skel):
|
| 821 |
+
k = np.ones((3,3), np.uint8)
|
| 822 |
+
n = cv2.filter2D(skel.astype(np.uint8), -1, k) * skel.astype(np.uint8)
|
| 823 |
+
return n >= 4
|
| 824 |
+
|
| 825 |
+
def _dominant_branch(skel):
|
| 826 |
+
bp = _find_branch_pts(skel)
|
| 827 |
+
sp = skel.copy().astype(np.uint8); sp[bp] = 0
|
| 828 |
+
lab = sk_label(sp)
|
| 829 |
+
if lab.max() == 0: return skel
|
| 830 |
+
sizes = np.bincount(lab.ravel()); sizes[0] = 0
|
| 831 |
+
return (lab == sizes.argmax())
|
| 832 |
+
|
| 833 |
+
def _order_centerline(skel):
|
| 834 |
+
pts = np.argwhere(skel)
|
| 835 |
+
if len(pts) == 0: return []
|
| 836 |
+
sk_set = set(map(tuple, pts))
|
| 837 |
+
def nbrs(p):
|
| 838 |
+
r,c = p
|
| 839 |
+
return [(r+dr,c+dc) for dr in [-1,0,1] for dc in [-1,0,1]
|
| 840 |
+
if (dr,dc)!=(0,0) and (r+dr,c+dc) in sk_set]
|
| 841 |
+
endpoints = [p for p in sk_set if len(nbrs(p))==1] or [tuple(pts[0])]
|
| 842 |
+
def trace(start):
|
| 843 |
+
visited={tuple(start)}; path=[tuple(start)]; cur=tuple(start)
|
| 844 |
+
while True:
|
| 845 |
+
cands=[n for n in nbrs(cur) if n not in visited]
|
| 846 |
+
if not cands: break
|
| 847 |
+
if len(path)>1:
|
| 848 |
+
pr=path[-2]; dr0=cur[0]-pr[0]; dc0=cur[1]-pr[1]
|
| 849 |
+
cands.sort(key=lambda n:abs(n[0]-cur[0]-dr0)+abs(n[1]-cur[1]-dc0))
|
| 850 |
+
cur=cands[0]; visited.add(cur); path.append(cur)
|
| 851 |
+
return path
|
| 852 |
+
best=[]
|
| 853 |
+
for ep in endpoints[:20]:
|
| 854 |
+
p=trace(ep)
|
| 855 |
+
if len(p)>len(best): best=p
|
| 856 |
+
return best
|
| 857 |
+
|
| 858 |
+
def _measure_diameters(cl_pts, refined_mask):
|
| 859 |
+
H,W=refined_mask.shape
|
| 860 |
+
step=FFR_CFG["sample_step"]; half=FFR_CFG["perp_half_len"]
|
| 861 |
+
arc_len=0.; arc_pos=[]; diams=[]; wall_pairs=[]; centers=[]
|
| 862 |
+
for i in range(0, len(cl_pts)-step, step):
|
| 863 |
+
p0=np.array(cl_pts[i], dtype=float)
|
| 864 |
+
p1=np.array(cl_pts[min(i+step,len(cl_pts)-1)], dtype=float)
|
| 865 |
+
tang=p1-p0; tl=np.linalg.norm(tang)
|
| 866 |
+
if tl<1e-9: continue
|
| 867 |
+
tang/=tl; norm=np.array([-tang[1],tang[0]])
|
| 868 |
+
def wd(d):
|
| 869 |
+
for dd in np.arange(0.5,half,0.5):
|
| 870 |
+
pt=p0+d*dd; r_=int(round(pt[0])); c_=int(round(pt[1]))
|
| 871 |
+
if r_<0 or r_>=H or c_<0 or c_>=W: return dd,pt
|
| 872 |
+
if refined_mask[r_,c_]==0: return dd,pt
|
| 873 |
+
return half, p0+d*half
|
| 874 |
+
dl,pl=wd( norm); dr,pr=wd(-norm); diam=dl+dr
|
| 875 |
+
if diam<1. or diam>half*1.8: continue
|
| 876 |
+
if len(centers)==0: step_arc=0.
|
| 877 |
+
else: step_arc=float(np.linalg.norm(p0-np.array(centers[-1])))
|
| 878 |
+
arc_len+=min(step_arc, FFR_CFG["sample_step"]*1.5)
|
| 879 |
+
arc_pos.append(arc_len); diams.append(diam)
|
| 880 |
+
wall_pairs.append((pl,pr)); centers.append(p0)
|
| 881 |
+
return (np.array(arc_pos), np.array(diams,dtype=np.float32),
|
| 882 |
+
wall_pairs, centers)
|
| 883 |
+
|
| 884 |
+
def _ds_to_ffr(ds):
|
| 885 |
+
return max(0., min(1., 1.-(0.33*ds+0.60*ds**2)))
|
| 886 |
+
|
| 887 |
+
def run_ffr(frame_rgb, binary_mask, detections):
|
| 888 |
+
"""Full FFR pipeline v4. Returns result dict + annotated figure (np array)."""
|
| 889 |
+
img_gray = cv2.cvtColor(frame_rgb, cv2.COLOR_RGB2GRAY)
|
| 890 |
+
px_mm = FFR_CFG["px_per_mm"]
|
| 891 |
+
|
| 892 |
+
# ── Debug: report mask quality before refinement ──────────────────────────
|
| 893 |
+
bm_sum = int(binary_mask.sum())
|
| 894 |
+
bm_pct = 100 * binary_mask.mean()
|
| 895 |
+
print(f"[FFR] binary_mask entering FFR: sum={bm_sum} {bm_pct:.1f}% of image")
|
| 896 |
+
|
| 897 |
+
# ── Use binary mask directly — skip Otsu refinement ─────────────────────
|
| 898 |
+
# ResUNet already produces a clean vessel segmentation (Dice=0.80).
|
| 899 |
+
# Otsu refinement was designed for use with ground-truth binary masks from
|
| 900 |
+
# the ARCADE dataset where it further removes bright background pixels.
|
| 901 |
+
# On video frames, Otsu degrades the ResUNet mask because the vessel/bg
|
| 902 |
+
# intensity distribution differs from training PNGs.
|
| 903 |
+
# Using the binary mask directly preserves all vessel pixels.
|
| 904 |
+
refined = binary_mask.copy()
|
| 905 |
+
print(f"[FFR] Using binary mask directly (skip Otsu): sum={refined.sum()}")
|
| 906 |
+
|
| 907 |
+
refined = _morph_clean(refined)
|
| 908 |
+
print(f"[FFR] After morph cleanup: sum={refined.sum()}")
|
| 909 |
+
|
| 910 |
+
skel_raw = skeletonize(refined.astype(bool))
|
| 911 |
+
print(f"[FFR] Skeleton pixels: {skel_raw.sum()}")
|
| 912 |
+
|
| 913 |
+
# Prune spurs
|
| 914 |
+
sk = skel_raw.copy().astype(np.uint8)
|
| 915 |
+
for _ in range(FFR_CFG["min_branch_len"]):
|
| 916 |
+
k=np.ones((3,3),np.uint8)
|
| 917 |
+
nb=cv2.filter2D(sk,-1,k)*sk
|
| 918 |
+
ep=(nb==2).astype(np.uint8)
|
| 919 |
+
sk=cv2.subtract(sk,ep)
|
| 920 |
+
if ep.sum()==0: break
|
| 921 |
+
skel_pruned=sk.astype(bool)
|
| 922 |
+
print(f"[FFR] Pruned skeleton: {skel_pruned.sum()}")
|
| 923 |
+
|
| 924 |
+
skel_dom = _dominant_branch(skel_pruned)
|
| 925 |
+
print(f"[FFR] Dominant branch: {skel_dom.sum()}")
|
| 926 |
+
cl_pts = _order_centerline(skel_dom)
|
| 927 |
+
print(f"[FFR] Ordered centerline: {len(cl_pts)} pts")
|
| 928 |
+
|
| 929 |
+
# Fallback: only if dominant branch is very short (< 30 pts)
|
| 930 |
+
# Do NOT fall back to full skeleton — it traces through noise fragments
|
| 931 |
+
if len(cl_pts) < 30:
|
| 932 |
+
print(f"[FFR] Dominant branch short ({len(cl_pts)} pts) — trying full pruned skeleton")
|
| 933 |
+
cl_pts_full = _order_centerline(skel_pruned)
|
| 934 |
+
if len(cl_pts_full) > len(cl_pts):
|
| 935 |
+
cl_pts = cl_pts_full
|
| 936 |
+
print(f"[FFR] Full skeleton centerline: {len(cl_pts)} pts")
|
| 937 |
+
|
| 938 |
+
if len(cl_pts) < 5:
|
| 939 |
+
return {
|
| 940 |
+
"error": (
|
| 941 |
+
f"Centerline too short ({len(cl_pts)} pts). "
|
| 942 |
+
f"binary_mask={bm_sum}px ({bm_pct:.1f}%), "
|
| 943 |
+
f"skeleton={int(skel_raw.sum())}px."
|
| 944 |
+
)
|
| 945 |
+
}, None
|
| 946 |
+
|
| 947 |
+
arc_pos, diams_px, wall_pairs, centers = _measure_diameters(cl_pts, refined)
|
| 948 |
+
if len(diams_px)==0:
|
| 949 |
+
return {"error":"No diameter measurements obtained."}, None
|
| 950 |
+
|
| 951 |
+
win=FFR_CFG["smooth_window"]
|
| 952 |
+
if len(diams_px)<win: win=len(diams_px) if len(diams_px)%2==1 else max(3,len(diams_px)-1)
|
| 953 |
+
diams_sm = savgol_filter(diams_px, win, FFR_CFG["smooth_poly"]) if win>=3 else diams_px.copy()
|
| 954 |
+
|
| 955 |
+
ref_px = np.percentile(diams_sm, FFR_CFG["reference_pct"])
|
| 956 |
+
ref_mm = ref_px / px_mm
|
| 957 |
+
|
| 958 |
+
# ── Physiological sanity clamp ────────────────────────────────────────────
|
| 959 |
+
# Coronary vessels are 1.5–5.0 mm in diameter.
|
| 960 |
+
# If ref_mm is outside this range, the skeleton traced through noise/background.
|
| 961 |
+
# Clamp to physiological limits and recompute ref_px accordingly.
|
| 962 |
+
ref_mm_raw = ref_mm
|
| 963 |
+
ref_mm = float(np.clip(ref_mm,
|
| 964 |
+
FFR_CFG["ref_diam_min_mm"],
|
| 965 |
+
FFR_CFG["ref_diam_max_mm"]))
|
| 966 |
+
ref_px = ref_mm * px_mm
|
| 967 |
+
if abs(ref_mm_raw - ref_mm) > 0.1:
|
| 968 |
+
print(f"[FFR] ref_diam clamped: {ref_mm_raw:.2f}mm → {ref_mm:.2f}mm "
|
| 969 |
+
f"(physiological range {FFR_CFG['ref_diam_min_mm']}–"
|
| 970 |
+
f"{FFR_CFG['ref_diam_max_mm']}mm)")
|
| 971 |
+
|
| 972 |
+
# Also filter out diameter measurements that exceed 2× ref_px (noise)
|
| 973 |
+
valid_mask = diams_sm <= (ref_px * 2.5)
|
| 974 |
+
if valid_mask.sum() > 5:
|
| 975 |
+
arc_pos = arc_pos[valid_mask]
|
| 976 |
+
diams_px = diams_px[valid_mask]
|
| 977 |
+
diams_sm = diams_sm[valid_mask]
|
| 978 |
+
wall_pairs = [wp for wp, v in zip(wall_pairs, valid_mask) if v]
|
| 979 |
+
centers = [c for c, v in zip(centers, valid_mask) if v]
|
| 980 |
+
centers_arr = np.array(centers)
|
| 981 |
+
print(f"[FFR] After noise filter: {len(diams_sm)} measurements kept")
|
| 982 |
+
|
| 983 |
+
stenoses=[]; unloc=[]
|
| 984 |
+
|
| 985 |
+
for det in detections:
|
| 986 |
+
bbox = det.get("bbox",[])
|
| 987 |
+
score = det.get("score",1.)
|
| 988 |
+
overlap = det.get("overlap",0.)
|
| 989 |
+
if len(bbox)<4: continue
|
| 990 |
+
if overlap < FFR_CFG["min_overlap_for_ffr"]:
|
| 991 |
+
unloc.append({**det,"reason":"low-overlap"}); continue
|
| 992 |
+
x1,y1,x2,y2=bbox
|
| 993 |
+
inside=np.where((centers_arr[:,0]>=y1)&(centers_arr[:,0]<=y2)&
|
| 994 |
+
(centers_arr[:,1]>=x1)&(centers_arr[:,1]<=x2))[0]
|
| 995 |
+
idx = int(inside[np.argmin(diams_sm[inside])]) if len(inside)>0 else \
|
| 996 |
+
int(np.argmin(np.linalg.norm(centers_arr-np.array([(y1+y2)/2,(x1+x2)/2]),axis=1)))
|
| 997 |
+
if idx>=len(diams_sm): continue
|
| 998 |
+
d_sten=float(diams_sm[idx]); ds=max(0.,1.-d_sten/ref_px)
|
| 999 |
+
if ds < FFR_CFG["min_ds_fraction"]:
|
| 1000 |
+
unloc.append({**det,"reason":"no-narrowing"}); continue
|
| 1001 |
+
ffr_est=_ds_to_ffr(ds)
|
| 1002 |
+
stenoses.append({"index":idx,"position_mm":float(arc_pos[idx]/px_mm),
|
| 1003 |
+
"d_mm":float(d_sten/px_mm),"ref_mm":float(ref_mm),
|
| 1004 |
+
"pct_ds":float(ds*100),"ffr":ffr_est,
|
| 1005 |
+
"score":score,"significant":ffr_est<=0.80})
|
| 1006 |
+
|
| 1007 |
+
global_ffr=1.
|
| 1008 |
+
for s in stenoses: global_ffr*=s["ffr"]
|
| 1009 |
+
global_ffr=max(0.,global_ffr)
|
| 1010 |
+
|
| 1011 |
+
# Build figure
|
| 1012 |
+
fig, axes = plt.subplots(1,3, figsize=(15,5))
|
| 1013 |
+
fig.patch.set_facecolor("white")
|
| 1014 |
+
|
| 1015 |
+
# Panel 1: stenosis overlay
|
| 1016 |
+
axes[0].imshow(frame_rgb, cmap="gray")
|
| 1017 |
+
for det in detections:
|
| 1018 |
+
b=det.get("bbox",[]); sc=det.get("score",0)
|
| 1019 |
+
if len(b)<4: continue
|
| 1020 |
+
x1,y1,x2,y2=b
|
| 1021 |
+
col="red" if sc>=0.5 else "orange"
|
| 1022 |
+
axes[0].add_patch(mpatches.Rectangle(
|
| 1023 |
+
(x1,y1),x2-x1,y2-y1,fill=False,edgecolor=col,linewidth=2))
|
| 1024 |
+
axes[0].text(x1,y1-5,f"{sc:.2f}",color=col,fontsize=8,fontweight="bold")
|
| 1025 |
+
axes[0].axis("off"); axes[0].set_title("Stenosis detections", fontsize=11, color="#1a6fa8")
|
| 1026 |
+
|
| 1027 |
+
# Panel 2: centerline + diameter lines
|
| 1028 |
+
overlay=frame_rgb.copy()
|
| 1029 |
+
sk_pts=np.argwhere(skel_dom)
|
| 1030 |
+
for r,c in sk_pts:
|
| 1031 |
+
if 0<=r<512 and 0<=c<512: overlay[r,c]=[0,206,209]
|
| 1032 |
+
axes[1].imshow(overlay)
|
| 1033 |
+
for i,(wl,wr) in enumerate(wall_pairs):
|
| 1034 |
+
axes[1].plot([wl[1],wr[1]],[wl[0],wr[0]],
|
| 1035 |
+
color="lime",alpha=0.4,linewidth=0.6)
|
| 1036 |
+
for st in stenoses:
|
| 1037 |
+
idx=st["index"]
|
| 1038 |
+
if idx<len(centers):
|
| 1039 |
+
cp=centers[idx]
|
| 1040 |
+
axes[1].plot(cp[1],cp[0],"rv",markersize=10,zorder=6,
|
| 1041 |
+
markeredgecolor="white",markeredgewidth=0.5)
|
| 1042 |
+
axes[1].annotate(f'FFR≈{st["ffr"]:.2f}',
|
| 1043 |
+
xy=(cp[1],cp[0]),xytext=(cp[1]+14,cp[0]-14),
|
| 1044 |
+
color="red",fontsize=8,fontweight="bold",
|
| 1045 |
+
arrowprops=dict(arrowstyle="->",color="red",lw=0.8))
|
| 1046 |
+
axes[1].axis("off"); axes[1].set_title("Centerline + FFR markers", fontsize=11, color="#1a6fa8")
|
| 1047 |
+
|
| 1048 |
+
# Panel 3: diameter profile
|
| 1049 |
+
if len(arc_pos)>0:
|
| 1050 |
+
pos_mm=arc_pos/px_mm
|
| 1051 |
+
axes[2].plot(pos_mm,diams_px/px_mm,color="#cce0f0",lw=0.8,label="Raw")
|
| 1052 |
+
axes[2].plot(pos_mm,diams_sm/px_mm,color="#1a6fa8",lw=1.8,label="Smoothed")
|
| 1053 |
+
axes[2].axhline(ref_mm,color="#0d9e6e",ls="--",lw=1.2,label=f"Ref {ref_mm:.2f}mm")
|
| 1054 |
+
for st in stenoses:
|
| 1055 |
+
axes[2].axvline(st["position_mm"],color="red",ls=":",lw=1.,alpha=0.8)
|
| 1056 |
+
axes[2].annotate(f'{st["pct_ds"]:.0f}%DS\nFFR≈{st["ffr"]:.2f}',
|
| 1057 |
+
xy=(st["position_mm"],st["d_mm"]),
|
| 1058 |
+
xytext=(st["position_mm"]+2,st["d_mm"]+ref_mm*0.15),
|
| 1059 |
+
color="red",fontsize=7,fontweight="bold",
|
| 1060 |
+
arrowprops=dict(arrowstyle="->",color="red",lw=0.7))
|
| 1061 |
+
axes[2].set_xlabel("Position (mm)",fontsize=9)
|
| 1062 |
+
axes[2].set_ylabel("Diameter (mm)",fontsize=9)
|
| 1063 |
+
axes[2].legend(fontsize=8); axes[2].grid(alpha=0.3); axes[2].set_ylim(0)
|
| 1064 |
+
axes[2].set_title("Lumen diameter profile", fontsize=11, color="#1a6fa8")
|
| 1065 |
+
|
| 1066 |
+
ffr_label = ("⚠ ISCHEMIC" if global_ffr<=0.80 else "✓ Non-ischemic")
|
| 1067 |
+
fig.suptitle(f"FFR Analysis — Global FFR: {global_ffr:.3f} {ffr_label} | "
|
| 1068 |
+
f"Stenoses: {len(stenoses)} | Ref: {ref_mm:.2f} mm",
|
| 1069 |
+
fontsize=12,fontweight="bold",color="#1a2533",y=1.01)
|
| 1070 |
+
plt.tight_layout()
|
| 1071 |
+
|
| 1072 |
+
# Convert figure to numpy array — compatible with all matplotlib versions
|
| 1073 |
+
fig.canvas.draw()
|
| 1074 |
+
w, h = fig.canvas.get_width_height()
|
| 1075 |
+
try:
|
| 1076 |
+
# matplotlib >= 3.8
|
| 1077 |
+
buf = np.frombuffer(fig.canvas.buffer_rgba(), dtype=np.uint8)
|
| 1078 |
+
buf = buf.reshape(h, w, 4)[:, :, :3] # drop alpha channel
|
| 1079 |
+
except AttributeError:
|
| 1080 |
+
# matplotlib < 3.8 fallback
|
| 1081 |
+
buf = np.frombuffer(fig.canvas.tostring_rgb(), dtype=np.uint8)
|
| 1082 |
+
buf = buf.reshape(h, w, 3)
|
| 1083 |
+
plt.close(fig)
|
| 1084 |
+
|
| 1085 |
+
result = {
|
| 1086 |
+
"global_ffr" : round(global_ffr,3),
|
| 1087 |
+
"ischemic" : global_ffr<=0.80,
|
| 1088 |
+
"n_stenoses" : len(stenoses),
|
| 1089 |
+
"stenoses" : stenoses,
|
| 1090 |
+
"ref_diam_mm" : round(ref_mm,2),
|
| 1091 |
+
"vessel_len_mm": round(float(arc_pos[-1]/px_mm),1) if len(arc_pos)>0 else 0,
|
| 1092 |
+
"n_unlocalized": len(unloc),
|
| 1093 |
+
}
|
| 1094 |
+
return result, buf
|
| 1095 |
+
|
| 1096 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1097 |
+
# MODULE 6 — nnUNet OVERLAY
|
| 1098 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1099 |
+
|
| 1100 |
+
# 26 maximally distinct colours — one per ARCADE class (0 = background, black)
|
| 1101 |
+
# Generated by evenly spacing hue across HSV wheel at two brightness levels,
|
| 1102 |
+
# then manually verified for distinctness. Each colour is unique.
|
| 1103 |
+
# Format: (R, G, B)
|
| 1104 |
+
ARCADE_COLOURS = {
|
| 1105 |
+
0 : ( 0, 0, 0), # background — black (not rendered)
|
| 1106 |
+
1 : (255, 0, 0), # LMCA — red
|
| 1107 |
+
2 : ( 0, 120, 255), # LAD proximal — azure blue
|
| 1108 |
+
3 : ( 0, 220, 0), # LAD mid — lime green
|
| 1109 |
+
4 : (255, 165, 0), # LAD distal — orange
|
| 1110 |
+
5 : (180, 0, 255), # D1 — violet
|
| 1111 |
+
6 : ( 0, 230, 230), # D2 — cyan
|
| 1112 |
+
7 : (255, 230, 0), # D3 — yellow
|
| 1113 |
+
8 : (255, 0, 180), # Septal — hot pink
|
| 1114 |
+
9 : ( 0, 180, 80), # LAD var — jade green
|
| 1115 |
+
10: (255, 100, 0), # LCX proximal — deep orange
|
| 1116 |
+
11: ( 40, 40, 255), # LCX mid — royal blue
|
| 1117 |
+
12: ( 0, 255, 140), # LCX distal — spring green
|
| 1118 |
+
13: (220, 0, 80), # OM1 — crimson
|
| 1119 |
+
14: ( 0, 200, 255), # OM2 — sky blue
|
| 1120 |
+
15: (200, 255, 0), # OM3 — chartreuse
|
| 1121 |
+
16: (140, 0, 200), # LCX var — purple
|
| 1122 |
+
17: (255, 200, 120), # RCA proximal — peach
|
| 1123 |
+
18: ( 80, 255, 80), # RCA mid — bright green
|
| 1124 |
+
19: (255, 60, 200), # RCA distal — rose pink
|
| 1125 |
+
20: ( 0, 80, 200), # PDA — cobalt blue
|
| 1126 |
+
21: (200, 140, 0), # PLV — dark amber
|
| 1127 |
+
22: ( 80, 200, 200), # AM branch — teal
|
| 1128 |
+
23: (255, 255, 120), # Conus — light yellow
|
| 1129 |
+
24: (160, 60, 0), # SAN artery — brown
|
| 1130 |
+
25: (180, 180, 180), # Other — silver grey
|
| 1131 |
+
}
|
| 1132 |
+
|
| 1133 |
+
ARCADE_LABELS = {
|
| 1134 |
+
1:"LMCA", 2:"LAD prox", 3:"LAD mid", 4:"LAD dist",
|
| 1135 |
+
5:"D1", 6:"D2", 7:"D3", 8:"Septal",
|
| 1136 |
+
9:"LAD var", 10:"LCX prox", 11:"LCX mid", 12:"LCX dist",
|
| 1137 |
+
13:"OM1", 14:"OM2", 15:"OM3", 16:"LCX var",
|
| 1138 |
+
17:"RCA prox",18:"RCA mid", 19:"RCA dist", 20:"PDA",
|
| 1139 |
+
21:"PLV", 22:"AM", 23:"Conus", 24:"SAN",
|
| 1140 |
+
25:"Other",
|
| 1141 |
+
}
|
| 1142 |
+
|
| 1143 |
+
|
| 1144 |
+
def render_nnunet_overlay(frame_rgb, seg_map, alpha=0.55):
|
| 1145 |
+
"""
|
| 1146 |
+
Renders nnUNet segmentation overlay.
|
| 1147 |
+
Handles two cases:
|
| 1148 |
+
- Binary output (classes 0,1): single vessel colour overlay
|
| 1149 |
+
- 26-class output (ARCADE SYNTAX): full colour-coded overlay with legend
|
| 1150 |
+
"""
|
| 1151 |
+
unique_cls = [c for c in np.unique(seg_map) if c > 0]
|
| 1152 |
+
overlay = frame_rgb.copy().astype(np.float32)
|
| 1153 |
+
present_classes = []
|
| 1154 |
+
|
| 1155 |
+
# ── Binary case (0/1 only) ────────────────────────────────────────────────
|
| 1156 |
+
if len(unique_cls) == 1 and unique_cls[0] == 1:
|
| 1157 |
+
mask = (seg_map == 1)
|
| 1158 |
+
col = np.array([0, 200, 255], dtype=np.float32) # cyan for vessel
|
| 1159 |
+
overlay[mask] = overlay[mask] * (1 - alpha) + col * alpha
|
| 1160 |
+
result = overlay.astype(np.uint8)
|
| 1161 |
+
# Simple label
|
| 1162 |
+
cv2.putText(result, "Vessel mask (binary)",
|
| 1163 |
+
(8, 20), cv2.FONT_HERSHEY_SIMPLEX,
|
| 1164 |
+
0.5, (0, 200, 255), 1, cv2.LINE_AA)
|
| 1165 |
+
return result
|
| 1166 |
+
|
| 1167 |
+
# ── 26-class case ─────────────────────────────────────────────────────────
|
| 1168 |
+
for cls_id, rgb_col in ARCADE_COLOURS.items():
|
| 1169 |
+
if cls_id == 0:
|
| 1170 |
+
continue
|
| 1171 |
+
mask = (seg_map == cls_id)
|
| 1172 |
+
if not mask.any():
|
| 1173 |
+
continue
|
| 1174 |
+
col = np.array(rgb_col, dtype=np.float32)
|
| 1175 |
+
overlay[mask] = overlay[mask] * (1 - alpha) + col * alpha
|
| 1176 |
+
present_classes.append(cls_id)
|
| 1177 |
+
|
| 1178 |
+
result = overlay.astype(np.uint8)
|
| 1179 |
+
|
| 1180 |
+
# Compact legend — bottom-left, only present classes, max 13 rows
|
| 1181 |
+
if present_classes:
|
| 1182 |
+
try:
|
| 1183 |
+
n_rows = min(len(present_classes), 13)
|
| 1184 |
+
row_h = 18
|
| 1185 |
+
legend_h = n_rows * row_h + 10
|
| 1186 |
+
legend_w = 148
|
| 1187 |
+
H, W = result.shape[:2]
|
| 1188 |
+
x0, y0 = 6, H - legend_h - 6
|
| 1189 |
+
sub = result[y0:y0+legend_h, x0:x0+legend_w].astype(np.float32)
|
| 1190 |
+
result[y0:y0+legend_h, x0:x0+legend_w] = (sub * 0.4).astype(np.uint8)
|
| 1191 |
+
for i, cls_id in enumerate(present_classes[:13]):
|
| 1192 |
+
r, g, b = ARCADE_COLOURS[cls_id]
|
| 1193 |
+
lbl = ARCADE_LABELS.get(cls_id, f"cls{cls_id}")
|
| 1194 |
+
yy = y0 + 6 + i * row_h
|
| 1195 |
+
cv2.rectangle(result, (x0+4, yy), (x0+16, yy+11), (b, g, r), -1)
|
| 1196 |
+
cv2.rectangle(result, (x0+4, yy), (x0+16, yy+11), (255,255,255), 1)
|
| 1197 |
+
cv2.putText(result, lbl, (x0+20, yy+9),
|
| 1198 |
+
cv2.FONT_HERSHEY_SIMPLEX, 0.35,
|
| 1199 |
+
(255,255,255), 1, cv2.LINE_AA)
|
| 1200 |
+
except Exception:
|
| 1201 |
+
pass
|
| 1202 |
+
|
| 1203 |
+
return result
|
| 1204 |
+
|
| 1205 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1206 |
+
# MODULE 7 — SYNTAX SCORE (simplified anatomical estimate)
|
| 1207 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1208 |
+
|
| 1209 |
+
def compute_syntax_score(stenoses, seg_map):
|
| 1210 |
+
"""
|
| 1211 |
+
Simplified SYNTAX estimate:
|
| 1212 |
+
Each stenosis contributes a weight based on DS% and
|
| 1213 |
+
anatomical territory (estimated from nnUNet class ID).
|
| 1214 |
+
Reference: Sianos et al. EuroIntervention 2005.
|
| 1215 |
+
Note: true SYNTAX requires angiographic expertise; this is
|
| 1216 |
+
an automated surrogate for display purposes.
|
| 1217 |
+
"""
|
| 1218 |
+
if not stenoses:
|
| 1219 |
+
return 0.0, "Low (0)"
|
| 1220 |
+
|
| 1221 |
+
base_score = 0.
|
| 1222 |
+
for st in stenoses:
|
| 1223 |
+
ds = st["pct_ds"] / 100
|
| 1224 |
+
# Weight by position: proximal vessels score higher
|
| 1225 |
+
pos_w = max(0.5, 1.5 - st["position_mm"] / 100)
|
| 1226 |
+
# Occlusion multiplier
|
| 1227 |
+
occ_w = 5.0 if ds >= 0.99 else (2.5 if ds >= 0.70 else 1.5 if ds >= 0.50 else 1.0)
|
| 1228 |
+
base_score += pos_w * occ_w * (1 + ds)
|
| 1229 |
+
|
| 1230 |
+
# Add segment count from nnUNet
|
| 1231 |
+
n_segs = len(np.unique(seg_map)) - 1 # exclude background
|
| 1232 |
+
base_score += n_segs * 0.3
|
| 1233 |
+
|
| 1234 |
+
score = round(base_score, 1)
|
| 1235 |
+
if score < 23: tier = "Low (<23)"
|
| 1236 |
+
elif score < 33: tier = "Intermediate (23–32)"
|
| 1237 |
+
else: tier = "High (≥33)"
|
| 1238 |
+
return score, tier
|
| 1239 |
+
|
| 1240 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1241 |
+
# JSON ENCODER — handles numpy float32/int64 etc.
|
| 1242 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1243 |
+
|
| 1244 |
+
class _NpEncoder(json.JSONEncoder):
|
| 1245 |
+
def default(self, obj):
|
| 1246 |
+
if isinstance(obj, np.integer): return int(obj)
|
| 1247 |
+
if isinstance(obj, np.floating): return float(obj)
|
| 1248 |
+
if isinstance(obj, np.ndarray): return obj.tolist()
|
| 1249 |
+
if isinstance(obj, np.bool_): return bool(obj)
|
| 1250 |
+
return super().default(obj)
|
| 1251 |
+
|
| 1252 |
+
|
| 1253 |
+
def _to_json(obj) -> str:
|
| 1254 |
+
return json.dumps(obj, indent=2, cls=_NpEncoder)
|
| 1255 |
+
|
| 1256 |
+
|
| 1257 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1258 |
+
# MAIN PIPELINE FUNCTION
|
| 1259 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1260 |
+
|
| 1261 |
+
def analyse(video_path, sten_conf, seg_conf, px_per_mm_override, progress=gr.Progress()):
|
| 1262 |
+
"""
|
| 1263 |
+
Full pipeline. Called by Gradio on button click.
|
| 1264 |
+
Returns: (status_html, frame_img, seg_img, ffr_img, metrics_html, json_str)
|
| 1265 |
+
"""
|
| 1266 |
+
if video_path is None:
|
| 1267 |
+
return (_status("error","No video uploaded."), None, None, None, "", "")
|
| 1268 |
+
|
| 1269 |
+
FFR_CFG["px_per_mm"] = px_per_mm_override
|
| 1270 |
+
|
| 1271 |
+
try:
|
| 1272 |
+
# Step 1 — keyframe
|
| 1273 |
+
progress(0.10, desc="Extracting best frame…")
|
| 1274 |
+
frame_rgb, frame_idx, frame_score, total_frames = extract_best_frame(video_path)
|
| 1275 |
+
|
| 1276 |
+
# Step 2 — stenosis detection
|
| 1277 |
+
progress(0.30, desc="Running Mask2Former…")
|
| 1278 |
+
detections = run_mask2former(frame_rgb, sten_conf)
|
| 1279 |
+
|
| 1280 |
+
# Compute vessel overlap for each detection
|
| 1281 |
+
binary_mask_for_overlap = np.ones((512,512),dtype=np.uint8) # placeholder until ResUNet
|
| 1282 |
+
for det in detections:
|
| 1283 |
+
b=det.get("bbox",[]);
|
| 1284 |
+
if len(b)<4: det["overlap"]=0.; continue
|
| 1285 |
+
x1,y1,x2,y2=[int(v) for v in b]
|
| 1286 |
+
area=(x2-x1)*(y2-y1)
|
| 1287 |
+
det["overlap"]=min(float(binary_mask_for_overlap[y1:y2,x1:x2].sum())/max(area,1),1.)
|
| 1288 |
+
|
| 1289 |
+
# Step 3a — binary segmentation (ResUNet)
|
| 1290 |
+
progress(0.50, desc="Running ResUNet…")
|
| 1291 |
+
binary_mask, binary_overlay = run_resunet(frame_rgb)
|
| 1292 |
+
print(f"[ResUNet] binary_mask sum={binary_mask.sum()} "
|
| 1293 |
+
f"unique={np.unique(binary_mask)} "
|
| 1294 |
+
f"vessel%={100*binary_mask.mean():.1f}")
|
| 1295 |
+
# Recompute overlap with real mask
|
| 1296 |
+
for det in detections:
|
| 1297 |
+
b=det.get("bbox",[])
|
| 1298 |
+
if len(b)<4: continue
|
| 1299 |
+
x1,y1,x2,y2=[int(v) for v in b]
|
| 1300 |
+
area=(x2-x1)*(y2-y1)
|
| 1301 |
+
det["overlap"]=float(binary_mask[y1:y2,x1:x2].sum())/max(area,1)
|
| 1302 |
+
|
| 1303 |
+
# Step 3b — YOLOv8m-seg 26-class segmentation
|
| 1304 |
+
progress(0.65, desc="Running YOLOv8m-seg…")
|
| 1305 |
+
seg_map = run_yolo_seg(frame_rgb, seg_conf)
|
| 1306 |
+
seg_overlay = render_nnunet_overlay(frame_rgb, seg_map)
|
| 1307 |
+
|
| 1308 |
+
# Step 4 — FFR
|
| 1309 |
+
progress(0.80, desc="Computing FFR…")
|
| 1310 |
+
ffr_result, ffr_fig = run_ffr(frame_rgb, binary_mask, detections)
|
| 1311 |
+
|
| 1312 |
+
if "error" in ffr_result:
|
| 1313 |
+
return (_status("warn", ffr_result["error"]),
|
| 1314 |
+
frame_rgb, binary_overlay, seg_overlay, None,
|
| 1315 |
+
_metrics_html(ffr_result, {}, frame_idx, total_frames),
|
| 1316 |
+
_to_json(ffr_result))
|
| 1317 |
+
|
| 1318 |
+
# Step 5 — SYNTAX score
|
| 1319 |
+
progress(0.92, desc="Computing SYNTAX score…")
|
| 1320 |
+
syntax_score, syntax_tier = compute_syntax_score(ffr_result["stenoses"], seg_map)
|
| 1321 |
+
ffr_result["syntax_score"] = syntax_score
|
| 1322 |
+
ffr_result["syntax_tier"] = syntax_tier
|
| 1323 |
+
|
| 1324 |
+
progress(1.00, desc="Done.")
|
| 1325 |
+
status = _status("ok",
|
| 1326 |
+
f"Analysis complete — Frame {frame_idx}/{total_frames} "
|
| 1327 |
+
f"(quality score {frame_score:.1f})")
|
| 1328 |
+
metrics = _metrics_html(ffr_result, {"score":frame_score,"idx":frame_idx},
|
| 1329 |
+
frame_idx, total_frames)
|
| 1330 |
+
|
| 1331 |
+
return (status, frame_rgb, binary_overlay, seg_overlay, ffr_fig, metrics,
|
| 1332 |
+
_to_json(ffr_result))
|
| 1333 |
+
|
| 1334 |
+
except FileNotFoundError as e:
|
| 1335 |
+
return (_status("error", str(e)), None, None, None, None,
|
| 1336 |
+
_metrics_html({},{},0,0),
|
| 1337 |
+
_to_json({"error":str(e)}))
|
| 1338 |
+
except Exception as e:
|
| 1339 |
+
import traceback
|
| 1340 |
+
tb = traceback.format_exc()
|
| 1341 |
+
return (_status("error", f"{type(e).__name__}: {e}"), None, None, None, None,
|
| 1342 |
+
_metrics_html({},{},0,0),
|
| 1343 |
+
_to_json({"error":str(e),"traceback":tb}))
|
| 1344 |
+
|
| 1345 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1346 |
+
# HTML HELPERS
|
| 1347 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1348 |
+
|
| 1349 |
+
def _status(kind: str, msg: str) -> str:
|
| 1350 |
+
colours = {"ok":"#0d9e6e","warn":"#d98b00","error":"#d64040"}
|
| 1351 |
+
icons = {"ok":"✓","warn":"⚠","error":"✗"}
|
| 1352 |
+
col = colours.get(kind, "#1a6fa8")
|
| 1353 |
+
ico = icons.get(kind, "•")
|
| 1354 |
+
return (f'<div id="status-box" style="border-color:{col};color:{col};'
|
| 1355 |
+
f'background:{"#f0fbf7" if kind=="ok" else "#fff8f0" if kind=="warn" else "#fff0f0"}">'
|
| 1356 |
+
f'{ico} {msg}</div>')
|
| 1357 |
+
|
| 1358 |
+
|
| 1359 |
+
def _metrics_html(ffr_result, frame_info, frame_idx, total) -> str:
|
| 1360 |
+
if not ffr_result or "error" in ffr_result:
|
| 1361 |
+
return '<div class="card"><div class="card-title">Metrics</div><p style="color:#5a7a96">Run analysis to see results.</p></div>'
|
| 1362 |
+
|
| 1363 |
+
gffr = ffr_result.get("global_ffr", "—")
|
| 1364 |
+
isch = ffr_result.get("ischemic", None)
|
| 1365 |
+
ns = ffr_result.get("n_stenoses", 0)
|
| 1366 |
+
ref = ffr_result.get("ref_diam_mm","—")
|
| 1367 |
+
vlen = ffr_result.get("vessel_len_mm","—")
|
| 1368 |
+
syn = ffr_result.get("syntax_score","—")
|
| 1369 |
+
stier = ffr_result.get("syntax_tier","—")
|
| 1370 |
+
|
| 1371 |
+
if isch is True: ffr_cls,ffr_lbl = "ischemic","⚠ ISCHEMIC"
|
| 1372 |
+
elif isch is False: ffr_cls,ffr_lbl = "ok","✓ NON-ISCHEMIC"
|
| 1373 |
+
else: ffr_cls,ffr_lbl = "","—"
|
| 1374 |
+
|
| 1375 |
+
syn_val = float(syn) if isinstance(syn,float) else 0
|
| 1376 |
+
syn_cls = "ischemic" if syn_val>=33 else ("borderline" if syn_val>=23 else "ok")
|
| 1377 |
+
|
| 1378 |
+
stenosis_rows = ""
|
| 1379 |
+
for i,st in enumerate(ffr_result.get("stenoses",[])):
|
| 1380 |
+
sig_col = "#d64040" if st.get("significant") else "#0d9e6e"
|
| 1381 |
+
stenosis_rows += (
|
| 1382 |
+
f'<tr>'
|
| 1383 |
+
f'<td style="padding:4px 8px">#{i+1}</td>'
|
| 1384 |
+
f'<td style="padding:4px 8px">{st["position_mm"]:.1f} mm</td>'
|
| 1385 |
+
f'<td style="padding:4px 8px">{st["pct_ds"]:.1f}%</td>'
|
| 1386 |
+
f'<td style="padding:4px 8px">{st["d_mm"]:.2f} mm</td>'
|
| 1387 |
+
f'<td style="padding:4px 8px;color:{sig_col};font-weight:600">{st["ffr"]:.3f}</td>'
|
| 1388 |
+
f'</tr>'
|
| 1389 |
+
)
|
| 1390 |
+
|
| 1391 |
+
return f"""
|
| 1392 |
+
<div class="card">
|
| 1393 |
+
<div class="card-title">Key metrics</div>
|
| 1394 |
+
<div class="metric-row">
|
| 1395 |
+
<div class="metric-badge">
|
| 1396 |
+
<div class="val {ffr_cls}">{gffr if gffr!="—" else "—"}</div>
|
| 1397 |
+
<div class="lbl">Global FFR</div>
|
| 1398 |
+
</div>
|
| 1399 |
+
<div class="metric-badge">
|
| 1400 |
+
<div class="val {ffr_cls}" style="font-size:14px">{ffr_lbl}</div>
|
| 1401 |
+
<div class="lbl">Ischaemia</div>
|
| 1402 |
+
</div>
|
| 1403 |
+
<div class="metric-badge">
|
| 1404 |
+
<div class="val">{ns}</div>
|
| 1405 |
+
<div class="lbl">Stenoses</div>
|
| 1406 |
+
</div>
|
| 1407 |
+
<div class="metric-badge">
|
| 1408 |
+
<div class="val {syn_cls}">{syn}</div>
|
| 1409 |
+
<div class="lbl">SYNTAX score</div>
|
| 1410 |
+
</div>
|
| 1411 |
+
<div class="metric-badge">
|
| 1412 |
+
<div class="val">{ref} mm</div>
|
| 1413 |
+
<div class="lbl">Ref diameter</div>
|
| 1414 |
+
</div>
|
| 1415 |
+
<div class="metric-badge">
|
| 1416 |
+
<div class="val">{vlen} mm</div>
|
| 1417 |
+
<div class="lbl">Vessel length</div>
|
| 1418 |
+
</div>
|
| 1419 |
+
</div>
|
| 1420 |
+
<div style="font-size:12px;color:#5a7a96;margin-top:4px">
|
| 1421 |
+
SYNTAX tier: <b>{stier}</b> |
|
| 1422 |
+
Frame {frame_idx}/{total} |
|
| 1423 |
+
Scale: {FFR_CFG["px_per_mm"]} px/mm
|
| 1424 |
+
</div>
|
| 1425 |
+
</div>
|
| 1426 |
+
{'<div class="card"><div class="card-title">Stenosis table</div><table style="width:100%;border-collapse:collapse;font-size:13px"><thead><tr style="background:#e8f3fb;color:#1a6fa8"><th style="padding:6px 8px;text-align:left">#</th><th style="padding:6px 8px;text-align:left">Position</th><th style="padding:6px 8px;text-align:left">DS%</th><th style="padding:6px 8px;text-align:left">Min diam</th><th style="padding:6px 8px;text-align:left">FFR</th></tr></thead><tbody>' + stenosis_rows + '</tbody></table></div>'
|
| 1427 |
+
if stenosis_rows else ""}
|
| 1428 |
+
"""
|
| 1429 |
+
|
| 1430 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1431 |
+
# GRADIO APP
|
| 1432 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1433 |
+
|
| 1434 |
+
with gr.Blocks(css=CSS, title="Angio AI") as demo:
|
| 1435 |
+
|
| 1436 |
+
# Header
|
| 1437 |
+
gr.HTML(_header_html())
|
| 1438 |
+
|
| 1439 |
+
with gr.Tabs() as tabs:
|
| 1440 |
+
|
| 1441 |
+
# ── PAGE 1: UPLOAD ────────────────────────────────────────────────
|
| 1442 |
+
with gr.TabItem("📤 Upload & Configure", id="tab-upload"):
|
| 1443 |
+
gr.HTML('<div style="height:12px"></div>')
|
| 1444 |
+
|
| 1445 |
+
with gr.Row():
|
| 1446 |
+
# Left — upload + controls
|
| 1447 |
+
with gr.Column(scale=2):
|
| 1448 |
+
gr.HTML('<div class="card"><div class="card-title">Angiographic video</div>')
|
| 1449 |
+
video_input = gr.Video(
|
| 1450 |
+
label="Upload XCA video (mp4 / avi / dicom-wrapped)",
|
| 1451 |
+
elem_id="upload-zone",
|
| 1452 |
+
height=240,
|
| 1453 |
+
)
|
| 1454 |
+
gr.HTML('</div>')
|
| 1455 |
+
|
| 1456 |
+
gr.HTML('<div class="card"><div class="card-title">Model controls</div>')
|
| 1457 |
+
sten_conf = gr.Slider(
|
| 1458 |
+
minimum=0.05, maximum=0.95, value=0.30, step=0.05,
|
| 1459 |
+
label="Stenosis confidence threshold (Mask2Former)",
|
| 1460 |
+
info="Detections below this confidence are discarded",
|
| 1461 |
+
)
|
| 1462 |
+
seg_conf = gr.Slider(
|
| 1463 |
+
minimum=0.05, maximum=0.95, value=0.25, step=0.05,
|
| 1464 |
+
label="Segmentation confidence threshold (YOLOv8m-seg)",
|
| 1465 |
+
info="Detections below this confidence are discarded by YOLO NMS",
|
| 1466 |
+
)
|
| 1467 |
+
px_per_mm = gr.Slider(
|
| 1468 |
+
minimum=2.0, maximum=6.0, value=3.75, step=0.25,
|
| 1469 |
+
label="Scale (px / mm)",
|
| 1470 |
+
info="ARCADE default = 3.75 px/mm (0.267 mm/px). Adjust if using non-ARCADE data.",
|
| 1471 |
+
)
|
| 1472 |
+
gr.HTML('</div>')
|
| 1473 |
+
|
| 1474 |
+
# Right — info card
|
| 1475 |
+
with gr.Column(scale=1):
|
| 1476 |
+
gr.HTML("""
|
| 1477 |
+
<div class="card">
|
| 1478 |
+
<div class="card-title">Pipeline overview</div>
|
| 1479 |
+
<ol style="font-size:13px;color:#1a2533;line-height:1.9;padding-left:16px">
|
| 1480 |
+
<li>Best frame extracted from video</li>
|
| 1481 |
+
<li>Stenosis detection — Mask2Former</li>
|
| 1482 |
+
<li>Coronary segmentation — YOLOv8m-seg (26-class)</li>
|
| 1483 |
+
<li>Binary vessel mask — ResUNet</li>
|
| 1484 |
+
<li>FFR estimation — QFR v4</li>
|
| 1485 |
+
<li>SYNTAX score computation</li>
|
| 1486 |
+
</ol>
|
| 1487 |
+
<hr style="border-color:#cce0f0;margin:12px 0">
|
| 1488 |
+
<div style="font-size:12px;color:#5a7a96">
|
| 1489 |
+
<b>FFR threshold:</b> ≤ 0.80 → ischemic<br>
|
| 1490 |
+
<b>Formula:</b> 1 − (0.33·DS + 0.60·DS²)<br>
|
| 1491 |
+
<b>Reference:</b> Tu et al. JACC 2016
|
| 1492 |
+
</div>
|
| 1493 |
+
</div>
|
| 1494 |
+
<div class="card">
|
| 1495 |
+
<div class="card-title">Model checkpoints</div>
|
| 1496 |
+
<div style="font-size:12px;color:#5a7a96;line-height:1.8">
|
| 1497 |
+
Auto-downloaded from<br>
|
| 1498 |
+
<code>MuhammadAdil63/</code><br>
|
| 1499 |
+
<code>angio-ai-checkpoints</code><br><br>
|
| 1500 |
+
<code>mask2former_best.pth</code><br>
|
| 1501 |
+
<code>binary_best.pth</code><br>
|
| 1502 |
+
<code>best.pt</code>
|
| 1503 |
+
</div>
|
| 1504 |
+
</div>
|
| 1505 |
+
""")
|
| 1506 |
+
|
| 1507 |
+
with gr.Row():
|
| 1508 |
+
with gr.Column(scale=2):
|
| 1509 |
+
btn_analyse = gr.Button("▶ Run analysis",
|
| 1510 |
+
elem_id="btn-analyse", variant="primary")
|
| 1511 |
+
with gr.Column(scale=1):
|
| 1512 |
+
btn_reset = gr.Button("↺ Clear", elem_id="btn-reset")
|
| 1513 |
+
|
| 1514 |
+
status_out = gr.HTML(
|
| 1515 |
+
'<div id="status-box">Upload a video and press Run analysis.</div>')
|
| 1516 |
+
|
| 1517 |
+
# ── PAGE 2: RESULTS ───────────────────────────────────────────────
|
| 1518 |
+
with gr.TabItem("📊 Results", id="tab-results"):
|
| 1519 |
+
gr.HTML('<div style="height:12px"></div>')
|
| 1520 |
+
|
| 1521 |
+
status_out2 = gr.HTML()
|
| 1522 |
+
metrics_out = gr.HTML()
|
| 1523 |
+
|
| 1524 |
+
with gr.Row():
|
| 1525 |
+
with gr.Column():
|
| 1526 |
+
gr.HTML('<div class="card-title" style="margin-bottom:6px">Best frame extracted</div>')
|
| 1527 |
+
frame_out = gr.Image(label="Best keyframe", height=300)
|
| 1528 |
+
with gr.Column():
|
| 1529 |
+
gr.HTML('<div class="card-title" style="margin-bottom:6px">Binary vessel mask (ResUNet)</div>')
|
| 1530 |
+
binary_out = gr.Image(label="Green overlay — vessel pixels", height=300)
|
| 1531 |
+
|
| 1532 |
+
with gr.Row():
|
| 1533 |
+
with gr.Column():
|
| 1534 |
+
gr.HTML('<div class="card-title" style="margin-bottom:6px">Coronary segmentation (YOLOv8m-seg)</div>')
|
| 1535 |
+
seg_out = gr.Image(label="26-class overlay", height=300)
|
| 1536 |
+
with gr.Column():
|
| 1537 |
+
gr.HTML('<div class="card-title" style="margin-bottom:6px">FFR analysis</div>')
|
| 1538 |
+
ffr_out = gr.Image(label="FFR pipeline output", height=300)
|
| 1539 |
+
|
| 1540 |
+
with gr.Row():
|
| 1541 |
+
with gr.Column():
|
| 1542 |
+
gr.HTML('<div class="card-title" style="margin-bottom:6px">Full results (JSON)</div>')
|
| 1543 |
+
json_out = gr.Code(label="", language="json", lines=12)
|
| 1544 |
+
|
| 1545 |
+
# Footer
|
| 1546 |
+
gr.HTML("""
|
| 1547 |
+
<div id="footer">
|
| 1548 |
+
Angio AI · MS Data Science · Information Technology University Lahore ·
|
| 1549 |
+
Supervisor: Dr. Arif Mehmood |
|
| 1550 |
+
Image-based QFR — Tu et al. JACC 2016 · FAVOR II trial
|
| 1551 |
+
</div>""")
|
| 1552 |
+
|
| 1553 |
+
# ── WIRING ────────────────────────────────────────────────────────────
|
| 1554 |
+
|
| 1555 |
+
def run_and_switch(video, sc, sg, px):
|
| 1556 |
+
return analyse(video, sc, sg, px)
|
| 1557 |
+
|
| 1558 |
+
btn_analyse.click(
|
| 1559 |
+
fn=run_and_switch,
|
| 1560 |
+
inputs=[video_input, sten_conf, seg_conf, px_per_mm],
|
| 1561 |
+
outputs=[status_out, frame_out, binary_out, seg_out, ffr_out, metrics_out, json_out],
|
| 1562 |
+
).then(
|
| 1563 |
+
# Copy status HTML to results page status box
|
| 1564 |
+
fn=lambda s: s,
|
| 1565 |
+
inputs=[status_out],
|
| 1566 |
+
outputs=[status_out2],
|
| 1567 |
+
).then(
|
| 1568 |
+
# Switch to Results tab via JavaScript
|
| 1569 |
+
fn=None,
|
| 1570 |
+
js="""
|
| 1571 |
+
() => {
|
| 1572 |
+
const tabs = document.querySelectorAll('.tab-nav button');
|
| 1573 |
+
if (tabs.length >= 2) tabs[1].click();
|
| 1574 |
+
}
|
| 1575 |
+
""",
|
| 1576 |
+
)
|
| 1577 |
+
|
| 1578 |
+
btn_reset.click(
|
| 1579 |
+
fn=lambda: (
|
| 1580 |
+
None, None, None, None, None,
|
| 1581 |
+
'<div id="status-box">Upload a video and press Run analysis.</div>',
|
| 1582 |
+
"", "",
|
| 1583 |
+
),
|
| 1584 |
+
outputs=[video_input, frame_out, binary_out, seg_out, ffr_out,
|
| 1585 |
+
status_out, metrics_out, json_out],
|
| 1586 |
+
)
|
| 1587 |
+
|
| 1588 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1589 |
+
# ENTRY POINT
|
| 1590 |
+
# ─────────────────────────────────────────────────────────────────────────────
|
| 1591 |
+
|
| 1592 |
+
if __name__ == "__main__":
|
| 1593 |
+
demo.launch(
|
| 1594 |
+
server_name="0.0.0.0",
|
| 1595 |
+
server_port=7860,
|
| 1596 |
+
share=False,
|
| 1597 |
+
inbrowser=False, # must be False on HF Spaces
|
| 1598 |
+
)
|
packages.txt
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# System packages installed via apt before pip
|
| 2 |
+
# Required by opencv-python-headless on HF Spaces Ubuntu base image
|
| 3 |
+
libgl1-mesa-glx
|
| 4 |
+
libglib2.0-0
|
| 5 |
+
libsm6
|
| 6 |
+
libxext6
|
| 7 |
+
libxrender-dev
|
requirements.txt
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ── Angio AI — HuggingFace Spaces requirements ────────────────────────────────
|
| 2 |
+
# Python 3.10 | CPU-only free tier
|
| 3 |
+
# Pin versions that are known to work together on HF Spaces CPU.
|
| 4 |
+
|
| 5 |
+
# ── Core ML ───────────────────────────────────────────────────────────────────
|
| 6 |
+
torch==2.2.2+cpu
|
| 7 |
+
torchvision==0.17.2+cpu
|
| 8 |
+
--extra-index-url https://download.pytorch.org/whl/cpu
|
| 9 |
+
|
| 10 |
+
# ── Transformers (Mask2Former) ────────────────────────────────────────────────
|
| 11 |
+
transformers==4.40.2
|
| 12 |
+
accelerate==0.30.0
|
| 13 |
+
|
| 14 |
+
# ── YOLO (ultralytics) ───────────────────────────────────────────────────────
|
| 15 |
+
ultralytics==8.2.18
|
| 16 |
+
|
| 17 |
+
# ── HuggingFace Hub (checkpoint download) ────────────────────────────────────
|
| 18 |
+
huggingface_hub==0.23.2
|
| 19 |
+
|
| 20 |
+
# ── Computer Vision ───────────────────────────────────────────────────────────
|
| 21 |
+
opencv-python-headless==4.9.0.80
|
| 22 |
+
scikit-image==0.22.0
|
| 23 |
+
scipy==1.13.0
|
| 24 |
+
|
| 25 |
+
# ── Visualisation / UI ────────────────────────────────────────────────────────
|
| 26 |
+
gradio==4.31.5
|
| 27 |
+
matplotlib==3.8.4
|
| 28 |
+
numpy==1.26.4
|