MuhammadAdil63 commited on
Commit
a185e27
·
1 Parent(s): 0e91bec

Initial deployment — Angio AI

Browse files
Files changed (4) hide show
  1. README.md +52 -9
  2. app.py +1598 -0
  3. packages.txt +7 -0
  4. requirements.txt +28 -0
README.md CHANGED
@@ -1,15 +1,58 @@
1
  ---
2
- title: AngioAI
3
- emoji: ⚡
4
- colorFrom: indigo
5
- colorTo: red
6
  sdk: gradio
7
- sdk_version: 6.18.0
8
- python_version: '3.13'
9
  app_file: app.py
10
  pinned: false
11
- license: apache-2.0
12
- short_description: Automation of Angiography Segment and Stenosis Detection
13
  ---
14
 
15
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Angio AI
3
+ emoji: 🫀
4
+ colorFrom: red
5
+ colorTo: blue
6
  sdk: gradio
7
+ sdk_version: 4.31.5
 
8
  app_file: app.py
9
  pinned: false
10
+ license: mit
 
11
  ---
12
 
13
+ # Angio AI — Coronary Angiography Analysis System
14
+
15
+ **MS Data Science Thesis · Information Technology University Lahore**
16
+ **Supervisor: Dr. Arif Mehmood**
17
+
18
+ ---
19
+
20
+ ## What it does
21
+
22
+ Upload a coronary X-ray angiography (XCA) video and the pipeline automatically:
23
+
24
+ | Step | Module | Output |
25
+ |------|--------|--------|
26
+ | 1 | Keyframe extraction | Best diagnostic frame (contrast × sharpness score) |
27
+ | 2 | Mask2Former | Stenosis detection — bounding boxes + instance masks |
28
+ | 3a | ResUNet | Binary vessel mask (Dice 0.8015 on ARCADE) |
29
+ | 3b | YOLOv8m-seg | 26-class coronary anatomy segmentation |
30
+ | 4 | FFR Pipeline v4 | Quantitative Flow Ratio (QFR) estimation |
31
+ | 5 | SYNTAX Score | Lesion complexity scoring |
32
+
33
+ ---
34
+
35
+ ## Models
36
+
37
+ All checkpoints are stored in [`MuhammadAdil63/angio-ai-checkpoints`](https://huggingface.co/MuhammadAdil63/angio-ai-checkpoints) and downloaded automatically on first run.
38
+
39
+ | File | Architecture | Task | Dice / mAP |
40
+ |------|-------------|------|-----------|
41
+ | `mask2former_best.pth` | Mask2Former Swin-Base | Stenosis detection | — |
42
+ | `binary_best.pth` | ResUNet (16→256 ch) | Binary vessel seg | 0.8015 |
43
+ | `best.pt` | YOLOv8m-seg nc=26 | 26-class anatomy seg | — |
44
+
45
+ Dataset: [ARCADE Challenge](https://arcade.grand-challenge.org/) (syntax + stenosis splits)
46
+
47
+ ---
48
+
49
+ ## Notes
50
+
51
+ - **Hardware**: CPU-only (HF free tier) — inference takes ~30–60 seconds per video
52
+ - **Input**: MP4 / AVI coronary angiography video, ideally ≥5 seconds
53
+ - **Scale**: Default 3.75 px/mm (ARCADE hardware). Adjust slider for non-ARCADE data.
54
+ - **FFR formula**: QFR = 1 − (0.33·DS + 0.60·DS²) — Tu et al. JACC 2016 / FAVOR II
55
+
56
+ ---
57
+
58
+ *This tool is for research purposes only and is not intended for clinical use.*
app.py ADDED
@@ -0,0 +1,1598 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ── angio_ai_app.py ─────────────────────────────────────────────────────────────────────────────
2
+ # Angio AI — Coronary Angiography Analysis System
3
+ # Muhammad Adil · MS Data Science · ITU Lahore
4
+ # Supervisor: Dr. Arif Mehmood
5
+ #
6
+ # ── DEPLOYMENT — HuggingFace Spaces ──────────────────────────────────────────
7
+ # Checkpoints are loaded from HF Hub: MuhammadAdil63/angio-ai-checkpoints
8
+ # Set HF_TOKEN as a Space Secret (Settings → Secrets) for private repo access.
9
+ # Files expected in that repo:
10
+ # mask2former_best.pth (Mask2Former Swin-Base fine-tuned)
11
+ # binary_best.pth (ResUNet binary vessel segmentation)
12
+ # best.pt (YOLOv8m-seg 26-class ARCADE syntax)
13
+ #
14
+ # ── DEPENDENCIES ────────────────────────────────────────────────────────────────────────────
15
+ # pip install gradio torch torchvision opencv-python-headless
16
+ # scikit-image scipy matplotlib transformers
17
+ # ultralytics (for YOLOv8m-seg inference)
18
+ # ─────────────────────────────────────────────────────────────────────────────
19
+
20
+ import os
21
+ import sys
22
+ import json
23
+ import base64
24
+ import tempfile
25
+ import warnings
26
+ warnings.filterwarnings("ignore")
27
+
28
+ import cv2
29
+ import numpy as np
30
+ import matplotlib
31
+ matplotlib.use("Agg")
32
+ import matplotlib.pyplot as plt
33
+ import matplotlib.patches as mpatches
34
+ from pathlib import Path
35
+
36
+ import gradio as gr
37
+ import torch
38
+ import huggingface_hub # noqa: F401 (ensure installed; used in _hf_ckpt)
39
+
40
+ # ─────────────────────────────────────────────────────────────────────────────
41
+ # CONFIG — edit these paths before running
42
+ # ─────────────────────────────────────────────────────────────────────────────
43
+
44
+ # ── Checkpoint download from HuggingFace Hub ─────────────────────────────────
45
+ # Checkpoints live in: MuhammadAdil63/angio-ai-checkpoints (private repo)
46
+ # huggingface_hub reads HF_TOKEN from environment (set as a Space secret).
47
+ def _hf_ckpt(filename: str) -> str:
48
+ """Download checkpoint from HF Hub on first call, return local cache path."""
49
+ from huggingface_hub import hf_hub_download
50
+ return hf_hub_download(
51
+ repo_id = "MuhammadAdil63/angio-ai-checkpoints",
52
+ filename = filename,
53
+ repo_type= "model",
54
+ token = os.environ.get("HF_TOKEN"), # set in Space Secrets
55
+ )
56
+
57
+ CONFIG = {
58
+ # ── Checkpoint paths (resolved lazily via HF Hub) ─────────────────────────
59
+ # These callables are invoked once inside each model's _load_*() function.
60
+ "MASK2FORMER_CKPT" : lambda: _hf_ckpt("mask2former_best.pth"),
61
+ "RESUNET_CKPT" : lambda: _hf_ckpt("binary_best.pth"),
62
+ "YOLO_CKPT" : lambda: _hf_ckpt("best.pt"),
63
+
64
+ # ── YOLO inference params ─────────────────────────────────────────────────
65
+ "YOLO_CONF" : 0.25,
66
+ "YOLO_IOU" : 0.70,
67
+ "YOLO_IMGSZ" : 512,
68
+
69
+ # ── FFR pipeline scale (ARCADE hardcoded) ────────────────────────────────
70
+ "PX_PER_MM" : 3.75,
71
+
72
+ # ── Device — CPU on HF free tier ─────────────────────────────────────────
73
+ "DEVICE" : "cuda:0" if torch.cuda.is_available() else "cpu",
74
+ }
75
+
76
+ # ─────────────────────────────────────────────────────────────────────────────
77
+ # ITU LOGO — encode to base64 for embedding in HTML
78
+ # Place ITULog.png in the same directory as this script.
79
+ # ─────────────────────────────────────────────────────────────────────────────
80
+
81
+ def _encode_logo(path: str) -> str:
82
+ try:
83
+ with open(path, "rb") as f:
84
+ return "data:image/png;base64," + base64.b64encode(f.read()).decode()
85
+ except FileNotFoundError:
86
+ return ""
87
+
88
+ LOGO_ITU = _encode_logo(str(Path(__file__).parent / "ITULog.png"))
89
+
90
+ # ─────────────────────────────────────────────────────────────────────────────
91
+ # CSS THEME — white / bluish, minimalist, medical
92
+ # ─────────────────────────────────────────────────────────────────────────────
93
+
94
+ CSS = """
95
+ /* ── Base ── */
96
+ :root {
97
+ --primary: #1a6fa8;
98
+ --primary-lt: #e8f3fb;
99
+ --primary-mid: #4a9fd4;
100
+ --accent: #0d9e6e; /* Angio AI green */
101
+ --bg: #f7fafd;
102
+ --surface: #ffffff;
103
+ --border: #cce0f0;
104
+ --text: #1a2533;
105
+ --text-sec: #5a7a96;
106
+ --danger: #d64040;
107
+ --warn: #d98b00;
108
+ --radius: 10px;
109
+ --shadow: 0 2px 12px rgba(26,111,168,0.10);
110
+ }
111
+
112
+ body, .gradio-container {
113
+ background: var(--bg) !important;
114
+ font-family: 'Segoe UI', system-ui, sans-serif !important;
115
+ color: var(--text) !important;
116
+ }
117
+
118
+ /* ── Header bar ── */
119
+ #header {
120
+ background: var(--surface);
121
+ border-bottom: 2px solid var(--border);
122
+ padding: 14px 28px;
123
+ display: flex;
124
+ align-items: center;
125
+ justify-content: space-between;
126
+ box-shadow: var(--shadow);
127
+ margin-bottom: 0 !important;
128
+ }
129
+ #logo-itu img { height: 48px; }
130
+ #logo-angio {
131
+ font-size: 26px;
132
+ font-weight: 700;
133
+ color: var(--accent);
134
+ letter-spacing: 1px;
135
+ }
136
+ #logo-angio span {
137
+ font-size: 13px;
138
+ color: var(--text-sec);
139
+ font-weight: 400;
140
+ display: block;
141
+ letter-spacing: 0;
142
+ }
143
+
144
+ /* ── Tab bar ── */
145
+ .tab-nav button {
146
+ color: var(--text-sec) !important;
147
+ border-bottom: 3px solid transparent !important;
148
+ font-size: 15px !important;
149
+ font-weight: 500 !important;
150
+ padding: 10px 24px !important;
151
+ background: transparent !important;
152
+ }
153
+ .tab-nav button.selected {
154
+ color: var(--primary) !important;
155
+ border-bottom-color: var(--primary) !important;
156
+ }
157
+
158
+ /* ── Cards ── */
159
+ .card {
160
+ background: var(--surface);
161
+ border: 1px solid var(--border);
162
+ border-radius: var(--radius);
163
+ padding: 22px;
164
+ box-shadow: var(--shadow);
165
+ margin-bottom: 16px;
166
+ }
167
+ .card-title {
168
+ font-size: 14px;
169
+ font-weight: 600;
170
+ color: var(--primary);
171
+ text-transform: uppercase;
172
+ letter-spacing: 0.5px;
173
+ margin-bottom: 12px;
174
+ padding-bottom: 8px;
175
+ border-bottom: 1px solid var(--primary-lt);
176
+ }
177
+
178
+ /* ── Upload zone ── */
179
+ #upload-zone {
180
+ border: 2px dashed var(--primary-mid) !important;
181
+ border-radius: var(--radius) !important;
182
+ background: var(--primary-lt) !important;
183
+ min-height: 200px !important;
184
+ }
185
+
186
+ /* ── Sliders ── */
187
+ input[type=range] { accent-color: var(--primary); }
188
+ .gradio-slider label { color: var(--text-sec) !important; font-size: 13px !important; }
189
+
190
+ /* ── Buttons ── */
191
+ #btn-analyse {
192
+ background: var(--primary) !important;
193
+ color: white !important;
194
+ border: none !important;
195
+ border-radius: var(--radius) !important;
196
+ font-size: 15px !important;
197
+ font-weight: 600 !important;
198
+ padding: 12px 32px !important;
199
+ cursor: pointer !important;
200
+ width: 100% !important;
201
+ transition: background 0.2s !important;
202
+ }
203
+ #btn-analyse:hover { background: #155d90 !important; }
204
+ #btn-reset {
205
+ background: transparent !important;
206
+ color: var(--text-sec) !important;
207
+ border: 1px solid var(--border) !important;
208
+ border-radius: var(--radius) !important;
209
+ font-size: 13px !important;
210
+ padding: 8px 20px !important;
211
+ cursor: pointer !important;
212
+ width: 100% !important;
213
+ }
214
+
215
+ /* ── Metric badges ── */
216
+ .metric-row {
217
+ display: flex;
218
+ gap: 12px;
219
+ flex-wrap: wrap;
220
+ margin-bottom: 12px;
221
+ }
222
+ .metric-badge {
223
+ flex: 1;
224
+ min-width: 110px;
225
+ background: var(--primary-lt);
226
+ border: 1px solid var(--border);
227
+ border-radius: 8px;
228
+ padding: 10px 14px;
229
+ text-align: center;
230
+ }
231
+ .metric-badge .val {
232
+ font-size: 22px;
233
+ font-weight: 700;
234
+ color: var(--primary);
235
+ }
236
+ .metric-badge .val.ischemic { color: var(--danger); }
237
+ .metric-badge .val.borderline { color: var(--warn); }
238
+ .metric-badge .val.ok { color: var(--accent); }
239
+ .metric-badge .lbl {
240
+ font-size: 11px;
241
+ color: var(--text-sec);
242
+ text-transform: uppercase;
243
+ letter-spacing: 0.4px;
244
+ }
245
+
246
+ /* ── Result images ── */
247
+ #result-images img {
248
+ border-radius: var(--radius);
249
+ border: 1px solid var(--border);
250
+ }
251
+
252
+ /* ── Status ── */
253
+ #status-box {
254
+ padding: 10px 16px;
255
+ border-radius: 8px;
256
+ font-size: 13px;
257
+ font-weight: 500;
258
+ border: 1px solid var(--border);
259
+ background: var(--primary-lt);
260
+ color: var(--primary);
261
+ }
262
+
263
+ /* ── Footer ── */
264
+ #footer {
265
+ text-align: center;
266
+ font-size: 12px;
267
+ color: var(--text-sec);
268
+ padding: 20px;
269
+ border-top: 1px solid var(--border);
270
+ margin-top: 24px;
271
+ }
272
+ """
273
+
274
+ # ─────────────────────────────────────────────────────────────────────────────
275
+ # HEADER HTML
276
+ # ─────────────────────────────────────────────────────────────────────────────
277
+
278
+ def _header_html() -> str:
279
+ itu_tag = (f'<img src="{LOGO_ITU}" alt="ITU Logo" style="height:48px">'
280
+ if LOGO_ITU else
281
+ '<span style="font-weight:700;color:#1a6fa8;font-size:18px">ITU</span>')
282
+ return f"""
283
+ <div id="header">
284
+ <div id="logo-itu">{itu_tag}</div>
285
+ <div id="logo-angio">
286
+ Angio AI
287
+ <span>Coronary Analysis System · ITU Lahore</span>
288
+ </div>
289
+ </div>"""
290
+
291
+ # ─────────────────────────────────────────────────────────────────────────────
292
+ # MODULE 1 — KEYFRAME EXTRACTION (from app.py)
293
+ # ─────────────────────────────────────────────────────────────────────────────
294
+
295
+ def extract_best_frame(video_path: str, n: int = 1) -> tuple:
296
+ """
297
+ Extract the single best frame from the video.
298
+ Returns (frame_rgb_512, frame_index, score).
299
+ """
300
+ cap = cv2.VideoCapture(video_path)
301
+ total = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
302
+ if total == 0:
303
+ cap.release()
304
+ raise ValueError("Could not read video — check format/codec.")
305
+
306
+ step = max(1, total // min(120, total))
307
+ scores = []
308
+ fidx = 0
309
+
310
+ while cap.isOpened():
311
+ ret, frame = cap.read()
312
+ if not ret:
313
+ break
314
+ if fidx % step == 0:
315
+ gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
316
+ h, w = gray.shape
317
+ roi = gray[int(h*0.2):int(h*0.8), int(w*0.2):int(w*0.8)]
318
+ contrast = float(roi.mean())
319
+ sharpness = float(cv2.Laplacian(roi, cv2.CV_64F).var())
320
+ score = contrast * 0.6 + min(sharpness, 5000) / 5000 * 100 * 0.4
321
+ scores.append((score, fidx, frame.copy()))
322
+ fidx += 1
323
+ cap.release()
324
+
325
+ if not scores:
326
+ raise ValueError("No frames could be sampled from video.")
327
+
328
+ scores.sort(key=lambda x: x[0], reverse=True)
329
+ best_score, best_idx, best_frame = scores[0]
330
+
331
+ rgb = cv2.cvtColor(best_frame, cv2.COLOR_BGR2RGB)
332
+ rgb = cv2.resize(rgb, (512, 512))
333
+ return rgb, best_idx, best_score, total
334
+
335
+ # ─────────────────────────────────────────────────────────────────────────────
336
+ # MODULE 2 — MASK2FORMER INFERENCE
337
+ # ─────────────────────────────────────────────────────────────────────────────
338
+
339
+ _m2f_model = None
340
+ _m2f_proc = None
341
+
342
+ def _load_mask2former():
343
+ global _m2f_model, _m2f_proc
344
+ if _m2f_model is not None:
345
+ return
346
+ from transformers import (AutoImageProcessor,
347
+ Mask2FormerForUniversalSegmentation)
348
+ ckpt = CONFIG["MASK2FORMER_CKPT"]() # resolves via HF Hub download
349
+ if not os.path.exists(ckpt):
350
+ raise FileNotFoundError(f"Mask2Former checkpoint not found:\n{ckpt}")
351
+
352
+ _m2f_proc = AutoImageProcessor.from_pretrained(
353
+ "facebook/mask2former-swin-base-coco-instance")
354
+ _m2f_model = Mask2FormerForUniversalSegmentation.from_pretrained(
355
+ "facebook/mask2former-swin-base-coco-instance",
356
+ num_labels=2, ignore_mismatched_sizes=True)
357
+
358
+ state = torch.load(ckpt, map_location="cpu")
359
+ weights = state.get("model_state", state)
360
+ _m2f_model.load_state_dict(weights, strict=False)
361
+ _m2f_model.to(CONFIG["DEVICE"]).eval()
362
+
363
+
364
+ def run_mask2former(frame_rgb: np.ndarray, conf_thr: float) -> list:
365
+ """
366
+ Returns list of dicts: [{bbox:[x1,y1,x2,y2], score:float, mask:np.ndarray}, ...]
367
+ """
368
+ _load_mask2former()
369
+ from PIL import Image
370
+ pil = Image.fromarray(frame_rgb)
371
+ inputs = _m2f_proc(images=pil, return_tensors="pt")
372
+ inputs = {k: v.to(CONFIG["DEVICE"]) for k, v in inputs.items()}
373
+
374
+ with torch.no_grad():
375
+ outputs = _m2f_model(**inputs)
376
+
377
+ results = _m2f_proc.post_process_instance_segmentation(
378
+ outputs, target_sizes=[(512, 512)])[0]
379
+
380
+ detections = []
381
+ for seg_info in results["segments_info"]:
382
+ score = float(seg_info["score"])
383
+ if score < conf_thr:
384
+ continue
385
+ mask = (results["segmentation"] == seg_info["id"]).cpu().numpy().astype(np.uint8)
386
+ ys, xs = np.where(mask)
387
+ if len(ys) == 0:
388
+ continue
389
+ x1, y1, x2, y2 = int(xs.min()), int(ys.min()), int(xs.max()), int(ys.max())
390
+ detections.append({"bbox": [x1, y1, x2, y2], "score": score, "mask": mask})
391
+
392
+ return detections
393
+
394
+ # ─────────────────────────────────────────────────────────────────────────────
395
+ # MODULE 3 — RESUNET BINARY SEGMENTATION
396
+ # ─────────────────────────────────────────────────────────────────────────────
397
+
398
+ _resunet_model = None
399
+
400
+ # ── ResUNet architecture — exactly matching binary_best.pth ──────────────────
401
+ #
402
+ # Decoded from checkpoint key names via dump_keys.py:
403
+ #
404
+ # enc0 Conv2d(1, 16, 3, 3, bias=True) plain entry conv
405
+ # down1..4 ConvTranspose2d strided ×2 downsampler 16→32→64→128→256
406
+ # res1..4 _ResBlock(ch) encoder residual blocks ch=16,32,64,128
407
+ # bot.0..2 _ResBlock(256) × 3 bottleneck
408
+ # up3..0 ConvTranspose2d ×2 upsampler 256→128→64→32→16
409
+ # res12..15 _ResBlock(ch) decoder residual blocks ch=128,64,32,16
410
+ # head Sequential[Conv3×3, ReLU, BN, output head
411
+ # Conv3×3, ReLU, BN, Conv1×1]
412
+ #
413
+ # _ResBlock structure (from key pattern res*.bn_in, res*.units.N.*, res*.bn_out):
414
+ # bn_in → BN
415
+ # units → ModuleList of N sub-units, each: [Conv3×3, ReLU, Conv3×3, BN]
416
+ # bn_out → BN
417
+ # forward: x → bn_in → for each unit: residual(x + unit(x)) → bn_out
418
+ # ─────────────────────────────────────────────────────────────────────────────
419
+
420
+ class _ResBlock(torch.nn.Module):
421
+ """
422
+ Matches checkpoint pattern:
423
+ res*.bn_in.*
424
+ res*.units.0.0.weight (Conv3×3)
425
+ res*.units.0.0.bias
426
+ res*.units.0.2.weight (Conv3×3)
427
+ res*.units.0.2.bias
428
+ res*.units.0.3.* (BN)
429
+ ... (3 units total)
430
+ res*.bn_out.*
431
+ Sequential index mapping inside each unit:
432
+ 0 = Conv2d
433
+ 1 = ReLU (no params — not in state dict)
434
+ 2 = Conv2d
435
+ 3 = BatchNorm2d
436
+ """
437
+ def __init__(self, ch, n_units=3):
438
+ super().__init__()
439
+ self.bn_in = torch.nn.BatchNorm2d(ch)
440
+ self.units = torch.nn.ModuleList([
441
+ torch.nn.Sequential(
442
+ torch.nn.Conv2d(ch, ch, 3, padding=1, bias=True),
443
+ torch.nn.ReLU(inplace=True),
444
+ torch.nn.Conv2d(ch, ch, 3, padding=1, bias=True),
445
+ torch.nn.BatchNorm2d(ch),
446
+ )
447
+ for _ in range(n_units)
448
+ ])
449
+ self.bn_out = torch.nn.BatchNorm2d(ch)
450
+
451
+ def forward(self, x):
452
+ x = self.bn_in(x)
453
+ for unit in self.units:
454
+ x = x + unit(x) # no extra ReLU on residual sum
455
+ return self.bn_out(x)
456
+
457
+
458
+ class ResUNet(torch.nn.Module):
459
+ """
460
+ Exact architecture matching binary_best.pth (Dice = 0.8015).
461
+ Key name mapping:
462
+ enc0 → self.enc0 (plain Conv2d)
463
+ down1..4 → self.down1..down4 (ConvTranspose2d strided downsampler)
464
+ res1..4 → self.res1..res4 (encoder ResBlocks)
465
+ bot.0..2 → self.bot[0..2] (bottleneck ResBlocks)
466
+ up3..0 → self.up3..up0 (ConvTranspose2d upsampler)
467
+ res12..15 → self.res12..res15 (decoder ResBlocks)
468
+ head → self.head (Sequential output head)
469
+ """
470
+ def __init__(self):
471
+ super().__init__()
472
+ # ── Entry conv ────────────────────────────────────────────────────────
473
+ self.enc0 = torch.nn.Conv2d(1, 16, 3, padding=1, bias=True)
474
+
475
+ # ── Encoder downsamples — strided Conv2d (not ConvTranspose2d) ─────────
476
+ # Checkpoint shape (out, in, kH, kW): down1=(32,16,2,2) → Conv2d(16→32)
477
+ # stride=2 kernel=2 halves spatial dimensions while increasing channels
478
+ self.down1 = torch.nn.Conv2d(16, 32, 2, stride=2)
479
+ self.down2 = torch.nn.Conv2d(32, 64, 2, stride=2)
480
+ self.down3 = torch.nn.Conv2d(64, 128, 2, stride=2)
481
+ self.down4 = torch.nn.Conv2d(128, 256, 2, stride=2)
482
+
483
+ # ── Encoder residual blocks ───────────────────────���───────────────────
484
+ self.res1 = _ResBlock(16)
485
+ self.res2 = _ResBlock(32)
486
+ self.res3 = _ResBlock(64)
487
+ self.res4 = _ResBlock(128)
488
+
489
+ # ── Bottleneck ────────────────────────────────────────────────────────
490
+ self.bot = torch.nn.ModuleList([
491
+ _ResBlock(256),
492
+ _ResBlock(256),
493
+ _ResBlock(256),
494
+ ])
495
+
496
+ # ── Decoder upsamplers ────────────────────────────────────────────────
497
+ # Shape from keys: up3=(256,128,2,2) bias=(128,)
498
+ # up2=(128,64,2,2) bias=(64,)
499
+ # up1=(64,32,2,2) bias=(32,)
500
+ # up0=(32,16,2,2) bias=(16,)
501
+ self.up3 = torch.nn.ConvTranspose2d(256, 128, 2, stride=2)
502
+ self.up2 = torch.nn.ConvTranspose2d(128, 64, 2, stride=2)
503
+ self.up1 = torch.nn.ConvTranspose2d(64, 32, 2, stride=2)
504
+ self.up0 = torch.nn.ConvTranspose2d(32, 16, 2, stride=2)
505
+
506
+ # ── Decoder residual blocks ───────────────────────────────────────────
507
+ self.res12 = _ResBlock(128)
508
+ self.res13 = _ResBlock(64)
509
+ self.res14 = _ResBlock(32)
510
+ self.res15 = _ResBlock(16)
511
+
512
+ # ── Output head ───────────────────────────────────────────────────────
513
+ # From keys: head.0=Conv(16,16,3,3), head.2=BN(16), head.3=Conv(16,16,3,3),
514
+ # head.5=BN(16), head.6=Conv(2,16,1,1)
515
+ # Index 1 and 4 are ReLU (no state dict entries)
516
+ self.head = torch.nn.Sequential(
517
+ torch.nn.Conv2d(16, 16, 3, padding=1, bias=True), # 0
518
+ torch.nn.ReLU(inplace=True), # 1
519
+ torch.nn.BatchNorm2d(16), # 2
520
+ torch.nn.Conv2d(16, 16, 3, padding=1, bias=True), # 3
521
+ torch.nn.ReLU(inplace=True), # 4
522
+ torch.nn.BatchNorm2d(16), # 5
523
+ torch.nn.Conv2d(16, 2, 1, bias=True), # 6
524
+ )
525
+
526
+ def forward(self, x):
527
+ # ── Encoder ──────────────────────────────────────────────────────────
528
+ # Entry conv → ResBlock → down. Skip taken AFTER ResBlock.
529
+ x = self.enc0(x) # (B,16,512,512)
530
+ r1 = self.res1(x); x = self.down1(r1) # skip r1, down → (B,32,256,256)
531
+ r2 = self.res2(x); x = self.down2(r2) # skip r2, down → (B,64,128,128)
532
+ r3 = self.res3(x); x = self.down3(r3) # skip r3, down → (B,128,64,64)
533
+ r4 = self.res4(x); x = self.down4(r4) # skip r4, down → (B,256,32,32)
534
+
535
+ # ── Bottleneck ───────────────────────────────────────────────────────
536
+ for blk in self.bot:
537
+ x = blk(x)
538
+
539
+ # ── Decoder — ResBlock first, then add skip ───────────────────────────
540
+ x = self.res12(self.up3(x)) + r4 # (B,128,64,64)
541
+ x = self.res13(self.up2(x)) + r3 # (B,64,128,128)
542
+ x = self.res14(self.up1(x)) + r2 # (B,32,256,256)
543
+ x = self.res15(self.up0(x)) + r1 # (B,16,512,512)
544
+
545
+ return self.head(x)
546
+
547
+
548
+ def _load_resunet():
549
+ global _resunet_model
550
+ if _resunet_model is not None:
551
+ return
552
+ ckpt = CONFIG["RESUNET_CKPT"]() # resolves via HF Hub download
553
+ if not os.path.exists(ckpt):
554
+ raise FileNotFoundError(f"ResUNet checkpoint not found:\n{ckpt}")
555
+ _resunet_model = ResUNet()
556
+ state = torch.load(ckpt, map_location="cpu")
557
+ weights = state.get("model_state", state)
558
+
559
+ # Try strict first; if it fails print exact mismatches and retry strict=False
560
+ try:
561
+ _resunet_model.load_state_dict(weights, strict=True)
562
+ print(f"[ResUNet] ✓ Perfect load (strict=True) — "
563
+ f"epoch={state.get('epoch','?')} "
564
+ f"dice={state.get('best_val_dice',0):.4f}")
565
+ except RuntimeError as e:
566
+ print(f"[ResUNet] strict=True failed:\n {e}")
567
+ result = _resunet_model.load_state_dict(weights, strict=False)
568
+ print(f"[ResUNet] strict=False — "
569
+ f"missing={len(result.missing_keys)} "
570
+ f"unexpected={len(result.unexpected_keys)}")
571
+ for k in result.missing_keys[:10]:
572
+ print(f" MISSING {k}")
573
+ for k in result.unexpected_keys[:10]:
574
+ print(f" UNEXPECTED {k}")
575
+
576
+ _resunet_model.to(CONFIG["DEVICE"]).eval()
577
+ print(f"[ResUNet] Loaded (eval mode) — "
578
+ f"epoch={state.get('epoch','?')} "
579
+ f"dice={state.get('best_val_dice',0):.4f}")
580
+
581
+
582
+ def _preprocess_resunet(rgb: np.ndarray) -> torch.Tensor:
583
+ """ARCADE preprocessing: invert → top-hat → subtract → CLAHE → normalise."""
584
+ gray = cv2.cvtColor(rgb, cv2.COLOR_RGB2GRAY).astype(np.uint8)
585
+ inv = cv2.bitwise_not(gray)
586
+ se = cv2.getStructuringElement(cv2.MORPH_RECT, (50, 50))
587
+ th = cv2.morphologyEx(inv, cv2.MORPH_TOPHAT, se)
588
+ sub = cv2.subtract(gray, th)
589
+ clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8, 8))
590
+ proc = clahe.apply(sub).astype(np.float32) / 255.0
591
+ t = torch.from_numpy(proc).unsqueeze(0).unsqueeze(0) # (1,1,512,512)
592
+ return t
593
+
594
+
595
+ def run_resunet(frame_rgb: np.ndarray):
596
+ """
597
+ Returns:
598
+ binary_mask : (512,512) uint8, values 0/1
599
+ binary_overlay: (512,512,3) uint8 RGB — green vessel overlay on original
600
+ Architecture fix applied: skip connections are now res(up(x)) + skip
601
+ (not res(up(x) + skip)), and ResBlock has no extra ReLU on residual sum.
602
+ Model runs in eval() mode — BN running stats are now correct.
603
+ """
604
+ _load_resunet()
605
+ _resunet_model.eval()
606
+ t = _preprocess_resunet(frame_rgb).to(CONFIG["DEVICE"])
607
+ with torch.no_grad():
608
+ logits = _resunet_model(t)
609
+ pred = logits.argmax(dim=1).squeeze().cpu().numpy().astype(np.uint8)
610
+ vessel_px = int((pred == 1).sum())
611
+ print(f"[ResUNet] vessel px={vessel_px} ({100*(pred==1).mean():.1f}%)")
612
+
613
+ # Build green overlay for display in Results tab
614
+ overlay = frame_rgb.copy().astype(np.float32)
615
+ green = np.array([0, 255, 80], dtype=np.float32)
616
+ mask3d = pred == 1
617
+ overlay[mask3d] = overlay[mask3d] * 0.35 + green * 0.65
618
+ overlay = overlay.astype(np.uint8)
619
+ # Draw contours for sharper vessel edges
620
+ contours, _ = cv2.findContours(pred, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
621
+ cv2.drawContours(overlay, contours, -1, (0, 255, 80), 1)
622
+ # Coverage label
623
+ cov = 100 * vessel_px / pred.size
624
+ cv2.putText(overlay, f"Vessel coverage: {cov:.1f}%",
625
+ (8, 20), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 255, 80), 1, cv2.LINE_AA)
626
+
627
+ return pred, overlay
628
+
629
+ # ─────────────────────────────────────────────────────────────────────────────
630
+ # MODULE 4 — YOLOv8m-seg INFERENCE (replaces nnUNet)
631
+ # ─────────────────────────────────────────────────────────────────────────────
632
+ #
633
+ # Checkpoint : CONFIG["YOLO_CKPT"] -> best.pt from YOLOv8m-seg training
634
+ # Model : YOLOv8m-seg, nc=26, trained on ARCADE syntax dataset
635
+ #
636
+ # Input : frame_rgb -- (512,512,3) uint8 RGB
637
+ # seg_conf -- confidence threshold from UI slider
638
+ # Output: seg_map -- (512,512) uint8 label map, values 0..25
639
+ # 0 = background (no detection)
640
+ # 1..25 = ARCADE class index
641
+ #
642
+ # Output format is identical to the old run_nnunet() so that
643
+ # render_nnunet_overlay() and compute_syntax_score() work unchanged.
644
+ # ─────────────────────────────────────────────────────────────────────────────
645
+
646
+ # ARCADE 26-class name list (index = YOLO cls_id + 1)
647
+ _YOLO_CLASS_NAMES = [
648
+ "background", # 0 (background, not predicted by YOLO)
649
+ "LMCA", # 1
650
+ "LAD prox", # 2
651
+ "LAD mid", # 3
652
+ "LAD dist", # 4
653
+ "D1", # 5
654
+ "D2", # 6
655
+ "D3", # 7
656
+ "Septal", # 8
657
+ "LAD var", # 9
658
+ "LCX prox", # 10
659
+ "LCX mid", # 11
660
+ "LCX dist", # 12
661
+ "OM1", # 13
662
+ "OM2", # 14
663
+ "OM3", # 15
664
+ "LCX var", # 16
665
+ "RCA prox", # 17
666
+ "RCA mid", # 18
667
+ "RCA dist", # 19
668
+ "PDA", # 20
669
+ "PLV", # 21
670
+ "AM", # 22
671
+ "Conus", # 23
672
+ "SAN", # 24
673
+ "Other", # 25
674
+ ]
675
+
676
+ _yolo_model = None # singleton -- loaded once on first call, reused every frame
677
+
678
+
679
+ def _load_yolo():
680
+ """Lazy-load YOLOv8m-seg checkpoint into module-level singleton."""
681
+ global _yolo_model
682
+ if _yolo_model is not None:
683
+ return
684
+ try:
685
+ from ultralytics import YOLO
686
+ except ImportError:
687
+ raise ImportError(
688
+ "ultralytics not installed. Run: pip install ultralytics"
689
+ )
690
+ ckpt = CONFIG["YOLO_CKPT"]() # resolves via HF Hub download
691
+ if not os.path.exists(ckpt):
692
+ raise FileNotFoundError(
693
+ f"YOLOv8 checkpoint not found:\n{ckpt}\n"
694
+ f"Check HF_TOKEN secret and repo MuhammadAdil63/angio-ai-checkpoints"
695
+ )
696
+ print(f"[YOLO] Loading checkpoint: {ckpt}")
697
+ _yolo_model = YOLO(ckpt)
698
+ print(f"[YOLO] Model loaded OK")
699
+
700
+
701
+ def run_yolo_seg(frame_rgb: np.ndarray, seg_conf: float) -> np.ndarray:
702
+ """
703
+ Run YOLOv8m-seg on a single RGB frame.
704
+ Returns seg_map (512x512 uint8), label values 0..25.
705
+
706
+ Label map composition:
707
+ - Initialised to 0 (background).
708
+ - Instances sorted ascending by confidence so higher-conf masks
709
+ overwrite lower-conf ones at overlapping pixels.
710
+ - YOLO cls_id is 0-based (0..24 for nc=25 visible classes).
711
+ We store label_val = cls_id + 1 so 0 stays background,
712
+ matching ARCADE_COLOURS / ARCADE_LABELS which use keys 1..25.
713
+ """
714
+ _load_yolo()
715
+
716
+ device = CONFIG["DEVICE"]
717
+ conf = float(seg_conf) if seg_conf else CONFIG["YOLO_CONF"]
718
+ iou = CONFIG["YOLO_IOU"]
719
+ imgsz = CONFIG["YOLO_IMGSZ"]
720
+
721
+ # YOLO expects BGR
722
+ frame_bgr = cv2.cvtColor(frame_rgb, cv2.COLOR_RGB2BGR)
723
+
724
+ try:
725
+ results = _yolo_model.predict(
726
+ source = frame_bgr,
727
+ conf = conf,
728
+ iou = iou,
729
+ imgsz = imgsz,
730
+ device = device,
731
+ retina_masks = False,
732
+ verbose = False,
733
+ )
734
+ except Exception as e:
735
+ print(f"[YOLO] predict() failed: {e}")
736
+ return np.zeros((512, 512), dtype=np.uint8)
737
+
738
+ result = results[0]
739
+ h, w = frame_rgb.shape[:2]
740
+ seg_map = np.zeros((h, w), dtype=np.uint8)
741
+
742
+ if result.masks is None or result.boxes is None:
743
+ print("[YOLO] No masks returned -- check conf threshold or checkpoint.")
744
+ return seg_map
745
+
746
+ masks = result.masks.data.cpu().numpy() # (N, Hm, Wm) float32
747
+ cls_ids = result.boxes.cls.cpu().int().numpy() # (N,) 0-based
748
+ confs = result.boxes.conf.cpu().numpy() # (N,)
749
+
750
+ # Ascending confidence sort so higher-conf masks win at overlaps
751
+ order = np.argsort(confs)
752
+
753
+ n_written = 0
754
+ for i in order:
755
+ cls_id = int(cls_ids[i])
756
+ mask_resized = cv2.resize(
757
+ masks[i], (w, h), interpolation=cv2.INTER_LINEAR
758
+ )
759
+ binary = mask_resized > 0.5
760
+ label_val = min(cls_id + 1, 25) # clamp to valid 1-25 range
761
+ seg_map[binary] = label_val
762
+ n_written += int(binary.sum())
763
+
764
+ detected_cls = [_YOLO_CLASS_NAMES[min(int(c) + 1, 25)] for c in cls_ids]
765
+ print(f"[YOLO] Detections: {len(cls_ids)} | classes: {detected_cls} | "
766
+ f"vessel px: {n_written} ({100 * n_written / (h * w):.1f}%)")
767
+ print(f"[YOLO] seg_map unique labels: {np.unique(seg_map)}")
768
+ return seg_map
769
+
770
+ # ─────────────────────────────────────────────────────────────────────────────
771
+ # MODULE 5 — FFR PIPELINE (inline, from ffr_pipeline_v4.py)
772
+ # ─────────────────────────────────────────────────────────────────────────────
773
+
774
+ from skimage.morphology import skeletonize, remove_small_objects, disk
775
+ from skimage.measure import label as sk_label
776
+ from scipy.ndimage import distance_transform_edt
777
+ from scipy.signal import argrelmin, savgol_filter
778
+ from skimage.filters import threshold_otsu
779
+
780
+ FFR_CFG = {
781
+ "close_radius" : 3,
782
+ "open_radius" : 1,
783
+ "min_vessel_px" : 100,
784
+ "stenosis_overlap_thr": 0.05,
785
+ # Lowered from 0.20 → 0.05 for video frames which have sparser masks
786
+ # than training PNGs — prevents all detections being unlocalized
787
+ "min_overlap_for_ffr": 0.05,
788
+ "min_ds_fraction" : 0.10,
789
+ "min_branch_len" : 15,
790
+ "use_dominant_branch": True,
791
+ "min_coverage_warn" : 0.50,
792
+ "sample_step" : 3,
793
+ "perp_half_len" : 40, # reduced from 50 to avoid hitting background
794
+ "smooth_window" : 11,
795
+ "smooth_poly" : 2,
796
+ "reference_pct" : 75,
797
+ # Physiological cap: coronary vessels are 1.5–5mm in diameter
798
+ # If auto-computed ref exceeds this, clamp it
799
+ "ref_diam_max_mm" : 5.0,
800
+ "ref_diam_min_mm" : 1.0,
801
+ "px_per_mm" : 3.75,
802
+ }
803
+
804
+ def _refine_mask(img_gray, binary_mask):
805
+ masked = img_gray[binary_mask == 1]
806
+ if len(masked) == 0:
807
+ return binary_mask.copy()
808
+ thr = threshold_otsu(masked)
809
+ return ((binary_mask == 1) & (img_gray < thr)).astype(np.uint8)
810
+
811
+ def _morph_clean(mask):
812
+ m = mask.copy()
813
+ k_c = disk(FFR_CFG["close_radius"]).astype(np.uint8)
814
+ k_o = disk(FFR_CFG["open_radius"]).astype(np.uint8)
815
+ m = cv2.morphologyEx(m, cv2.MORPH_CLOSE, k_c)
816
+ m = cv2.morphologyEx(m, cv2.MORPH_OPEN, k_o)
817
+ return remove_small_objects(m.astype(bool),
818
+ min_size=FFR_CFG["min_vessel_px"]).astype(np.uint8)
819
+
820
+ def _find_branch_pts(skel):
821
+ k = np.ones((3,3), np.uint8)
822
+ n = cv2.filter2D(skel.astype(np.uint8), -1, k) * skel.astype(np.uint8)
823
+ return n >= 4
824
+
825
+ def _dominant_branch(skel):
826
+ bp = _find_branch_pts(skel)
827
+ sp = skel.copy().astype(np.uint8); sp[bp] = 0
828
+ lab = sk_label(sp)
829
+ if lab.max() == 0: return skel
830
+ sizes = np.bincount(lab.ravel()); sizes[0] = 0
831
+ return (lab == sizes.argmax())
832
+
833
+ def _order_centerline(skel):
834
+ pts = np.argwhere(skel)
835
+ if len(pts) == 0: return []
836
+ sk_set = set(map(tuple, pts))
837
+ def nbrs(p):
838
+ r,c = p
839
+ return [(r+dr,c+dc) for dr in [-1,0,1] for dc in [-1,0,1]
840
+ if (dr,dc)!=(0,0) and (r+dr,c+dc) in sk_set]
841
+ endpoints = [p for p in sk_set if len(nbrs(p))==1] or [tuple(pts[0])]
842
+ def trace(start):
843
+ visited={tuple(start)}; path=[tuple(start)]; cur=tuple(start)
844
+ while True:
845
+ cands=[n for n in nbrs(cur) if n not in visited]
846
+ if not cands: break
847
+ if len(path)>1:
848
+ pr=path[-2]; dr0=cur[0]-pr[0]; dc0=cur[1]-pr[1]
849
+ cands.sort(key=lambda n:abs(n[0]-cur[0]-dr0)+abs(n[1]-cur[1]-dc0))
850
+ cur=cands[0]; visited.add(cur); path.append(cur)
851
+ return path
852
+ best=[]
853
+ for ep in endpoints[:20]:
854
+ p=trace(ep)
855
+ if len(p)>len(best): best=p
856
+ return best
857
+
858
+ def _measure_diameters(cl_pts, refined_mask):
859
+ H,W=refined_mask.shape
860
+ step=FFR_CFG["sample_step"]; half=FFR_CFG["perp_half_len"]
861
+ arc_len=0.; arc_pos=[]; diams=[]; wall_pairs=[]; centers=[]
862
+ for i in range(0, len(cl_pts)-step, step):
863
+ p0=np.array(cl_pts[i], dtype=float)
864
+ p1=np.array(cl_pts[min(i+step,len(cl_pts)-1)], dtype=float)
865
+ tang=p1-p0; tl=np.linalg.norm(tang)
866
+ if tl<1e-9: continue
867
+ tang/=tl; norm=np.array([-tang[1],tang[0]])
868
+ def wd(d):
869
+ for dd in np.arange(0.5,half,0.5):
870
+ pt=p0+d*dd; r_=int(round(pt[0])); c_=int(round(pt[1]))
871
+ if r_<0 or r_>=H or c_<0 or c_>=W: return dd,pt
872
+ if refined_mask[r_,c_]==0: return dd,pt
873
+ return half, p0+d*half
874
+ dl,pl=wd( norm); dr,pr=wd(-norm); diam=dl+dr
875
+ if diam<1. or diam>half*1.8: continue
876
+ if len(centers)==0: step_arc=0.
877
+ else: step_arc=float(np.linalg.norm(p0-np.array(centers[-1])))
878
+ arc_len+=min(step_arc, FFR_CFG["sample_step"]*1.5)
879
+ arc_pos.append(arc_len); diams.append(diam)
880
+ wall_pairs.append((pl,pr)); centers.append(p0)
881
+ return (np.array(arc_pos), np.array(diams,dtype=np.float32),
882
+ wall_pairs, centers)
883
+
884
+ def _ds_to_ffr(ds):
885
+ return max(0., min(1., 1.-(0.33*ds+0.60*ds**2)))
886
+
887
+ def run_ffr(frame_rgb, binary_mask, detections):
888
+ """Full FFR pipeline v4. Returns result dict + annotated figure (np array)."""
889
+ img_gray = cv2.cvtColor(frame_rgb, cv2.COLOR_RGB2GRAY)
890
+ px_mm = FFR_CFG["px_per_mm"]
891
+
892
+ # ── Debug: report mask quality before refinement ──────────────────────────
893
+ bm_sum = int(binary_mask.sum())
894
+ bm_pct = 100 * binary_mask.mean()
895
+ print(f"[FFR] binary_mask entering FFR: sum={bm_sum} {bm_pct:.1f}% of image")
896
+
897
+ # ── Use binary mask directly — skip Otsu refinement ─────────────────────
898
+ # ResUNet already produces a clean vessel segmentation (Dice=0.80).
899
+ # Otsu refinement was designed for use with ground-truth binary masks from
900
+ # the ARCADE dataset where it further removes bright background pixels.
901
+ # On video frames, Otsu degrades the ResUNet mask because the vessel/bg
902
+ # intensity distribution differs from training PNGs.
903
+ # Using the binary mask directly preserves all vessel pixels.
904
+ refined = binary_mask.copy()
905
+ print(f"[FFR] Using binary mask directly (skip Otsu): sum={refined.sum()}")
906
+
907
+ refined = _morph_clean(refined)
908
+ print(f"[FFR] After morph cleanup: sum={refined.sum()}")
909
+
910
+ skel_raw = skeletonize(refined.astype(bool))
911
+ print(f"[FFR] Skeleton pixels: {skel_raw.sum()}")
912
+
913
+ # Prune spurs
914
+ sk = skel_raw.copy().astype(np.uint8)
915
+ for _ in range(FFR_CFG["min_branch_len"]):
916
+ k=np.ones((3,3),np.uint8)
917
+ nb=cv2.filter2D(sk,-1,k)*sk
918
+ ep=(nb==2).astype(np.uint8)
919
+ sk=cv2.subtract(sk,ep)
920
+ if ep.sum()==0: break
921
+ skel_pruned=sk.astype(bool)
922
+ print(f"[FFR] Pruned skeleton: {skel_pruned.sum()}")
923
+
924
+ skel_dom = _dominant_branch(skel_pruned)
925
+ print(f"[FFR] Dominant branch: {skel_dom.sum()}")
926
+ cl_pts = _order_centerline(skel_dom)
927
+ print(f"[FFR] Ordered centerline: {len(cl_pts)} pts")
928
+
929
+ # Fallback: only if dominant branch is very short (< 30 pts)
930
+ # Do NOT fall back to full skeleton — it traces through noise fragments
931
+ if len(cl_pts) < 30:
932
+ print(f"[FFR] Dominant branch short ({len(cl_pts)} pts) — trying full pruned skeleton")
933
+ cl_pts_full = _order_centerline(skel_pruned)
934
+ if len(cl_pts_full) > len(cl_pts):
935
+ cl_pts = cl_pts_full
936
+ print(f"[FFR] Full skeleton centerline: {len(cl_pts)} pts")
937
+
938
+ if len(cl_pts) < 5:
939
+ return {
940
+ "error": (
941
+ f"Centerline too short ({len(cl_pts)} pts). "
942
+ f"binary_mask={bm_sum}px ({bm_pct:.1f}%), "
943
+ f"skeleton={int(skel_raw.sum())}px."
944
+ )
945
+ }, None
946
+
947
+ arc_pos, diams_px, wall_pairs, centers = _measure_diameters(cl_pts, refined)
948
+ if len(diams_px)==0:
949
+ return {"error":"No diameter measurements obtained."}, None
950
+
951
+ win=FFR_CFG["smooth_window"]
952
+ if len(diams_px)<win: win=len(diams_px) if len(diams_px)%2==1 else max(3,len(diams_px)-1)
953
+ diams_sm = savgol_filter(diams_px, win, FFR_CFG["smooth_poly"]) if win>=3 else diams_px.copy()
954
+
955
+ ref_px = np.percentile(diams_sm, FFR_CFG["reference_pct"])
956
+ ref_mm = ref_px / px_mm
957
+
958
+ # ── Physiological sanity clamp ────────────────────────────────────────────
959
+ # Coronary vessels are 1.5–5.0 mm in diameter.
960
+ # If ref_mm is outside this range, the skeleton traced through noise/background.
961
+ # Clamp to physiological limits and recompute ref_px accordingly.
962
+ ref_mm_raw = ref_mm
963
+ ref_mm = float(np.clip(ref_mm,
964
+ FFR_CFG["ref_diam_min_mm"],
965
+ FFR_CFG["ref_diam_max_mm"]))
966
+ ref_px = ref_mm * px_mm
967
+ if abs(ref_mm_raw - ref_mm) > 0.1:
968
+ print(f"[FFR] ref_diam clamped: {ref_mm_raw:.2f}mm → {ref_mm:.2f}mm "
969
+ f"(physiological range {FFR_CFG['ref_diam_min_mm']}–"
970
+ f"{FFR_CFG['ref_diam_max_mm']}mm)")
971
+
972
+ # Also filter out diameter measurements that exceed 2× ref_px (noise)
973
+ valid_mask = diams_sm <= (ref_px * 2.5)
974
+ if valid_mask.sum() > 5:
975
+ arc_pos = arc_pos[valid_mask]
976
+ diams_px = diams_px[valid_mask]
977
+ diams_sm = diams_sm[valid_mask]
978
+ wall_pairs = [wp for wp, v in zip(wall_pairs, valid_mask) if v]
979
+ centers = [c for c, v in zip(centers, valid_mask) if v]
980
+ centers_arr = np.array(centers)
981
+ print(f"[FFR] After noise filter: {len(diams_sm)} measurements kept")
982
+
983
+ stenoses=[]; unloc=[]
984
+
985
+ for det in detections:
986
+ bbox = det.get("bbox",[])
987
+ score = det.get("score",1.)
988
+ overlap = det.get("overlap",0.)
989
+ if len(bbox)<4: continue
990
+ if overlap < FFR_CFG["min_overlap_for_ffr"]:
991
+ unloc.append({**det,"reason":"low-overlap"}); continue
992
+ x1,y1,x2,y2=bbox
993
+ inside=np.where((centers_arr[:,0]>=y1)&(centers_arr[:,0]<=y2)&
994
+ (centers_arr[:,1]>=x1)&(centers_arr[:,1]<=x2))[0]
995
+ idx = int(inside[np.argmin(diams_sm[inside])]) if len(inside)>0 else \
996
+ int(np.argmin(np.linalg.norm(centers_arr-np.array([(y1+y2)/2,(x1+x2)/2]),axis=1)))
997
+ if idx>=len(diams_sm): continue
998
+ d_sten=float(diams_sm[idx]); ds=max(0.,1.-d_sten/ref_px)
999
+ if ds < FFR_CFG["min_ds_fraction"]:
1000
+ unloc.append({**det,"reason":"no-narrowing"}); continue
1001
+ ffr_est=_ds_to_ffr(ds)
1002
+ stenoses.append({"index":idx,"position_mm":float(arc_pos[idx]/px_mm),
1003
+ "d_mm":float(d_sten/px_mm),"ref_mm":float(ref_mm),
1004
+ "pct_ds":float(ds*100),"ffr":ffr_est,
1005
+ "score":score,"significant":ffr_est<=0.80})
1006
+
1007
+ global_ffr=1.
1008
+ for s in stenoses: global_ffr*=s["ffr"]
1009
+ global_ffr=max(0.,global_ffr)
1010
+
1011
+ # Build figure
1012
+ fig, axes = plt.subplots(1,3, figsize=(15,5))
1013
+ fig.patch.set_facecolor("white")
1014
+
1015
+ # Panel 1: stenosis overlay
1016
+ axes[0].imshow(frame_rgb, cmap="gray")
1017
+ for det in detections:
1018
+ b=det.get("bbox",[]); sc=det.get("score",0)
1019
+ if len(b)<4: continue
1020
+ x1,y1,x2,y2=b
1021
+ col="red" if sc>=0.5 else "orange"
1022
+ axes[0].add_patch(mpatches.Rectangle(
1023
+ (x1,y1),x2-x1,y2-y1,fill=False,edgecolor=col,linewidth=2))
1024
+ axes[0].text(x1,y1-5,f"{sc:.2f}",color=col,fontsize=8,fontweight="bold")
1025
+ axes[0].axis("off"); axes[0].set_title("Stenosis detections", fontsize=11, color="#1a6fa8")
1026
+
1027
+ # Panel 2: centerline + diameter lines
1028
+ overlay=frame_rgb.copy()
1029
+ sk_pts=np.argwhere(skel_dom)
1030
+ for r,c in sk_pts:
1031
+ if 0<=r<512 and 0<=c<512: overlay[r,c]=[0,206,209]
1032
+ axes[1].imshow(overlay)
1033
+ for i,(wl,wr) in enumerate(wall_pairs):
1034
+ axes[1].plot([wl[1],wr[1]],[wl[0],wr[0]],
1035
+ color="lime",alpha=0.4,linewidth=0.6)
1036
+ for st in stenoses:
1037
+ idx=st["index"]
1038
+ if idx<len(centers):
1039
+ cp=centers[idx]
1040
+ axes[1].plot(cp[1],cp[0],"rv",markersize=10,zorder=6,
1041
+ markeredgecolor="white",markeredgewidth=0.5)
1042
+ axes[1].annotate(f'FFR≈{st["ffr"]:.2f}',
1043
+ xy=(cp[1],cp[0]),xytext=(cp[1]+14,cp[0]-14),
1044
+ color="red",fontsize=8,fontweight="bold",
1045
+ arrowprops=dict(arrowstyle="->",color="red",lw=0.8))
1046
+ axes[1].axis("off"); axes[1].set_title("Centerline + FFR markers", fontsize=11, color="#1a6fa8")
1047
+
1048
+ # Panel 3: diameter profile
1049
+ if len(arc_pos)>0:
1050
+ pos_mm=arc_pos/px_mm
1051
+ axes[2].plot(pos_mm,diams_px/px_mm,color="#cce0f0",lw=0.8,label="Raw")
1052
+ axes[2].plot(pos_mm,diams_sm/px_mm,color="#1a6fa8",lw=1.8,label="Smoothed")
1053
+ axes[2].axhline(ref_mm,color="#0d9e6e",ls="--",lw=1.2,label=f"Ref {ref_mm:.2f}mm")
1054
+ for st in stenoses:
1055
+ axes[2].axvline(st["position_mm"],color="red",ls=":",lw=1.,alpha=0.8)
1056
+ axes[2].annotate(f'{st["pct_ds"]:.0f}%DS\nFFR≈{st["ffr"]:.2f}',
1057
+ xy=(st["position_mm"],st["d_mm"]),
1058
+ xytext=(st["position_mm"]+2,st["d_mm"]+ref_mm*0.15),
1059
+ color="red",fontsize=7,fontweight="bold",
1060
+ arrowprops=dict(arrowstyle="->",color="red",lw=0.7))
1061
+ axes[2].set_xlabel("Position (mm)",fontsize=9)
1062
+ axes[2].set_ylabel("Diameter (mm)",fontsize=9)
1063
+ axes[2].legend(fontsize=8); axes[2].grid(alpha=0.3); axes[2].set_ylim(0)
1064
+ axes[2].set_title("Lumen diameter profile", fontsize=11, color="#1a6fa8")
1065
+
1066
+ ffr_label = ("⚠ ISCHEMIC" if global_ffr<=0.80 else "✓ Non-ischemic")
1067
+ fig.suptitle(f"FFR Analysis — Global FFR: {global_ffr:.3f} {ffr_label} | "
1068
+ f"Stenoses: {len(stenoses)} | Ref: {ref_mm:.2f} mm",
1069
+ fontsize=12,fontweight="bold",color="#1a2533",y=1.01)
1070
+ plt.tight_layout()
1071
+
1072
+ # Convert figure to numpy array — compatible with all matplotlib versions
1073
+ fig.canvas.draw()
1074
+ w, h = fig.canvas.get_width_height()
1075
+ try:
1076
+ # matplotlib >= 3.8
1077
+ buf = np.frombuffer(fig.canvas.buffer_rgba(), dtype=np.uint8)
1078
+ buf = buf.reshape(h, w, 4)[:, :, :3] # drop alpha channel
1079
+ except AttributeError:
1080
+ # matplotlib < 3.8 fallback
1081
+ buf = np.frombuffer(fig.canvas.tostring_rgb(), dtype=np.uint8)
1082
+ buf = buf.reshape(h, w, 3)
1083
+ plt.close(fig)
1084
+
1085
+ result = {
1086
+ "global_ffr" : round(global_ffr,3),
1087
+ "ischemic" : global_ffr<=0.80,
1088
+ "n_stenoses" : len(stenoses),
1089
+ "stenoses" : stenoses,
1090
+ "ref_diam_mm" : round(ref_mm,2),
1091
+ "vessel_len_mm": round(float(arc_pos[-1]/px_mm),1) if len(arc_pos)>0 else 0,
1092
+ "n_unlocalized": len(unloc),
1093
+ }
1094
+ return result, buf
1095
+
1096
+ # ─────────────────────────────────────────────────────────────────────────────
1097
+ # MODULE 6 — nnUNet OVERLAY
1098
+ # ─────────────────────────────────────────────────────────────────────────────
1099
+
1100
+ # 26 maximally distinct colours — one per ARCADE class (0 = background, black)
1101
+ # Generated by evenly spacing hue across HSV wheel at two brightness levels,
1102
+ # then manually verified for distinctness. Each colour is unique.
1103
+ # Format: (R, G, B)
1104
+ ARCADE_COLOURS = {
1105
+ 0 : ( 0, 0, 0), # background — black (not rendered)
1106
+ 1 : (255, 0, 0), # LMCA — red
1107
+ 2 : ( 0, 120, 255), # LAD proximal — azure blue
1108
+ 3 : ( 0, 220, 0), # LAD mid — lime green
1109
+ 4 : (255, 165, 0), # LAD distal — orange
1110
+ 5 : (180, 0, 255), # D1 — violet
1111
+ 6 : ( 0, 230, 230), # D2 — cyan
1112
+ 7 : (255, 230, 0), # D3 — yellow
1113
+ 8 : (255, 0, 180), # Septal — hot pink
1114
+ 9 : ( 0, 180, 80), # LAD var — jade green
1115
+ 10: (255, 100, 0), # LCX proximal — deep orange
1116
+ 11: ( 40, 40, 255), # LCX mid — royal blue
1117
+ 12: ( 0, 255, 140), # LCX distal — spring green
1118
+ 13: (220, 0, 80), # OM1 — crimson
1119
+ 14: ( 0, 200, 255), # OM2 — sky blue
1120
+ 15: (200, 255, 0), # OM3 — chartreuse
1121
+ 16: (140, 0, 200), # LCX var — purple
1122
+ 17: (255, 200, 120), # RCA proximal — peach
1123
+ 18: ( 80, 255, 80), # RCA mid — bright green
1124
+ 19: (255, 60, 200), # RCA distal — rose pink
1125
+ 20: ( 0, 80, 200), # PDA — cobalt blue
1126
+ 21: (200, 140, 0), # PLV — dark amber
1127
+ 22: ( 80, 200, 200), # AM branch — teal
1128
+ 23: (255, 255, 120), # Conus — light yellow
1129
+ 24: (160, 60, 0), # SAN artery — brown
1130
+ 25: (180, 180, 180), # Other — silver grey
1131
+ }
1132
+
1133
+ ARCADE_LABELS = {
1134
+ 1:"LMCA", 2:"LAD prox", 3:"LAD mid", 4:"LAD dist",
1135
+ 5:"D1", 6:"D2", 7:"D3", 8:"Septal",
1136
+ 9:"LAD var", 10:"LCX prox", 11:"LCX mid", 12:"LCX dist",
1137
+ 13:"OM1", 14:"OM2", 15:"OM3", 16:"LCX var",
1138
+ 17:"RCA prox",18:"RCA mid", 19:"RCA dist", 20:"PDA",
1139
+ 21:"PLV", 22:"AM", 23:"Conus", 24:"SAN",
1140
+ 25:"Other",
1141
+ }
1142
+
1143
+
1144
+ def render_nnunet_overlay(frame_rgb, seg_map, alpha=0.55):
1145
+ """
1146
+ Renders nnUNet segmentation overlay.
1147
+ Handles two cases:
1148
+ - Binary output (classes 0,1): single vessel colour overlay
1149
+ - 26-class output (ARCADE SYNTAX): full colour-coded overlay with legend
1150
+ """
1151
+ unique_cls = [c for c in np.unique(seg_map) if c > 0]
1152
+ overlay = frame_rgb.copy().astype(np.float32)
1153
+ present_classes = []
1154
+
1155
+ # ── Binary case (0/1 only) ────────────────────────────────────────────────
1156
+ if len(unique_cls) == 1 and unique_cls[0] == 1:
1157
+ mask = (seg_map == 1)
1158
+ col = np.array([0, 200, 255], dtype=np.float32) # cyan for vessel
1159
+ overlay[mask] = overlay[mask] * (1 - alpha) + col * alpha
1160
+ result = overlay.astype(np.uint8)
1161
+ # Simple label
1162
+ cv2.putText(result, "Vessel mask (binary)",
1163
+ (8, 20), cv2.FONT_HERSHEY_SIMPLEX,
1164
+ 0.5, (0, 200, 255), 1, cv2.LINE_AA)
1165
+ return result
1166
+
1167
+ # ── 26-class case ─────────────────────────────────────────────────────────
1168
+ for cls_id, rgb_col in ARCADE_COLOURS.items():
1169
+ if cls_id == 0:
1170
+ continue
1171
+ mask = (seg_map == cls_id)
1172
+ if not mask.any():
1173
+ continue
1174
+ col = np.array(rgb_col, dtype=np.float32)
1175
+ overlay[mask] = overlay[mask] * (1 - alpha) + col * alpha
1176
+ present_classes.append(cls_id)
1177
+
1178
+ result = overlay.astype(np.uint8)
1179
+
1180
+ # Compact legend — bottom-left, only present classes, max 13 rows
1181
+ if present_classes:
1182
+ try:
1183
+ n_rows = min(len(present_classes), 13)
1184
+ row_h = 18
1185
+ legend_h = n_rows * row_h + 10
1186
+ legend_w = 148
1187
+ H, W = result.shape[:2]
1188
+ x0, y0 = 6, H - legend_h - 6
1189
+ sub = result[y0:y0+legend_h, x0:x0+legend_w].astype(np.float32)
1190
+ result[y0:y0+legend_h, x0:x0+legend_w] = (sub * 0.4).astype(np.uint8)
1191
+ for i, cls_id in enumerate(present_classes[:13]):
1192
+ r, g, b = ARCADE_COLOURS[cls_id]
1193
+ lbl = ARCADE_LABELS.get(cls_id, f"cls{cls_id}")
1194
+ yy = y0 + 6 + i * row_h
1195
+ cv2.rectangle(result, (x0+4, yy), (x0+16, yy+11), (b, g, r), -1)
1196
+ cv2.rectangle(result, (x0+4, yy), (x0+16, yy+11), (255,255,255), 1)
1197
+ cv2.putText(result, lbl, (x0+20, yy+9),
1198
+ cv2.FONT_HERSHEY_SIMPLEX, 0.35,
1199
+ (255,255,255), 1, cv2.LINE_AA)
1200
+ except Exception:
1201
+ pass
1202
+
1203
+ return result
1204
+
1205
+ # ─────────────────────────────────────────────────────────────────────────────
1206
+ # MODULE 7 — SYNTAX SCORE (simplified anatomical estimate)
1207
+ # ─────────────────────────────────────────────────────────────────────────────
1208
+
1209
+ def compute_syntax_score(stenoses, seg_map):
1210
+ """
1211
+ Simplified SYNTAX estimate:
1212
+ Each stenosis contributes a weight based on DS% and
1213
+ anatomical territory (estimated from nnUNet class ID).
1214
+ Reference: Sianos et al. EuroIntervention 2005.
1215
+ Note: true SYNTAX requires angiographic expertise; this is
1216
+ an automated surrogate for display purposes.
1217
+ """
1218
+ if not stenoses:
1219
+ return 0.0, "Low (0)"
1220
+
1221
+ base_score = 0.
1222
+ for st in stenoses:
1223
+ ds = st["pct_ds"] / 100
1224
+ # Weight by position: proximal vessels score higher
1225
+ pos_w = max(0.5, 1.5 - st["position_mm"] / 100)
1226
+ # Occlusion multiplier
1227
+ occ_w = 5.0 if ds >= 0.99 else (2.5 if ds >= 0.70 else 1.5 if ds >= 0.50 else 1.0)
1228
+ base_score += pos_w * occ_w * (1 + ds)
1229
+
1230
+ # Add segment count from nnUNet
1231
+ n_segs = len(np.unique(seg_map)) - 1 # exclude background
1232
+ base_score += n_segs * 0.3
1233
+
1234
+ score = round(base_score, 1)
1235
+ if score < 23: tier = "Low (<23)"
1236
+ elif score < 33: tier = "Intermediate (23–32)"
1237
+ else: tier = "High (≥33)"
1238
+ return score, tier
1239
+
1240
+ # ─────────────────────────────────────────────────────────────────────────────
1241
+ # JSON ENCODER — handles numpy float32/int64 etc.
1242
+ # ─────────────────────────────────────────────────────────────────────────────
1243
+
1244
+ class _NpEncoder(json.JSONEncoder):
1245
+ def default(self, obj):
1246
+ if isinstance(obj, np.integer): return int(obj)
1247
+ if isinstance(obj, np.floating): return float(obj)
1248
+ if isinstance(obj, np.ndarray): return obj.tolist()
1249
+ if isinstance(obj, np.bool_): return bool(obj)
1250
+ return super().default(obj)
1251
+
1252
+
1253
+ def _to_json(obj) -> str:
1254
+ return json.dumps(obj, indent=2, cls=_NpEncoder)
1255
+
1256
+
1257
+ # ─────────────────────────────────────────────────────────────────────────────
1258
+ # MAIN PIPELINE FUNCTION
1259
+ # ─────────────────────────────────────────────────────────────────────────────
1260
+
1261
+ def analyse(video_path, sten_conf, seg_conf, px_per_mm_override, progress=gr.Progress()):
1262
+ """
1263
+ Full pipeline. Called by Gradio on button click.
1264
+ Returns: (status_html, frame_img, seg_img, ffr_img, metrics_html, json_str)
1265
+ """
1266
+ if video_path is None:
1267
+ return (_status("error","No video uploaded."), None, None, None, "", "")
1268
+
1269
+ FFR_CFG["px_per_mm"] = px_per_mm_override
1270
+
1271
+ try:
1272
+ # Step 1 — keyframe
1273
+ progress(0.10, desc="Extracting best frame…")
1274
+ frame_rgb, frame_idx, frame_score, total_frames = extract_best_frame(video_path)
1275
+
1276
+ # Step 2 — stenosis detection
1277
+ progress(0.30, desc="Running Mask2Former…")
1278
+ detections = run_mask2former(frame_rgb, sten_conf)
1279
+
1280
+ # Compute vessel overlap for each detection
1281
+ binary_mask_for_overlap = np.ones((512,512),dtype=np.uint8) # placeholder until ResUNet
1282
+ for det in detections:
1283
+ b=det.get("bbox",[]);
1284
+ if len(b)<4: det["overlap"]=0.; continue
1285
+ x1,y1,x2,y2=[int(v) for v in b]
1286
+ area=(x2-x1)*(y2-y1)
1287
+ det["overlap"]=min(float(binary_mask_for_overlap[y1:y2,x1:x2].sum())/max(area,1),1.)
1288
+
1289
+ # Step 3a — binary segmentation (ResUNet)
1290
+ progress(0.50, desc="Running ResUNet…")
1291
+ binary_mask, binary_overlay = run_resunet(frame_rgb)
1292
+ print(f"[ResUNet] binary_mask sum={binary_mask.sum()} "
1293
+ f"unique={np.unique(binary_mask)} "
1294
+ f"vessel%={100*binary_mask.mean():.1f}")
1295
+ # Recompute overlap with real mask
1296
+ for det in detections:
1297
+ b=det.get("bbox",[])
1298
+ if len(b)<4: continue
1299
+ x1,y1,x2,y2=[int(v) for v in b]
1300
+ area=(x2-x1)*(y2-y1)
1301
+ det["overlap"]=float(binary_mask[y1:y2,x1:x2].sum())/max(area,1)
1302
+
1303
+ # Step 3b — YOLOv8m-seg 26-class segmentation
1304
+ progress(0.65, desc="Running YOLOv8m-seg…")
1305
+ seg_map = run_yolo_seg(frame_rgb, seg_conf)
1306
+ seg_overlay = render_nnunet_overlay(frame_rgb, seg_map)
1307
+
1308
+ # Step 4 — FFR
1309
+ progress(0.80, desc="Computing FFR…")
1310
+ ffr_result, ffr_fig = run_ffr(frame_rgb, binary_mask, detections)
1311
+
1312
+ if "error" in ffr_result:
1313
+ return (_status("warn", ffr_result["error"]),
1314
+ frame_rgb, binary_overlay, seg_overlay, None,
1315
+ _metrics_html(ffr_result, {}, frame_idx, total_frames),
1316
+ _to_json(ffr_result))
1317
+
1318
+ # Step 5 — SYNTAX score
1319
+ progress(0.92, desc="Computing SYNTAX score…")
1320
+ syntax_score, syntax_tier = compute_syntax_score(ffr_result["stenoses"], seg_map)
1321
+ ffr_result["syntax_score"] = syntax_score
1322
+ ffr_result["syntax_tier"] = syntax_tier
1323
+
1324
+ progress(1.00, desc="Done.")
1325
+ status = _status("ok",
1326
+ f"Analysis complete — Frame {frame_idx}/{total_frames} "
1327
+ f"(quality score {frame_score:.1f})")
1328
+ metrics = _metrics_html(ffr_result, {"score":frame_score,"idx":frame_idx},
1329
+ frame_idx, total_frames)
1330
+
1331
+ return (status, frame_rgb, binary_overlay, seg_overlay, ffr_fig, metrics,
1332
+ _to_json(ffr_result))
1333
+
1334
+ except FileNotFoundError as e:
1335
+ return (_status("error", str(e)), None, None, None, None,
1336
+ _metrics_html({},{},0,0),
1337
+ _to_json({"error":str(e)}))
1338
+ except Exception as e:
1339
+ import traceback
1340
+ tb = traceback.format_exc()
1341
+ return (_status("error", f"{type(e).__name__}: {e}"), None, None, None, None,
1342
+ _metrics_html({},{},0,0),
1343
+ _to_json({"error":str(e),"traceback":tb}))
1344
+
1345
+ # ─────────────────────────────────────────────────────────────────────────────
1346
+ # HTML HELPERS
1347
+ # ─────────────────────────────────────────────────────────────────────────────
1348
+
1349
+ def _status(kind: str, msg: str) -> str:
1350
+ colours = {"ok":"#0d9e6e","warn":"#d98b00","error":"#d64040"}
1351
+ icons = {"ok":"✓","warn":"⚠","error":"✗"}
1352
+ col = colours.get(kind, "#1a6fa8")
1353
+ ico = icons.get(kind, "•")
1354
+ return (f'<div id="status-box" style="border-color:{col};color:{col};'
1355
+ f'background:{"#f0fbf7" if kind=="ok" else "#fff8f0" if kind=="warn" else "#fff0f0"}">'
1356
+ f'{ico} {msg}</div>')
1357
+
1358
+
1359
+ def _metrics_html(ffr_result, frame_info, frame_idx, total) -> str:
1360
+ if not ffr_result or "error" in ffr_result:
1361
+ return '<div class="card"><div class="card-title">Metrics</div><p style="color:#5a7a96">Run analysis to see results.</p></div>'
1362
+
1363
+ gffr = ffr_result.get("global_ffr", "—")
1364
+ isch = ffr_result.get("ischemic", None)
1365
+ ns = ffr_result.get("n_stenoses", 0)
1366
+ ref = ffr_result.get("ref_diam_mm","—")
1367
+ vlen = ffr_result.get("vessel_len_mm","—")
1368
+ syn = ffr_result.get("syntax_score","—")
1369
+ stier = ffr_result.get("syntax_tier","—")
1370
+
1371
+ if isch is True: ffr_cls,ffr_lbl = "ischemic","⚠ ISCHEMIC"
1372
+ elif isch is False: ffr_cls,ffr_lbl = "ok","✓ NON-ISCHEMIC"
1373
+ else: ffr_cls,ffr_lbl = "","—"
1374
+
1375
+ syn_val = float(syn) if isinstance(syn,float) else 0
1376
+ syn_cls = "ischemic" if syn_val>=33 else ("borderline" if syn_val>=23 else "ok")
1377
+
1378
+ stenosis_rows = ""
1379
+ for i,st in enumerate(ffr_result.get("stenoses",[])):
1380
+ sig_col = "#d64040" if st.get("significant") else "#0d9e6e"
1381
+ stenosis_rows += (
1382
+ f'<tr>'
1383
+ f'<td style="padding:4px 8px">#{i+1}</td>'
1384
+ f'<td style="padding:4px 8px">{st["position_mm"]:.1f} mm</td>'
1385
+ f'<td style="padding:4px 8px">{st["pct_ds"]:.1f}%</td>'
1386
+ f'<td style="padding:4px 8px">{st["d_mm"]:.2f} mm</td>'
1387
+ f'<td style="padding:4px 8px;color:{sig_col};font-weight:600">{st["ffr"]:.3f}</td>'
1388
+ f'</tr>'
1389
+ )
1390
+
1391
+ return f"""
1392
+ <div class="card">
1393
+ <div class="card-title">Key metrics</div>
1394
+ <div class="metric-row">
1395
+ <div class="metric-badge">
1396
+ <div class="val {ffr_cls}">{gffr if gffr!="—" else "—"}</div>
1397
+ <div class="lbl">Global FFR</div>
1398
+ </div>
1399
+ <div class="metric-badge">
1400
+ <div class="val {ffr_cls}" style="font-size:14px">{ffr_lbl}</div>
1401
+ <div class="lbl">Ischaemia</div>
1402
+ </div>
1403
+ <div class="metric-badge">
1404
+ <div class="val">{ns}</div>
1405
+ <div class="lbl">Stenoses</div>
1406
+ </div>
1407
+ <div class="metric-badge">
1408
+ <div class="val {syn_cls}">{syn}</div>
1409
+ <div class="lbl">SYNTAX score</div>
1410
+ </div>
1411
+ <div class="metric-badge">
1412
+ <div class="val">{ref} mm</div>
1413
+ <div class="lbl">Ref diameter</div>
1414
+ </div>
1415
+ <div class="metric-badge">
1416
+ <div class="val">{vlen} mm</div>
1417
+ <div class="lbl">Vessel length</div>
1418
+ </div>
1419
+ </div>
1420
+ <div style="font-size:12px;color:#5a7a96;margin-top:4px">
1421
+ SYNTAX tier: <b>{stier}</b> &nbsp;|&nbsp;
1422
+ Frame {frame_idx}/{total} &nbsp;|&nbsp;
1423
+ Scale: {FFR_CFG["px_per_mm"]} px/mm
1424
+ </div>
1425
+ </div>
1426
+ {'<div class="card"><div class="card-title">Stenosis table</div><table style="width:100%;border-collapse:collapse;font-size:13px"><thead><tr style="background:#e8f3fb;color:#1a6fa8"><th style="padding:6px 8px;text-align:left">#</th><th style="padding:6px 8px;text-align:left">Position</th><th style="padding:6px 8px;text-align:left">DS%</th><th style="padding:6px 8px;text-align:left">Min diam</th><th style="padding:6px 8px;text-align:left">FFR</th></tr></thead><tbody>' + stenosis_rows + '</tbody></table></div>'
1427
+ if stenosis_rows else ""}
1428
+ """
1429
+
1430
+ # ─────────────────────────────────────────────────────────────────────────────
1431
+ # GRADIO APP
1432
+ # ─────────────────────────────────────────────────────────────────────────────
1433
+
1434
+ with gr.Blocks(css=CSS, title="Angio AI") as demo:
1435
+
1436
+ # Header
1437
+ gr.HTML(_header_html())
1438
+
1439
+ with gr.Tabs() as tabs:
1440
+
1441
+ # ── PAGE 1: UPLOAD ────────────────────────────────────────────────
1442
+ with gr.TabItem("📤 Upload & Configure", id="tab-upload"):
1443
+ gr.HTML('<div style="height:12px"></div>')
1444
+
1445
+ with gr.Row():
1446
+ # Left — upload + controls
1447
+ with gr.Column(scale=2):
1448
+ gr.HTML('<div class="card"><div class="card-title">Angiographic video</div>')
1449
+ video_input = gr.Video(
1450
+ label="Upload XCA video (mp4 / avi / dicom-wrapped)",
1451
+ elem_id="upload-zone",
1452
+ height=240,
1453
+ )
1454
+ gr.HTML('</div>')
1455
+
1456
+ gr.HTML('<div class="card"><div class="card-title">Model controls</div>')
1457
+ sten_conf = gr.Slider(
1458
+ minimum=0.05, maximum=0.95, value=0.30, step=0.05,
1459
+ label="Stenosis confidence threshold (Mask2Former)",
1460
+ info="Detections below this confidence are discarded",
1461
+ )
1462
+ seg_conf = gr.Slider(
1463
+ minimum=0.05, maximum=0.95, value=0.25, step=0.05,
1464
+ label="Segmentation confidence threshold (YOLOv8m-seg)",
1465
+ info="Detections below this confidence are discarded by YOLO NMS",
1466
+ )
1467
+ px_per_mm = gr.Slider(
1468
+ minimum=2.0, maximum=6.0, value=3.75, step=0.25,
1469
+ label="Scale (px / mm)",
1470
+ info="ARCADE default = 3.75 px/mm (0.267 mm/px). Adjust if using non-ARCADE data.",
1471
+ )
1472
+ gr.HTML('</div>')
1473
+
1474
+ # Right — info card
1475
+ with gr.Column(scale=1):
1476
+ gr.HTML("""
1477
+ <div class="card">
1478
+ <div class="card-title">Pipeline overview</div>
1479
+ <ol style="font-size:13px;color:#1a2533;line-height:1.9;padding-left:16px">
1480
+ <li>Best frame extracted from video</li>
1481
+ <li>Stenosis detection — Mask2Former</li>
1482
+ <li>Coronary segmentation — YOLOv8m-seg (26-class)</li>
1483
+ <li>Binary vessel mask — ResUNet</li>
1484
+ <li>FFR estimation — QFR v4</li>
1485
+ <li>SYNTAX score computation</li>
1486
+ </ol>
1487
+ <hr style="border-color:#cce0f0;margin:12px 0">
1488
+ <div style="font-size:12px;color:#5a7a96">
1489
+ <b>FFR threshold:</b> ≤ 0.80 → ischemic<br>
1490
+ <b>Formula:</b> 1 − (0.33·DS + 0.60·DS²)<br>
1491
+ <b>Reference:</b> Tu et al. JACC 2016
1492
+ </div>
1493
+ </div>
1494
+ <div class="card">
1495
+ <div class="card-title">Model checkpoints</div>
1496
+ <div style="font-size:12px;color:#5a7a96;line-height:1.8">
1497
+ Auto-downloaded from<br>
1498
+ <code>MuhammadAdil63/</code><br>
1499
+ <code>angio-ai-checkpoints</code><br><br>
1500
+ <code>mask2former_best.pth</code><br>
1501
+ <code>binary_best.pth</code><br>
1502
+ <code>best.pt</code>
1503
+ </div>
1504
+ </div>
1505
+ """)
1506
+
1507
+ with gr.Row():
1508
+ with gr.Column(scale=2):
1509
+ btn_analyse = gr.Button("▶ Run analysis",
1510
+ elem_id="btn-analyse", variant="primary")
1511
+ with gr.Column(scale=1):
1512
+ btn_reset = gr.Button("↺ Clear", elem_id="btn-reset")
1513
+
1514
+ status_out = gr.HTML(
1515
+ '<div id="status-box">Upload a video and press Run analysis.</div>')
1516
+
1517
+ # ── PAGE 2: RESULTS ───────────────────────────────────────────────
1518
+ with gr.TabItem("📊 Results", id="tab-results"):
1519
+ gr.HTML('<div style="height:12px"></div>')
1520
+
1521
+ status_out2 = gr.HTML()
1522
+ metrics_out = gr.HTML()
1523
+
1524
+ with gr.Row():
1525
+ with gr.Column():
1526
+ gr.HTML('<div class="card-title" style="margin-bottom:6px">Best frame extracted</div>')
1527
+ frame_out = gr.Image(label="Best keyframe", height=300)
1528
+ with gr.Column():
1529
+ gr.HTML('<div class="card-title" style="margin-bottom:6px">Binary vessel mask (ResUNet)</div>')
1530
+ binary_out = gr.Image(label="Green overlay — vessel pixels", height=300)
1531
+
1532
+ with gr.Row():
1533
+ with gr.Column():
1534
+ gr.HTML('<div class="card-title" style="margin-bottom:6px">Coronary segmentation (YOLOv8m-seg)</div>')
1535
+ seg_out = gr.Image(label="26-class overlay", height=300)
1536
+ with gr.Column():
1537
+ gr.HTML('<div class="card-title" style="margin-bottom:6px">FFR analysis</div>')
1538
+ ffr_out = gr.Image(label="FFR pipeline output", height=300)
1539
+
1540
+ with gr.Row():
1541
+ with gr.Column():
1542
+ gr.HTML('<div class="card-title" style="margin-bottom:6px">Full results (JSON)</div>')
1543
+ json_out = gr.Code(label="", language="json", lines=12)
1544
+
1545
+ # Footer
1546
+ gr.HTML("""
1547
+ <div id="footer">
1548
+ Angio AI · MS Data Science · Information Technology University Lahore ·
1549
+ Supervisor: Dr. Arif Mehmood &nbsp;|&nbsp;
1550
+ Image-based QFR — Tu et al. JACC 2016 · FAVOR II trial
1551
+ </div>""")
1552
+
1553
+ # ── WIRING ────────────────────────────────────────────────────────────
1554
+
1555
+ def run_and_switch(video, sc, sg, px):
1556
+ return analyse(video, sc, sg, px)
1557
+
1558
+ btn_analyse.click(
1559
+ fn=run_and_switch,
1560
+ inputs=[video_input, sten_conf, seg_conf, px_per_mm],
1561
+ outputs=[status_out, frame_out, binary_out, seg_out, ffr_out, metrics_out, json_out],
1562
+ ).then(
1563
+ # Copy status HTML to results page status box
1564
+ fn=lambda s: s,
1565
+ inputs=[status_out],
1566
+ outputs=[status_out2],
1567
+ ).then(
1568
+ # Switch to Results tab via JavaScript
1569
+ fn=None,
1570
+ js="""
1571
+ () => {
1572
+ const tabs = document.querySelectorAll('.tab-nav button');
1573
+ if (tabs.length >= 2) tabs[1].click();
1574
+ }
1575
+ """,
1576
+ )
1577
+
1578
+ btn_reset.click(
1579
+ fn=lambda: (
1580
+ None, None, None, None, None,
1581
+ '<div id="status-box">Upload a video and press Run analysis.</div>',
1582
+ "", "",
1583
+ ),
1584
+ outputs=[video_input, frame_out, binary_out, seg_out, ffr_out,
1585
+ status_out, metrics_out, json_out],
1586
+ )
1587
+
1588
+ # ─────────────────────────────────────────────────────────────────────────────
1589
+ # ENTRY POINT
1590
+ # ─────────────────────────────────────────────────────────────────────────────
1591
+
1592
+ if __name__ == "__main__":
1593
+ demo.launch(
1594
+ server_name="0.0.0.0",
1595
+ server_port=7860,
1596
+ share=False,
1597
+ inbrowser=False, # must be False on HF Spaces
1598
+ )
packages.txt ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ # System packages installed via apt before pip
2
+ # Required by opencv-python-headless on HF Spaces Ubuntu base image
3
+ libgl1-mesa-glx
4
+ libglib2.0-0
5
+ libsm6
6
+ libxext6
7
+ libxrender-dev
requirements.txt ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ── Angio AI — HuggingFace Spaces requirements ────────────────────────────────
2
+ # Python 3.10 | CPU-only free tier
3
+ # Pin versions that are known to work together on HF Spaces CPU.
4
+
5
+ # ── Core ML ───────────────────────────────────────────────────────────────────
6
+ torch==2.2.2+cpu
7
+ torchvision==0.17.2+cpu
8
+ --extra-index-url https://download.pytorch.org/whl/cpu
9
+
10
+ # ── Transformers (Mask2Former) ────────────────────────────────────────────────
11
+ transformers==4.40.2
12
+ accelerate==0.30.0
13
+
14
+ # ── YOLO (ultralytics) ───────────────────────────────────────────────────────
15
+ ultralytics==8.2.18
16
+
17
+ # ── HuggingFace Hub (checkpoint download) ────────────────────────────────────
18
+ huggingface_hub==0.23.2
19
+
20
+ # ── Computer Vision ───────────────────────────────────────────────────────────
21
+ opencv-python-headless==4.9.0.80
22
+ scikit-image==0.22.0
23
+ scipy==1.13.0
24
+
25
+ # ── Visualisation / UI ────────────────────────────────────────────────────────
26
+ gradio==4.31.5
27
+ matplotlib==3.8.4
28
+ numpy==1.26.4