Spaces:
Sleeping
Sleeping
Solo framing + interpretability, maturity signals, case study
Browse files- Reframe as solo end-to-end applied ML case study (CREDITS, README, UI footer)
- Add CASE_STUDY.md (narrative write-up)
- Add SHAP interpretability, calibration, and screening-threshold analysis
(executed notebook + assets/*.png referenced in README/case study)
- Add [analysis] optional deps
- CASE_STUDY.md +110 -0
- CREDITS.md +12 -18
- README.md +24 -9
- assets/calibration.png +0 -0
- assets/shap_summary.png +0 -0
- assets/threshold.png +0 -0
- frontend/index.html +2 -2
- notebooks/analysis.ipynb +0 -0
- pyproject.toml +3 -0
CASE_STUDY.md
ADDED
|
@@ -0,0 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Case Study β NephroScreen
|
| 2 |
+
|
| 3 |
+
*An end-to-end applied machine-learning project: screening for chronic kidney
|
| 4 |
+
disease from routine lab values, built and analysed solo.*
|
| 5 |
+
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
## At a glance
|
| 9 |
+
|
| 10 |
+
| | |
|
| 11 |
+
|---|---|
|
| 12 |
+
| **Problem** | Binary classification β predict CKD vs. healthy from routine clinical/lab features |
|
| 13 |
+
| **Data** | UCI Chronic Kidney Disease dataset β 400 patients, 24 attributes, ~11% cells missing |
|
| 14 |
+
| **Best model** | Random Forest β 100% test accuracy, ROC-AUC 1.0000 (read the caveat below) |
|
| 15 |
+
| **Stack** | scikit-learn Β· FastAPI Β· vanilla JS frontend Β· Docker/Render |
|
| 16 |
+
| **What makes it mature** | Leakage-safe pipeline Β· SHAP interpretability Β· calibration Β· screening-tuned threshold Β· tests |
|
| 17 |
+
|
| 18 |
+
## 1. The problem
|
| 19 |
+
|
| 20 |
+
Chronic kidney disease often progresses silently until it is advanced, yet the
|
| 21 |
+
warning signs are already present in routine blood and urine tests. The task:
|
| 22 |
+
given a patient's lab panel, flag likely CKD early enough to matter. This is a
|
| 23 |
+
classic **screening** problem, which shapes every later decision β especially
|
| 24 |
+
how errors are weighted.
|
| 25 |
+
|
| 26 |
+
## 2. The data β and an honest catch
|
| 27 |
+
|
| 28 |
+
The UCI CKD dataset is small (400 rows) and the two classes are **highly
|
| 29 |
+
separable** on a few biomarkers. Every model I trained scored in the high 90s,
|
| 30 |
+
and Random Forest reached **100% on the held-out test set**.
|
| 31 |
+
|
| 32 |
+
**That number is a red flag, not a trophy.** With an 80-patient test set drawn
|
| 33 |
+
from one clean, single-source dataset, near-perfect accuracy reflects the
|
| 34 |
+
*separability of this data*, not real-world clinical performance. I treat it that
|
| 35 |
+
way throughout β the engineering value here is doing the workflow *correctly*,
|
| 36 |
+
not chasing a headline metric.
|
| 37 |
+
|
| 38 |
+
## 3. Approach
|
| 39 |
+
|
| 40 |
+
**Leakage-safe preprocessing.** All imputation, encoding, and feature
|
| 41 |
+
engineering live in a custom scikit-learn transformer (`CKDPreprocessor`) wrapped
|
| 42 |
+
in a `Pipeline`. Every statistic β medians, modes, the scaler, the `age_group`
|
| 43 |
+
fill value β is fit on the **training fold only**, so cross-validation and the
|
| 44 |
+
live API never see test-set information.
|
| 45 |
+
|
| 46 |
+
**Domain feature engineering.** Four clinically motivated features, e.g.
|
| 47 |
+
`kidney_stress_index = serum_creatinine Γ blood_urea / hemoglobin` (the CKD triad
|
| 48 |
+
in one number). SHAP later confirmed it earns its place in the top five.
|
| 49 |
+
|
| 50 |
+
**Three models compared** with 10-fold stratified CV:
|
| 51 |
+
|
| 52 |
+
| Model | Accuracy | Recall | F1 | ROC-AUC | 10-Fold CV |
|
| 53 |
+
|---|---|---|---|---|---|
|
| 54 |
+
| Logistic Regression | 98.75% | 98% | 98.99% | 1.0000 | 99.38% Β± 1.25% |
|
| 55 |
+
| **Random Forest** | **100%** | **100%** | **100%** | **1.0000** | 98.75% Β± 2.86% |
|
| 56 |
+
| Decision Tree | 97.50% | 98% | 98.00% | 0.9893 | 96.56% Β± 2.19% |
|
| 57 |
+
|
| 58 |
+
## 4. Interpretability β what the model actually uses
|
| 59 |
+
|
| 60 |
+

|
| 61 |
+
|
| 62 |
+
SHAP attributions are clinically coherent and build trust in the model:
|
| 63 |
+
|
| 64 |
+
- **Low hemoglobin and PCV** push strongly toward CKD (anemia of kidney disease).
|
| 65 |
+
- **High serum creatinine** and the engineered **kidney_stress_index** are top drivers.
|
| 66 |
+
- **Hypertension and diabetes = "yes"** push toward CKD β the two leading real-world causes.
|
| 67 |
+
|
| 68 |
+
This is the difference between "the model is accurate" and "I understand *why*
|
| 69 |
+
the model is accurate" β the latter is what makes a result defensible.
|
| 70 |
+
|
| 71 |
+
## 5. Maturity signals
|
| 72 |
+
|
| 73 |
+
**Calibration.** Do the probabilities mean what they say? The reliability curve
|
| 74 |
+
checks whether an "0.8" prediction is CKD ~80% of the time β essential before
|
| 75 |
+
anyone acts on a probability.
|
| 76 |
+
|
| 77 |
+

|
| 78 |
+
|
| 79 |
+
**A screening-tuned threshold.** For screening, a *missed* CKD case (false
|
| 80 |
+
negative) is far costlier than a false alarm. Instead of the default 0.5, the API
|
| 81 |
+
serves a threshold tuned to keep **recall β₯ 0.99**, accepting more false
|
| 82 |
+
positives as the safer trade.
|
| 83 |
+
|
| 84 |
+

|
| 85 |
+
|
| 86 |
+
## 6. From model to product
|
| 87 |
+
|
| 88 |
+
The trained pipeline is persisted with `joblib` and served by a **FastAPI**
|
| 89 |
+
backend; a dependency-free web frontend collects a lab panel (missing fields are
|
| 90 |
+
imputed) and shows the probability, a risk band, and which entered values fall
|
| 91 |
+
outside normal clinical ranges. Containerised for free-tier deployment.
|
| 92 |
+
|
| 93 |
+
```
|
| 94 |
+
Browser β FastAPI /api/predict β joblib pipeline (CKDPreprocessor β RandomForest)
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
## 7. Limitations & what I'd do next
|
| 98 |
+
|
| 99 |
+
- **Single small dataset** β results will not transfer to real clinical
|
| 100 |
+
populations without external validation on an independent cohort.
|
| 101 |
+
- **No serving-time calibration** yet; probabilities should be calibrated
|
| 102 |
+
(e.g. isotonic) before being shown as clinical likelihoods.
|
| 103 |
+
- **Next experiment:** find a **minimal, low-cost feature subset** β how much
|
| 104 |
+
accuracy is lost if the expensive labs are dropped? That's the question with
|
| 105 |
+
real value for screening in resource-limited settings.
|
| 106 |
+
|
| 107 |
+
---
|
| 108 |
+
|
| 109 |
+
*Built solo by Sadiq Mansoor. Dataset: UCI ML Repository β Chronic Kidney
|
| 110 |
+
Disease (ID 336). Educational project, not a medical device.*
|
CREDITS.md
CHANGED
|
@@ -1,24 +1,18 @@
|
|
| 1 |
# Credits & Origin
|
| 2 |
|
| 3 |
-
NephroScreen
|
| 4 |
-
|
| 5 |
-
dataset. That original submission β an exploratory notebook, a comparison of
|
| 6 |
-
three classifiers, and a written report β was completed and graded.
|
| 7 |
|
| 8 |
-
|
|
|
|
|
|
|
| 9 |
|
| 10 |
-
-
|
| 11 |
-
|
| 12 |
-
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
and SHAP analysis, and an automated test suite.
|
| 17 |
-
- Containerised and configured for free-tier cloud deployment.
|
| 18 |
|
| 19 |
-
|
| 20 |
-
spirit of honest attribution. The engineering in this repository β the package,
|
| 21 |
-
API, frontend, deployment, and added analysis β is my own work.
|
| 22 |
-
|
| 23 |
-
**Original coursework contributors:** Sadiq Mansoor, Muhammad Rayyan Hassan.
|
| 24 |
**Dataset:** UCI Machine Learning Repository β Chronic Kidney Disease (ID 336).
|
|
|
|
| 1 |
# Credits & Origin
|
| 2 |
|
| 3 |
+
NephroScreen is a **solo project**. The design, data analysis, machine-learning
|
| 4 |
+
pipeline, API, web interface, deployment, and write-up are entirely my own work.
|
|
|
|
|
|
|
| 5 |
|
| 6 |
+
An earlier version of the core analysis was submitted for a university Machine
|
| 7 |
+
Learning course. This repository is a **complete independent rebuild and
|
| 8 |
+
extension** of that idea, adding:
|
| 9 |
|
| 10 |
+
- An installable package with a leakage-safe, train-only preprocessing pipeline.
|
| 11 |
+
- A **FastAPI** service and a **professional web frontend** for live screening.
|
| 12 |
+
- **Serving-oriented ML work**: model persistence, a recall-tuned decision
|
| 13 |
+
threshold, rule-based clinical indicators, calibration and SHAP analysis, and
|
| 14 |
+
an automated test suite.
|
| 15 |
+
- Containerisation and free-tier cloud deployment.
|
|
|
|
|
|
|
| 16 |
|
| 17 |
+
**Author:** Sadiq Mansoor
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
**Dataset:** UCI Machine Learning Repository β Chronic Kidney Disease (ID 336).
|
README.md
CHANGED
|
@@ -1,12 +1,13 @@
|
|
| 1 |
# NephroScreen β Early Chronic Kidney Disease Screening
|
| 2 |
|
| 3 |
-
An end-to-end machine-learning
|
| 4 |
-
(CKD) risk from routine clinical lab values β a **leakage-safe
|
| 5 |
-
pipeline**, a **FastAPI** service, and a **clean web interface**,
|
| 6 |
-
a free tier.
|
| 7 |
|
| 8 |
-
|
| 9 |
-
|
|
|
|
| 10 |
|
| 11 |
---
|
| 12 |
|
|
@@ -48,6 +49,20 @@ Held-out test set (80 patients), 10-fold stratified CV, `random_state=42`:
|
|
| 48 |
Top Random Forest predictors: hemoglobin, packed cell volume, serum creatinine,
|
| 49 |
`kidney_stress_index` (engineered), red blood cell count.
|
| 50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
## Architecture
|
| 52 |
|
| 53 |
```
|
|
@@ -111,6 +126,6 @@ curl -X POST http://127.0.0.1:8000/api/predict \
|
|
| 111 |
|
| 112 |
## Credits
|
| 113 |
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
Licensed under [MIT](LICENSE).
|
|
|
|
| 1 |
# NephroScreen β Early Chronic Kidney Disease Screening
|
| 2 |
|
| 3 |
+
**An end-to-end applied machine-learning case study.** It estimates chronic
|
| 4 |
+
kidney disease (CKD) risk from routine clinical lab values β a **leakage-safe
|
| 5 |
+
scikit-learn pipeline**, a **FastAPI** service, and a **clean web interface**,
|
| 6 |
+
deployable on a free tier.
|
| 7 |
|
| 8 |
+
π **[Read the case study β](CASE_STUDY.md)** Β· βοΈ *Educational project β not a
|
| 9 |
+
medical device.* Trained on a small public dataset (400 patients) and *not*
|
| 10 |
+
validated for clinical use.
|
| 11 |
|
| 12 |
---
|
| 13 |
|
|
|
|
| 49 |
Top Random Forest predictors: hemoglobin, packed cell volume, serum creatinine,
|
| 50 |
`kidney_stress_index` (engineered), red blood cell count.
|
| 51 |
|
| 52 |
+
## Interpretability & maturity
|
| 53 |
+
|
| 54 |
+
Full write-up in the **[case study](CASE_STUDY.md)**. In short:
|
| 55 |
+
|
| 56 |
+
| SHAP β why the model decides | Calibration | Screening threshold |
|
| 57 |
+
|---|---|---|
|
| 58 |
+
|  |  |  |
|
| 59 |
+
|
| 60 |
+
- **SHAP** attributions are clinically coherent (low hemoglobin/PCV, high
|
| 61 |
+
creatinine, hypertension/diabetes drive CKD predictions).
|
| 62 |
+
- **Calibration** curve checks the probabilities are trustworthy, not just the labels.
|
| 63 |
+
- **Recall-tuned threshold (0.62)** β for screening, a missed case costs more
|
| 64 |
+
than a false alarm, so the served threshold keeps recall β₯ 0.99.
|
| 65 |
+
|
| 66 |
## Architecture
|
| 67 |
|
| 68 |
```
|
|
|
|
| 126 |
|
| 127 |
## Credits
|
| 128 |
|
| 129 |
+
Solo project by **Sadiq Mansoor** β see [CREDITS.md](CREDITS.md). An independent
|
| 130 |
+
rebuild and extension of an earlier university coursework analysis. Dataset: UCI
|
| 131 |
+
ML Repository, Chronic Kidney Disease (ID 336). Licensed under [MIT](LICENSE).
|
assets/calibration.png
ADDED
|
assets/shap_summary.png
ADDED
|
assets/threshold.png
ADDED
|
frontend/index.html
CHANGED
|
@@ -57,8 +57,8 @@
|
|
| 57 |
Educational portfolio project β not a medical device.
|
| 58 |
</p>
|
| 59 |
<p class="muted">
|
| 60 |
-
|
| 61 |
-
|
| 62 |
</p>
|
| 63 |
</footer>
|
| 64 |
|
|
|
|
| 57 |
Educational portfolio project β not a medical device.
|
| 58 |
</p>
|
| 59 |
<p class="muted">
|
| 60 |
+
A solo end-to-end applied ML case study by Sadiq Mansoor β
|
| 61 |
+
model, API, and interface engineered from scratch.
|
| 62 |
</p>
|
| 63 |
</footer>
|
| 64 |
|
notebooks/analysis.ipynb
CHANGED
|
The diff for this file is too large to render.
See raw diff
|
|
|
pyproject.toml
CHANGED
|
@@ -10,6 +10,9 @@ readme = "README.md"
|
|
| 10 |
requires-python = ">=3.10"
|
| 11 |
license = { text = "MIT" }
|
| 12 |
|
|
|
|
|
|
|
|
|
|
| 13 |
[tool.setuptools.packages.find]
|
| 14 |
where = ["src"]
|
| 15 |
|
|
|
|
| 10 |
requires-python = ">=3.10"
|
| 11 |
license = { text = "MIT" }
|
| 12 |
|
| 13 |
+
[project.optional-dependencies]
|
| 14 |
+
analysis = ["shap==0.46.0", "matplotlib==3.9.2", "notebook", "nbformat"]
|
| 15 |
+
|
| 16 |
[tool.setuptools.packages.find]
|
| 17 |
where = ["src"]
|
| 18 |
|