Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,72 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- tabular-classification
|
| 5 |
+
- healthcare
|
| 6 |
+
- synthetic-data
|
| 7 |
+
- ehr
|
| 8 |
+
library_name: pytorch
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# Patient Timeline Forecasting — ens_lr_gbdt_tx
|
| 12 |
+
|
| 13 |
+
Predicts which of 40 target conditions are **newly** diagnosed in the five
|
| 14 |
+
years after a patient's anchor date, from structured Synthea EHR events
|
| 15 |
+
recorded strictly before it.
|
| 16 |
+
|
| 17 |
+
Trained for the M31 research-intern take-home. **Synthetic data only. Not a
|
| 18 |
+
clinical device, and not usable for any decision about a real person.**
|
| 19 |
+
|
| 20 |
+
## Task
|
| 21 |
+
|
| 22 |
+
- 3,514 Synthea patients, split 2,791 train / 365 validation / 358 test.
|
| 23 |
+
- Anchor = last recorded encounter minus five calendar years, floored to midnight.
|
| 24 |
+
- Label = **first-ever** diagnosis of the condition falls in `[anchor, anchor+5y]`.
|
| 25 |
+
A patient already diagnosed before the anchor is *prevalent*: structurally
|
| 26 |
+
negative, and reported separately rather than counted as an ordinary negative.
|
| 27 |
+
|
| 28 |
+
## Results (validation, n=365)
|
| 29 |
+
|
| 30 |
+
| metric | value |
|
| 31 |
+
|---|---|
|
| 32 |
+
| val_macro_auroc | 0.7700 |
|
| 33 |
+
| val_macro_ap | 0.2681 |
|
| 34 |
+
|
| 35 |
+
The validation set resolves differences of roughly **±0.01 macro AUROC** and no
|
| 36 |
+
better. Per-condition AUROC on the rarest label (5 positives) carries a
|
| 37 |
+
Hanley–McNeil 95% interval of about ±0.24. Numbers below that separation are
|
| 38 |
+
reported but should not be read as rankings.
|
| 39 |
+
|
| 40 |
+
## What the model is actually learning
|
| 41 |
+
|
| 42 |
+
The anchor is defined from each patient's last encounter, so for the **42.9%**
|
| 43 |
+
of training patients with a death date, the outcome window is exactly their
|
| 44 |
+
last five years of life — 100% of those deaths fall within 30 days of the
|
| 45 |
+
window end. A substantial part of the achievable signal is therefore
|
| 46 |
+
"is this record about to end", which is a property of how the task was
|
| 47 |
+
constructed rather than of clinical prediction. The same rule generated the
|
| 48 |
+
test anchors, so this is not leakage, but it does bound how the results should
|
| 49 |
+
be interpreted.
|
| 50 |
+
|
| 51 |
+
## Leakage controls
|
| 52 |
+
|
| 53 |
+
`DEATHDATE`, `HEALTHCARE_EXPENSES` and `HEALTHCARE_COVERAGE` are refused at
|
| 54 |
+
load time. `STOP`-derived durations are excluded: the organisers blanked
|
| 55 |
+
post-anchor stops in the test split, so such a feature would both leak and
|
| 56 |
+
shift. Every fitted statistic — vocabulary, quantile edges, scalers — is fitted
|
| 57 |
+
on training patients only. 63 automated checks cover this, including a grep
|
| 58 |
+
test that no module outside the time utility parses a timestamp.
|
| 59 |
+
|
| 60 |
+
## Limitations
|
| 61 |
+
|
| 62 |
+
Synthea is not real EHR data. A discriminator separates it from MIMIC at
|
| 63 |
+
AUC 0.999, and across 19 health datasets the winning classifier agreed between
|
| 64 |
+
real- and synthetic-trained models in only 21–26% of cases. **Whichever model
|
| 65 |
+
wins here, that ranking should not be assumed to transfer.**
|
| 66 |
+
|
| 67 |
+
## Reproducing
|
| 68 |
+
|
| 69 |
+
```bash
|
| 70 |
+
git clone https://github.com/Sallamsaka/M31-Coding-Test && pip install -r requirements.txt
|
| 71 |
+
./run_all.ps1
|
| 72 |
+
```
|