sallamsaka commited on
Commit
33da14d
·
verified ·
1 Parent(s): 5994842

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +72 -0
README.md ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - tabular-classification
5
+ - healthcare
6
+ - synthetic-data
7
+ - ehr
8
+ library_name: pytorch
9
+ ---
10
+
11
+ # Patient Timeline Forecasting — ens_lr_gbdt_tx
12
+
13
+ Predicts which of 40 target conditions are **newly** diagnosed in the five
14
+ years after a patient's anchor date, from structured Synthea EHR events
15
+ recorded strictly before it.
16
+
17
+ Trained for the M31 research-intern take-home. **Synthetic data only. Not a
18
+ clinical device, and not usable for any decision about a real person.**
19
+
20
+ ## Task
21
+
22
+ - 3,514 Synthea patients, split 2,791 train / 365 validation / 358 test.
23
+ - Anchor = last recorded encounter minus five calendar years, floored to midnight.
24
+ - Label = **first-ever** diagnosis of the condition falls in `[anchor, anchor+5y]`.
25
+ A patient already diagnosed before the anchor is *prevalent*: structurally
26
+ negative, and reported separately rather than counted as an ordinary negative.
27
+
28
+ ## Results (validation, n=365)
29
+
30
+ | metric | value |
31
+ |---|---|
32
+ | val_macro_auroc | 0.7700 |
33
+ | val_macro_ap | 0.2681 |
34
+
35
+ The validation set resolves differences of roughly **±0.01 macro AUROC** and no
36
+ better. Per-condition AUROC on the rarest label (5 positives) carries a
37
+ Hanley–McNeil 95% interval of about ±0.24. Numbers below that separation are
38
+ reported but should not be read as rankings.
39
+
40
+ ## What the model is actually learning
41
+
42
+ The anchor is defined from each patient's last encounter, so for the **42.9%**
43
+ of training patients with a death date, the outcome window is exactly their
44
+ last five years of life — 100% of those deaths fall within 30 days of the
45
+ window end. A substantial part of the achievable signal is therefore
46
+ "is this record about to end", which is a property of how the task was
47
+ constructed rather than of clinical prediction. The same rule generated the
48
+ test anchors, so this is not leakage, but it does bound how the results should
49
+ be interpreted.
50
+
51
+ ## Leakage controls
52
+
53
+ `DEATHDATE`, `HEALTHCARE_EXPENSES` and `HEALTHCARE_COVERAGE` are refused at
54
+ load time. `STOP`-derived durations are excluded: the organisers blanked
55
+ post-anchor stops in the test split, so such a feature would both leak and
56
+ shift. Every fitted statistic — vocabulary, quantile edges, scalers — is fitted
57
+ on training patients only. 63 automated checks cover this, including a grep
58
+ test that no module outside the time utility parses a timestamp.
59
+
60
+ ## Limitations
61
+
62
+ Synthea is not real EHR data. A discriminator separates it from MIMIC at
63
+ AUC 0.999, and across 19 health datasets the winning classifier agreed between
64
+ real- and synthetic-trained models in only 21–26% of cases. **Whichever model
65
+ wins here, that ranking should not be assumed to transfer.**
66
+
67
+ ## Reproducing
68
+
69
+ ```bash
70
+ git clone https://github.com/Sallamsaka/M31-Coding-Test && pip install -r requirements.txt
71
+ ./run_all.ps1
72
+ ```