Delower commited on
Commit
16fb189
·
verified ·
1 Parent(s): 26b6659

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +102 -0
README.md ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ tags:
4
+ - herg
5
+ - cardiotoxicity
6
+ - xgboost
7
+ - cheminformatics
8
+ - drug-discovery
9
+ ---
10
+
11
+ # hERG XGBoost — Phase 5 (locked-pool finals)
12
+
13
+ XGBoost models for **hERG (Kv11.1) cardiotoxicity prediction**: a blocker
14
+ classifier and a pIC50 regressor, each in a random-split and a scaffold-split
15
+ protocol variant. The scaffold-protocol models are the headline versions.
16
+ These are the Phase 5 "Approach A" finals of a comprehensive hERG rebuild and
17
+ benchmark, trained on comprehensive molecular features (Morgan/ECFP/FCFP
18
+ fingerprints and counts, MACCS keys, 217 RDKit descriptors, physicochemical,
19
+ hERG-targeted and 2D-pharmacophore blocks — the CDK/PaDEL block was dropped).
20
+
21
+ ## Files
22
+
23
+ | File | Contents |
24
+ |---|---|
25
+ | `models/*.pkl` | The four model pickles, exactly as saved by the training run. Each is a dict: `{model, task, blocks, params, preps, protocol, trained_on}` — the fitted preprocessing states (`preps`) travel with the weights. |
26
+ | `config.json` | Feature blocks, XGBoost hyperparameters, preprocessing specification, locked thresholds, training-pool description. |
27
+ | `feature_extraction.py` | End-to-end SMILES → feature blocks (RDKit only), identical to the training pipeline. |
28
+ | `predict_herg.py` | Standalone inference: SMILES/CSV → blocker probability or predicted pIC50. |
29
+ | `validation_results.json` | Full honest external metrics (machine-readable). |
30
+ | `requirements.txt` | Pinned dependencies used for verification. |
31
+
32
+ ## Quick start
33
+
34
+ ```bash
35
+ pip install -r requirements.txt
36
+ python predict_herg.py --model "models/xgboostA_cls_base+physchem_extra+morgan_counts+ecfp6+fcfp4+herg+pharm+morgan1024+morgan1024_counts_scaffold.pkl" \
37
+ "Fc1ccc(C(OCCN2CCN(CCCc3ccccc3)CC2)c2ccc(F)cc2)cc1"
38
+ # -> {"prob_blocker": "0.9902", "call_10uM": 1} (vanoxerine)
39
+ ```
40
+
41
+ Classification call: blocker if probability ≥ 0.5 at the locked **10 µM**
42
+ cutoff (pIC50 ≥ 5.0). Regression output is predicted pIC50; the derived
43
+ 10 µM call is pIC50 ≥ 5.0. A secondary 1 µM cutoff is reported in the
44
+ project's evaluation but is not the headline threshold.
45
+
46
+ ## Training data and decontamination
47
+
48
+ Trained on curated public hERG data (ChEMBL + BindingDB + PubChem),
49
+ canonical-deduplicated: **12,702 compounds** (classification pool) and
50
+ **8,657** (regression pool; functional IC50 only — Ki was never pooled into
51
+ the primary regression). Before training, **1,738 unique compounds covering
52
+ every evaluation set** (internal test splits, hERG-70, hERG-60, the
53
+ new-compounds slice, the 36-drug FDA panel and the regulatory-failure set)
54
+ were removed by InChIKey connectivity block; the post-lock overlap was
55
+ verified to be exactly 0. Headline finals are refits on the protocol
56
+ train+val keys, with all preprocessing fitted on those keys only.
57
+
58
+ ## Validation results (honest, locked-out sets)
59
+
60
+ Classification (10 µM cutoff):
61
+
62
+ | Model | Internal test ACC / MCC | hERG-70 ACC / MCC | New compounds ACC / MCC (SEN) | FDA panel errors /36 |
63
+ |---|---|---|---|---|
64
+ | XGB cls random | 0.816 / 0.623 | 0.760 / 0.537 | 0.515 / 0.202 (0.331) | 4 (blocker 2, clean 2, withdrawn 0) |
65
+ | XGB cls scaffold ⭐ | 0.783 / 0.549 | 0.751 / 0.519 | 0.505 / 0.168 (0.331) | **2** (blocker 1, clean 1, withdrawn 0) |
66
+
67
+ Regression (pIC50):
68
+
69
+ | Model | Internal RMSE / MAE | hERG-70 RMSE / MAE | New compounds RMSE / R² | FDA MAE |
70
+ |---|---|---|---|---|
71
+ | XGB reg random | 0.540 / 0.379 | 0.816 / 0.581 | 0.807 / −0.052 | 0.811 |
72
+ | XGB reg scaffold ⭐ | 0.640 / 0.464 | 0.812 / 0.588 | 0.821 / −0.091 | 0.827 |
73
+
74
+ Regulatory-failure set (development-stage hERG/QT failures, Tier 1):
75
+ classifiers score **2/5** on the IC50-labeled compounds (vanoxerine ✓,
76
+ sematilide ✓; almokalant ✗, AZD1305 ✗, tedisamil ✗ — all under-called);
77
+ 2/6 counting d-sotalol's history-based call (no functional IC50 exists for
78
+ it). Regression-derived calls get 5/5, though almokalant's potency is
79
+ underestimated by ~2.3 log units.
80
+
81
+ ## Known limitations — read before use
82
+
83
+ - **Novel-compound sensitivity is ~0.33.** On 967 compounds absent from
84
+ UnihERG_DB, both classifiers catch only about a third of true blockers.
85
+ These models under-call moderate blockers on novel chemistry (the
86
+ regulatory-failure misses above are the same failure mode). **Do not use
87
+ as a sole hERG safety gate**; pair with a high-sensitivity model and treat
88
+ "non-blocker" calls on novel scaffolds as weak evidence.
89
+ - Regression R² is negative on the new-compounds slice — on truly novel
90
+ chemistry the regressor is near the experimental noise floor of public
91
+ hERG data (σ ≈ 0.68 log units).
92
+ - Probabilities from a fresh-SMILES featurization can differ in the second
93
+ decimal from cache-based scoring if the RDKit version differs from
94
+ `requirements.txt`; calls are unaffected in verification.
95
+
96
+ ## Provenance and status
97
+
98
+ Part of the hERG toxicity prediction rebuild (goal: comprehensive rebuild
99
+ and benchmark, 2026-11). A **deployment refit on all labeled data** (with
100
+ the benchmark sets folded back in) will supersede these locked-pool weights
101
+ later; scores of that future refit on the absorbed sets must never be
102
+ reported as validation. This repo is a model release only — no app/Space.