Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- herg
|
| 5 |
+
- cardiotoxicity
|
| 6 |
+
- xgboost
|
| 7 |
+
- cheminformatics
|
| 8 |
+
- drug-discovery
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# hERG XGBoost — Phase 5 (locked-pool finals)
|
| 12 |
+
|
| 13 |
+
XGBoost models for **hERG (Kv11.1) cardiotoxicity prediction**: a blocker
|
| 14 |
+
classifier and a pIC50 regressor, each in a random-split and a scaffold-split
|
| 15 |
+
protocol variant. The scaffold-protocol models are the headline versions.
|
| 16 |
+
These are the Phase 5 "Approach A" finals of a comprehensive hERG rebuild and
|
| 17 |
+
benchmark, trained on comprehensive molecular features (Morgan/ECFP/FCFP
|
| 18 |
+
fingerprints and counts, MACCS keys, 217 RDKit descriptors, physicochemical,
|
| 19 |
+
hERG-targeted and 2D-pharmacophore blocks — the CDK/PaDEL block was dropped).
|
| 20 |
+
|
| 21 |
+
## Files
|
| 22 |
+
|
| 23 |
+
| File | Contents |
|
| 24 |
+
|---|---|
|
| 25 |
+
| `models/*.pkl` | The four model pickles, exactly as saved by the training run. Each is a dict: `{model, task, blocks, params, preps, protocol, trained_on}` — the fitted preprocessing states (`preps`) travel with the weights. |
|
| 26 |
+
| `config.json` | Feature blocks, XGBoost hyperparameters, preprocessing specification, locked thresholds, training-pool description. |
|
| 27 |
+
| `feature_extraction.py` | End-to-end SMILES → feature blocks (RDKit only), identical to the training pipeline. |
|
| 28 |
+
| `predict_herg.py` | Standalone inference: SMILES/CSV → blocker probability or predicted pIC50. |
|
| 29 |
+
| `validation_results.json` | Full honest external metrics (machine-readable). |
|
| 30 |
+
| `requirements.txt` | Pinned dependencies used for verification. |
|
| 31 |
+
|
| 32 |
+
## Quick start
|
| 33 |
+
|
| 34 |
+
```bash
|
| 35 |
+
pip install -r requirements.txt
|
| 36 |
+
python predict_herg.py --model "models/xgboostA_cls_base+physchem_extra+morgan_counts+ecfp6+fcfp4+herg+pharm+morgan1024+morgan1024_counts_scaffold.pkl" \
|
| 37 |
+
"Fc1ccc(C(OCCN2CCN(CCCc3ccccc3)CC2)c2ccc(F)cc2)cc1"
|
| 38 |
+
# -> {"prob_blocker": "0.9902", "call_10uM": 1} (vanoxerine)
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
Classification call: blocker if probability ≥ 0.5 at the locked **10 µM**
|
| 42 |
+
cutoff (pIC50 ≥ 5.0). Regression output is predicted pIC50; the derived
|
| 43 |
+
10 µM call is pIC50 ≥ 5.0. A secondary 1 µM cutoff is reported in the
|
| 44 |
+
project's evaluation but is not the headline threshold.
|
| 45 |
+
|
| 46 |
+
## Training data and decontamination
|
| 47 |
+
|
| 48 |
+
Trained on curated public hERG data (ChEMBL + BindingDB + PubChem),
|
| 49 |
+
canonical-deduplicated: **12,702 compounds** (classification pool) and
|
| 50 |
+
**8,657** (regression pool; functional IC50 only — Ki was never pooled into
|
| 51 |
+
the primary regression). Before training, **1,738 unique compounds covering
|
| 52 |
+
every evaluation set** (internal test splits, hERG-70, hERG-60, the
|
| 53 |
+
new-compounds slice, the 36-drug FDA panel and the regulatory-failure set)
|
| 54 |
+
were removed by InChIKey connectivity block; the post-lock overlap was
|
| 55 |
+
verified to be exactly 0. Headline finals are refits on the protocol
|
| 56 |
+
train+val keys, with all preprocessing fitted on those keys only.
|
| 57 |
+
|
| 58 |
+
## Validation results (honest, locked-out sets)
|
| 59 |
+
|
| 60 |
+
Classification (10 µM cutoff):
|
| 61 |
+
|
| 62 |
+
| Model | Internal test ACC / MCC | hERG-70 ACC / MCC | New compounds ACC / MCC (SEN) | FDA panel errors /36 |
|
| 63 |
+
|---|---|---|---|---|
|
| 64 |
+
| XGB cls random | 0.816 / 0.623 | 0.760 / 0.537 | 0.515 / 0.202 (0.331) | 4 (blocker 2, clean 2, withdrawn 0) |
|
| 65 |
+
| XGB cls scaffold ⭐ | 0.783 / 0.549 | 0.751 / 0.519 | 0.505 / 0.168 (0.331) | **2** (blocker 1, clean 1, withdrawn 0) |
|
| 66 |
+
|
| 67 |
+
Regression (pIC50):
|
| 68 |
+
|
| 69 |
+
| Model | Internal RMSE / MAE | hERG-70 RMSE / MAE | New compounds RMSE / R² | FDA MAE |
|
| 70 |
+
|---|---|---|---|---|
|
| 71 |
+
| XGB reg random | 0.540 / 0.379 | 0.816 / 0.581 | 0.807 / −0.052 | 0.811 |
|
| 72 |
+
| XGB reg scaffold ⭐ | 0.640 / 0.464 | 0.812 / 0.588 | 0.821 / −0.091 | 0.827 |
|
| 73 |
+
|
| 74 |
+
Regulatory-failure set (development-stage hERG/QT failures, Tier 1):
|
| 75 |
+
classifiers score **2/5** on the IC50-labeled compounds (vanoxerine ✓,
|
| 76 |
+
sematilide ✓; almokalant ✗, AZD1305 ✗, tedisamil ✗ — all under-called);
|
| 77 |
+
2/6 counting d-sotalol's history-based call (no functional IC50 exists for
|
| 78 |
+
it). Regression-derived calls get 5/5, though almokalant's potency is
|
| 79 |
+
underestimated by ~2.3 log units.
|
| 80 |
+
|
| 81 |
+
## Known limitations — read before use
|
| 82 |
+
|
| 83 |
+
- **Novel-compound sensitivity is ~0.33.** On 967 compounds absent from
|
| 84 |
+
UnihERG_DB, both classifiers catch only about a third of true blockers.
|
| 85 |
+
These models under-call moderate blockers on novel chemistry (the
|
| 86 |
+
regulatory-failure misses above are the same failure mode). **Do not use
|
| 87 |
+
as a sole hERG safety gate**; pair with a high-sensitivity model and treat
|
| 88 |
+
"non-blocker" calls on novel scaffolds as weak evidence.
|
| 89 |
+
- Regression R² is negative on the new-compounds slice — on truly novel
|
| 90 |
+
chemistry the regressor is near the experimental noise floor of public
|
| 91 |
+
hERG data (σ ≈ 0.68 log units).
|
| 92 |
+
- Probabilities from a fresh-SMILES featurization can differ in the second
|
| 93 |
+
decimal from cache-based scoring if the RDKit version differs from
|
| 94 |
+
`requirements.txt`; calls are unaffected in verification.
|
| 95 |
+
|
| 96 |
+
## Provenance and status
|
| 97 |
+
|
| 98 |
+
Part of the hERG toxicity prediction rebuild (goal: comprehensive rebuild
|
| 99 |
+
and benchmark, 2026-11). A **deployment refit on all labeled data** (with
|
| 100 |
+
the benchmark sets folded back in) will supersede these locked-pool weights
|
| 101 |
+
later; scores of that future refit on the absorbed sets must never be
|
| 102 |
+
reported as validation. This repo is a model release only — no app/Space.
|