hERG XGBoost β Phase 5 (locked-pool finals)
XGBoost models for hERG (Kv11.1) cardiotoxicity prediction: a blocker classifier and a pIC50 regressor, each in a random-split and a scaffold-split protocol variant. The scaffold-protocol models are the headline versions. These are the Phase 5 "Approach A" finals of a comprehensive hERG rebuild and benchmark, trained on comprehensive molecular features (Morgan/ECFP/FCFP fingerprints and counts, MACCS keys, 217 RDKit descriptors, physicochemical, hERG-targeted and 2D-pharmacophore blocks β the CDK/PaDEL block was dropped).
Files
| File | Contents |
|---|---|
models/*.pkl |
The four model pickles, exactly as saved by the training run. Each is a dict: {model, task, blocks, params, preps, protocol, trained_on} β the fitted preprocessing states (preps) travel with the weights. |
config.json |
Feature blocks, XGBoost hyperparameters, preprocessing specification, locked thresholds, training-pool description. |
feature_extraction.py |
End-to-end SMILES β feature blocks (RDKit only), identical to the training pipeline. |
predict_herg.py |
Standalone inference: SMILES/CSV β blocker probability or predicted pIC50. |
validation_results.json |
Full honest external metrics (machine-readable). |
regulatory_inference_sample.csv |
Worked inference sample over ALL 7 regulatory-failure compounds: input SMILES + expected outputs (see below). |
requirements.txt |
Pinned dependencies used for verification. |
Quick start
pip install -r requirements.txt
python predict_herg.py --model "models/xgboostA_cls_base+physchem_extra+morgan_counts+ecfp6+fcfp4+herg+pharm+morgan1024+morgan1024_counts_scaffold.pkl" \
"Fc1ccc(C(OCCN2CCN(CCCc3ccccc3)CC2)c2ccc(F)cc2)cc1"
# -> {"prob_blocker": "0.9902", "call_10uM": 1} (vanoxerine)
Classification call: blocker if probability β₯ 0.5 at the locked 10 Β΅M cutoff (pIC50 β₯ 5.0). Regression output is predicted pIC50; the derived 10 Β΅M call is pIC50 β₯ 5.0. A secondary 1 Β΅M cutoff is reported in the project's evaluation but is not the headline threshold.
Training data and decontamination
Trained on curated public hERG data (ChEMBL + BindingDB + PubChem), canonical-deduplicated: 12,702 compounds (classification pool) and 8,657 (regression pool; functional IC50 only β Ki was never pooled into the primary regression). Before training, 1,738 unique compounds covering every evaluation set (internal test splits, hERG-70, hERG-60, the new-compounds slice, the 36-drug FDA panel and the regulatory-failure set) were removed by InChIKey connectivity block; the post-lock overlap was verified to be exactly 0. Headline finals are refits on the protocol train+val keys, with all preprocessing fitted on those keys only.
Validation results (honest, locked-out sets)
Classification (10 Β΅M cutoff):
| Model | Internal test ACC / MCC | hERG-70 ACC / MCC | New compounds ACC / MCC (SEN) | FDA panel errors /36 |
|---|---|---|---|---|
| XGB cls random | 0.816 / 0.623 | 0.760 / 0.537 | 0.515 / 0.202 (0.331) | 4 (blocker 2, clean 2, withdrawn 0) |
| XGB cls scaffold β | 0.783 / 0.549 | 0.751 / 0.519 | 0.505 / 0.168 (0.331) | 2 (blocker 1, clean 1, withdrawn 0) |
Regression (pIC50):
| Model | Internal RMSE / MAE | hERG-70 RMSE / MAE | New compounds RMSE / RΒ² | FDA MAE |
|---|---|---|---|---|
| XGB reg random | 0.540 / 0.379 | 0.816 / 0.581 | 0.807 / β0.052 | 0.811 |
| XGB reg scaffold β | 0.640 / 0.464 | 0.812 / 0.588 | 0.821 / β0.091 | 0.827 |
Inference sample β regulatory-failure set
regulatory_inference_sample.csv is the worked example for the inference
script: all 7 compounds of the regulatory-failure set with their SMILES
(input) and the models' actual outputs β blocker probabilities for both
classifier protocols, classifier calls, predicted pIC50 and error for both
regressors, and correct yes/no per compound. Feed its smiles column
through predict_herg.py to reproduce the prediction columns. Headline on
this set: Tier-1 classifiers 2/5 correct on the IC50-labeled compounds
(vanoxerine β, sematilide β; almokalant, AZD1305, tedisamil under-called).
d-Sotalol is a by-design case β it has no functional IC50 (Ki only), its
expected call rests on SWORD/proarrhythmia history, and both models call it
non-blocker, consistent with its weak measured potency; it is excluded from
the IC50-labeled count. Tecastemizole (Tier 2) sits in the training pools
and is never counted.
Regulatory-failure set (development-stage hERG/QT failures, Tier 1): classifiers score 2/5 on the IC50-labeled compounds (vanoxerine β, sematilide β; almokalant β, AZD1305 β, tedisamil β β all under-called); 2/6 counting d-sotalol's history-based call (no functional IC50 exists for it). Regression-derived calls get 5/5, though almokalant's potency is underestimated by ~2.3 log units.
Known limitations β read before use
- Novel-compound sensitivity is ~0.33. On 967 compounds absent from UnihERG_DB, both classifiers catch only about a third of true blockers. These models under-call moderate blockers on novel chemistry (the regulatory-failure misses above are the same failure mode). Do not use as a sole hERG safety gate; pair with a high-sensitivity model and treat "non-blocker" calls on novel scaffolds as weak evidence.
- Regression RΒ² is negative on the new-compounds slice β on truly novel chemistry the regressor is near the experimental noise floor of public hERG data (Ο β 0.68 log units).
- Probabilities from a fresh-SMILES featurization can differ in the second
decimal from cache-based scoring if the RDKit version differs from
requirements.txt; calls are unaffected in verification.
Provenance and status
Part of the hERG toxicity prediction rebuild (goal: comprehensive rebuild and benchmark, 2026-11). A deployment refit on all labeled data (with the benchmark sets folded back in) will supersede these locked-pool weights later; scores of that future refit on the absorbed sets must never be reported as validation. This repo is a model release only β no app/Space.
- Downloads last month
- 51