| --- |
| language: |
| - en |
| license: mit |
| tags: |
| - drug-discovery |
| - binding-affinity |
| - protein-ligand |
| - graph-neural-network |
| - esm2 |
| - drug-repurposing |
| - multimodal |
| - transfer-learning |
| datasets: |
| - pdbbind-v2020 |
| metrics: |
| - rmse |
| - pearsonr |
| pipeline_tag: other |
| --- |
| |
| # DeepPharm: Multi-Modal Transfer Learning for Drug-Target Affinity Prediction |
|
|
| ## Model Description |
|
|
| **DeepPharm** is a multi-modal deep learning framework for predicting protein–ligand binding affinity ($pK$). It combines: |
|
|
| - **GATv2** molecular graph encoder (3 layers, 4 heads) |
| - **ECFP4** fingerprint MLP encoder (2048→128) |
| - **Gated Fusion** mechanism for adaptive ligand representation |
| - **ESM-2** protein language model (150M params, fine-tuned) |
| - **Stacked Cross-Attention** (2 layers, 4 heads) for drug-protein interaction |
| - **Residual Prediction Head** with SiLU activation |
|
|
| ### Two Modes of Operation |
|
|
| | Mode | Task | Input | Output | |
| |------|------|-------|--------| |
| | **Mode A** | Supervised affinity prediction | Drug SMILES + Protein sequence | pK value | |
| | **Mode B** | Weakly supervised drug repurposing | Drug + Disease signature | Ranked candidates | |
|
|
| ## Performance |
|
|
| ### Systematic Ablation (PDBbind v2020, $N_{test}=3{,}775$) |
| |
| | Config | RMSE ↓ | Pearson ↑ | Spearman ↑ | |
| |--------|--------|-----------|------------| |
| | V1 Baseline (ESM-35M) | 1.266 | 0.743 | 0.743 | |
| | V2 Architecture | 1.258 | 0.748 | 0.746 | |
| | V2 + CosineWR | 1.244 | 0.753 | 0.750 | |
| | **V2 + ESM-150M (Best)** | **1.229** | **0.762** | **0.760** | |
| | V2 + EMA | 1.247 | 0.753 | 0.753 | |
| |
| ### Five-Seed Ensemble (Best Configuration) |
| |
| | Metric | Mean ± Std | |
| |--------|-----------| |
| | RMSE | 1.246 ± 0.005 | |
| | Pearson r | 0.751 ± 0.002 | |
| | Spearman ρ | 0.750 ± 0.002 | |
| |
| CV < 0.4% confirms high reproducibility. |
| |
| ### Baselines (all re-implemented on same split) |
| |
| | Model | RMSE ↓ | Pearson ↑ | |
| |-------|--------|-----------| |
| | DeepDTA (CNN) | 1.48 | 0.61 | |
| | GraphDTA (GCN) | 1.39 | 0.67 | |
| | MolCLR* | 1.30 | 0.74 | |
| | DrugBAN | 1.28 | 0.76 | |
| | **DeepPharm V2** | **1.23** | **0.76** | |
| |
| ## Intended Use |
| |
| - High-throughput virtual screening of drug candidates |
| - Binding affinity prediction for drug-target pairs |
| - Hypothesis generation for drug repurposing in orphan diseases |
| - Research and academic purposes |
| |
| ## Limitations |
| |
| - 2D topological encoder; cannot distinguish stereoisomers |
| - Trained on PDBbind v2020, which overrepresents kinases |
| - Mode B uses drug priors (guilt-by-association), not zero-shot inference |
| - Predictions require experimental validation |
| |
| ## Training Details |
| |
| - **Dataset:** PDBbind v2020 General Set (15,100 train / 3,775 test, seed=42) |
| - **Hardware:** 1× NVIDIA H100 80 GB |
| - **Optimizer:** AdamW (backbone LR: 5e-6, head LR: 8e-4) |
| - **Scheduler:** CosineAnnealing with Warm Restarts ($T_0$=10, $T_{mult}$=2) |
| - **Loss:** MSE + 0.3·RankingLoss + 0.2·HuberLoss |
| - **Training time:** ~11 min/epoch (ESM-2 150M), best checkpoint at epoch 18 |
| |
| ## Available Checkpoints |
| |
| | File | Description | RMSE | |
| |------|-------------|------| |
| | `best_v2_esm150m.pt` | Best V2 model (ESM-2 150M) | 1.229 | |
| | `best_v1_esm35m.pt` | V1 Baseline (ESM-2 35M) | 1.266 | |
| |
| ## How to Use |
| |
| ```python |
| from huggingface_hub import hf_hub_download |
|
|
| # Download the best model |
| path = hf_hub_download("chamoso/DeepPharm", "best_v2_esm150m.pt") |
|
|
| # Load in PyTorch |
| import torch |
| checkpoint = torch.load(path, map_location="cpu") |
| ``` |
| |
| For full inference with data preprocessing: |
| |
| ```bash |
| git clone https://github.com/chamoso/DeepPharm.git |
| cd DeepPharm |
| python scripts/predict.py \ |
| --checkpoint weights/best_v2_esm150m.pt \ |
| --smiles "CC(=O)Oc1ccccc1C(=O)O" \ |
| --sequence "MKTAYIAKQRQISFVKSHFSRQLE..." |
| ``` |
| |
| ## Links |
| |
| - **GitHub:** [chamoso/DeepPharm](https://github.com/chamoso/DeepPharm) |
| - **Live Demo:** [HuggingFace Spaces](https://huggingface.co/spaces/chamoso/DeepPharm) |
| |
| ## Citation |
| |
| *Preprint coming soon.* |
| |
| ## License |
| |
| MIT License |
| |
