ynuozhang commited on
Commit ·
07fc6ea
1
Parent(s): 60cd1e8
update readme
Browse files
README.md
CHANGED
|
@@ -518,48 +518,48 @@ print(predictor.predict_property("nf", "smiles", smiles, uncertainty=True
|
|
| 518 |
You can also find the same description in the paper or in the PeptiVerse app `Documentation` tab.
|
| 519 |
|
| 520 |
---
|
| 521 |
-
###
|
| 522 |
-
50% of read blood cells being lysed at x ug/ml concetration (HC50). If HC50 < 100uM, considered as hemolytic, otherwise non-hemolytic, resulting in a binary 0/1 dataset. The predicted probability should therefore be interpreted as a risk indicator, not an exact concentration estimate.
|
| 523 |
-
**Output interpretation:**
|
| 524 |
- Score close to 1.0 = high probability of red blood cell membrane disruption
|
| 525 |
- Score close to 0.0 = non-hemolytic
|
| 526 |
---
|
| 527 |
|
| 528 |
-
###
|
| 529 |
-
Outputs a probability (0–1) that a peptide remains soluble in aqueous conditions.
|
| 530 |
-
**Output interpretation:**
|
| 531 |
- Score close to 1.0 = highly soluble
|
| 532 |
- Score close to 0.0 = poorly soluble
|
| 533 |
|
| 534 |
---
|
| 535 |
|
| 536 |
-
###
|
| 537 |
-
Higher scores indicate stronger non-fouling behavior, desirable for circulation and surface-exposed applications.
|
| 538 |
-
**Output interpretation:**
|
| 539 |
- Score close to 1.0 = non-fouling
|
| 540 |
- Score close to 0.0 = fouling
|
| 541 |
---
|
| 542 |
|
| 543 |
-
###
|
| 544 |
-
Predicts membrane permeability on a log P scale.
|
| 545 |
-
**Output interpretation:**
|
| 546 |
- Higher values = more permeable (>-6.0)
|
| 547 |
- For penetrance predictions, it is a classification prediction, so within the [0, 1] range, closer to 1 indicates more permeable.
|
| 548 |
---
|
| 549 |
|
| 550 |
-
###
|
| 551 |
-
**Interpretation:** Predicted values reflect relative peptide stability for the unit in hours. Higher scores indicate longer persistence in serum, while lower scores suggest faster degradation.
|
| 552 |
|
| 553 |
---
|
| 554 |
|
| 555 |
-
###
|
| 556 |
-
**Interpretation:** Outputs a probability (0–1) that a peptide exhibits toxic effects. Higher scores indicate increased toxicity risk.
|
| 557 |
|
| 558 |
---
|
| 559 |
|
| 560 |
-
###
|
| 561 |
|
| 562 |
-
Predicts peptide-protein binding affinity. Requires both peptide and target protein sequence.
|
| 563 |
|
| 564 |
**Interpretation:**<br>
|
| 565 |
- Scores ≥ 9 correspond to tight binders (K ≤ 10⁻⁹ M, nanomolar to picomolar range)<br>
|
|
@@ -569,15 +569,15 @@ Predicts peptide-protein binding affinity. Requires both peptide and target prot
|
|
| 569 |
|
| 570 |
---
|
| 571 |
|
| 572 |
-
### Uncertainty Interpretation
|
| 573 |
-
#### Entropy (classifiers)
|
| 574 |
|
| 575 |
-
Binary predictive entropy of the output probability $\bar{p}$:
|
| 576 |
|
| 577 |
-
$$\mathcal{H} = -\bar{p}\log\bar{p} - (1 - \bar{p})\log(1 - \bar{p})$$
|
| 578 |
|
| 579 |
-
- For **DNN classifiers**: $\bar{p}$ is the mean probability across 5 independently seeded models (deep ensemble). High entropy reflects both epistemic uncertainty (seed disagreement) and aleatoric uncertainty (collectively diffuse predictions).
|
| 580 |
-
- For **XGBoost / SVM / ElasticNet classifiers**: $\bar{p}$ is the single model's output probability (or sigmoid of decision function for ElasticNet). Entropy reflects output confidence of a single model only.
|
| 581 |
|
| 582 |
| Range | Interpretation |
|
| 583 |
|---|---|
|
|
@@ -589,33 +589,33 @@ $$\mathcal{H} = -\bar{p}\log\bar{p} - (1 - \bar{p})\log(1 - \bar{p})$$
|
|
| 589 |
|
| 590 |
---
|
| 591 |
|
| 592 |
-
#### Adaptive Conformal Prediction Interval (regressors)
|
| 593 |
|
| 594 |
-
Returned as a tuple `(lo, hi)` with 90% marginal coverage guarantee.
|
| 595 |
|
| 596 |
-
We implement the **residual normalised conformity score** following [Lei et al. (2018)](https://doi.org/10.1080/01621459.2017.1307116) and [Cordier et al. (2023) / MAPIE](https://proceedings.mlr.press/v204/cordier23a.html). An auxiliary XGBoost model $\hat{\sigma}(\mathbf{x})$ is trained on held-out embeddings and absolute residuals $|y_i - \hat{y}_i|$. At inference:
|
| 597 |
|
| 598 |
$$[\hat{y}(\mathbf{x}) - q \cdot \hat{\sigma}(\mathbf{x}),\ \hat{y}(\mathbf{x}) + q \cdot \hat{\sigma}(\mathbf{x})]$$
|
| 599 |
|
| 600 |
-
where $q$ is the $\lceil(n+1)(1-\alpha)\rceil / n$ quantile of the normalized scores $s_i = |y_i - \hat{y}_i| / \hat{\sigma}(\mathbf{x}_i)$.
|
| 601 |
|
| 602 |
-
- **Interval width varies per input** -- molecules more dissimilar to training data tend to receive wider intervals
|
| 603 |
-
- **Coverage guarantee**: on exchangeable data, $P(y \in [\hat{y} - q\hat{\sigma},\ \hat{y} + q\hat{\sigma}]) \geq 0.90$
|
| 604 |
-
- **The guarantee is marginal**, not conditional, as an unusually narrow interval on an out-of-distribution molecule does not guarantee correctness
|
| 605 |
-
- **Full access**:
|
| 606 |
|
| 607 |
---
|
| 608 |
|
| 609 |
-
#### Generating a MAPIE Bundle for a New Model
|
| 610 |
|
| 611 |
-
To enable conformal uncertainty for a newly trained regression model:
|
| 612 |
|
| 613 |
```bash
|
| 614 |
# Fit adaptive conformal bundle from val_predictions.csv
|
| 615 |
python fit_mapie_adaptive.py --root training_classifiers --prop <property_name>
|
| 616 |
```
|
| 617 |
|
| 618 |
-
The script reads `sequence`/`smiles` and `y_pred`/`y_true` columns from the CSV, recomputes embeddings, fits the XGBoost $\hat{\sigma}$ model, and saves `mapie_calibration.joblib` into the model directory. The bundle is automatically detected and loaded by `PeptiVersePredictor` on next
|
| 619 |
|
| 620 |
|
| 621 |
|
|
|
|
| 518 |
You can also find the same description in the paper or in the PeptiVerse app `Documentation` tab.
|
| 519 |
|
| 520 |
---
|
| 521 |
+
### 🩸 Hemolysis Prediction<br>
|
| 522 |
+
50% of read blood cells being lysed at x ug/ml concetration (HC50). If HC50 < 100uM, considered as hemolytic, otherwise non-hemolytic, resulting in a binary 0/1 dataset. The predicted probability should therefore be interpreted as a risk indicator, not an exact concentration estimate.<br>
|
| 523 |
+
**Output interpretation:**<br>
|
| 524 |
- Score close to 1.0 = high probability of red blood cell membrane disruption
|
| 525 |
- Score close to 0.0 = non-hemolytic
|
| 526 |
---
|
| 527 |
|
| 528 |
+
### 💧 Solubility Prediction<br>
|
| 529 |
+
Outputs a probability (0–1) that a peptide remains soluble in aqueous conditions.<br>
|
| 530 |
+
**Output interpretation:**<br>
|
| 531 |
- Score close to 1.0 = highly soluble
|
| 532 |
- Score close to 0.0 = poorly soluble
|
| 533 |
|
| 534 |
---
|
| 535 |
|
| 536 |
+
### 👯 Non-Fouling Prediction<br>
|
| 537 |
+
Higher scores indicate stronger non-fouling behavior, desirable for circulation and surface-exposed applications.<br>
|
| 538 |
+
**Output interpretation:**<br>
|
| 539 |
- Score close to 1.0 = non-fouling
|
| 540 |
- Score close to 0.0 = fouling
|
| 541 |
---
|
| 542 |
|
| 543 |
+
### 🪣 Permeability Prediction<br>
|
| 544 |
+
Predicts membrane permeability on a log P scale.<br>
|
| 545 |
+
**Output interpretation:**<br>
|
| 546 |
- Higher values = more permeable (>-6.0)
|
| 547 |
- For penetrance predictions, it is a classification prediction, so within the [0, 1] range, closer to 1 indicates more permeable.
|
| 548 |
---
|
| 549 |
|
| 550 |
+
### ⏱️ Half-Life Prediction<br>
|
| 551 |
+
**Interpretation:** Predicted values reflect relative peptide stability for the unit in hours. Higher scores indicate longer persistence in serum, while lower scores suggest faster degradation.<br>
|
| 552 |
|
| 553 |
---
|
| 554 |
|
| 555 |
+
### ☠️ Toxicity Prediction<br>
|
| 556 |
+
**Interpretation:** Outputs a probability (0–1) that a peptide exhibits toxic effects. Higher scores indicate increased toxicity risk.<br>
|
| 557 |
|
| 558 |
---
|
| 559 |
|
| 560 |
+
### 🔗 Binding Affinity Prediction <br>
|
| 561 |
|
| 562 |
+
Predicts peptide-protein binding affinity. Requires both peptide and target protein sequence.<br>
|
| 563 |
|
| 564 |
**Interpretation:**<br>
|
| 565 |
- Scores ≥ 9 correspond to tight binders (K ≤ 10⁻⁹ M, nanomolar to picomolar range)<br>
|
|
|
|
| 569 |
|
| 570 |
---
|
| 571 |
|
| 572 |
+
### Uncertainty Interpretation <br>
|
| 573 |
+
#### Entropy (classifiers)<br>
|
| 574 |
|
| 575 |
+
Binary predictive entropy of the output probability $\bar{p}$:<br>
|
| 576 |
|
| 577 |
+
$$\mathcal{H} = -\bar{p}\log\bar{p} - (1 - \bar{p})\log(1 - \bar{p})$$<br>
|
| 578 |
|
| 579 |
+
- For **DNN classifiers**: $\bar{p}$ is the mean probability across 5 independently seeded models (deep ensemble). High entropy reflects both epistemic uncertainty (seed disagreement) and aleatoric uncertainty (collectively diffuse predictions).<br>
|
| 580 |
+
- For **XGBoost / SVM / ElasticNet classifiers**: $\bar{p}$ is the single model's output probability (or sigmoid of decision function for ElasticNet). Entropy reflects output confidence of a single model only.<br>
|
| 581 |
|
| 582 |
| Range | Interpretation |
|
| 583 |
|---|---|
|
|
|
|
| 589 |
|
| 590 |
---
|
| 591 |
|
| 592 |
+
#### Adaptive Conformal Prediction Interval (regressors)<br>
|
| 593 |
|
| 594 |
+
Returned as a tuple `(lo, hi)` with 90% marginal coverage guarantee.<br>
|
| 595 |
|
| 596 |
+
We implement the **residual normalised conformity score** following [Lei et al. (2018)](https://doi.org/10.1080/01621459.2017.1307116) and [Cordier et al. (2023) / MAPIE](https://proceedings.mlr.press/v204/cordier23a.html). An auxiliary XGBoost model $\hat{\sigma}(\mathbf{x})$ is trained on held-out embeddings and absolute residuals $|y_i - \hat{y}_i|$. At inference:<br>
|
| 597 |
|
| 598 |
$$[\hat{y}(\mathbf{x}) - q \cdot \hat{\sigma}(\mathbf{x}),\ \hat{y}(\mathbf{x}) + q \cdot \hat{\sigma}(\mathbf{x})]$$
|
| 599 |
|
| 600 |
+
where $q$ is the $\lceil(n+1)(1-\alpha)\rceil / n$ quantile of the normalized scores $s_i = |y_i - \hat{y}_i| / \hat{\sigma}(\mathbf{x}_i)$.<br>
|
| 601 |
|
| 602 |
+
- **Interval width varies per input** -- molecules more dissimilar to training data tend to receive wider intervals<br>
|
| 603 |
+
- **Coverage guarantee**: on exchangeable data, $P(y \in [\hat{y} - q\hat{\sigma},\ \hat{y} + q\hat{\sigma}]) \geq 0.90$<br>
|
| 604 |
+
- **The guarantee is marginal**, not conditional, as an unusually narrow interval on an out-of-distribution molecule does not guarantee correctness<br>
|
| 605 |
+
- **Full access**: We already computed MAPIE for all regression models; users are allowed to directly use them for customized model lists.<br>
|
| 606 |
|
| 607 |
---
|
| 608 |
|
| 609 |
+
#### Generating a MAPIE Bundle for a New Model<br>
|
| 610 |
|
| 611 |
+
To enable conformal uncertainty for a newly trained regression model:<br>
|
| 612 |
|
| 613 |
```bash
|
| 614 |
# Fit adaptive conformal bundle from val_predictions.csv
|
| 615 |
python fit_mapie_adaptive.py --root training_classifiers --prop <property_name>
|
| 616 |
```
|
| 617 |
|
| 618 |
+
The script reads `sequence`/`smiles` and `y_pred`/`y_true` columns from the CSV, recomputes embeddings, fits the XGBoost $\hat{\sigma}$ model, and saves `mapie_calibration.joblib` into the model directory. The bundle is automatically detected and loaded by `PeptiVersePredictor` on the next initialization.<br>
|
| 619 |
|
| 620 |
|
| 621 |
|