Joblib
ynuozhang commited on
Commit
07fc6ea
·
1 Parent(s): 60cd1e8

update readme

Browse files
Files changed (1) hide show
  1. README.md +35 -35
README.md CHANGED
@@ -518,48 +518,48 @@ print(predictor.predict_property("nf", "smiles", smiles, uncertainty=True
518
  You can also find the same description in the paper or in the PeptiVerse app `Documentation` tab.
519
 
520
  ---
521
- #### 🩸 Hemolysis Prediction
522
- 50% of read blood cells being lysed at x ug/ml concetration (HC50). If HC50 < 100uM, considered as hemolytic, otherwise non-hemolytic, resulting in a binary 0/1 dataset. The predicted probability should therefore be interpreted as a risk indicator, not an exact concentration estimate.
523
- **Output interpretation:**
524
  - Score close to 1.0 = high probability of red blood cell membrane disruption
525
  - Score close to 0.0 = non-hemolytic
526
  ---
527
 
528
- #### 💧 Solubility Prediction
529
- Outputs a probability (0–1) that a peptide remains soluble in aqueous conditions.
530
- **Output interpretation:**
531
  - Score close to 1.0 = highly soluble
532
  - Score close to 0.0 = poorly soluble
533
 
534
  ---
535
 
536
- #### 👯 Non-Fouling Prediction
537
- Higher scores indicate stronger non-fouling behavior, desirable for circulation and surface-exposed applications.
538
- **Output interpretation:**
539
  - Score close to 1.0 = non-fouling
540
  - Score close to 0.0 = fouling
541
  ---
542
 
543
- #### 🪣 Permeability Prediction
544
- Predicts membrane permeability on a log P scale.
545
- **Output interpretation:**
546
  - Higher values = more permeable (>-6.0)
547
  - For penetrance predictions, it is a classification prediction, so within the [0, 1] range, closer to 1 indicates more permeable.
548
  ---
549
 
550
- #### ⏱️ Half-Life Prediction
551
- **Interpretation:** Predicted values reflect relative peptide stability for the unit in hours. Higher scores indicate longer persistence in serum, while lower scores suggest faster degradation.
552
 
553
  ---
554
 
555
- #### ☠️ Toxicity Prediction
556
- **Interpretation:** Outputs a probability (0–1) that a peptide exhibits toxic effects. Higher scores indicate increased toxicity risk.
557
 
558
  ---
559
 
560
- #### 🔗 Binding Affinity Prediction
561
 
562
- Predicts peptide-protein binding affinity. Requires both peptide and target protein sequence.
563
 
564
  **Interpretation:**<br>
565
  - Scores ≥ 9 correspond to tight binders (K ≤ 10⁻⁹ M, nanomolar to picomolar range)<br>
@@ -569,15 +569,15 @@ Predicts peptide-protein binding affinity. Requires both peptide and target prot
569
 
570
  ---
571
 
572
- ### Uncertainty Interpretation
573
- #### Entropy (classifiers)
574
 
575
- Binary predictive entropy of the output probability $\bar{p}$:
576
 
577
- $$\mathcal{H} = -\bar{p}\log\bar{p} - (1 - \bar{p})\log(1 - \bar{p})$$
578
 
579
- - For **DNN classifiers**: $\bar{p}$ is the mean probability across 5 independently seeded models (deep ensemble). High entropy reflects both epistemic uncertainty (seed disagreement) and aleatoric uncertainty (collectively diffuse predictions).
580
- - For **XGBoost / SVM / ElasticNet classifiers**: $\bar{p}$ is the single model's output probability (or sigmoid of decision function for ElasticNet). Entropy reflects output confidence of a single model only.
581
 
582
  | Range | Interpretation |
583
  |---|---|
@@ -589,33 +589,33 @@ $$\mathcal{H} = -\bar{p}\log\bar{p} - (1 - \bar{p})\log(1 - \bar{p})$$
589
 
590
  ---
591
 
592
- #### Adaptive Conformal Prediction Interval (regressors)
593
 
594
- Returned as a tuple `(lo, hi)` with 90% marginal coverage guarantee.
595
 
596
- We implement the **residual normalised conformity score** following [Lei et al. (2018)](https://doi.org/10.1080/01621459.2017.1307116) and [Cordier et al. (2023) / MAPIE](https://proceedings.mlr.press/v204/cordier23a.html). An auxiliary XGBoost model $\hat{\sigma}(\mathbf{x})$ is trained on held-out embeddings and absolute residuals $|y_i - \hat{y}_i|$. At inference:
597
 
598
  $$[\hat{y}(\mathbf{x}) - q \cdot \hat{\sigma}(\mathbf{x}),\ \hat{y}(\mathbf{x}) + q \cdot \hat{\sigma}(\mathbf{x})]$$
599
 
600
- where $q$ is the $\lceil(n+1)(1-\alpha)\rceil / n$ quantile of the normalized scores $s_i = |y_i - \hat{y}_i| / \hat{\sigma}(\mathbf{x}_i)$.
601
 
602
- - **Interval width varies per input** -- molecules more dissimilar to training data tend to receive wider intervals
603
- - **Coverage guarantee**: on exchangeable data, $P(y \in [\hat{y} - q\hat{\sigma},\ \hat{y} + q\hat{\sigma}]) \geq 0.90$
604
- - **The guarantee is marginal**, not conditional, as an unusually narrow interval on an out-of-distribution molecule does not guarantee correctness
605
- - **Full access**: we already computed MAPIE for all regression models, users are allowed to directly use them for customized model lists.
606
 
607
  ---
608
 
609
- #### Generating a MAPIE Bundle for a New Model
610
 
611
- To enable conformal uncertainty for a newly trained regression model:
612
 
613
  ```bash
614
  # Fit adaptive conformal bundle from val_predictions.csv
615
  python fit_mapie_adaptive.py --root training_classifiers --prop <property_name>
616
  ```
617
 
618
- The script reads `sequence`/`smiles` and `y_pred`/`y_true` columns from the CSV, recomputes embeddings, fits the XGBoost $\hat{\sigma}$ model, and saves `mapie_calibration.joblib` into the model directory. The bundle is automatically detected and loaded by `PeptiVersePredictor` on next initialisation.
619
 
620
 
621
 
 
518
  You can also find the same description in the paper or in the PeptiVerse app `Documentation` tab.
519
 
520
  ---
521
+ ### 🩸 Hemolysis Prediction<br>
522
+ 50% of read blood cells being lysed at x ug/ml concetration (HC50). If HC50 < 100uM, considered as hemolytic, otherwise non-hemolytic, resulting in a binary 0/1 dataset. The predicted probability should therefore be interpreted as a risk indicator, not an exact concentration estimate.<br>
523
+ **Output interpretation:**<br>
524
  - Score close to 1.0 = high probability of red blood cell membrane disruption
525
  - Score close to 0.0 = non-hemolytic
526
  ---
527
 
528
+ ### 💧 Solubility Prediction<br>
529
+ Outputs a probability (0–1) that a peptide remains soluble in aqueous conditions.<br>
530
+ **Output interpretation:**<br>
531
  - Score close to 1.0 = highly soluble
532
  - Score close to 0.0 = poorly soluble
533
 
534
  ---
535
 
536
+ ### 👯 Non-Fouling Prediction<br>
537
+ Higher scores indicate stronger non-fouling behavior, desirable for circulation and surface-exposed applications.<br>
538
+ **Output interpretation:**<br>
539
  - Score close to 1.0 = non-fouling
540
  - Score close to 0.0 = fouling
541
  ---
542
 
543
+ ### 🪣 Permeability Prediction<br>
544
+ Predicts membrane permeability on a log P scale.<br>
545
+ **Output interpretation:**<br>
546
  - Higher values = more permeable (>-6.0)
547
  - For penetrance predictions, it is a classification prediction, so within the [0, 1] range, closer to 1 indicates more permeable.
548
  ---
549
 
550
+ ### ⏱️ Half-Life Prediction<br>
551
+ **Interpretation:** Predicted values reflect relative peptide stability for the unit in hours. Higher scores indicate longer persistence in serum, while lower scores suggest faster degradation.<br>
552
 
553
  ---
554
 
555
+ ### ☠️ Toxicity Prediction<br>
556
+ **Interpretation:** Outputs a probability (0–1) that a peptide exhibits toxic effects. Higher scores indicate increased toxicity risk.<br>
557
 
558
  ---
559
 
560
+ ### 🔗 Binding Affinity Prediction <br>
561
 
562
+ Predicts peptide-protein binding affinity. Requires both peptide and target protein sequence.<br>
563
 
564
  **Interpretation:**<br>
565
  - Scores ≥ 9 correspond to tight binders (K ≤ 10⁻⁹ M, nanomolar to picomolar range)<br>
 
569
 
570
  ---
571
 
572
+ ### Uncertainty Interpretation <br>
573
+ #### Entropy (classifiers)<br>
574
 
575
+ Binary predictive entropy of the output probability $\bar{p}$:<br>
576
 
577
+ $$\mathcal{H} = -\bar{p}\log\bar{p} - (1 - \bar{p})\log(1 - \bar{p})$$<br>
578
 
579
+ - For **DNN classifiers**: $\bar{p}$ is the mean probability across 5 independently seeded models (deep ensemble). High entropy reflects both epistemic uncertainty (seed disagreement) and aleatoric uncertainty (collectively diffuse predictions).<br>
580
+ - For **XGBoost / SVM / ElasticNet classifiers**: $\bar{p}$ is the single model's output probability (or sigmoid of decision function for ElasticNet). Entropy reflects output confidence of a single model only.<br>
581
 
582
  | Range | Interpretation |
583
  |---|---|
 
589
 
590
  ---
591
 
592
+ #### Adaptive Conformal Prediction Interval (regressors)<br>
593
 
594
+ Returned as a tuple `(lo, hi)` with 90% marginal coverage guarantee.<br>
595
 
596
+ We implement the **residual normalised conformity score** following [Lei et al. (2018)](https://doi.org/10.1080/01621459.2017.1307116) and [Cordier et al. (2023) / MAPIE](https://proceedings.mlr.press/v204/cordier23a.html). An auxiliary XGBoost model $\hat{\sigma}(\mathbf{x})$ is trained on held-out embeddings and absolute residuals $|y_i - \hat{y}_i|$. At inference:<br>
597
 
598
  $$[\hat{y}(\mathbf{x}) - q \cdot \hat{\sigma}(\mathbf{x}),\ \hat{y}(\mathbf{x}) + q \cdot \hat{\sigma}(\mathbf{x})]$$
599
 
600
+ where $q$ is the $\lceil(n+1)(1-\alpha)\rceil / n$ quantile of the normalized scores $s_i = |y_i - \hat{y}_i| / \hat{\sigma}(\mathbf{x}_i)$.<br>
601
 
602
+ - **Interval width varies per input** -- molecules more dissimilar to training data tend to receive wider intervals<br>
603
+ - **Coverage guarantee**: on exchangeable data, $P(y \in [\hat{y} - q\hat{\sigma},\ \hat{y} + q\hat{\sigma}]) \geq 0.90$<br>
604
+ - **The guarantee is marginal**, not conditional, as an unusually narrow interval on an out-of-distribution molecule does not guarantee correctness<br>
605
+ - **Full access**: We already computed MAPIE for all regression models; users are allowed to directly use them for customized model lists.<br>
606
 
607
  ---
608
 
609
+ #### Generating a MAPIE Bundle for a New Model<br>
610
 
611
+ To enable conformal uncertainty for a newly trained regression model:<br>
612
 
613
  ```bash
614
  # Fit adaptive conformal bundle from val_predictions.csv
615
  python fit_mapie_adaptive.py --root training_classifiers --prop <property_name>
616
  ```
617
 
618
+ The script reads `sequence`/`smiles` and `y_pred`/`y_true` columns from the CSV, recomputes embeddings, fits the XGBoost $\hat{\sigma}$ model, and saves `mapie_calibration.joblib` into the model directory. The bundle is automatically detected and loaded by `PeptiVersePredictor` on the next initialization.<br>
619
 
620
 
621