README: add 'Reproducing the main benchmark' section (CPU reproduce_benchmark.py)
Browse files
README.md
CHANGED
|
@@ -140,6 +140,25 @@ package version.
|
|
| 140 |
measurable lift over random subsampling. Treat this package as a research
|
| 141 |
baseline, not a turnkey accuracy improvement.
|
| 142 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 143 |
## Citation
|
| 144 |
|
| 145 |
If you use this code, please cite the SF-Cluster paper (forthcoming) and
|
|
|
|
| 140 |
measurable lift over random subsampling. Treat this package as a research
|
| 141 |
baseline, not a turnkey accuracy improvement.
|
| 142 |
|
| 143 |
+
## Reproducing the main benchmark
|
| 144 |
+
|
| 145 |
+
The scoring/evaluation code, reference structures, region definitions, and the
|
| 146 |
+
AlphaFold2 prediction sets for the headline arms are included so the main
|
| 147 |
+
`minority_hit_rate` table can be regenerated on CPU (no GPU):
|
| 148 |
+
|
| 149 |
+
```bash
|
| 150 |
+
pip install numpy biopython pyyaml
|
| 151 |
+
python reproduce_benchmark.py --mode score # re-score all 1440 prediction PDBs
|
| 152 |
+
python reproduce_benchmark.py --mode precomputed # aggregate shipped evals.tsv (fast)
|
| 153 |
+
```
|
| 154 |
+
|
| 155 |
+
The driver re-runs the Biopython Superimposer scorer (`eval/evaluate_prediction.py`,
|
| 156 |
+
hit = Cα RMSD ≤ 3 Å on common core AND pLDDT ≥ 70 overall AND ≥ 70 in the switch
|
| 157 |
+
region) over `bench/` and asserts each cell matches the published table
|
| 158 |
+
(mosaic-SF KaiB 0.95, GA98 0.925, GB98 0.1875; AF-Cluster baseline 0.36/0.46/0.44;
|
| 159 |
+
full table and the GA_GB scoring caveat in `README_benchmark.md`). All 15 cells
|
| 160 |
+
reproduce exactly. TMalign is optional (extra columns only).
|
| 161 |
+
|
| 162 |
## Citation
|
| 163 |
|
| 164 |
If you use this code, please cite the SF-Cluster paper (forthcoming) and
|