|
Download README.md from Syam26/ec-classifier-v3: direct link, hf CLI and curl.
- Browser
- Download file 6.19 kB
-
https://huggingface.co/Syam26/ec-classifier-v3/resolve/main/README.md
- Command line
-
hf download hf://Syam26/ec-classifier-v3/README.md
-
curl -L -o README.md https://huggingface.co/Syam26/ec-classifier-v3/resolve/main/README.md
6.19 kB
| license: cc-by-4.0 | |
| tags: | |
| - protein | |
| - enzyme-commission | |
| - ec-number | |
| - biology | |
| - leak-curber | |
| # Enzyme EC classification, v3: homology, residue profiles, a hierarchical head and a blended ranker | |
| Submission for the **Leak CURBER enzyme classification** benchmark (DA3410 Deep Learning Lab, | |
| Project 2). Given an enzyme amino-acid sequence, the model predicts one fully specified EC number. | |
| It runs on CPU only and offline, inside the evaluator's 2 vCPU / 16 GB / 10 minute budget. | |
| ## Local results | |
| Five folds cover all 14,466 training clusters at 50% identity. Evaluation uses 14,103 non-redundant | |
| queries (one per cluster, best hit under 60% identity). Every score is out-of-fold for the | |
| reference, the head and the ranker. *Matched* reweights queries to the identity histogram of the | |
| real inputs; *raw* does not and is pessimistic. Treat the two as a range. | |
| | system | matched L4 | L3 | L2 | L1 | raw L4 | | |
| |---|---|---|---|---|---| | |
| | DIAMOND bitscore vote | 0.719 | 0.884 | 0.900 | 0.921 | 0.669 | | |
| | v1 as shipped (approximation) | 0.732 | 0.906 | 0.924 | 0.951 | 0.683 | | |
| | v2 | 0.739 | 0.912 | 0.930 | 0.956 | 0.692 | | |
| | **v3 (this model)** | **0.741** | 0.912 | 0.930 | 0.956 | 0.694 | | |
| | *perfect ranking of v2's candidates* | *0.794* | *0.925* | *0.940* | *0.961* | *0.754* | | |
| v3 against v2, paired on identical queries: **L4 +0.0020, 95% CI [+0.0004, +0.0036]**. On folds 3-4 | |
| alone, which never guided any v3 choice: +0.0024 [-0.0002, +0.0051]. | |
| These are local estimates, not leaderboard scores. | |
| ## What changed from v2 | |
| 1. **A blend of three rankers.** Five-seed bags of LightGBM models trained with LambdaRank, | |
| XE-NDCG and a pointwise binary objective. Alone each scores like v2; averaged, their different | |
| errors partly cancel. | |
| 2. **A wider ESM-2 gate**: inputs whose best bitscore is under 300 (v2: 150) get head evidence, | |
| embedded weakest first in three tiers so that a slower machine still covers the inputs that gain | |
| most. | |
| 3. **Reproducible ranking.** Candidates with equal scores were ordered by Python's per-process | |
| hash seed in v2; v3 breaks ties by EC name. | |
| 4. **A deadline for the larger ranker**: evidence gathering stops at 500 s (v2: 540 s). | |
| The reference database, residue-profile features and hierarchical head are v2's, unchanged. | |
| ## What was measured and not adopted | |
| | idea | source | L4 vs v2 | | |
| |---|---|---| | |
| | Ranker hyperparameter search (60 configurations, confirmed on held-out folds) | — | +0.0000 | | |
| | Query-relative (listwise) features | learning-to-rank practice | +0.0001 | | |
| | More head-only candidates (2, 10, 15 instead of 5) | — | -0.0002 to -0.0008 | | |
| | Residue-profile constants (6 variants) | HIT-EC / GraphEC residue focus | within ±0.0008 | | |
| | Up to 10 training queries per cluster instead of 3 | — | +0.0000 | | |
| | Hard-negative contrastive batches + compositional label weights in the head | CLEAN; HIT-EC, CREEP/Horizyn | -0.0001 | | |
| | Learned mix of all 13 ESM-2 layers in the head | — | worse on fold 0 | | |
| | Rescue search with e-value 10 for no-hit inputs | — | best hit right 6% of the time | | |
| | Embedding every input | — | below gate 300 | | |
| ## How it works | |
| 1. **Homology.** DIAMOND `--very-sensitive` against all 186,488 unique training sequences, 200 hits | |
| with full alignments, read through a pipe. | |
| 2. **Residue profiles.** Each candidate EC's aligned homologs form a Henikoff-weighted profile on | |
| the query's columns, compared with its competitors' (the fourth digit is decided by a few | |
| specificity-determining residues). | |
| 3. **Protein language model.** ESM-2 (35M) and a head combining CLEAN's supervised contrastive | |
| projection with HIT-EC's hierarchical L1-L4 outputs: probabilities, centroid similarity, | |
| nearest neighbours and extra candidates. | |
| 4. **Ranking.** 15 LightGBM models score each (query, candidate EC) pair on 47 features; z-scored | |
| raw scores are summed. | |
| 5. **Fallback.** Anything with no evidence receives the most frequent training EC. | |
| ## Reliability | |
| * **No file over 16 MiB** written during inference: the DIAMOND index is built in the Dockerfile, | |
| `--hit-membuf` keeps DIAMOND's data in memory, hits arrive through a pipe. Verified on all 11,367 | |
| inputs with `RLIMIT_FSIZE` = 16 MiB. | |
| * **Runtime:** 496 s for 11,367 inputs at 2 threads on an Apple M4 (limit 600 s). The embedding | |
| stage deliberately fills the time up to its deadline; a slower machine embeds fewer inputs. | |
| * **Failure tests**, all with valid output for every input: the main search killed by its deadline | |
| (`EC_TIME_BUDGET=130`, 85% agreement with the full run), and an unusable DIAMOND index. | |
| ## Usage | |
| ```bash | |
| docker build --platform linux/amd64 -t my-enzyme-model . | |
| docker run --rm --platform linux/amd64 --user 1000:1000 --network none --cpus 2 --memory 16g \ | |
| -v "$PWD/../../data:/input:ro" -v "$PWD/output:/output" \ | |
| my-enzyme-model python /app/predict.py \ | |
| --input /input/val_inputs.parquet --output /output/predictions.jsonl | |
| ``` | |
| `EC_TIME_BUDGET` (default 500 s) and `EC_THREADS` (default 2) override the defaults. | |
| ## References | |
| * Yu, Cui, Li, Luo, Jiang, Zhao. *Enzyme function prediction using contrastive learning* (CLEAN). Science 379(6639), 2023. | |
| * Dumontet, Han, Lee, Oh, Kang. *Trustworthy prediction of enzyme commission numbers using a hierarchical interpretable transformer* (HIT-EC). Nature Communications 17(1), 2026. | |
| * Song, Yuan, Chen, Zeng, Zhao, Yang. *Accurately predicting enzyme functions through geometric graph learning on ESMFold-predicted structures* (GraphEC). Nature Communications 15(1), 2024. | |
| * Yang, Mora, Liu, Wittmann, Anandkumar, Arnold, Yue. *CARE: a Benchmark Suite for the Classification and Retrieval of Enzymes*. arXiv:2406.15669, 2025. | |
| * Rocks et al. *Dual-encoder contrastive learning accelerates enzyme discovery* (Horizyn). PNAS 123(12), 2026; Mikhael, Chinn, Barzilay. *CLIPZyme*. ICML 2024. | |
| * Buchfink, Reuter, Drost. *Sensitive protein alignments at tree-of-life scale using DIAMOND.* Nature Methods 18, 2021. | |
| * Lin et al. *Evolutionary-scale prediction of atomic-level protein structure with a language model* (ESM-2). Science 379, 2023. | |
| * Ke et al. *LightGBM.* NeurIPS 2017. | |
| EC vocabulary from ExPASy ENZYME (SIB, CC BY 4.0). | |