--- license: cc-by-4.0 tags: - protein - enzyme-commission - ec-number - biology - leak-curber --- # Enzyme EC classification, v3: homology, residue profiles, a hierarchical head and a blended ranker Submission for the **Leak CURBER enzyme classification** benchmark (DA3410 Deep Learning Lab, Project 2). Given an enzyme amino-acid sequence, the model predicts one fully specified EC number. It runs on CPU only and offline, inside the evaluator's 2 vCPU / 16 GB / 10 minute budget. ## Local results Five folds cover all 14,466 training clusters at 50% identity. Evaluation uses 14,103 non-redundant queries (one per cluster, best hit under 60% identity). Every score is out-of-fold for the reference, the head and the ranker. *Matched* reweights queries to the identity histogram of the real inputs; *raw* does not and is pessimistic. Treat the two as a range. | system | matched L4 | L3 | L2 | L1 | raw L4 | |---|---|---|---|---|---| | DIAMOND bitscore vote | 0.719 | 0.884 | 0.900 | 0.921 | 0.669 | | v1 as shipped (approximation) | 0.732 | 0.906 | 0.924 | 0.951 | 0.683 | | v2 | 0.739 | 0.912 | 0.930 | 0.956 | 0.692 | | **v3 (this model)** | **0.741** | 0.912 | 0.930 | 0.956 | 0.694 | | *perfect ranking of v2's candidates* | *0.794* | *0.925* | *0.940* | *0.961* | *0.754* | v3 against v2, paired on identical queries: **L4 +0.0020, 95% CI [+0.0004, +0.0036]**. On folds 3-4 alone, which never guided any v3 choice: +0.0024 [-0.0002, +0.0051]. These are local estimates, not leaderboard scores. ## What changed from v2 1. **A blend of three rankers.** Five-seed bags of LightGBM models trained with LambdaRank, XE-NDCG and a pointwise binary objective. Alone each scores like v2; averaged, their different errors partly cancel. 2. **A wider ESM-2 gate**: inputs whose best bitscore is under 300 (v2: 150) get head evidence, embedded weakest first in three tiers so that a slower machine still covers the inputs that gain most. 3. **Reproducible ranking.** Candidates with equal scores were ordered by Python's per-process hash seed in v2; v3 breaks ties by EC name. 4. **A deadline for the larger ranker**: evidence gathering stops at 500 s (v2: 540 s). The reference database, residue-profile features and hierarchical head are v2's, unchanged. ## What was measured and not adopted | idea | source | L4 vs v2 | |---|---|---| | Ranker hyperparameter search (60 configurations, confirmed on held-out folds) | — | +0.0000 | | Query-relative (listwise) features | learning-to-rank practice | +0.0001 | | More head-only candidates (2, 10, 15 instead of 5) | — | -0.0002 to -0.0008 | | Residue-profile constants (6 variants) | HIT-EC / GraphEC residue focus | within ±0.0008 | | Up to 10 training queries per cluster instead of 3 | — | +0.0000 | | Hard-negative contrastive batches + compositional label weights in the head | CLEAN; HIT-EC, CREEP/Horizyn | -0.0001 | | Learned mix of all 13 ESM-2 layers in the head | — | worse on fold 0 | | Rescue search with e-value 10 for no-hit inputs | — | best hit right 6% of the time | | Embedding every input | — | below gate 300 | ## How it works 1. **Homology.** DIAMOND `--very-sensitive` against all 186,488 unique training sequences, 200 hits with full alignments, read through a pipe. 2. **Residue profiles.** Each candidate EC's aligned homologs form a Henikoff-weighted profile on the query's columns, compared with its competitors' (the fourth digit is decided by a few specificity-determining residues). 3. **Protein language model.** ESM-2 (35M) and a head combining CLEAN's supervised contrastive projection with HIT-EC's hierarchical L1-L4 outputs: probabilities, centroid similarity, nearest neighbours and extra candidates. 4. **Ranking.** 15 LightGBM models score each (query, candidate EC) pair on 47 features; z-scored raw scores are summed. 5. **Fallback.** Anything with no evidence receives the most frequent training EC. ## Reliability * **No file over 16 MiB** written during inference: the DIAMOND index is built in the Dockerfile, `--hit-membuf` keeps DIAMOND's data in memory, hits arrive through a pipe. Verified on all 11,367 inputs with `RLIMIT_FSIZE` = 16 MiB. * **Runtime:** 496 s for 11,367 inputs at 2 threads on an Apple M4 (limit 600 s). The embedding stage deliberately fills the time up to its deadline; a slower machine embeds fewer inputs. * **Failure tests**, all with valid output for every input: the main search killed by its deadline (`EC_TIME_BUDGET=130`, 85% agreement with the full run), and an unusable DIAMOND index. ## Usage ```bash docker build --platform linux/amd64 -t my-enzyme-model . docker run --rm --platform linux/amd64 --user 1000:1000 --network none --cpus 2 --memory 16g \ -v "$PWD/../../data:/input:ro" -v "$PWD/output:/output" \ my-enzyme-model python /app/predict.py \ --input /input/val_inputs.parquet --output /output/predictions.jsonl ``` `EC_TIME_BUDGET` (default 500 s) and `EC_THREADS` (default 2) override the defaults. ## References * Yu, Cui, Li, Luo, Jiang, Zhao. *Enzyme function prediction using contrastive learning* (CLEAN). Science 379(6639), 2023. * Dumontet, Han, Lee, Oh, Kang. *Trustworthy prediction of enzyme commission numbers using a hierarchical interpretable transformer* (HIT-EC). Nature Communications 17(1), 2026. * Song, Yuan, Chen, Zeng, Zhao, Yang. *Accurately predicting enzyme functions through geometric graph learning on ESMFold-predicted structures* (GraphEC). Nature Communications 15(1), 2024. * Yang, Mora, Liu, Wittmann, Anandkumar, Arnold, Yue. *CARE: a Benchmark Suite for the Classification and Retrieval of Enzymes*. arXiv:2406.15669, 2025. * Rocks et al. *Dual-encoder contrastive learning accelerates enzyme discovery* (Horizyn). PNAS 123(12), 2026; Mikhael, Chinn, Barzilay. *CLIPZyme*. ICML 2024. * Buchfink, Reuter, Drost. *Sensitive protein alignments at tree-of-life scale using DIAMOND.* Nature Methods 18, 2021. * Lin et al. *Evolutionary-scale prediction of atomic-level protein structure with a language model* (ESM-2). Science 379, 2023. * Ke et al. *LightGBM.* NeurIPS 2017. EC vocabulary from ExPASy ENZYME (SIB, CC BY 4.0).