File size: 6,186 Bytes
cab6030
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
---
license: cc-by-4.0
tags:
  - protein
  - enzyme-commission
  - ec-number
  - biology
  - leak-curber
---

# Enzyme EC classification, v3: homology, residue profiles, a hierarchical head and a blended ranker

Submission for the **Leak CURBER enzyme classification** benchmark (DA3410 Deep Learning Lab,
Project 2). Given an enzyme amino-acid sequence, the model predicts one fully specified EC number.
It runs on CPU only and offline, inside the evaluator's 2 vCPU / 16 GB / 10 minute budget.

## Local results

Five folds cover all 14,466 training clusters at 50% identity. Evaluation uses 14,103 non-redundant
queries (one per cluster, best hit under 60% identity). Every score is out-of-fold for the
reference, the head and the ranker. *Matched* reweights queries to the identity histogram of the
real inputs; *raw* does not and is pessimistic. Treat the two as a range.

| system | matched L4 | L3 | L2 | L1 | raw L4 |
|---|---|---|---|---|---|
| DIAMOND bitscore vote | 0.719 | 0.884 | 0.900 | 0.921 | 0.669 |
| v1 as shipped (approximation) | 0.732 | 0.906 | 0.924 | 0.951 | 0.683 |
| v2 | 0.739 | 0.912 | 0.930 | 0.956 | 0.692 |
| **v3 (this model)** | **0.741** | 0.912 | 0.930 | 0.956 | 0.694 |
| *perfect ranking of v2's candidates* | *0.794* | *0.925* | *0.940* | *0.961* | *0.754* |

v3 against v2, paired on identical queries: **L4 +0.0020, 95% CI [+0.0004, +0.0036]**. On folds 3-4
alone, which never guided any v3 choice: +0.0024 [-0.0002, +0.0051].

These are local estimates, not leaderboard scores.

## What changed from v2

1. **A blend of three rankers.** Five-seed bags of LightGBM models trained with LambdaRank,
   XE-NDCG and a pointwise binary objective. Alone each scores like v2; averaged, their different
   errors partly cancel.
2. **A wider ESM-2 gate**: inputs whose best bitscore is under 300 (v2: 150) get head evidence,
   embedded weakest first in three tiers so that a slower machine still covers the inputs that gain
   most.
3. **Reproducible ranking.** Candidates with equal scores were ordered by Python's per-process
   hash seed in v2; v3 breaks ties by EC name.
4. **A deadline for the larger ranker**: evidence gathering stops at 500 s (v2: 540 s).

The reference database, residue-profile features and hierarchical head are v2's, unchanged.

## What was measured and not adopted

| idea | source | L4 vs v2 |
|---|---|---|
| Ranker hyperparameter search (60 configurations, confirmed on held-out folds) | — | +0.0000 |
| Query-relative (listwise) features | learning-to-rank practice | +0.0001 |
| More head-only candidates (2, 10, 15 instead of 5) | — | -0.0002 to -0.0008 |
| Residue-profile constants (6 variants) | HIT-EC / GraphEC residue focus | within ±0.0008 |
| Up to 10 training queries per cluster instead of 3 | — | +0.0000 |
| Hard-negative contrastive batches + compositional label weights in the head | CLEAN; HIT-EC, CREEP/Horizyn | -0.0001 |
| Learned mix of all 13 ESM-2 layers in the head | — | worse on fold 0 |
| Rescue search with e-value 10 for no-hit inputs | — | best hit right 6% of the time |
| Embedding every input | — | below gate 300 |

## How it works

1. **Homology.** DIAMOND `--very-sensitive` against all 186,488 unique training sequences, 200 hits
   with full alignments, read through a pipe.
2. **Residue profiles.** Each candidate EC's aligned homologs form a Henikoff-weighted profile on
   the query's columns, compared with its competitors' (the fourth digit is decided by a few
   specificity-determining residues).
3. **Protein language model.** ESM-2 (35M) and a head combining CLEAN's supervised contrastive
   projection with HIT-EC's hierarchical L1-L4 outputs: probabilities, centroid similarity,
   nearest neighbours and extra candidates.
4. **Ranking.** 15 LightGBM models score each (query, candidate EC) pair on 47 features; z-scored
   raw scores are summed.
5. **Fallback.** Anything with no evidence receives the most frequent training EC.

## Reliability

* **No file over 16 MiB** written during inference: the DIAMOND index is built in the Dockerfile,
  `--hit-membuf` keeps DIAMOND's data in memory, hits arrive through a pipe. Verified on all 11,367
  inputs with `RLIMIT_FSIZE` = 16 MiB.
* **Runtime:** 496 s for 11,367 inputs at 2 threads on an Apple M4 (limit 600 s). The embedding
  stage deliberately fills the time up to its deadline; a slower machine embeds fewer inputs.
* **Failure tests**, all with valid output for every input: the main search killed by its deadline
  (`EC_TIME_BUDGET=130`, 85% agreement with the full run), and an unusable DIAMOND index.

## Usage

```bash
docker build --platform linux/amd64 -t my-enzyme-model .
docker run --rm --platform linux/amd64 --user 1000:1000 --network none --cpus 2 --memory 16g \
  -v "$PWD/../../data:/input:ro" -v "$PWD/output:/output" \
  my-enzyme-model python /app/predict.py \
  --input /input/val_inputs.parquet --output /output/predictions.jsonl
```

`EC_TIME_BUDGET` (default 500 s) and `EC_THREADS` (default 2) override the defaults.

## References

* Yu, Cui, Li, Luo, Jiang, Zhao. *Enzyme function prediction using contrastive learning* (CLEAN). Science 379(6639), 2023.
* Dumontet, Han, Lee, Oh, Kang. *Trustworthy prediction of enzyme commission numbers using a hierarchical interpretable transformer* (HIT-EC). Nature Communications 17(1), 2026.
* Song, Yuan, Chen, Zeng, Zhao, Yang. *Accurately predicting enzyme functions through geometric graph learning on ESMFold-predicted structures* (GraphEC). Nature Communications 15(1), 2024.
* Yang, Mora, Liu, Wittmann, Anandkumar, Arnold, Yue. *CARE: a Benchmark Suite for the Classification and Retrieval of Enzymes*. arXiv:2406.15669, 2025.
* Rocks et al. *Dual-encoder contrastive learning accelerates enzyme discovery* (Horizyn). PNAS 123(12), 2026; Mikhael, Chinn, Barzilay. *CLIPZyme*. ICML 2024.
* Buchfink, Reuter, Drost. *Sensitive protein alignments at tree-of-life scale using DIAMOND.* Nature Methods 18, 2021.
* Lin et al. *Evolutionary-scale prediction of atomic-level protein structure with a language model* (ESM-2). Science 379, 2023.
* Ke et al. *LightGBM.* NeurIPS 2017.

EC vocabulary from ExPASy ENZYME (SIB, CC BY 4.0).