infernogod commited on
Commit
ecfc4dd
·
verified ·
1 Parent(s): cdb4903
README.md CHANGED
@@ -9,7 +9,7 @@ tags:
9
  Submission for the Leak CURBER enzyme classification task (`leak-curber-enzyme-classification`).
10
 
11
  ## External data (disclosure)
12
- The course permits external data provided it is declared. This model uses three external resources, all from UniProtKB (UniProt Consortium, CC BY 4.0), downloaded on 31 Oct 2026:
13
 
14
  - **UniProtKB/Swiss-Prot**, reviewed entries with EC annotations (UniProt Consortium, CC BY 4.0), downloaded on 31 Oct 2026 from
15
  `https://rest.uniprot.org/uniprotkb/stream?compressed=true&format=tsv&fields=accession,ec,sequence&query=reviewed:true`.
@@ -19,8 +19,9 @@ The course permits external data provided it is declared. This model uses three
19
  73,351 entries. After the same filtering (vocabulary ECs only, skip sequences already in train or Swiss-Prot, merge duplicates), **62,670 unique sequences** are added as `un_<accession>`.
20
  - **UniProtKB/TrEMBL, unreviewed entries with transcript-level evidence and an EC annotation (v16)**, from the same endpoint with
21
  `query=(reviewed:false) AND (ec:*) AND (existence:2)`, downloaded one EC class at a time. 250,795 entries. After the same filtering, **206,689 unique sequences** (`u2_<accession>`) form a separate database, `weights/trembl_ev2.dmnd`, searched in DIAMOND's fast mode as an optional second pass.
22
- - The external records are shipped in plain text in `weights/swissprot_ext.fa.gz`, `weights/trembl_ext.fa.gz` and `weights/trembl_ev2_ext.fa.gz`, and their labels are in `weights/labels.json.gz`. `tools/build_external_reference.py` rebuilds the reference and accepts `--exclude` to drop any sequences, for example test-split candidates.
23
- - **Leakage note.** 9,663 of the 11,367 unique validation inputs (85%) appear verbatim in Swiss-Prot, 170 more in the protein-level TrEMBL set and 122 more in the transcript-level set. On the public validation set this external reference therefore acts largely as a lookup, and the public score overstates generalisation. To measure the genuine benefit, we removed every validation sequence from the external part and re-ran the pipeline:
 
24
 
25
  | Reference database | Val L4 (9,663 inputs with Swiss-Prot labels) |
26
  |---|---|
@@ -38,7 +39,7 @@ The labels for the validation check come from Swiss-Prot. They agree with the of
38
  No external data was used for training any learned component. ESM-2 and other pretrained protein language models were used only in experiments and are not part of this submission.
39
 
40
  ## Pipeline
41
- 1. **Homology search.** Every unique input is searched against the reference (186,488 train + 31,985 Swiss-Prot + 62,670 TrEMBL sequences) with DIAMOND v2.1.10 (`blastp --very-sensitive`, 50 hits).
42
  2. **Second pass.** Inputs with no hit, or a best hit of 30% identity or less, are searched again with `--ultra-sensitive` when the time budget clearly allows it.
43
  3. **Extra reference pass (v16).** All inputs are also searched against the transcript-level TrEMBL database in DIAMOND's `--fast` mode, about 25 s. This runs only if less than 330 s have passed. It only adds hits and is skipped on any error. For the hard validation proteins not in Swiss-Prot, it doubles the number with a relative at 80% identity or more (53 to 103, with validation sequences excluded).
44
  4. **Vote.** Each EC gets `sum over the top 50 hits of (bitscore / best bitscore)^12 x w_source`, split evenly across a hit's ECs. `w_source` is 1 for train and Swiss-Prot references, 0.2 for protein-level TrEMBL references and 0.1 for transcript-level TrEMBL references, whose automatic annotations are noisier. The highest-scoring EC is predicted, with ties broken by training frequency. Settings are in `weights/mode.json`. They were tuned on the validation inputs that have Swiss-Prot labels, with every validation sequence removed from the external reference. Tuning used alternate halves: the best setting chosen on one half improved the other half by the same margin (+0.5 points over k=20, power 8, unweighted).
 
9
  Submission for the Leak CURBER enzyme classification task (`leak-curber-enzyme-classification`).
10
 
11
  ## External data (disclosure)
12
+ The course permits external data provided it is declared. This model uses four external resources: three from UniProtKB (UniProt Consortium, CC BY 4.0), downloaded on 31 Oct 2026, and CatPred-DB (v19):
13
 
14
  - **UniProtKB/Swiss-Prot**, reviewed entries with EC annotations (UniProt Consortium, CC BY 4.0), downloaded on 31 Oct 2026 from
15
  `https://rest.uniprot.org/uniprotkb/stream?compressed=true&format=tsv&fields=accession,ec,sequence&query=reviewed:true`.
 
19
  73,351 entries. After the same filtering (vocabulary ECs only, skip sequences already in train or Swiss-Prot, merge duplicates), **62,670 unique sequences** are added as `un_<accession>`.
20
  - **UniProtKB/TrEMBL, unreviewed entries with transcript-level evidence and an EC annotation (v16)**, from the same endpoint with
21
  `query=(reviewed:false) AND (ec:*) AND (existence:2)`, downloaded one EC class at a time. 250,795 entries. After the same filtering, **206,689 unique sequences** (`u2_<accession>`) form a separate database, `weights/trembl_ev2.dmnd`, searched in DIAMOND's fast mode as an optional second pass.
22
+ - **CatPred-DB (v19)**: enzyme kinetics data compiled from BRENDA and SABIO-RK, with sequences, UniProt IDs and EC numbers (Boorla & Maranas, *Nature Communications* 2025; Hugging Face `RosettaCommons/CatPred-DB`, `datasets/processed/*_wt_singleSeqs_wpdbs.csv`). There are 13,204 unique sequences, 12,686 with an allowed EC. **2,354 are new** (not in train or the UniProt sets) and are added as `cp_<n>_<uniprot>` with the union of their ECs.
23
+ - The external records are shipped in plain text in `weights/swissprot_ext.fa.gz`, `weights/trembl_ext.fa.gz`, `weights/trembl_ev2_ext.fa.gz` and `weights/catpred_ext.fa.gz`, and their labels are in `weights/labels.json.gz`. `tools/build_external_reference.py` rebuilds the reference and accepts `--exclude` to drop any sequences, for example test-split candidates.
24
+ - **Leakage note.** 9,663 of the 11,367 unique validation inputs (85%) appear verbatim in Swiss-Prot, 170 more in the protein-level TrEMBL set, 122 more in the transcript-level set, and 459 more in CatPred-DB. On the public validation set this external reference therefore acts largely as a lookup, and the public score overstates generalisation. To measure the genuine benefit, we removed every validation sequence from the external part and re-ran the pipeline:
25
 
26
  | Reference database | Val L4 (9,663 inputs with Swiss-Prot labels) |
27
  |---|---|
 
39
  No external data was used for training any learned component. ESM-2 and other pretrained protein language models were used only in experiments and are not part of this submission.
40
 
41
  ## Pipeline
42
+ 1. **Homology search.** Every unique input is searched against the reference (186,488 train + 31,985 Swiss-Prot + 62,670 TrEMBL + 2,354 CatPred-DB sequences) with DIAMOND v2.1.10 (`blastp --very-sensitive`, 50 hits).
43
  2. **Second pass.** Inputs with no hit, or a best hit of 30% identity or less, are searched again with `--ultra-sensitive` when the time budget clearly allows it.
44
  3. **Extra reference pass (v16).** All inputs are also searched against the transcript-level TrEMBL database in DIAMOND's `--fast` mode, about 25 s. This runs only if less than 330 s have passed. It only adds hits and is skipped on any error. For the hard validation proteins not in Swiss-Prot, it doubles the number with a relative at 80% identity or more (53 to 103, with validation sequences excluded).
45
  4. **Vote.** Each EC gets `sum over the top 50 hits of (bitscore / best bitscore)^12 x w_source`, split evenly across a hit's ECs. `w_source` is 1 for train and Swiss-Prot references, 0.2 for protein-level TrEMBL references and 0.1 for transcript-level TrEMBL references, whose automatic annotations are noisier. The highest-scoring EC is predicted, with ties broken by training frequency. Settings are in `weights/mode.json`. They were tuned on the validation inputs that have Swiss-Prot labels, with every validation sequence removed from the external reference. Tuning used alternate halves: the best setting chosen on one half improved the other half by the same margin (+0.5 points over k=20, power 8, unweighted).
weights/catpred_ext.fa.gz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9435192600ae554d577d8e28454684480fba5248a2d2326b0db9c43e324326ac
3
+ size 588449
weights/labels.json.gz CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:62f567bd0e387932e7cef3a5e8545204b4836981442990edef3e161a92e6bf1e
3
- size 2809589
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ab3b173b33b5fcda184a007526c1da5d3668c61b2e721e3842bc01adf658e379
3
+ size 2833370
weights/train.dmnd CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:91f98ad221364ab1c4ee6007d9019b121990645ee83f53978e7e8cf4dfbdf128
3
- size 143298378
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b87593cf033d04ab37e766662fd82db18cbf46aad4f03b6d9cbc1922506f0e8d
3
+ size 144388995