foldmatch-esm-codes / README.md
jseguramora's picture
esm-codes-v1 bundle
3a14f46 verified
|
Raw History Blame Contribute Delete
10.7 kB
---
license: mit
tags:
- biology
- protein
- esm3
- sequence-alignment
- foldmatch
---
# FoldMatch ESM3 residue codes (`esm-codes-v1`)
Encoders, statistics table and calibration panel for FoldMatch's ESM3 residue-code Stage-2 scoring. Every residue of a
database sequence is stored as an 8-byte binary code computed from its ESM3 embedding; a query keeps a 64-dimensional
fp16 encoding. Stage 2 aligns query and target with Smith-Waterman on the row/column-normalised code similarity and
reports calibrated E-values. It is an alternative to BLOSUM62, trained so that its alignments reproduce structural
(US-align) residue correspondences.
## Intended use
These files are made for the ESM3 residue-code scoring that is being added to
[FoldMatch](https://github.com/rcsb/foldmatch); FoldMatch's release notes will say from which version on it is
available. That FoldMatch downloads the files from this repository at a pinned revision and checks their sha256s
before use. It adds two options:
- `fm-search build sequences --residue-codes` writes the per-residue codes next to the sequence database (8 bytes per
residue, from the same ESM3 pass that computes the chain embeddings);
- `fm-search query sequences --scoring esm-codes` scores the Stage-1 candidates with the codes (one scoring per query;
BLOSUM62 scoring, `--scoring blosum62`, is unchanged).
The files are not meant to be used outside FoldMatch: the exact precision chain below is part of the method.
## Inputs
- ESM3 `esm3_sm_open_v1`, sequence-only, through `rcsb/rcsb-esm@636acb420daf674df37ab0220f56061a5de2f169` (weights file
`data/weights/esm3_sm_open_v1.pth`, sha256 `5ead5a135c658068db6a4f1b933e72d6110992c4668822e1c0e2dcc53e38acd9`, the same blob as
`EvolutionaryScale/esm3-sm-open-v1@47f0545b2b6daf26a93439a3cd610f4f7f3d5478`).
- Tap: `per_residue_embedding` from `forward_and_sample(..., SamplingConfig(return_per_residue_embeddings=True))`, i.e.
the last block's output before the final LayerNorm; rows 1..L (BOS and EOS removed), `[L, 1536]`.
- Sequences are normalised first: ASCII whitespace removed, ASCII letters upper-cased, non-ASCII refused.
## Method (system `esm-codes-v1`)
1. Transform (zcw50_clean, not folded): `x16 = fp16(x)`; `xs = fp32(x16[:, dims]) - offset` (dims = 0..1535 without
1361 and 1489); `xw = fp32(fp16(xs @ M))`; `offset` fp32[1534], `M` fp32[1534, 1534].
2. Encoders `f_q`, `f_t`: `f(x) = lin(x) + fc2(GELU_exact(fc1(x)))`, 1534 -> 1024 -> 64 with a linear skip, fp32 with
no TF32, no autocast and no dropout (3,471,360 parameters in total).
3. Target code: `bit_k = (f_t(xw)_k >= 0)`, `numpy.packbits(bits, axis=1)` MSB-first -> `uint8[L, 8]`; decoded as
`2 bit - 1`.
4. Query encoding: `q = fp16(f_q(xw))`, `[L, 64]`.
5. Scoring (setting D): cosine similarity of `q` and the decoded codes (fp32 GEMM per pair), row/column z-scores
(population std; a row or column with std < 1e-6 contributes 0), Smith-Waterman on `z - 1.0` with gap open 5.0 and
extend 0.5 (DP id `fmsw/1`). Score `S = S_f - S_r`, where `S_r` aligns the row-reversed similarity.
6. Statistics: `S' = S - Q90(query)`, with Q90 the 0.9 quantile of S over a fixed panel of 1,000 UniRef50 sequences;
`ln E = lnK(m) + ln m + ln N - lambda(m) S'` with lambda and lnK linear in ln m between six length bins and clamped
outside them (m = query length, N = database residues).
Calibration version: `foldmatch-evalue-M8_s0-D_fmr-pnlU50s1000-M3t999-v1`.
## Files
| File | Bytes | sha256 |
|---|---:|---|
| `esm_codes_v1_encoders.pt` | 23,324,662 | `1696ab91089ece8bce787d38b9d4d68bbc8ca4abff06f836f6e9363a23a1c02b` |
| `esm_codes_v1_scoring.npz` | 2,983,002 | `a5bebd626ec874626fb9372a63d10c32a001ed09c358717448bc9e1fd0ddedd6` |
| `bundle.json` | 10,830 | `dcd29bddbff0bb7dc93209bbe292d422d7bff21dd48c80f6e1f6b74f23789fe8` |
- `esm_codes_v1_encoders.pt`: `torch.load(path, weights_only=True)`. Format `foldmatch-esm-codes-encoders/1`; the transform, both
state_dicts, the architecture, conventions and provenance.
- `esm_codes_v1_scoring.npz`: `numpy.load(path, allow_pickle=False)`. Format `foldmatch-esm-codes-scoring/1`; `panel_codes` uint8
[355626, 8], `panel_offsets`, `panel_lengths`, `panel_ids` (UniRef50), `bin_centres`, `lam`, `lnK` (float64[6]) and
`meta_json` with every scoring constant.
- `bundle.json`: system name, calibration version, the sha256s above, content digests (transform
`e8989b1cefc83978...`, query encoder `d8bed195068be23a...`, target encoder
`58da263e005a79fd...`, panel `7aea2b00a24ee41c...`), the ESM3 identity, the DP
id, licence and attribution.
- `SHA256SUMS`: `shasum -a 256 -c SHA256SUMS` checks every file.
## Training and calibration
The encoders (research model L2m_B8_s0, epoch 7) were trained on sequence-only ESM3 embeddings of CATH domains to
reproduce US-align residue correspondences (a cross-entropy on distance-filtered US-align targets, distillation from the
full-precision whitened ESM3 similarity and a Smith-Waterman margin term), on a training pool that excludes every
evaluation domain and its superfamily; the checkpoint and the Smith-Waterman setting were chosen on development pairs
only. Smith-Waterman F1 against US-align at the shipped setting: 0.640 on validation related CATH pairs, 0.266 on
validation remote pairs, 0.262 on test remote pairs and 0.613 on 582 similar multi-domain chain pairs (predicted 3Di at
its own tuned setting: 0.604, 0.212, 0.203 and 0.536). E-values were fitted (censored Gumbel ML at q = 0.999 per
query-length bin) on 1,465 SCOP-annotated benchmark queries against 50,000 targets (8.6 million residues), with the
real null = targets that share no SCOP fold with the query; the per-query location Q90 comes from 1,000 UniRef50
release 2026_03 cluster representatives (amended stratified draw, 355,626 residues). On held-out query halves the
false-positive ratio (observed / expected) was 0.82 and 1.13 at E <= 1, 1.26 and 0.93 at E <= 0.1, and 2.7 and 2.2
at E <= 0.01; sensitivity at E <= 1e-3 was 0.15 and 0.20. FoldMatch's own ESM3 path reproduces the research codes on
99.5-99.6% of bits on GPU, CPU and Apple MPS (median |delta ln E| about 0.05, 0.049-0.051), and Mac and Linux CPUs
agree on 99.997% of bits.
## Limitations
- Sequence (FASTA) inputs only. The encoders were trained on sequence-only ESM3 embeddings; structure-conditioned
embeddings are out of distribution, and FoldMatch's residue-code options are for sequence inputs only.
- Calibrated domain: queries of 21-966 residues, targets up to 3,000 residues, at most 1% non-standard residues on
either side and targets with at least 2 distinct codes. Outside it FoldMatch still reports the alignment but sets
`ecal = 0` and warns.
- Long queries: for queries of 1,000-3,000 residues the E-values are optimistic (about 2x at E <= 1, 38x at E <= 0.01
and 180x at E <= 1e-3 on an 8.6-million-residue database); such hits need E far below the threshold or independent
confirmation. Long targets (1,000-3,000 residues) are calibrated (false-positive ratio 1.00 at E <= 1).
- In general E <= 0.01 is about 2.5x optimistic while E <= 1 is about right; the panel and table are fixed, so any
change is a new system name.
- A query of one residue, or whose encoding rows are all identical, carries no information and reports no hits.
- Results are bit-identical on one machine and BLAS build; across machines they agree within FoldMatch's tolerance
contract (scores within 5e-5 relative, Q90 and ln E within 1e-4).
- Only the ESM3 weights with sha256 `5ead5a135c658068...` are supported.
- The codes are trained for alignment quality, not for homology detection; candidates come from FoldMatch's Stage-1
chain-embedding search, which bounds recall.
## Licence and attribution
Copyright (c) 2026 - now, RCSB PDB and contributors. The encoders, the transform, the statistics table, the panel codes and this documentation are
released under the MIT licence (see `LICENSE`), like FoldMatch. The panel member identifiers (`panel_ids`) are UniProt
data and are not covered by the MIT licence: they stay under CC BY 4.0, with the attribution below.
- UniRef50: Panel member identifiers are UniRef50 cluster identifiers (UniProt release 2026_03) from the UniProt Consortium, https://www.uniprot.org, distributed under the Creative Commons Attribution 4.0 International licence (CC BY 4.0, https://creativecommons.org/licenses/by/4.0/). The panel codes were computed from ESM3 embeddings of those representative sequences; the sequences themselves are not included.
- ESM3: The encoders take per-residue embeddings of ESM3 esm3_sm_open_v1 (EvolutionaryScale) as input; no ESM3 weights are included. FoldMatch loads the weights from rcsb/rcsb-esm@636acb420daf674df37ab0220f56061a5de2f169 (MIT; weights sha256 5ead5a135c658068db6a4f1b933e72d6110992c4668822e1c0e2dcc53e38acd9), the same blob as EvolutionaryScale/esm3-sm-open-v1@47f0545b2b6daf26a93439a3cd610f4f7f3d5478, whose model card points to the MIT licence of the evolutionaryscale/esm repository. Older releases of the same weights (e.g. the gated EvolutionaryScale/esm3-sm-open-v1 card at 2feda652 and the LICENSE.md shipped in the esm 3.2.3 wheel) carry the EvolutionaryScale Cambrian Non-Commercial License; each copy of the weights is governed by the licence it was obtained under.
## Citation
- FoldMatch: Segura, J., et al. (2026). Multi-scale structural similarity embedding search across entire proteomes.
https://doi.org/10.1093/bioinformatics/btag058
- ESM3: Hayes, T., et al. (2025). Simulating 500 million years of evolution with a language model. Science 387,
850-858. https://doi.org/10.1126/science.ads0018
- UniProt: The UniProt Consortium (2025). UniProt: the Universal Protein Knowledgebase in 2025. Nucleic Acids Research
53(D1), D609-D617. https://doi.org/10.1093/nar/gkae1010
## Verification
Before writing these files the export script (`rcsb-embedding-search src/scripts/save_esm_codes_for_production.py`)
checked that the encoders reload with `weights_only=True`, rebuild with `strict=True` and equal the research
checkpoint, and that re-encoding FoldMatch's test fixtures gives the reference codes (F2
20,416/20,416 bits, F2c 19,392/19,392,
F2b 22,976/22,976) and query encodings (20,416 of
20,416 F2 values bit-equal). With this panel and table the research kernel reproduces the 50
fixture pairs (50/50 paths; max |delta Q90| 0, max |delta ln E|
0) and 20 benchmark Q90s (max |delta| 0), and the scoring file
loads with `allow_pickle=False` in a process without torch. These numbers come from the export machine (Apple silicon
CPU). On other CPUs (checked on Linux x86-64) the codes are identical and the query encodings agree within one fp16 ulp
of each row's largest value.