--- license: mit tags: - biology - protein - esm3 - sequence-alignment - foldmatch --- # FoldMatch ESM3 residue codes (`esm-codes-v1`) Encoders, statistics table and calibration panel for FoldMatch's ESM3 residue-code Stage-2 scoring. Every residue of a database sequence is stored as an 8-byte binary code computed from its ESM3 embedding; a query keeps a 64-dimensional fp16 encoding. Stage 2 aligns query and target with Smith-Waterman on the row/column-normalised code similarity and reports calibrated E-values. It is an alternative to BLOSUM62, trained so that its alignments reproduce structural (US-align) residue correspondences. ## Intended use These files are made for the ESM3 residue-code scoring that is being added to [FoldMatch](https://github.com/rcsb/foldmatch); FoldMatch's release notes will say from which version on it is available. That FoldMatch downloads the files from this repository at a pinned revision and checks their sha256s before use. It adds two options: - `fm-search build sequences --residue-codes` writes the per-residue codes next to the sequence database (8 bytes per residue, from the same ESM3 pass that computes the chain embeddings); - `fm-search query sequences --scoring esm-codes` scores the Stage-1 candidates with the codes (one scoring per query; BLOSUM62 scoring, `--scoring blosum62`, is unchanged). The files are not meant to be used outside FoldMatch: the exact precision chain below is part of the method. ## Inputs - ESM3 `esm3_sm_open_v1`, sequence-only, through `rcsb/rcsb-esm@636acb420daf674df37ab0220f56061a5de2f169` (weights file `data/weights/esm3_sm_open_v1.pth`, sha256 `5ead5a135c658068db6a4f1b933e72d6110992c4668822e1c0e2dcc53e38acd9`, the same blob as `EvolutionaryScale/esm3-sm-open-v1@47f0545b2b6daf26a93439a3cd610f4f7f3d5478`). - Tap: `per_residue_embedding` from `forward_and_sample(..., SamplingConfig(return_per_residue_embeddings=True))`, i.e. the last block's output before the final LayerNorm; rows 1..L (BOS and EOS removed), `[L, 1536]`. - Sequences are normalised first: ASCII whitespace removed, ASCII letters upper-cased, non-ASCII refused. ## Method (system `esm-codes-v1`) 1. Transform (zcw50_clean, not folded): `x16 = fp16(x)`; `xs = fp32(x16[:, dims]) - offset` (dims = 0..1535 without 1361 and 1489); `xw = fp32(fp16(xs @ M))`; `offset` fp32[1534], `M` fp32[1534, 1534]. 2. Encoders `f_q`, `f_t`: `f(x) = lin(x) + fc2(GELU_exact(fc1(x)))`, 1534 -> 1024 -> 64 with a linear skip, fp32 with no TF32, no autocast and no dropout (3,471,360 parameters in total). 3. Target code: `bit_k = (f_t(xw)_k >= 0)`, `numpy.packbits(bits, axis=1)` MSB-first -> `uint8[L, 8]`; decoded as `2 bit - 1`. 4. Query encoding: `q = fp16(f_q(xw))`, `[L, 64]`. 5. Scoring (setting D): cosine similarity of `q` and the decoded codes (fp32 GEMM per pair), row/column z-scores (population std; a row or column with std < 1e-6 contributes 0), Smith-Waterman on `z - 1.0` with gap open 5.0 and extend 0.5 (DP id `fmsw/1`). Score `S = S_f - S_r`, where `S_r` aligns the row-reversed similarity. 6. Statistics: `S' = S - Q90(query)`, with Q90 the 0.9 quantile of S over a fixed panel of 1,000 UniRef50 sequences; `ln E = lnK(m) + ln m + ln N - lambda(m) S'` with lambda and lnK linear in ln m between six length bins and clamped outside them (m = query length, N = database residues). Calibration version: `foldmatch-evalue-M8_s0-D_fmr-pnlU50s1000-M3t999-v1`. ## Files | File | Bytes | sha256 | |---|---:|---| | `esm_codes_v1_encoders.pt` | 23,324,662 | `1696ab91089ece8bce787d38b9d4d68bbc8ca4abff06f836f6e9363a23a1c02b` | | `esm_codes_v1_scoring.npz` | 2,983,002 | `a5bebd626ec874626fb9372a63d10c32a001ed09c358717448bc9e1fd0ddedd6` | | `bundle.json` | 10,830 | `dcd29bddbff0bb7dc93209bbe292d422d7bff21dd48c80f6e1f6b74f23789fe8` | - `esm_codes_v1_encoders.pt`: `torch.load(path, weights_only=True)`. Format `foldmatch-esm-codes-encoders/1`; the transform, both state_dicts, the architecture, conventions and provenance. - `esm_codes_v1_scoring.npz`: `numpy.load(path, allow_pickle=False)`. Format `foldmatch-esm-codes-scoring/1`; `panel_codes` uint8 [355626, 8], `panel_offsets`, `panel_lengths`, `panel_ids` (UniRef50), `bin_centres`, `lam`, `lnK` (float64[6]) and `meta_json` with every scoring constant. - `bundle.json`: system name, calibration version, the sha256s above, content digests (transform `e8989b1cefc83978...`, query encoder `d8bed195068be23a...`, target encoder `58da263e005a79fd...`, panel `7aea2b00a24ee41c...`), the ESM3 identity, the DP id, licence and attribution. - `SHA256SUMS`: `shasum -a 256 -c SHA256SUMS` checks every file. ## Training and calibration The encoders (research model L2m_B8_s0, epoch 7) were trained on sequence-only ESM3 embeddings of CATH domains to reproduce US-align residue correspondences (a cross-entropy on distance-filtered US-align targets, distillation from the full-precision whitened ESM3 similarity and a Smith-Waterman margin term), on a training pool that excludes every evaluation domain and its superfamily; the checkpoint and the Smith-Waterman setting were chosen on development pairs only. Smith-Waterman F1 against US-align at the shipped setting: 0.640 on validation related CATH pairs, 0.266 on validation remote pairs, 0.262 on test remote pairs and 0.613 on 582 similar multi-domain chain pairs (predicted 3Di at its own tuned setting: 0.604, 0.212, 0.203 and 0.536). E-values were fitted (censored Gumbel ML at q = 0.999 per query-length bin) on 1,465 SCOP-annotated benchmark queries against 50,000 targets (8.6 million residues), with the real null = targets that share no SCOP fold with the query; the per-query location Q90 comes from 1,000 UniRef50 release 2026_03 cluster representatives (amended stratified draw, 355,626 residues). On held-out query halves the false-positive ratio (observed / expected) was 0.82 and 1.13 at E <= 1, 1.26 and 0.93 at E <= 0.1, and 2.7 and 2.2 at E <= 0.01; sensitivity at E <= 1e-3 was 0.15 and 0.20. FoldMatch's own ESM3 path reproduces the research codes on 99.5-99.6% of bits on GPU, CPU and Apple MPS (median |delta ln E| about 0.05, 0.049-0.051), and Mac and Linux CPUs agree on 99.997% of bits. ## Limitations - Sequence (FASTA) inputs only. The encoders were trained on sequence-only ESM3 embeddings; structure-conditioned embeddings are out of distribution, and FoldMatch's residue-code options are for sequence inputs only. - Calibrated domain: queries of 21-966 residues, targets up to 3,000 residues, at most 1% non-standard residues on either side and targets with at least 2 distinct codes. Outside it FoldMatch still reports the alignment but sets `ecal = 0` and warns. - Long queries: for queries of 1,000-3,000 residues the E-values are optimistic (about 2x at E <= 1, 38x at E <= 0.01 and 180x at E <= 1e-3 on an 8.6-million-residue database); such hits need E far below the threshold or independent confirmation. Long targets (1,000-3,000 residues) are calibrated (false-positive ratio 1.00 at E <= 1). - In general E <= 0.01 is about 2.5x optimistic while E <= 1 is about right; the panel and table are fixed, so any change is a new system name. - A query of one residue, or whose encoding rows are all identical, carries no information and reports no hits. - Results are bit-identical on one machine and BLAS build; across machines they agree within FoldMatch's tolerance contract (scores within 5e-5 relative, Q90 and ln E within 1e-4). - Only the ESM3 weights with sha256 `5ead5a135c658068...` are supported. - The codes are trained for alignment quality, not for homology detection; candidates come from FoldMatch's Stage-1 chain-embedding search, which bounds recall. ## Licence and attribution Copyright (c) 2026 - now, RCSB PDB and contributors. The encoders, the transform, the statistics table, the panel codes and this documentation are released under the MIT licence (see `LICENSE`), like FoldMatch. The panel member identifiers (`panel_ids`) are UniProt data and are not covered by the MIT licence: they stay under CC BY 4.0, with the attribution below. - UniRef50: Panel member identifiers are UniRef50 cluster identifiers (UniProt release 2026_03) from the UniProt Consortium, https://www.uniprot.org, distributed under the Creative Commons Attribution 4.0 International licence (CC BY 4.0, https://creativecommons.org/licenses/by/4.0/). The panel codes were computed from ESM3 embeddings of those representative sequences; the sequences themselves are not included. - ESM3: The encoders take per-residue embeddings of ESM3 esm3_sm_open_v1 (EvolutionaryScale) as input; no ESM3 weights are included. FoldMatch loads the weights from rcsb/rcsb-esm@636acb420daf674df37ab0220f56061a5de2f169 (MIT; weights sha256 5ead5a135c658068db6a4f1b933e72d6110992c4668822e1c0e2dcc53e38acd9), the same blob as EvolutionaryScale/esm3-sm-open-v1@47f0545b2b6daf26a93439a3cd610f4f7f3d5478, whose model card points to the MIT licence of the evolutionaryscale/esm repository. Older releases of the same weights (e.g. the gated EvolutionaryScale/esm3-sm-open-v1 card at 2feda652 and the LICENSE.md shipped in the esm 3.2.3 wheel) carry the EvolutionaryScale Cambrian Non-Commercial License; each copy of the weights is governed by the licence it was obtained under. ## Citation - FoldMatch: Segura, J., et al. (2026). Multi-scale structural similarity embedding search across entire proteomes. https://doi.org/10.1093/bioinformatics/btag058 - ESM3: Hayes, T., et al. (2025). Simulating 500 million years of evolution with a language model. Science 387, 850-858. https://doi.org/10.1126/science.ads0018 - UniProt: The UniProt Consortium (2025). UniProt: the Universal Protein Knowledgebase in 2025. Nucleic Acids Research 53(D1), D609-D617. https://doi.org/10.1093/nar/gkae1010 ## Verification Before writing these files the export script (`rcsb-embedding-search src/scripts/save_esm_codes_for_production.py`) checked that the encoders reload with `weights_only=True`, rebuild with `strict=True` and equal the research checkpoint, and that re-encoding FoldMatch's test fixtures gives the reference codes (F2 20,416/20,416 bits, F2c 19,392/19,392, F2b 22,976/22,976) and query encodings (20,416 of 20,416 F2 values bit-equal). With this panel and table the research kernel reproduces the 50 fixture pairs (50/50 paths; max |delta Q90| 0, max |delta ln E| 0) and 20 benchmark Q90s (max |delta| 0), and the scoring file loads with `allow_pickle=False` in a process without torch. These numbers come from the export machine (Apple silicon CPU). On other CPUs (checked on Linux x86-64) the codes are identical and the query encodings agree within one fp16 ulp of each row's largest value.