FoldMatch ESM3 residue codes (esm-codes-v1)

Encoders, statistics table and calibration panel for FoldMatch's ESM3 residue-code Stage-2 scoring. Every residue of a database sequence is stored as an 8-byte binary code computed from its ESM3 embedding; a query keeps a 64-dimensional fp16 encoding. Stage 2 aligns query and target with Smith-Waterman on the row/column-normalised code similarity and reports calibrated E-values. It is an alternative to BLOSUM62, trained so that its alignments reproduce structural (US-align) residue correspondences.

Intended use

These files are made for the ESM3 residue-code scoring that is being added to FoldMatch; FoldMatch's release notes will say from which version on it is available. That FoldMatch downloads the files from this repository at a pinned revision and checks their sha256s before use. It adds two options:

  • fm-search build sequences --residue-codes writes the per-residue codes next to the sequence database (8 bytes per residue, from the same ESM3 pass that computes the chain embeddings);
  • fm-search query sequences --scoring esm-codes scores the Stage-1 candidates with the codes (one scoring per query; BLOSUM62 scoring, --scoring blosum62, is unchanged).

The files are not meant to be used outside FoldMatch: the exact precision chain below is part of the method.

Inputs

  • ESM3 esm3_sm_open_v1, sequence-only, through rcsb/rcsb-esm@636acb420daf674df37ab0220f56061a5de2f169 (weights file data/weights/esm3_sm_open_v1.pth, sha256 5ead5a135c658068db6a4f1b933e72d6110992c4668822e1c0e2dcc53e38acd9, the same blob as EvolutionaryScale/esm3-sm-open-v1@47f0545b2b6daf26a93439a3cd610f4f7f3d5478).
  • Tap: per_residue_embedding from forward_and_sample(..., SamplingConfig(return_per_residue_embeddings=True)), i.e. the last block's output before the final LayerNorm; rows 1..L (BOS and EOS removed), [L, 1536].
  • Sequences are normalised first: ASCII whitespace removed, ASCII letters upper-cased, non-ASCII refused.

Method (system esm-codes-v1)

  1. Transform (zcw50_clean, not folded): x16 = fp16(x); xs = fp32(x16[:, dims]) - offset (dims = 0..1535 without 1361 and 1489); xw = fp32(fp16(xs @ M)); offset fp32[1534], M fp32[1534, 1534].
  2. Encoders f_q, f_t: f(x) = lin(x) + fc2(GELU_exact(fc1(x))), 1534 -> 1024 -> 64 with a linear skip, fp32 with no TF32, no autocast and no dropout (3,471,360 parameters in total).
  3. Target code: bit_k = (f_t(xw)_k >= 0), numpy.packbits(bits, axis=1) MSB-first -> uint8[L, 8]; decoded as 2 bit - 1.
  4. Query encoding: q = fp16(f_q(xw)), [L, 64].
  5. Scoring (setting D): cosine similarity of q and the decoded codes (fp32 GEMM per pair), row/column z-scores (population std; a row or column with std < 1e-6 contributes 0), Smith-Waterman on z - 1.0 with gap open 5.0 and extend 0.5 (DP id fmsw/1). Score S = S_f - S_r, where S_r aligns the row-reversed similarity.
  6. Statistics: S' = S - Q90(query), with Q90 the 0.9 quantile of S over a fixed panel of 1,000 UniRef50 sequences; ln E = lnK(m) + ln m + ln N - lambda(m) S' with lambda and lnK linear in ln m between six length bins and clamped outside them (m = query length, N = database residues).

Calibration version: foldmatch-evalue-M8_s0-D_fmr-pnlU50s1000-M3t999-v1.

Files

File Bytes sha256
esm_codes_v1_encoders.pt 23,324,662 1696ab91089ece8bce787d38b9d4d68bbc8ca4abff06f836f6e9363a23a1c02b
esm_codes_v1_scoring.npz 2,983,002 a5bebd626ec874626fb9372a63d10c32a001ed09c358717448bc9e1fd0ddedd6
bundle.json 10,830 dcd29bddbff0bb7dc93209bbe292d422d7bff21dd48c80f6e1f6b74f23789fe8
  • esm_codes_v1_encoders.pt: torch.load(path, weights_only=True). Format foldmatch-esm-codes-encoders/1; the transform, both state_dicts, the architecture, conventions and provenance.
  • esm_codes_v1_scoring.npz: numpy.load(path, allow_pickle=False). Format foldmatch-esm-codes-scoring/1; panel_codes uint8 [355626, 8], panel_offsets, panel_lengths, panel_ids (UniRef50), bin_centres, lam, lnK (float64[6]) and meta_json with every scoring constant.
  • bundle.json: system name, calibration version, the sha256s above, content digests (transform e8989b1cefc83978..., query encoder d8bed195068be23a..., target encoder 58da263e005a79fd..., panel 7aea2b00a24ee41c...), the ESM3 identity, the DP id, licence and attribution.
  • SHA256SUMS: shasum -a 256 -c SHA256SUMS checks every file.

Training and calibration

The encoders (research model L2m_B8_s0, epoch 7) were trained on sequence-only ESM3 embeddings of CATH domains to reproduce US-align residue correspondences (a cross-entropy on distance-filtered US-align targets, distillation from the full-precision whitened ESM3 similarity and a Smith-Waterman margin term), on a training pool that excludes every evaluation domain and its superfamily; the checkpoint and the Smith-Waterman setting were chosen on development pairs only. Smith-Waterman F1 against US-align at the shipped setting: 0.640 on validation related CATH pairs, 0.266 on validation remote pairs, 0.262 on test remote pairs and 0.613 on 582 similar multi-domain chain pairs (predicted 3Di at its own tuned setting: 0.604, 0.212, 0.203 and 0.536). E-values were fitted (censored Gumbel ML at q = 0.999 per query-length bin) on 1,465 SCOP-annotated benchmark queries against 50,000 targets (8.6 million residues), with the real null = targets that share no SCOP fold with the query; the per-query location Q90 comes from 1,000 UniRef50 release 2026_03 cluster representatives (amended stratified draw, 355,626 residues). On held-out query halves the false-positive ratio (observed / expected) was 0.82 and 1.13 at E <= 1, 1.26 and 0.93 at E <= 0.1, and 2.7 and 2.2 at E <= 0.01; sensitivity at E <= 1e-3 was 0.15 and 0.20. FoldMatch's own ESM3 path reproduces the research codes on 99.5-99.6% of bits on GPU, CPU and Apple MPS (median |delta ln E| about 0.05, 0.049-0.051), and Mac and Linux CPUs agree on 99.997% of bits.

Limitations

  • Sequence (FASTA) inputs only. The encoders were trained on sequence-only ESM3 embeddings; structure-conditioned embeddings are out of distribution, and FoldMatch's residue-code options are for sequence inputs only.
  • Calibrated domain: queries of 21-966 residues, targets up to 3,000 residues, at most 1% non-standard residues on either side and targets with at least 2 distinct codes. Outside it FoldMatch still reports the alignment but sets ecal = 0 and warns.
  • Long queries: for queries of 1,000-3,000 residues the E-values are optimistic (about 2x at E <= 1, 38x at E <= 0.01 and 180x at E <= 1e-3 on an 8.6-million-residue database); such hits need E far below the threshold or independent confirmation. Long targets (1,000-3,000 residues) are calibrated (false-positive ratio 1.00 at E <= 1).
  • In general E <= 0.01 is about 2.5x optimistic while E <= 1 is about right; the panel and table are fixed, so any change is a new system name.
  • A query of one residue, or whose encoding rows are all identical, carries no information and reports no hits.
  • Results are bit-identical on one machine and BLAS build; across machines they agree within FoldMatch's tolerance contract (scores within 5e-5 relative, Q90 and ln E within 1e-4).
  • Only the ESM3 weights with sha256 5ead5a135c658068... are supported.
  • The codes are trained for alignment quality, not for homology detection; candidates come from FoldMatch's Stage-1 chain-embedding search, which bounds recall.

Licence and attribution

Copyright (c) 2026 - now, RCSB PDB and contributors. The encoders, the transform, the statistics table, the panel codes and this documentation are released under the MIT licence (see LICENSE), like FoldMatch. The panel member identifiers (panel_ids) are UniProt data and are not covered by the MIT licence: they stay under CC BY 4.0, with the attribution below.

  • UniRef50: Panel member identifiers are UniRef50 cluster identifiers (UniProt release 2026_03) from the UniProt Consortium, https://www.uniprot.org, distributed under the Creative Commons Attribution 4.0 International licence (CC BY 4.0, https://creativecommons.org/licenses/by/4.0/). The panel codes were computed from ESM3 embeddings of those representative sequences; the sequences themselves are not included.
  • ESM3: The encoders take per-residue embeddings of ESM3 esm3_sm_open_v1 (EvolutionaryScale) as input; no ESM3 weights are included. FoldMatch loads the weights from rcsb/rcsb-esm@636acb420daf674df37ab0220f56061a5de2f169 (MIT; weights sha256 5ead5a135c658068db6a4f1b933e72d6110992c4668822e1c0e2dcc53e38acd9), the same blob as EvolutionaryScale/esm3-sm-open-v1@47f0545b2b6daf26a93439a3cd610f4f7f3d5478, whose model card points to the MIT licence of the evolutionaryscale/esm repository. Older releases of the same weights (e.g. the gated EvolutionaryScale/esm3-sm-open-v1 card at 2feda652 and the LICENSE.md shipped in the esm 3.2.3 wheel) carry the EvolutionaryScale Cambrian Non-Commercial License; each copy of the weights is governed by the licence it was obtained under.

Citation

Verification

Before writing these files the export script (rcsb-embedding-search src/scripts/save_esm_codes_for_production.py) checked that the encoders reload with weights_only=True, rebuild with strict=True and equal the research checkpoint, and that re-encoding FoldMatch's test fixtures gives the reference codes (F2 20,416/20,416 bits, F2c 19,392/19,392, F2b 22,976/22,976) and query encodings (20,416 of 20,416 F2 values bit-equal). With this panel and table the research kernel reproduces the 50 fixture pairs (50/50 paths; max |delta Q90| 0, max |delta ln E| 0) and 20 benchmark Q90s (max |delta| 0), and the scoring file loads with allow_pickle=False in a process without torch. These numbers come from the export machine (Apple silicon CPU). On other CPUs (checked on Linux x86-64) the codes are identical and the query encodings agree within one fp16 ulp of each row's largest value.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support