Anonymoususer2223
/

ProtCompass_Embeddings

@@ -1,176 +1,402 @@
 ---
 license: mit
 ---
-# ProtCompass Embeddings
-Pre-computed protein embeddings from 70+ encoders across 15 downstream tasks, plus probing results and evaluation outputs.
-## Dataset Structure
-```
-embeddings/                          # Compressed embeddings (~150GB)
-├── contact_prediction/              # Per-encoder compressed (300GB → ~60GB)
-│   ├── esm2.tar.gz
-│   ├── gearnet.tar.gz
-│   └── ... (36 encoders)
-├── secondary_structure/             # Per-encoder compressed (129GB → ~30GB)
-├── ppi_site/                        # Per-encoder compressed (80GB → ~20GB)
-├── metal_binding/                   # Per-encoder compressed (41GB → ~10GB)
-├── mutation_effect.tar.gz           # Per-task compressed (27GB → ~7GB)
-├── go_bp.tar.gz                     # Per-task compressed (7.9GB → ~2GB)
-├── stability.tar.gz                 # Per-task compressed (4.1GB → ~1GB)
-├── solubility.tar.gz                # Per-task compressed (3.6GB → ~900MB)
-├── go_mf.tar.gz                     # Per-task compressed (3.1GB → ~800MB)
-├── fluorescence.tar.gz              # Per-task compressed (3.0GB → ~800MB)
-├── ec_classification.tar.gz         # Per-task compressed (1.9GB → ~500MB)
-├── subcellular_localization.tar.gz  # Per-task compressed (1.4GB → ~400MB)
-├── membrane_soluble.tar.gz          # Per-task compressed (1.4GB → ~400MB)
-├── remote_homology.tar.gz           # Per-task compressed (805MB → ~200MB)
-└── ppi_affinity.tar.gz              # Per-task compressed (169MB → ~50MB)
-probing_IF/                          # Probing results (2.8GB)
-├── probing_embeddings/              # Invariant family embeddings (12 encoders)
-└── probing_results_architecture_full/  # Full probing results (195 files)
-results/                             # Evaluation results (6.9MB)
-└── {encoder}/{task}/                # Per-encoder, per-task results
-outputs/                             # Analysis outputs (12MB)
-├── alignment_analysis/              # Alignment analysis figures
-├── paper_figures_v12/               # Final paper figures
-└── uncertainty_appendix/            # Uncertainty analysis
-```
-## Decompression Instructions
-All embeddings are compressed with gzip. Decompress before use:
-### Large Tasks (per-encoder compression)
-For `contact_prediction`, `secondary_structure`, `ppi_site`, `metal_binding`:
-```bash
-# Decompress all encoders in a task
-cd embeddings/contact_prediction/
-for f in *.tar.gz; do tar -xzf "$f"; done
-# Or decompress specific encoder
-tar -xzf esm2.tar.gz
-```
-### Medium/Small Tasks (per-task compression)
-For all other tasks:
-```bash
-# Decompress entire task
-cd embeddings/
-tar -xzf mutation_effect.tar.gz
-tar -xzf secondary_structure.tar.gz
-# etc.
-# Or decompress all tasks at once
-for f in *.tar.gz; do tar -xzf "$f"; done
-```
-## File Format
-After decompression, each encoder directory contains:
-- `train_embeddings.npy`: Training set embeddings (N × D)
-- `test_embeddings.npy`: Test set embeddings (M × D)
-- `train_labels.npy`: Training labels
-- `test_labels.npy`: Test labels
-- `train_ids.txt`: Protein IDs for training set
-- `test_ids.txt`: Protein IDs for test set
-- `meta.json`: Metadata (encoder name, dimensions, dataset info)
-## Usage
 ```python
 import numpy as np
-from huggingface_hub import hf_hub_download
-import tarfile
-# Download and decompress embeddings
-tar_path = hf_hub_download(
-    repo_id="Anonymoususer2223/ProtCompass_Embeddings",
-    filename="embeddings/mutation_effect.tar.gz",
-    repo_type="dataset"
-)
-# Extract
-with tarfile.open(tar_path, 'r:gz') as tar:
-    tar.extractall(path="./embeddings/")
-# Load embeddings
-train_emb = np.load("embeddings/mutation_effect/esm2/train_embeddings.npy")
-test_emb = np.load("embeddings/mutation_effect/esm2/test_embeddings.npy")
-train_labels = np.load("embeddings/mutation_effect/esm2/train_labels.npy")
-test_labels = np.load("embeddings/mutation_effect/esm2/test_labels.npy")
-# Use for downstream tasks
-from sklearn.linear_model import Ridge
-model = Ridge()
-model.fit(train_emb, train_labels)
-score = model.score(test_emb, test_labels)
-print(f"Test R²: {score:.3f}")
 ```
-## Encoders Included
-### Sequence Encoders (8)
-ESM2 (650M, 150M, 35M), ESM1b, ESM3, ProtTrans, ProstT5, ProteinBERT-BFD, Ankh
-### Structure Encoders (50+)
-GearNet, GCPNet, EGNN, GVP, IPA, TFN, SchNet, DimeNet, MACE, CDConv, ProteinMPNN, PottsMPNN, dMaSIF, and more
-### Multimodal Encoders (5)
-SaProt, ESM-IF, FoldVision
-### Baselines (5)
-Random, Length, Torsion, One-hot, BLOSUM
-## Dataset Statistics
-- **Compressed size**: ~150GB
-- **Uncompressed size**: ~600GB
-- **Total encoders**: 70+
-- **Total tasks**: 15
-- **Total proteins**: ~500K across all tasks
-- **Compression ratio**: ~4x (gzip)
-## Compression Details
-- **Large tasks** (>30GB): Per-encoder compression for flexibility
-  - Users can download only specific encoders
-  - Enables parallel decompression
-- **Medium/Small tasks** (<30GB): Per-task compression
-  - Single archive per task
-  - Faster download for complete task data
-## Citation
-If you use these embeddings, please cite:
-```bibtex
-@article{protcompass2026,
-  title={ProtCompass: Interpretable Benchmarking and Task-Aware Evaluation of Protein Encoders},
-  author={Your Name et al.},
-  journal={NeurIPS},
-  year={2026}
-}
 ```
-## Related Resources
-- **Code Repository**: [GitHub](https://github.com/yourusername/protcompass)
-- **Raw Datasets**: [ProtEnv on HuggingFace](https://huggingface.co/datasets/Anonymoususer2223/ProtEnv)
-- **Paper**: [arXiv](https://arxiv.org/abs/xxxx.xxxxx)
-## License
-MIT License
-## Contact
-For questions or issues, please open an issue on the [GitHub repository](https://github.com/yourusername/protcompass).

 ---
 license: mit
+tags:
+- biology
+- protein
+- protein-embeddings
+- benchmark
 ---
+# ProtCompass embeddings
+Frozen protein embeddings, input-audit records and label files for the paper
+**ProtCompass: Biological Information and Predictive Utility in Protein Representations**
+(anonymous submission to ICLR 2027).
+Protein encoders are usually evaluated by structural probing, downstream prediction and concept erasure. These
+measurements can mix signal learned by the encoder with information that is already present in the input or that the
+evaluation adds. ProtCompass separates these sources. It audits which inputs each encoder actually reads, compares
+structural probe scores with non-learned descriptions of the same inputs, compares downstream scores with simple
+sequence baselines, and controls concept-erasure experiments for the representation capacity that is removed together
+with the target property. The paper evaluates 33 encoders on a probing suite of 21 probes built from CATH 4.3 domains
+and on ten downstream tasks.
+- Code, result tables and analysis scripts: <https://anonymous.4open.science/r/ProEmbedEva-C4F6>
+- Predicted structures for four downstream tasks: <https://huggingface.co/datasets/Anonymoususer2223/ProtEnv>
+This repository holds the release that backs the paper: embeddings, labels, the input audit and the pretrained
+weights of the structure encoders. Its paths mirror the working directory of the code (for example
+`extract/embeddings/<task>/<encoder>/` and `checkpoints/`). An earlier release was removed; see
+[Earlier release (removed)](#earlier-release-removed).
+## Contents
+Sizes are in GB (10^9 bytes).
+| Path | Content | Files | Size | Paper appendix |
+|---|---|---:|---:|---|
+| `extract/embeddings/<task>/<encoder>/` | pooled embeddings of 33 encoders and one non-learned reference on the ten downstream tasks | 2,237 | 6.5 | Tasks and protocols (The downstream tasks); Controls for the downstream comparison; Controls for the erasure analysis; Seed variability |
+| `extract/manifest/<task>.csv` | one row per protein: id, split, sequence, structure file, label | 11 | 0.12 | Tasks and protocols (The downstream tasks) |
+| `extract/pairs/ppi_affinity_pairs.csv` | chain pairs and labels of the PPI affinity task | 1 | <0.01 | Tasks and protocols (The downstream tasks) |
+| `extract/encoder_input_audit.csv`, `encoder_atom_audit.csv`, `encoder_sidechain_audit.csv` | measured input audit | 3 | <0.01 | Study Design; The Encoder Panel |
+| `probing_embeddings/<encoder>/` | per-residue and pooled embeddings of the 33 encoders on 19,461 CATH 4.3 domains | 101 | 96.9 | Tasks and protocols (Probe protocol); Controls for the probing comparison; Two residue-level tasks, and what the pooled readout hides |
+| `probing_embeddings_vframe/<encoder>_vframe/` | GVP, GCPNet (random and pretrained) and MACE with the direction of their vector channels kept | 19 | 5.2 | Controls for the probing comparison (Equivariant encoders with the direction of their vector channels kept) |
+| `probing_data/labels.h5`, `probing_data/splits/` | probe labels, coordinates and splits of the CATH domains | 8 | 0.32 | Tasks and protocols (The probing suite; Probe protocol) |
+| `extract/ab_modality/embeddings/`, `extract/ab_modality/probing_embeddings/` | ESM3 and SaProt with the structure input removed, and SaProt with shuffled 3Di tokens | 210 | 24.9 | Controls for the downstream comparison (Removing the structure input from the same checkpoint) |
+| `extract/ab_modality/encoder_input_audit_ab*.csv` | input audit of those three sources | 2 | <0.01 | same |
+| `confirmation/`, `confirmation2/` | probing cohorts A and B: CATH 4.4 domains absent from CATH 4.3 | 52 | 6.3 | Confirmation on data and splits the development did not use (The probing headline on CATH 4.4 domains absent from CATH 4.3) |
+| `confirmation_downstream/` | held-out downstream proteins never used in development | 280 | 0.54 | Confirmation on data and splits the development did not use (The demand results on downstream proteins never scored) |
+| `masif_site/` | residue-level interface task rebuilt from MaSIF-site | 90 | 20.2 | Two residue-level tasks, and what the pooled readout hides (Interface residues) |
+| `atlas/data_v4/` | latent-space atlas: 2D maps, k-NN purity and CKA for every task layer | 12 | 0.06 | Latent-Space Atlas |
+| `checkpoints/<encoder>/` | pretrained weights of six structure encoders, with licences and `SHA256SUMS` | 14 | 0.16 | The Encoder Panel |
+The current release has 3,040 files and 161.2 GB in total. The appendix names are section and subsection titles of
+the paper.
+## Encoders
+The paper evaluates 33 encoders. They are declared as 9 sequence-based, 19 structure-based and 5 multimodal models.
+The analyses group them by a measured input audit, which finds that 10 encoders do not read coordinates
+(sequence-only) and 23 do (coordinate-reading). The only encoder whose measured group differs from its declared class
+is ProstT5, which is declared multimodal and read on its amino-acid track only.
+The directory name is the key used in every path. *Width* is the width of the stored array. *Atoms read* comes from
+the atom audit (`extract/encoder_atom_audit.csv`); "side chain" marks the three encoders that the side-chain audit
+(`extract/encoder_sidechain_audit.csv`) finds to read atoms beyond CB.
+| Directory | Paper name | Declared class | Weights | Width | Audit group | Reads residue identity | Atoms read |
+|---|---|---|---|---:|---|---|---|
+| `ankh_base` | Ankh | sequence | pretrained | 768 | sequence-only | yes | - |
+| `esm1b` | ESM-1b | sequence | pretrained | 1280 | sequence-only | yes | - |
+| `esm2_8m` | ESM2-8M | sequence | pretrained | 320 | sequence-only | yes | - |
+| `esm2_35m` | ESM2-35M | sequence | pretrained | 480 | sequence-only | yes | - |
+| `esm2_150m` | ESM2-150M | sequence | pretrained | 640 | sequence-only | yes | - |
+| `esm2_650m` | ESM2-650M | sequence | pretrained | 1280 | sequence-only | yes | - |
+| `esm2_3b` | ESM2-3B | sequence | pretrained | 2560 | sequence-only | yes | - |
+| `protbert_bfd` | ProtBert | sequence | pretrained | 1024 | sequence-only | yes | - |
+| `prottrans_t5_xl_u50` | ProtT5-XL | sequence | pretrained | 1024 | sequence-only | yes | - |
+| `prostt5` | ProstT5 | multimodal | pretrained | 1024 | sequence-only | yes | - |
+| `dimenet` | DimeNet | structure | random | 128 | coordinate-reading | yes | CA |
+| `esm_if1` | ESM-IF1 | structure | pretrained | 512 | coordinate-reading | no | N, CA, C |
+| `foldseek_3di` | Foldseek | structure | pretrained (tokeniser, no learned parameters) | 20 | coordinate-reading | no | N, CA, CB |
+| `gcpnet_random` | GCPNet | structure | random | 116 | coordinate-reading | yes | N, CA, C, O, CB, side chain |
+| `gcpnet_pretrained` | GCPNet-pre | structure | pretrained | 116 | coordinate-reading | yes | N, CA, C, O, CB, side chain |
+| `gvp` | GVP | structure | random | 116 | coordinate-reading | yes | N, CA, C |
+| `gearnet` | GearNet | structure | random | 512 | coordinate-reading | yes | CA |
+| `gearnet_pretrained` | GearNet-pre | structure | pretrained | 512 | coordinate-reading | yes | CA |
+| `ipa` | IPA | structure | random | 128 | coordinate-reading | yes | CA |
+| `ipa_pretrained` | IPA-pre | structure | pretrained | 128 | coordinate-reading | yes | CA |
+| `mace` | MACE | structure | random | 128 | coordinate-reading | yes | CA |
+| `pottsmpnn` | PottsMPNN | structure | random | 128 | coordinate-reading | no | N, CA, C, O |
+| `pottsmpnn_ft` | PottsMPNN-ft | structure | pretrained | 128 | coordinate-reading | no | N, CA, C, O |
+| `pottsmpnn_pretrained` | PottsMPNN-pre | structure | pretrained | 128 | coordinate-reading | no | N, CA, C, O |
+| `pottsmpnn_soluble` | PottsMPNN-sol | structure | pretrained | 128 | coordinate-reading | no | N, CA, C, O |
+| `proteinmpnn` | ProteinMPNN | structure | pretrained | 128 | coordinate-reading | no | N, CA, C, O |
+| `schnet` | SchNet | structure | random | 128 | coordinate-reading | yes | CA |
+| `tfn` | TFN | structure | random | 128 | coordinate-reading | yes | CA |
+| `tfn_pretrained` | TFN-pre | structure | pretrained | 128 | coordinate-reading | yes | CA |
+| `cdconv` | CDConv | multimodal | random | 256 | coordinate-reading | yes | CA |
+| `esm3` | ESM3 | multimodal | pretrained | 1536 | coordinate-reading | yes | N, CA, C |
+| `foldvision` | FoldVision | multimodal | pretrained | 1024 | coordinate-reading | yes | N, CA, C, O, CB, side chain |
+| `saprot` | SaProt | multimodal | pretrained | 1280 | coordinate-reading | yes | N, CA, CB |
+Randomly initialised encoders use one fixed seed for all proteins; the hash of their weights is recorded in
+`meta.json`. Three further encoders were audited and are not released: EGNN (its embeddings do not change when the
+coordinates change), GeoBPE (its output changes under a rigid motion) and dMaSIF (all-zero embeddings). FoldVision has
+no extraction on remote homology, so it is scored on nine tasks.
+### Which layer is read
+Every embedding is taken from the last representation the encoder computes before any task head or decoder. Where an
+encoder has residue states, a protein is the masked mean of its residue states. The rule of each encoder is recorded in
+the `rule` field of its `meta.json`.
+| Encoders | Representation read |
+|---|---|
+| Ankh, ESM-1b, ESM2 (all sizes), ProtBert, ProtT5-XL | final hidden state, special tokens removed |
+| ProstT5 | final hidden state of the amino-acid track with the `<AA2fold>` prefix, special tokens removed; no 3Di input |
+| ESM3 | final trunk residue embeddings, sequence and structure tracks |
+| SaProt | final hidden states at the residue positions of the combined amino-acid and 3Di tokens |
+| SchNet, TFN, DimeNet, MACE, GearNet, CDConv, IPA (random and pretrained versions) | final invariant node states; IPA reads the structure in a canonical pose |
+| GVP, GCPNet (random and pretrained) | final scalar channels plus the norm of each vector channel (100 + 16 = 116) |
+| ProteinMPNN, PottsMPNN (all four) | encoder output h_V, before the decoder |
+| ESM-IF1 | output of the GVP-Transformer encoder, before decoding |
+| Foldseek | one-hot 3Di state per residue; the protein vector is the 3Di state frequency |
+| FoldVision | the model's global embedding; no residue states |
+## Pretrained weights
+`checkpoints/` holds the local checkpoint files that six of the released structure encoders read. Every file is
+byte-identical to its upstream release, and loading it with the project's wrapper on CPU reproduces the weight hash
+(`weights_sha256`) recorded in the `meta.json` of the released embeddings. The upstream licence is stored next to each
+file, and `checkpoints/SHA256SUMS` lists the full hashes. `checkpoints/README.md` has a script that links the files
+to the paths the code reads.
+| Encoder | File in this repository | Size (MB) | SHA-256 (first 16) | Source | Licence | Path the code reads |
+|---|---|---:|---|---|---|---|
+| ProteinMPNN | `checkpoints/proteinmpnn/proteinmpnn_v48_020.pt` | 6.7 | `c9cb4a671d796041` | ProteinMPNN repository, `vanilla_model_weights/v_48_020.pt` | MIT | `protcompass/checkpoints/proteinmpnn_v48_020.pt` |
+| GearNet-pre | `checkpoints/gearnet_pretrained/mc_gearnet_edge.pth` | 80.7 | `6dc11e2db6eff3c9` | Zenodo, DOI 10.5281/zenodo.7593637 | CC BY 4.0 | `protcompass/checkpoints/gearnet/mc_gearnet_edge.pth` |
+| GCPNet-pre (GCPNet reads its hyper-parameters) | `checkpoints/gcpnet_pretrained/model_epoch_115_..._globalkendall_0_676.ckpt` | 14.3 | `ff7b73be01fb57d5` | GCPNet repository, `checkpoints/PSR/` (PSR task model) | MIT | `protcompass/external_repos/GCPNet/checkpoints/PSR/` |
+| PottsMPNN-pre | `checkpoints/pottsmpnn_pretrained/pottsmpnn_20.pt` | 20.5 | `28b1620a71883142` | PottsMPNN repository, `proteinmpnn_compatible_model_weights/` | MIT | `protcompass/external_repos/PottsMPNN/proteinmpnn_compatible_model_weights/pottsmpnn_20.pt` |
+| PottsMPNN-sol | `checkpoints/pottsmpnn_soluble/sol_pottsmpnn_20.pt` | 20.7 | `f203a69773df41a1` | PottsMPNN repository, `soluble_model_weights/` | MIT | `protcompass/external_repos/PottsMPNN/soluble_model_weights/sol_pottsmpnn_20.pt` |
+| PottsMPNN-ft | `checkpoints/pottsmpnn_ft/potts_ft.pt` | 20.7 | `b672ff4ed1492560` | PottsMPNN repository, `ft_model_weights/` | MIT | `protcompass/external_repos/PottsMPNN/ft_model_weights/potts_ft.pt` |
+The code paths are relative to the root of the code repository, whose `protcompass/` folder mirrors this repository.
+The upstream repositories are github.com/dauparas/ProteinMPNN (commit 8907e66), github.com/BioinfoMachineLearning/GCPNet
+(commit 172733b) and github.com/KeatingLab/PottsMPNN (commit a6ccac3); a clone of GCPNet or PottsMPNN at that commit
+already holds its files at the paths the code reads.
+Weights loaded from a public source, which this repository does not copy:
+| Encoder | Source | Revision |
+|---|---|---|
+| ESM-1b | `facebook/esm1b_t33_650M_UR50S` | `7b37824baec4d3658e1df7479222a7c79b465b76` |
+| ESM2-8M | `facebook/esm2_t6_8M_UR50D` | `c731040fcd8d73dceaa04b0a8e6329b345b0f5df` |
+| ESM2-35M | `facebook/esm2_t12_35M_UR50D` | `6fbf070e65b0b7291e7bbcd451118c216cff79d8` |
+| ESM2-150M | `facebook/esm2_t30_150M_UR50D` | `a695f6045e2e32885fa60af20c13cb35398ce30c` |
+| ESM2-650M | `facebook/esm2_t33_650M_UR50D` | `08e4846e537177426273712802403f7ba8261b6c` |
+| ESM2-3B | `facebook/esm2_t36_3B_UR50D` | `476b639933c8baad5ad09a60ac1a87f987b656fc` |
+| ProtBert | `Rostlab/prot_bert_bfd` | `6c5c8a55a52ff08a664dfd584aa1773f125a0487` |
+| ProtT5-XL | `Rostlab/prot_t5_xl_half_uniref50-enc` | `94a6abc029ae13029317b140b7424e012bf8dfbf` |
+| Ankh | `ElnaggarLab/ankh-base` | `d99cb6b966530dfc2ae96bc69d9255c2a07308b0` |
+| ProstT5 | `Rostlab/ProstT5` | `d7d097d5bf9a993ab8f68488b4681d6ca70db9e5` |
+| SaProt | `westlake-repl/SaProt_650M_AF2` | `d9b9ad00ef61c0990e611b2b43f2231c7de24b38` |
+| FoldVision | `AlexanderKroll/foldvision-encoder` | not recorded |
+| ESM-IF1 | `torch.hub.load("facebookresearch/esm:main", "esm_if1_gvp4_t16_142M_UR50")` (ESM release, MIT) | not recorded |
+| Foldseek 3Di | the `mini3di` package, version 0.2.1 (BSD-3-Clause); no weights file | 0.2.1 |
+The first twelve are Hugging Face hub ids; the revisions are those recorded in the `checkpoint_revision` field of each
+`meta.json`.
+- **ESM3.** The `esm3-sm-open-v1` weights are distributed by EvolutionaryScale under the Cambrian Non-Commercial
+  License Agreement, which does not allow redistribution, so they are not in this repository. Accept the licence and
+  download them from the gated Hugging Face repository `EvolutionaryScale/esm3-sm-open-v1`. The code expects them under
+  `protcompass/esm3-sm-open-v1/` (weights file `data/weights/esm3_sm_open_v1.pth`).
+- **TFN-pre and IPA-pre.** Their weights were taken from the models' released repositories and are not redistributed
+  here. The `meta.json` of their embeddings records the SHA-256 prefix of the file that was used (`17af58de61a4674c`
+  for TFN-pre, `cd4ca8e7f61daba2` for IPA-pre). The code expects them at `protcompass/checkpoints/tfn_pretrained/best.pt`
+  and `protcompass/checkpoints/ipa_pretrained/best.pt`.
+- **Randomly initialised encoders** (SchNet, TFN, DimeNet, MACE, GearNet, IPA, GVP, GCPNet, CDConv, PottsMPNN) have no
+  weights file. They are built with the fixed seed 20260919, and the hash of the resulting weights is in each
+  `meta.json`.
+## Downstream tasks
+Fourteen tasks were assembled. Four (contact prediction, mutation effect, PPI sites and metal binding) were excluded
+because their labels or metric were unsuitable (appendix "Tasks and analyses the paper evaluated and does not use"),
+and ten are scored. Every encoder reads the same rows of a task. For
+each task the embeddings cover the rows of the manifest with `stage1 == 1`.
+| Directory | Task | Metric | Source dataset | Structures | train / valid / test |
+|---|---|---|---|---|---|
+| `ec_classification` | EC classification | accuracy | ProteinWorkshop | experimental (PDB) | 6,000 / 1,500 / 3,000 |
+| `go_mf` | GO molecular function | F_max | ProteinWorkshop | experimental (PDB) | 6,000 / 1,500 / 2,904 |
+| `go_bp` | GO biological process | F_max | ProteinWorkshop | experimental (PDB) | 6,000 / 1,500 / 2,903 |
+| `remote_homology` | remote homology (fold) | accuracy | TAPE | experimental (PDB) | 6,000 / 506 / 480 |
+| `subcellular_localization` | localisation, 10 classes | accuracy | DeepLoc 2 | predicted (AlphaFold DB) | 6,000 / 1,500 / 2,473 |
+| `membrane_soluble` | membrane versus soluble | AUROC | DeepLoc 2 | predicted (AlphaFold DB) | 6,000 / 1,500 / 3,000 |
+| `stability` | stability | Spearman | TAPE | predicted (ESMFold) | 6,000 / 1,500 / 3,000 |
+| `fluorescence` | fluorescence | Spearman | TAPE | predicted (ESMFold) | 6,000 / 1,500 / 3,000 |
+| `solubility` | solubility | AUROC | FLIP | predicted (ESMFold) | 5,417 / 1,363 / 1,329 |
+| `ppi_affinity` | PPI affinity | Spearman | SKEMPI2 | predicted (ESMFold) | 2,589 chains; 2,352 / 194 / 326 pairs |
+The structure sources and split sizes are those of the paper's table "Structures and splits of the ten scored tasks".
+Every extracted row has a structure file whose sequence equals the task sequence (`structure_coverage` in `meta.json`).
+## File formats
+### Downstream embeddings: `extract/embeddings/<task>/<encoder>/`
+| File | Content |
+|---|---|
+| `train.npy`, `valid.npy`, `test.npy` | float32 array of shape (number of proteins in the split, width) |
+| `ids_train.txt`, `ids_valid.txt`, `ids_test.txt` | one protein id per line; line i is row i of the array of the same split |
+| `meta.json` | extraction record (fields below) |
+| `chains.npy`, `ids_chains.txt` | PPI affinity only: one row per chain, in place of the three splits |
+- Row order equals the order of the manifest rows with `stage1 == 1` and the same `split`. The id files are identical
+  for every encoder of a task.
+- Labels are in the `label` column of `extract/manifest/<task>.csv`: an integer class index (EC, localisation), a
+  fold index (remote homology), a list of term indices written as a string (GO-MF, GO-BP), 0 or 1 (solubility,
+  membrane versus soluble) or a real value (stability, fluorescence).
+- PPI affinity: `extract/pairs/ppi_affinity_pairs.csv` has columns `id, split, chain_a, chain_b, label`. The paper
+  represents a pair by concatenating the embeddings of `chain_a` and `chain_b` in that order.
+- `ref_composition/` in every task directory is not an encoder. It is the non-learned composition reference of the
+  paper: 20 amino-acid frequencies, log length, length/600 and the fraction of residues outside the 20-letter
+  alphabet (23 values).
+- `extract/manifest/<task>.csv` has columns `id, split, sequence, structure_path, length, label, stage1`, plus
+  `align_path, struct_coverage` (experimental and AlphaFold DB structures) or `mean_plddt, chain_break` (ESMFold
+  structures).
+Fields of `meta.json` (not every field is present for every encoder):
+| Field | Meaning |
+|---|---|
+| `encoder`, `task` | directory keys |
+| `family`, `target`, `patch`, `env` | how the encoder was loaded: a Hugging Face language model (`hf_plm`), a model wrapper (`registry`) or a patched official implementation (`patched`) |
+| `weights` | `pretrained`, `random` or `none` |
+| `checkpoint`, `checkpoint_revision`, `checkpoint_sha256`, `checkpoint_loaded_by`, `weights_sha256`, `seed` | the weights used. Hub checkpoints give the hub id and revision; local checkpoint paths are given relative to the project directory |
+| `rule` | which representation is read (see [Which layer is read](#which-layer-is-read)) |
+| `pooling` | how residue states become one protein vector (masked mean over all residues of the chain) |
+| `inputs_passed`, `pose` | what the encoder receives |
+| `dtype`, `hidden_size`, `n_layers`, `n_parameters` | model details |
+| `manifest`, `rows`, `structure_coverage`, `n_per_split` | which manifest rows were extracted |
+| `health` | `n_rows`, `dim`, `nan_values`, `all_zero_rows`, `distinct_rows`, `numerical_rank`, `max_abs` of the stored array |
+| `input_audit` | relative change of the embedding under a 1 Å coordinate jitter, a coordinate swap, a sequence shuffle and a rigid motion, and the audit verdict |
+| `date`, `seconds` | when and how long |
+### Probing embeddings: `probing_embeddings/`
+The probing set is 19,461 CATH 4.3 domains with 2,823,525 residues. The same row convention holds for every probing
+directory in this repository.
+| File | Content |
+|---|---|
+| `ids.txt` | CATH domain id per line; line k is domain k |
+| `offsets.npy`, `lengths.npy` | int64, shape (19461,); the residues of domain k are rows `offsets[k]` to `offsets[k] + lengths[k] - 1` |
+| `<encoder>/residue_fp16.npy` | float16, shape (2823525, width); row r is row r of `residue/*` in `probing_data/labels.h5` |
+| `<encoder>/pooled_fp32.npy` | float32, shape (19461, width); row k is domain k (mean over its residues) |
+| `<encoder>/meta.json` | extraction record; its `row_convention` or `rows` field states the row order |
+FoldVision has no residue states, so `probing_embeddings/foldvision/` holds `pooled_fp32.npy` only, and only the
+protein-level probes apply to it. Language models were run on the sequence of the residues kept in `labels.h5`; a
+discontinuous CATH domain is passed as one concatenated sequence.
+`probing_data/labels.h5` holds three groups. `protein/` has one row per domain (id, length, CATH labels, split codes,
+protein-level targets). `residue/` has one row per residue (amino acid, N, CA, C, O and CB coordinates with CB built
+from ideal geometry where it is absent, DSSP 3- and 8-state, relative solvent accessibility, B-factor, backbone angles, contact counts and other residue labels). `pair/`
+has the sampled residue pairs with their distances. The split codes are `split_superfamily` (15,565 train, 1,948
+validation, 1,948 test; no superfamily is shared between splits) and `split_topology` (15,378 / 2,159 / 1,924; the
+topology-disjoint split of the confirmation experiments), with 0 = train, 1 = validation and 2 = test.
+`probing_data/splits/` lists the same splits as domain ids. The CATH domain structure files are public and are not
+redistributed here.
+`probing_embeddings_vframe/<encoder>_vframe/` follows the same layout. It stores the scalar channels together with the
+vector channels projected onto a per-residue frame built from N, CA and C (width 148 for GVP and the two GCPNets, 176
+for MACE). The two GCPNet directories also hold `residue_atommean_fp16.npy` and `pooled_atommean_fp32.npy`, the same
+representation averaged over the heavy atoms of each residue. The frame definition is in each `meta.json`.
+### Modality ablation: `extract/ab_modality/`
+Three extra sources hold the weights fixed and change only the input.
+| Source | Input |
+|---|---|
+| `esm3_seqonly` | ESM3 with the true sequence and the mask token at every structure position (no coordinates) |
+| `saprot_seqonly` | SaProt with every 3Di token replaced by `#` (the token for a residue without structure) |
+| `saprot_shuf3di` | SaProt with each protein's own 3Di tokens permuted within the protein (seed from the protein id) |
+`extract/ab_modality/embeddings/<task>/<source>/` follows the downstream format for all ten tasks, and
+`extract/ab_modality/probing_embeddings/<source>/` follows the probing format. The audit rows of these sources are in
+`extract/ab_modality/encoder_input_audit_ab.csv` and `encoder_input_audit_ab_shuf.csv`. The full-input ESM3 and
+SaProt embeddings they are compared with are the `esm3` and `saprot` entries of the main release.
+### Confirmation sets
+| Path | Content |
+|---|---|
+| `confirmation/` | cohort A: 1,990 CATH v4.4 S40 domains absent from CATH v4.3. `embeddings/<encoder>/` in the probing format for Ankh, ESM2-650M, ESM3, ESM-IF1, ProteinMPNN and ProtT5-XL; `labels_v44.h5` in the schema of `labels.h5`; `cohort_manifest.csv` with the CATH class, architecture, topology and superfamily of each domain |
+| `confirmation2/` | cohort B: 1,316 further CATH v4.4 domains, same layout, the six encoders of cohort A plus both GCPNets |
+| `confirmation_downstream/` | eight held-out downstream cohorts, 13,632 proteins taken from manifest rows that were never extracted or scored in development. `embeddings/<task>/<encoder>/` holds `test.npy`, `ids_test.txt` and `meta.json`; `ids_<task>.txt` and `cohort_<task>.csv` list the proteins and labels. Solubility and PPI affinity have no unused rows and have no cohort |
+### Residue-level interface task: `masif_site/`
+Interface residues rebuilt from MaSIF-site with a label defined from geometry: a residue of the target chain is an
+interface residue if any of its heavy atoms lies within 5 Å of a heavy atom of another chain. There are 3,327 chains
+(2,677 train, 297 validation, 353 test) and 647,332 residues. `embeddings/<encoder>/` follows the probing format for 28
+of the 33 encoders (SaProt, Foldseek, both GCPNets and FoldVision are not scored on this task). `labels.h5` holds the
+residue labels (`residue/interface`) and the split (`protein/split_masif`); `exclusions.csv` lists the 35 entries that
+were removed.
+### Latent-space atlas: `atlas/data_v4/`
+`atlas.json` holds every layer; the other files hold one task layer each (`ec`, `go_mf`, `go_bp`, `localisation`,
+`homology`, `solubility`, `membrane`, `stability`, `fluorescence`, `affinity`). Each layer is a stratified sample of
+about 2,000 proteins of the task's own rows. For every protein it gives the id, split and label; for every encoder the
+UMAP coordinates (k = 10, seed 42, Procrustes-aligned to ESM2-150M), the k-NN purity with its chance level and a
+five-fold linear probe accuracy; and between encoders the
+debiased linear CKA and k-NN overlap, both computed on the full embeddings. `summary.json` holds the per-layer
+summary. The atlas is built from the embeddings in `extract/embeddings/`.
+### Input audit
+`extract/encoder_input_audit.csv` has one row per encoder and dataset (`probing` for the CATH set, `stage1` for the
+downstream set). Each row gives the relative change of the pooled embedding over the same 32 proteins under a 1 Å
+coordinate jitter, a coordinate swap with another protein, a sequence shuffle and a rigid motion, the measured
+`uses_coordinates`, `uses_residue_identity` and `invariant`, the declared inputs, and a `verdict`. The file also holds
+the failing rows of EGNN and GeoBPE. `extract/encoder_atom_audit.csv` gives the change when each of the N, CA, C, O
+and CB coordinates is perturbed and the resulting `atom_set`. `extract/encoder_sidechain_audit.csv` gives the same for
+side-chain atoms.
+## Loading example
 ```python
 import numpy as np
+import pandas as pd
+from huggingface_hub import snapshot_download
+repo = "Anonymoususer2223/ProtCompass_Embeddings"
+task, encoder = "subcellular_localization", "esm2_650m"
+# Download one encoder on one task and the task manifest (about 65 MB).
+local = snapshot_download(repo, allow_patterns=[f"extract/embeddings/{task}/{encoder}/*",
+                                                f"extract/manifest/{task}.csv"])
+d = f"{local}/extract/embeddings/{task}/{encoder}"
+splits = ("train", "valid", "test")
+X = {s: np.load(f"{d}/{s}.npy") for s in splits}
+ids = {s: open(f"{d}/ids_{s}.txt").read().split() for s in splits}
+manifest = pd.read_csv(f"{local}/extract/manifest/{task}.csv")
+manifest = manifest[manifest.stage1 == 1].set_index("id")
+y = {s: manifest.loc[ids[s], "label"].to_numpy() for s in splits}
+# A quick check with a logistic regression. The paper's heads (linear and MLP-2, selected on validation,
+# three seeds) are in extract/score2.py of the code repository.
+from sklearn.linear_model import LogisticRegression
+from sklearn.preprocessing import StandardScaler
+scaler = StandardScaler().fit(X["train"])
+clf = LogisticRegression(max_iter=1000).fit(scaler.transform(X["train"]), y["train"])
+print("test accuracy", clf.score(scaler.transform(X["test"]), y["test"]))
 ```
+Per-residue probing embeddings are large (up to 14.5 GB for ESM2-3B), so open them with `mmap_mode="r"`:
+```python
+import h5py
+# In allow_patterns a * also matches '/', so the top-level files are named one by one.
+local = snapshot_download(repo, allow_patterns=["probing_embeddings/esm2_650m/*", "probing_embeddings/offsets.npy",
+                                                "probing_embeddings/lengths.npy", "probing_embeddings/ids.txt",
+                                                "probing_data/labels.h5"])
+R = np.load(f"{local}/probing_embeddings/esm2_650m/residue_fp16.npy", mmap_mode="r")
+offsets = np.load(f"{local}/probing_embeddings/offsets.npy")
+lengths = np.load(f"{local}/probing_embeddings/lengths.npy")
+with h5py.File(f"{local}/probing_data/labels.h5", "r") as f:
+    k = 10
+    rows = slice(offsets[k], offsets[k] + lengths[k])
+    residue_embeddings = R[rows]          # (lengths[k], 1280)
+    dssp3 = f["residue/dssp3"][rows]      # 0 = H, 1 = E, 2 = C
+    split = f["protein/split_superfamily"][k]
 ```
+## Structures and datasets
+The manifests give, for every row, the structure file the extraction read (`structure_path`, relative to the
+directory that holds the code repository's `protcompass/` folder). The ESMFold structures of stability, fluorescence, solubility and PPI affinity are in the
+[ProtEnv](https://huggingface.co/datasets/Anonymoususer2223/ProtEnv) dataset under `predicted_structures/`; the last
+two components of `structure_path` (for example `predictions_fluorescence/fluor_20897.pdb`) are the member name in the
+corresponding archive. The experimental (PDB), AlphaFold DB and CATH structures are public and are not redistributed.
+The scripts that fetch them and build the manifests are in the code repository.
+## Earlier release (removed)
+An earlier release, produced by an earlier extraction pipeline for a previous version of this work, was stored
+under `embeddings/` and `probing_IF/`. The current paper does not use it. An audit in September 2026 found defects
+in it: structure encoders received placeholder or all-zero coordinates on several tasks, some entries were
+degenerate or mislabelled, and four of its tasks had defects in their labels or metric. It was removed from this
+repository. Every file the paper uses is in the current release.
+## License
+The embeddings, labels, manifests, audit files and atlas data produced for this work are released under the MIT
+licence. The files under `checkpoints/` keep their upstream licences: MIT for the ProteinMPNN, GCPNet and PottsMPNN
+files (the licence text is next to each file) and CC BY 4.0 for the GearNet weights (`checkpoints/gearnet_pretrained/NOTICE.md`).