| --- |
| license: cc-by-nc-4.0 |
| library_name: pytorch |
| tags: |
| - biology |
| - bioinformatics |
| - protein-language-model |
| - mhc |
| - mhc-class-ii |
| - peptide-binding |
| - bertmhc |
| - non-commercial-research |
| metrics: |
| - roc_auc |
| - pearsonr |
| --- |
| |
| # BERTMHC MHC class II binding model |
|
|
| This repository contains a locally trained, paper-aligned BERTMHC checkpoint |
| for MHC class II peptide-binding affinity prediction. It is intended for |
| academic and non-profit non-commercial research. |
|
|
| > **Release scope:** this is one fold and one random seed, not the complete |
| > five-fold, three-initialization-per-fold experiment from the BERTMHC paper. |
| > It is not a clinical model and does not predict antigen presentation, |
| > T-cell response, or immunogenicity. |
|
|
| ## Model description |
|
|
| The checkpoint fine-tunes the TAPE ProteinBERT base model on fold 1 of the |
| Jensen et al. MHC class II binding dataset used by NetMHCIIpan-3.2 and BERTMHC. |
| It uses: |
|
|
| - a 34-residue MHC class II pseudosequence; |
| - peptide padding or end-truncation to 24 residues; |
| - TAPE ProteinBERT with 12 layers, 12 attention heads, and hidden size 768; |
| - mean pooling over non-padding token representations; |
| - a two-layer MLP with hidden size 512; |
| - end-to-end fine-tuning with mean squared error on normalized affinity; and |
| - the first output logit, transformed with a sigmoid, as the binding score. |
|
|
| The normalized target is |
|
|
| ```text |
| y = 1 - log(IC50) / log(50000) |
| ``` |
|
|
| where IC50 is expressed in nM. A 500 nM binder threshold corresponds to a |
| normalized target of approximately 0.426. |
|
|
| ## Evaluation |
|
|
| The best checkpoint was selected at epoch 10 using fold-1 evaluation auROC. |
| The evaluation partition contains 27,332 measurements, including 11,013 |
| binders at the 500 nM-derived threshold. |
|
|
| | Metric | Fold-1 evaluation | |
| |---|---:| |
| | auROC | 0.8660506295 | |
| | Pearson R | 0.7330023050 | |
| | MSE | 0.0320102610 | |
| | Average precision | 0.8205408974 | |
| | Kendall tau | 0.5383985861 | |
|
|
| Fold 1 controlled learning-rate scheduling, checkpoint selection, and early |
| stopping. These numbers are therefore validation results, not an independent |
| test result. The BERTMHC paper reported aggregate five-fold performance of |
| auROC 0.8822 and Pearson R 0.759 using three model initializations per fold. |
| The two result sets are not directly equivalent. |
|
|
| ## Files |
|
|
| | File | Purpose | |
| |---|---| |
| | `model.safetensors` | 92,356,612-parameter PyTorch state dictionary | |
| | `bertmhc_config.json` | Architecture, input, and target configuration | |
| | `mhc_pseudosequences.csv` | 72 human HLA class II pseudosequences used by the local adapter | |
| | `mhc_pseudosequences_full.csv` | Full 80-molecule human and murine training mapping | |
| | `metrics.json` | Best-checkpoint metrics and evaluation scope | |
| | `training_metadata.json` | Sanitized training, environment, and provenance metadata | |
| | `THIRD_PARTY_NOTICES.md` | BERTMHC, TAPE, and dataset notices | |
| | `MODEL_LICENSE.md` | Model-weights license and scope | |
| | `SHA256SUMS` | Release-file checksums | |
|
|
| ## Loading the state dictionary |
|
|
| This is a custom PyTorch/TAPE checkpoint, not a Hugging Face Transformers |
| `AutoModel` repository. The repository intentionally does not redistribute the |
| official or locally modified BERTMHC source code. Researchers must obtain the |
| official BERTMHC software separately, review and accept its non-commercial |
| license, and use a runtime that implements the paper-specified masked mean |
| pooling behavior. |
|
|
| After installing an authorized BERTMHC runtime, the state dictionary can be |
| loaded as follows: |
|
|
| ```python |
| from huggingface_hub import hf_hub_download |
| from safetensors.torch import load_file |
| from tape import ProteinBertConfig |
| |
| from bertmhc.bertmhc import BERTMHC |
| |
| repo_id = "zmactep/bertmhc" |
| config_path = hf_hub_download(repo_id, "bertmhc_config.json") |
| weights_path = hf_hub_download(repo_id, "model.safetensors") |
| |
| config = ProteinBertConfig.from_pretrained(config_path) |
| model = BERTMHC(config) |
| model.load_state_dict(load_file(weights_path), strict=True) |
| model.eval() |
| ``` |
|
|
| The released upstream BERTMHC implementation mean-pools all encoded positions, |
| whereas the paper specifies masking padding positions before pooling. This |
| checkpoint was trained with masked mean pooling. Using it with unmasked pooling |
| will not reproduce the reported scores. |
|
|
| For input preparation, follow the official BERTMHC binding schema and |
| tokenization. Each row needs an allele label, its 34-residue MHC |
| pseudosequence, and a peptide. The mapping files in this repository are |
| provided for convenience and provenance. |
|
|
| ## Intended use |
|
|
| Suitable uses include: |
|
|
| - non-commercial academic investigation of MHC-II peptide binding; |
| - replication and method-comparison studies; |
| - research-only ranking of peptide candidates; and |
| - evaluation as one component of an MHC-II ensemble. |
|
|
| ## Out-of-scope use |
|
|
| Do not use this checkpoint as: |
|
|
| - a clinical, diagnostic, or therapeutic decision system; |
| - a standalone predictor of antigen presentation or immunogenicity; |
| - evidence of efficacy or safety of a vaccine candidate; |
| - a calibrated probability of biochemical binding; or |
| - a commercial model or service. |
|
|
| ## Limitations |
|
|
| - Only fold 1 and seed 1111 were trained. |
| - The published 15-model cross-validation design has not been completed. |
| - The authors' independent 2,413-pair benchmark has not yet been evaluated. |
| - No local leave-one-molecule-out experiment was performed. |
| - Peptides longer than 24 residues are truncated at the end. |
| - Source duplicates and repeated peptide-allele measurements were preserved. |
| - Apple MPS runs were stable to reported precision but not proven bitwise |
| deterministic across hardware and software versions. |
| - Scores have not been independently calibrated across alleles. |
|
|
| ## Training data |
|
|
| The model was trained from the official Jensen/NetMHCIIpan-3.2 binding folds: |
|
|
| - fold-1 training: 106,949 measurements; |
| - fold-1 evaluation: 27,332 measurements; |
| - complete source dataset: 134,281 measurements; |
| - complete source coverage: 80 MHC-II molecules, including 72 human and 8 |
| murine molecules; and |
| - observed peptide lengths: 9-37 residues. |
|
|
| No training rows are redistributed in this model repository. Dataset source |
| and processed-artifact hashes are recorded in `training_metadata.json`. |
|
|
| ## Paper and code alignment |
|
|
| The peer-reviewed paper specifies MSE regression, Pearson correlation, and |
| padding-aware mean pooling for the binding task. The released BERTMHC code uses |
| binary cross-entropy for the continuous binding target, computes Kendall tau as |
| `val_cor`, and averages over padding positions. This checkpoint follows the |
| paper for these three choices. GitHub issue 8 records the unresolved public |
| question about the loss and metric discrepancy. |
|
|
| ## Licensing and attribution |
|
|
| The model weights are released under CC BY-NC 4.0; see |
| `MODEL_LICENSE.md`. The official BERTMHC software is not included and remains |
| subject to its separate NEC Laboratories Europe non-commercial research |
| license. TAPE notices are reproduced in `THIRD_PARTY_NOTICES.md`. |
|
|
| Publications using this checkpoint must cite the BERTMHC and TAPE papers and |
| must explicitly acknowledge: |
|
|
| > The BERTMHC software was developed by NEC Laboratories Europe GmbH. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{cheng2021bertmhc, |
| title = {BERTMHC: improved MHC-peptide class II interaction prediction with transformer and multiple instance learning}, |
| author = {Cheng, Jun and Bendjama, Ka{\"i}dre and Rittner, Karola and Malone, Brandon}, |
| journal = {Bioinformatics}, |
| volume = {37}, |
| number = {22}, |
| pages = {4172--4179}, |
| year = {2021}, |
| doi = {10.1093/bioinformatics/btab422} |
| } |
| |
| @inproceedings{rao2019tape, |
| title = {Evaluating Protein Transfer Learning with TAPE}, |
| author = {Rao, Roshan and Bhattacharya, Nicholas and Thomas, Neil and Duan, Yan and Chen, Xi and Canny, John and Abbeel, Pieter and Song, Yun S.}, |
| booktitle = {Advances in Neural Information Processing Systems}, |
| volume = {32}, |
| year = {2019} |
| } |
| ``` |
|
|
| ## Sources |
|
|
| - BERTMHC paper: https://doi.org/10.1093/bioinformatics/btab422 |
| - Official BERTMHC repository: https://github.com/s6juncheng/BERTMHC |
| - Paper/code discrepancy issue: https://github.com/s6juncheng/BERTMHC/issues/8 |
| - Jensen et al. dataset paper: https://doi.org/10.1111/imm.12889 |
| - TAPE paper and code: https://pmc.ncbi.nlm.nih.gov/articles/PMC7774645/ |
| - NetMHCIIpan-3.2 data page: https://services.healthtech.dtu.dk/services/NetMHCIIpan-3.2/ |
|
|
