ESMCapsid-C

Built with ESM.

ESMCapsid-C is a continued masked-language-model fine-tune of Synthyra/ESMplusplus_large for the representation stage of the ESMCapsid workflow. The repository includes the model weights, tokenizer, configuration, and custom Transformers implementation required for loading the model.

About

ESM++ Large provides a Transformers-compatible implementation of ESMC 600M. ESMCapsid-C was initialized from that checkpoint and continued with masked-language-model training. In the released ESMCapsid pipeline, sequences passing the ESMCapsid-S screen are encoded with Layer 35 and attention-mask mean pooling.

Property	Value
Base model	Synthyra/ESMplusplus_large
Model type	Continued MLM fine-tune
Transformer layers	36
Hidden size	1,152
Attention heads	18
ESMCapsid representation	Layer 35, attention-mask mean pooling

Usage

Installation

pip install torch transformers tokenizers

Load the model and extract Layer 35 embeddings

import torch
from transformers import AutoModel, AutoTokenizer

model_id = "Shuofang127/ESMCapsid-C"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
).eval()

batch = tokenizer(
    ["MSTNPKPQRKTKRNT", "MKTIIALSYIFCLVFA"],
    padding=True,
    return_tensors="pt",
)

with torch.inference_mode():
    output = model(**batch, output_hidden_states=True)

# The packaged implementation returns block states without an input-embedding state.
layer35 = output.hidden_states[35]
mask = batch["attention_mask"].unsqueeze(-1).to(layer35.dtype)
embedding = (layer35 * mask).sum(dim=1) / mask.sum(dim=1).clamp_min(1)

print(embedding.shape)  # (batch_size, 1152)

For offline loading, replace model_id with the local model directory and set local_files_only=True in both from_pretrained calls.

Property heads

Nine pretrained heads are provided in heads/property_heads/: realm, kingdom, phylum, class, order, family, genome_label, host_group, and fold_label. Each includes its checkpoint, class mapping, and prediction threshold.

Use the ESMCapsid Inference toolkit with --property-heads /path/to/heads/property_heads for annotate or predict. See the head package README for a heads-only download example. These heads use a distinct layer-33, residue-only representation computed internally by the toolkit, not the public layer-35 embedding vectors. They are auxiliary model assets and do not change AutoModel.from_pretrained().

Important

• ESMCapsid-C is a representation model, not the first-stage capsid classifier. Use Shuofang127/ESMCapsid-S for candidate screening.
• Embedding similarity does not establish infectivity, host range, taxonomy, structure, or biological function.
• Keep the released layer, sequence preprocessing, truncation policy, and pooling method together.
• This repository uses custom code and requires trust_remote_code=True. Review the code before loading it.
• The packaged modeling_esm_plusplus.py is the runtime implementation for this release; features added later to the base repository are not automatically supported here.

Citation

• Base model: Synthyra/ESMplusplus_large, DOI 10.57967/hf/3726.
• Hallee et al. FastPLMs: Bringing ESMC and ESM3 to Transformers. DOI 10.5281/zenodo.16466575.
• Liu, S., Xia, S., and Wang, H. Capsid-specialized protein language models reveal higher-order viral architecture from sequence. bioRxiv (2026). DOI 10.64898/2026.09.06.749605.

License

ESMCapsid-C is a derivative of ESM++ Large and ESMC 600M. Review the repository's LICENSE and NOTICE.txt before use or redistribution.