Token Classification
Transformers
PEFT
PyTorch
protein-language-model
protein-annotation
bioinformatics
esm2
Instructions to use Wangwpi/PALM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Wangwpi/PALM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="Wangwpi/PALM")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Wangwpi/PALM", device_map="auto") - PEFT
How to use Wangwpi/PALM with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
|
Download README.md from Wangwpi/PALM: direct link, hf CLI and curl.
- Browser
- Download file 3.47 kB
-
https://huggingface.co/Wangwpi/PALM/resolve/main/README.md
- Command line
-
hf download hf://Wangwpi/PALM/README.md
-
curl -L -o README.md https://huggingface.co/Wangwpi/PALM/resolve/main/README.md
3.47 kB
| license: mit | |
| base_model: facebook/esm2_t30_150M_UR50D | |
| tags: | |
| - protein-language-model | |
| - protein-annotation | |
| - bioinformatics | |
| - esm2 | |
| - peft | |
| - pytorch | |
| library_name: transformers | |
| pipeline_tag: token-classification | |
| # PALM soft-target epoch 8 | |
| This repository contains the final PALM soft-target overlap-aware checkpoint | |
| and optional ClinVar-trained pathogenicity heads. | |
| ## Files | |
| ```text | |
| palm_soft_target_epoch8.pt | |
| pathogenicity/random_split_fold_1.pt | |
| pathogenicity/random_split_fold_2.pt | |
| pathogenicity/random_split_fold_3.pt | |
| pathogenicity/random_split_fold_4.pt | |
| pathogenicity/random_split_fold_5.pt | |
| interpro_classes.tsv | |
| checkpoint_manifest.json | |
| ``` | |
| `interpro_classes.tsv` maps every output class to its InterPro entry: | |
| ```text | |
| head class_index interpro_id interpro_type name | |
| domain 0 IPR000001 Domain Kringle | |
| site 0 IPR000035 Conserved_site Alkylbase DNA glycosidase, conserved site | |
| ``` | |
| `class_index` is 0-based and per head: domain indices run 0-17950, and site | |
| indices restart at 0 and run to 1382. The checkpoint itself carries the same | |
| accession-to-index maps as `ipr2domain` and `ipr2site`; this file adds the | |
| human-readable names. | |
| The annotation checkpoint predicts four residue-level outputs: | |
| - domain occurrence: none, one, or multiple; | |
| - site occurrence: none, one, or multiple; | |
| - InterPro domain classes; and | |
| - InterPro site classes. | |
| Overlapping domain/site labels use uniform soft targets over all true classes. | |
| The final model uses ESM2-150M, LoRA on the last four ESM layers, learned | |
| all-layer pooling, and a bidirectional LSTM residue encoder. | |
| ## Use | |
| Install the code from the PALM GitHub repository, then use the Hugging Face | |
| artifact reference directly: | |
| ```python | |
| from palm import PALMPredictor | |
| predictor = PALMPredictor( | |
| "hf://Wangwpi/PALM::palm_soft_target_epoch8.pt", | |
| device="cuda", | |
| ) | |
| annotation = predictor.annotate("MSEQUENCE...") | |
| ``` | |
| The checkpoint is downloaded from the | |
| [`Wangwpi/PALM`](https://huggingface.co/Wangwpi/PALM) model repository and | |
| cached by `huggingface_hub`. | |
| ## Pathogenicity heads | |
| The five random-split heads were selected at epochs 3, 4, 4, 3, and 4 using | |
| inner-validation AUROC within a common 15-epoch budget. Their final prediction | |
| is the unweighted arithmetic mean of the five pathogenicity probabilities. | |
| These are supervised ClinVar-trained models. Their outputs must not be called | |
| zero-shot. PALM annotation/embedding WT-mutant disruption can be evaluated | |
| separately without loading these heads. | |
| ## Limitations | |
| - The annotation and pathogenicity models are research models, not clinical | |
| diagnostic tools. | |
| - Variant results depend on the protein transcript supplied to the model. | |
| - Long proteins are limited by the ESM2 context window; variant workflows crop | |
| around the amino-acid change. | |
| - Cohort filters and transcript aggregation must be reported with benchmark | |
| results. | |
| ## License | |
| Released under the MIT license, the same terms as the GitHub repository | |
| (`https://github.com/WGLab/PALM`). | |
| PALM is a fine-tune of `facebook/esm2_t30_150M_UR50D`, which is itself MIT | |
| licensed. The annotation model was trained on UniRef50 sequences labelled with | |
| InterPro domain and site entries, and the optional pathogenicity heads on | |
| ClinVar variant classifications; anyone redistributing derived data should | |
| check the terms of those sources as well. | |
| ## Citation | |
| Citation information will be added when the PALM manuscript is available. | |