KlaBenchmark

KlaBenchmark is a benchmark for lysine lactylation (Kla) site prediction. Published predictors were trained and scored on different datasets and background-lysine definitions, so those published numbers cannot be compared directly. This release keeps seven published predictors on the same KlaAtlas sites, splits, and evaluation protocol.

The seven predictors are PCBert-Kla, HybridKla, PBertKla, DeepKlapred, ABFF-Kla, Auto-Kla, and DeepKla. This Hugging Face package contains the benchmark TSV files and the trained weights. Training and prediction code are in the KlaBenchmark GitHub repository.

Relationship to the GitHub Repository

GitHub repository	This Hugging Face package
Source code for the seven predictors	Complete benchmark datasets and pretrained model weights
Protein-level split and original-model datasets	The remaining partitions
Project documentation	Extra files required for prediction

Download a method folder from dataset/ and the matching checkpoint, then run the scripts from the GitHub repository. On GitHub, each method includes only KlaAtlas_benchmark_summary/protein_split_dataset/ and External_generalization/original_model_dataset/. The other groups below are in this package.

Repository Contents

KlaBenchmark/
├── dataset/
│   ├── ABFF-Kla/
│   ├── Auto-Kla/
│   ├── DeepKla/
│   ├── DeepKlapred/
│   ├── HybridKla/
│   ├── PBertKla/
│   └── PCBert-Kla/
├── checkpoint/
└── checkpoint_original/

The files under each method are grouped in the same order as Supplementary Table S6:

dataset/<method>/
├── KlaAtlas_benchmark_summary/
│   ├── protein_split_dataset/
│   ├── sequence_split_dataset/
│   └── unseen_external/
├── External_generalization/
│   └── original_model_dataset/
├── Background_definition/
│   ├── background_ms_observable/
│   ├── background_protein_level/
│   ├── background_rsa_lt_0.2/
│   └── background_bias_removed/
├── Sequence_context/
│   ├── 15aa_window_protein_split_dataset/
│   ├── 31aa_window_protein_split_dataset/
│   ├── 41aa_window_protein_split_dataset/
│   └── 51aa_window_protein_split_dataset/
├── Cross_species_transfer/
│   └── cross_species_dataset/
├── Species_specific_training/
│   └── species_dataset/
└── DIA_external_performance/
    └── dia_external/

Keep this directory structure intact when possible. The group names and the dataset folder names inside them are part of the benchmark paths.

Required Columns

For most predictors, each train.tsv, val.tsv, and test.tsv must include at least:

• Sequence: local sequence window centered on the candidate lysine. Padding is _.
• Label: 1 for an experimentally identified Kla site, 0 for a background lysine site

ABFF-Kla also needs structure-related fields such as Contact.

Protein-level files use each method's input window. ABFF-Kla uses 35 aa windows. PBertKla uses 45 aa windows. The other methods use 51 aa windows.

Optional Columns

Depending on the split, files may also include:

• Site: site-level identifier
• ACC_ID: protein accession (e.g. UniProt)
• Mod_positions: lysine position in the protein
• Species: source species
• UniProtTaxID: NCBI taxonomy identifier

Benchmark Partitions

KlaAtlas_benchmark_summary/

Directory	Benchmark setting
protein_split_dataset/	Protein-level train/validation/test split. Samples from the same protein stay in one subset.
sequence_split_dataset/	The same KlaAtlas sites, repartitioned after clustering lysine-centered windows at 50% identity (CD-HIT).
unseen_external/	Class-balanced unseen-site external test. Ten repeats, test.tsv only. Sites absent from the published training sets are held out from the KlaAtlas protein-level training and validation proteins.

External_generalization/

Directory	Benchmark setting
original_model_dataset/	Train, validation, and test files aligned with each published model's own corpus. Direct external testing uses the KlaAtlas protein-level test in KlaAtlas_benchmark_summary/protein_split_dataset/.

Background_definition/

Each background is a separate dataset with repeat_01 through repeat_10. Every repeat has train.tsv, val.tsv, and test.tsv.

Directory	Benchmark setting
background_ms_observable/	Lysines covered by experimentally identified peptides and lacking Kla evidence.
background_protein_level/	Lysines on Kla-containing proteins that were not identified as Kla sites, irrespective of peptide-level MS observability.
background_rsa_lt_0.2/	MS-observable lysines with RSA < 0.2.
background_bias_removed/	MS-observable background after the peptide-construction bias is removed.

Sequence_context/

These datasets use the protein-level partition. Only the lysine-centered window length changes.

Directory	Benchmark setting
15aa_window_protein_split_dataset/	15 aa window
31aa_window_protein_split_dataset/	31 aa window
41aa_window_protein_split_dataset/	41 aa window
51aa_window_protein_split_dataset/	51 aa window

Cross_species_transfer/

Directory	Benchmark setting
cross_species_dataset/	Train and validate on human proteins. Non-human species are used only in the external test.

Species_specific_training/

Directory	Benchmark setting
species_dataset/<Species>/	Protein-level split built from that species alone.

species_dataset/ contains Homo_sapiens, Mus_musculus, Rattus_norvegicus, Caenorhabditis_elegans, Escherichia_coli, and Streptococcus_mutans.

DIA_external_performance/

Directory	Benchmark setting
dia_external/repeat_01/ ... repeat_10/	Independent DIA evaluation. These files are not used for training or model selection.

Each repeat contains LAC293-Kla.tsv, PGH-Kla.tsv, RAT-Kla.tsv, and RIH-Kla.tsv.

Most other partitions provide:

train.tsv
val.tsv
test.tsv

Selecting paths by scenario

Scenario	Typical data paths
Protein-level split	KlaAtlas_benchmark_summary/protein_split_dataset/train.tsv, etc.
Sequence-level split	KlaAtlas_benchmark_summary/sequence_split_dataset/train.tsv, etc.
Unseen-site external test	KlaAtlas_benchmark_summary/unseen_external/repeat_01/test.tsv
Original-model setting	External_generalization/original_model_dataset/train.tsv, etc.
Background comparison	Background_definition/background_ms_observable/repeat_01/train.tsv, and the matching repeat under the other three background folders
Fixed window (15 / 31 / 41 / 51 aa)	Sequence_context/15aa_window_protein_split_dataset/..., and the 31, 41, and 51 aa folders
Cross-species transfer	Cross_species_transfer/cross_species_dataset/train.tsv, etc.
Species-specific training	Species_specific_training/species_dataset/<Species>/train.tsv, etc.
DIA external evaluation	DIA_external_performance/dia_external/repeat_01/LAC293-Kla.tsv, and the other three DIA files in that repeat

For fixed-window partitions, use the window length that matches the dataset name when configuring the model. Report that window length with the result.

Checkpoints

checkpoint/<method>/ contains the seed-42 model trained on KlaAtlas_benchmark_summary/protein_split_dataset. Supplementary Table S6 reports the mean of three training seeds, so one checkpoint does not by itself equal that mean. checkpoint_original/<method>/ contains the seed-42 model trained on that method's External_generalization/original_model_dataset. Both packages use the same layout.

checkpoint/<method>/
├── preprocess.json
└── model/
    ├── best_model.<ext>
    └── extra/

Method	Main checkpoint	Additional artifacts
ABFF-Kla	model/best_model.keras	preprocess.json
Auto-Kla	model/best_model.pt	preprocess.json
DeepKla	model/best_model.keras	preprocess.json
DeepKlapred	model/best_model.pt	preprocess.json
HybridKla	model/best_model.pth	ESM2 state dict, feature DNNs, GPS encoder, LSTM weights, vocabulary, feature order, preprocess.json
PBertKla	model/best_model.weights.h5	ProteinBERT input and output encoders, preprocess.json
PCBert-Kla	model/best_model.pt	Scaler model, preprocess.json

Place a downloaded package in the matching local folder before prediction. PCBert-Kla reads model/extra/scaler_model.pkl, PBertKla reads its encoder files, and HybridKla reads its branch files.

Usage

Run these commands from the GitHub repository after the files above are available locally:

python methods/<method>/train.py \
  --train_tsv dataset/<method>/KlaAtlas_benchmark_summary/protein_split_dataset/train.tsv \
  --val_tsv dataset/<method>/KlaAtlas_benchmark_summary/protein_split_dataset/val.tsv \
  --test_tsv dataset/<method>/KlaAtlas_benchmark_summary/protein_split_dataset/test.tsv \
  --run_name <run_name>

python methods/<method>/predict.py \
  --checkpoint_path checkpoint/<method>/model/<checkpoint_file> \
  --input_tsv dataset/<method>/KlaAtlas_benchmark_summary/protein_split_dataset/test.tsv \
  --output_dir <path/to/output>

<method> is the folder name of one predictor. <checkpoint_file> is the main checkpoint named in the table above.

Label Caveat

Background lysine sites (Label = 0) are not experimentally verified negatives. They are lysine residues that were not identified as Kla sites under the rule used by that partition. Treat benchmark tasks as positive versus background classification, not positive versus confirmed-negative.

Usage Guidelines

• Keep official train/validation/test splits unchanged when reproducing published results.
• State which partition (protein-level, sequence-level, fixed window, species, cross-species, background, unseen-site, DIA, or original-model) was used.

Redistribution

Benchmark data and the released checkpoints are for research use. When redistributing derivatives, comply with the terms of the original proteomics databases, UniProt, pretrained backbone models, and source publications, and cite the KlaBenchmark manuscript and underlying studies.