LineVul-MoE

A research release of 110 fine-tuned CodeBERT checkpoints: 11 model variants, five training seeds, and two fixed BigVul splits. The base encoder is microsoft/codebert-base. The original LineVul is the reference for the first-layer attention scoring method.

Checkpoints

  • All-in-One: binary vulnerability detection over the full training split.
  • Exclude: one classifier per target CWE, trained on its vulnerable functions and all non-vulnerable functions. Vulnerable functions outside the target CWE are excluded.
  • One-Hot: one classifier per target CWE, initialized from the corresponding All-in-One checkpoint. The target CWE is positive and the other four top CWE classes are negative. Non-vulnerable functions are outside this training task.

The five target classes are CWE-119, CWE-20, CWE-399, CWE-125, and CWE-200. The training seeds are 42, 2026, 31415, 27182, and 123456. The official split preserves the published LineVul split. The project split groups projects, CVEs, commits, and matching function hashes before splitting. Exact prepared split fingerprints are recorded in reproduction.json.

Download a single checkpoint

from huggingface_hub import hf_hub_download
from safetensors.torch import load_file

path = hf_hub_download(
    repo_id="<owner>/LineVul-MoE",
    revision="<published-Hub-commit-SHA>",
    subfolder="checkpoints/LineVul-MoE_Exclude_official_CWE-119_seed42",
    filename="model.safetensors",
)
state = load_file(path, device="cpu")
# Build the original binary classifier with src/models.py in the experiment code.
# Then use model.load_state_dict(state, strict=True) and model.eval().

These files contain the original encoder and one-logit classification head tensors. Load them with the experiment's src/models.py model definition. Use manifest.csv to select a checkpoint and verify its SHA-256.

Reproduce inference and evaluation

Experiment code on GitHub: repository URL and fixed commit pending. The upstream LineVul repository above describes the reference method.

  1. Reconstruct the fixed split and tokenize with the original CodeBERT tokenizer and 512-token input window. The fingerprints in reproduction.json identify the source run configuration, tokenizer, and prepared split files.
  2. For function-level inference, use the binary model's logit and apply sigmoid(logit / temperature). Use the validation-derived threshold stored in reproduction.json to reproduce the binary prediction. The One-Hot classifier estimates the target CWE among vulnerable Top-5 functions.
  3. For line-level ranking, reproduce the original evaluator: sum first-layer attention across heads and query positions, zero the beginning and ending special-token scores, and aggregate token scores by source line. The reference evaluator includes padded query positions in this aggregation. Evaluate only the designated Top-5 vulnerable functions with a verified flaw line inside the input window. Compute the per-function metrics and average over the reported population and training seeds.

Distribution terms

The upstream CodeBERT code repository carries an MIT license. The license for distribution of this fine-tuned checkpoint collection remains to be confirmed before public release.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Hietan/LineVul-MoE

Finetuned
(158)
this model