LineVul-MoE / README.md
Hietan's picture
Add files using upload-large-folder tool
dc36a52 verified
|
Raw History Blame Contribute Delete
3.56 kB
---
base_model: microsoft/codebert-base
tags:
- codebert
- linevul
- vulnerability-detection
- research
---
# LineVul-MoE
A research release of 110 fine-tuned CodeBERT checkpoints: 11 model variants, five training seeds, and two fixed BigVul splits. The base encoder is [microsoft/codebert-base](https://huggingface.co/microsoft/codebert-base). The original [LineVul](https://github.com/awsm-research/LineVul) is the reference for the first-layer attention scoring method.
## Checkpoints
- **All-in-One**: binary vulnerability detection over the full training split.
- **Exclude**: one classifier per target CWE, trained on its vulnerable functions and all non-vulnerable functions. Vulnerable functions outside the target CWE are excluded.
- **One-Hot**: one classifier per target CWE, initialized from the corresponding All-in-One checkpoint. The target CWE is positive and the other four top CWE classes are negative. Non-vulnerable functions are outside this training task.
The five target classes are CWE-119, CWE-20, CWE-399, CWE-125, and CWE-200. The training seeds are 42, 2026, 31415, 27182, and 123456. The `official` split preserves the published LineVul split. The `project` split groups projects, CVEs, commits, and matching function hashes before splitting. Exact prepared split fingerprints are recorded in `reproduction.json`.
## Download a single checkpoint
```python
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
path = hf_hub_download(
repo_id="<owner>/LineVul-MoE",
revision="<published-Hub-commit-SHA>",
subfolder="checkpoints/LineVul-MoE_Exclude_official_CWE-119_seed42",
filename="model.safetensors",
)
state = load_file(path, device="cpu")
# Build the original binary classifier with src/models.py in the experiment code.
# Then use model.load_state_dict(state, strict=True) and model.eval().
```
These files contain the original encoder and one-logit classification head tensors. Load them with the experiment's `src/models.py` model definition. Use `manifest.csv` to select a checkpoint and verify its SHA-256.
## Reproduce inference and evaluation
**Experiment code on GitHub:** repository URL and fixed commit pending. The upstream LineVul repository above describes the reference method.
1. Reconstruct the fixed split and tokenize with the original CodeBERT tokenizer and 512-token input window. The fingerprints in `reproduction.json` identify the source run configuration, tokenizer, and prepared split files.
2. For function-level inference, use the binary model's logit and apply `sigmoid(logit / temperature)`. Use the validation-derived threshold stored in `reproduction.json` to reproduce the binary prediction. The One-Hot classifier estimates the target CWE among vulnerable Top-5 functions.
3. For line-level ranking, reproduce the original evaluator: sum first-layer attention across heads and query positions, zero the beginning and ending special-token scores, and aggregate token scores by source line. The reference evaluator includes padded query positions in this aggregation. Evaluate only the designated Top-5 vulnerable functions with a verified flaw line inside the input window. Compute the per-function metrics and average over the reported population and training seeds.
## Distribution terms
The [upstream CodeBERT code repository](https://github.com/microsoft/CodeBERT/blob/master/LICENSE) carries an MIT license. The license for distribution of this fine-tuned checkpoint collection remains to be confirmed before public release.