phazertron's picture
Add model card
2814d28 verified
|
Raw History Blame Contribute Delete
4.99 kB
metadata
library_name: peft
pipeline_tag: text-generation
tags:
  - lora
  - qlora
  - peft
  - vulnerability-detection
  - security
  - cwe
  - code
language:
  - en
base_model:
  - Qwen/Qwen2.5-Coder-7B-Instruct
  - deepseek-ai/deepseek-coder-6.7b-instruct
  - mistralai/Mistral-7B-Instruct-v0.3
license: other
license_name: mixed-per-adapter
license_link: https://github.com/Phazertron/Vulnerability_Detector

Vulnerability Detector — Phase 2 SFT adapters (working detectors)

Status: functional. These three QLoRA adapters work as C/C++ vulnerability detectors and can be used for inference. They also reproduce Section 3 of the thesis. Read the honesty note before trusting a score.

From the MSc thesis Software Vulnerability Detection with Advanced Machine Learning Techniques (Claudio Bendini, University of Parma, 2025–2026, advisor Prof.ssa Eleonora Iotti). Each adapter classifies a C/C++ function as CLEAN or VULNERABLE: CWE-<id>. Detection only, no remediation.

The adapters

Supervised fine-tuning on real-world BigVul functions. Results on the BigVul test split:

Subfolder Base model Binary F1 Recall FPR Base license
deepseek-coder-6.7b DeepSeek-Coder-6.7B-instruct 0.5813 0.8343 0.0469 DeepSeek Model License
mistral-7b Mistral-7B-Instruct-v0.3 0.4923 0.7130 0.0536 Apache 2.0
qwen2.5-coder-7b Qwen2.5-Coder-7B-Instruct 0.2156 0.1400 0.0072 Apache 2.0

Use deepseek-coder-6.7b. It catches 83% of vulnerable functions while flagging under 5% of clean ones, a 7x binary-F1 gain over the zero-shot base. The headline is the false-positive rate, not the F1. The Qwen adapter is included for completeness; it is a valid run but a weak detector.

Honesty note (read before trusting any number)

On PrimeVul's paired split, where each vulnerable function sits beside its own patched version, none of these models tells the two apart at better than chance (thesis Section 15 / log Section 15). The BigVul numbers are real, but most of what they measure is surface pattern, not the vulnerability. A high score is not a safe signal that the model understands the bug.

How to load

Rank 32, alpha 64, dropout 0.05, all attention and MLP projections, 4-bit base (QLoRA). PEFT 0.13.2, TRL 1.5.0, Transformers 4.57.6.

Subfolder Base model id
deepseek-coder-6.7b deepseek-ai/deepseek-coder-6.7b-instruct
mistral-7b mistralai/Mistral-7B-Instruct-v0.3
qwen2.5-coder-7b Qwen/Qwen2.5-Coder-7B-Instruct
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "deepseek-ai/deepseek-coder-6.7b-instruct"
repo    = "phazertron/vulnerability-detector-sft"
sub     = "deepseek-coder-6.7b"

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(repo, subfolder=sub)
base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, repo, subfolder=sub)

Prompt the model exactly as it was trained:

You are a security auditor. Analyze the following C function.
Output ONLY: VULNERABLE: CWE-<id>   OR   CLEAN
Do not suggest fixes. Do not explain.

<function source here>

Expected output is one line: VULNERABLE: CWE-787 or CLEAN.

Training data

BigVul (real-world CVE functions). CASTLE and PrimeVul were held out and never trained on. The adapters contain learned weights, not dataset content.

Licensing

Each adapter inherits its base model's license, so this repository is mixed-licensed:

  • mistral-7b, qwen2.5-coder-7b: Apache 2.0.
  • deepseek-coder-6.7b: DeepSeek Model License. It permits redistribution and derivatives but requires that its use-based restrictions (Attachment A) be passed on and kept in force. By using this adapter you agree to them.

Citation

@mastersthesis{bendini2026vulndetect,
  author = {Claudio Bendini},
  title  = {Software Vulnerability Detection with Advanced Machine Learning Techniques},
  school = {University of Parma},
  year   = {2026}
}