Vulnerability Detector β€” DPO adapters (negative result)

Status: NOT functional for detection. Do not deploy. These three adapters are a negative result, published only so that Sections 13 and 19 of the thesis can be reproduced. They detect fewer vulnerabilities than the Phase 2 SFT adapters they were built from.

From the MSc thesis Software Vulnerability Detection with Advanced Machine Learning Techniques (Claudio Bendini, University of Parma, 2025–2026, advisor Prof.ssa Eleonora Iotti).

Why this is a negative result

The "Agentic Forum" had three fine-tuned models vote on each function, debate the disagreements, and then retrain the losers with DPO on paired (correct reasoning, incorrect reasoning) examples, weighting harder cases more. The hypothesis was a self-improving flywheel. Measured on BigVul, DPO instead cost about 26 points of recall (0.8110 to 0.5508) while only halving FPR (0.0451 to 0.0239). Precision rose, recall collapsed; the ensemble detected far fewer real vulnerabilities after DPO than before. The structural cause is in Section 19 of the log.

The adapters

Each is the Phase 2 SFT adapter after the DPO stage.

Subfolder Base model Base license
qwen Qwen2.5-Coder-7B-Instruct Apache 2.0
deepseek DeepSeek-Coder-6.7B-instruct DeepSeek Model License
mistral Mistral-7B-Instruct-v0.3 Apache 2.0

How to load

Rank 32, alpha 64, dropout 0.05, all attention and MLP projections, 4-bit base (QLoRA). PEFT 0.13.2, TRL 1.5.0, Transformers 4.57.6.

Subfolder Base model id
qwen Qwen/Qwen2.5-Coder-7B-Instruct
deepseek deepseek-ai/deepseek-coder-6.7b-instruct
mistral mistralai/Mistral-7B-Instruct-v0.3
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "Qwen/Qwen2.5-Coder-7B-Instruct"
repo    = "phazertron/vulnerability-detector-dpo-negative"
sub     = "qwen"   # or "deepseek", "mistral"

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(repo, subfolder=sub)
base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, repo, subfolder=sub)

Training data

DPO preference pairs built from inter-model disagreement on BigVul, after Phase 2 SFT on the same corpus. The adapters contain learned weights, not dataset content.

Licensing

Each adapter inherits its base model's license, so this repository is mixed-licensed:

  • qwen, mistral: Apache 2.0.
  • deepseek: DeepSeek Model License. It permits redistribution and derivatives but requires that its use-based restrictions (Attachment A) be passed on and kept in force. By using this adapter you agree to them.

Citation

@mastersthesis{bendini2026vulndetect,
  author = {Claudio Bendini},
  title  = {Software Vulnerability Detection with Advanced Machine Learning Techniques},
  school = {University of Parma},
  year   = {2026}
}
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for phazertron/vulnerability-detector-dpo-negative

Base model

Qwen/Qwen2.5-7B
Adapter
(666)
this model