phazertron's picture
Add model card
2814d28 verified
|
Raw History Blame Contribute Delete
4.99 kB
---
library_name: peft
pipeline_tag: text-generation
tags:
- lora
- qlora
- peft
- vulnerability-detection
- security
- cwe
- code
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-7B-Instruct
- deepseek-ai/deepseek-coder-6.7b-instruct
- mistralai/Mistral-7B-Instruct-v0.3
license: other
license_name: mixed-per-adapter
license_link: https://github.com/Phazertron/Vulnerability_Detector
---
# Vulnerability Detector — Phase 2 SFT adapters (working detectors)
> **Status: functional.** These three QLoRA adapters work as C/C++ vulnerability
> detectors and can be used for inference. They also reproduce Section 3 of the
> thesis. Read the honesty note before trusting a score.
From the MSc thesis *Software Vulnerability Detection with Advanced Machine
Learning Techniques* (Claudio Bendini, University of Parma, 2025–2026, advisor
Prof.ssa Eleonora Iotti). Each adapter classifies a C/C++ function as `CLEAN` or
`VULNERABLE: CWE-<id>`. Detection only, no remediation.
- **Code, data pipeline, full results:** https://github.com/Phazertron/Vulnerability_Detector
- **Canonical record:** [`docs/EXPERIMENT_LOG.md`](https://github.com/Phazertron/Vulnerability_Detector/blob/main/docs/EXPERIMENT_LOG.md), Section 3.
- **Companion repos:** [IR vs source](https://huggingface.co/phazertron/vulnerability-detector-ir-source) (a valid positive finding) and [DPO](https://huggingface.co/phazertron/vulnerability-detector-dpo-negative) (a negative result).
## The adapters
Supervised fine-tuning on real-world BigVul functions. Results on the BigVul
test split:
| Subfolder | Base model | Binary F1 | Recall | FPR | Base license |
|---|---|---|---|---|---|
| `deepseek-coder-6.7b` | DeepSeek-Coder-6.7B-instruct | **0.5813** | **0.8343** | 0.0469 | DeepSeek Model License |
| `mistral-7b` | Mistral-7B-Instruct-v0.3 | 0.4923 | 0.7130 | 0.0536 | Apache 2.0 |
| `qwen2.5-coder-7b` | Qwen2.5-Coder-7B-Instruct | 0.2156 | 0.1400 | 0.0072 | Apache 2.0 |
**Use `deepseek-coder-6.7b`.** It catches 83% of vulnerable functions while
flagging under 5% of clean ones, a 7x binary-F1 gain over the zero-shot base.
The headline is the false-positive rate, not the F1. The Qwen adapter is
included for completeness; it is a valid run but a weak detector.
### Honesty note (read before trusting any number)
On PrimeVul's paired split, where each vulnerable function sits beside its own
patched version, none of these models tells the two apart at better than chance
(thesis Section 15 / log Section 15). The BigVul numbers are real, but most of
what they measure is surface pattern, not the vulnerability. A high score is not
a safe signal that the model understands the bug.
## How to load
Rank 32, alpha 64, dropout 0.05, all attention and MLP projections, 4-bit base
(QLoRA). PEFT 0.13.2, TRL 1.5.0, Transformers 4.57.6.
| Subfolder | Base model id |
|---|---|
| `deepseek-coder-6.7b` | `deepseek-ai/deepseek-coder-6.7b-instruct` |
| `mistral-7b` | `mistralai/Mistral-7B-Instruct-v0.3` |
| `qwen2.5-coder-7b` | `Qwen/Qwen2.5-Coder-7B-Instruct` |
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
base_id = "deepseek-ai/deepseek-coder-6.7b-instruct"
repo = "phazertron/vulnerability-detector-sft"
sub = "deepseek-coder-6.7b"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(repo, subfolder=sub)
base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, repo, subfolder=sub)
```
Prompt the model exactly as it was trained:
```
You are a security auditor. Analyze the following C function.
Output ONLY: VULNERABLE: CWE-<id> OR CLEAN
Do not suggest fixes. Do not explain.
<function source here>
```
Expected output is one line: `VULNERABLE: CWE-787` or `CLEAN`.
## Training data
[BigVul](https://github.com/ZeoVan/MSR_20_Code_vulnerability_CSV_Dataset)
(real-world CVE functions). CASTLE and PrimeVul were held out and never trained
on. The adapters contain learned weights, not dataset content.
## Licensing
Each adapter inherits its base model's license, so this repository is
**mixed-licensed**:
- `mistral-7b`, `qwen2.5-coder-7b`: [Apache 2.0](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/blob/main/LICENSE).
- `deepseek-coder-6.7b`: [DeepSeek Model License](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct/blob/main/LICENSE). It permits redistribution and derivatives but requires that its use-based restrictions (Attachment A) be passed on and kept in force. By using this adapter you agree to them.
## Citation
```bibtex
@mastersthesis{bendini2026vulndetect,
author = {Claudio Bendini},
title = {Software Vulnerability Detection with Advanced Machine Learning Techniques},
school = {University of Parma},
year = {2026}
}
```