--- library_name: peft pipeline_tag: text-generation tags: - lora - qlora - peft - vulnerability-detection - security - cwe - code language: - en base_model: - Qwen/Qwen2.5-Coder-7B-Instruct - deepseek-ai/deepseek-coder-6.7b-instruct - mistralai/Mistral-7B-Instruct-v0.3 license: other license_name: mixed-per-adapter license_link: https://github.com/Phazertron/Vulnerability_Detector --- # Vulnerability Detector — Phase 2 SFT adapters (working detectors) > **Status: functional.** These three QLoRA adapters work as C/C++ vulnerability > detectors and can be used for inference. They also reproduce Section 3 of the > thesis. Read the honesty note before trusting a score. From the MSc thesis *Software Vulnerability Detection with Advanced Machine Learning Techniques* (Claudio Bendini, University of Parma, 2025–2026, advisor Prof.ssa Eleonora Iotti). Each adapter classifies a C/C++ function as `CLEAN` or `VULNERABLE: CWE-`. Detection only, no remediation. - **Code, data pipeline, full results:** https://github.com/Phazertron/Vulnerability_Detector - **Canonical record:** [`docs/EXPERIMENT_LOG.md`](https://github.com/Phazertron/Vulnerability_Detector/blob/main/docs/EXPERIMENT_LOG.md), Section 3. - **Companion repos:** [IR vs source](https://huggingface.co/phazertron/vulnerability-detector-ir-source) (a valid positive finding) and [DPO](https://huggingface.co/phazertron/vulnerability-detector-dpo-negative) (a negative result). ## The adapters Supervised fine-tuning on real-world BigVul functions. Results on the BigVul test split: | Subfolder | Base model | Binary F1 | Recall | FPR | Base license | |---|---|---|---|---|---| | `deepseek-coder-6.7b` | DeepSeek-Coder-6.7B-instruct | **0.5813** | **0.8343** | 0.0469 | DeepSeek Model License | | `mistral-7b` | Mistral-7B-Instruct-v0.3 | 0.4923 | 0.7130 | 0.0536 | Apache 2.0 | | `qwen2.5-coder-7b` | Qwen2.5-Coder-7B-Instruct | 0.2156 | 0.1400 | 0.0072 | Apache 2.0 | **Use `deepseek-coder-6.7b`.** It catches 83% of vulnerable functions while flagging under 5% of clean ones, a 7x binary-F1 gain over the zero-shot base. The headline is the false-positive rate, not the F1. The Qwen adapter is included for completeness; it is a valid run but a weak detector. ### Honesty note (read before trusting any number) On PrimeVul's paired split, where each vulnerable function sits beside its own patched version, none of these models tells the two apart at better than chance (thesis Section 15 / log Section 15). The BigVul numbers are real, but most of what they measure is surface pattern, not the vulnerability. A high score is not a safe signal that the model understands the bug. ## How to load Rank 32, alpha 64, dropout 0.05, all attention and MLP projections, 4-bit base (QLoRA). PEFT 0.13.2, TRL 1.5.0, Transformers 4.57.6. | Subfolder | Base model id | |---|---| | `deepseek-coder-6.7b` | `deepseek-ai/deepseek-coder-6.7b-instruct` | | `mistral-7b` | `mistralai/Mistral-7B-Instruct-v0.3` | | `qwen2.5-coder-7b` | `Qwen/Qwen2.5-Coder-7B-Instruct` | ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel base_id = "deepseek-ai/deepseek-coder-6.7b-instruct" repo = "phazertron/vulnerability-detector-sft" sub = "deepseek-coder-6.7b" bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16) tok = AutoTokenizer.from_pretrained(repo, subfolder=sub) base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto") model = PeftModel.from_pretrained(base, repo, subfolder=sub) ``` Prompt the model exactly as it was trained: ``` You are a security auditor. Analyze the following C function. Output ONLY: VULNERABLE: CWE- OR CLEAN Do not suggest fixes. Do not explain. ``` Expected output is one line: `VULNERABLE: CWE-787` or `CLEAN`. ## Training data [BigVul](https://github.com/ZeoVan/MSR_20_Code_vulnerability_CSV_Dataset) (real-world CVE functions). CASTLE and PrimeVul were held out and never trained on. The adapters contain learned weights, not dataset content. ## Licensing Each adapter inherits its base model's license, so this repository is **mixed-licensed**: - `mistral-7b`, `qwen2.5-coder-7b`: [Apache 2.0](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/blob/main/LICENSE). - `deepseek-coder-6.7b`: [DeepSeek Model License](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct/blob/main/LICENSE). It permits redistribution and derivatives but requires that its use-based restrictions (Attachment A) be passed on and kept in force. By using this adapter you agree to them. ## Citation ```bibtex @mastersthesis{bendini2026vulndetect, author = {Claudio Bendini}, title = {Software Vulnerability Detection with Advanced Machine Learning Techniques}, school = {University of Parma}, year = {2026} } ```