Text Generation
PEFT
Safetensors
English
lora
qlora
vulnerability-detection
security
cwe
llvm-ir
code
Instructions to use phazertron/vulnerability-detector-ir-source with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use phazertron/vulnerability-detector-ir-source with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
File size: 4,231 Bytes
26b6b80 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | ---
library_name: peft
pipeline_tag: text-generation
tags:
- lora
- qlora
- peft
- vulnerability-detection
- security
- cwe
- llvm-ir
- code
language:
- en
base_model:
- Qwen/Qwen2.5-Coder-7B-Instruct
license: apache-2.0
---
# Vulnerability Detector — IR vs source adapters (Phase 3B)
> **Status: valid research artifacts.** Both adapters are functional, but their
> purpose is to reproduce the one genuine positive finding of the thesis
> (Section 16): training on LLVM IR detects worse yet classifies the CWE far
> better than training on the matched C source. Use them to reproduce that
> dissociation, not as a general detector.
From the MSc thesis *Software Vulnerability Detection with Advanced Machine
Learning Techniques* (Claudio Bendini, University of Parma, 2025–2026, advisor
Prof.ssa Eleonora Iotti). Both adapters fine-tune the same base model,
Qwen2.5-Coder-7B-Instruct, on the same SARD Juliet programs, one arm on the C
source and the other on the `clang -O0 -emit-llvm` intermediate representation.
- **Code, data pipeline, full results:** https://github.com/Phazertron/Vulnerability_Detector
- **Canonical record:** [`docs/EXPERIMENT_LOG.md`](https://github.com/Phazertron/Vulnerability_Detector/blob/main/docs/EXPERIMENT_LOG.md), Section 16.
- **Companion repos:** [Phase 2 SFT](https://huggingface.co/phazertron/vulnerability-detector-sft) (working detectors) and [DPO](https://huggingface.co/phazertron/vulnerability-detector-dpo-negative) (a negative result).
## The finding
On the parity split (246 records both arms could handle):
| Metric | `source` arm | `ir` arm | Gap |
|---|---|---|---|
| Binary F1 (detection) | 0.5069 | 0.3116 | -0.1954 |
| CWE accuracy given detection | 0.1455 | **0.7742** | **+0.6287** |
Restricted to the 11 records both arms detected correctly, CWE accuracy is
1/11 for source against 8/11 for IR (McNemar exact p = 0.039; 95% CI on the gap
[+0.18, +1.00], excludes zero). IR is **worse at deciding whether code is
vulnerable** and **much better at naming which weakness it is**, and the
defensible claim is the +0.18 lower bound, not the point estimate. The effect
spreads across 13 CWEs rather than concentrating on memory-safety ones.
## The adapters
| Subfolder | Trained on | Role |
|---|---|---|
| `ir` | SARD Juliet, LLVM IR | the IR arm (strong CWE attribution) |
| `source` | SARD Juliet, C source | the matched baseline for comparison |
Both are for reproducing Section 16. Neither is a strong real-world detector:
they are trained on synthetic Juliet programs and do not transfer to real-world
code (Section 14).
## How to load
Both use base `Qwen/Qwen2.5-Coder-7B-Instruct`. Rank 32, alpha 64, dropout 0.05,
all attention and MLP projections, 4-bit base (QLoRA), 2048-token sequence
budget. PEFT 0.13.2, TRL 1.5.0, Transformers 4.57.6.
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
base_id = "Qwen/Qwen2.5-Coder-7B-Instruct"
repo = "phazertron/vulnerability-detector-ir-source"
sub = "ir" # or "source"
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(repo, subfolder=sub)
base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, repo, subfolder=sub)
```
The `source` arm is prompted with the C function; the `ir` arm with the
function's LLVM IR (see the pipeline in the GitHub repo). Expected output is one
line: `VULNERABLE: CWE-787` or `CLEAN`.
## Training data
[SARD Juliet](https://samate.nist.gov/SARD/test-suites/112) (compilable
synthetic C). The adapters contain learned weights, not dataset content.
## Licensing
Both adapters derive from Qwen2.5-Coder-7B-Instruct and are released under
[Apache 2.0](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/blob/main/LICENSE).
## Citation
```bibtex
@mastersthesis{bendini2026vulndetect,
author = {Claudio Bendini},
title = {Software Vulnerability Detection with Advanced Machine Learning Techniques},
school = {University of Parma},
year = {2026}
}
```
|