phazertron's picture
Add model card
26b6b80 verified
|
Raw History Blame Contribute Delete
4.23 kB
metadata
library_name: peft
pipeline_tag: text-generation
tags:
  - lora
  - qlora
  - peft
  - vulnerability-detection
  - security
  - cwe
  - llvm-ir
  - code
language:
  - en
base_model:
  - Qwen/Qwen2.5-Coder-7B-Instruct
license: apache-2.0

Vulnerability Detector — IR vs source adapters (Phase 3B)

Status: valid research artifacts. Both adapters are functional, but their purpose is to reproduce the one genuine positive finding of the thesis (Section 16): training on LLVM IR detects worse yet classifies the CWE far better than training on the matched C source. Use them to reproduce that dissociation, not as a general detector.

From the MSc thesis Software Vulnerability Detection with Advanced Machine Learning Techniques (Claudio Bendini, University of Parma, 2025–2026, advisor Prof.ssa Eleonora Iotti). Both adapters fine-tune the same base model, Qwen2.5-Coder-7B-Instruct, on the same SARD Juliet programs, one arm on the C source and the other on the clang -O0 -emit-llvm intermediate representation.

The finding

On the parity split (246 records both arms could handle):

Metric source arm ir arm Gap
Binary F1 (detection) 0.5069 0.3116 -0.1954
CWE accuracy given detection 0.1455 0.7742 +0.6287

Restricted to the 11 records both arms detected correctly, CWE accuracy is 1/11 for source against 8/11 for IR (McNemar exact p = 0.039; 95% CI on the gap [+0.18, +1.00], excludes zero). IR is worse at deciding whether code is vulnerable and much better at naming which weakness it is, and the defensible claim is the +0.18 lower bound, not the point estimate. The effect spreads across 13 CWEs rather than concentrating on memory-safety ones.

The adapters

Subfolder Trained on Role
ir SARD Juliet, LLVM IR the IR arm (strong CWE attribution)
source SARD Juliet, C source the matched baseline for comparison

Both are for reproducing Section 16. Neither is a strong real-world detector: they are trained on synthetic Juliet programs and do not transfer to real-world code (Section 14).

How to load

Both use base Qwen/Qwen2.5-Coder-7B-Instruct. Rank 32, alpha 64, dropout 0.05, all attention and MLP projections, 4-bit base (QLoRA), 2048-token sequence budget. PEFT 0.13.2, TRL 1.5.0, Transformers 4.57.6.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

base_id = "Qwen/Qwen2.5-Coder-7B-Instruct"
repo    = "phazertron/vulnerability-detector-ir-source"
sub     = "ir"   # or "source"

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained(repo, subfolder=sub)
base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(base, repo, subfolder=sub)

The source arm is prompted with the C function; the ir arm with the function's LLVM IR (see the pipeline in the GitHub repo). Expected output is one line: VULNERABLE: CWE-787 or CLEAN.

Training data

SARD Juliet (compilable synthetic C). The adapters contain learned weights, not dataset content.

Licensing

Both adapters derive from Qwen2.5-Coder-7B-Instruct and are released under Apache 2.0.

Citation

@mastersthesis{bendini2026vulndetect,
  author = {Claudio Bendini},
  title  = {Software Vulnerability Detection with Advanced Machine Learning Techniques},
  school = {University of Parma},
  year   = {2026}
}