--- library_name: peft pipeline_tag: text-generation tags: - lora - qlora - peft - vulnerability-detection - security - cwe - llvm-ir - code language: - en base_model: - Qwen/Qwen2.5-Coder-7B-Instruct license: apache-2.0 --- # Vulnerability Detector — IR vs source adapters (Phase 3B) > **Status: valid research artifacts.** Both adapters are functional, but their > purpose is to reproduce the one genuine positive finding of the thesis > (Section 16): training on LLVM IR detects worse yet classifies the CWE far > better than training on the matched C source. Use them to reproduce that > dissociation, not as a general detector. From the MSc thesis *Software Vulnerability Detection with Advanced Machine Learning Techniques* (Claudio Bendini, University of Parma, 2025–2026, advisor Prof.ssa Eleonora Iotti). Both adapters fine-tune the same base model, Qwen2.5-Coder-7B-Instruct, on the same SARD Juliet programs, one arm on the C source and the other on the `clang -O0 -emit-llvm` intermediate representation. - **Code, data pipeline, full results:** https://github.com/Phazertron/Vulnerability_Detector - **Canonical record:** [`docs/EXPERIMENT_LOG.md`](https://github.com/Phazertron/Vulnerability_Detector/blob/main/docs/EXPERIMENT_LOG.md), Section 16. - **Companion repos:** [Phase 2 SFT](https://huggingface.co/phazertron/vulnerability-detector-sft) (working detectors) and [DPO](https://huggingface.co/phazertron/vulnerability-detector-dpo-negative) (a negative result). ## The finding On the parity split (246 records both arms could handle): | Metric | `source` arm | `ir` arm | Gap | |---|---|---|---| | Binary F1 (detection) | 0.5069 | 0.3116 | -0.1954 | | CWE accuracy given detection | 0.1455 | **0.7742** | **+0.6287** | Restricted to the 11 records both arms detected correctly, CWE accuracy is 1/11 for source against 8/11 for IR (McNemar exact p = 0.039; 95% CI on the gap [+0.18, +1.00], excludes zero). IR is **worse at deciding whether code is vulnerable** and **much better at naming which weakness it is**, and the defensible claim is the +0.18 lower bound, not the point estimate. The effect spreads across 13 CWEs rather than concentrating on memory-safety ones. ## The adapters | Subfolder | Trained on | Role | |---|---|---| | `ir` | SARD Juliet, LLVM IR | the IR arm (strong CWE attribution) | | `source` | SARD Juliet, C source | the matched baseline for comparison | Both are for reproducing Section 16. Neither is a strong real-world detector: they are trained on synthetic Juliet programs and do not transfer to real-world code (Section 14). ## How to load Both use base `Qwen/Qwen2.5-Coder-7B-Instruct`. Rank 32, alpha 64, dropout 0.05, all attention and MLP projections, 4-bit base (QLoRA), 2048-token sequence budget. PEFT 0.13.2, TRL 1.5.0, Transformers 4.57.6. ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig from peft import PeftModel base_id = "Qwen/Qwen2.5-Coder-7B-Instruct" repo = "phazertron/vulnerability-detector-ir-source" sub = "ir" # or "source" bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16) tok = AutoTokenizer.from_pretrained(repo, subfolder=sub) base = AutoModelForCausalLM.from_pretrained(base_id, quantization_config=bnb, device_map="auto") model = PeftModel.from_pretrained(base, repo, subfolder=sub) ``` The `source` arm is prompted with the C function; the `ir` arm with the function's LLVM IR (see the pipeline in the GitHub repo). Expected output is one line: `VULNERABLE: CWE-787` or `CLEAN`. ## Training data [SARD Juliet](https://samate.nist.gov/SARD/test-suites/112) (compilable synthetic C). The adapters contain learned weights, not dataset content. ## Licensing Both adapters derive from Qwen2.5-Coder-7B-Instruct and are released under [Apache 2.0](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/blob/main/LICENSE). ## Citation ```bibtex @mastersthesis{bendini2026vulndetect, author = {Claudio Bendini}, title = {Software Vulnerability Detection with Advanced Machine Learning Techniques}, school = {University of Parma}, year = {2026} } ```