Agent 2 β€” CodeBERT Vulnerability Classifier

CodeBERT-base fine-tuned to flag vulnerabilities in C/C++ source code. Part of a three-agent DevSecOps auditor (metrics β†’ transformer β†’ gatekeeper).

Model details

Field Value
Version 1.0.0 (initial release)
Base model microsoft/codebert-base
Architecture RobertaForSequenceClassification
Task Binary text classification
Labels SAFE (0), VULNERABLE (1)
Max length 384 tokens
Threshold 0.45
Training data CodeXGLUE defect detection + DiverseVul (C/C++)
Dataset version Not pinned β€” local snapshots, revisions not recorded

Files

File Purpose
config.json Architecture; label2id = {"SAFE": 0, "VULNERABLE": 1}
model.safetensors Fine-tuned weights (~476 MB)
tokenizer.json Fast tokenizer
tokenizer_config.json Tokenizer settings
meta.json Extra wrapper metadata (threshold, max length, metrics)

meta.json is not used by transformers; the values are repeated here.

Input and preprocessing

  • Input: one raw source-code string. No manual feature extraction.
  • Feature ordering: N/A β€” the tokenizer produces the input IDs and attention mask.
  • Tokenize with truncation=True, max_length=384. Longer code is truncated, so only the first 384 tokens affect the score.
  • Labels are taken from config.json (label2id).

Output

Raw: logits of shape (1, 2); apply softmax and read index 1:

p_vuln = softmax(logits, dim=-1)[0, model.config.label2id["VULNERABLE"]].item()
vulnerable = p_vuln >= 0.45

Reference wrapper output:

{"transformer_risk_score": 0.3013, "vulnerable": false, "threshold": 0.45, "status": "ok", "reason": ""}

Scores are softmax outputs, not calibrated probabilities. Recall is prioritized over precision, so false positives are expected. Non-C/C++ code is out-of-distribution.

Metrics

Metric Value
F1 0.6881
Precision 0.5804
Recall 0.8450
ROC-AUC 0.7475

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

name = "your-username/agent2-codebert-vuln"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name).eval()

inputs = tok(code, return_tensors="pt", truncation=True, max_length=384)
with torch.inference_mode():
    logits = model(**inputs).logits

p_vuln = torch.softmax(logits, -1)[0, model.config.label2id["VULNERABLE"]].item()
vulnerable = p_vuln >= 0.45

License

Not specified. Update the license field before publishing.

Downloads last month
-
Safetensors
Model size
0.1B params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for MhamedRayan/devsecops-transformer

Finetuned
(159)
this model