File size: 2,901 Bytes
6d2d9cd
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2341dac
 
 
 
 
 
 
 
 
 
 
 
 
 
6d2d9cd
 
 
 
 
 
 
 
 
 
 
 
2341dac
 
 
 
 
 
 
 
 
 
6d2d9cd
2341dac
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6d2d9cd
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
---
license: other
library_name: transformers
pipeline_tag: text-classification
base_model: microsoft/codebert-base
tags:
  - codebert
  - cybersecurity
  - vulnerability-detection
  - code
  - security
  - text-classification
---

# Agent 2 — CodeBERT Vulnerability Classifier

CodeBERT-base fine-tuned to flag vulnerabilities in C/C++ source code.
Part of a three-agent DevSecOps auditor (metrics → transformer → gatekeeper).

## Model details

| Field | Value |
|---|---|
| Version | 1.0.0 (initial release) |
| Base model | `microsoft/codebert-base` |
| Architecture | `RobertaForSequenceClassification` |
| Task | Binary text classification |
| Labels | `SAFE` (0), `VULNERABLE` (1) |
| Max length | **384** tokens |
| Threshold | **0.45** |
| Training data | CodeXGLUE defect detection + DiverseVul (C/C++) |
| Dataset version | Not pinned — local snapshots, revisions not recorded |

## Files

| File | Purpose |
|---|---|
| `config.json` | Architecture; `label2id` = `{"SAFE": 0, "VULNERABLE": 1}` |
| `model.safetensors` | Fine-tuned weights (~476 MB) |
| `tokenizer.json` | Fast tokenizer |
| `tokenizer_config.json` | Tokenizer settings |
| `meta.json` | Extra wrapper metadata (threshold, max length, metrics) |

`meta.json` is not used by `transformers`; the values are repeated here.

## Input and preprocessing

- Input: one raw source-code string. No manual feature extraction.
- Feature ordering: **N/A** — the tokenizer produces the input IDs and
  attention mask.
- Tokenize with `truncation=True, max_length=384`. Longer code is truncated,
  so only the first 384 tokens affect the score.
- Labels are taken from `config.json` (`label2id`).

## Output

Raw: logits of shape `(1, 2)`; apply softmax and read index 1:

```python
p_vuln = softmax(logits, dim=-1)[0, model.config.label2id["VULNERABLE"]].item()
vulnerable = p_vuln >= 0.45
```

Reference wrapper output:

```json
{"transformer_risk_score": 0.3013, "vulnerable": false, "threshold": 0.45, "status": "ok", "reason": ""}
```

Scores are softmax outputs, not calibrated probabilities. Recall is prioritized
over precision, so false positives are expected. Non-C/C++ code is
out-of-distribution.

## Metrics

| Metric | Value |
|---|---|
| F1 | 0.6881 |
| Precision | 0.5804 |
| Recall | 0.8450 |
| ROC-AUC | 0.7475 |

## Usage

```python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

name = "your-username/agent2-codebert-vuln"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name).eval()

inputs = tok(code, return_tensors="pt", truncation=True, max_length=384)
with torch.inference_mode():
    logits = model(**inputs).logits

p_vuln = torch.softmax(logits, -1)[0, model.config.label2id["VULNERABLE"]].item()
vulnerable = p_vuln >= 0.45
```

## License

Not specified. Update the `license` field before publishing.