tanaymitra01 commited on
Commit
832a08e
·
verified ·
1 Parent(s): c61043c

Add model card with metadata, metrics, and usage

Browse files
Files changed (1) hide show
  1. README.md +129 -0
README.md ADDED
@@ -0,0 +1,129 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: transformers
4
+ pipeline_tag: text-classification
5
+ base_model: microsoft/graphcodebert-base
6
+ tags:
7
+ - code
8
+ - solidity
9
+ - smart-contracts
10
+ - vulnerability-detection
11
+ - graphcodebert
12
+ - roberta
13
+ language:
14
+ - en
15
+ metrics:
16
+ - accuracy
17
+ - f1
18
+ model-index:
19
+ - name: graphcodebert-vulnerability-detector
20
+ results:
21
+ - task:
22
+ type: text-classification
23
+ name: Solidity vulnerability classification
24
+ dataset:
25
+ name: SmartBugs-Wild (subset, tool-consensus labels)
26
+ type: smartbugs-wild
27
+ metrics:
28
+ - type: accuracy
29
+ value: 0.5622
30
+ name: Test accuracy
31
+ - type: f1
32
+ value: 0.4655
33
+ name: Macro F1
34
+ ---
35
+
36
+ # Graph CodeBERT — Solidity Vulnerability Detector
37
+
38
+ Fine-tuned [`microsoft/graphcodebert-base`](https://huggingface.co/microsoft/graphcodebert-base) for **Solidity smart-contract vulnerability type classification**.
39
+
40
+ Part of [SolidityGuard](https://github.com/tanaymitra54/solidity_guard_razorpay) — used as a first-pass detector alongside Slither and an LLM auditor.
41
+
42
+ ## Intended use
43
+
44
+ - Input: Solidity source code (string)
45
+ - Output: one of 12 labels (see below) + confidence
46
+ - Best as a **screening** signal, not a sole security audit
47
+
48
+ ## Labels
49
+
50
+ `safe`, `reentrancy`, `access_control`, `tx_origin_auth`, `integer_overflow`, `unsafe_delegatecall`, `weak_randomness`, `unbounded_loop`, `redundant_storage`, `gas_optimization`, `best_practice`, `other`
51
+
52
+ ## Training data
53
+
54
+ - **SmartBugs-Wild** contracts with tool-consensus labels from [SmartBugs results](https://github.com/smartbugs/smartbugs-results) (`metadata/results_wild.json`)
55
+ - Cap: 5,000 contracts; train/val/test split 70/15/15 with light augmentation
56
+
57
+ ## Held-out test metrics
58
+
59
+ | Metric | Value |
60
+ |--------|-------|
61
+ | Accuracy | 0.562 |
62
+ | Macro F1 | 0.466 |
63
+ | F1 `safe` | 0.694 |
64
+ | F1 `integer_overflow` | 0.634 |
65
+ | F1 `reentrancy` | 0.461 |
66
+ | F1 `other` | 0.340 |
67
+ | F1 `access_control` | 0.200 |
68
+
69
+ Labels are noisy (static-analysis consensus), so scores are moderate by design.
70
+
71
+ ## Quick start
72
+
73
+ ```python
74
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
75
+ import torch
76
+
77
+ repo = "tanaymitra01/graphcodebert-vulnerability-detector"
78
+ tok = AutoTokenizer.from_pretrained(repo)
79
+ model = AutoModelForSequenceClassification.from_pretrained(repo)
80
+ model.eval()
81
+
82
+ code = """
83
+ pragma solidity ^0.8.0;
84
+ contract Vault {
85
+ mapping(address => uint) public bal;
86
+ function withdraw() public {
87
+ uint amount = bal[msg.sender];
88
+ (bool ok,) = msg.sender.call{value: amount}("");
89
+ require(ok);
90
+ bal[msg.sender] = 0;
91
+ }
92
+ }
93
+ """
94
+ inputs = tok(code, return_tensors="pt", truncation=True, max_length=512)
95
+ with torch.no_grad():
96
+ probs = torch.softmax(model(**inputs).logits, dim=-1)[0]
97
+ pred = int(probs.argmax())
98
+ print(model.config.id2label[pred], float(probs[pred]))
99
+ ```
100
+
101
+ ### With SolidityGuard
102
+
103
+ ```bash
104
+ export GRAPHCODEBERT_PATH=tanaymitra01/graphcodebert-vulnerability-detector
105
+ ```
106
+
107
+ ## Files
108
+
109
+ - `model.safetensors` — weights
110
+ - `config.json` — RobertaForSequenceClassification config + label maps
111
+ - `label_map.json` — label list / id maps used in training
112
+
113
+ ## Limitations
114
+
115
+ - Tool-derived labels ≠ audited ground truth
116
+ - Truncation at 512 tokens; large contracts lose context
117
+ - Rare classes (e.g. access control) have low F1
118
+ - Not a replacement for professional security review
119
+
120
+ ## Citation
121
+
122
+ ```bibtex
123
+ @misc{solidityguard-graphcodebert,
124
+ title = {Graph CodeBERT Vulnerability Detector for Solidity},
125
+ author = {Tanay Mitra},
126
+ year = {2026},
127
+ url = {https://huggingface.co/tanaymitra01/graphcodebert-vulnerability-detector}
128
+ }
129
+ ```