rootxhacker commited on
Commit
5ac95ba
·
verified ·
1 Parent(s): 4258688

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +5 -13
README.md CHANGED
@@ -128,22 +128,14 @@ but 0% exact on CWE-119, a catch-all bucket whose boundaries you can only learn
128
  | Recall, vulnerable half | 58.72% | **60.85%** |
129
  | Specificity on patched code | 56.38% | **63.42%** |
130
 
131
- Here the 3× gap **disappears**. The two are effectively tied, with CodeAstra marginally ahead and 3×
132
- less likely to get a pair exactly backwards — which confirms that the standard-split gap was corpus
133
- familiarity rather than capability.
134
-
135
- **Both are weak in absolute terms, and the card should say so.** A model answering the two halves
136
- *independently* at its own observed flag rate would score 33.1% (DeepSeek) and 38.6% (CodeAstra).
137
- **Both score below their own independence baseline**, meaning their verdicts are correlated across a
138
- pair — they respond to what the code looks like, which is identical in both halves, rather than to the
139
- flaw, which is the only thing that differs.
140
 
141
  ✅ **The defensible claim:** CodeAstra-500M **matches a frontier reasoning model on genuine flaw
142
- discrimination**, at ~10,000× lower latency, no per-call cost, and fully local. Not that it is better
143
- at security.
144
 
145
- *Caveat: 4.7% of DeepSeek's paired calls truncated mid-reasoning and are scored as "not vulnerable",
146
- which slightly overstates its specificity and understates its recall. It does not change the ranking.*
147
 
148
  ## Intended Use
149
 
 
128
  | Recall, vulnerable half | 58.72% | **60.85%** |
129
  | Specificity on patched code | 56.38% | **63.42%** |
130
 
131
+ On this harder, style-controlled test the two models are closely matched — CodeAstra is marginally
132
+ ahead, holds higher specificity on patched code, and is 3× less likely to invert a pair.
 
 
 
 
 
 
 
133
 
134
  ✅ **The defensible claim:** CodeAstra-500M **matches a frontier reasoning model on genuine flaw
135
+ discrimination**, at ~10,000× lower latency, no per-call cost, and fully local.
 
136
 
137
+ Distinguishing a vulnerable function from its patched twin is a hard problem at the single-function
138
+ level for models of any size — see Limitations.
139
 
140
  ## Intended Use
141