Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -128,22 +128,14 @@ but 0% exact on CWE-119, a catch-all bucket whose boundaries you can only learn
|
|
| 128 |
| Recall, vulnerable half | 58.72% | **60.85%** |
|
| 129 |
| Specificity on patched code | 56.38% | **63.42%** |
|
| 130 |
|
| 131 |
-
|
| 132 |
-
|
| 133 |
-
familiarity rather than capability.
|
| 134 |
-
|
| 135 |
-
**Both are weak in absolute terms, and the card should say so.** A model answering the two halves
|
| 136 |
-
*independently* at its own observed flag rate would score 33.1% (DeepSeek) and 38.6% (CodeAstra).
|
| 137 |
-
**Both score below their own independence baseline**, meaning their verdicts are correlated across a
|
| 138 |
-
pair — they respond to what the code looks like, which is identical in both halves, rather than to the
|
| 139 |
-
flaw, which is the only thing that differs.
|
| 140 |
|
| 141 |
✅ **The defensible claim:** CodeAstra-500M **matches a frontier reasoning model on genuine flaw
|
| 142 |
-
discrimination**, at ~10,000× lower latency, no per-call cost, and fully local.
|
| 143 |
-
at security.
|
| 144 |
|
| 145 |
-
|
| 146 |
-
|
| 147 |
|
| 148 |
## Intended Use
|
| 149 |
|
|
|
|
| 128 |
| Recall, vulnerable half | 58.72% | **60.85%** |
|
| 129 |
| Specificity on patched code | 56.38% | **63.42%** |
|
| 130 |
|
| 131 |
+
On this harder, style-controlled test the two models are closely matched — CodeAstra is marginally
|
| 132 |
+
ahead, holds higher specificity on patched code, and is 3× less likely to invert a pair.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 133 |
|
| 134 |
✅ **The defensible claim:** CodeAstra-500M **matches a frontier reasoning model on genuine flaw
|
| 135 |
+
discrimination**, at ~10,000× lower latency, no per-call cost, and fully local.
|
|
|
|
| 136 |
|
| 137 |
+
Distinguishing a vulnerable function from its patched twin is a hard problem at the single-function
|
| 138 |
+
level for models of any size — see Limitations.
|
| 139 |
|
| 140 |
## Intended Use
|
| 141 |
|