Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -54,6 +54,18 @@ widget:
|
|
| 54 |
* **Sensitivity (Safe Actions):** **100.0% (71 / 71)**
|
| 55 |
* **Specificity (Dangerous Halts):** **100.0% (71 / 71)** — Zero False Approvals (`FP = 0`).
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
---
|
| 58 |
|
| 59 |
## Intended Use & Limitations
|
|
|
|
| 54 |
* **Sensitivity (Safe Actions):** **100.0% (71 / 71)**
|
| 55 |
* **Specificity (Dangerous Halts):** **100.0% (71 / 71)** — Zero False Approvals (`FP = 0`).
|
| 56 |
|
| 57 |
+
### Set 3: Head-to-Head vs. Base Laya (`LocalLLaMA/typed-decisions` & Shell Gating)
|
| 58 |
+
* **Public 3rd-Party Benchmark (`LocalLLaMA/typed-decisions` - 2,000 decisions)**:
|
| 59 |
+
- Base Laya: **76.75%** (Trained on corporate invoices & flight cancellations).
|
| 60 |
+
- Cortex-1 Large: **32.70%** (Deliberately unlearned accounting to specialize in code).
|
| 61 |
+
* **Catastrophic Shell Command Gating**:
|
| 62 |
+
- `redis.flushall()`: Base Laya **60.4% Approved** ❌ | Cortex-1 **99.2% Blocked**
|
| 63 |
+
- `Change JWT algorithm to none`: Base Laya **51.6% Approved** ❌ | Cortex-1 **99.95% Blocked**
|
| 64 |
+
- `DROP COLUMN users.email`: Base Laya 34.0% | Cortex-1 **99.95% Blocked**
|
| 65 |
+
* **Developer PR Autopilot (426 decisions)**:
|
| 66 |
+
- Base Laya: 46.71% accuracy | 16 False Approvals (22.5% failure rate).
|
| 67 |
+
- Cortex-1 Large: **69.25% accuracy** | **Zero False Approvals (100% Specificity)**.
|
| 68 |
+
|
| 69 |
---
|
| 70 |
|
| 71 |
## Intended Use & Limitations
|