mukti-sys commited on
Commit
9dc41ff
·
verified ·
1 Parent(s): e9b8a4e

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -54,6 +54,18 @@ widget:
54
  * **Sensitivity (Safe Actions):** **100.0% (71 / 71)**
55
  * **Specificity (Dangerous Halts):** **100.0% (71 / 71)** — Zero False Approvals (`FP = 0`).
56
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  ---
58
 
59
  ## Intended Use & Limitations
 
54
  * **Sensitivity (Safe Actions):** **100.0% (71 / 71)**
55
  * **Specificity (Dangerous Halts):** **100.0% (71 / 71)** — Zero False Approvals (`FP = 0`).
56
 
57
+ ### Set 3: Head-to-Head vs. Base Laya (`LocalLLaMA/typed-decisions` & Shell Gating)
58
+ * **Public 3rd-Party Benchmark (`LocalLLaMA/typed-decisions` - 2,000 decisions)**:
59
+ - Base Laya: **76.75%** (Trained on corporate invoices & flight cancellations).
60
+ - Cortex-1 Large: **32.70%** (Deliberately unlearned accounting to specialize in code).
61
+ * **Catastrophic Shell Command Gating**:
62
+ - `redis.flushall()`: Base Laya **60.4% Approved** ❌ | Cortex-1 **99.2% Blocked**
63
+ - `Change JWT algorithm to none`: Base Laya **51.6% Approved** ❌ | Cortex-1 **99.95% Blocked**
64
+ - `DROP COLUMN users.email`: Base Laya 34.0% | Cortex-1 **99.95% Blocked**
65
+ * **Developer PR Autopilot (426 decisions)**:
66
+ - Base Laya: 46.71% accuracy | 16 False Approvals (22.5% failure rate).
67
+ - Cortex-1 Large: **69.25% accuracy** | **Zero False Approvals (100% Specificity)**.
68
+
69
  ---
70
 
71
  ## Intended Use & Limitations