Update mobile adapter

mobile/README.md

@@ -23,27 +23,27 @@ base_model: fastino/GLiNER2.5-Decide
 metrics:
   - accuracy
 model-index:
-  - name: haruspex-mobile
+  - name: haruspex
     results:
       - task:
           type: text-classification
           name: Security decision (boolean / choice / score)
         dataset:
           type: custom
-          name: mobile (MASVS) held-out (41 rows)
+          name: mobile pack test split (117 rows)
         metrics:
           - type: accuracy
-            value: 0.9756
+            value: 0.9231
             name: accuracy
           - type: ece
-            value: 0.0244
-            name: ECE
+            value: 0.0598
+            name: ece
           - type: brier
-            value: 0.0244
-            name: BRIER
+            value: 0.0598
+            name: brier
 ---
 
-# TyrusRC/haruspex-mobile
+# TyrusRC/haruspex (mobile)
 
 **Haruspex** is a small, local **decision layer** for security operations. It reads a
 situation and returns a *calibrated* typed verdict — **boolean** (yes/no + probability),
@@ -55,8 +55,9 @@ Haruspex **reuses** the published decision encoder
 specializes it on security decisions. It does not train a decision model from scratch.
 
 - Base: `fastino/GLiNER2.5-Decide`.
-- Added: a typed decision contract, security specialization data, and a
-  split-conformal **selective-calibration** layer (accept/abstain against a risk budget).
+- Added: a typed decision contract, security specialization data, a certified
+  **selective-calibration** layer (accept/abstain against a risk budget), conformal
+  **prediction sets**, and a **novelty gate** that abstains on unfamiliar input.
 - Serves the TypeSafe `POST /v1/systemone` and native `/v1/evaluate` contracts.
 
 ## Intended use (security, both sides)
@@ -64,8 +65,10 @@ specializes it on security decisions. It does not train a decision model from sc
 - **Offensive (pentest / red-team):** is this template/probe relevant to the target?
   is this scanner hit a real finding? finding severity; verdict discipline
   (vulnerable / not-vulnerable / inconclusive); attack-path next step.
-- **Defensive (SOC / blue-team):** alert triage (true vs false positive), alert
-  severity/priority, detection-rule applicability, incident routing, CVE applicability.
+- **Defensive (SOC / blue-team):** alert and incident triage (true / benign / false
+  positive), alert severity/priority, detection-rule applicability, incident routing, CVE
+  applicability, ATT&CK tactic of Windows/Sysmon events. Raw Defender, Sentinel, Elastic
+  ECS and Windows event JSON is normalised before the model reads it.
 
 It is a classifier for bounded decisions — not a chatbot, not an exploit generator.
 
@@ -73,7 +76,7 @@ It is a classifier for bounded decisions — not a chatbot, not an exploit gener
 
 ```python
 from haruspex import Haruspex
-m = Haruspex.load("TyrusRC/haruspex-mobile", device="cuda")   # or "mps" / "cpu"
+m = Haruspex.load("TyrusRC/haruspex", subfolder="mobile", device="cuda")   # or "mps" / "cpu"
 print(m.decide({"type":"boolean","state":"Target: WordPress 6.2, PHP 8.1",
                  "question":"Is a WordPress xmlrpc.php template relevant?",
                  "criteria":{"true":"WordPress target","false":"not WordPress"}}))
@@ -83,27 +86,39 @@ Or serve the decision API (keyless, self-hosted):
 
 ```sh
 pip install "haruspex[serve] @ git+https://github.com/TyrusRC/haruspex"
-haruspex-serve --model TyrusRC/haruspex-mobile --port 3000      # POST /v1/systemone  ·  /v1/evaluate
+haruspex-serve --adapters mobile=TyrusRC/haruspex#mobile --port 3000      # POST /v1/systemone  ·  /v1/evaluate
 ```
 
 ## Calibration
 
-Fitted calibrator: accept threshold `0.920385`, target risk `0.2`, coverage `1.0` (n=30).
+Fitted calibrator: accept threshold `0.0`, target risk `0.2` at delta `0.1`, coverage on the calibration split `1.0` (n=78), temperature `1.0`.
+
+Each verdict carries the full probability distribution over its options (temperature-
+scaled). A Learn-then-Test selective layer accepts a verdict only when its confidence
+clears a threshold certified on a held-out calibration split, so that with probability
+1 - delta the error among accepted verdicts stays within the target risk; otherwise it
+abstains with `"route": "escalate"`. A conformal prediction set lists the options the
+true answer is among, and a novelty gate abstains on inputs unlike the training data.
+Fit with `python -m haruspex.calibrate` and `python -m haruspex.ood`; evaluate with
+`python -m haruspex.eval`.
 
 ## Evaluation
 
-On a 41-row held-out split of the security pack (recipe selected on a separate calibration split, never on test): **accuracy 0.9756**, ECE 0.0244, Brier 0.0244. The adapter beats a prompted general 8B model on the same set at a small fraction of the parameters; see the repo's `docs/benchmark.md` comparison card.
+Held-out test sets, never used for training, model selection or calibration:
+
+- **mobile pack test split (117 rows)**: accuracy 0.9231, ECE 0.0598, Brier 0.0598; with the calibrator it answers 1.0 of rows at accuracy 0.9391 and abstains on the rest.
 
-A split-conformal selective layer gates each verdict: it accepts when the top-1
-confidence clears a threshold fitted to hold the error AMONG accepted items at a target
-risk, and abstains otherwise. Fit it with `python -m haruspex.calibrate`; evaluate
-coverage/risk with `python -m haruspex.eval`.
+Comparisons against general LLMs are in the repo's `docs/benchmark.md`.
 
 ## Limitations
 
 - A specialist for the decision types above; novel free-form judgments belong to a
   general LLM.
-- Calibrated on the training mix — recalibrate for a very different domain.
-- Prediction SETS (not just abstention) await a GLiNER2 fix for per-label scores.
+- Calibrated on this domain's held-out split; recalibrate before relying on the
+  guarantee on your own data.
+- Prediction-set coverage is marginal over the calibration mix, not per input.
+- A calibrator fitted on a mix of domains holds its risk target on average over that
+  mix, not on each domain: check each test set's accepted accuracy under Evaluation,
+  and route a domain with its own adapter to that adapter.
 
 Apache-2.0. Repo + recipe: https://github.com/TyrusRC/haruspex

mobile/calibrator.json

@@ -1,7 +1,12 @@
 {
-  "threshold": 0.99997,
+  "threshold": 0.0,
   "target_risk": 0.2,
-  "achieved_risk": 0.038462,
+  "achieved_risk": 0.012821,
   "coverage": 1.0,
-  "n": 78
+  "n": 78,
+  "delta": 0.1,
+  "temperature": 1.0,
+  "temperature_scope": "all",
+  "set_q": 0.0,
+  "set_alpha": 0.1
 }

mobile/eval.json

@@ -1,7 +1,29 @@
 {
-  "accuracy": 0.9466,
-  "ece": 0.0529,
-  "brier": 0.0523,
-  "test_n": 131,
-  "dataset": "mobile held-out"
+ "n": 117,
+ "accuracy": 0.9231,
+ "accuracy_ci95": [
+  0.8602,
+  0.959
+ ],
+ "brier": 0.0598,
+ "ece": 0.0598,
+ "ece_adaptive": 0.0598,
+ "p50_ms": 41.9,
+ "by_source": {
+  "model": {
+   "n": 115,
+   "accuracy": 0.9391
+  },
+  "model_declined": {
+   "n": 2,
+   "accuracy": 0.0
+  }
+ },
+ "selective": {
+  "coverage": 1.0,
+  "accepted_accuracy": 0.9391,
+  "accepted_risk": 0.0609
+ },
+ "test_n": 117,
+ "dataset": "mobile pack test split"
 }

mobile/final/adapter_config.json

@@ -35,22 +35,22 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "classifier.2",
-    "projector.0",
     "out_project.3",
-    "project_end.0",
     "out_project.0",
-    "query_proj",
     "project_end.3",
-    "classifier.0",
-    "project_start.0",
-    "project_start.3",
-    "projector.2",
+    "classifier.2",
     "count_pred.2",
-    "value_proj",
+    "project_start.3",
     "count_pred.0",
+    "project_start.0",
     "dense",
-    "key_proj"
+    "query_proj",
+    "key_proj",
+    "projector.0",
+    "classifier.0",
+    "projector.2",
+    "value_proj",
+    "project_end.0"
   ],
   "target_parameters": null,
   "task_type": null,

mobile/final/adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2acdb5e92e793f024569e1dddbd174fe2f7374b86e9888fd4be7708bb7ff805f
+oid sha256:45fd02c28672f68373fc5ef17bea5fa3a6935e70be24ba39f4be3c2e2d6545e4
 size 31766440

mobile/ood.json

@@ -0,0 +1 @@
+{"alpha": 0.01, "dim": 2048, "cal_scores": [0.530308, 0.610907, 0.618065, 0.621887, 0.646794, 0.67057, 0.672206, 0.709406, 0.729524, 0.831814, 0.833628, 0.859725, 0.999999, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]}

mobile/ood_train.npy

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:48e1c49ef47f8ea2c07bcde77552c94f01b4c1f2172156a4c4466590b87513ea
+size 1892480