README.md

---
license: apache-2.0
library_name: gliner2
pipeline_tag: text-classification
language:
  - en
tags:
  - decision-model
  - calibration
  - selective-prediction
  - conformal-prediction
  - abstention
  - risk-aware
  - security
  - cybersecurity
  - security-operations
  - soc
  - pentest
  - red-team
  - blue-team
  - gliner2
base_model: fastino/GLiNER2.5-Decide
metrics:
  - accuracy
model-index:
  - name: haruspex
    results:
      - task:
          type: text-classification
          name: Security decision (boolean / choice / score)
        dataset:
          type: custom
          name: web held-out (156 rows)
        metrics:
          - type: accuracy
            value: 0.8718
            name: accuracy
          - type: ece
            value: 0.1076
            name: ECE
          - type: brier
            value: 0.1071
            name: BRIER
---

# TyrusRC/haruspex

**Haruspex** is a small, local **decision layer** for security operations. It reads a
situation and returns a *calibrated* typed verdict — **boolean** (yes/no + probability),
**choice** (pick one option), **score** (rate on a rubric) — and **abstains** when it is
not confident enough for your risk budget.

Haruspex **reuses** the published decision encoder
[`fastino/GLiNER2.5-Decide`](https://huggingface.co/fastino/GLiNER2.5-Decide) (Apache-2.0, non-autoregressive, CPU/GPU) and
specializes it on security decisions. It does not train a decision model from scratch.

- Base: `fastino/GLiNER2.5-Decide`.
- Added: a typed decision contract, security specialization data, and a
  split-conformal **selective-calibration** layer (accept/abstain against a risk budget).
- Serves the TypeSafe `POST /v1/systemone` and native `/v1/evaluate` contracts.

## Intended use (security, both sides)

- **Offensive (pentest / red-team):** is this template/probe relevant to the target?
  is this scanner hit a real finding? finding severity; verdict discipline
  (vulnerable / not-vulnerable / inconclusive); attack-path next step.
- **Defensive (SOC / blue-team):** alert triage (true vs false positive), alert
  severity/priority, detection-rule applicability, incident routing, CVE applicability.

It is a classifier for bounded decisions — not a chatbot, not an exploit generator.

## Use

```python
from haruspex import Haruspex
m = Haruspex.load("TyrusRC/haruspex", device="cuda")   # or "mps" / "cpu"
print(m.decide({"type":"boolean","state":"Target: WordPress 6.2, PHP 8.1",
                 "question":"Is a WordPress xmlrpc.php template relevant?",
                 "criteria":{"true":"WordPress target","false":"not WordPress"}}))
```

Or serve the decision API (keyless, self-hosted):

```sh
pip install "haruspex[serve] @ git+https://github.com/TyrusRC/haruspex"
haruspex-serve --model TyrusRC/haruspex --port 3000      # POST /v1/systemone  ·  /v1/evaluate
```

## Calibration

Fitted calibrator: accept threshold `0.0`, target risk `0.2`, coverage `1.0` (n=116).

## Evaluation

On a 156-row held-out split of the security pack (recipe selected on a separate calibration split, never on test): **accuracy 0.8718**, ECE 0.1076, Brier 0.1071. The adapter beats a prompted general 8B model on the same set at a small fraction of the parameters; see the repo's `docs/benchmark.md` comparison card.

A split-conformal selective layer gates each verdict: it accepts when the top-1
confidence clears a threshold fitted to hold the error AMONG accepted items at a target
risk, and abstains otherwise. Fit it with `python -m haruspex.calibrate`; evaluate
coverage/risk with `python -m haruspex.eval`.

## Limitations

- A specialist for the decision types above; novel free-form judgments belong to a
  general LLM.
- Calibrated on the training mix — recalibrate for a very different domain.
- Prediction SETS (not just abstention) await a GLiNER2 fix for per-label scores.

Apache-2.0. Repo + recipe: https://github.com/TyrusRC/haruspex