CRP Intent SetFit — speech-act classifier

Part of the Context Relay Protocol (CRP) ML-first governance layer. Classifies a user turn into one of four CRP speech acts — request, question, assertion, expressive — used by crp/isa/intent.py (SPEC-051 Intent & Speech Acts) to drive routing and operation framing in agentic pipelines.

Few-shot SetFit model: contrastive-tuned all-MiniLM-L6-v2 body + LogisticRegression head fit on the full training mix.

Verified results (independent harness, 2026-07-28)

Metric	Value
Held-out accuracy (2,000 unseen examples)	0.9340
F1 assertion / expressive / question / request	0.980 / 1.000 / 0.880 / 0.933
Production-style CRP prompts	18/20

Training data

Banking77 + SNIPS (intent names heuristically mapped to the four CRP speech acts) + templated synthetic examples per class. 64-shot contrastive body tuning, then the classifier head refit on the full mix.

Usage

from setfit import SetFitModel

model = SetFitModel.from_pretrained("AutoCyberAI/crp-intent-setfit")
model.predict(["Please scan the repository for compliance issues."])
# -> ['request']

In the CRP SDK this model is the default intent backend:

CRP_INTENT_MODEL=AutoCyberAI/crp-intent-setfit  # default; no env needed

Limitations

Four speech acts only; trained on English service/assistant phrasing. The held-out score is in-distribution (same data mix); the production-prompt score above is the out-of-distribution signal.

License

Elastic License 2.0 — see the CRP repository for details.

Downloads last month: 48

Safetensors

Model size

22.7M params

Tensor type

F32

Model tree for AutoCyberAI/crp-intent-setfit

Base model

nreimers/MiniLM-L6-H384-uncased

Quantized

sentence-transformers/all-MiniLM-L6-v2

Finetuned

(994)

this model

Datasets used to train AutoCyberAI/crp-intent-setfit

Evaluation results

Held-out accuracy (2,000 examples) on CRP speech-act held-out mix
self-reported

0.934