ConstantinosVidiniotis commited on
Commit
bd69fcf
·
verified ·
1 Parent(s): 0d5d600

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +39 -89
README.md CHANGED
@@ -1,89 +1,39 @@
1
- ---
2
- license: other
3
- pipeline_tag: text-classification
4
- library_name: setfit
5
- base_model: sentence-transformers/all-MiniLM-L6-v2
6
- tags:
7
- - setfit
8
- - sentence-transformers
9
- - text-classification
10
- - crp
11
- - context-relay-protocol
12
- - intent-classification
13
- - speech-acts
14
- datasets:
15
- - banking77
16
- - snips_built_in_intents
17
- metrics:
18
- - accuracy
19
- model-index:
20
- - name: crp-intent-setfit
21
- results:
22
- - task:
23
- type: text-classification
24
- name: Speech-act classification (4-class)
25
- dataset:
26
- type: banking77
27
- name: CRP speech-act held-out mix
28
- metrics:
29
- - type: accuracy
30
- value: 0.9340
31
- name: Held-out accuracy (2,000 examples)
32
- widget:
33
- - text: "Please scan the repository for compliance issues."
34
- - text: "What is the current deployment status?"
35
- - text: "I believe the server is down."
36
- - text: "This is frustrating and slow."
37
- inference: true
38
- ---
39
-
40
- # CRP Intent SetFit — speech-act classifier
41
-
42
- Part of the [Context Relay Protocol (CRP)](https://crprotocol.io) ML-first
43
- governance layer. Classifies a user turn into one of four CRP speech acts —
44
- **request, question, assertion, expressive** — used by `crp/isa/intent.py`
45
- (SPEC-051 Intent & Speech Acts) to drive routing and operation framing in
46
- agentic pipelines.
47
-
48
- Few-shot SetFit model: contrastive-tuned `all-MiniLM-L6-v2` body +
49
- LogisticRegression head fit on the full training mix.
50
-
51
- ## Verified results (independent harness, 2026-07-28)
52
-
53
- | Metric | Value |
54
- |---|---|
55
- | Held-out accuracy (2,000 unseen examples) | **0.9340** |
56
- | F1 assertion / expressive / question / request | 0.980 / 1.000 / 0.880 / 0.933 |
57
- | Production-style CRP prompts | **18/20** |
58
-
59
- ## Training data
60
-
61
- Banking77 + SNIPS (intent names heuristically mapped to the four CRP speech
62
- acts) + templated synthetic examples per class. 64-shot contrastive body
63
- tuning, then the classifier head refit on the full mix.
64
-
65
- ## Usage
66
-
67
- ```python
68
- from setfit import SetFitModel
69
-
70
- model = SetFitModel.from_pretrained("AutoCyberAI/crp-intent-setfit")
71
- model.predict(["Please scan the repository for compliance issues."])
72
- # -> ['request']
73
- ```
74
-
75
- In the CRP SDK this model is the default intent backend:
76
-
77
- ```bash
78
- CRP_INTENT_MODEL=AutoCyberAI/crp-intent-setfit # default; no env needed
79
- ```
80
-
81
- ## Limitations
82
-
83
- Four speech acts only; trained on English service/assistant phrasing. The
84
- held-out score is in-distribution (same data mix); the production-prompt
85
- score above is the out-of-distribution signal.
86
-
87
- ## License
88
-
89
- Elastic License 2.0 — see the CRP repository for details.
 
1
+ # CRP Intent / Speech-Act Classifier
2
+
3
+ A SetFit sentence-transformer classifier that maps a user turn into one of four CRP speech acts: `request`, `question`, `assertion`, or `expressive`. Trained on Banking77, SNIPS, and synthetic CRP-style templates. Used by `crp.isa.intent` to decide how a turn should be routed and framed in the positioned agent loop.
4
+
5
+ ## Model description
6
+
7
+ - **Architecture:** SetFit on `sentence-transformers/all-MiniLM-L6-v2` (22M params).
8
+ - **Labels:** `request`, `question`, `assertion`, `expressive`.
9
+ - **Held-out accuracy:** 0.934 (2,000-example held-out slice from the training mix).
10
+ - **Production prompt score:** 18/20 correctly classified.
11
+ - **Inference budget:** ~10 ms on CPU; governed by `crp.ml.registry.ModelManager`.
12
+
13
+ ## Intended use
14
+
15
+ ```python
16
+ from setfit import SetFitModel
17
+ model = SetFitModel.from_pretrained('AutoCyberAI/crp-intent-setfit')
18
+ print(model.predict(['Please scan the repository for compliance issues.'])) # ['request']
19
+ ```
20
+
21
+ ## Limitations
22
+
23
+ - The model is trained on English banking/intent datasets plus synthetic CRP templates; performance may degrade on code-heavy or non-English inputs.
24
+ - It is an advisory classifier — the rule-based fallback in `crp.isa.intent` remains the degraded path if the model is unavailable or the latency budget is exceeded.
25
+
26
+ ## Citation
27
+
28
+ ```bibtex
29
+ @misc{crp-intent-setfit,
30
+ title={{CRP Intent / Speech-Act Classifier}},
31
+ author={{AutoCyber AI}},
32
+ year={2026},
33
+ howpublished={\url{https://huggingface.co/AutoCyberAI/crp-intent-setfit}}
34
+ }
35
+ ```
36
+
37
+ ---
38
+
39
+ *This model is part of the Context Relay Protocol (CRP) v6 Phase A managed-model suite. Learn more at https://crprotocol.io.*