Text Classification
setfit
Safetensors
sentence-transformers
bert
crp
context-relay-protocol
intent-classification
speech-acts
Eval Results (legacy)
text-embeddings-inference
Instructions to use AutoCyberAI/crp-intent-setfit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- setfit
How to use AutoCyberAI/crp-intent-setfit with setfit:
from setfit import SetFitModel model = SetFitModel.from_pretrained("AutoCyberAI/crp-intent-setfit") preds = model.predict(["i loved the spiderman movie!", "pineapple on pizza is the worst"]) print(preds) - sentence-transformers
How to use AutoCyberAI/crp-intent-setfit with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("AutoCyberAI/crp-intent-setfit") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Push model using huggingface_hub.
Browse files- README.md +80 -41
- config_setfit.json +2 -2
- model.safetensors +1 -1
- model_head.pkl +1 -1
README.md
CHANGED
|
@@ -5,31 +5,17 @@ tags:
|
|
| 5 |
- text-classification
|
| 6 |
- generated_from_setfit_trainer
|
| 7 |
widget:
|
| 8 |
-
- text: Why is
|
| 9 |
-
- text:
|
| 10 |
-
- text:
|
| 11 |
-
- text:
|
| 12 |
-
- text:
|
| 13 |
metrics:
|
| 14 |
- accuracy
|
| 15 |
pipeline_tag: text-classification
|
| 16 |
library_name: setfit
|
| 17 |
inference: true
|
| 18 |
base_model: sentence-transformers/all-MiniLM-L6-v2
|
| 19 |
-
model-index:
|
| 20 |
-
- name: SetFit with sentence-transformers/all-MiniLM-L6-v2
|
| 21 |
-
results:
|
| 22 |
-
- task:
|
| 23 |
-
type: text-classification
|
| 24 |
-
name: Text Classification
|
| 25 |
-
dataset:
|
| 26 |
-
name: Unknown
|
| 27 |
-
type: unknown
|
| 28 |
-
split: test
|
| 29 |
-
metrics:
|
| 30 |
-
- type: accuracy
|
| 31 |
-
value: 0.695
|
| 32 |
-
name: Accuracy
|
| 33 |
---
|
| 34 |
|
| 35 |
# SetFit with sentence-transformers/all-MiniLM-L6-v2
|
|
@@ -60,19 +46,12 @@ The model has been trained using an efficient few-shot learning technique that i
|
|
| 60 |
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
|
| 61 |
|
| 62 |
### Model Labels
|
| 63 |
-
| Label | Examples
|
| 64 |
-
|:-----------|:---------------------------------------------------------------------------------------------------------------------------------------------
|
| 65 |
-
|
|
| 66 |
-
|
|
| 67 |
-
|
|
| 68 |
-
| expressive | <ul><li>'Great, the
|
| 69 |
-
|
| 70 |
-
## Evaluation
|
| 71 |
-
|
| 72 |
-
### Metrics
|
| 73 |
-
| Label | Accuracy |
|
| 74 |
-
|:--------|:---------|
|
| 75 |
-
| **all** | 0.695 |
|
| 76 |
|
| 77 |
## Uses
|
| 78 |
|
|
@@ -92,7 +71,7 @@ from setfit import SetFitModel
|
|
| 92 |
# Download from the 🤗 Hub
|
| 93 |
model = SetFitModel.from_pretrained("AutoCyberAI/crp-intent-setfit")
|
| 94 |
# Run inference
|
| 95 |
-
preds = model("
|
| 96 |
```
|
| 97 |
|
| 98 |
<!--
|
|
@@ -124,14 +103,14 @@ preds = model("Why is the policy slow?")
|
|
| 124 |
### Training Set Metrics
|
| 125 |
| Training set | Min | Median | Max |
|
| 126 |
|:-------------|:----|:-------|:----|
|
| 127 |
-
| Word count | 4 | 7.
|
| 128 |
|
| 129 |
| Label | Training Sample Count |
|
| 130 |
|:-----------|:----------------------|
|
| 131 |
-
| assertion |
|
| 132 |
-
| expressive |
|
| 133 |
-
| question |
|
| 134 |
-
| request |
|
| 135 |
|
| 136 |
### Training Hyperparameters
|
| 137 |
- batch_size: (16, 16)
|
|
@@ -149,13 +128,73 @@ preds = model("Why is the policy slow?")
|
|
| 149 |
- l2_weight: 0.01
|
| 150 |
- seed: 42
|
| 151 |
- eval_max_steps: -1
|
| 152 |
-
- load_best_model_at_end:
|
| 153 |
|
| 154 |
### Training Results
|
| 155 |
| Epoch | Step | Training Loss | Validation Loss |
|
| 156 |
|:------:|:----:|:-------------:|:---------------:|
|
| 157 |
-
| 0.
|
| 158 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 159 |
|
| 160 |
### Framework Versions
|
| 161 |
- Python: 3.11.6
|
|
|
|
| 5 |
- text-classification
|
| 6 |
- generated_from_setfit_trainer
|
| 7 |
widget:
|
| 8 |
+
- text: Why is a transfer I did recently still processing?
|
| 9 |
+
- text: Can you find the port?
|
| 10 |
+
- text: It seems that the balance find.
|
| 11 |
+
- text: the compliance posture appears to be updated.
|
| 12 |
+
- text: I believe the report is correct.
|
| 13 |
metrics:
|
| 14 |
- accuracy
|
| 15 |
pipeline_tag: text-classification
|
| 16 |
library_name: setfit
|
| 17 |
inference: true
|
| 18 |
base_model: sentence-transformers/all-MiniLM-L6-v2
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
---
|
| 20 |
|
| 21 |
# SetFit with sentence-transformers/all-MiniLM-L6-v2
|
|
|
|
| 46 |
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
|
| 47 |
|
| 48 |
### Model Labels
|
| 49 |
+
| Label | Examples |
|
| 50 |
+
|:-----------|:---------------------------------------------------------------------------------------------------------------------------------------------|
|
| 51 |
+
| question | <ul><li>'Cash withdrawal was incorrect at ATM.'</li><li>'What is the server?'</li><li>'What is the invoice?'</li></ul> |
|
| 52 |
+
| request | <ul><li>'I need you to show the policy.'</li><li>'Can you summarise the compliance posture?'</li><li>'Can you find the port?'</li></ul> |
|
| 53 |
+
| assertion | <ul><li>'It seems that the report schedule.'</li><li>'I believe the balance is secure.'</li><li>'This is the compliance posture.'</li></ul> |
|
| 54 |
+
| expressive | <ul><li>'Great, the host is working!'</li><li>'I am frustrated with the port.'</li><li>'Amazing, the compliance posture is fixed!'</li></ul> |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
## Uses
|
| 57 |
|
|
|
|
| 71 |
# Download from the 🤗 Hub
|
| 72 |
model = SetFitModel.from_pretrained("AutoCyberAI/crp-intent-setfit")
|
| 73 |
# Run inference
|
| 74 |
+
preds = model("Can you find the port?")
|
| 75 |
```
|
| 76 |
|
| 77 |
<!--
|
|
|
|
| 103 |
### Training Set Metrics
|
| 104 |
| Training set | Min | Median | Max |
|
| 105 |
|:-------------|:----|:-------|:----|
|
| 106 |
+
| Word count | 4 | 7.4883 | 53 |
|
| 107 |
|
| 108 |
| Label | Training Sample Count |
|
| 109 |
|:-----------|:----------------------|
|
| 110 |
+
| assertion | 64 |
|
| 111 |
+
| expressive | 64 |
|
| 112 |
+
| question | 64 |
|
| 113 |
+
| request | 64 |
|
| 114 |
|
| 115 |
### Training Hyperparameters
|
| 116 |
- batch_size: (16, 16)
|
|
|
|
| 128 |
- l2_weight: 0.01
|
| 129 |
- seed: 42
|
| 130 |
- eval_max_steps: -1
|
| 131 |
+
- load_best_model_at_end: False
|
| 132 |
|
| 133 |
### Training Results
|
| 134 |
| Epoch | Step | Training Loss | Validation Loss |
|
| 135 |
|:------:|:----:|:-------------:|:---------------:|
|
| 136 |
+
| 0.0003 | 1 | 0.4095 | - |
|
| 137 |
+
| 0.0163 | 50 | 0.3272 | - |
|
| 138 |
+
| 0.0326 | 100 | 0.2618 | - |
|
| 139 |
+
| 0.0488 | 150 | 0.1939 | - |
|
| 140 |
+
| 0.0651 | 200 | 0.1145 | - |
|
| 141 |
+
| 0.0814 | 250 | 0.0689 | - |
|
| 142 |
+
| 0.0977 | 300 | 0.0714 | - |
|
| 143 |
+
| 0.1139 | 350 | 0.0542 | - |
|
| 144 |
+
| 0.1302 | 400 | 0.0306 | - |
|
| 145 |
+
| 0.1465 | 450 | 0.0145 | - |
|
| 146 |
+
| 0.1628 | 500 | 0.0044 | - |
|
| 147 |
+
| 0.1790 | 550 | 0.0026 | - |
|
| 148 |
+
| 0.1953 | 600 | 0.0022 | - |
|
| 149 |
+
| 0.2116 | 650 | 0.0015 | - |
|
| 150 |
+
| 0.2279 | 700 | 0.0013 | - |
|
| 151 |
+
| 0.2441 | 750 | 0.0012 | - |
|
| 152 |
+
| 0.2604 | 800 | 0.001 | - |
|
| 153 |
+
| 0.2767 | 850 | 0.0008 | - |
|
| 154 |
+
| 0.2930 | 900 | 0.0009 | - |
|
| 155 |
+
| 0.3092 | 950 | 0.0008 | - |
|
| 156 |
+
| 0.3255 | 1000 | 0.0006 | - |
|
| 157 |
+
| 0.3418 | 1050 | 0.0006 | - |
|
| 158 |
+
| 0.3581 | 1100 | 0.0006 | - |
|
| 159 |
+
| 0.3743 | 1150 | 0.0006 | - |
|
| 160 |
+
| 0.3906 | 1200 | 0.0005 | - |
|
| 161 |
+
| 0.4069 | 1250 | 0.0005 | - |
|
| 162 |
+
| 0.4232 | 1300 | 0.0005 | - |
|
| 163 |
+
| 0.4395 | 1350 | 0.0005 | - |
|
| 164 |
+
| 0.4557 | 1400 | 0.0004 | - |
|
| 165 |
+
| 0.4720 | 1450 | 0.0004 | - |
|
| 166 |
+
| 0.4883 | 1500 | 0.0004 | - |
|
| 167 |
+
| 0.5046 | 1550 | 0.0004 | - |
|
| 168 |
+
| 0.5208 | 1600 | 0.0004 | - |
|
| 169 |
+
| 0.5371 | 1650 | 0.0004 | - |
|
| 170 |
+
| 0.5534 | 1700 | 0.0004 | - |
|
| 171 |
+
| 0.5697 | 1750 | 0.0003 | - |
|
| 172 |
+
| 0.5859 | 1800 | 0.0003 | - |
|
| 173 |
+
| 0.6022 | 1850 | 0.0003 | - |
|
| 174 |
+
| 0.6185 | 1900 | 0.0003 | - |
|
| 175 |
+
| 0.6348 | 1950 | 0.0003 | - |
|
| 176 |
+
| 0.6510 | 2000 | 0.0003 | - |
|
| 177 |
+
| 0.6673 | 2050 | 0.0003 | - |
|
| 178 |
+
| 0.6836 | 2100 | 0.0003 | - |
|
| 179 |
+
| 0.6999 | 2150 | 0.0003 | - |
|
| 180 |
+
| 0.7161 | 2200 | 0.0003 | - |
|
| 181 |
+
| 0.7324 | 2250 | 0.0003 | - |
|
| 182 |
+
| 0.7487 | 2300 | 0.0003 | - |
|
| 183 |
+
| 0.7650 | 2350 | 0.0003 | - |
|
| 184 |
+
| 0.7812 | 2400 | 0.0003 | - |
|
| 185 |
+
| 0.7975 | 2450 | 0.0003 | - |
|
| 186 |
+
| 0.8138 | 2500 | 0.0002 | - |
|
| 187 |
+
| 0.8301 | 2550 | 0.0002 | - |
|
| 188 |
+
| 0.8464 | 2600 | 0.0003 | - |
|
| 189 |
+
| 0.8626 | 2650 | 0.0002 | - |
|
| 190 |
+
| 0.8789 | 2700 | 0.0002 | - |
|
| 191 |
+
| 0.8952 | 2750 | 0.0002 | - |
|
| 192 |
+
| 0.9115 | 2800 | 0.0003 | - |
|
| 193 |
+
| 0.9277 | 2850 | 0.0002 | - |
|
| 194 |
+
| 0.9440 | 2900 | 0.0002 | - |
|
| 195 |
+
| 0.9603 | 2950 | 0.0002 | - |
|
| 196 |
+
| 0.9766 | 3000 | 0.0002 | - |
|
| 197 |
+
| 0.9928 | 3050 | 0.0002 | - |
|
| 198 |
|
| 199 |
### Framework Versions
|
| 200 |
- Python: 3.11.6
|
config_setfit.json
CHANGED
|
@@ -1,9 +1,9 @@
|
|
| 1 |
{
|
| 2 |
-
"normalize_embeddings": false,
|
| 3 |
"labels": [
|
| 4 |
"assertion",
|
| 5 |
"expressive",
|
| 6 |
"question",
|
| 7 |
"request"
|
| 8 |
-
]
|
|
|
|
| 9 |
}
|
|
|
|
| 1 |
{
|
|
|
|
| 2 |
"labels": [
|
| 3 |
"assertion",
|
| 4 |
"expressive",
|
| 5 |
"question",
|
| 6 |
"request"
|
| 7 |
+
],
|
| 8 |
+
"normalize_embeddings": false
|
| 9 |
}
|
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 90864192
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:39a4e71b3ba068c43a85886c12db3b6694290671f278529ee2b9314d6e1fddcd
|
| 3 |
size 90864192
|
model_head.pkl
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 7159
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1f801a60e79115c275b21fc1d116c853079e09f3ba8405ccd74e5b3ad4ddce93
|
| 3 |
size 7159
|