Update README.md
Browse files
README.md
CHANGED
|
@@ -22,18 +22,52 @@ widget:
|
|
| 22 |
base_model:
|
| 23 |
- distilbert/distilbert-base-uncased
|
| 24 |
---
|
|
|
|
| 25 |
# ๐ฏ Customer Ticket Routing & Intent Classifier (DistilBERT)
|
|
|
|
| 26 |
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) for **intent classification and automated customer ticket routing** across **77 banking categories** on the [Banking77](https://huggingface.co/datasets/PolyAI/banking77) dataset.
|
|
|
|
| 27 |
It is designed for production microservices to automatically route incoming customer support requests to appropriate departments.
|
| 28 |
-
|
| 29 |
---
|
|
|
|
| 30 |
## ๐ Benchmark & Evaluation Results
|
| 31 |
-
|
|
|
|
|
|
|
| 32 |
| Metric | Score |
|
| 33 |
-
| :--- | :---
|
| 34 |
-
|
|
| 35 |
-
|
|
| 36 |
-
|
|
| 37 |
-
|
|
| 38 |
-
|
|
| 39 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
base_model:
|
| 23 |
- distilbert/distilbert-base-uncased
|
| 24 |
---
|
| 25 |
+
|
| 26 |
# ๐ฏ Customer Ticket Routing & Intent Classifier (DistilBERT)
|
| 27 |
+
|
| 28 |
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) for **intent classification and automated customer ticket routing** across **77 banking categories** on the [Banking77](https://huggingface.co/datasets/PolyAI/banking77) dataset.
|
| 29 |
+
|
| 30 |
It is designed for production microservices to automatically route incoming customer support requests to appropriate departments.
|
| 31 |
+
|
| 32 |
---
|
| 33 |
+
|
| 34 |
## ๐ Benchmark & Evaluation Results
|
| 35 |
+
|
| 36 |
+
Evaluated on the official Banking77 test set (3,080 samples):
|
| 37 |
+
|
| 38 |
| Metric | Score |
|
| 39 |
+
| :--- | :--- |
|
| 40 |
+
| Test Macro F1 | 0.9175 |
|
| 41 |
+
| Test Accuracy | 0.9175 |
|
| 42 |
+
| Test Precision (Macro) | 0.9209 |
|
| 43 |
+
| Test Recall (Macro) | 0.9175 |
|
| 44 |
+
| Test Loss | 0.3478 |
|
| 45 |
+
|
| 46 |
+
### Comparison with Baselines
|
| 47 |
+
|
| 48 |
+
| Approach | Trainable Params | Test Macro F1 | Test Accuracy | Test Loss |
|
| 49 |
+
| :--- | :--- | :--- | :--- | :--- |
|
| 50 |
+
| Baseline (TF-IDF + LogReg + Bayes) | - | 0.8901 | - | - |
|
| 51 |
+
| DistilBERT + LoRA (r=32) | ~2.6% (~1.8M) | 0.9132 | 0.9133 | 0.3189 |
|
| 52 |
+
| DistilBERT (Full Fine-Tuning) | 100% (66.9M) | 0.9175 | 0.9175 | 0.3478 |
|
| 53 |
+
|
| 54 |
+
---
|
| 55 |
+
|
| 56 |
+
## ๐ How to Use
|
| 57 |
+
|
| 58 |
+
### 1. Using transformers pipeline (Quickest)
|
| 59 |
+
|
| 60 |
+
```python
|
| 61 |
+
from transformers import pipeline
|
| 62 |
+
|
| 63 |
+
classifier = pipeline(
|
| 64 |
+
"text-classification",
|
| 65 |
+
model="nzgl21/ticket-routing-distilbert"
|
| 66 |
+
)
|
| 67 |
+
|
| 68 |
+
# Example ticket
|
| 69 |
+
ticket = "I noticed an unknown fee charged on my credit card statement."
|
| 70 |
+
prediction = classifier(ticket)
|
| 71 |
+
|
| 72 |
+
print(prediction)
|
| 73 |
+
# Output: [{'label': 'card_payment_fee_charged', 'score': 0.964}]
|