| # DistilBERT-Based Intent Detection Model for Banking Customer Queries |
|
|
| This repository contains a fine-tuned **DistilBERT** model for **intent detection** in banking customer support scenarios. It is trained on the **BANKING77 dataset** and designed to accurately classify user queries into 77 distinct banking-related intents. |
|
|
| ## Model Details |
|
|
| - **Model Architecture:** DistilBERT Base Uncased |
| - **Task:** Intent Detection for Banking Queries |
| - **Dataset:** [BANKING77](https://huggingface.co/datasets/banking77) |
| - **Fine-tuning Framework:** Hugging Face Transformers |
| - **Language:** English |
| - **Number of Labels:** 77 |
| - **Quantization:** *Not applied (full precision)* |
|
|
| ## Usage |
|
|
| ### Installation |
|
|
| ```bash |
| pip install transformers torch datasets |
| ``` |
|
|
| ### Loading the Model |
|
|
| ```python |
| from transformers import DistilBertTokenizer, DistilBertForSequenceClassification |
| import torch |
| |
| # Load fine-tuned model |
| model_path = "./banking77-distilbert" # Adjust path if different |
| model = DistilBertForSequenceClassification.from_pretrained(model_path) |
| tokenizer = DistilBertTokenizer.from_pretrained(model_path) |
| model.eval() |
| |
| # Sample query |
| text = "I need to reset my online banking password." |
| |
| # Tokenize and predict |
| inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True) |
| with torch.no_grad(): |
| outputs = model(**inputs) |
| predicted_class = torch.argmax(outputs.logits, dim=1).item() |
| |
| # Example label map (replace with the full BANKING77 map) |
| label_map = {0: "activate_my_card", 1: "balance", 2: "card_arrival", ..., 76: "why_was_my_card_declined"} |
| print(f"Predicted Intent: {label_map[predicted_class]}") |
| ``` |
|
|
| ## Performance Metrics |
|
|
| - **Accuracy:** ~95% (on the BANKING77 test split) |
| - **Loss:** ~0.13 (after fine-tuning for 4 epochs) |
|
|
| ## Fine-Tuning Details |
|
|
| ### Dataset |
|
|
| - **Name:** BANKING77 |
| - **Size:** ~13,000 customer support queries |
| - **Intents:** 77 unique labeled banking intents |
|
|
| ### Training |
|
|
| - **Epochs:** 4 |
| - **Batch Size:** 16 |
| - **Learning Rate:** 2e-5 |
| - **Optimizer:** AdamW |
| - **Evaluation Strategy:** per epoch |
| - **Loss Function:** CrossEntropyLoss |
|
|
| ### Hardware |
|
|
| - **GPU Used:** NVIDIA Tesla T4 (via Google Colab) |
| - **Training Time:** ~15 minutes |
|
|
| ## Repository Structure |
|
|
| ``` |
| . |
| ├── banking77-distilbert/ # Fine-tuned model directory (saved via trainer.save_model) |
| │ ├── config.json |
| │ ├── pytorch_model.bin |
| │ ├── tokenizer_config.json |
| │ ├── vocab.txt |
| ├── intent_predictor.py # Script for predicting intents (with preprocessing) |
| ├── README.md # Model documentation |
| ``` |
|
|
| ## Limitations |
|
|
| - The model is trained only on banking-related intents; it may misclassify out-of-domain queries. |
| - Multilingual support is not included — limited to English. |
| - Model does not handle multiple intents per query. |
|
|
| ## Contributing |
|
|
| Contributions and suggestions are welcome. Please open an issue or pull request for improvements or additional features. |
|
|
