Update README.md

a71f2d2 verified 10 days ago

21.2 kB

	---
	tags:
	- setfit
	- sentence-transformers
	- text-classification
	- generated_from_setfit_trainer
	widget:
	- text: What do I do? My card is broken.
	- text: Where should I withdraw money from?
	- text: >-
	I can't seem to make a standard bank transfer. I have tried at least five
	times already but none of them are going through. Please tell me what is
	wrong?
	- text: What are the currency exchange fees?
	- text: I have an unauthorized charge.
	metrics:
	- accuracy
	pipeline_tag: text-classification
	library_name: setfit
	inference: true
	base_model: BAAI/bge-small-en-v1.5
	model_name: TinyModels/setfit-banking-intent
	license: apache-2.0
	language:
	- en
	---

	<div align="center">

	<img src="https://capsule-render.vercel.app/api?type=waving&color=gradient&customColorList=6,11,20,24,30&height=220&section=header&text=SetFit%20Banking%20Intent&fontSize=54&fontColor=ffffff&animation=fadeIn&fontAlignY=32&desc=77-Class%20Intent%20Classification%20%E2%80%A2%20Few-Shot%20Learning%20%E2%80%A2%20SetFit&descAlignY=55&descSize=18&descColor=e0d7ff" width="100%"/>

	<a href="https://huggingface.co/TinyModels/setfit-banking-intent">
	<img src="https://readme-typing-svg.demolab.com?font=Fira+Code&weight=700&size=24&duration=2800&pause=900&color=A78BFA&center=true&vCenter=true&multiline=true&width=800&height=90&lines=Classify+banking+intents+in+one+line+of+code.;77+classes+%7C+4+samples+per+class+%7C+SetFit+few-shot.;Powered+by+BAAI%2Fbge-small-en-v1.5.;Built+with+%E2%9D%A4%EF%B8%8F+by+TinyModels." alt="Typing SVG" />
	</a>

	<br/>


	![Task](https://img.shields.io/badge/Task-Text%20Classification-6C63FF?style=for-the-badge&logo=huggingface&logoColor=white)
	![Library](https://img.shields.io/badge/Library-SetFit%201.2.0-FF6F91?style=for-the-badge&logo=python&logoColor=white)
	![Backbone](https://img.shields.io/badge/Backbone-bge--small--en--v1.5-00C2A8?style=for-the-badge&logo=ai&logoColor=white)
	![Classes](https://img.shields.io/badge/Classes-77-F7B32B?style=for-the-badge&logo=target&logoColor=white)
	![F1](https://img.shields.io/badge/Metric-Accuracy-34D399?style=for-the-badge&logo=checkmarx&logoColor=white)

	![Python](https://img.shields.io/badge/Python-3.13-3776AB?style=flat-square&logo=python&logoColor=white)
	![PyTorch](https://img.shields.io/badge/PyTorch-2.11-EE4C2C?style=flat-square&logo=pytorch&logoColor=white)
	![Transformers](https://img.shields.io/badge/Transformers-5.16-FFD21E?style=flat-square&logo=huggingface&logoColor=black)
	![SentenceTransformers](https://img.shields.io/badge/SentenceTransformers-5.7-8A2BE2?style=flat-square&logo=scikitlearn&logoColor=white)
	![scikit-learn](https://img.shields.io/badge/scikit--learn-LogReg-F7931E?style=flat-square&logo=scikitlearn&logoColor=white)

	![Status](https://img.shields.io/badge/Status-Active-brightgreen?style=flat-square)
	![Maintained](https://img.shields.io/badge/Maintained-Yes-success?style=flat-square)
	![PRs](https://img.shields.io/badge/PRs-Welcome-blueviolet?style=flat-square)
	![License](https://img.shields.io/badge/License-See%20Hub-lightgrey?style=flat-square)

	<br/>


	<img src="https://media.giphy.com/media/3o7aCTfyhYawdOXcFW/giphy.gif" width="140" alt="animated spark"/>

	<br/><br/>

	> ### 🏦 A tiny, fast, and accurate SetFit classifier that routes banking customer intents — trained with just 4 examples per class (308 sentences total).

	</div>

	<img src="https://user-images.githubusercontent.com/74038190/212284100-561aa473-3905-4a80-b561-0d28506553ee.gif" width="100%">

	---


	<div align="center">

	### 🧭 Table of Contents

	<a href="#-model-description"><kbd>📖 Description</kbd></a>  •
	<a href="#-architecture"><kbd>🏗️ Architecture</kbd></a>  •
	<a href="#-quick-start"><kbd>🚀 Quick Start</kbd></a>  •
	<a href="#-model-labels-77-classes"><kbd>🏷️ Labels</kbd></a>  •
	<a href="#-training-details"><kbd>🎓 Training</kbd></a>  •
	<a href="#-bias-risks--limitations"><kbd>⚠️ Risks</kbd></a>  •
	<a href="#-citation"><kbd>📜 Citation</kbd></a>

	</div>

	---

	## 📖 Model Description

	<div align="center">

	\| 🔍 \| 💡 \|
	\|:--:\|:--\|
	\| Model Type \| `SetFit` — Efficient Few-Shot Learning \|
	\| Author \| [TinyModels](https://huggingface.co/TinyModels) \|
	\| Sentence Transformer \| [`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-small-en-v1.5) \|
	\| Classification Head \| [`sklearn.linear_model.LogisticRegression`](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) \|
	\| Max Sequence Length \| `512` tokens \|
	\| Number of Classes \| `77` banking intents \|
	\| Training Samples \| `4` per class — `308` total \|
	\| Language \| English 🇬🇧 \|
	\| Domain \| 🏦 Banking / Fintech customer support \|

	</div>

	### 🌟 Why SetFit?

	> SetFit achieves strong accuracy without prompts and without large labeled datasets.
	> It first fine-tunes a sentence embedding model with contrastive learning, then trains a lightweight classifier on top — perfect for low-data, high-class-count problems like this one.

	---

	## 🏗️ Architecture

	```mermaid
	flowchart LR
	A["📝 Input Text<br/>(customer message)"] --> B["🔤 BAAI/bge-small-en-v1.5<br/>Sentence Transformer"]
	B --> C["🧬 Sentence Embedding<br/>(384-dim)"]
	C --> D["🎯 LogisticRegression<br/>Classification Head"]
	D --> E["🏷️ Predicted Intent<br/>(1 of 77 classes)"]

	subgraph TRAIN["🎓 Training Pipeline"]
	direction TB
	T1["Contrastive Fine-tuning<br/>(CosineSimilarityLoss)"] --> T2["Embedding Extraction"]
	T2 --> T3["LogReg Head Training"]
	end

	style A fill:#6C63FF,color:#fff,stroke:#4B4BFF,stroke-width:2px
	style B fill:#FF6F91,color:#fff,stroke:#E0567A,stroke-width:2px
	style C fill:#00C2A8,color:#fff,stroke:#009E88,stroke-width:2px
	style D fill:#F7B32B,color:#000,stroke:#D99A1F,stroke-width:2px
	style E fill:#34D399,color:#000,stroke:#25A67A,stroke-width:2px
	style TRAIN fill:#1a1a2e,color:#fff,stroke:#6C63FF,stroke-dasharray: 5 5
	```

	<img src="https://user-images.githubusercontent.com/74038190/212284100-561aa473-3905-4a80-b561-0d28506553ee.gif" width="100%">

	---

	## 🚀 Quick Start

	<details open>
	<summary><b>🐍 Python — one-liner inference</b></summary>

	```bash
	pip install setfit
	```

	```python
	from setfit import SetFitModel

	# 🔽 Load from the Hub
	model = SetFitModel.from_pretrained("TinyModels/setfit-banking-intent")

	# 🎯 Run inference
	preds = model("I have an unauthorized charge.")
	print(preds)
	# → ['direct_debit_payment_not_recognised']
	```

	</details>

	<details>
	<summary><b>📦 Batch inference with confidence scores</b></summary>

	```python
	from setfit import SetFitModel
	import numpy as np

	model = SetFitModel.from_pretrained("TinyModels/setfit-banking-intent")

	texts = [
	"What do I do? My card is broken.",
	"Where should I withdraw money from?",
	"What are the currency exchange fees?",
	"I have an unauthorized charge.",
	]

	# Predicted labels
	labels = model(texts)
	print(labels)

	# Probability distribution per class
	probs = model.predict_proba(texts)
	top_idx = np.argmax(probs, axis=1)
	top_conf = probs[np.arange(len(texts)), top_idx]

	for t, l, c in zip(texts, labels, top_conf):
	print(f"[{c:0.2%}] {l:<40} ← {t}")
	```

	</details>

	<details>
	<summary><b>⚡ FastAPI microservice snippet</b></summary>

	```python
	from fastapi import FastAPI
	from pydantic import BaseModel
	from setfit import SetFitModel

	app = FastAPI(title="Banking Intent API")
	model = SetFitModel.from_pretrained("TinyModels/setfit-banking-intent")

	class Query(BaseModel):
	text: str

	@app.post("/classify")
	def classify(q: Query):
	label = model([q.text])[0]
	return {"intent": label, "text": q.text}
	```

	</details>

	---

	## 🏷️ Model Labels (77 Classes)

	<div align="center">

	![Classes](https://img.shields.io/badge/Total%20Intents-77-6C63FF?style=for-the-badge)
	![Categories](https://img.shields.io/badge/Categories-10-FF6F91?style=for-the-badge)

	</div>

	<details>
	<summary>💳 <b>Card Management</b> — <i>20 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `activate_my_card` \| I have a new card and need to activate it \|
	\| `card_about_to_expire` \| Do I need to do something to get a new card once it expires? \|
	\| `card_acceptance` \| Where can I use my Mastercard? \|
	\| `card_arrival` \| Where is the card I ordered 2 weeks ago? \|
	\| `card_delivery_estimate` \| What is the delivery time for US? \|
	\| `card_linking` \| My new card isn't in my app, how do I get it in there? \|
	\| `card_not_working` \| What do I do? My card is broken. \|
	\| `card_swallowed` \| Please send a new card; the ATM ate mine. \|
	\| `contactless_not_working` \| Why is my contactless not working? \|
	\| `country_support` \| Will my new card work outside of the EU? \|
	\| `disposable_card_limits` \| Is there a limit to how many times I can use my disposable virtual card? \|
	\| `get_disposable_virtual_card` \| Where can I order a disposable virtual card? \|
	\| `get_physical_card` \| In the app, where do I find my card PIN? \|
	\| `getting_spare_card` \| Is it possible to get another card? \|
	\| `getting_virtual_card` \| How do I receive a virtual card? \|
	\| `lost_or_stolen_card` \| I've lost my card. What can I do about that? \|
	\| `order_physical_card` \| How do I ask for a physical card? \|
	\| `supported_cards_and_currencies` \| What currencies are approved to add money? \|
	\| `virtual_card_not_working` \| Why isn't my disposable virtual card working? \|
	\| `visa_or_mastercard` \| Which one are you? Visa or Mastercard? \|

	</details>

	<details>
	<summary>💸 <b>Payments</b> — <i>9 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `apple_pay_or_google_pay` \| How can I get my Google pay top up to work? \|
	\| `card_payment_fee_charged` \| Why am I getting charged more for using my card? \|
	\| `card_payment_not_recognised` \| I don't understand where this charge came from. \|
	\| `card_payment_wrong_exchange_rate` \| My exchange rate isn't correct. \|
	\| `declined_card_payment` \| You have declined my payment. \|
	\| `extra_charge_on_statement` \| Why is there a $1 charge on my statement? \|
	\| `pending_card_payment` \| A card payment on my account is shown as pending. \|
	\| `reverted_card_payment?` \| I did a payment but it was reverted by the app. \|
	\| `transaction_charged_twice` \| I have a duplicate charge. \|

	</details>

	<details>
	<summary>🔁 <b>Transfers</b> — <i>12 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `balance_not_updated_after_bank_transfer` \| I just transferred some money and do not see it updated yet. \|
	\| `beneficiary_not_allowed` \| Why can't I use my beneficiary? \|
	\| `cancel_transfer` \| Please cancel the transfer I just made. \|
	\| `declined_transfer` \| I got a message that my transfer was declined. \|
	\| `failed_transfer` \| I can't seem to make a standard bank transfer. \|
	\| `pending_transfer` \| I am still waiting for a transfer to show up. \|
	\| `receiving_money` \| How can my boss pay me directly to the card? \|
	\| `transfer_fee_charged` \| Why was I charged a fee for transferring money? \|
	\| `transfer_into_account` \| How can I transfer money to this account from another bank? \|
	\| `transfer_not_received_by_recipient` \| I transferred money and it didn't get there. \|
	\| `transfer_timing` \| How long until transfers from Europe go through? \|
	\| `top_up_by_bank_transfer_charge` \| If I top up by transfer, am I going to be charged? \|

	</details>

	<details>
	<summary>⬆️ <b>Top-ups</b> — <i>9 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `automatic_top_up` \| How can I setup automatic top-up? \|
	\| `pending_top_up` \| Why is the top-up I made still pending? \|
	\| `top_up_by_card_charge` \| Do you have any fees if I want to add money using an international card? \|
	\| `top_up_by_cash_or_cheque` \| Can I top up with check? \|
	\| `top_up_failed` \| I don't think that my top-up worked. \|
	\| `top_up_limits` \| What's the top-up limit? \|
	\| `top_up_reverted` \| Why did my top-up get reverted? \|
	\| `topping_up_by_card` \| I can't see my top up in my wallet! \|
	\| `verify_top_up` \| How are top-ups verified? \|

	</details>

	<details>
	<summary>🏧 <b>Cash & ATM</b> — <i>8 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `atm_support` \| Where should I withdraw money from? \|
	\| `balance_not_updated_after_cheque_or_cash_deposit` \| Why isn't my cash deposit showing up in my account? \|
	\| `cash_withdrawal_charge` \| I got charged fees for withdrawing cash! \|
	\| `cash_withdrawal_not_recognised` \| I see cash withdrawals that I did not authorize. \|
	\| `declined_cash_withdrawal` \| Is my card broken? I can't get cash out of the ATM. \|
	\| `pending_cash_withdrawal` \| My ATM withdrawal is taking forever. \|
	\| `wrong_amount_of_cash_received` \| I only got $20 of the $100 that I attempted to withdraw. \|
	\| `wrong_exchange_rate_for_cash_withdrawal` \| I received the incorrect exchange rate. \|

	</details>

	<details>
	<summary>🔐 <b>Security & PIN</b> — <i>5 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `change_pin` \| Can you tell me how to change my PIN? \|
	\| `compromised_card` \| The card has suffered a security breach. \|
	\| `lost_or_stolen_phone` \| My app was on the phone and I was mugged. \|
	\| `passcode_forgotten` \| I thought I knew my password but I guess I was wrong. \|
	\| `pin_blocked` \| Help me unblock my account. I entered the PIN wrong too many times. \|

	</details>

	<details>
	<summary>🪪 <b>Identity Verification</b> — <i>4 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `unable_to_verify_identity` \| The app is not able to realize that it is me. \|
	\| `verify_my_identity` \| Can I get information on the identity checks? \|
	\| `verify_source_of_funds` \| I'd like to know where my funds come from. \|
	\| `why_verify_identity` \| What is the function of the identity check? \|

	</details>

	<details>
	<summary>👤 <b>Account Management</b> — <i>3 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `age_limit` \| At what age can a person open an account? \|
	\| `edit_personal_details` \| I just got married and need to change my name on the account. \|
	\| `terminate_account` \| How can I delete my account? \|

	</details>

	<details>
	<summary>💱 <b>Fees & Currency</b> — <i>4 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `exchange_charge` \| What are the currency exchange fees? \|
	\| `exchange_rate` \| How are the exchange rates determined? \|
	\| `exchange_via_app` \| Can I exchange USD and GBP from the app? \|
	\| `fiat_currency_support` \| How many currencies can I have? \|

	</details>

	<details>
	<summary>🧾 <b>Refunds & Disputes</b> — <i>3 classes</i></summary>

	\| Label \| Example Utterance \|
	\|:------\|:------------------\|
	\| `direct_debit_payment_not_recognised` \| I have an unauthorized charge. \|
	\| `Refund_not_showing_up` \| Why can't I see my refund in my statement? \|
	\| `request_refund` \| I need a refund for something I bought. \|

	</details>

	<img src="https://user-images.githubusercontent.com/74038190/212284100-561aa473-3905-4a80-b561-0d28506553ee.gif" width="100%">

	---

	## 🎓 Training Details

	<div align="center">

	### ⚙️ Hyperparameters

	\| Parameter \| Value \| \| Parameter \| Value \|
	\|:----------\|:-----:\|-\|:----------\|:-----:\|
	\| `batch_size` \| `(32, 32)` \| \| `num_epochs` \| `(1, 1)` \|
	\| `max_steps` \| `-1` \| \| `sampling_strategy` \| `oversampling` \|
	\| `num_iterations` \| `2` \| \| `body_learning_rate` \| `2e-05` \|
	\| `head_learning_rate` \| `2e-05` \| \| `loss` \| `CosineSimilarityLoss` \|
	\| `distance_metric` \| `cosine_distance` \| \| `margin` \| `0.25` \|
	\| `end_to_end` \| `False` \| \| `use_amp` \| `False` \|
	\| `warmup_proportion` \| `0.1` \| \| `l2_weight` \| `0.01` \|
	\| `seed` \| `42` \| \| `load_best_model_at_end` \| `False` \|

	</div>

	### 📚 Training Set Statistics

	<div align="center">

	\| Metric \| Min \| Median \| Max \|
	\|:-------\|:---:\|:------:\|:---:\|
	\| 📝 Word count \| 4 \| 11.66 \| 78 \|
	\| 🏷️ Samples per class \| 4 \| 4 \| 4 \|
	\| 🗂️ Total samples \| — \| 308 \| — \|

	</div>

	### 📉 Training Curve

	\| Epoch \| Step \| Training Loss \| Validation Loss \|
	\|:-----:\|:----:\|:-------------:\|:---------------:\|
	\| `0.0256` \| `1` \| 0.2141 \| — \|

	### 🧪 Framework Versions

	<div align="center">

	![Python](https://img.shields.io/badge/Python-3.13.15-3776AB?style=for-the-badge&logo=python&logoColor=white)
	![SetFit](https://img.shields.io/badge/SetFit-1.2.0-FF6F91?style=for-the-badge)
	![SentenceTransformers](https://img.shields.io/badge/SentenceTransformers-5.7.0-8A2BE2?style=for-the-badge)
	![Transformers](https://img.shields.io/badge/Transformers-5.16.1-FFD21E?style=for-the-badge&logo=huggingface&logoColor=black)
	![PyTorch](https://img.shields.io/badge/PyTorch-2.11.0%2Bcu128-EE4C2C?style=for-the-badge&logo=pytorch&logoColor=white)
	![Datasets](https://img.shields.io/badge/Datasets-5.0.1-34D399?style=for-the-badge)
	![Tokenizers](https://img.shields.io/badge/Tokenizers-0.23.1-F7B32B?style=for-the-badge)

	</div>

	---

	## ⚠️ Bias, Risks & Limitations

	<details>
	<summary><b>🔍 Click to expand</b></summary>

	\| ⚠️ Area \| 📝 Notes \|
	\|:--------\|:---------\|
	\| Data size \| Only 4 examples per class — rare phrasings may be misclassified. \|
	\| Language \| Trained only on English — will not generalize to other languages without re-training. \|
	\| Domain shift \| Vocabulary is banking-specific; out-of-domain inputs may produce confident but wrong labels. \|
	\| Class ambiguity \| Some classes are semantically close (e.g. `pending_transfer` vs. `transfer_not_received_by_recipient`). \|
	\| No calibration \| LogReg probabilities are not temperature-calibrated — treat confidence as approximate. \|
	\| PII \| Do not feed real customer PII into shared demos; use anonymized text. \|

	</details>

	### 🚫 Out-of-Scope Use

	- ❌ Legal, medical, or financial advice generation
	- ❌ Production decision-making without human-in-the-loop
	- ❌ Non-English customer messages
	- ❌ Emotion / sentiment / toxicity detection

	---

	## 🧩 Model Card Recipe

	```yaml
	🎯 base_model: BAAI/bge-small-en-v1.5
	🧠 head: LogisticRegression
	📚 technique: SetFit (few-shot contrastive)
	🏷️ classes: 77
	📈 metric: accuracy
	🔤 max_tokens: 512
	🌱 seed: 42
	```

	---

	## 🗺️ Roadmap

	- [x] Train on 77 banking intents
	- [x] Publish to Hugging Face Hub
	- [ ] Add calibrated confidence scores
	- [ ] Multilingual variant (XLM-R backbone)
	- [ ] ONNX / quantized export for edge
	- [ ] Evaluation on public banking benchmark

	---

	## 📜 Citation

	If you use this model, please cite the SetFit paper:

	```bibtex
	@article{tunstall2022setfit,
	doi = {10.48550/ARXIV.2209.11055},
	url = {https://arxiv.org/abs/2209.11055},
	author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
	keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences},
	title = {Efficient Few-Shot Learning Without Prompts},
	publisher = {arXiv},
	year = {2022},
	copyright = {Creative Commons Attribution 4.0 International}
	}
	```

	---

	## 📚 Glossary

	\| Term \| Meaning \|
	\|:-----\|:--------\|
	\| SetFit \| Sentence Transformer Fine-tuning — few-shot text classification technique. \|
	\| Contrastive Learning \| Training method that pulls similar pairs together, pushes dissimilar apart. \|
	\| Sentence Transformer \| Encoder that maps text → dense vector embedding. \|
	\| LogisticRegression Head \| Simple linear classifier on top of embeddings. \|
	\| Intent \| The user's goal behind a message (e.g. `cancel_transfer`). \|

	---

	## 🌟 Star History

	<div align="center">

	<a href="https://star-history.com/#TinyModels/setfit-banking-intent&Date">
	<img src="https://api.star-history.com/svg?repos=TinyModels/setfit-banking-intent&type=Date" width="600" alt="Star History Chart"/>
	</a>

	</div>

	---


	<div align="center">

	### 💜 Made with love by [TinyModels](https://huggingface.co/TinyModels)

	<a href="https://huggingface.co/TinyModels/setfit-banking-intent">
	<img src="https://img.shields.io/badge/🤗%20Hugging%20Face-Model%20Card-FFD21E?style=for-the-badge&logoColor=black"/>
	</a>
	<a href="https://github.com/huggingface/setfit">
	<img src="https://img.shields.io/badge/GitHub-SetFit-181717?style=for-the-badge&logo=github"/>
	</a>
	<a href="https://arxiv.org/abs/2209.11055">
	<img src="https://img.shields.io/badge/arXiv-2209.11055-B31B1B?style=for-the-badge&logo=arxiv"/>
	</a>

	<br/><br/>

	<img src="https://capsule-render.vercel.app/api?type=waving&color=gradient&customColorList=6,11,20,24,30&height=140&section=footer&text=Thanks%20for%20stopping%20by!&fontSize=24&fontColor=ffffff&animation=twinkling&fontAlignY=60" width="100%"/>

	<sub>⭐ If this model helped you, consider leaving a like on the Hub/Repo!</sub>

	</div>