TinyModels/Setfit-banking-intent

🏦 A tiny, fast, and accurate SetFit classifier that routes banking customer intents — trained with just 4 examples per class (308 sentences total).

---

🧭 Table of Contents

📖 Description  •  🏗️ Architecture  •  🚀 Quick Start  •  🏷️ Labels  •  🎓 Training  •  ⚠️ Risks  •  📜 Citation

---

📖 Model Description

🔍	💡
Model Type	SetFit — Efficient Few-Shot Learning
Author	TinyModels
Sentence Transformer	BAAI/bge-small-en-v1.5
Classification Head	sklearn.linear_model.LogisticRegression
Max Sequence Length	512 tokens
Number of Classes	77 banking intents
Training Samples	4 per class — 308 total
Language	English 🇬🇧
Domain	🏦 Banking / Fintech customer support

🌟 Why SetFit?

SetFit achieves strong accuracy without prompts and without large labeled datasets.
It first fine-tunes a sentence embedding model with contrastive learning, then trains a lightweight classifier on top — perfect for low-data, high-class-count problems like this one.

---

🏗️ Architecture

flowchart LR
    A["📝 Input Text<br/>(customer message)"] --> B["🔤 BAAI/bge-small-en-v1.5<br/>Sentence Transformer"]
    B --> C["🧬 Sentence Embedding<br/>(384-dim)"]
    C --> D["🎯 LogisticRegression<br/>Classification Head"]
    D --> E["🏷️ Predicted Intent<br/>(1 of 77 classes)"]

    subgraph TRAIN["🎓 Training Pipeline"]
        direction TB
        T1["Contrastive Fine-tuning<br/>(CosineSimilarityLoss)"] --> T2["Embedding Extraction"]
        T2 --> T3["LogReg Head Training"]
    end

    style A fill:#6C63FF,color:#fff,stroke:#4B4BFF,stroke-width:2px
    style B fill:#FF6F91,color:#fff,stroke:#E0567A,stroke-width:2px
    style C fill:#00C2A8,color:#fff,stroke:#009E88,stroke-width:2px
    style D fill:#F7B32B,color:#000,stroke:#D99A1F,stroke-width:2px
    style E fill:#34D399,color:#000,stroke:#25A67A,stroke-width:2px
    style TRAIN fill:#1a1a2e,color:#fff,stroke:#6C63FF,stroke-dasharray: 5 5

---

🚀 Quick Start

🐍 Python — one-liner inference

pip install setfit

from setfit import SetFitModel

# 🔽 Load from the Hub
model = SetFitModel.from_pretrained("TinyModels/setfit-banking-intent")

# 🎯 Run inference
preds = model("I have an unauthorized charge.")
print(preds)
# → ['direct_debit_payment_not_recognised']

📦 Batch inference with confidence scores

from setfit import SetFitModel
import numpy as np

model = SetFitModel.from_pretrained("TinyModels/setfit-banking-intent")

texts = [
    "What do I do? My card is broken.",
    "Where should I withdraw money from?",
    "What are the currency exchange fees?",
    "I have an unauthorized charge.",
]

# Predicted labels
labels = model(texts)
print(labels)

# Probability distribution per class
probs = model.predict_proba(texts)
top_idx = np.argmax(probs, axis=1)
top_conf = probs[np.arange(len(texts)), top_idx]

for t, l, c in zip(texts, labels, top_conf):
    print(f"[{c:0.2%}] {l:<40} ← {t}")

⚡ FastAPI microservice snippet

from fastapi import FastAPI
from pydantic import BaseModel
from setfit import SetFitModel

app = FastAPI(title="Banking Intent API")
model = SetFitModel.from_pretrained("TinyModels/setfit-banking-intent")

class Query(BaseModel):
    text: str

@app.post("/classify")
def classify(q: Query):
    label = model([q.text])[0]
    return {"intent": label, "text": q.text}

---

🏷️ Model Labels (77 Classes)

💳 Card Management — 20 classes

Label	Example Utterance
activate_my_card	I have a new card and need to activate it
card_about_to_expire	Do I need to do something to get a new card once it expires?
card_acceptance	Where can I use my Mastercard?
card_arrival	Where is the card I ordered 2 weeks ago?
card_delivery_estimate	What is the delivery time for US?
card_linking	My new card isn't in my app, how do I get it in there?
card_not_working	What do I do? My card is broken.
card_swallowed	Please send a new card; the ATM ate mine.
contactless_not_working	Why is my contactless not working?
country_support	Will my new card work outside of the EU?
disposable_card_limits	Is there a limit to how many times I can use my disposable virtual card?
get_disposable_virtual_card	Where can I order a disposable virtual card?
get_physical_card	In the app, where do I find my card PIN?
getting_spare_card	Is it possible to get another card?
getting_virtual_card	How do I receive a virtual card?
lost_or_stolen_card	I've lost my card. What can I do about that?
order_physical_card	How do I ask for a physical card?
supported_cards_and_currencies	What currencies are approved to add money?
virtual_card_not_working	Why isn't my disposable virtual card working?
visa_or_mastercard	Which one are you? Visa or Mastercard?

💸 Payments — 9 classes

Label	Example Utterance
apple_pay_or_google_pay	How can I get my Google pay top up to work?
card_payment_fee_charged	Why am I getting charged more for using my card?
card_payment_not_recognised	I don't understand where this charge came from.
card_payment_wrong_exchange_rate	My exchange rate isn't correct.
declined_card_payment	You have declined my payment.
extra_charge_on_statement	Why is there a $1 charge on my statement?
pending_card_payment	A card payment on my account is shown as pending.
reverted_card_payment?	I did a payment but it was reverted by the app.
transaction_charged_twice	I have a duplicate charge.

🔁 Transfers — 12 classes

Label	Example Utterance
balance_not_updated_after_bank_transfer	I just transferred some money and do not see it updated yet.
beneficiary_not_allowed	Why can't I use my beneficiary?
cancel_transfer	Please cancel the transfer I just made.
declined_transfer	I got a message that my transfer was declined.
failed_transfer	I can't seem to make a standard bank transfer.
pending_transfer	I am still waiting for a transfer to show up.
receiving_money	How can my boss pay me directly to the card?
transfer_fee_charged	Why was I charged a fee for transferring money?
transfer_into_account	How can I transfer money to this account from another bank?
transfer_not_received_by_recipient	I transferred money and it didn't get there.
transfer_timing	How long until transfers from Europe go through?
top_up_by_bank_transfer_charge	If I top up by transfer, am I going to be charged?

⬆️ Top-ups — 9 classes

Label	Example Utterance
automatic_top_up	How can I setup automatic top-up?
pending_top_up	Why is the top-up I made still pending?
top_up_by_card_charge	Do you have any fees if I want to add money using an international card?
top_up_by_cash_or_cheque	Can I top up with check?
top_up_failed	I don't think that my top-up worked.
top_up_limits	What's the top-up limit?
top_up_reverted	Why did my top-up get reverted?
topping_up_by_card	I can't see my top up in my wallet!
verify_top_up	How are top-ups verified?

🏧 Cash & ATM — 8 classes

Label	Example Utterance
atm_support	Where should I withdraw money from?
balance_not_updated_after_cheque_or_cash_deposit	Why isn't my cash deposit showing up in my account?
cash_withdrawal_charge	I got charged fees for withdrawing cash!
cash_withdrawal_not_recognised	I see cash withdrawals that I did not authorize.
declined_cash_withdrawal	Is my card broken? I can't get cash out of the ATM.
pending_cash_withdrawal	My ATM withdrawal is taking forever.
wrong_amount_of_cash_received	I only got $20 of the $100 that I attempted to withdraw.
wrong_exchange_rate_for_cash_withdrawal	I received the incorrect exchange rate.

🔐 Security & PIN — 5 classes

Label	Example Utterance
change_pin	Can you tell me how to change my PIN?
compromised_card	The card has suffered a security breach.
lost_or_stolen_phone	My app was on the phone and I was mugged.
passcode_forgotten	I thought I knew my password but I guess I was wrong.
pin_blocked	Help me unblock my account. I entered the PIN wrong too many times.

🪪 Identity Verification — 4 classes

Label	Example Utterance
unable_to_verify_identity	The app is not able to realize that it is me.
verify_my_identity	Can I get information on the identity checks?
verify_source_of_funds	I'd like to know where my funds come from.
why_verify_identity	What is the function of the identity check?

👤 Account Management — 3 classes

Label	Example Utterance
age_limit	At what age can a person open an account?
edit_personal_details	I just got married and need to change my name on the account.
terminate_account	How can I delete my account?

💱 Fees & Currency — 4 classes

Label	Example Utterance
exchange_charge	What are the currency exchange fees?
exchange_rate	How are the exchange rates determined?
exchange_via_app	Can I exchange USD and GBP from the app?
fiat_currency_support	How many currencies can I have?

🧾 Refunds & Disputes — 3 classes

Label	Example Utterance
direct_debit_payment_not_recognised	I have an unauthorized charge.
Refund_not_showing_up	Why can't I see my refund in my statement?
request_refund	I need a refund for something I bought.

---

🎓 Training Details

⚙️ Hyperparameters

Parameter	Value		Parameter	Value
batch_size	(32, 32)		num_epochs	(1, 1)
max_steps	-1		sampling_strategy	oversampling
num_iterations	2		body_learning_rate	2e-05
head_learning_rate	2e-05		loss	CosineSimilarityLoss
distance_metric	cosine_distance		margin	0.25
end_to_end	False		use_amp	False
warmup_proportion	0.1		l2_weight	0.01
seed	42		load_best_model_at_end	False

📚 Training Set Statistics

Metric	Min	Median	Max
📝 Word count	4	11.66	78
🏷️ Samples per class	4	4	4
🗂️ Total samples	—	308	—

📉 Training Curve

Epoch	Step	Training Loss	Validation Loss
0.0256	1	0.2141	—

🧪 Framework Versions

---

⚠️ Bias, Risks & Limitations

🔍 Click to expand

⚠️ Area	📝 Notes
Data size	Only 4 examples per class — rare phrasings may be misclassified.
Language	Trained only on English — will not generalize to other languages without re-training.
Domain shift	Vocabulary is banking-specific; out-of-domain inputs may produce confident but wrong labels.
Class ambiguity	Some classes are semantically close (e.g. pending_transfer vs. transfer_not_received_by_recipient).
No calibration	LogReg probabilities are not temperature-calibrated — treat confidence as approximate.
PII	Do not feed real customer PII into shared demos; use anonymized text.

🚫 Out-of-Scope Use

• ❌ Legal, medical, or financial advice generation
• ❌ Production decision-making without human-in-the-loop
• ❌ Non-English customer messages
• ❌ Emotion / sentiment / toxicity detection

---

🧩 Model Card Recipe

🎯 base_model:    BAAI/bge-small-en-v1.5
🧠 head:          LogisticRegression
📚 technique:     SetFit (few-shot contrastive)
🏷️ classes:       77
📈 metric:        accuracy
🔤 max_tokens:    512
🌱 seed:          42

---

🗺️ Roadmap

• Train on 77 banking intents
• Publish to Hugging Face Hub
• Add calibrated confidence scores
• Multilingual variant (XLM-R backbone)
• ONNX / quantized export for edge
• Evaluation on public banking benchmark

---

📜 Citation

If you use this model, please cite the SetFit paper:

@article{tunstall2022setfit,
    doi       = {10.48550/ARXIV.2209.11055},
    url       = {https://arxiv.org/abs/2209.11055},
    author    = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords  = {Computation and Language (cs.CL), FOS: Computer and information sciences},
    title     = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year      = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

---

📚 Glossary

Term	Meaning
SetFit	Sentence Transformer Fine-tuning — few-shot text classification technique.
Contrastive Learning	Training method that pulls similar pairs together, pushes dissimilar apart.
Sentence Transformer	Encoder that maps text → dense vector embedding.
LogisticRegression Head	Simple linear classifier on top of embeddings.
Intent	The user's goal behind a message (e.g. cancel_transfer).

---

🌟 Star History

---

💜 Made with love by TinyModels

_{⭐ If this model helped you, consider leaving a like on the Hub/Repo!}