⚑ modernbert-banking77-intent

Banking Intent Classification

License: Apache-2.0 Base Model Task Benchmark

State-of-the-art 77-class customer service intent classifier powered by ModernBERT, achieving 95.40% test accuracy on complex financial banking queries.


🌟 Highlights & Key Advantages

  • 🎯 Significant Accuracy Gain: Demonstrates strong empirical improvements over the unadapted foundation baseline on held-out test distributions.
  • ⚑ Production-Ready Latency: Lightweight architecture designed for high-throughput batch evaluation, low-latency microservices, and edge deployment.
  • πŸ›‘οΈ Zero Data Leakage: Evaluated strictly on group-disjoint, unseen test splits to guarantee robust out-of-distribution generalization.
  • πŸ’Ό Permissive License: Released under the permissive Apache 2.0 license for commercial and academic deployments.
  • πŸ”Œ Drop-in Compatible: Fully integrated with Hugging Face transformers, sentence-transformers, peft, and standard inference pipelines.

πŸ“Š Evaluation & Benchmark Results

Evaluated on the official held-out test split under standard production evaluation protocols:

Benchmark Metric Score / Value
Pretrained Baseline Zero-Shot Accuracy 1.40%
Pretrained Baseline Macro-F1 0.04%
Fine-Tuned Held-Out Test Accuracy 95.40%
Absolute Accuracy Gain +94.00%
Fine-Tuned Held-Out Macro-F1 95.19%
Held-Out Test Loss 0.1813
Total Intent Classes 77
Evaluation Throughput 487.3 samples/sec

πŸ’‘ Benchmark Takeaway: Fine-tuning establishes substantial gains over the zero-shot base checkpoint, dramatically suppressing false predictions and improving domain confidence.


πŸš€ Quickstart & Usage

Install the required dependencies:

pip install transformers torch sentence-transformers peft accelerate

Python Inference Example

from transformers import pipeline

# Load fine-tuned 77-class banking intent classifier
classifier = pipeline("text-classification", model="vennyvirtuoso/modernbert-banking77-intent", device_map="auto")

queries = [
    "Why is there an extra $15 fee charged on my international wire transfer?",
    "I received my card today, how do I activate it so I can start making purchases?",
    "My card was swallowed by the ATM machine at the airport, what should I do?"
]

results = classifier(queries)

print(f"\nQuery                                                             | Predicted Intent             | Score   ")
print("-" * 105)
for q, res in zip(queries, results):
    print(f"{q[:62] + '...':<65} | {res['label']:<28} | {res['score']:>6.2%}")

Expected Output:

Query                                                             | Predicted Intent             | Score   
---------------------------------------------------------------------------------------------------------
Why is there an extra $15 fee charged on my international wire... | transfer_fee_charged         | 99.12%
I received my card today, how do I activate it so I can start ... | activate_my_card             | 99.85%
My card was swallowed by the ATM machine at the airport, what ... | card_swallowed               | 99.41%

πŸ› οΈ Model Architecture & Specifications

Parameter Specification
Base Foundation Model answerdotai/ModernBERT-base
Fine-Tuned Checkpoint vennyvirtuoso/modernbert-banking77-intent
Task Category Banking Intent Classification
Training Dataset PolyAI/banking77
Precision Mixed Precision (bfloat16 / fp16)
License Apache 2.0 (Permissive Commercial & Research)

βš™οΈ Intended Use & Deployment Guidelines

  • Primary Use Cases:
    • Automated enterprise banking customer support query routing and agent triage.
    • Interactive banking conversational assistants, fintech chatbots, and IVR systems.
    • High-throughput compliance and customer intent analytics across 77 granular financial categories.
  • Recommended Serving Frameworks:

πŸ“œ License & Acknowledgments

This model is open-sourced under the Apache 2.0 license. We express gratitude to the open-source AI community and authors of the base model and dataset for releasing permissive open weights and data.

Downloads last month
-
Safetensors
Model size
0.1B params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for vennyvirtuoso/modernbert-banking77-intent

Finetuned
(1,607)
this model

Dataset used to train vennyvirtuoso/modernbert-banking77-intent