Text Classification
Transformers
Safetensors
English
distilbert
consumer-complaints
finance
text-embeddings-inference
Instructions to use mai-data-ai/complaint-distilbert with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mai-data-ai/complaint-distilbert with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="mai-data-ai/complaint-distilbert")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("mai-data-ai/complaint-distilbert") model = AutoModelForSequenceClassification.from_pretrained("mai-data-ai/complaint-distilbert", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Consumer Complaint Classifier (DistilBERT)
distilbert-base-uncased fine-tuned to route a customer's written complaint to one of 5 product categories:
credit_card · credit_reporting · debt_collection · mortgages_and_loans · retail_banking
- Live demo: mai-data-ai/complaint-classifier
- Code and notebook: GitHub
Results (held-out test set, 12,430 complaints)
| Metric | Score |
|---|---|
| Accuracy | 0.8612 |
| Macro precision | 0.8499 |
| Macro recall | 0.8486 |
| Macro F1 | 0.8491 |
Per-class results:
| Class | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| credit_card | 0.8302 | 0.7902 | 0.8097 | 1,497 |
| credit_reporting | 0.8917 | 0.8967 | 0.8942 | 5,617 |
| debt_collection | 0.7936 | 0.7871 | 0.7903 | 2,100 |
| mortgages_and_loans | 0.8555 | 0.8615 | 0.8585 | 1,870 |
| retail_banking | 0.8784 | 0.9071 | 0.8925 | 1,346 |
Comparison with models trained from scratch (same split)
| Model | Accuracy | Macro F1 |
|---|---|---|
| SimpleRNN | 0.7973 | 0.7747 |
| LSTM | 0.8191 | 0.8063 |
| GRU | 0.8250 | 0.8184 |
| DistilBERT (this model) | 0.8612 | 0.8491 |
The SimpleRNN, LSTM and GRU baselines are Keras models (128-d embedding, 128 units, ~3.9M parameters) trained on the same train/validation/test split. Scores for the from-scratch models vary by about ±0.5 points between training runs; the ranking (DistilBERT > GRU ≈ LSTM > SimpleRNN) was the same in every run.
Training
- Data: Kaggle Consumer Complaints Dataset for NLP. 162,421 rows reduced to 124,300 complaints after removing empty narratives, exact duplicates and texts with conflicting labels.
- Split: stratified 80/10/10 (99,440 train / 12,430 validation / 12,430 test), seed 42.
- Class imbalance:
credit_reportingis ~45% of the data, so training used class-weighted cross-entropy (weights computed from the training set only). - Hyperparameters: learning rate 2e-5, weight decay 0.01, warmup 900 steps, batch size 32, 3 epochs, fp16, max length 256. The best checkpoint was selected by validation macro-F1.
Usage
The model expects text cleaned the same way as the training data: lowercase, no punctuation or numbers, no stop words, lemmatized.
import re, nltk
from nltk.corpus import stopwords
from nltk.stem import WordNetLemmatizer
from transformers import pipeline
nltk.download("stopwords"); nltk.download("wordnet")
STOP = set(stopwords.words("english"))
lemm = WordNetLemmatizer()
def clean_text(text):
text = re.sub(r"[^a-z\s]", " ", str(text).lower())
return " ".join(lemm.lemmatize(w) for w in text.split() if w not in STOP)
clf = pipeline("text-classification",
model="mai-data-ai/complaint-distilbert", top_k=None)
complaint = "A debt collector keeps calling me about a loan I already paid off."
print(clf(clean_text(complaint), truncation=True, max_length=256))
Limitations
- Categories overlap. Most of the 1,725 test errors involve
credit_reportinganddebt_collection(credit_reporting→debt_collection: 325,debt_collection→credit_reporting: 295). - Trained on English complaint text from a single source (US consumer complaints), so performance on other domains or languages is not guaranteed.
- Only 5 coarse product categories; it does not predict issue type or sub-product.
- Intended for assisting complaint routing, not for fully automated decisions without human review.
- Downloads last month
- 97
Model tree for mai-data-ai/complaint-distilbert
Base model
distilbert/distilbert-base-uncased