google-research-datasets/go_emotions
Viewer • Updated • 265k • 27.7k • 284
How to use mai-data-ai/emotion-distilbert with Transformers:
# Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-classification", model="mai-data-ai/emotion-distilbert") # pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification
tokenizer = AutoTokenizer.from_pretrained("mai-data-ai/emotion-distilbert")
model = AutoModelForSequenceClassification.from_pretrained("mai-data-ai/emotion-distilbert", device_map="auto")A fine-tuned DistilBERT model that classifies English text into one of six emotions: joy, sadness, anger, fear, surprise, disgust.
It was trained on GoEmotions (Reddit comments), with the 27 fine-grained labels mapped to the 6 Ekman emotions.
from transformers import pipeline
clf = pipeline("text-classification", model="mai-data-ai/emotion-distilbert", top_k=None)
print(clf("I am so happy and grateful for everything you did for me!"))
Or with the model directly:
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification
name = "mai-data-ai/emotion-distilbert"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForSequenceClassification.from_pretrained(name).eval()
enc = tok("Stop wasting my time, this is so annoying", return_tensors="pt",
truncation=True, max_length=64)
with torch.no_grad():
probs = torch.softmax(model(**enc).logits, dim=-1)[0]
print({model.config.id2label[i]: round(p.item(), 3) for i, p in enumerate(probs)})
GoEmotions (simplified config) labels were mapped to six Ekman emotions:
| Ekman emotion | GoEmotions labels |
|---|---|
| anger | anger, annoyance, disapproval |
| disgust | disgust |
| fear | fear, nervousness |
| joy | joy, amusement, approval, excitement, gratitude, love, optimism, relief, pride, admiration, desire, caring |
| sadness | sadness, disappointment, embarrassment, grief, remorse |
| surprise | surprise, realization, confusion, curiosity |
Only comments whose labels map to exactly one Ekman emotion were kept. Comments with mixed emotions, and those labeled only neutral, were dropped.
| Split | Comments |
|---|---|
| Train | 28,104 |
| Validation | 3,527 |
| Test | 3,539 |
Train class distribution: joy 15,809 · anger 4,590 · surprise 4,160 · sadness 2,455 · fear 566 · disgust 524.
distilbert-base-uncasedTrainerTest set (3,539 comments):
| Metric | Score |
|---|---|
| Accuracy | 0.7934 |
| Macro F1 | 0.6962 |
| Emotion | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| joy | 0.937 | 0.854 | 0.894 | 1,937 |
| sadness | 0.615 | 0.674 | 0.643 | 298 |
| anger | 0.675 | 0.704 | 0.689 | 609 |
| fear | 0.625 | 0.765 | 0.688 | 85 |
| surprise | 0.675 | 0.771 | 0.720 | 533 |
| disgust | 0.490 | 0.610 | 0.543 | 77 |
| Model | Accuracy | Macro F1 |
|---|---|---|
| DistilBERT (this model) | 0.7934 | 0.6962 |
| BiLSTM + Attention (trainable embeddings) | 0.7426 | 0.6526 |
| LSTM (trainable embeddings) | 0.7251 | 0.6359 |
| GRU (trainable embeddings) | 0.7321 | 0.6289 |
| GRU (frozen embeddings) | 0.7098 | 0.6164 |
| BiLSTM + Attention (frozen embeddings) | 0.7056 | 0.6164 |
| LSTM (frozen embeddings) | 0.6917 | 0.5867 |
Base model
distilbert/distilbert-base-uncased