google-research-datasets/go_emotions
Viewer • Updated • 265k • 27.4k • 284
Model fine-tuned from Mamba3-SISO-187M on the GoEmotions dataset for multi-label emotion classification.
The model uses a custom classifier developed by me specifically for Mamba3.
Classifier implementation: GitHub repository.
GoEmotions is a dataset of Reddit comments annotated with 27 emotion categories and neutral. Each sample can have one or multiple emotion labels. A threshold of 0.5 is used for prediction. For more details, see the original paper.
The model was fine-tuned using the following configuration:
| Parameter | Value |
|---|---|
| Pretrained model | Mamba3-SISO-187M |
| Tokenizer | Llama 3.1-8B |
| Classification head | Custom RoBERTa-style head |
| Pooling | Last non-padding token |
| Learning rate | 4.38 × 10⁻⁵ |
| Weight decay | 0.01497 |
| Batch size | 16 |
| Warmup ratio | 0.01828 |
| Learning-rate scheduler | Cosine |
| Maximum gradient norm | 1.0 |
| Dropout | 0.1491 |
| Positive-weight clamp | 10.0 |
| Maximum sequence length | 256 |
| Random seed | 42 |
| Decision threshold | 0.5 |
Clone Classifier implementation and install requirements.txt.
import torch
from src.model_loader import load_model
model = load_model("FeDuck113/mamba3-go_emotions")
predictions = model.predict("I am so grateful for your help!")
print(predictions)
The loaded model can be fine-tuned with the Hugging Face Trainer. This example continues training on GoEmotions
import torch
from transformers import Trainer, TrainingArguments
from src.model_loader import load_model
from src.utils.load_datasets import load_go_emotions
model = load_model("FeDuck113/mamba3-go_emotions")
tokenizer = model.tokenizer
train_ds, val_ds, _, pos_weight, _ = load_go_emotions(
model.tokenize_batch, num_labels=model.num_labels, return_label_names=True
)
model.pos_weight = torch.as_tensor(pos_weight, dtype=torch.float32).clamp(max=10.0)
training_args = TrainingArguments(...)
trainer = Trainer(...)
trainer.train()
| Split | Examples | Subset accuracy | Macro precision | Macro recall | Macro F1 | Micro F1 | Macro AP | Hamming loss |
|---|---|---|---|---|---|---|---|---|
| Validation | 5,426 | 0.2326 | 0.3935 | 0.6752 | 0.4857 | 0.5297 | 0.5008 | 0.0567 |
| Test | 5,427 | 0.2091 | 0.3767 | 0.6649 | 0.4700 | 0.5225 | 0.4849 | 0.0579 |
| Model | Precision | Recall | F1 | Source |
|---|---|---|---|---|
| Mamba3 GoEmotions (this work) | 37.67 | 66.49 | 47.00 | This repository |
| RoBERTa GoEmotions | 57.52 | 39.62 | 45.03 | SamLowe/roberta-base-go_emotions |
| UltraBERT GoEmotions | 65.67 | 42.98 | 49.67 | cirimus/ultrabert-380M-go-emotions |
| Emotion | Support | Precision | Recall | F1 | AP |
|---|---|---|---|---|---|
| admiration | 504 | 0.4619 | 0.8671 | 0.6028 | 0.7260 |
| amusement | 264 | 0.6295 | 0.9205 | 0.7477 | 0.8060 |
| anger | 198 | 0.2762 | 0.6515 | 0.3880 | 0.3955 |
| annoyance | 320 | 0.2032 | 0.6750 | 0.3124 | 0.3202 |
| approval | 351 | 0.2479 | 0.6695 | 0.3618 | 0.4344 |
| caring | 135 | 0.2877 | 0.6222 | 0.3934 | 0.3634 |
| confusion | 153 | 0.2359 | 0.7386 | 0.3576 | 0.4011 |
| curiosity | 284 | 0.4091 | 0.8873 | 0.5600 | 0.5123 |
| desire | 83 | 0.4128 | 0.5422 | 0.4688 | 0.4513 |
| disappointment | 151 | 0.1555 | 0.4768 | 0.2345 | 0.2523 |
| disapproval | 267 | 0.2600 | 0.6816 | 0.3764 | 0.3597 |
| disgust | 123 | 0.3255 | 0.6748 | 0.4392 | 0.4985 |
| embarrassment | 37 | 0.4500 | 0.4865 | 0.4675 | 0.4045 |
| excitement | 103 | 0.2651 | 0.5534 | 0.3585 | 0.4209 |
| fear | 78 | 0.4745 | 0.8333 | 0.6047 | 0.7170 |
| gratitude | 352 | 0.7557 | 0.9403 | 0.8380 | 0.9553 |
| grief | 6 | 0.5000 | 0.5000 | 0.5000 | 0.3611 |
| joy | 161 | 0.3296 | 0.7329 | 0.4547 | 0.5408 |
| love | 238 | 0.6324 | 0.9034 | 0.7439 | 0.8430 |
| nervousness | 23 | 0.1739 | 0.3478 | 0.2319 | 0.2009 |
| optimism | 186 | 0.3664 | 0.6559 | 0.4701 | 0.5719 |
| pride | 16 | 0.4286 | 0.3750 | 0.4000 | 0.2552 |
| realization | 145 | 0.2090 | 0.3517 | 0.2622 | 0.2213 |
| relief | 11 | 0.2000 | 0.3636 | 0.2581 | 0.1376 |
| remorse | 56 | 0.5426 | 0.9107 | 0.6800 | 0.6147 |
| sadness | 156 | 0.3567 | 0.7500 | 0.4835 | 0.5773 |
| surprise | 141 | 0.3731 | 0.6879 | 0.4838 | 0.5332 |
| neutral | 1,787 | 0.5840 | 0.8170 | 0.6811 | 0.7027 |