| ---
|
| license: mit
|
| tags:
|
| - emotion-classification
|
| - mental-health
|
| - multi-label
|
| - transformers
|
| - distilbert
|
| - goemotions
|
| language:
|
| - en
|
| metrics:
|
| - f1
|
| - precision
|
| - recall
|
| pipeline_tag: text-classification
|
| base_model: distilbert-base-uncased
|
| ---
|
|
|
| # Mental Health Emotion Detection - Enhanced DistilBERT
|
|
|
| This model is a fine-tuned DistilBERT for multi-label emotion classification in mental health applications, detecting 28 different emotions from text input with enhanced architecture and advanced training techniques.
|
|
|
| ## Model Description
|
|
|
| - **Model Type:** Enhanced DistilBERT (Fine-tuned)
|
| - **Base Model:** distilbert-base-uncased
|
| - **Task:** Multi-label emotion classification
|
| - **Dataset:** GoEmotions (balanced and enhanced)
|
| - **Languages:** English
|
| - **Architecture:** Enhanced with additional layers, focal loss, and class balancing
|
|
|
| ## Performance
|
|
|
| | Metric | Score |
|
| |--------|-------|
|
| | F1-Score | 0.298 |
|
| | Precision | 0.459 |
|
| | Recall | 0.260 |
|
| | Accuracy | 89.5% |
|
| | Improvement | 7.6x over baseline |
|
|
|
| ## Emotions Detected
|
|
|
| The model can detect 28 emotions: admiration, amusement, anger, annoyance, approval, caring, confusion, curiosity, desire, disappointment, disapproval, disgust, embarrassment, excitement, fear, gratitude, grief, joy, love, nervousness, optimism, pride, realization, relief, remorse, sadness, surprise, neutral.
|
|
|
| ## Usage
|
|
|
| ```python
|
| from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| import torch
|
|
|
| # Load model and tokenizer
|
| tokenizer = AutoTokenizer.from_pretrained("YOUR_USERNAME/mental-health-enhanced-distilbert")
|
| model = AutoModelForSequenceClassification.from_pretrained("YOUR_USERNAME/mental-health-enhanced-distilbert")
|
|
|
| # Example usage
|
| text = "I'm feeling really anxious about tomorrow"
|
| inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True, max_length=128)
|
|
|
| with torch.no_grad():
|
| outputs = model(**inputs)
|
| predictions = torch.sigmoid(outputs.logits)
|
|
|
| # Get emotion labels
|
| emotions = []
|
| for i, score in enumerate(predictions[0]):
|
| if score > 0.4: # Threshold
|
| emotion = model.config.id2label[i]
|
| emotions.append((emotion, score.item()))
|
|
|
| print(emotions)
|
| ```
|
|
|
| ## Training Details
|
|
|
| ### Enhanced Architecture
|
| - **Base:** DistilBERT with additional hidden layers
|
| - **Enhancements:**
|
| - Layer normalization
|
| - Dropout regularization
|
| - Enhanced forward pass with ReLU activations
|
| - Multi-layer classification head (768 → 512 → 256 → 128 → 28)
|
|
|
| ### Advanced Training Techniques
|
| - **Loss Function:** Focal Loss for class imbalance handling
|
| - **Class Weighting:** Advanced weighting for rare emotions
|
| - **Data Balancing:** Oversampling rare emotions, undersampling common ones
|
| - **Optimization:** AdamW with cosine scheduling
|
| - **Early Stopping:** Patience-based with best model saving
|
|
|
| ### Training Data
|
| - **Dataset:** GoEmotions (balanced subset)
|
| - **Training Samples:** ~12,750
|
| - **Validation Samples:** ~2,250
|
| - **Preprocessing:** Contraction expansion, lowercase normalization
|
| - **Balancing:** Advanced sampling for 28 emotion categories
|
|
|
| ## Model Architecture
|
|
|
| ```
|
| Input Text → DistilBERT Encoder → Enhanced Classification Head
|
| ↓
|
| Hidden Layer 1 (768→512)
|
| ↓
|
| Hidden Layer 2 (512→256)
|
| ↓
|
| Hidden Layer 3 (256→128)
|
| ↓
|
| Output Layer (128→28)
|
| ```
|
|
|
| ## Intended Use
|
|
|
| This model is designed for:
|
| - Mental health chatbots and companions
|
| - Emotion-aware dialogue systems
|
| - Mental health screening tools
|
| - Research in computational psychology
|
| - Empathetic AI applications
|
|
|
| ## Limitations
|
|
|
| - Trained primarily on English text
|
| - Performance may vary with very informal language
|
| - Should not be used as sole diagnostic tool for mental health
|
| - Requires context for optimal performance
|
|
|
| ## Training Metrics by Epoch
|
|
|
| | Epoch | F1-Score | Precision | Recall |
|
| |-------|----------|-----------|--------|
|
| | 1 | 0.0145 | 0.0419 | 0.0089 |
|
| | 2 | 0.1430 | 0.2797 | 0.1211 |
|
| | 3 | 0.2141 | 0.4751 | 0.1804 |
|
| | 4 | 0.2749 | 0.4317 | 0.2340 |
|
| | 5 | 0.2897 | 0.4524 | 0.2533 |
|
| | 6 | 0.2981 | 0.4592 | 0.2597 |
|
|
|
| ## Citation
|
|
|
| If you use this model, please cite:
|
|
|
| ```
|
| @misc{mental-health-emotion-distilbert,
|
| title={Mental Health Emotion Detection - Enhanced DistilBERT},
|
| author={Your Name},
|
| year={2024},
|
| publisher={Hugging Face},
|
| url={https://huggingface.co/YOUR_USERNAME/mental-health-enhanced-distilbert}
|
| }
|
| ```
|
|
|
| ## Acknowledgments
|
|
|
| - Built on DistilBERT by Hugging Face
|
| - Trained on GoEmotions dataset
|
| - Enhanced with advanced ML techniques for mental health applications
|
|
|
