mental-bert-depemo2 / README.md
mahir2021's picture
Upload Mental-BERT
eac9e39 verified
|
Raw History Blame Contribute Delete
1.68 kB
---
library_name: transformers
tags: [mental-health, multi-label, mental-bert-base-uncased]
---
# Mental-BERT
Multi-label emotion classifier trained on the depEMO dataset.
- **Base encoder:** `mental/mental-bert-base-uncased`
- **Architecture:** CLSDiscriminator (encoder → CLS → head)
- **Labels (8):** anger, brain dysfunction (forget), emptiness, hopelessness, loneliness, sadness, suicide intent, worthlessness
- **Best seed:** 43, **Epochs:** 5, **LR:** 2e-05, **MaxLen:** 192
- **Per-class threshold tuning** on the validation set was applied.
## Test metrics (seed-averaged predictions)
- **eval_P-Mic**: 0.7567
- **eval_R-Mic**: 0.8927
- **eval_F1-Mic**: 0.8191
- **eval_P-Mac**: 0.7294
- **eval_R-Mac**: 0.8595
- **eval_F1-Mac**: 0.7870
- **eval_loss**: 0.3720
## Files
- `config.json`, `model.safetensors`, `tokenizer*`: HuggingFace encoder
- `custom_head.bin`: `torch.state_dict` of the classification head (plus adv head and/or BiLSTM if applicable)
- `custom_config.json`: metadata to rebuild the head
- `emotion_mapping.json`: label ↔ index
- `hyperparameters.json`, `split_indices.json`, `training_log.json`
- `per_label_report.csv`, `test_results.json`
## Reload
```python
from transformers import AutoModel, AutoTokenizer
import torch, json
from huggingface_hub import hf_hub_download
repo_id = "mahir2021/mental-bert-depemo2"
cfg = json.load(open(hf_hub_download(repo_id, 'custom_config.json')))
enc = AutoModel.from_pretrained(repo_id)
tok = AutoTokenizer.from_pretrained(repo_id)
head = torch.load(hf_hub_download(repo_id, 'custom_head.bin'), map_location='cpu')
# then instantiate the head class from cfg['arch'] and load_state_dict(head)
```