| --- |
| license: mit |
| language: |
| - multilingual |
| library_name: transformers |
| pipeline_tag: text-classification |
| widget: |
| - text: "You wont believe what happened to me today" |
| - text: "You wont believe what happened to me today!" |
| - text: "You wont believe what happened to me today..." |
| - text: "You wont believe what happened to me today <3" |
| - text: "You wont believe what happened to me today :)" |
| - text: "You wont believe what happened to me today :(" |
| --- |
| This is an emotion classification model based on fine-tuning of a Bernice model, which is a pre-trained model trained on multilingual Twitter data. |
| The fine-tuning dataset is a subset of the self-labeled emotion dataset (Lykousas et al., 2019) in English that corresponds to Anger, Fear, Sadness, Joy, and Affection. |
| See the paper, [LEIA: Linguistic Embeddings for the Identification of Affect](https://doi.org/10.1140/epjds/s13688-023-00427-0) for further details. |
|
|
| ## Evaluation |
| We evaluated LEIA-multilingual on posts with self-annotated emotion labels identified as non-English using an ensemble of language identification tools. |
| The table below shows the macro-F1 scores aggregated across emotion categories for each language: |
|
|
|
|
| |
| |Language|Macro-F1| |
| |:---:|:---:| |
| |ar |44.18[43.07,45.29]| |
| |da |65.44[60.96,69.83] | |
| |de |60.47[57.58,63.38] | |
| |es |61.67[60.79,62.55] | |
| |fi |45.1[40.96,49.14] | |
| |fr |65.78[63.19,68.36] | |
| |it |63.37[59.67,67.1] | |
| |pt |57.27[55.15,59.4] | |
| |tl |58.37[55.51,61.23] | |
| |tr |45.42[41.17,49.79]| |
|
|
|
|
|
|
| ## Citation |
| Please cite the following paper if you find the model useful for your work: |
| ```bibtex |
| @article{aroyehun2023leia, |
| title={LEIA: Linguistic Embeddings for the Identification of Affect}, |
| author={Aroyehun, Segun Taofeek and Malik, Lukas and Metzler, Hannah and Haimerl, Nikolas and Di Natale, Anna and Garcia, David}, |
| journal={EPJ Data Science}, |
| volume={12}, |
| year={2023}, |
| publisher={Springer} |
| } |
| ``` |
