|
Download README.md from Akash212/multicorpus-hate-speech-ensemble: direct link, hf CLI and curl.
- Browser
- Download file 1.68 kB
-
https://huggingface.co/Akash212/multicorpus-hate-speech-ensemble/resolve/main/README.md
- Command line
-
hf download hf://Akash212/multicorpus-hate-speech-ensemble/README.md
-
curl -L -o README.md https://huggingface.co/Akash212/multicorpus-hate-speech-ensemble/resolve/main/README.md
1.68 kB
| license: other | |
| language: [en, es, fr, ar, ur] | |
| tags: [text-classification, hate-speech, multilingual, ensemble] | |
| # Multi-corpus hate speech ensemble | |
| Binary classifier: 1 = abusive or hateful, 0 = neither. Trained on ten public corpora in English, Spanish, | |
| French, Arabic and Roman Urdu (Davidson, HateXplain, DynaHate, ETHOS, HatEval EN/ES, RUHSOLD, MLMA EN/FR/AR). | |
| The ensemble weights and threshold were chosen with HABC on validation data | |
| (minimax objective). | |
| ## Test results (5,874 texts, never used for any choice) | |
| accuracy 0.7673 | macro-F1 0.7386 | worst-corpus macro-F1 0.5312 | ROC-AUC 0.8264 | |
| | corpus | n | accuracy | macro-F1 | | |
| |---|---|---|---| | |
| | davidson2017 | 896 | 0.9118 | 0.8486 | | |
| | dynabench2021 | 900 | 0.6044 | 0.6042 | | |
| | ethos2022 | 150 | 0.6667 | 0.6563 | | |
| | hateval2019_en | 150 | 0.6867 | 0.6791 | | |
| | hateval2019_es | 75 | 0.6267 | 0.5879 | | |
| | hatexplain2021 | 899 | 0.6841 | 0.6714 | | |
| | mlma2019_ar | 492 | 0.7764 | 0.7180 | | |
| | mlma2019_en | 819 | 0.8449 | 0.5312 | | |
| | mlma2019_fr | 593 | 0.7707 | 0.6035 | | |
| | romanurdu2020 | 900 | 0.8333 | 0.8333 | | |
| ## Use | |
| ```python | |
| from huggingface_hub import snapshot_download | |
| import sys | |
| path = snapshot_download("Akash212/multicorpus-hate-speech-ensemble", token="<your token>") | |
| sys.path.insert(0, path) | |
| from predict import HateEnsemble | |
| model = HateEnsemble(path) | |
| model.predict(["text one", "text two"]) | |
| ``` | |
| ## Limitations | |
| Research model. Labels from different corpora follow different definitions, and hate and offensive language | |
| are merged. Scores drop on data unlike the training corpora. Use it to help human moderators, not to remove | |
| content automatically. Some training corpora allow research use only. | |