Add the model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,82 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language: id
|
| 3 |
+
license: cc-by-nc-sa-4.0
|
| 4 |
+
library_name: scikit-learn
|
| 5 |
+
pipeline_tag: text-classification
|
| 6 |
+
tags:
|
| 7 |
+
- abusive-language
|
| 8 |
+
- indonesian
|
| 9 |
+
- tf-idf
|
| 10 |
+
- logistic-regression
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# SlangGuard abusive language detector
|
| 14 |
+
|
| 15 |
+
A small model that says whether an Indonesian text contains **abusive language** (kata kasar).
|
| 16 |
+
It is the second model of [SlangGuard](https://github.com/Akmal76/slangguard), a portfolio
|
| 17 |
+
project. It is separate from the sentiment model, because the two questions differ: "ujaran ini
|
| 18 |
+
negatif?" and "ujaran ini kasar?" have different answers for friendly teasing and for polite anger.
|
| 19 |
+
|
| 20 |
+
It is a scikit-learn pipeline: TF-IDF over groups of 2 to 5 letters inside words
|
| 21 |
+
(`char_wb`), then logistic regression. Letter groups let it catch spellings like `k0ntol` and
|
| 22 |
+
`kntl` that a whole-word model has never seen. The file is 1.3 MB and runs on a CPU.
|
| 23 |
+
|
| 24 |
+
## How to use
|
| 25 |
+
|
| 26 |
+
The text must be normalized like the training text: lowercase, no punctuation, no standalone
|
| 27 |
+
`amp`, one space between words. The function is `normalize_text` in
|
| 28 |
+
[`src/data/clean.py`](https://github.com/Akmal76/slangguard/blob/main/src/data/clean.py).
|
| 29 |
+
|
| 30 |
+
```python
|
| 31 |
+
import joblib
|
| 32 |
+
from huggingface_hub import hf_hub_download
|
| 33 |
+
|
| 34 |
+
path = hf_hub_download("amc76/slangguard-abusive", "model.joblib", revision="<commit>")
|
| 35 |
+
model = joblib.load(path) # a pickle: load only a revision you trust
|
| 36 |
+
score = model.predict_proba(["dasar goblok lu"])[:, 1] # probability of "abusive"
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
**A pickle can run code when it is loaded.** Pin the revision and do not load files from repos
|
| 40 |
+
you do not trust.
|
| 41 |
+
|
| 42 |
+
## Training data
|
| 43 |
+
|
| 44 |
+
[Multi-label hate speech and abusive language detection in Indonesian Twitter](https://github.com/okkyibrohim/id-multi-label-hate-speech-and-abusive-language-detection)
|
| 45 |
+
(Ibrohim and Budi, 2019), column `Abusive`, **CC BY-NC-SA 4.0**. After cleaning there are 12,849
|
| 46 |
+
tweets (38% abusive), split 70/15/15 with seed 42: 8,994 train, 1,927 validation and 1,928 test.
|
| 47 |
+
The text of the data is not included here.
|
| 48 |
+
|
| 49 |
+
Cleaning removed what the source added and a real user would never type: the placeholders
|
| 50 |
+
`USER`, `URL` and `RT`, the escape codes `\n` and `\x..`, and non-ASCII characters. These were
|
| 51 |
+
not noise: `USER` is in 58% of the tweets and 47% of them are abusive (38% overall), so a model
|
| 52 |
+
could have learned it instead of words. Duplicates and texts with two different labels were dropped.
|
| 53 |
+
|
| 54 |
+
## Results
|
| 55 |
+
|
| 56 |
+
The setting `C` was chosen on the validation set only (30, from a grid of 0.3 to 100). The grid
|
| 57 |
+
was widened once, because the best value of the first grid (10) sat at its edge. The test scores
|
| 58 |
+
of both runs were printed, but only the validation scores were used to choose.
|
| 59 |
+
|
| 60 |
+
| Set | F1 (abusive) | Precision | Recall | Macro-F1 |
|
| 61 |
+
| --- | --- | --- | --- | --- |
|
| 62 |
+
| Validation | 0.886 | 0.906 | 0.867 | 0.909 |
|
| 63 |
+
| Test | 0.894 | 0.900 | 0.889 | 0.915 |
|
| 64 |
+
|
| 65 |
+
## Limits
|
| 66 |
+
|
| 67 |
+
- **It finds abusive words, not intent.** Friendly use among friends is flagged, and a polite
|
| 68 |
+
insult is not. Context is not read.
|
| 69 |
+
- **Spellings that it has seen little of are weak.** `anjg` scored 0.68, and a neutral sentence
|
| 70 |
+
was flagged at 0.56 in a hand check of 20 sentences. That check is not a measurement.
|
| 71 |
+
- **The data is Twitter from 2018 and 2019.** Newer slang and other platforms will do worse, and
|
| 72 |
+
the sentiment drift that SlangGuard studies applies here too.
|
| 73 |
+
- **It is not a hate speech detector.** Only the `Abusive` label was used.
|
| 74 |
+
- Do not use it alone to moderate or punish people. It is a signal for a human or a filter.
|
| 75 |
+
|
| 76 |
+
## Citation
|
| 77 |
+
|
| 78 |
+
Ibrohim, M. O. and Budi, I. 2019. Multi-label Hate Speech and Abusive Language Detection in
|
| 79 |
+
Indonesian Twitter. In Proceedings of the Third Workshop on Abusive Language Online, pp. 46-57.
|
| 80 |
+
|
| 81 |
+
The model is licensed **CC BY-NC-SA 4.0**, the same as its training data: attribution,
|
| 82 |
+
non-commercial use, and the same license for derived work.
|