amc76 commited on
Commit
ccf9a6d
·
verified ·
1 Parent(s): 0ae724d

Add the model card

Browse files
Files changed (1) hide show
  1. README.md +82 -0
README.md ADDED
@@ -0,0 +1,82 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: id
3
+ license: cc-by-nc-sa-4.0
4
+ library_name: scikit-learn
5
+ pipeline_tag: text-classification
6
+ tags:
7
+ - abusive-language
8
+ - indonesian
9
+ - tf-idf
10
+ - logistic-regression
11
+ ---
12
+
13
+ # SlangGuard abusive language detector
14
+
15
+ A small model that says whether an Indonesian text contains **abusive language** (kata kasar).
16
+ It is the second model of [SlangGuard](https://github.com/Akmal76/slangguard), a portfolio
17
+ project. It is separate from the sentiment model, because the two questions differ: "ujaran ini
18
+ negatif?" and "ujaran ini kasar?" have different answers for friendly teasing and for polite anger.
19
+
20
+ It is a scikit-learn pipeline: TF-IDF over groups of 2 to 5 letters inside words
21
+ (`char_wb`), then logistic regression. Letter groups let it catch spellings like `k0ntol` and
22
+ `kntl` that a whole-word model has never seen. The file is 1.3 MB and runs on a CPU.
23
+
24
+ ## How to use
25
+
26
+ The text must be normalized like the training text: lowercase, no punctuation, no standalone
27
+ `amp`, one space between words. The function is `normalize_text` in
28
+ [`src/data/clean.py`](https://github.com/Akmal76/slangguard/blob/main/src/data/clean.py).
29
+
30
+ ```python
31
+ import joblib
32
+ from huggingface_hub import hf_hub_download
33
+
34
+ path = hf_hub_download("amc76/slangguard-abusive", "model.joblib", revision="<commit>")
35
+ model = joblib.load(path) # a pickle: load only a revision you trust
36
+ score = model.predict_proba(["dasar goblok lu"])[:, 1] # probability of "abusive"
37
+ ```
38
+
39
+ **A pickle can run code when it is loaded.** Pin the revision and do not load files from repos
40
+ you do not trust.
41
+
42
+ ## Training data
43
+
44
+ [Multi-label hate speech and abusive language detection in Indonesian Twitter](https://github.com/okkyibrohim/id-multi-label-hate-speech-and-abusive-language-detection)
45
+ (Ibrohim and Budi, 2019), column `Abusive`, **CC BY-NC-SA 4.0**. After cleaning there are 12,849
46
+ tweets (38% abusive), split 70/15/15 with seed 42: 8,994 train, 1,927 validation and 1,928 test.
47
+ The text of the data is not included here.
48
+
49
+ Cleaning removed what the source added and a real user would never type: the placeholders
50
+ `USER`, `URL` and `RT`, the escape codes `\n` and `\x..`, and non-ASCII characters. These were
51
+ not noise: `USER` is in 58% of the tweets and 47% of them are abusive (38% overall), so a model
52
+ could have learned it instead of words. Duplicates and texts with two different labels were dropped.
53
+
54
+ ## Results
55
+
56
+ The setting `C` was chosen on the validation set only (30, from a grid of 0.3 to 100). The grid
57
+ was widened once, because the best value of the first grid (10) sat at its edge. The test scores
58
+ of both runs were printed, but only the validation scores were used to choose.
59
+
60
+ | Set | F1 (abusive) | Precision | Recall | Macro-F1 |
61
+ | --- | --- | --- | --- | --- |
62
+ | Validation | 0.886 | 0.906 | 0.867 | 0.909 |
63
+ | Test | 0.894 | 0.900 | 0.889 | 0.915 |
64
+
65
+ ## Limits
66
+
67
+ - **It finds abusive words, not intent.** Friendly use among friends is flagged, and a polite
68
+ insult is not. Context is not read.
69
+ - **Spellings that it has seen little of are weak.** `anjg` scored 0.68, and a neutral sentence
70
+ was flagged at 0.56 in a hand check of 20 sentences. That check is not a measurement.
71
+ - **The data is Twitter from 2018 and 2019.** Newer slang and other platforms will do worse, and
72
+ the sentiment drift that SlangGuard studies applies here too.
73
+ - **It is not a hate speech detector.** Only the `Abusive` label was used.
74
+ - Do not use it alone to moderate or punish people. It is a signal for a human or a filter.
75
+
76
+ ## Citation
77
+
78
+ Ibrohim, M. O. and Budi, I. 2019. Multi-label Hate Speech and Abusive Language Detection in
79
+ Indonesian Twitter. In Proceedings of the Third Workshop on Abusive Language Online, pp. 46-57.
80
+
81
+ The model is licensed **CC BY-NC-SA 4.0**, the same as its training data: attribution,
82
+ non-commercial use, and the same license for derived work.