| --- |
| language: en |
| license: mit |
| base_model: distilbert/distilbert-base-uncased |
| pipeline_tag: text-classification |
| tags: |
| - toxicity |
| - content-moderation |
| datasets: |
| - google/jigsaw_unintended_bias |
| --- |
| |
| # DistilBERT toxicity classifier (soft labels + identity weights) |
|
|
| Fine-tuned `distilbert-base-uncased` that predicts whether an English comment is toxic. Built for the capstone project |
| *Trust in Automated Moderation of Toxic Comments* (ECE 4424 / CS 4824). Code: https://github.com/martinagg7/ML-ToxicModeration |
|
|
| ## Training |
|
|
| - Data: Jigsaw Unintended Bias in Toxicity Classification (`train.csv`, 1.8 M news comments), 80/10/10 split grouped by article. |
| - Soft labels: the model learns the share of annotators who marked a comment as toxic, not only 0/1. |
| - Identity weights: comments that mention an identity weigh more in the loss (2 if toxic, 3 if non-toxic). |
| - 2 epochs, learning rate 2e-5, batch size 16, max length 128 tokens. Epoch chosen on validation PR-AUC. |
|
|
| ## Results (held-out test set, 177,635 comments, threshold 0.5) |
|
|
| | Model | Precision | Recall | F1 | ROC AUC | PR-AUC | |
| |---|---|---|---|---|---| |
| | Word blocklist (LDNOOBW) | 0.27 | 0.10 | 0.15 | 0.54 | 0.10 | |
| | TF-IDF + logistic regression | 0.73 | 0.53 | 0.61 | 0.93 | 0.69 | |
| | **This model** | **0.80** | **0.58** | **0.67** | **0.96** | **0.78** | |
|
|
| ## Limitations |
|
|
| - Misses about 4 in 10 toxic comments at threshold 0.5. |
| - Still blocks more innocent comments when they mention an identity (for example 5.6% for gay/lesbian vs 1.3% overall). |
| - Fails on negations with insults ("You are not an idiot"), counter-speech and positive swearing. |
| - Evaluated only on English news comments from the same source as training. |
|
|
| Not intended for fully automatic moderation: uncertain predictions (probability 0.4–0.6) should go to human review. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import pipeline |
| clf = pipeline("text-classification", model="Martinagg/toxic-distilbert") |
| clf("You are an idiot") |
| ``` |
|
|
