Deeptanshuu commited on
Commit
7fb84fb
·
verified ·
1 Parent(s): cc5d2a4

Publish multilingual toxicity classifier: weights, card, thresholds, config

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ images/pr_curves.png filter=lfs diff=lfs merge=lfs -text
37
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,669 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ - ru
5
+ - tr
6
+ - es
7
+ - fr
8
+ - it
9
+ - pt
10
+ license: apache-2.0
11
+ library_name: transformers
12
+ pipeline_tag: text-classification
13
+ base_model: xlm-roberta-large
14
+ tags:
15
+ - toxicity
16
+ - content-moderation
17
+ - multilingual
18
+ - multi-label
19
+ - xlm-roberta
20
+ - custom_code
21
+ metrics:
22
+ - roc_auc
23
+ - f1
24
+ model-index:
25
+ - name: toxic-comment-multilingual-xlmr
26
+ results:
27
+ - task:
28
+ type: text-classification
29
+ name: Multi-label toxicity classification
30
+ dataset:
31
+ type: custom
32
+ name: Multilingual Jigsaw-derived toxicity corpus (7 languages)
33
+ split: test
34
+ metrics:
35
+ - type: roc_auc
36
+ name: Macro AUC-ROC
37
+ value: 0.9852
38
+ - type: f1
39
+ name: Macro F1 (tuned thresholds)
40
+ value: 0.8814
41
+ - type: f1
42
+ name: Macro F1 (threshold 0.5)
43
+ value: 0.8821
44
+ ---
45
+
46
+ # toxic-comment-multilingual-xlmr
47
+
48
+ Multi-label toxicity classification for online comments in seven languages:
49
+ English, Russian, Turkish, Spanish, French, Italian, Portuguese.
50
+
51
+ The model is XLM-RoBERTa-large with one extra attention block on top whose
52
+ attention scores carry a per-language bias, followed by a small classification
53
+ head. It emits six independent probabilities per comment.
54
+
55
+ **Read the [Known limitations](#known-limitations) and
56
+ [Out-of-scope use](#out-of-scope-use) sections before you deploy this.** They are
57
+ not boilerplate. This model has a documented history of a published version whose
58
+ headline number meant something quite different from what it appeared to mean,
59
+ and the card explains that in full.
60
+
61
+ ---
62
+
63
+ ## What the six labels mean
64
+
65
+ | Label | Index | Meaning in the source annotation scheme |
66
+ |---|---|---|
67
+ | `toxic` | 0 | Rude, disrespectful, or likely to make someone leave a discussion |
68
+ | `severe_toxic` | 1 | An extreme case of the above. In the source scheme this is a *subset* of `toxic` |
69
+ | `obscene` | 2 | Obscene or vulgar language |
70
+ | `threat` | 3 | A threat of violence against a person or group |
71
+ | `insult` | 4 | Insulting, inflammatory, or negative toward a person |
72
+ | `identity_hate` | 5 | Hatred toward an identity group: race, religion, gender, sexuality, nationality |
73
+
74
+ The index column is the position in the output vector. It is worth memorising
75
+ that index 1 is `severe_toxic` and index 2 is `obscene`, because that is neither
76
+ alphabetical nor the order most people assume.
77
+
78
+ ### Why six sigmoids and not a softmax
79
+
80
+ The labels are **non-exclusive**. A single comment can be toxic *and* obscene
81
+ *and* an insult at the same time, and most toxic comments in the training data
82
+ carry two or three labels at once. That is a different problem from picking one
83
+ class out of six.
84
+
85
+ A softmax forces the six scores to sum to 1, so raising one necessarily lowers
86
+ the others. That is exactly wrong here: the fact that a comment is obscene
87
+ should not reduce the model's belief that it is also an insult.
88
+
89
+ So the head produces six raw scores (logits) and each one goes through its own
90
+ sigmoid independently:
91
+
92
+ ```
93
+ p_k = 1 / (1 + exp(-logit_k)) for k = 0..5
94
+ ```
95
+
96
+ Each `p_k` is that class's own answer to its own yes/no question. The six values
97
+ do not sum to anything in particular. You compare each one to its own threshold
98
+ and take every class that clears it.
99
+
100
+ ---
101
+
102
+ ## Quick start
103
+
104
+ This is a **custom architecture**, not a stock Hugging Face model. `AutoModel`
105
+ alone will not build it. The model definition ships in this repo as
106
+ `modeling_toxic_xlmr.py`, and you need `trust_remote_code=True` so that
107
+ `transformers` will execute it.
108
+
109
+ ```python
110
+ import json, torch
111
+ from huggingface_hub import hf_hub_download
112
+ from transformers import AutoModel, AutoTokenizer
113
+
114
+ REPO = "Deeptanshuu/toxic-comment-multilingual-xlmr"
115
+ LANGUAGE_IDS = {"en": 0, "ru": 1, "tr": 2, "es": 3, "fr": 4, "it": 5, "pt": 6}
116
+ LABELS = ["toxic", "severe_toxic", "obscene", "threat", "insult", "identity_hate"]
117
+
118
+ model = AutoModel.from_pretrained(REPO, trust_remote_code=True).eval()
119
+ tokenizer = AutoTokenizer.from_pretrained(REPO)
120
+
121
+ with open(hf_hub_download(REPO, "thresholds.json")) as f:
122
+ thresholds = json.load(f)["thresholds"]
123
+ cut = torch.tensor([thresholds[name] for name in LABELS])
124
+
125
+ texts = ["You are an absolute idiot.", "Sei un cretino, vattene via."]
126
+ langs = ["en", "it"]
127
+
128
+ enc = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
129
+ lang_ids = torch.tensor([LANGUAGE_IDS[l] for l in langs])
130
+
131
+ with torch.no_grad():
132
+ probs = model(
133
+ input_ids=enc["input_ids"],
134
+ attention_mask=enc["attention_mask"],
135
+ lang_ids=lang_ids,
136
+ )["probabilities"]
137
+
138
+ for text, row in zip(texts, probs):
139
+ fired = [name for name, p, t in zip(LABELS, row, cut) if p >= t]
140
+ print(text, "->", fired or ["clean"])
141
+ ```
142
+
143
+ `inference_example.py` in this repo is the same thing as a runnable script with
144
+ batching, and it is the version that has actually been executed end to end.
145
+
146
+ ### If you would rather not run remote code
147
+
148
+ `modeling_toxic_xlmr.py` also contains a plain `load_model()` helper. Download
149
+ the repo, then:
150
+
151
+ ```python
152
+ from huggingface_hub import snapshot_download
153
+ import sys
154
+
155
+ path = snapshot_download("Deeptanshuu/toxic-comment-multilingual-xlmr")
156
+ sys.path.insert(0, path)
157
+
158
+ from modeling_toxic_xlmr import load_model
159
+ model, tokenizer = load_model(path, device="cpu")
160
+ ```
161
+
162
+ This constructs the module and calls `load_state_dict` directly. No auto
163
+ classes, no dynamic module machinery. You are still executing the same Python
164
+ file, but you can read it first, which is the point.
165
+
166
+ ---
167
+
168
+ ## The `lang_ids` input, which will trip you up
169
+
170
+ The forward pass takes **three** inputs, not the usual two:
171
+
172
+ ```python
173
+ model(input_ids=..., attention_mask=..., lang_ids=...)
174
+ ```
175
+
176
+ `lang_ids` is a `LongTensor` of shape `[batch_size]`, one integer per sequence:
177
+
178
+ | Language | `lang_id` |
179
+ |---|---|
180
+ | English | 0 |
181
+ | Russian | 1 |
182
+ | Turkish | 2 |
183
+ | Spanish | 3 |
184
+ | French | 4 |
185
+ | Italian | 5 |
186
+ | Portuguese | 6 |
187
+
188
+ The model looks the id up in a learned embedding table and uses it to bias the
189
+ attention scores in the final block. It is the one place language identity
190
+ enters the network.
191
+
192
+ **If you omit `lang_ids`, the model does not fail.** It fills the batch with
193
+ zeros, which means every comment is scored as if it were English. You get a
194
+ `UserWarning` once per process, and the raw logits shift by a measurable amount
195
+ (see [Known limitations](#known-limitations) for the measured size of that
196
+ shift). That shift is real but, as far as this project has been able to
197
+ measure, harmless: the language-conditioning ablation found no detectable
198
+ accuracy cost to disabling the pathway entirely. So "quietly worse" overstates
199
+ it — omitting `lang_ids` changes individual outputs without demonstrably
200
+ degrading them on average. Pass it anyway. The alternative is silently treating
201
+ unknown-language text as English with nothing telling you that happened, which
202
+ is still the easiest way to misuse this model by accident, even though it has
203
+ not been shown to cost accuracy.
204
+
205
+ The same applies to the `text-classification` pipeline, which works but cannot
206
+ pass `lang_ids`:
207
+
208
+ ```python
209
+ # Works, but treats every input as English and applies no thresholds.
210
+ from transformers import pipeline
211
+ pipe = pipeline("text-classification", model=REPO, trust_remote_code=True, top_k=None)
212
+ ```
213
+
214
+ Ids outside 0-6 are clamped into range with a warning rather than raising. If
215
+ your text is in a language not in the table, there is no good id to pass: this
216
+ model has never seen that language's toxicity labels, and picking the
217
+ typologically nearest option is a guess, not a fallback.
218
+
219
+ ---
220
+
221
+ ## Thresholds: do not use 0.5
222
+
223
+ `thresholds.json` holds one decision threshold per class. Use it.
224
+
225
+ Across every version of this model the pattern has been the same: **ranking
226
+ quality is good on all six classes, calibration is poor on the three rare
227
+ ones.** AUC sits high, which means the model orders comments correctly, but the
228
+ absolute probabilities for `severe_toxic`, `threat` and `identity_hate` cluster
229
+ well below 0.5 even for true positives, because those classes are only 2-5% of
230
+ the training data. Cutting at 0.5 therefore throws away most of their recall for
231
+ very little precision in return.
232
+
233
+ The thresholds in this repo were chosen to maximise per-class F1 on the
234
+ validation split, and every one of them sits below 0.5.
235
+
236
+ There is deliberately **no per-language threshold block** in this repo. The
237
+ evaluation script can emit one, but that code path builds an unshuffled
238
+ cross-validation split instead of a stratified one, so folds containing zero
239
+ positive examples combine with a `zero_division=1` setting to inject free F1
240
+ scores of 1.0. It reported English `severe_toxic` at F1 0.597 when the maximum
241
+ achievable at any threshold is 0.442. Nothing ever read those numbers, and they
242
+ are not published here.
243
+
244
+ ---
245
+
246
+ ## Intended use
247
+
248
+ - Flagging comments for **human** review in a moderation queue.
249
+ - Prioritising a review backlog by likely severity.
250
+ - Research and coursework on multilingual toxicity detection, multi-label
251
+ classification, or threshold calibration on imbalanced data.
252
+ - A baseline to beat.
253
+
254
+ ## Out-of-scope use
255
+
256
+ **Do not use this as the sole automated decision-maker for moderation.** No
257
+ comment should be deleted, no account banned, and no user penalised on this
258
+ model's output alone. It is a triage aid. Keep a human in the loop for anything
259
+ with a consequence attached.
260
+
261
+ Beyond that:
262
+
263
+ - **Performance is uneven across languages, measurably so.** In the previous
264
+ version's evaluation English led the worst language by about 5 points of macro
265
+ AUC (English 0.946, Turkish 0.894). The same ordering shows up here, but the
266
+ gap is much smaller (English 0.990, Turkish 0.973 — see
267
+ [Results](#results) for the full breakdown). A single global threshold will
268
+ still be tighter or looser in practice depending on the language, even though
269
+ the number itself is the same.
270
+ - **Reclaimed slurs and in-group language are a known failure mode of this class
271
+ of model.** Communities that use slurs about themselves affectionately, drag
272
+ and roast culture, AAVE, and queer in-group speech all reliably draw false
273
+ positives from toxicity classifiers trained on annotator judgements of the
274
+ kind used here. This model has not been evaluated for that, which means the
275
+ problem is unmeasured, not absent. Deploying it against a community that talks
276
+ that way will disproportionately flag that community.
277
+ - **Quoted and counter-speech toxicity is scored the same as first-person
278
+ toxicity.** A comment reporting abuse, or quoting a slur to object to it,
279
+ looks much like the abuse itself to this model.
280
+ - **Not a substitute for a threat-assessment process.** The `threat` class is
281
+ the rarest and among the weakest. Do not route safety-critical escalation
282
+ through it.
283
+ - **Not calibrated probabilities.** A score of 0.8 does not mean 80% of
284
+ annotators would call the comment toxic. Treat the outputs as ranks, and use
285
+ the thresholds.
286
+ - **The base rate in the evaluation data is nothing like a real queue.** Roughly
287
+ half the comments in this corpus are labelled `toxic`. A live moderation
288
+ stream is typically a few percent. Precision on real traffic will be
289
+ substantially worse than any precision figure in this card, because precision
290
+ depends on the base rate and the base rate here is inflated by roughly an
291
+ order of magnitude. This is the single most common way people are disappointed
292
+ by a toxicity classifier in production.
293
+ - Not intended for languages outside the seven listed, for long-form documents,
294
+ or for detecting misinformation, spam, or self-harm content.
295
+
296
+ ---
297
+
298
+ ## Training data
299
+
300
+ Built from Jigsaw toxic comment data, extended to seven languages. 356,580
301
+ comments, split 285,264 train / 35,658 validation / 35,658 test, roughly balanced
302
+ across languages (about 14.5% each; English 13.0%).
303
+
304
+ Class balance is heavily skewed: roughly 48-50% `toxic`, but only 2-5% for
305
+ `threat`, `severe_toxic` and `identity_hate`.
306
+
307
+ ### Provenance limits you should know about
308
+
309
+ These are real caveats, not disclaimers.
310
+
311
+ - **The script that produced the multilingual corpus is not in the source
312
+ repository.** The pipeline is reproducible from the multilingual CSV onward,
313
+ but the step that got from Jigsaw's English data to seven languages is not.
314
+ Nobody outside the project can audit or rerun it.
315
+ - **The label definitions are English Wikipedia talk-page definitions.** The
316
+ original Jigsaw annotation guidelines were written for one community, in one
317
+ language, at one point in time. Whatever the missing step did to carry those
318
+ labels into Russian, Turkish, Spanish, French, Italian and Portuguese,
319
+ the label semantics for those six languages are inherited from an unauditable
320
+ process. Non-English performance numbers should be read with that in mind.
321
+ - **Rare classes were topped up with synthetic data.** `threat` in particular
322
+ was augmented with samples generated by Mistral-7B-Instruct-v0.3 in 4-bit.
323
+ Their labels come from the generating prompt, not from annotation, so they are
324
+ weakly labelled. The lightweight classifier used to filter the generated
325
+ samples was trained on the same distribution it filters, which is circular:
326
+ it keeps the synthetic samples that look like what the model already believes
327
+ toxicity looks like, and discards the ones that would have taught it something
328
+ new.
329
+ - **Measured near-duplicate leakage.** Exact `comment_text` overlap between
330
+ splits is zero. But near-duplicates (character 3-4-gram TF-IDF, cosine at or
331
+ above 0.9 against train) account for **3.8% of English validation rows and
332
+ 0.6% of Russian**. The cause is that augmentation runs before the split, so
333
+ LLM-generated variants of the same seed text can land on both sides. Exact-hash
334
+ deduplication does not catch these. Held-out numbers are therefore mildly
335
+ optimistic on top of everything else.
336
+
337
+ ---
338
+
339
+ ## Evaluation protocol
340
+
341
+ Thresholds are tuned on the **validation** split. All reported metrics are
342
+ computed on the **held-out test** split, which was not used for tuning or model
343
+ selection.
344
+
345
+ **Why that separation matters.** Per-class thresholds are free parameters, six of
346
+ them. If you pick the thresholds that maximise F1 on a split and then report F1
347
+ on that same split, you have fitted those six parameters to that split's noise
348
+ and are reporting the fit as if it were a prediction. The number goes up and
349
+ means less. AUC is threshold-free and so is unaffected, which is exactly why AUC
350
+ and F1 can disagree about whether a change helped.
351
+
352
+ An earlier version of this project's results reported tuned-threshold metrics on
353
+ the same split it tuned on. Correcting the protocol moved the numbers by less
354
+ than 0.003 in the end, so nobody was materially misled in practice. It was still
355
+ wrong in principle, and it is fixed.
356
+
357
+ Evaluation uses `max_length=512`, matching training. An earlier evaluation
358
+ defaulted to 128, which truncated about 16% of real comments in the test split
359
+ and quietly changed what was being measured.
360
+
361
+ ---
362
+
363
+ ## Results
364
+
365
+
366
+ ![F1 by class, previous version versus this one](images/f1_gains_by_class.png)
367
+
368
+ ![Threat probability distributions, previous version versus this one](images/threat_probability_shift.png)
369
+
370
+ Both versions rank `threat` comparably (ROC-AUC 0.905 against 0.975), but the previous one's threat
371
+ scores sat on top of the non-threat scores and mostly below any usable cut-off: 80% of real threats
372
+ scored under 0.5, against 16% here. That is why F1 moved four times as much as AUC did. A model can
373
+ rank acceptably and still be unusable at every threshold.
374
+
375
+ ![Precision-recall curves for all six labels](images/pr_curves.png)
376
+
377
+ Read the precision-recall curves rather than the ROC ones if you are deciding whether to deploy
378
+ this. With three labels at 2-5% positive rate, ROC flatters the model because the false-positive
379
+ rate is divided by an enormous negative pool. Precision-recall prices the same predictions against
380
+ the positives, and each legend entry carries the label's base rate so the curve can be read against
381
+ the prevalence it was measured at.
382
+
383
+ Final metrics for this version, measured on `best_model` (epoch 5 of a 6-epoch
384
+ run that completed all 6 epochs), evaluation run
385
+ `evaluation_results/eval_20260830_072515`.
386
+
387
+ | Metric (test split) | Value |
388
+ |---|---|
389
+ | Macro AUC-ROC | 0.9852 |
390
+ | Weighted AUC-ROC | 0.9890 |
391
+ | Macro F1 at threshold 0.5 | 0.8821 |
392
+ | Macro F1 at tuned thresholds | 0.8814 |
393
+ | Weighted F1 at tuned thresholds | 0.9332 |
394
+ | Exact match | 0.8772 |
395
+
396
+ Per class, on the test split at tuned thresholds:
397
+
398
+ | Class | AUC | Threshold | Precision | Recall | F1 |
399
+ |---|---|---|---|---|---|
400
+ | `toxic` | 0.9921 | 0.4724 | 0.9534 | 0.9750 | 0.9641 |
401
+ | `severe_toxic` | 0.9863 | 0.4724 | 0.7139 | 0.8010 | 0.7549 |
402
+ | `obscene` | 0.9899 | 0.5276 | 0.9419 | 0.9338 | 0.9378 |
403
+ | `threat` | 0.9755 | 0.5276 | 0.8846 | 0.8003 | 0.8403 |
404
+ | `insult` | 0.9855 | 0.5643 | 0.9204 | 0.9266 | 0.9235 |
405
+ | `identity_hate` | 0.9818 | 0.5643 | 0.8959 | 0.8419 | 0.8680 |
406
+
407
+ Per language, macro AUC on the test split:
408
+
409
+ | Language | Macro AUC |
410
+ |---|---|
411
+ | English | 0.9902 |
412
+ | Russian | 0.9790 |
413
+ | Turkish | 0.9726 |
414
+ | Spanish | 0.9882 |
415
+ | French | 0.9877 |
416
+ | Italian | 0.9893 |
417
+ | Portuguese | 0.9832 |
418
+
419
+ `metrics.json` in this repo carries the same values in machine-readable form.
420
+
421
+ ---
422
+
423
+ ## What this looks like on real traffic
424
+
425
+ Every number above is measured on a corpus that is roughly **50% toxic**. Real
426
+ moderation queues are not. A typical forum or comment stream runs somewhere
427
+ between 1% and 5% toxic, and that difference matters more than any metric on this
428
+ page.
429
+
430
+ Recall and false-positive rate are properties of the model and do not change with
431
+ the mix of your data. Precision does:
432
+
433
+ ```
434
+ precision = TPR * p / (TPR * p + FPR * (1 - p))
435
+ ```
436
+
437
+ where `p` is the share of your traffic that is genuinely toxic. Applying the
438
+ model's measured TPR and FPR at three prevalences:
439
+
440
+ | Label | Recall | FPR | Precision @ 50% | @ 5% | @ 1% |
441
+ |---|---|---|---|---|---|
442
+ | toxic | 0.975 | 0.0470 | 0.953 | 0.522 | **0.173** |
443
+ | insult | 0.927 | 0.0321 | 0.920 | 0.603 | 0.226 |
444
+ | obscene | 0.934 | 0.0184 | 0.942 | 0.728 | 0.339 |
445
+ | severe_toxic | 0.801 | 0.0156 | 0.714 | 0.730 | 0.342 |
446
+ | identity_hate | 0.842 | 0.0055 | 0.896 | 0.890 | 0.608 |
447
+ | threat | 0.800 | 0.0023 | 0.885 | 0.948 | **0.779** |
448
+
449
+ **Read the last column before deploying anything.** On a stream that is 1% toxic,
450
+ the `toxic` head flags roughly six comments for every one that is genuinely
451
+ toxic. That is not a defect in training -- it is arithmetic. A 4.7% false-positive
452
+ rate against 99% clean traffic produces far more false positives than there are
453
+ true positives to find.
454
+
455
+ Two consequences worth planning around:
456
+
457
+ - **The rare-class heads hold up best.** `threat` and `identity_hate` have
458
+ false-positive rates of 0.2% and 0.6%, so they stay usable at low prevalence
459
+ (0.78 and 0.61 precision at 1%). This is the opposite of the usual expectation
460
+ and is worth exploiting: the heads you would expect to be weakest are the ones
461
+ that survive a realistic base rate.
462
+ - **Raise the thresholds if you need precision.** The shipped thresholds maximise
463
+ F1 on a balanced corpus. If you are triaging a low-toxicity stream and care
464
+ about not wasting reviewer time, tune your own thresholds on a sample of *your*
465
+ traffic, not on this corpus. `thresholds.json` is a starting point, not a
466
+ recommendation for your distribution.
467
+
468
+ Nothing here is unique to this model; any classifier trained on a balanced corpus
469
+ behaves this way off-distribution. It is stated explicitly because most model
470
+ cards do not, and people are repeatedly surprised by it in production.
471
+
472
+ ## The previous version, and why its 0.9147 does not mean what it looks like
473
+
474
+ This section is the most useful thing in this card for anyone comparing versions.
475
+ It is here because the honest version of this project's history is more
476
+ instructive than a clean one would be.
477
+
478
+ ### What was wrong
479
+
480
+ An earlier version of this model was published in April 2025 with a macro AUC of
481
+ 0.9147. **That model never fine-tuned its XLM-RoBERTa backbone.** Measured
482
+ directly: 4.8M of 564.7M parameters, **0.8%**, actually received a gradient. All
483
+ 381 encoder tensors had `grad = None` after a backward pass.
484
+
485
+ Two bugs multiplied together to cause it:
486
+
487
+ 1. A layer-freezing option intended to freeze the bottom encoder layers instead
488
+ froze the first eight *parameter tensors* of the base model. That is 258.6M
489
+ parameters, almost all of it the 256M-row word-embedding matrix. The
490
+ assertion written to catch exactly this checked the same wrong slice, so it
491
+ passed.
492
+ 2. With the embeddings frozen, the input to the gradient-checkpointed segment no
493
+ longer required grad. PyTorch's `gradient_checkpointing_enable()` defaults to
494
+ `use_reentrant=True`, and a reentrant checkpoint whose input does not require
495
+ grad builds **no backward graph at all** through the segment. Every encoder
496
+ layer above the frozen embeddings silently stopped receiving gradient.
497
+
498
+ Neither bug throws. Training runs, loss goes down, AUC comes out at 0.9147. What
499
+ was actually being trained was the small head on top of a frozen feature
500
+ extractor: a linear probe on stock XLM-R representations.
501
+
502
+ That is a legitimate thing to build. It is not what the card said it was.
503
+
504
+ ### The other thing that was inert
505
+
506
+ The architecture's central claim is that biasing attention by language helps.
507
+ In the April version that bias had a shape that was constant along the axis the
508
+ softmax normalises over. Softmax is shift-invariant along that axis, so the bias
509
+ cancelled **exactly**. `lang_ids` had literally no effect on the output.
510
+
511
+ Measured directly on the shipped `best_model` checkpoint: swapping `lang_ids`
512
+ between two languages on identical text moves the 6 output logits by an L2 norm
513
+ of about 0.19 (mean 0.187 across 3 example texts compared pairwise across all 7
514
+ languages against English; individual comparisons ranged 0.10-0.30 depending on
515
+ text and language pair). In the April code the same test moved the logits by
516
+ 3.6e-07, which is float32 rounding noise — this version's effect is still about
517
+ five orders of magnitude larger. The fix adds the language vector to the
518
+ attention *queries* rather than to the scores or the keys, which is the only one
519
+ of the three placements that survives the softmax.
520
+
521
+ Worth knowing if you are testing something similar: the broken version leaked
522
+ about 4e-08 of float noise into the language embedding's gradient, so a naive
523
+ `assert grad is not None and grad.sum() != 0` **passes** on the broken model. A
524
+ test for "does this parameter actually learn" needs a magnitude threshold, not a
525
+ comparison against exact zero.
526
+
527
+ ### What changed in this version
528
+
529
+ | | April 2025 version | This version |
530
+ |---|---|---|
531
+ | Parameters receiving gradient | 4.8M of 564.7M (0.8%) | 307.1M of 564.7M (54.4%) |
532
+ | Encoder fine-tuned | No | Yes |
533
+ | `lang_ids` affects output | No (cancels under softmax) | Yes (~0.19 logit delta, L2 norm, measured on `best_model`) |
534
+ | Sampler | Drew with replacement; 36.9% of the training set never seen in an epoch | Exact one-pass, 285,264 unique |
535
+ | Class weighting | Never activated | Active, rare classes get about 2.6x the weight of `toxic` |
536
+ | LR warmup | Computed, then never applied | Linear warmup over 10% of steps, then cosine decay |
537
+ | Validation during training | None; best checkpoint picked by hand | Per-epoch, with per-class AUC and automatic model selection |
538
+ | Serving sequence length | 128 (truncated 15.7% of test rows) | 512, matching training |
539
+ | Run completion | Crashed at epoch 4 of 6 on a logging auth error; the published checkpoint is epoch 2 | Completed all 6 of 6 epochs; best checkpoint by validation macro AUC is epoch 5 |
540
+
541
+ The word/position embedding module is **still frozen in this version, on
542
+ purpose**. It is 256M of the 564.7M parameters, and freezing it removes the
543
+ optimizer update that dominates step time without measurably costing quality.
544
+ That is a deliberate choice, unlike last time.
545
+
546
+ For reference, the April model's real numbers, measured on the test split with
547
+ tuned thresholds under the corrected protocol:
548
+
549
+ | Metric | April 2025 version |
550
+ |---|---|
551
+ | Macro AUC | 0.9147 |
552
+ | Macro F1 at 0.5 | 0.5284 |
553
+ | Macro F1 at tuned thresholds | 0.6036 |
554
+ | Weighted F1 | 0.7732 |
555
+ | Exact match | 0.6194 |
556
+
557
+ Per class: `toxic` 0.9666 AUC / 0.9038 F1, `obscene` 0.9278 / 0.7392, `insult`
558
+ 0.9035 / 0.7248, `threat` 0.9051 / 0.4189, `severe_toxic` 0.8988 / 0.3980,
559
+ `identity_hate` 0.8866 / 0.4370. Per language macro AUC: English 0.9463, French
560
+ 0.9157, Spanish 0.9152, Italian 0.9139, Portuguese 0.9088, Russian 0.9065,
561
+ Turkish 0.8944.
562
+
563
+ **These are the previous version's numbers.** They are recorded so the two can
564
+ be compared. They are not this model's results — this model's results are the
565
+ [Results](#results) section above.
566
+
567
+ ---
568
+
569
+ ## Known limitations
570
+
571
+ - **Language conditioning: a measured null result.** The ablation has been run:
572
+ a control model, trained identically except for `disable_lang_conditioning=True`,
573
+ scores macro AUC 0.9849 on the test split against 0.9852 for the treatment — a
574
+ difference of +0.0003 with a paired-bootstrap 95% CI of [-0.0007, +0.0012]
575
+ (p = 0.588). The interval includes zero, and the per-language breakdown is
576
+ mixed-sign (3 of 7 languages score worse with conditioning on), so this is not
577
+ a case of a small effect concentrated where it should be. Language-aware
578
+ attention does not measurably improve accuracy on this corpus with this
579
+ backbone. That is a measured outcome, not a failure: the mechanism is
580
+ confirmed live (see the `lang_ids` section above), it simply does not buy
581
+ anything here. The practical consequence is useful to a downstream user:
582
+ `lang_ids` can be omitted at inference with no measurable cost to accuracy,
583
+ even though the pathway still shifts individual logits by a measurable amount
584
+ when it is used.
585
+ - Non-English label semantics are inherited from a corpus-building step that is
586
+ not in the repository and cannot be audited. See
587
+ [Training data](#training-data).
588
+ - Rare-class performance is limited by rare-class data, and part of that data is
589
+ weakly labelled synthetic text.
590
+ - The measured near-duplicate leakage (3.8% English validation) makes held-out
591
+ metrics slightly optimistic.
592
+ - Fairness has not been evaluated. There is no subgroup analysis, no test for
593
+ identity-term bias, and no evaluation on reclaimed-slur or in-group language.
594
+ Absence of a measured problem is not evidence of absence.
595
+ - Weights ship as a pickled `pytorch_model.bin` state dict rather than
596
+ safetensors, because that is what the training loop writes.
597
+ - `severe_toxic` is a subset of `toxic` in the annotation scheme, so the two are
598
+ strongly correlated by construction. Independent sigmoids do not know that.
599
+
600
+ ---
601
+
602
+ ## Files in this repo
603
+
604
+ | File | What it is |
605
+ |---|---|
606
+ | `modeling_toxic_xlmr.py` | The model definition. Loaded by `trust_remote_code=True`, or importable directly for `load_model()` |
607
+ | `config.json` | Architecture config, including the nested XLM-RoBERTa encoder config and the `auto_map` that points the auto classes at the file above |
608
+ | `pytorch_model.bin` | Trained weights, a raw state dict (about 2.2 GB) |
609
+ | `thresholds.json` | Per-class decision thresholds tuned on validation, plus their provenance |
610
+ | `metrics.json` | Evaluation results, machine-readable, including the previous version's for comparison |
611
+ | `training_config.json` | The exact configuration the training run used |
612
+ | `inference_example.py` | Runnable batch-prediction example with thresholds applied |
613
+ | tokenizer files | Stock `xlm-roberta-large` tokenizer, unmodified, shipped so the repo is self-contained |
614
+
615
+ ## Training configuration
616
+
617
+ Full detail in `training_config.json`. The parts that matter:
618
+
619
+ | Setting | Value |
620
+ |---|---|
621
+ | Base model | `xlm-roberta-large` (24 layers, hidden 1024, 16 heads) |
622
+ | Max sequence length | 512 |
623
+ | Batch size | 64 with gradient accumulation 2 (effective 128) |
624
+ | Epochs | 6 |
625
+ | Optimizer | AdamW, lr 2e-5, weight decay 2e-7 |
626
+ | Schedule | Linear warmup over 10% of steps, then cosine to 1% of peak |
627
+ | Mixed precision | fp16 |
628
+ | Gradient checkpointing | Enabled, `use_reentrant=False` |
629
+ | Frozen | Embedding module only (256M parameters), deliberately |
630
+ | Loss | Focal loss over BCE-with-logits, with per-language dynamic class weights |
631
+ | Hardware | Single Quadro RTX 6000 (24 GB) |
632
+
633
+ `use_reentrant=False` is not a detail. It costs about 4x the step time and twice
634
+ the memory of the reentrant variant, and it is the difference between the
635
+ encoder receiving gradients and receiving nothing at all.
636
+
637
+ ---
638
+
639
+ ## License
640
+
641
+ Apache 2.0.
642
+
643
+ The base model, `xlm-roberta-large`, is MIT licensed. Apache-2.0 is a permissive
644
+ license compatible with that, and it additionally grants an explicit patent
645
+ license that plain MIT does not — that is why it was chosen here rather than
646
+ just inheriting MIT as-is.
647
+
648
+ This license covers the model weights and the code in this repository. It does
649
+ **not** extend to the underlying training data:
650
+
651
+ - The training data derives from Jigsaw's toxic comment dataset and Wikipedia
652
+ talk-page comments, which carry their own terms.
653
+ - Rare-class synthetic augmentation was generated with
654
+ Mistral-7B-Instruct-v0.3, which carries its own license.
655
+
656
+ Using these weights does not grant any rights to that underlying data. This
657
+ card licenses what this repository ships — weights and code — and makes no
658
+ claim about, and does not settle, what the source data's own terms permit.
659
+
660
+ ## Citation
661
+
662
+ ```bibtex
663
+ @misc{toxic_comment_multilingual_xlmr,
664
+ author = {Deeptanshu Lal},
665
+ title = {toxic-comment-multilingual-xlmr: multilingual multi-label toxicity classification},
666
+ year = {2026},
667
+ howpublished = {\url{https://huggingface.co/Deeptanshuu/toxic-comment-multilingual-xlmr}}
668
+ }
669
+ ```
config.json ADDED
@@ -0,0 +1,112 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_cross_attention": false,
3
+ "architectures": [
4
+ "ToxicCommentModel"
5
+ ],
6
+ "auto_map": {
7
+ "AutoConfig": "modeling_toxic_xlmr.ToxicCommentConfig",
8
+ "AutoModel": "modeling_toxic_xlmr.ToxicCommentModel",
9
+ "AutoModelForSequenceClassification": "modeling_toxic_xlmr.ToxicCommentModel"
10
+ },
11
+ "bad_words_ids": null,
12
+ "begin_suppress_tokens": null,
13
+ "bos_token_id": null,
14
+ "chunk_size_feed_forward": 0,
15
+ "cross_attention_hidden_size": null,
16
+ "decoder_start_token_id": null,
17
+ "disable_lang_conditioning": false,
18
+ "diversity_penalty": 0.0,
19
+ "do_sample": false,
20
+ "dropout": 0.0,
21
+ "early_stopping": false,
22
+ "encoder": {
23
+ "attention_probs_dropout_prob": 0.1,
24
+ "bos_token_id": 0,
25
+ "eos_token_id": 2,
26
+ "hidden_act": "gelu",
27
+ "hidden_dropout_prob": 0.1,
28
+ "hidden_size": 1024,
29
+ "initializer_range": 0.02,
30
+ "intermediate_size": 4096,
31
+ "layer_norm_eps": 1e-05,
32
+ "max_position_embeddings": 514,
33
+ "model_type": "xlm-roberta",
34
+ "num_attention_heads": 16,
35
+ "num_hidden_layers": 24,
36
+ "pad_token_id": 1,
37
+ "position_embedding_type": "absolute",
38
+ "type_vocab_size": 1,
39
+ "use_cache": true,
40
+ "vocab_size": 250002
41
+ },
42
+ "encoder_no_repeat_ngram_size": 0,
43
+ "eos_token_id": null,
44
+ "exponential_decay_length_penalty": null,
45
+ "finetuning_task": null,
46
+ "forced_bos_token_id": null,
47
+ "forced_eos_token_id": null,
48
+ "hidden_size": 1024,
49
+ "id2label": {
50
+ "0": "toxic",
51
+ "1": "severe_toxic",
52
+ "2": "obscene",
53
+ "3": "threat",
54
+ "4": "insult",
55
+ "5": "identity_hate"
56
+ },
57
+ "is_decoder": false,
58
+ "is_encoder_decoder": false,
59
+ "label2id": {
60
+ "identity_hate": 5,
61
+ "insult": 4,
62
+ "obscene": 2,
63
+ "severe_toxic": 1,
64
+ "threat": 3,
65
+ "toxic": 0
66
+ },
67
+ "languages": {
68
+ "en": 0,
69
+ "es": 3,
70
+ "fr": 4,
71
+ "it": 5,
72
+ "pt": 6,
73
+ "ru": 1,
74
+ "tr": 2
75
+ },
76
+ "length_penalty": 1.0,
77
+ "max_length": 20,
78
+ "max_seq_length": 512,
79
+ "min_length": 0,
80
+ "model_type": "toxic_comment_xlmr",
81
+ "no_repeat_ngram_size": 0,
82
+ "num_attention_heads": 16,
83
+ "num_beam_groups": 1,
84
+ "num_beams": 1,
85
+ "num_return_sequences": 1,
86
+ "output_attentions": false,
87
+ "output_hidden_states": false,
88
+ "output_scores": false,
89
+ "pad_token_id": null,
90
+ "prefix": null,
91
+ "problem_type": "multi_label_classification",
92
+ "pruned_heads": {},
93
+ "remove_invalid_values": false,
94
+ "repetition_penalty": 1.0,
95
+ "return_dict": true,
96
+ "return_dict_in_generate": false,
97
+ "sep_token_id": null,
98
+ "suppress_tokens": null,
99
+ "task_specific_params": null,
100
+ "temperature": 1.0,
101
+ "tf_legacy_loss": false,
102
+ "tie_encoder_decoder": false,
103
+ "tie_word_embeddings": true,
104
+ "tokenizer_class": null,
105
+ "top_k": 50,
106
+ "top_p": 1.0,
107
+ "torch_dtype": null,
108
+ "torchscript": false,
109
+ "transformers_version": "4.48.2",
110
+ "typical_p": 1.0,
111
+ "use_bfloat16": false
112
+ }
images/f1_gains_by_class.png ADDED
images/pr_curves.png ADDED

Git LFS Details

  • SHA256: 38cf6e621b4c7349259afac19eb6e451016feafc07504cfa65487e05c0e5c4d2
  • Pointer size: 131 Bytes
  • Size of remote file: 142 kB
images/threat_probability_shift.png ADDED
inference_example.py ADDED
@@ -0,0 +1,162 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """Runnable example: batch toxicity prediction with tuned per-class thresholds.
3
+
4
+ python inference_example.py # pull from the Hub
5
+ python inference_example.py /path/to/local/repo # use a local checkout
6
+ python inference_example.py --device cuda
7
+
8
+ Only needs `transformers`, `torch` and `huggingface_hub`. Nothing from the
9
+ training repo is imported; `modeling_toxic_xlmr.py` ships alongside this file
10
+ and is loaded for you by `trust_remote_code=True`.
11
+
12
+ The three things people get wrong with this model:
13
+
14
+ 1. It needs `lang_ids`. Omit it and every row is scored as English (id 0),
15
+ silently and with no error.
16
+ 2. The six labels are independent, not one choice out of six. Any number of
17
+ them can fire on the same comment.
18
+ 3. 0.5 is the wrong cut. Use `thresholds.json`; the rare classes need cuts
19
+ near 0.37.
20
+ """
21
+
22
+ import argparse
23
+ import json
24
+ import os
25
+ import sys
26
+
27
+ import torch
28
+ from transformers import AutoModel, AutoTokenizer
29
+
30
+ REPO_ID = "Deeptanshuu/toxic-comment-multilingual-xlmr"
31
+
32
+ # Baked into the trained language embedding table. Do not renumber.
33
+ LANGUAGE_IDS = {"en": 0, "ru": 1, "tr": 2, "es": 3, "fr": 4, "it": 5, "pt": 6}
34
+
35
+ # Classifier output order. Index 1 is severe_toxic and index 2 is obscene,
36
+ # which is neither alphabetical nor the order most people would guess.
37
+ LABEL_NAMES = ["toxic", "severe_toxic", "obscene", "threat", "insult", "identity_hate"]
38
+
39
+
40
+ def load_thresholds(source: str) -> dict:
41
+ """Read thresholds.json from a local directory or from the Hub."""
42
+ if os.path.isdir(source):
43
+ path = os.path.join(source, "thresholds.json")
44
+ else:
45
+ from huggingface_hub import hf_hub_download
46
+
47
+ path = hf_hub_download(repo_id=source, filename="thresholds.json")
48
+ with open(path) as f:
49
+ return json.load(f)["thresholds"]
50
+
51
+
52
+ @torch.no_grad()
53
+ def predict(model, tokenizer, texts, langs, thresholds, batch_size=8, max_length=512):
54
+ """Score a list of texts and apply the per-class thresholds.
55
+
56
+ Args:
57
+ texts: list of strings.
58
+ langs: list of language codes, one per text, from LANGUAGE_IDS.
59
+ thresholds: {label_name: float}.
60
+
61
+ Returns:
62
+ list of dicts, one per input text.
63
+ """
64
+ if len(texts) != len(langs):
65
+ raise ValueError(f"{len(texts)} texts but {len(langs)} language codes")
66
+
67
+ unknown = sorted({l for l in langs if l not in LANGUAGE_IDS})
68
+ if unknown:
69
+ raise ValueError(
70
+ f"Unsupported language codes {unknown}. This model knows "
71
+ f"{sorted(LANGUAGE_IDS)}. Pick the closest supported language rather "
72
+ "than letting it default silently."
73
+ )
74
+
75
+ device = next(model.parameters()).device
76
+ cut = torch.tensor([thresholds[name] for name in LABEL_NAMES], device=device)
77
+
78
+ results = []
79
+ for start in range(0, len(texts), batch_size):
80
+ chunk = texts[start : start + batch_size]
81
+ chunk_langs = langs[start : start + batch_size]
82
+
83
+ enc = tokenizer(
84
+ chunk,
85
+ padding=True,
86
+ truncation=True,
87
+ max_length=max_length, # must be 512; training used 512
88
+ return_tensors="pt",
89
+ ).to(device)
90
+ lang_ids = torch.tensor(
91
+ [LANGUAGE_IDS[l] for l in chunk_langs], dtype=torch.long, device=device
92
+ )
93
+
94
+ out = model(
95
+ input_ids=enc["input_ids"],
96
+ attention_mask=enc["attention_mask"],
97
+ lang_ids=lang_ids,
98
+ )
99
+ probs = out["probabilities"]
100
+ fired = probs >= cut
101
+
102
+ for text, lang, prob_row, fired_row in zip(chunk, chunk_langs, probs, fired):
103
+ labels = [n for n, f in zip(LABEL_NAMES, fired_row.tolist()) if f]
104
+ results.append(
105
+ {
106
+ "text": text,
107
+ "language": lang,
108
+ "probabilities": {
109
+ n: round(float(p), 4) for n, p in zip(LABEL_NAMES, prob_row.tolist())
110
+ },
111
+ "labels": labels,
112
+ "any_toxic": bool(labels),
113
+ }
114
+ )
115
+ return results
116
+
117
+
118
+ def main() -> int:
119
+ parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
120
+ parser.add_argument(
121
+ "source",
122
+ nargs="?",
123
+ default=REPO_ID,
124
+ help="Hub repo id or a local directory holding config.json and pytorch_model.bin",
125
+ )
126
+ parser.add_argument("--device", default="cuda" if torch.cuda.is_available() else "cpu")
127
+ parser.add_argument("--batch-size", type=int, default=8)
128
+ args = parser.parse_args()
129
+
130
+ print(f"loading {args.source} on {args.device}", file=sys.stderr)
131
+ model = AutoModel.from_pretrained(args.source, trust_remote_code=True)
132
+ model.to(args.device).eval()
133
+ # The tokenizer is stock xlm-roberta-large, unmodified. It ships in this
134
+ # repo so the whole thing is self-contained, but the line below works
135
+ # equally well against "xlm-roberta-large".
136
+ tokenizer = AutoTokenizer.from_pretrained(args.source)
137
+ thresholds = load_thresholds(args.source)
138
+
139
+ samples = [
140
+ ("You are an absolute idiot and everyone here hates you.", "en"),
141
+ ("Thanks for catching that typo, much appreciated.", "en"),
142
+ ("Sei un cretino, vattene via da qui.", "it"),
143
+ ("Merci beaucoup pour ton aide, c'etait tres clair.", "fr"),
144
+ ("Defol git buradan, seni asagilik herif.", "tr"),
145
+ ]
146
+ texts = [t for t, _ in samples]
147
+ langs = [l for _, l in samples]
148
+
149
+ for row in predict(model, tokenizer, texts, langs, thresholds, batch_size=args.batch_size):
150
+ verdict = ", ".join(row["labels"]) if row["labels"] else "clean"
151
+ print(f"\n[{row['language']}] {row['text']}")
152
+ print(f" fires: {verdict}")
153
+ print(
154
+ " probs: "
155
+ + " ".join(f"{n}={row['probabilities'][n]:.3f}" for n in LABEL_NAMES)
156
+ )
157
+ print("\nthresholds used: " + json.dumps(thresholds))
158
+ return 0
159
+
160
+
161
+ if __name__ == "__main__":
162
+ raise SystemExit(main())
metrics.json ADDED
@@ -0,0 +1,243 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_status": "FINAL. Numbers below are measured on the final best_model checkpoint (epoch 5 of a 6-epoch run that completed all 6 epochs), evaluation run evaluation_results/eval_20260830_072515. Test-split metrics use per-class thresholds tuned on validation and applied unchanged; see the protocol block below.",
3
+ "label_order": [
4
+ "toxic",
5
+ "severe_toxic",
6
+ "obscene",
7
+ "threat",
8
+ "insult",
9
+ "identity_hate"
10
+ ],
11
+ "protocol": {
12
+ "thresholds_tuned_on": "dataset/split/val.csv",
13
+ "metrics_reported_on": "dataset/split/test.csv",
14
+ "why_this_matters": "Thresholds are free parameters. Choosing them on the same split you report on lets the model fit noise in that split and inflates every threshold-dependent metric (F1, precision, recall, exact match). AUC is threshold-free and is not affected.",
15
+ "max_length": 512,
16
+ "test_split_size": 35658,
17
+ "known_leakage": "Near-duplicate leakage measured against train (char 3-4-gram TF-IDF, cosine >= 0.9): 3.8% of English validation rows, 0.6% of Russian. Exact duplicate overlap is 0. Held-out numbers are therefore mildly optimistic."
18
+ },
19
+ "this_version": {
20
+ "description": "Encoder fine-tuned. 307.1M of 564.7M parameters (54.4%) receive gradient; the word/position embedding module is frozen deliberately.",
21
+ "checkpoint": "weights/toxic_classifier_xlmr_v2/best_model (epoch 5 of a 6-epoch run that completed all 6 epochs)",
22
+ "epochs_completed": 6,
23
+ "selected_epoch": 5,
24
+ "validation": {
25
+ "auc_macro": 0.9868
26
+ },
27
+ "test": {
28
+ "auc_macro": 0.9852,
29
+ "auc_weighted": 0.9890,
30
+ "f1_macro_at_0.5": 0.8821,
31
+ "f1_macro_at_tuned": 0.8814,
32
+ "f1_weighted_at_tuned": 0.9332,
33
+ "precision_macro_at_tuned": 0.8850,
34
+ "recall_macro_at_tuned": 0.8798,
35
+ "exact_match_at_tuned": 0.8772,
36
+ "hamming_loss_at_tuned": 0.0257,
37
+ "per_class": {
38
+ "toxic": {
39
+ "auc": 0.9921,
40
+ "threshold": 0.4724,
41
+ "precision": 0.9534,
42
+ "recall": 0.9750,
43
+ "f1": 0.9641,
44
+ "support": 17697
45
+ },
46
+ "severe_toxic": {
47
+ "auc": 0.9863,
48
+ "threshold": 0.4724,
49
+ "precision": 0.7139,
50
+ "recall": 0.8010,
51
+ "f1": 0.7549,
52
+ "support": 1648
53
+ },
54
+ "obscene": {
55
+ "auc": 0.9899,
56
+ "threshold": 0.5276,
57
+ "precision": 0.9419,
58
+ "recall": 0.9338,
59
+ "f1": 0.9378,
60
+ "support": 8626
61
+ },
62
+ "threat": {
63
+ "auc": 0.9755,
64
+ "threshold": 0.5276,
65
+ "precision": 0.8846,
66
+ "recall": 0.8003,
67
+ "f1": 0.8403,
68
+ "support": 766
69
+ },
70
+ "insult": {
71
+ "auc": 0.9855,
72
+ "threshold": 0.5643,
73
+ "precision": 0.9204,
74
+ "recall": 0.9266,
75
+ "f1": 0.9235,
76
+ "support": 10199
77
+ },
78
+ "identity_hate": {
79
+ "auc": 0.9818,
80
+ "threshold": 0.5643,
81
+ "precision": 0.8959,
82
+ "recall": 0.8419,
83
+ "f1": 0.8680,
84
+ "support": 1891
85
+ }
86
+ },
87
+ "per_language": {
88
+ "en": {
89
+ "lang_id": 0,
90
+ "auc_macro": 0.9902,
91
+ "f1_macro": 0.9025,
92
+ "sample_count": 4638
93
+ },
94
+ "ru": {
95
+ "lang_id": 1,
96
+ "auc_macro": 0.9790,
97
+ "f1_macro": 0.8545,
98
+ "sample_count": 5193
99
+ },
100
+ "tr": {
101
+ "lang_id": 2,
102
+ "auc_macro": 0.9726,
103
+ "f1_macro": 0.8128,
104
+ "sample_count": 5163
105
+ },
106
+ "es": {
107
+ "lang_id": 3,
108
+ "auc_macro": 0.9882,
109
+ "f1_macro": 0.9023,
110
+ "sample_count": 5168
111
+ },
112
+ "fr": {
113
+ "lang_id": 4,
114
+ "auc_macro": 0.9877,
115
+ "f1_macro": 0.9075,
116
+ "sample_count": 5158
117
+ },
118
+ "it": {
119
+ "lang_id": 5,
120
+ "auc_macro": 0.9893,
121
+ "f1_macro": 0.8985,
122
+ "sample_count": 5146
123
+ },
124
+ "pt": {
125
+ "lang_id": 6,
126
+ "auc_macro": 0.9832,
127
+ "f1_macro": 0.8891,
128
+ "sample_count": 5192
129
+ }
130
+ }
131
+ }
132
+ },
133
+ "previous_version_april_2025": {
134
+ "description": "The model published in April 2025. Its XLM-RoBERTa encoder was never fine-tuned: 4.8M of 564.7M parameters (0.8%) received gradient and all 381 encoder tensors had grad=None. These numbers describe a linear probe on frozen XLM-R features, not a fine-tuned model. They are recorded here so the two versions can be compared honestly. They are NOT this model's numbers.",
135
+ "checkpoint": "weights/toxic_classifier_xlm-roberta-large/latest (epoch 2 of a 6-epoch run that crashed at epoch 4)",
136
+ "evaluation_run": "evaluation_results/eval_20260830_011818",
137
+ "test": {
138
+ "auc_macro": 0.9147,
139
+ "auc_weighted": 0.9351,
140
+ "f1_macro_at_0.5": 0.5284,
141
+ "f1_macro_at_tuned": 0.6036,
142
+ "f1_weighted_at_tuned": 0.7732,
143
+ "precision_macro_at_tuned": 0.5651,
144
+ "recall_macro_at_tuned": 0.6526,
145
+ "exact_match_at_tuned": 0.6194,
146
+ "hamming_loss_at_tuned": 0.0941,
147
+ "per_class": {
148
+ "toxic": {
149
+ "auc": 0.9666,
150
+ "threshold": 0.4959,
151
+ "precision": 0.9039,
152
+ "recall": 0.9037,
153
+ "f1": 0.9038,
154
+ "support": 17697
155
+ },
156
+ "severe_toxic": {
157
+ "auc": 0.8988,
158
+ "threshold": 0.3816,
159
+ "precision": 0.337,
160
+ "recall": 0.486,
161
+ "f1": 0.398,
162
+ "support": 1648
163
+ },
164
+ "obscene": {
165
+ "auc": 0.9278,
166
+ "threshold": 0.4633,
167
+ "precision": 0.7158,
168
+ "recall": 0.7643,
169
+ "f1": 0.7392,
170
+ "support": 8626
171
+ },
172
+ "threat": {
173
+ "auc": 0.9051,
174
+ "threshold": 0.3653,
175
+ "precision": 0.3867,
176
+ "recall": 0.4569,
177
+ "f1": 0.4189,
178
+ "support": 766
179
+ },
180
+ "insult": {
181
+ "auc": 0.9035,
182
+ "threshold": 0.4633,
183
+ "precision": 0.6585,
184
+ "recall": 0.8061,
185
+ "f1": 0.7248,
186
+ "support": 10199
187
+ },
188
+ "identity_hate": {
189
+ "auc": 0.8866,
190
+ "threshold": 0.3653,
191
+ "precision": 0.3889,
192
+ "recall": 0.4987,
193
+ "f1": 0.437,
194
+ "support": 1891
195
+ }
196
+ },
197
+ "per_language": {
198
+ "en": {
199
+ "lang_id": 0,
200
+ "auc_macro": 0.9463,
201
+ "f1_macro": 0.6508,
202
+ "sample_count": 4638
203
+ },
204
+ "fr": {
205
+ "lang_id": 4,
206
+ "auc_macro": 0.9157,
207
+ "f1_macro": 0.6035,
208
+ "sample_count": 5158
209
+ },
210
+ "es": {
211
+ "lang_id": 3,
212
+ "auc_macro": 0.9152,
213
+ "f1_macro": 0.5978,
214
+ "sample_count": 5168
215
+ },
216
+ "it": {
217
+ "lang_id": 5,
218
+ "auc_macro": 0.9139,
219
+ "f1_macro": 0.613,
220
+ "sample_count": 5146
221
+ },
222
+ "pt": {
223
+ "lang_id": 6,
224
+ "auc_macro": 0.9088,
225
+ "f1_macro": 0.5971,
226
+ "sample_count": 5192
227
+ },
228
+ "ru": {
229
+ "lang_id": 1,
230
+ "auc_macro": 0.9065,
231
+ "f1_macro": 0.595,
232
+ "sample_count": 5193
233
+ },
234
+ "tr": {
235
+ "lang_id": 2,
236
+ "auc_macro": 0.8944,
237
+ "f1_macro": 0.5627,
238
+ "sample_count": 5163
239
+ }
240
+ }
241
+ }
242
+ }
243
+ }
modeling_toxic_xlmr.py ADDED
@@ -0,0 +1,368 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Model definition for Deeptanshuu/toxic-comment-multilingual-xlmr.
2
+
3
+ This is a custom architecture, not a stock Hugging Face model. It is
4
+ XLM-RoBERTa-large followed by one extra attention block whose scores carry a
5
+ per-language bias, then a small classification head with six independent
6
+ sigmoid outputs.
7
+
8
+ Two ways to load it, both supported:
9
+
10
+ 1. Auto classes with remote code (the file you are reading is fetched from the
11
+ repo and executed, which is why `trust_remote_code=True` is required):
12
+
13
+ >>> from transformers import AutoModel, AutoTokenizer
14
+ >>> model = AutoModel.from_pretrained(REPO, trust_remote_code=True)
15
+ >>> tok = AutoTokenizer.from_pretrained(REPO)
16
+
17
+ 2. The explicit helper at the bottom of this file, which needs no auto-class
18
+ machinery and no remote-code execution:
19
+
20
+ >>> from modeling_toxic_xlmr import load_model
21
+ >>> model, tokenizer = load_model("/path/to/local/checkout")
22
+
23
+ Note the `>>> ` prefixes above. transformers scans this file line by line for
24
+ import statements when it loads it as remote code, and an unprefixed
25
+ `from modeling_toxic_xlmr import ...` in a docstring is read as a dependency on
26
+ a PyPI package of that name, which then fails to install.
27
+
28
+ The forward pass takes a third input beyond the usual pair:
29
+
30
+ model(input_ids=..., attention_mask=..., lang_ids=...)
31
+
32
+ `lang_ids` is a LongTensor of shape [batch] holding one id per sequence, using
33
+ LANGUAGE_IDS below. If you omit it the model falls back to id 0 (English) for
34
+ every row and warns once. That is a silent quality loss on non-English text,
35
+ not an error.
36
+ """
37
+
38
+ from dataclasses import dataclass
39
+ from typing import Optional
40
+ import json
41
+ import logging
42
+ import os
43
+ import warnings
44
+
45
+ import torch
46
+ import torch.nn as nn
47
+ import torch.nn.functional as F
48
+ from transformers import PretrainedConfig, PreTrainedModel, XLMRobertaConfig, XLMRobertaModel
49
+ from transformers.utils import ModelOutput
50
+
51
+ logger = logging.getLogger(__name__)
52
+
53
+ # Language id mapping. These integers are baked into the trained lang_embed
54
+ # table; changing the mapping silently changes which language the model thinks
55
+ # it is looking at.
56
+ LANGUAGE_IDS = {
57
+ "en": 0, # English
58
+ "ru": 1, # Russian
59
+ "tr": 2, # Turkish
60
+ "es": 3, # Spanish
61
+ "fr": 4, # French
62
+ "it": 5, # Italian
63
+ "pt": 6, # Portuguese
64
+ }
65
+ ID_TO_LANGUAGE = {v: k for k, v in LANGUAGE_IDS.items()}
66
+
67
+ # Output order of the classifier. Do not reorder: index 1 is severe_toxic and
68
+ # index 2 is obscene, which is not the alphabetical or the intuitive order.
69
+ LABEL_NAMES = [
70
+ "toxic",
71
+ "severe_toxic",
72
+ "obscene",
73
+ "threat",
74
+ "insult",
75
+ "identity_hate",
76
+ ]
77
+
78
+ _WARNED_MISSING_LANG_IDS = False
79
+
80
+
81
+ class ToxicCommentConfig(PretrainedConfig):
82
+ """Configuration for :class:`ToxicCommentModel`.
83
+
84
+ The XLM-RoBERTa encoder configuration is nested under ``encoder`` as a plain
85
+ dict so the whole architecture is described by one config.json and the
86
+ encoder can be rebuilt without a network fetch.
87
+ """
88
+
89
+ model_type = "toxic_comment_xlmr"
90
+
91
+ def __init__(
92
+ self,
93
+ encoder=None,
94
+ hidden_size: int = 1024,
95
+ num_attention_heads: int = 16,
96
+ dropout: float = 0.0,
97
+ disable_lang_conditioning: bool = False,
98
+ languages=None,
99
+ max_seq_length: int = 512,
100
+ **kwargs,
101
+ ):
102
+ # Nested encoder config. Default is whatever XLMRobertaConfig defaults
103
+ # to; the shipped config.json carries the real xlm-roberta-large values.
104
+ self.encoder = encoder if encoder is not None else XLMRobertaConfig().to_dict()
105
+ self.hidden_size = hidden_size
106
+ self.num_attention_heads = num_attention_heads
107
+ self.dropout = dropout
108
+ self.disable_lang_conditioning = disable_lang_conditioning
109
+ self.languages = dict(languages) if languages is not None else dict(LANGUAGE_IDS)
110
+ # Training-time tokenizer truncation length. Named max_seq_length and not
111
+ # max_length because PretrainedConfig already owns max_length as a text
112
+ # generation parameter and would overwrite it with 20.
113
+ self.max_seq_length = max_seq_length
114
+
115
+ kwargs.setdefault("problem_type", "multi_label_classification")
116
+ kwargs.setdefault("id2label", {i: name for i, name in enumerate(LABEL_NAMES)})
117
+ kwargs.setdefault("label2id", {name: i for i, name in enumerate(LABEL_NAMES)})
118
+ super().__init__(**kwargs)
119
+
120
+
121
+ @dataclass
122
+ class ToxicCommentOutput(ModelOutput):
123
+ """Output of :class:`ToxicCommentModel`.
124
+
125
+ Supports both ``out.logits`` and ``out["probabilities"]`` access.
126
+ """
127
+
128
+ loss: Optional[torch.FloatTensor] = None
129
+ logits: Optional[torch.FloatTensor] = None
130
+ probabilities: Optional[torch.FloatTensor] = None
131
+
132
+
133
+ class ToxicCommentModel(PreTrainedModel):
134
+ """XLM-RoBERTa-large + one language-conditioned attention block + MLP head."""
135
+
136
+ config_class = ToxicCommentConfig
137
+ base_model_prefix = "base_model"
138
+ supports_gradient_checkpointing = True
139
+ _no_split_modules = ["XLMRobertaLayer"]
140
+
141
+ def __init__(self, config: ToxicCommentConfig):
142
+ super().__init__(config)
143
+
144
+ encoder_config = XLMRobertaConfig.from_dict(config.encoder)
145
+ # Constructed from config rather than from_pretrained: every weight is
146
+ # about to be overwritten by the checkpoint, so downloading the stock
147
+ # xlm-roberta-large weights first would be 2.2 GB of wasted transfer.
148
+ self.base_model = XLMRobertaModel(encoder_config)
149
+
150
+ hidden_size = config.hidden_size
151
+ num_heads = config.num_attention_heads
152
+ num_labels = len(LABEL_NAMES)
153
+
154
+ self.original_hidden_size = encoder_config.hidden_size
155
+ self.needs_projection = hidden_size != self.original_hidden_size
156
+ if self.needs_projection:
157
+ self.dim_projection = nn.Sequential(
158
+ nn.Linear(self.original_hidden_size, hidden_size),
159
+ nn.LayerNorm(hidden_size),
160
+ nn.GELU(),
161
+ )
162
+ self.working_hidden_size = hidden_size if self.needs_projection else self.original_hidden_size
163
+
164
+ num_languages = len(config.languages)
165
+ self.lang_embed = nn.Embedding(num_languages, 64)
166
+ self.register_buffer("valid_lang_ids", torch.arange(num_languages))
167
+
168
+ self.lang_proj = nn.Sequential(
169
+ nn.Linear(64, hidden_size),
170
+ nn.LayerNorm(hidden_size),
171
+ nn.Tanh(), # bounded, so the bias cannot saturate the softmax
172
+ )
173
+
174
+ self.head_dim = hidden_size // num_heads
175
+ self.num_attention_heads = num_heads
176
+ self.scale = self.head_dim ** -0.5
177
+
178
+ self.q_proj = nn.Linear(hidden_size, hidden_size)
179
+ self.k_proj = nn.Linear(hidden_size, hidden_size)
180
+ self.v_proj = nn.Linear(hidden_size, hidden_size)
181
+ self.dropout = nn.Dropout(config.dropout)
182
+
183
+ self.post_attention = nn.Sequential(
184
+ nn.Linear(hidden_size, hidden_size),
185
+ nn.LayerNorm(hidden_size),
186
+ nn.GELU(),
187
+ )
188
+
189
+ self.classifier = nn.Sequential(
190
+ nn.Linear(hidden_size, 512),
191
+ nn.LayerNorm(512),
192
+ nn.GELU(),
193
+ nn.Linear(512, num_labels),
194
+ )
195
+
196
+ self.disable_lang_conditioning = bool(config.disable_lang_conditioning)
197
+
198
+ self.post_init()
199
+
200
+ # PreTrainedModel exposes `base_model` as a property that resolves
201
+ # `base_model_prefix`. Our encoder submodule is literally named
202
+ # `base_model` (the checkpoint keys are `base_model.*`), so the inherited
203
+ # property would call getattr(self, "base_model") on itself and recurse.
204
+ # Read it out of the module registry instead.
205
+ @property
206
+ def base_model(self) -> nn.Module:
207
+ module = self._modules.get("base_model")
208
+ return self if module is None else module
209
+
210
+ def _init_weights(self, module):
211
+ """Initialize the custom head.
212
+
213
+ Only reached when a model is built from scratch; `from_pretrained` runs
214
+ under `no_init_weights` and skips this entirely.
215
+ """
216
+ if isinstance(module, nn.Linear):
217
+ if module in (self.q_proj, self.k_proj, self.v_proj):
218
+ nn.init.normal_(module.weight, std=0.02)
219
+ else:
220
+ nn.init.xavier_uniform_(module.weight)
221
+ if module.bias is not None:
222
+ nn.init.zeros_(module.bias)
223
+ elif isinstance(module, nn.LayerNorm):
224
+ nn.init.ones_(module.weight)
225
+ nn.init.zeros_(module.bias)
226
+ elif isinstance(module, nn.Embedding):
227
+ nn.init.normal_(module.weight, std=0.02)
228
+
229
+ def validate_lang_ids(self, lang_ids: torch.Tensor) -> torch.Tensor:
230
+ """Coerce to long, then clamp anything outside 0..6 into range."""
231
+ if not isinstance(lang_ids, torch.Tensor):
232
+ lang_ids = torch.tensor(lang_ids, dtype=torch.long, device=self.valid_lang_ids.device)
233
+ elif lang_ids.dtype != torch.long:
234
+ lang_ids = lang_ids.long()
235
+
236
+ num_invalid = int((~torch.isin(lang_ids, self.valid_lang_ids)).sum().item())
237
+ if num_invalid:
238
+ logger.warning(
239
+ "Found %d language ids outside 0-%d; clamping into range.",
240
+ num_invalid,
241
+ len(self.config.languages) - 1,
242
+ )
243
+ lang_ids = torch.clamp(lang_ids, min=0, max=len(self.config.languages) - 1)
244
+ return lang_ids
245
+
246
+ def forward(
247
+ self,
248
+ input_ids: torch.Tensor,
249
+ attention_mask: Optional[torch.Tensor] = None,
250
+ lang_ids: Optional[torch.Tensor] = None,
251
+ labels: Optional[torch.Tensor] = None,
252
+ return_dict: Optional[bool] = None,
253
+ ):
254
+ device = input_ids.device
255
+ batch_size = input_ids.size(0)
256
+
257
+ if lang_ids is None:
258
+ global _WARNED_MISSING_LANG_IDS
259
+ if not _WARNED_MISSING_LANG_IDS:
260
+ _WARNED_MISSING_LANG_IDS = True
261
+ warnings.warn(
262
+ "lang_ids was not supplied, so every sequence is being treated as "
263
+ "English (id 0). Pass lang_ids for non-English text; see "
264
+ "LANGUAGE_IDS in modeling_toxic_xlmr.py.",
265
+ stacklevel=2,
266
+ )
267
+ lang_ids = torch.zeros(batch_size, dtype=torch.long, device=device)
268
+ lang_ids = self.validate_lang_ids(lang_ids.to(device))
269
+
270
+ hidden_states = self.base_model(
271
+ input_ids=input_ids,
272
+ attention_mask=attention_mask,
273
+ ).last_hidden_state
274
+
275
+ if self.needs_projection:
276
+ hidden_states = self.dim_projection(hidden_states)
277
+
278
+ batch_size, seq_len, hidden_size = hidden_states.shape
279
+ num_heads = self.num_attention_heads
280
+ head_dim = hidden_size // num_heads
281
+
282
+ q = self.q_proj(hidden_states).view(batch_size, seq_len, num_heads, head_dim).transpose(1, 2)
283
+ k = self.k_proj(hidden_states).view(batch_size, seq_len, num_heads, head_dim).transpose(1, 2)
284
+ v = self.v_proj(hidden_states).view(batch_size, seq_len, num_heads, head_dim).transpose(1, 2)
285
+
286
+ attn_scores = torch.matmul(q, k.transpose(-2, -1)) * self.scale
287
+
288
+ # Language bias. The language vector is added to the QUERIES, which
289
+ # contributes lang_vec . k_j to the score for key j. That varies along
290
+ # the key axis, so softmax does not cancel it. Adding a per-query
291
+ # constant instead (the shape this model shipped with in April 2025)
292
+ # cancels exactly and makes lang_ids a no-op.
293
+ if not self.disable_lang_conditioning:
294
+ lang_emb = self.lang_embed(lang_ids)
295
+ lang_vec = self.lang_proj(lang_emb).view(batch_size, num_heads, 1, head_dim)
296
+ attn_scores = attn_scores + torch.matmul(lang_vec, k.transpose(-2, -1)) * self.scale
297
+
298
+ # Mask after the bias, so the language term can never leak attention
299
+ # onto padding positions.
300
+ if attention_mask is not None:
301
+ attn_scores = attn_scores.masked_fill(
302
+ ~attention_mask.bool().unsqueeze(1).unsqueeze(2),
303
+ float("-inf"),
304
+ )
305
+
306
+ attn_weights = self.dropout(F.softmax(attn_scores, dim=-1))
307
+ attention_output = torch.matmul(attn_weights, v)
308
+ attention_output = attention_output.transpose(1, 2).contiguous().view(
309
+ batch_size, seq_len, hidden_size
310
+ )
311
+
312
+ output = self.post_attention(attention_output)
313
+ logits = self.classifier(output[:, 0]) # [CLS] position
314
+ probabilities = torch.sigmoid(logits)
315
+
316
+ loss = None
317
+ if labels is not None:
318
+ loss = F.binary_cross_entropy_with_logits(logits, labels.float())
319
+
320
+ if return_dict is False:
321
+ return tuple(x for x in (loss, logits, probabilities) if x is not None)
322
+ return ToxicCommentOutput(loss=loss, logits=logits, probabilities=probabilities)
323
+
324
+
325
+ # --------------------------------------------------------------------------
326
+ # Explicit loader: no auto classes, no trust_remote_code, no magic.
327
+ # --------------------------------------------------------------------------
328
+
329
+ def load_model(model_dir: str, device: str = "cpu", tokenizer_name: str = None):
330
+ """Load the model and its tokenizer from a local directory.
331
+
332
+ Args:
333
+ model_dir: directory holding config.json and pytorch_model.bin.
334
+ device: torch device string.
335
+ tokenizer_name: where to load the tokenizer from. Defaults to
336
+ `model_dir` if it has tokenizer files, otherwise "xlm-roberta-large"
337
+ (the tokenizer is stock and unmodified).
338
+
339
+ Returns:
340
+ (model, tokenizer), model in eval mode on `device`.
341
+ """
342
+ from transformers import AutoTokenizer
343
+
344
+ with open(os.path.join(model_dir, "config.json")) as f:
345
+ config = ToxicCommentConfig.from_dict(json.load(f))
346
+
347
+ model = ToxicCommentModel(config)
348
+
349
+ weights = os.path.join(model_dir, "pytorch_model.bin")
350
+ if not os.path.exists(weights):
351
+ raise FileNotFoundError(f"No pytorch_model.bin in {model_dir}")
352
+ state_dict = torch.load(weights, map_location="cpu", weights_only=True)
353
+ missing, unexpected = model.load_state_dict(state_dict, strict=True)
354
+ if missing or unexpected: # strict=True already raises, this is belt and braces
355
+ raise RuntimeError(f"state_dict mismatch: missing={missing} unexpected={unexpected}")
356
+
357
+ model.to(device)
358
+ model.eval()
359
+
360
+ if tokenizer_name is None:
361
+ has_local = any(
362
+ os.path.exists(os.path.join(model_dir, n))
363
+ for n in ("tokenizer.json", "sentencepiece.bpe.model")
364
+ )
365
+ tokenizer_name = model_dir if has_local else "xlm-roberta-large"
366
+ tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)
367
+
368
+ return model, tokenizer
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:54b4fd10ad5e369324fe9764061cff6fc5d0d97218c5929407c3cfcbd366bb0e
3
+ size 2258929598
sentencepiece.bpe.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cfc8146abe2a0488e9e2a0c56de7952f7c11ab059eca145a0a727afce0db2865
3
+ size 5069051
special_tokens_map.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": "<s>",
3
+ "cls_token": "<s>",
4
+ "eos_token": "</s>",
5
+ "mask_token": {
6
+ "content": "<mask>",
7
+ "lstrip": true,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false
11
+ },
12
+ "pad_token": "<pad>",
13
+ "sep_token": "</s>",
14
+ "unk_token": "<unk>"
15
+ }
thresholds.json ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_readme": [
3
+ "Per-class decision thresholds. A probability at or above the threshold for a",
4
+ "class means that class fires. The six classes are independent: any number of",
5
+ "them can fire on the same comment.",
6
+ "",
7
+ "Do not use 0.5. The rare classes (severe_toxic, threat, identity_hate) are",
8
+ "well ordered but badly calibrated, so a 0.5 cut throws away most of their",
9
+ "recall. Every threshold below sits under 0.5 for that reason.",
10
+ "",
11
+ "Thresholds were chosen to maximise per-class F1 on the validation split and",
12
+ "are then applied unchanged to the held-out test split. They are a single",
13
+ "global set: there is no per-language block here, deliberately. See",
14
+ "_provenance.per_language_block_omitted."
15
+ ],
16
+ "_provenance": {
17
+ "status": "FINAL. These thresholds are tuned on validation using the final best_model checkpoint (epoch 5 of a 6-epoch run that completed all 6 epochs) and applied unchanged to the test split. They match the shipped weights.",
18
+ "tuned_on": "dataset/split/val.csv",
19
+ "applied_to": "dataset/split/test.csv",
20
+ "source_run": "evaluation_results/eval_20260830_072515",
21
+ "source_checkpoint": "weights/toxic_classifier_xlmr_v2/best_model (epoch 5 of 6, encoder fine-tuned)",
22
+ "search": "direct sweep over candidate thresholds in [0.05, 0.95], per class independently, maximising F1 on validation",
23
+ "per_language_block_omitted": "The evaluation script also emits a per-language threshold block. It is not shipped: it comes from a code path with a known bug and nothing in the serving path ever reads it. An earlier run of that code path reported English severe_toxic at F1 0.597 when the maximum achievable at any threshold is 0.442."
24
+ },
25
+ "label_order": [
26
+ "toxic",
27
+ "severe_toxic",
28
+ "obscene",
29
+ "threat",
30
+ "insult",
31
+ "identity_hate"
32
+ ],
33
+ "thresholds": {
34
+ "toxic": 0.4724489795918367,
35
+ "severe_toxic": 0.4724489795918367,
36
+ "obscene": 0.5275510204081633,
37
+ "threat": 0.5275510204081633,
38
+ "insult": 0.5642857142857143,
39
+ "identity_hate": 0.5642857142857143
40
+ },
41
+ "validation_f1_at_threshold": {
42
+ "_note": "F1 achieved at the threshold above, on the validation split, by THIS (shipped) checkpoint. This is the value the threshold search maximised, not a test-split number.",
43
+ "toxic": 0.964104847236908,
44
+ "severe_toxic": 0.7429062768701634,
45
+ "obscene": 0.937078390323331,
46
+ "threat": 0.8424657534246576,
47
+ "insult": 0.9223998826463253,
48
+ "identity_hate": 0.8753432180120813
49
+ },
50
+ "validation_positive_support": {
51
+ "_note": "Positive examples per class in the 35,658-row validation split used for tuning.",
52
+ "toxic": 17697,
53
+ "severe_toxic": 1655,
54
+ "obscene": 8626,
55
+ "threat": 760,
56
+ "insult": 10199,
57
+ "identity_hate": 1878,
58
+ "total_samples": 35658
59
+ }
60
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a56def25aa40facc030ea8b0b87f3688e4b3c39eb8b45d5702b3a1300fe2a20
3
+ size 17082734
tokenizer_config.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "<s>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "<pad>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "</s>",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "<unk>",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "250001": {
36
+ "content": "<mask>",
37
+ "lstrip": true,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "bos_token": "<s>",
45
+ "clean_up_tokenization_spaces": false,
46
+ "cls_token": "<s>",
47
+ "eos_token": "</s>",
48
+ "extra_special_tokens": {},
49
+ "mask_token": "<mask>",
50
+ "model_max_length": 512,
51
+ "pad_token": "<pad>",
52
+ "sep_token": "</s>",
53
+ "tokenizer_class": "XLMRobertaTokenizer",
54
+ "unk_token": "<unk>"
55
+ }
training_config.json ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "xlm-roberta-large",
3
+ "max_length": 512,
4
+ "hidden_size": 1024,
5
+ "num_attention_heads": 16,
6
+ "model_dropout": 0.0,
7
+ "freeze_layers": 0,
8
+ "freeze_embeddings": true,
9
+ "cache_dir": "cached_dataset",
10
+ "checkpoint_dir": "weights/toxic_classifier_xlmr_v2",
11
+ "label_columns": [
12
+ "toxic",
13
+ "severe_toxic",
14
+ "obscene",
15
+ "threat",
16
+ "insult",
17
+ "identity_hate"
18
+ ],
19
+ "batch_size": 64,
20
+ "grad_accum_steps": 2,
21
+ "epochs": 6,
22
+ "lr": 2e-05,
23
+ "num_cycles": 2,
24
+ "weight_decay": 2e-07,
25
+ "max_grad_norm": 1.0,
26
+ "warmup_ratio": 0.1,
27
+ "label_smoothing": 0.01,
28
+ "min_lr_ratio": 0.01,
29
+ "activation_checkpointing": true,
30
+ "mixed_precision": "fp16",
31
+ "_num_workers": 8,
32
+ "gc_frequency": 500,
33
+ "tensor_float_32": false,
34
+ "train_file": "dataset/split/train.csv",
35
+ "val_file": "dataset/split/val.csv",
36
+ "test_file": "dataset/split/test.csv",
37
+ "dynamic_padding": true,
38
+ "pad_to_multiple_of": 8,
39
+ "use_class_weights": true,
40
+ "use_warmup": true,
41
+ "data_parallel": true,
42
+ "eval_batch_size": 256,
43
+ "eval_every_epoch": true,
44
+ "lang_weights": "weights/language_class_weights.json",
45
+ "disable_lang_conditioning": false,
46
+ "class_weights_file": "weights/language_class_weights.json",
47
+ "device": "cuda"
48
+ }