--- license: apache-2.0 language: - hi - bn - mr - ta - te - kn - ml - gu - pa - or - as - ur - en library_name: transformers pipeline_tag: text-classification base_model: jhu-clsp/mmBERT-small tags: - profanity - offensive-language - hate-speech - content-moderation - indic - multilingual - code-mixed - multi-label --- # cbx-indic-profanity-detector-36M An offensive-content classifier for 12 Indian languages and English: Hindi, Bengali, Marathi, Tamil, Telugu, Kannada, Malayalam, Gujarati, Punjabi, Odia, Assamese, Urdu and English, written in native script, in Latin letters (romanized) or code-mixed. Give it a comment, message or short passage and it returns four independent labels, `profanity`, `insult`, `hate` and `sexual`, plus an overall `offensive` flag. It has 36.5M parameters: the 58M with every second layer removed (11 of 22 layers), distilled from a fine-tuned mmBERT-base. Other sizes: 58M, 17M. On a CPU (8 threads, fp32) one short comment takes about 8 ms and a full 256-token chunk about 27 ms. **Licence: Apache-2.0.** Read [Intended use and disclaimer](#intended-use-and-disclaimer) before using it. In short: it is an aid for people who review content, not a judge of content or of people. ## Labels A text can have any number of labels; none means clean. Precision, recall, F1 and PR-AUC are on the gold test set (below). | Label | Yes when the text … | Precision | Recall | F1 | PR-AUC | |---|---|---:|---:|---:|---:| | `profanity` | uses a swear word, obscene word or vulgar expression, aimed at someone or not (judged by the words) | 0.96 | 0.55 | 0.70 | 0.90 | | `insult` | attacks, demeans or abuses a specific person or group (name-calling, degrading comparisons, threats) | 0.87 | 0.58 | 0.70 | 0.83 | | `hate` | attacks a group because of religion, caste, ethnicity, nationality, region, language, gender, sexual orientation or disability | 0.54 | 0.48 | 0.51 | 0.48 | | `sexual` | sexually explicit text or a sexual proposition, including porn / escort / sex-chat advertising (judged by what it describes; curse words with a sexual literal meaning are `profanity`) | 0.96 | 0.74 | 0.83 | 0.94 | | `offensive` | any of the four (a separate head; the most reliable single flag) | 0.87 | 0.91 | 0.89 | 0.96 | The four labels are precise but miss many cases; `offensive` finds about 91% of offensive texts. Use `offensive` to decide whether a text needs attention and the four labels to say why. News about crime, sex education, medicine, criticism of a policy, and a word that is only mentioned or discussed are meant to be clean. ## Quick start ```bash pip install -r requirements.txt python example.py ``` ```python from cbx_profanity import ProfanityClassifier clf = ProfanityClassifier(".") # folder with the model files; CPU or GPU r = clf.classify("saala harami, teri aukaat kya hai") r["offensive"], r["labels"] # flag and the labels at or above their thresholds r["scores"] # score per label and for `offensive` clf.classify(["first comment", "second comment"]) # a list gives a list clf.classify_document(long_text) # 256-token chunks, highest score per label ``` `python example.py` prints (texts invented; the errors are the model's own, see Limitations): ``` clean - Thanks for sharing the recipe, my mother makes it the same way every Diwali. clean - शाम को बारिश हो सकती है, छाता साथ रखना। clean - நாளைக்கு கூட்டம் பத்து மணிக்கு தொடங்கும். OFFENSIVE profanity bc aaj traffic bahut kharab hai, ek ghanta phas gaya OFFENSIVE profanity,insult,hate saala harami, teri aukaat kya hai OFFENSIVE - What a stupid useless idiot, nobody asked for your opinion. ``` Thresholds (picked on validation for F1): profanity 0.85, insult 0.51, hate 0.44, sexual 0.94, offensive 0.41. Pass `thresholds={"insult": 0.3}` to `ProfanityClassifier` to trade precision for recall on one label. Lower thresholds catch more profanity and insults but flag more clean text: in our tests, thresholds tuned on a sample that was half offensive raised profanity / insult F1 by about 0.05 and tripled the share of clean texts flagged. ## Input - One comment, message or passage. The model reads at most 256 tokens; longer text is cut (`"truncated": True`). For long documents use `classify_document`, which returns the highest score of each label over 256-token chunks and the number of flagged chunks. - Native script, romanized and code-mixed text are all supported. Judge the text alone: the model does not see the conversation, the image or the video a comment refers to. ## Evaluation **Gold test set:** 3,980 texts never used in training: about 300 social-media comments per language (from the test splits of the source datasets; none for Gujarati, Punjabi, Odia and Assamese, which have no labelled comment data) and about 107 web passages per Indic language (FineWeb-2 documents held out from training). Each text was labelled by two LLMs independently (Gemma 4 31B and Claude Haiku 5.5) following a written annotation guide; Claude Sonnet 5.5 decided their disagreements. **No person reviewed the gold labels**, so they contain errors. F1 per label at the shipped thresholds, the share of clean gold texts (all four labels no, 1,693 texts) on which any label fires, and recall of `hate` on Hindi HateCheck (Röttger et al. 2022; 2,488 hateful test cases): | Model | profanity | insult | hate | sexual | offensive | Clean texts flagged | HateCheck-hi hate recall | |---|---:|---:|---:|---:|---:|---:|---:| | 58M (same family) | 0.80 | 0.69 | 0.49 | 0.85 | 0.89 | 3.5% | 0.23 | | **this model (36M)** | 0.70 | 0.70 | 0.51 | 0.83 | 0.89 | 3.6% | 0.17 | | 17M (same family) | 0.76 | 0.66 | 0.49 | 0.85 | 0.88 | 4.0% | 0.24 | | Lexicon rule | 0.78 | — | 0.36 | 0.27 | 0.70 | 2.9% | 0.00 | | TF-IDF + logistic regression | 0.64 | 0.55 | 0.46 | 0.85 | 0.87 | 7.6% | 0.08 | Lexicon rule: a word list of 598 terms we wrote; it has no `insult` label. TF-IDF: character n-grams with logistic regression, trained on the same data. **By language** (`offensive` F1 and clean texts flagged, this model): | Language | offensive F1 | Clean flagged | Gold texts | |---|---:|---:|---| | Hindi (`hi`) | 0.90 | 0.0% | comments + web | | Bengali (`bn`) | 0.89 | 2.7% | comments + web | | Marathi (`mr`) | 0.87 | 5.9% | comments + web | | Tamil (`ta`) | 0.88 | 5.1% | comments + web | | Telugu (`te`) | 0.84 | 2.3% | comments + web | | Kannada (`kn`) | 0.92 | 6.4% | comments + web | | Malayalam (`ml`) | 0.88 | 2.5% | comments + web | | Gujarati (`gu`) | 0.96 | 0.0% | web text only | | Punjabi (`pa`) | 0.87 | 4.7% | web text only | | Odia (`or`) | 0.94 | 22.2% | web text only | | Assamese (`as`) | 0.94 | 2.8% | web text only | | Urdu (`ur`) | 0.88 | 2.1% | comments + web | | English (`en`) | 0.89 | 2.1% | comments | Gujarati, Punjabi, Odia and Assamese are scored on about 107 web passages each, with no social-media comments: their high F1 says little about comments in those languages, where no labelled data exists. **By script** (`offensive` F1): romanized 0.91, native script 0.87, mixed 0.82. **By text type** (F1, comments / web): profanity 0.65 / 0.77, insult 0.71 / 0.29, sexual 0.11 / 0.92. ## Training data About 248,000 texts: social-media comments and posts from nine public datasets (credited in the table and in NOTICE) and about 7,000 web passages from FineWeb-2. The source datasets each label "offensive" their own way; their labels were mapped to the four labels by a written guide and left blank where they do not say (for example, a dataset that marks a comment "abusive" says nothing about which label applies). About 12,000 comments and 7,000 web passages were labelled with all four labels by two LLMs (Gemma 4 31B and Claude Haiku 5.5), with gpt-5.4-mini deciding where they disagreed. The models were then distilled from a larger fine-tuned mmBERT-base. | Dataset | Languages | Licence | |---|---|---| | [MACD (Multilingual Abusive Comment Dataset), ShareChat / Moj comments, ShareChat](https://github.com/ShareChatAI/MACD) | hi, ta, te, ml, kn | no licence file in the repository; used with attribution | | [DravidianCodeMix offensive language identification (OffensEval Dravidian), Chakravarthi et al.](https://huggingface.co/datasets/community-datasets/offenseval_dravidian) | ta, ml, kn | CC BY 4.0 | | [BD-SHS, Romim et al., LREC 2022](https://www.kaggle.com/datasets/naurosromim/bdshs) | bn | CC BY 4.0 | | [Bengali Hate Speech, Karim et al.](https://huggingface.co/datasets/rezacsedu/bn_hate_speech) | bn | MIT | | [Telugu-Hatespeech, Marreddy et al., ACM TALLIP 2022](https://huggingface.co/datasets/mounikaiiith/Telugu-Hatespeech) | te | CC BY 4.0 | | [MOLD (Marathi Offensive Language Dataset) 1.0 and 2.0, Gaikwad, Zampieri et al.](https://github.com/tharindudr/MOLD) | mr | CC BY 4.0 | | [RUHSOLD (Roman Urdu Hate-Speech and Offensive Language Dataset), Rizwan et al., EMNLP 2020](https://github.com/haroonshakeel/roman_urdu_hate_speech) | ur (romanized) | MIT | | [Hate Speech and Offensive Language, Davidson et al., ICWSM 2017](https://github.com/t-davidson/hate-speech-and-offensive-language) | en | MIT | | [HateXplain, Mathew et al., AAAI 2021](https://github.com/hate-alert/HateXplain) | en | CC BY 4.0 | | [FineWeb-2 (web text), Hugging Face](https://huggingface.co/datasets/HuggingFaceFW/fineweb-2) | all 12 Indic languages | ODC-By 1.0 | MACD has no licence file; we use it with attribution to its authors and ShareChat. Hindi HateCheck (CC BY 4.0) was used for testing only. ## Limitations - **`hate` is weak.** On the gold set about 46% of `hate` flags are wrong (often strong personal abuse with no identity target) and it finds 48% of hateful texts; on Hindi HateCheck it finds 17%. Do not rely on it to find hate speech, and check every `hate` flag. - **Mild insults are often missed** (`insult` recall 0.58): rude but not vulgar words such as "idiot" or "bewakoof" often score below the threshold, sometimes on `offensive` too. - **Insults in web text** are rarely found (F1 0.29), and **sexual content in comments** is weak (F1 0.11). - **False alarms vary by language:** 22% (Odia) and 6% (Kannada) of clean gold texts are flagged in the worst languages, against 3.6% overall. - **Gujarati, Punjabi, Odia, Assamese** and native-script Urdu have little or no labelled comment data; expect more errors on comments in these languages. - **Bias.** Dialect words, caste and community terms, reclaimed words and counter-speech that quotes abuse can be flagged wrongly; mild words can be missed. Errors are not spread evenly across languages, scripts or communities. - **Labels are from LLMs**, not people, both for training and for the gold set. The guide's choices (for example, that criticism of a political party is not `hate`) are one reasonable reading, not the only one. - **Context.** The model sees one text. Sarcasm, quotes, in-group banter and comments about images or videos are often misread. ## Intended use and disclaimer **Intended use:** helping people review user-generated content (ranking or routing comments for a moderator, hiding by default with a way to show, warning before posting); research on offensive language in Indian languages; measuring how much offensive content a corpus contains; comparing moderation tools. **Not intended for, and not supported:** - **The sole basis for removing content, suspending accounts or any decision about a person.** Use it with human review and an appeal route. It makes mistakes, and they fall unevenly on languages, dialects and communities. - **Legal determinations**, such as whether text is hate speech, obscene, defamatory or otherwise unlawful under any law. Its labels are not legal findings or advice. - **Profiling or surveillance** of people or groups, or inferring anyone's religion, caste or other identity. - **Statements about real people.** A label means the model scored a text that way; it is not a finding that anyone said or meant anything offensive. **No responsibility for use.** The model is provided "as is", without warranties or conditions of any kind, and the authors and Cobuild Tech Private Limited (CobuildX) are not liable for any use of it (Apache License 2.0, sections 7 and 8). You alone are responsible for how you use the model and its outputs, for any decision taken with them, and for complying with the law that applies to you. The model runs on your systems; CobuildX does not receive your data. Scores on this page are measured on specific test sets and are not a guarantee of performance on your text. ## Files | File | | |---|---| | `model.safetensors`, `config.json`, `tokenizer.json`, `tokenizer_config.json` | the model: mmBERT encoder with a five-way sigmoid head | | `profanity_config.json` | labels, thresholds, input limit, base model, scores | | `cbx_profanity.py`, `example.py`, `requirements.txt` | loader and example | | `LICENSE`, `NOTICE` | Apache-2.0 licence, third-party notices and dataset credits | | `checksums.sha256` | SHA-256 of every file | ## Contact Questions: hello@cobuildx.ai. Release v0.1 (2026-11-19).