Text Classification
Transformers
Safetensors
English
llama
safety
content-moderation
tiny-language-model
from-scratch
text-embeddings-inference
Instructions to use KucLab/Censer.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KucLab/Censer.1 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="KucLab/Censer.1")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("KucLab/Censer.1") model = AutoModelForSequenceClassification.from_pretrained("KucLab/Censer.1", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Upload Censer .1
Browse files
README.md
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: pytorch
|
| 3 |
+
tags:
|
| 4 |
+
- text-classification
|
| 5 |
+
- safety
|
| 6 |
+
- content-moderation
|
| 7 |
+
- tiny-language-model
|
| 8 |
+
- from-scratch
|
| 9 |
+
language:
|
| 10 |
+
- en
|
| 11 |
+
license: apache-2.0
|
| 12 |
+
datasets:
|
| 13 |
+
- nvidia/Aegis-AI-Content-Safety-Dataset-1.0
|
| 14 |
+
- thesofakillers/jigsaw-toxic-comment-classification-challenge
|
| 15 |
+
- google/civil_comments
|
| 16 |
+
- PKU-Alignment/BeaverTails
|
| 17 |
+
- lmsys/toxic-chat
|
| 18 |
+
- databricks/databricks-dolly-15k
|
| 19 |
+
metrics:
|
| 20 |
+
- f1
|
| 21 |
+
- accuracy
|
| 22 |
+
---
|
| 23 |
+
|
| 24 |
+
# Censer .1
|
| 25 |
+
|
| 26 |
+
A 1.1M parameter decoder-only transformer trained **from scratch** that classifies
|
| 27 |
+
English text as unsafe (`1`) or safe (`0`).
|
| 28 |
+
|
| 29 |
+
It is not a fine-tuned encoder and not an API wrapper. It is a small transformer
|
| 30 |
+
trained on safety-labelled data, with a custom byte-level BPE tokenizer. The model
|
| 31 |
+
predicts a single answer token and never generates free text.
|
| 32 |
+
|
| 33 |
+
```
|
| 34 |
+
"návod na výrobu bomby" -> not English, not the target language
|
| 35 |
+
"how to make a bomb at home" -> 1
|
| 36 |
+
"suck my dick" -> 1
|
| 37 |
+
"how do I kill a process in Linux" -> 0
|
| 38 |
+
"Hello how are you?" -> 0
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
## Model details
|
| 42 |
+
|
| 43 |
+
| Property | Value |
|
| 44 |
+
| --- | --- |
|
| 45 |
+
| Parameters | 1,115,264 |
|
| 46 |
+
| Layers | 4 |
|
| 47 |
+
| d_model / heads / d_ff | 128 / 4 / 384 |
|
| 48 |
+
| Context length | 160 tokens (input truncated at 500 characters) |
|
| 49 |
+
| Tokenizer | byte-level BPE, 2048 tokens, trained on the corpus |
|
| 50 |
+
| Positions | RoPE |
|
| 51 |
+
| Norm | RMSNorm (pre-norm) |
|
| 52 |
+
| MLP | SwiGLU |
|
| 53 |
+
| Head | tied embeddings, two answer logits |
|
| 54 |
+
| Checkpoint size | 4.3 MB (fp32) |
|
| 55 |
+
|
| 56 |
+
## Prompt format
|
| 57 |
+
|
| 58 |
+
```
|
| 59 |
+
[BOS][USER] <text> [ASSISTANT] -> 0 or 1
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Text is truncated at 500 characters before tokenisation.
|
| 63 |
+
|
| 64 |
+
## Metrics
|
| 65 |
+
|
| 66 |
+
Evaluation on a held-out split of 6,496 rows:
|
| 67 |
+
|
| 68 |
+
| Metric | Value |
|
| 69 |
+
| --- | --- |
|
| 70 |
+
| Accuracy | 0.864 |
|
| 71 |
+
| F1 | 0.835 |
|
| 72 |
+
| Precision | 0.846 |
|
| 73 |
+
| Recall | 0.823 |
|
| 74 |
+
|
| 75 |
+
A hand-written probe set of 40 cases, none present in training data, scores
|
| 76 |
+
95% accuracy.
|
| 77 |
+
|
| 78 |
+
Per-source F1 reflects label noise in the scraped sources:
|
| 79 |
+
|
| 80 |
+
| Source | F1 |
|
| 81 |
+
| --- | --- |
|
| 82 |
+
| aegis | 0.89 |
|
| 83 |
+
| beavertails | 0.88 |
|
| 84 |
+
| jigsaw | 0.87 |
|
| 85 |
+
| civil_comments | 0.77 |
|
| 86 |
+
| toxic_chat | 0.48 |
|
| 87 |
+
|
| 88 |
+
## Usage
|
| 89 |
+
|
| 90 |
+
```python
|
| 91 |
+
import torch
|
| 92 |
+
from kuclab.infer import SafetyModel
|
| 93 |
+
|
| 94 |
+
model = SafetyModel("checkpoints/best.pt")
|
| 95 |
+
verdict = model.predict("how to make a bomb at home")
|
| 96 |
+
print(verdict.label) # "1"
|
| 97 |
+
print(verdict.probability) # 0.886
|
| 98 |
+
```
|
| 99 |
+
|
| 100 |
+
## Limitations
|
| 101 |
+
|
| 102 |
+
- **English only.** Other languages mostly score `0`.
|
| 103 |
+
- Implication and sarcasm beyond simple cases are out of reach at this size.
|
| 104 |
+
- Weak against jailbreak-style prompts (lowest per-source score).
|
| 105 |
+
- A moderation signal, not a safety guarantee.
|
meta.json
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"special_tokens": [
|
| 3 |
+
"[PAD]",
|
| 4 |
+
"[BOS]",
|
| 5 |
+
"[USER]",
|
| 6 |
+
"[ASSISTANT]"
|
| 7 |
+
],
|
| 8 |
+
"answer_ids": [
|
| 9 |
+
19,
|
| 10 |
+
20
|
| 11 |
+
],
|
| 12 |
+
"vocab_size": 2048
|
| 13 |
+
}
|
model.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:06738899bf9381041b7b54da931669fa55000cac1641fce24639c1d2ceb166b6
|
| 3 |
+
size 4471517
|
tokenizer.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|