KucLab commited on
Commit
a9fe46a
·
verified ·
1 Parent(s): 120c3b3

Standard LlamaForSequenceClassification layout: loads with plain transformers, no trust_remote_code

Browse files
Files changed (3) hide show
  1. README.md +10 -11
  2. model.safetensors +1 -1
  3. tokenizer.json +0 -0
README.md CHANGED
@@ -143,14 +143,13 @@ Evaluation on a held-out split of 6,496 rows:
143
 
144
  | Metric | Value |
145
  | --- | --- |
146
- | Accuracy | 0.857 |
147
- | F1 | 0.824 |
148
- | Precision | 0.846 |
149
- | Recall | 0.804 |
150
 
151
- A hand-written probe set of 62 cases, none present in training data, scores
152
- **98.4% accuracy** (recall 1.0 — every unsafe probe caught). The previous
153
- Censer.1 scores 88.7% on the same probes.
154
 
155
  Per-source F1 reflects label noise in the scraped sources:
156
 
@@ -185,8 +184,8 @@ Benchmarked against public safety classifiers on the **same held-out test split*
185
 
186
  | Model | Params | Checkpoint | F1 | Latency | Throughput (b=32) | Peak VRAM |
187
  | --- | --- | --- | --- | --- | --- | --- |
188
- | **Censer.2 (ours)** | 7.3M | 28.0 MB | 0.824 | 3.14 ms | 497 texts/s | 177 MB |
189
- | KucLab/Censer.1 (previous) | 1.1M | 4.5 MB | 0.836 | 1.79 ms | 1,561 texts/s | 101 MB |
190
  | unitary/unbiased-toxic-roberta | 124.7M | 498.7 MB | 0.741 | 14.41 ms | 67 texts/s | 771 MB |
191
  | unitary/toxic-bert | 109.5M | 876.0 MB | 0.682 | 14.01 ms | 69 texts/s | 708 MB |
192
  | martin-ha/toxic-comment-model | 67.0M | 267.9 MB | 0.566 | 7.52 ms | 131 texts/s | 540 MB |
@@ -199,8 +198,8 @@ Benchmarked against public safety classifiers on the **same held-out test split*
199
  ![Metric bars](figs/metrics_bars.png)
200
 
201
  Censer.2 is ~9-23x smaller than the encoder baselines and 2.4-4.6x faster per text.
202
- On the scraped test split it is on par with Censer.1 (F1 0.824 vs 0.836); on the
203
- hand-written probe set it is far ahead (98.4% vs 88.7%).
204
 
205
  ## Usage
206
 
 
143
 
144
  | Metric | Value |
145
  | --- | --- |
146
+ | Accuracy | 0.863 |
147
+ | F1 | 0.830 |
148
+ | Precision | 0.862 |
149
+ | Recall | 0.799 |
150
 
151
+ A hand-written probe set of 85 cases, none present in training data, scores
152
+ **100% accuracy**. The previous Censer.1 scores 88.7% on the same probes.
 
153
 
154
  Per-source F1 reflects label noise in the scraped sources:
155
 
 
184
 
185
  | Model | Params | Checkpoint | F1 | Latency | Throughput (b=32) | Peak VRAM |
186
  | --- | --- | --- | --- | --- | --- | --- |
187
+ | **Censer.2 (ours)** | 7.3M | 28.0 MB | 0.830 | 3.12 ms | 491 texts/s | 177 MB |
188
+ | KucLab/Censer.1 (previous) | 1.1M | 4.5 MB | 0.833 | 1.89 ms | 1,643 texts/s | 101 MB |
189
  | unitary/unbiased-toxic-roberta | 124.7M | 498.7 MB | 0.741 | 14.41 ms | 67 texts/s | 771 MB |
190
  | unitary/toxic-bert | 109.5M | 876.0 MB | 0.682 | 14.01 ms | 69 texts/s | 708 MB |
191
  | martin-ha/toxic-comment-model | 67.0M | 267.9 MB | 0.566 | 7.52 ms | 131 texts/s | 540 MB |
 
198
  ![Metric bars](figs/metrics_bars.png)
199
 
200
  Censer.2 is ~9-23x smaller than the encoder baselines and 2.4-4.6x faster per text.
201
+ On the scraped test split it is on par with Censer.1 (F1 0.830 vs 0.833); on the
202
+ hand-written probe set it is at 100% vs 88.7% for Censer.1.
203
 
204
  ## Usage
205
 
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0f21ea032eb9c6a6fcbbeeb4cf7011a9664b69e343fdafcbffcf5640e026dd8d
3
  size 29381648
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b192b2a6dad7031cca40cff9dd25c00e771d3f7e2a1ddf661747b521189cfc21
3
  size 29381648
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff