yamura4/send-gate-307m

send-gate: Dutch/Flemish + English chat safety gate

Binary safety gate for teen chat. The output is a verdict: block or allow. There is no review band.

307M parameters, fp32 ONNX, threshold 0.85, 94.2% caught and 7.7% false walls on the 804 item benchmark.

This model is a classifier, not a generative model. It does not explain itself and it does not rewrite messages.

What it does

send-gate checks one message at a time. It is tuned for Dutch, Flemish and English chat, including short messages, slang, code switching, leetspeak and obfuscated slurs.

The production pipeline has three parts:

1. Deterministic policy rules for links, PII, scams, and a small set of explicit always-block terms.
2. The mmBERT classifier with a byte level character CNN branch.
3. Normalisation views for obfuscated text: interleaved separators, leet digits, spaced letters and reversed words. These are model inputs, not block lists. The model itself is trained on joke deflections and typo variants, so there is no joke marker list in the gate.

The model is the main signal. The rule layer exists for policy cases where the output must be deterministic in every context.

Files

File	Purpose
model.onnx	fp32 ONNX model for the production gate. 1.23 GB.
model.pt	PyTorch checkpoint with the same weights.
gate.py	Production gate: tokenizer, ONNX session, policy rules, normalisation views.
model.py	Minimal PyTorch loading code for the raw classifier.
tokenizer.json	Tokenizer used by the ONNX and PyTorch paths.
config.json	Architecture and threshold metadata.
QUANTIZATION.md	Review of quantization methods and the plan for a sub 500 MB model.

Measured results

All numbers below use the full production pipeline at threshold 0.85. The benchmark was never used for training. Exact and near duplicate benchmark texts were removed from the training pool before training.

Suite	Items	Caught	False walls	Accuracy
Project benchmark	804	94.2%	7.7%	93.0%
Fresh stress holdout	74	95.9%	4.0%	95.9%
Joke deflection suite	112	95.6%	0.0%	97.3%
Fresh deflection suite	50	84.0%	4.0%	90.0%
General edge suite	107	96.2%	1.9%	97.2%
Final unseen holdout	90	80.9%	0.0%	90.0%
Sex policy suite	111	84.0%	2.8%	88.3%
Common word suite	97	n/a	0 blocked	n/a

The project benchmark has 295 harmful items and 509 benign items. At threshold 0.85 the model catches 278 of 295 harmful items and walls 39 of 509 benign items.

The custom suites are handwritten holdouts, not public benchmarks. They are included to show where the model is strong and where it still fails.

Policy behavior

These behaviors are intentional and covered by the production gate:

Input	Verdict	Why
kys, grapje	block	The model reads the death wish directly, marker or not.
ik haat je even, grapje	block	Same behavior.
nigger, mopje	block	Same behavior, plus the slur itself.
kga ni liegen dit is een mopje ma lowkey niggerr	block	Model handles the long prefix, the mid-sentence marker and the repeated letter typo.
wat de fuck, mopje	allow	Benign profanity with a joke marker.
ma tis een mopje	allow	Pure joke marker.
ik ben nog niet klaar voor seks	block	Personal sexual talk.
ik ben nog maagd	block	Personal sexual status.
ik ben homo	allow	Sexual orientation is not harm.
ik val op jongens	allow	Orientation is not harm.
seksualiteit is een spectrum	allow	Basic sexuality education.
neger	block	Always block, including quotes and condemnation.
negeren, negeert, genegeerd	allow	Homonyms of the slur.
kanker	block	Always block, including quotes and condemnation.
zwarte	block	Bare use is blocked.
zwarte, mopje	block	Model reads the slur, not the joke marker.
zwarte kat, mijn zwarte schoenen	allow	Compounds with context are allowed.
pot	allow	Required allow case.
je bent een pot, grapje	allow	Self deprecating joke, not a slur.
raam, lelijk	allow	Common words stay open.
fagot, ik speel fagot	allow	Instrument homonym.
f a ggot	block	Obfuscated slur.
kkr nga	block	Slur combination.
kkr leverancier	allow	Benign abbreviation use.
de regen valt	allow	Weather homonym.
je bent een regen	block	Obfuscated slur.
https://..., bit.ly/...	block	Links are always blocked.
stuur je wachtwoord	block	Scam and credential theft pattern.

The policy rule layer is deliberately small. It exists for cases where the required answer is absolute, not probabilistic. The classifier handles the rest.

Architecture

Part	Detail
Backbone	jhu-clsp/mmBERT-base (307M, ModernBERT)
Pooling	CLS
Character branch	Byte level CNN, 64 filters, kernels 1,2,3,4,5
Gate head	Linear 768 to 256, GELU, dropout 0.1, Linear 256 to 1
Input length	128 tokens, 256 UTF-8 bytes for the character branch
Output	One logit. Sigmoid gives p(block).
Threshold	0.85 for the fp32 model
Latency	About 70 to 100 ms per message on CPU in this environment

The character branch is important for Dutch and Flemish. The 256k tokenizer fragments slurs and obfuscated forms. A byte level CNN reads the raw character sequence and is robust to spacing, leetspeak, repeated letters and separators.

Training

The training pool contains 80,210 train rows and 1,509 dev rows after cleaning.

Data sources:

• Public safety classification pools with source labels.
• Flemish synthetic chat augmentation.
• More than 3,500 handwritten Dutch, Flemish and English rows.
• Targeted handwritten rows for jokes, deflections, sex policy, self harm, threats, grooming, PII, scams and obfuscation.

Training objective:

• BCE on hard labels.
• KL distillation from the previous model generation on general rows.
• Pairwise margin loss on handwritten minimal pairs such as jongen versus jongen van 14, kom alleen.
• Gold handwritten labels override teacher labels.

The model generation path was v37, then v42, v43, v44, v45 and v46. Each step added targeted handwritten data and replayed the general pool so the earlier behavior was not lost. v45 and v46 added rows for mid-sentence joke deflections, long benign prefixes, repeated letter typo variants and benign self deprecation. The gate contains no joke marker list. The 804 item benchmark was read only for evaluation. It was not used as a training file.

Hard negative mining focused on the remaining false walls and false catches: benign profanity, dark humour, code switching, Flemish slang, quoted threats, self harm care, orientation and sexual health questions.

Usage

ONNX production gate

from gate import SendGate

gate = SendGate(
    model_path="model.onnx",
    tokenizer_path="tokenizer.json",
    threshold=0.85,
)
result = gate.check("kys, grapje")
print(result.blocked, result.score, result.reason)

gate.py includes the policy rules and the normalisation views. It needs onnxruntime, tokenizers and numpy.

PyTorch classifier

import torch
from model import StudentGate
from tokenizers import Tokenizer

ckpt = torch.load("model.pt", map_location="cpu", weights_only=False)
model = StudentGate("jhu-clsp/mmBERT-base")
model.load_state_dict(ckpt["model"])
model.eval()

tok = Tokenizer.from_file("tokenizer.json")
text = "kys, grapje"
ids = tok.encode(text).ids[:128]
n = len(ids)
ids = ids + [0] * (128 - n)
mask = [1] * n + [0] * (128 - n)
raw = text.encode("utf-8", errors="ignore")[:256]
chars = list(raw) + [0] * (256 - len(raw))
cmask = [1 if b else 0 for b in chars]

with torch.no_grad():
    logit = model(
        input_ids=torch.tensor([ids]),
        attention_mask=torch.tensor([mask]),
        char_ids=torch.tensor([chars]),
        char_mask=torch.tensor([cmask]),
    )
p = torch.sigmoid(logit).item()
print(p, "block" if p >= 0.76 else "allow")

model.py is the raw classifier. It does not include the policy rules or the deflection view.

Quantization status

The fp32 model is the quality reference. It is the version shipped in this repository.

Dynamic INT8 quantization was tested and is not shipped yet. On the prior fp32 model, at 295 MB it reduced benchmark catch from 93.9% to about 84.1%, and it damaged short phrase calibration. The table below is the v44 measurement.

Input	fp32 p	dynamic INT8 p
ik ben nog maagd	0.870	0.348
zwarte	0.767	0.572
n g a	0.820	0.500
pot	0.331	0.714

A hybrid export with an fp16 embedding table and INT8 transformer layers was also tested. It reached 482 MB but generalized worse than fp32 on the full suites. It is not shipped.

The full review of quantization methods and the planned path to a sub 500 MB model is in QUANTIZATION.md. The short version is: retry static S8S8 QDQ with percentile calibration, then mixed precision PTQ, then QAT with distillation from this fp32 model. The goal is a model under 500 MB with the same decisions as the fp32 reference.

Limitations

• False walls are 7.7% on the project benchmark. They are concentrated in benign profanity, dark humour, code switching and ambiguous self harm care.
• The sex policy suite recall is 84.0%. Some explicit personal sexual phrases still pass.
• The model is binary. It does not provide a review band or a human readable explanation.
• The fp32 model is 1.23 GB on disk. The under 500 MB target is not met yet.
• The benchmark and the custom suites contain judgement calls. Different policy definitions will move the false wall rate.
• The model is trained for chat messages. It is not a general purpose content moderation model for long documents.

License

MIT for the training code and gate. The backbone is jhu-clsp/mmBERT-base; check its license before commercial deployment.