henny52/axo-native-small-preview

axo-native-small (preview)

A small text moderation model for chat messages. It reads a message, decides whether it is toxic, and returns a fixed JSON structure: a score, an action (allow / flag / block), categories, and the intermediate results of four reasoning steps that led to the decision.

• 11.3M parameters, 64-token context; about 7 ms per message on 4 CPU threads, about 3 ms per message when batched
• Focus on German chat (gaming, communities, comment sections), plus English, Spanish, French, Portuguese and Russian
• Weights as safetensors and ONNX, Apache-2.0

This is a preview. The output format and the label set may still change before 1.0.

Results

Measured on ChatMod Bench, 600 German and English messages, half toxic. Every model runs with its own default decision rule.

Strong on chat-specific cases such as game banter, quotes, reports to moderators and implied insults. Weak on negation ("you are not stupid"), harsh but fair criticism, and counter-speech about groups.

Quickstart

The model has its own small architecture, so it ships with the code to run it (axo_native/, depends only on torch, numpy and safetensors).

pip install torch numpy safetensors huggingface_hub

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("henny52/axo-native-small-preview")
sys.path.insert(0, path)

from axo_native import AxoNativeSmall, AxoNativeTokenizer

model = AxoNativeSmall.from_pretrained(path)
tokenizer = AxoNativeTokenizer.from_pretrained(path)

result = model.predict_json("you are a worthless idiot, nobody wants you here", tokenizer)[0]
print(result.action, round(result.score, 2))   # block 1.0
print(result.to_json())

predict_json accepts a single string or a list. level selects one of three threshold presets:

level	flag from	block from
light	0.50	0.82
medium (default)	0.35	0.70
strict	0.22	0.55

Pass explain=False for batch scoring when you don't need the step-by-step trace.

ONNX

model.onnx takes input_ids and attention_mask (int64, shape [batch, seq]) and returns the raw logits of all heads (toxic_logits, severity_logits, categories_logits, lang_logits, trigger_logits, target_logits, context_logits). The toxicity score is softmax(toxic_logits)[:, 1]. Use AxoNativeTokenizer to produce the inputs; the tokenizer is plain Python with no dependencies beyond numpy.

Output

{
  "toxic": true,
  "score": 0.9998,
  "severity": 3,
  "categories": {"insult": 0.9958, "hate": 0.0016, "threat": 0.0001,
                 "self_harm": 0.0011, "sexual": 0.0003},
  "model": "axo-native-small",
  "lang": "en",
  "action": "block",
  "self_harm": false,
  "thinking": {
    "trigger": "insult_word",
    "target": "individual",
    "context": "direct_attack",
    "steps": ["..."],
    "explanation": "1. Trigger analysis: insult_word (98%), focus on 'worthless', 'nobody', 'idiot' -> toxicity 99%
..."
  }
}

Field	Values
categories	insult, hate, threat, self_harm, sexual
severity	0 none, 1 mild, 2 clear, 3 severe
lang	de, en, es, fr, pt, ru
thinking.trigger	none, profanity, insult_word, slur, threat_word, self_harm_word, sexual_word, obfuscated
thinking.target	nobody, self, game_object, individual, protected_group
thinking.context	neutral, gaming_banter, quote_or_report, sarcasm, fiction_or_hypothetical, direct_attack

self_harm: true forces block from a score of 0.35 so that these messages can be routed to a human. The model is not a crisis-detection tool.

How it works

text ─► tokenizer ─► 6-layer encoder ─► 4 reasoning steps ─► output heads ─► JSON
                    (RoPE, SwiGLU,     trigger → target →
                     RMSNorm)          context → verdict

• Tokenizer: 8,192-entry subword vocabulary. Before tokenizing, it folds look-alike characters (Cyrillic/Greek homoglyphs, fullwidth forms) and decodes common leetspeak, so i d 1 0 t, іdіot (Cyrillic і) and idiot end up close to each other.
• Encoder: 6 layers, 256 dimensions, 8 heads, 64-token context.
• Reasoning core: four sequential steps. Each step attends over the encoded tokens, predicts one concept (what triggered it, who is targeted, which context), and passes its state and prediction on to the next step. A small probe after each step estimates the toxicity at that point, so you can see how the decision forms.
• Heads: toxicity, severity, categories and language are linear heads on the final state. There is no text decoder; the model can only produce this structure.

The thinking trace shows the model's internal intermediate predictions and where each step attended. It is useful for debugging and for showing moderators why something was flagged, but it is not a guaranteed causal explanation.

---

Includes data derived from HOCON34k and GAHD, both licensed under CC BY 4.0.