πŸ›‘οΈ Horizon-10M-Guard

A Tiny, Experimental LLM-Based Safety & Content Moderation Classifier

Params Base Model Task Accuracy Context License Status

Built with πŸ–€ by Nebulixlabs

Instagram GitHub HuggingFace


🧠 Model Overview

Horizon-10M-Guard is a lightweight, LLM-based text safety classification model created by applying Supervised Fine-Tuning (SFT) on top of Nebulixlabs/Horizon-10M-Base.

Designed as an ultra-compact guard model (~10M parameters), it evaluates input text across broad safety domains and classifies whether the content is safe or unsafe, while also predicting the corresponding harm sub-category. Despite its tiny size, it demonstrates how modern decoder architectures (RoPE + SwiGLU + RMSNorm) can be adapted into fast, edge-friendly content moderation classifiers.


πŸ—οΈ Architecture Details

Component Value
🧬 Base Model Nebulixlabs/Horizon-10M-Base
🎯 Fine-Tuning Method Supervised Fine-Tuning (SFT)
πŸ”’ Parameters ~10.06M
πŸ“š Vocabulary size 8,192 (Custom Byte-Level BPE)
πŸ“ Hidden size (d_model) 320
🧱 Layers 6
πŸ‘οΈ Attention heads 5 (head_dim = 64)
πŸ” MLP hidden size 864 (SwiGLU)
πŸͺŸ Context length 128 tokens
🎯 Positional encoding Rotary (RoPE)
βš–οΈ Normalization RMSNorm (pre-norm)

πŸ“Š Evaluation & Performance

Metric Score Notes
🎯 Overall Accuracy ~60% Evaluated on held-out safety classification prompts

Note: Achieving ~60% multi-category safety classification accuracy with only 10.06M parameters and a 128-token context window highlights the model's efficiency as an experimental research baseline, though it is not suitable for standalone production moderation.


πŸ›‘οΈ Supported Classification Scope

Horizon-10M-Guard supports general-purpose text safety classification across diverse topics and outputs:

  1. Safety Verdict: safe or unsafe
  2. Sub-Category Tag: Identifies the specific risk/topic category associated with the input text.

πŸ’» How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM

repo_id = "Nebulixlabs/Horizon-10M-Guard"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)

# Format your input prompt according to your SFT prompt template
prompt = "Classify the following text:\nText: How do I bake a chocolate cake?\nOutput:"

output = model.generate_text(
    tokenizer,
    prompt=prompt,
    max_new_tokens=20,
    temperature=0.1,      # Low temperature recommended for deterministic classification
    top_k=10,
    top_p=0.9,
    repetition_penalty=1.05
)
print(output)
Downloads last month
31
Safetensors
Model size
12.7M params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Nebulixlabs/Horizon-10M-Guard

Finetuned
(4)
this model