---
license: apache-2.0
datasets:
- Nebulixlabs/safety-dataset
language:
- hi
- en
tags:
- Multilingual
- Hindi
- Text_Classification
- Guard
---
# Quantum-Guard-Multilingual
**A compact 1M-parameter Transformer for `safe` / `unsafe` text classification**
[]()
[](https://github.com/NebulixLabs)
[](https://instagram.com/nebulix_labs)
[]()
[]()
[]()
[]()
[]()
[]()
---
## 📖 Model Overview
**Quantum-Guard-Multilingual** is a highly efficient, custom-built Transformer model containing exactly **1,000,000 parameters**. Developed by **NebulixLabs**, it is designed for ultra-fast, lightweight binary text classification to determine whether a given text is `safe` or `unsafe`.
Because of its unique **byte-level (ByT5) encoding**, this model supports multilingual inputs out of the box, making it highly robust against prompt injections, toxic content, and spam across different languages without needing a massive vocabulary table.
## 🛠 Model Architecture
The model uses a custom `QuantumClassifier` architecture built entirely from scratch in PyTorch. By keeping the parameter count exactly at 1M, it offers blazing-fast inference speeds even on CPU.
| Specification | Value |
| :--- | :--- |
| **Total Parameters** | 1,000,000 |
| **Hidden Size (d_model)**| 112 |
| **Attention Heads** | 7 |
| **Transformer Layers** | 4 |
| **FFN Intermediate Size**| 336 |
| **Classifier Hidden Size**| 170 |
| **Max Context Length** | 128 Tokens |
| **Vocab Size** | 4096 (Base 384 + custom markers) |
| **Dropout** | 0.10 |
## 📊 Training Data
This model was trained exclusively to understand nuanced differences between safe and harmful text. The training pipeline utilized the **Nebulixlabs/safety-dataset** for high-quality instruction, query, and prompt safety alignments.
* **Training Target:** 200,000,000 non-padding tokens.
* **Batch Size:** 128
* **Optimizer:** AdamW (Mixed Precision FP16)
* **Objective:** Binary Cross-Entropy Loss (`0 = safe`, `1 = unsafe`).
## ⚙️ Tokenizer Semantics
Unlike standard BPE tokenizers, Quantum-Guard-Multilingual utilizes the official **google/byt5-small** tokenizer semantics.
* It uses a direct **raw UTF-8 byte mapping** (Byte value `b` maps to token ID `b + 3`).
* Two model-specific sequence markers were injected: `` (ID: 384) and `` (ID: 385).
* This eliminates "Out of Vocabulary" (OOV) errors entirely and allows it to process Hindi, Hinglish, emojis, and corrupted text flawlessly.
## 🚀 How to Use (Inference)
Because this is a custom architecture (`QuantumClasifier`), you need the base class definitions to run inference. Below is a simplified snippet of how the model processes text:
```python
import torch
import torch.nn.functional as F
# Example fast-encode function mirroring the training script
def byt5_fast_encode(text, max_len=128):
raw = text.encode("utf-8", errors="ignore")[:max_len-2]
ids = [384] + [b + 3 for b in raw] + [385] # 384=, 385=
length = len(ids)
if length < max_len:
ids.extend([0] * (max_len - length)) # 0=
mask = [1] * length + [0] * (max_len - length)
return ids, mask
# Assuming the model is loaded into `model` and on correct device
text = "Hello, is this safe?"
ids, mask = byt5_fast_encode(text)
input_ids = torch.tensor([ids], dtype=torch.long)
attention_mask = torch.tensor([mask], dtype=torch.bool)
model.eval()
with torch.no_grad():
logits = model(input_ids, attention_mask)
probabilities = torch.softmax(logits.float(), dim=-1)[0]
predicted_id = int(probabilities.argmax().item())
confidence = float(probabilities[predicted_id].item())
label = "safe" if predicted_id == 0 else "unsafe"
print(f"Prediction: {label} (Confidence: {confidence*100:.2f}%)")
```
## 👨💻 About the Author
* **Organization:** NebulixLabs
* **GitHub:** [@NebulixLabs](https://github.com/NebulixLabs)
* **Instagram:** [@nebulix_labs](https://instagram.com/nebulix_labs)
* **Hugging Face:** [@Nebulixlabs](https://huggingface.com/Nebulixlabs)
*Feel free to reach out for collaborations or questions regarding lightweight model architectures!*