smoll-swarm/SmolRouter-135M

🐝 SMOL ROUTER 135M

The ultra-fast routing gateway for SMOL SWARM.

135M Parameters · Runs on CPU · Deterministic JSON Output · Zero Cloud Dependency

---

📌 Overview

SmolRouter-135M is the gateway agent for the SMOL SWARM ecosystem.

Its sole job is to analyze incoming user queries in under 2 milliseconds on a laptop CPU and output a clean, deterministic routing decision. Instead of running a large, heavy LLM to decide what to do, SmolRouter rapidly points the task to the right specialized worker model.

Think of it as the receptionist of your multi-agent system: it doesn't do the work itself, but it instantly knows exactly which specialist should handle each incoming request.

🎯 Supported Routes

Route	Target Model	Intended Task
extractor	SmolExtractor-1.7B	Invoices, receipts, logs, messy text → strict JSON schemas
compressor	SmolCompressor-360M	Summarization, RAG chunk pruning, context reduction
planner	SmolPlanner-1.7B	Multi-step workflows, project roadmaps, goal decomposition
coder	SmolCoder-1.7B	Helper functions, scripts, regex, code debugging
direct	Fallback / Local LLM	Simple factual questions, casual chat, direct answers

---

⚡ Quick Start

import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "smoll-swarm/SmolRouter-135M"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float32,
    device_map="auto"
)

SYSTEM_PROMPT = """You are SmolRouter, the routing coordinator for SmolSwarm.
Your job is to analyze the user's input and select the optimal agent to handle the task.
You must respond ONLY with a valid JSON object matching this schema:
{"route": "extractor" | "compressor" | "planner" | "coder" | "direct", "confidence": float, "reason": "short explanation"}"""

user_input = "Extract the order ID and billing total from this receipt: Invoice #8841, total due $99.12."

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": user_input}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=80,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

generated = outputs[0][inputs.input_ids.shape[1]:]
response_text = tokenizer.decode(generated, skip_special_tokens=True).strip()

decision = json.loads(response_text)
print(decision)
# Output:
# {"route": "extractor", "confidence": 0.96, "reason": "Receipt parsing."}

---

📊 Output Protocol

Every response follows strict, valid JSON matching the Smol Swarm envelope:

{
  "route": "extractor",
  "confidence": 0.96,
  "reason": "Receipt parsing."
}

---

🏗️ Swarm Architecture

                        User Request
                             │
                             ▼
                   ┌──────────────────┐
                   │ SmolRouter-135M  │  ~2ms latency on CPU
                   │  (Gatekeeper)    │
                   └─────────┬────────┘
                             │
       ┌──────────────┬──────┴───────┬──────────────┐
       ▼              ▼              ▼              ▼
 ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐
 │ Extractor │  │Compressor │  │  Planner  │  │   Coder   │
 │  (1.7B)   │  │  (360M)   │  │  (1.7B)   │  │  (1.7B)   │
 └───────────┘  └───────────┘  └───────────┘  └───────────┘

---

🧪 Verified Routing Examples

User Input	Routed To	✅
"Extract the order ID and billing total from receipt #8841..."	extractor	✓
"Can you condense this 4000-word market research document..."	compressor	✓
"Write a Python script to download images from an S3 bucket."	coder	✓
"How should I organize a migration plan from AWS to GCP?"	planner	✓
"What is the theory of general relativity?"	direct	✓

---

⚙️ Recommended Inference Settings

For deterministic routing, always disable sampling:

generation_config = {
    "do_sample": False,
    "max_new_tokens": 80,
    "pad_token_id": tokenizer.eos_token_id
}

---

📦 Model Details

Property	Value
Base Model	HuggingFaceTB/SmolLM2-135M-Instruct
Parameters	135 Million
License	Apache 2.0

---

🎓 Training Details

• Dataset: 1,500 synthetically-generated routing samples balanced across all 5 Smol Swarm routes.
• Format: ChatML.
• Loss Masking: Only the assistant JSON response contributes to the loss, with the prompt tokens masked using -100.
• Optimizer: AdamW (lr 1e-4, weight decay 0.01).
• Scheduler: Cosine decay with 10 warmup steps.
• Epochs: 3.
• Batch Size: 16.
• Hardware: 2× NVIDIA T4 GPUs via Kaggle.
• Precision: FP32 (full precision for maximum stability).
• Final Loss: ~0.07.

---

⚠️ Limitations

• Routing-only: This model only decides a route; it does not perform the task itself. The downstream specialist models (SmolExtractor, SmolCompressor, etc.) do the actual work.
• Calibration: Confidence values are learned outputs, not statistically calibrated probabilities.
• Language: Primarily trained on English prompts.
• Prompt Format: Must use the exact system prompt provided above for best results.

---

🗺️ Smol Swarm Ecosystem

Follow the Smol Swarm Org to track upcoming model releases.

---

📚 Citation

@misc{smolrouter2025,
  title   = {SmolRouter-135M: An Ultra-Fast Gateway Model for Local Multi-Agent AI},
  author  = {QuillBytes},
  year    = {2026},
  url     = {https://huggingface.co/smoll-swarm/SmolRouter-135M}
}

---

Built by the SMOLL SWARM community

Small models. Big swarm. No cloud required.

🐝 Smol Swarm Org · ⚖️ License