Instructions to use switchboardllm/switchboard-router-0.5b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use switchboardllm/switchboard-router-0.5b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="switchboardllm/switchboard-router-0.5b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("switchboardllm/switchboard-router-0.5b")
model = AutoModelForCausalLM.from_pretrained("switchboardllm/switchboard-router-0.5b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use switchboardllm/switchboard-router-0.5b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "switchboardllm/switchboard-router-0.5b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "switchboardllm/switchboard-router-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/switchboardllm/switchboard-router-0.5b

SGLang

How to use switchboardllm/switchboard-router-0.5b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "switchboardllm/switchboard-router-0.5b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "switchboardllm/switchboard-router-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "switchboardllm/switchboard-router-0.5b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "switchboardllm/switchboard-router-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use switchboardllm/switchboard-router-0.5b with Docker Model Runner:
```
docker model run hf.co/switchboardllm/switchboard-router-0.5b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

switchboard-router-0.5b

Switchboard v0.1 is a 0.5B router: it reads a prompt and answers with one line of JSON, the cheapest model tier that can handle it. Send easy prompts to a small model and pay frontier prices only for the prompts that need it. Open weights, Apache 2.0.

Try it in your browser: Switchboard demo · Every number and raw answer: evaluation page

Run it

With Ollama, one line, no GPU needed:

ollama run hf.co/switchboardllm/switchboard-router-0.5b-GGUF "Write a haiku about rain"

{"tier":"small","domain":"writing"}

With Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer
import json

repo = "switchboardllm/switchboard-router-0.5b"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

msgs = [{"role": "user", "content": "Unscramble the following words: eglar, ngareo, kobo"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt")
out = model.generate(ids, max_new_tokens=24, do_sample=False)
print(json.loads(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True)))
# {'tier': 'frontier', 'domain': 'general'}

With Python, using the switchboard-router package (no other dependencies; it talks to Ollama, any OpenAI-compatible server, or Transformers):

pip install switchboard-router

from switchboard_router import Router

router = Router()  # uses Ollama when it's running
router.pick_model("Write a haiku about rain")
# 'anthropic/claude-haiku-5.5'

Use greedy decoding (do_sample=False, or temperature 0).

What it answers

Field	Values
`tier`	`small`, `mid` or `frontier`: the cheapest class of model expected to answer the prompt well
`domain`	`code`, `math`, `writing` or `general`: a hint, from keyword labels

If the output isn't valid JSON, send the prompt to frontier. A failed routing call should cost money, never quality.

Builds

Build	Size	Runs with	Same decision as this model
This repo (safetensors, BF16)	988 MB	Transformers, vLLM	reference
GGUF F16	994 MB	Ollama, llama.cpp	346-347 of 347 test prompts
ONNX 8-bit	660 MB	Transformers.js in the browser	344 of 347 test prompts

Results

Full-precision Switchboard v0.1 on 10,000 held-out prompts it never saw in training, in their natural mix:

	Switchboard v0.1	Random router*
Cost saved vs. always using Claude Opus 5.5	64.5%
Prompts served well (proxy, see below)	94.5%	90.0%
Sent too low (needed frontier, sent cheaper)	5.5%	10.0%
Hard prompts sent to frontier	59.1%	26.2%
Tier accuracy	60.3%
Invalid JSON outputs	0

*The random router sends the same share of prompts to frontier, picked at random. A perfect router would save 76.2%.

How it's measured. Each prompt in routellm/gpt4_dataset comes with a 2023 Mixtral-8x7B answer scored 1-5 by a GPT-4 judge. A prompt counts as served well if Switchboard sent it to frontier, or sent it lower and that answer scored at least 4. A 2026 small model is stronger than Mixtral-8x7B, so this proxy is cautious. Cost uses Surplus output prices per 1M tokens: Claude Haiku 5.5 ($0.50) for small, Claude Sonnet 5.5 ($10) for mid, Claude Opus 5.5 ($20) for frontier. Every answer is downloadable from the evaluation page.

Versions

	v0	v0.1
Cost saved	67.8%	64.5%
Sent too low	7.1%	5.5%
Hard prompts sent to frontier	47.5%	59.1%
Ranking AUC (hard vs. rest)	0.785	0.787

v0.1 trains on 1.5× more frontier examples. It calls the frontier model more often on purpose, giving up 3.3 points of savings for 22% fewer prompts sent too low. Its ability to rank hard prompts above easy ones (AUC) is about the same as v0. v0 is commit 016f5a9 in this repo's history.

The model


Base	Qwen2.5-0.5B-Instruct (Apache-2.0), full fine-tune in BF16
Parameters	494M
Training data	`routellm/gpt4_dataset` (Apache-2.0, LMSYS), 63,749 rows after balancing
Labels	Mixtral score 5 → small, 4 → mid, 1-3 → frontier; `domain` from keywords
Training	2 epochs in 55 minutes; the released weights are the best checkpoint by validation loss (end of epoch 1). One RTX 5070 Ti. Loss on the JSON answer only.
Prompt length	Trained on prompts cut to 512 tokens

Limitations

This is a 0.5B router and it misroutes some prompts: 5.5% of held-out prompts were sent to a cheaper tier than they needed.
Labels come from 2023 models and a GPT-4 judge, not from people. Mostly English prompts.
It reads one prompt at a time, not the whole conversation, and only the first 512 tokens.
A prompt can try to talk it into a cheaper tier. Don't use it as a security boundary.
domain comes from keyword labels and is only a hint.
Switchboard is an independent project and is not affiliated with Anthropic, Alibaba, Robinhood, Surplus or the Switchboard oracle network.

License

Apache-2.0, inherited from Qwen2.5-0.5B-Instruct and routellm/gpt4_dataset.

Downloads last month: -

Safetensors

Model size

0.5B params

Tensor type

BF16

Model tree for switchboardllm/switchboard-router-0.5b

Base model

Qwen/Qwen2.5-0.5B

Finetuned

Qwen/Qwen2.5-0.5B-Instruct

Finetuned

(1077)

this model

Quantizations

2 models

switchboardllm
/

switchboard-router-0.5b