coding-router — routing model

The routing model for coding-router: one local endpoint that optimizes coding requests between big and small models, so you get more usage at the same cost. Given a coding task (or a whole agent conversation), it picks the model and reasoning effort with the best measured cost/quality trade-off. Routing runs fully locally — this encoder runs in-process on your machine.

Files

path	what
router.json	model list, decision-rule parameters, provenance
router.npz	the evidence bank: task embeddings plus measured outcome and cost per model
encoder-fp16/	the tuned Qwen3-Embedding-0.6B, fp16 safetensors — CUDA / CPU via sentence-transformers
encoder-mlx-4bit/	the same tuned encoder, 4-bit MLX for Apple Silicon

Both encoder directories are the same model in two precisions. The server downloads only the one it needs: MLX on Apple Silicon, torch elsewhere.

What the router is

Qwen3-Embedding-0.6B, fine-tuned (LoRA on the attention projections) to route coding work. At inference: embed the request, compare it against a bank of tasks with measured outcomes for every model, and pick the cheapest model predicted to solve it. If nothing in the bank is similar to the request, the router does not guess — it escalates to the strongest model.

Benchmarks

Live-measured on held-out tasks the router never saw during fitting, against always using opus-5 (high), the strongest single model:

task type	router quality	opus-5 quality	cheaper	saved
short interactive requests	1.000	1.000	6.0×	83%
competitive programming (LiveCodeBench)	0.961	0.997	6.0×	83%
long-horizon software tasks (DeepSWE)	0.908	0.926	3.8×	74%

Quality differences are within statistical noise on every task type; the saving is the win.

Scope

The router was validated on repo-issue-style tasks (a paragraph or more of real problem description). Very short one-line prompts often have no close match in the evidence bank; the router then escalates to the strongest model rather than guess — safe, but no saving. Check off_distribution on the decision if you want to know when that happened.

Use

You never fetch this by hand — uv run python -m router.serve in coding-router downloads it on first run, then runs offline. New router versions overwrite this repo in place; old versions stay in the repo's git history.

This model contains only benchmark-task embeddings and measured outcomes — no user data of any kind.