# Using the router in a stack: the ideas that apply everywhere **Status legend.** *Documented* = checked against the tool's own documentation (2026-10-15). *Untested* = written from the documentation but not run against the real tool: read the debug log the first time. ## What you send to the router The **first user message of a task**, as plain text. Not the system prompt, not the history, not tool results. In agent harnesses a single user prompt triggers many model requests (tool loops, sub-agents, summaries). Route **once per user prompt** and keep that choice for the whole loop. ## One decision per prompt, sticky per session - Switching model in the middle of a conversation loses the prompt cache (you pay the prefix again) and can invalidate the reasoning blocks of earlier turns. The example policy therefore escalates and does not come back down within a session (`session.escalate_only`) and caps the number of switches. - Sub-agents start with their own first message (the task the parent gives them): they can be routed on their own. - Background requests (titles, classifiers, summaries) do not need the router; give them the smallest model. ## The policy is yours `integrations/policy.example.yaml` maps the router's outputs (`complexity_score`, `task`, `needs_tools`, `prompt_tokens`) to a tier (small / mid / large / frontier), a model and a reasoning effort, per harness profile. Thresholds sit halfway between the tier scores. They are a starting point: log the decisions for a week, read 50 of them, then adjust. Override with `!small`, `!mid`, `!large`, `!frontier` at the start of a prompt. ## Failure modes to design for - **Router down or slow**: use a short timeout (0.15 to 0.5 s) and fall back to the policy default, never block the request. - **`complexity_score` is a judgment of the labeller, not a measure of how hard the prompt is for your model.** Check the quality of the cheap tier on your own prompts before sending real traffic to it. - **Tools**: the example policy never sends a request that needs tools to the smallest tier. - **Privacy**: the router runs locally; prompts do not leave the machine for the routing step. - Not for moderation or safety decisions. ## Pieces in this repo | file | role | |---|---| | `s1d1_router.py` | load the model, decode the 21 outputs | | `integrations/policy.py`, `policy.example.yaml`, `test_policy.py` | policy engine, example policy, checks (run without a model) | | `integrations/serve.py` | `POST /route` over HTTP, standard library only | | `integrations/routed.py` | pick model and effort, then launch Claude Code or Codex | | `integrations/litellm/` | pointer to s1d1-tantive, the LiteLLM Auto Router classifier |