Text Classification
Transformers
Safetensors
ONNX
multilingual
modernbert
routing
llm-routing
prompt-classification
complexity-estimation
text-embeddings-inference
Instructions to use scombat/s1d1-router with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use scombat/s1d1-router with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="scombat/s1d1-router")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("scombat/s1d1-router") model = AutoModelForSequenceClassification.from_pretrained("scombat/s1d1-router", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download tutorials/02-litellm.md from scombat/s1d1-router: direct link, hf CLI and curl.
- Browser
- Download file 5.73 kB
-
https://huggingface.co/scombat/s1d1-router/resolve/main/tutorials/02-litellm.md
- Command line
-
hf download hf://scombat/s1d1-router/tutorials/02-litellm.md
-
curl -L -o 02-litellm.md https://huggingface.co/scombat/s1d1-router/resolve/main/tutorials/02-litellm.md
5.73 kB
| # LiteLLM | |
| The router as the classifier of LiteLLM's Auto Router. The auto router (`auto_router/complexity_router`) sorts each | |
| request into a tier and sends it to the model you put in that tier. The classifier is | |
| [s1d1-tantive](https://github.com/scombat/s1d1-tantive), a container image that serves the router: it answers the tier | |
| in a few milliseconds, with no LLM call. Keys, budgets, logs and fallbacks stay in LiteLLM. | |
| *Status: the `config.yaml` below was tested locally on 2026-10-25 with LiteLLM 1.106.4 against the hosted s1d1 API (then on s1d1-tantive 0.8.0). | |
| The admin-UI pitfalls of section 5 come from an earlier run on LiteLLM 1.103.2 with real | |
| opencode traffic.* | |
| The maintained reference is [s1d1-tantive docs/litellm.md](https://github.com/scombat/s1d1-tantive/blob/main/docs/litellm.md) | |
| (admin GUI, `curl` registration, Test Routing); this page keeps the short path and what went wrong once. | |
| ## 1. Run the service | |
| ```bash | |
| docker run --rm -p 8788:8788 ghcr.io/scombat/s1d1-tantive:0.8.1-small-onnx-int8 | |
| curl localhost:8788/ready # {"ready": true} once loaded and warmed up (a few seconds); 503 before | |
| ``` | |
| One image per variant (`full`, `compact`, `small`, `tiny`); the weights are inside, nothing is downloaded at start. The | |
| `small` one takes a few milliseconds per request on a CPU. Multi-arch, signed. Since 0.8.1 the int8 weights are the | |
| `reduce_range` ones (correction of 2026-10-25: earlier `small` images gave wrong scores on x86 CPUs without VNNI), so use | |
| 0.8.1 or later. To try the routing without running anything, the free hosted API <https://s1d1.qoo.gy/api> works as | |
| `api_base` (what you send is kept and used to retrain s1d1; read its terms first). | |
| ## 2. Configure LiteLLM (1.106 or later) | |
| Tested with LiteLLM 1.106.6. The service is registered as an ordinary deployment, and the router points at it by name: | |
| ```yaml | |
| model_list: | |
| # ... your four models: cheap, medium, strong, reasoning | |
| - model_name: s1d1-classifier | |
| litellm_params: | |
| model: hosted_vllm/s1d1-small # the name after the slash is sent and ignored by the service | |
| api_base: http://HOST:8788 # the service root, LiteLLM appends /v1/systemone | |
| model_info: | |
| mode: decisions | |
| - model_name: s1d1-auto | |
| litellm_params: | |
| model: auto_router/complexity_router | |
| complexity_router_default_model: medium | |
| complexity_router_config: | |
| classifier_type: oss_classifier | |
| opensource_classifier_config: | |
| deployment_name: s1d1-classifier | |
| timeout_ms: 1000 | |
| classifier_fallback: default_model | |
| tiers: | |
| SIMPLE: cheap | |
| MEDIUM: medium | |
| COMPLEX: strong | |
| REASONING: reasoning | |
| ``` | |
| Call `s1d1-auto` like any model. If the service is slow, unreachable or answers a name the router does not know, the | |
| request goes to the default model (`classifier_fallback: default_model`): a timeout opens a circuit breaker for 30 | |
| seconds, a stopped service falls back on each request. `fallback_tier` and `tier_definitions` are for LiteLLM Enterprise | |
| custom tiers; they are not needed here. On the service, `/metrics` counts `tantive_decisions_total` by tier: that is the | |
| check that the service was asked, since the header `x-litellm-complexity-router-cause` reads `jev_classifier` for any | |
| classifier. Where the admin GUI offers no deployment for the classifier, the fallback is `provider: laya`, | |
| `model: multilingual`, `api_base`, `timeout_ms` in place of `deployment_name` (LiteLLM's `laya` provider accepts only | |
| three model names; the service does not read it); it gave the same tiers on the same `/v1/systemone`. | |
| ## 3. The tier-name trap | |
| The router's tier names must be LiteLLM's four (`SIMPLE`, `MEDIUM`, `COMPLEX`, `REASONING`), which the service always | |
| recognises whatever its `S1D1_TIERS`. A name the service does not know sends every request to the fallback: if routing | |
| looks stuck on one model, check the names first. To move the bounds, set them on the service (`S1D1_TIERS`), not in | |
| LiteLLM. Scores above about 0.9 are rare, so the top tier sees little traffic. | |
| ## 4. LiteLLM before 1.106 | |
| No `oss_classifier` there. Mount the chat adapter (`S1D1_FORMATS=...,chat`), register the service as an | |
| OpenAI-compatible model with API base `http://HOST:8788/v1`, and in the auto router use classification method | |
| **LLM Classifier** with that model, timeout 1000 ms, fallback `heuristic`. Details: s1d1-tantive `docs/formats.md`. This | |
| is the path that was tested on 1.103.2. | |
| ## 5. Things that went wrong once (Admin UI) | |
| - **A wildcard model (`provider/*`) cannot be picked as a tier model.** Declare each upstream model explicitly. If the | |
| upstream is another LiteLLM, its name must not keep your local prefix. Saved credentials avoid retyping the key. | |
| - **Do not rename the standard tiers.** The UI sends its own labels to the classifier; with the four standard tiers | |
| leave display names empty. Custom names go through *Edit tiers* and must match the service (section 3). | |
| - **"Add tier" fails with `crypto.randomUUID is not a function`** over plain HTTP on an IP address: browsers only | |
| provide it on HTTPS or `localhost`. Use an SSH tunnel (`ssh -L 4000:localhost:4000 server`, then | |
| `http://localhost:4000/ui`) or HTTPS. | |
| - **"Add Auto Router" is greyed out with no tooltip.** A disabled button gets no hover; press Enter in the name field | |
| and the reason appears as an error message. | |
| ## 6. Limits | |
| - **Current message only.** The service classifies the last user message, what the router was trained on (single | |
| prompts, up to 1,024 tokens): a short follow-up such as "yes, go ahead" lands in a low tier. | |
| - **The tier is the only output** that reaches the auto router; task type and tool need are in the service's log. | |