Instructions to use Waytrine/task-router with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Laya
How to use Waytrine/task-router with Laya:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
# No code snippets available yet for this library.
# To use this model, check the repository files and the library's documentation.
# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.jsTask Router v4
A 322M-parameter classifier that answers ten routing questions about one request to an AI agent system, in one forward pass: what kind of task it is, how hard it is, which tools it needs and who should do it. It routes requests in production in PugPort, a multi-agent desktop app.
Code, data and evaluation: github.com/Waytrine/task-router ยท Case study: wyt.si/task-router ยท API reference: wyt.si/task-router/api
The ten questions
| Head | Labels |
|---|---|
task_type (10) |
conversation, coding, research, writing, translation, data_analysis, media, automation, mixed, unclear |
difficulty (4) |
simple, standard, complex, unclear |
web_search, document_read, run_commands, browser_use |
yes, no, unclear |
input_modality (6) |
text, image, audio, video, mixed, unclear |
media_output (9) |
none, image_create, image_edit, speech, music, video_generate, video_render, mixed, unclear |
project_access (4) |
none, read, write, unclear |
executor_intent (6) |
none, assign, exclude, reassign, mention_only, unclear |
The input is a bounded "state" of five parts: the message, recent context, quoted reference text, attachment metadata (kind, name and MIME type only, never contents) and the candidate executors. Token limit 1024. The full definitions the model sees are in questions.json.
Model details
- Architecture: Laya typed-decision model on a
jhu-clsp/mmBERT-baseencoder (ModernBERT, 22 layers, hidden size 768, vocabulary 256,000) with ten single-choice heads. - Parameters: 321,908,998 (encoder 306,939,648; decision heads 14,175,744; scorer 592,897; action head 198,402). Stored in FP16.
- Fine-tuned from:
convaiinnovations/laya-multilingual, revision1720e3e3357cfe1e281542e223f8273b0890ca34, with the Laya framework at commitb9fe351ce81e2d974bca2bf219c1e4030c0a05ea. - Languages: English and Chinese.
- Calibration: post-hoc temperature scaling, T = 3.51, fitted on a separate 60-row calibration split.
model.safetensorsSHA-256:4445f9401b321a94ebc3614d231847dc08beaa0c9ccc3fded8dc39f90e8bb4b9
Training data
Only synthetic data: the bilingual dataset pugport-routing-synthetic-v1, published in data/ of the GitHub repository. Requests were written by an AI assistant and reviewed by a second agent; 720 training rows for v4. No real user requests were used to train this checkpoint.
Training: soft cross-entropy, 4 epochs, batch 8 x gradient accumulation 2, encoder learning rate 2.5e-5, head learning rate 1e-4, weight decay 0.01, gradient clip 1.0, bf16 mixed precision, encoder not frozen.
Results
| Evaluation | Mean accuracy over the ten heads |
|---|---|
| Untuned base model, synthetic test (60 rows) | 44.0% |
| Task Router v4, synthetic test (60 rows) | 97.8% |
| Task Router v4, 30 sealed real requests | 72.0% |
The synthetic test was reused across rounds, so 97.8% is a regression number; 72.0% on real requests is the more honest measure. A later v5, trained with additional private real requests, reaches 91.0% on the same sealed set; its weights are not published because its training data is private. Details: eval/.
How to use
The checkpoint loads with the Laya framework. The GitHub repository has the inference service, an OpenAPI description and evaluation scripts:
git clone https://github.com/Waytrine/task-router.git && cd task-router
python -m pip install -r service/requirements.txt
python -c "from huggingface_hub import snapshot_download as d; d('Waytrine/task-router', local_dir='models/task-router-v4')"
TASK_ROUTER_CHECKPOINT=models/task-router-v4 TASK_ROUTER_DEVICE=cpu python service/app.py
Then POST /v1/judge with a request state; see the API reference. TASK_ROUTER_DEVICE can be cpu, cuda or mps.
Limits
- Real-traffic evaluation is small: 30 sealed items; one item moves a head by 3.3 points. 26 of the 30 are coding requests and none asks for media output.
- Synthetic-to-real gap: v4 underrates the difficulty of long real coding work that needs project write access and commands.
- Confidence is not a safeguard: raw confidences stay high even on wrong answers.
- Not compared end to end with an LLM judge or with PugPort's rule-based router.
- It classifies a request; it does not authorise tools, file writes or spending.
License and attribution
- These weights: Apache License 2.0. They are a fine-tuned derivative of
convaiinnovations/laya-multilingual(Apache-2.0), whose encoder isjhu-clsp/mmBERT-base(MIT). - The code in the GitHub repository is MIT-licensed.
- Task Router is an independent project by Waytrine. It is not a fork of Laya and not an official Laya or Convai Innovations product.
- Downloads last month
- 1
Model tree for Waytrine/task-router
Base model
convaiinnovations/laya-multilingual