ToolChooser-0.8B

A LoRA adapter on Qwen3.5-0.8B that chooses which tool answers a chat message when an embedding router is unsure. The router scores the message against every capability (tool) the chat offers; when two or more are still close, ToolChooser chooses one of them, or none, in which case the person is asked to choose.

It chooses only among the capabilities the router itself offered, and the host application decides what runs: a model's output alone never authorizes an operation.

It is a distillation: it learned the choices of a teacher 34 times its size, Qwen3.8-27B. On tie-breaks it never saw it makes the teacher's choice more often than a 35-billion-parameter model asked the same thing, and a wrong choice a tenth as often:

	201 tie-breaks of 31 October: the teacher's choice / the person asked / wrong	106 tie-breaks of 22 October	seconds a tie-break
ToolChooser-0.8B	184 / 15 / 2	91 / 13 / 2	0.31
its predecessor of 31 October	176 / 24 / 1	83 / 21 / 2	—
Qwen3.6-35B-A3B, thinking off (MoE, 3B active)	171 / 4 / 26	84 / 4 / 18	0.55
DeepSeek-V4-Flash, thinking off	152 / 24 / 24, 1 unreadable	71 / 20 / 15	1.91
LFM2.5-24B-A2B, thinking on (it cannot be turned off)	145 / 1 / 41, 14 unreadable	74 / 0 / 26, 6 unreadable	—
Qwen3.5-4B, thinking off	134 / 12 / 55	69 / 6 / 31	0.40

A wrong choice is a capability where the teacher chose none or another one; the person being asked is not a wrong choice, since nothing runs that was not asked for. The general models write their answer with the sampling their own cards give, and it is read with the host application's own parser; unreadable means the parser found no answer in it. ToolChooser's answer is read as a decision (below), in a few short requests: the template, the tokenizer and the next-token probabilities along the options; a general model writes its answer in one. Seconds are the median over the 201 tie-breaks, one request at a time on one NVIDIA A100 64 GB with llama.cpp, through the application's own call; DeepSeek-V4-Flash on three, which it needs, over its first 50.

Its sibling adapters load on the same base, each in a repository of its own: SemanticRepair-0.8B, which rewrites a message the router did not place, and QueryTranslator-0.8B, which writes a search request again in another language.

What is in this repository

file	what it is
qwen3.5-0.8b-base-2fc06364-q8_0.gguf	the base: Qwen3.5-0.8B at revision 2fc06364, Q8_0, 833,592,320 bytes, sha256 6a94a8273ab1fc5735743967b621eb5a3baba8e36a202f61c20dc47fbb80eb51
tool-chooser-0.8b-20261107-lora-f16.gguf	the adapter of 7 November 2026, F16 LoRA, 21,671,008 bytes, sha256 6713e46c8f7d2bf8a3b8f24cf7f66c6dbcb4f7acbc37c7ab747d69f59bb1bd9f
LICENSE	the base model's Apache-2.0 licence

Pin the sha256 of both files: a routing decision has to be reproducible.

Where it runs

With llama-server:

llama-server -m qwen3.5-0.8b-base-2fc06364-q8_0.gguf \
  --lora tool-chooser-0.8b-20261107-lora-f16.gguf \
  --jinja --host 127.0.0.1 --port 9250 -ngl 99 -c 32768 --no-context-shift --cache-ram 0

When the base also carries its sibling adapters, each request names this one at full scale and the others at none.

When it is asked, and how it is read

It reads the base model's chat template (llama-server --jinja, thinking off): a system message with the rules, and a user message listing the offered capabilities, each with its title, its description and what it gives (prose, tables, a chart…), the documents the person selected, and the message. The rules: choose by what the message asks for, not by the words it shares with a title; when the person selected documents and the message asks about what they hold, choose the capability that reads the selected documents; otherwise choose one only when the message plainly asks for what it gives and for nothing another one gives as well, and answer none when two would answer it equally. Its trained answer is one Markdown item:

- capability: <one id from the list, or none>

Nothing is generated. For each offered capability and for none, the host reads the probability of that whole answer, its end token included, from llama-server's next-token log-probabilities (/completion with n_probs), and normalizes them over the options. The likeliest capability is taken only when its probability is at least 0.85; below that the person is asked to choose. 0.85 is the lowest threshold with the fewest wrong choices on the 201 tie-breaks of 31 October, so their figures are not independent of it; the 106 tie-breaks of 22 October played no part in choosing it.

threshold	201: the teacher's choice / the person asked / wrong	106
0.8	182 / 15 / 4	90 / 13 / 3
0.85	184 / 15 / 2	91 / 13 / 2
0.9	182 / 17 / 2	90 / 14 / 2

A higher threshold can add choices the same as the teacher's: where the teacher chose none, a capability below the threshold becomes none.

What it is measured to do

201 tie-breaks of 31 October, 146 Italian and 55 English, from conversation groups that share nothing with the training rows, each answered by the teacher, Qwen3.8-27B, with the rules above; in 116 of them the person had selected documents.

106 tie-breaks of 22 October, 53 Italian and 53 English, answered by the teacher with the rules of that day, before the rule on selected documents.

In the chat's routing chain on 8 November, behind SemanticRepair-0.8B of 7 November, against its predecessor of 31 October: on 100 test messages, 50 Italian and 50 English, right 92 against 90 and wrong 6 against 8; on 481 real Italian questions right 178 against 177 and wrong 32 against 32.

What it does not fix

It chooses only among what the router offered. A capability the router did not put in the contest cannot be chosen, however right it would be.

The descriptions alone often do not settle it. Without selected documents it leaves more choices to the person than with them: a description says what a capability gives, not every question it answers.

Measured in Italian and English.

The data it was trained on

49,528 training rows, 40,216 answered with a capability and 9,312 with none: the 28,662 tie-breaks of the earlier training, recorded in scripted conversations played through the application and asked again on 31 October of the teacher with the rules above, and 20,866 recorded since 1 November and answered by the teacher with the same rules. 36,449 tie-breaks the teacher answered with the rules of 22 October while documents were selected are left out, since the rule on selected documents changed their answer; an answer the host's own parser could not read is left out. The 201 tie-breaks of the table were taken out by conversation group before training, and the 106 of 22 October and every question a benchmark measures stay out too.

Training

base	Qwen/Qwen3.5-0.8B at 2fc06364, its language model only
method	LoRA, rank 16, alpha 32, dropout 0.05, on every linear layer but the embeddings and the head (186 modules, 10.8M parameters)
schedule	one pass: 6,191 updates of 8 examples, learning rate 1×10⁻⁴, 100 warm-up steps
tokens	49.3M in the examples, 348K of them in the answers
loss	on the answer only
hardware	one NVIDIA A100 64 GB of the Leonardo supercomputer in Bologna, thanks to CINECA
release	F16 LoRA GGUF

It starts from the original base; it does not continue an earlier round.

Licence

Apache-2.0, as its base model Qwen3.5-0.8B. What it chooses is yours to check, and so is the responsibility for what you let it decide.