Cevra-ToolChoice-4B

Cevra-ToolChoice-4B is a merged, full-weight fine-tune of Qwen3.5-4B for selecting the next tool from a supplied set of candidate descriptions, including an optional no_tool decision. It scores decision-label tokens to produce a distribution over the supplied choices. It does not construct tool-call arguments or execute tools.

Usage

Use the same Tinker Cookbook DecisionRenderer (qwen3_5) and score the candidate-label token logits at the decision position. Ordinary text or JSON generation does not reproduce these probabilities. This example uses CUDA and the same scoring procedure used for evaluation. It requires sufficient GPU memory for the 4B BF16 model.

pip install torch accelerate 'transformers==5.22.2' 'tinker-cookbook==0.6.1'

import json
import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from tinker_cookbook import renderers
from tinker_cookbook.recipes.decisions import Decision, DecisionRenderer

repo = "EugeneEvstafev/cevra-toolchoice-4b"
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=False)
model = AutoModelForImageTextToText.from_pretrained(
    repo, dtype=torch.bfloat16, device_map={"": "cuda:0"},
    trust_remote_code=False,
).eval()
renderer = DecisionRenderer(renderers.get_renderer("qwen3_5", tokenizer))

state = "What is the current weather in London?"
instructions = "Select the most appropriate tool for the request."
choices = [
    ("weather", "Get current weather for a city."),
    ("no_tool", "No tool is needed."),
]
text = json.dumps(
    {"state": state, "instructions": instructions},
    ensure_ascii=False, sort_keys=True,
)
rendered = renderer.render(Decision(text, choices))
assert rendered.choice_pos == len(rendered.tokens) - 1
prompt = torch.tensor([rendered.tokens[:rendered.choice_pos]], device="cuda:0")
label_ids = torch.tensor(
    [rendered.label_tokens[label] for label in rendered.labels], device="cuda:0"
)
with torch.inference_mode():
    logits = model(input_ids=prompt, use_cache=False).logits[0, -1].float()
    values = torch.softmax(logits[label_ids], dim=0).cpu().tolist()
probabilities = dict(zip(rendered.labels, values, strict=True))
print(probabilities)

The example describes the interface rather than a validated prediction for this particular prompt. Check whether the selected tool is permitted and validate its arguments before execution.

Training

The model was trained with Tinker's Decision Model soft-label objective using a Qwen3.5-4B LoRA adapter (rank 16), merged into Hugging Face base revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. Tinker did not report its exact underlying base revision, so identity with the merge base cannot be asserted. The training snapshot contained 4,048 ToolBench-derived choice examples: 3,235 train, 407 validation and 406 test. Training used one epoch, 203 steps, batch size 16 and learning rate 0.0001. Validation selected the checkpoint. The dataset includes automatically generated and verified examples and one-hot or smoothed targets; Jev contributed to constructing the targets.

The source is the pinned Adorg/ToolBench snapshot, derived from ToolBench. The upstream project describes the data as intended for research and educational use and identifies Apache-2.0; its inputs include descriptions of third-party APIs. Review applicable rights and terms for your intended use. The training data and test examples are not included in this repository.

Evaluation

The public evaluation test split contains the 406 held-out examples used in this comparison. It is provided for evaluation and was not used as training data.

On the held-out test split, using the same inputs and candidates for the pinned Hugging Face base and the merged model:

Model	Agreement with dataset labels	Distribution Brier (lower is better)
Qwen3.5-4B base	78.57%	0.29733
Cevra-ToolChoice-4B	98.03%	0.02017
Jev through LLM7	98.28%	0.03755

These are agreement with dataset labels and target distributions, not independent estimates of real-world accuracy or probability calibration. The test set has 31 source groups, was exposed during development, and is not an independently human-labeled benchmark. Jev contributed to soft target construction, so this is not an independent comparison. The paired grouped-bootstrap 95% interval for Jev minus Cevra agreement was -4.03 to +4.99 percentage points. Local Laya could not process the 406 test examples without truncating tool descriptions and has no comparable score.

The merged model's probabilities were checked against Tinker on ten test inputs; the maximum absolute difference was 0.00598. These ten inputs are narrowly concentrated, so the check does not establish general equivalence. A separate comparison of the pinned Hugging Face base with the Tinker base on ten synthetic inputs had a maximum absolute probability difference of 0.02813. These limited numerical checks do not prove that the models are identical: the provider does not disclose the training base revision. Model performance may change for unseen tools, long contexts, ambiguous requests, different candidate sets or alternative renderers. Thresholds and confidence should be validated on independent, human-labeled data before deployment. Treat tool descriptions and prior tool results as untrusted input.

Paper and citation

Evgenii Evstafev, Cevra-ToolChoice-4B: A System Card for Probabilistic Tool Selection, Zenodo preprint, 12 November 2026.

@misc{evstafev2026cevra_toolchoice,
  author    = {Evstafev, Evgenii},
  title     = {Cevra-ToolChoice-4B: A System Card for Probabilistic Tool Selection},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.24159727},
  url       = {https://doi.org/10.5281/zenodo.24159727}
}

License and provenance

The Qwen3.5-4B base is distributed under Apache-2.0; its original license is included. This card does not make an independent determination of rights for third-party training data or API descriptions. Inspect the upstream sources and applicable terms for redistribution or commercial use.