Qwen3-0.6B-tools

Merged BF16 weights of the LoRA tool-calling fine-tune of Qwen/Qwen3-0.6B, trained so the model reliably emits OpenAI-style tool calls for agentic workflows.

Base model

This model was fine-tuned from:

Qwen/Qwen3-0.6B

Training

Supervised fine-tuning with LoRA, run with NeMo-RL, assistant-turn-masked cross-entropy over the tool-calling logs.

  • LoRA: dim 32, alpha 32, dropout 0.05, target modules *.proj (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)
  • Max sequence length: 4096
  • Precision: bfloat16
  • Optimizer: AdamW, lr 5e-5, weight decay 0.01
  • Schedule: 15-step linear warmup, then cosine decay to 1e-6
  • Global batch size: 64 (micro batch 2)
  • Training steps: 336 (validation every 84)
  • Loss masked to assistant turns via the chat template's generation keyword

The tool-calling chat template is the model's contract: it is the template the base model's inline template was replaced with so that tool schemas render verbatim and the assistant's tool_calls block is the trainable span.

Dataset

Training used the sft_tools split of the r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation dataset (OpenAI messages format; assistant turns carry tool_calls).

Intended use

  • Structured tool / function calling
  • Agent-style multi-step interactions
  • Small-footprint on-device or edge deployment

It is not intended to be a general replacement for larger Qwen models.

Sibling artifacts

  • GGUF quantizations: iromu/Qwen3-0.6B-tools-GGUF
  • NVFP4 checkpoint: iromu/Qwen3-0.6B-tools-NVFP4

Usage

Serve the NVFP4 checkpoint with TensorRT-LLM:

trtllm-serve serve iromu/Qwen3-0.6B-tools-NVFP4 --port 8000

Load a GGUF quant with llama.cpp:

llama-cli -hf iromu/Qwen3-0.6B-tools-GGUF:Q4_K_M

Load the merged BF16 model with transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "iromu/Qwen3-0.6B-tools", torch_dtype="bfloat16", device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("iromu/Qwen3-0.6B-tools")

Validation

Model Quant Cap n Tool call emitted Names match Exact args match Δ exact vs BASE (gated)
Qwen/Qwen3-0.6B BF16 384 tok 274 48/274 (17.5%) 9/274 (3.3%) 2/274 (0.7%) (no predecessor)
Qwen3-0.6B-tools BF16 384 tok 274 272/274 (99.3%) 228/274 (83.2%) 175/274 (63.9%) +63.1pp

Measured on the sft_tools held-out validation split, greedy decoding. Throughput is single-stream greedy decode, not serving throughput.

suppressed in the Δ column means no comparison was admissible, which is a statement about the provenance of the two rows and not a measurement of no change. The reason is printed in the cell.

Downloads last month
512
Safetensors
Model size
0.6B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for iromu/Qwen3-0.6B-tools

Finetuned
Qwen/Qwen3-0.6B
Adapter
(769)
this model
Adapters
4 models

Dataset used to train iromu/Qwen3-0.6B-tools

Space using iromu/Qwen3-0.6B-tools 1

Collection including iromu/Qwen3-0.6B-tools