| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct |
| tags: |
| - code |
| - function-calling |
| - tool-use |
| - agent |
| - small-language-model |
| datasets: |
| - NousResearch/hermes-function-calling-v1 |
| language: |
| - en |
| pipeline_tag: text-generation |
| --- |
| |
| # smolcode-coder-1.5b-tools |
|
|
| A LoRA fine-tune of **Qwen2.5-Coder-1.5B-Instruct** that teaches the model to emit |
| **native `<tool_call>` function calls**, so a 1.5B *coder* model can actually drive an |
| agentic write β run β fix β verify loop. |
|
|
| Built for [**smolcode**](https://gitea.poyner.ai/sean/smolcode) β an SLM-optimized |
| agentic coding assistant β for the Hugging Face **Build Small** hackathon. |
|
|
| ## Why |
| Out of the box, small Qwen-Coder models describe tool calls as plain-text/```json |
| instead of emitting the native `<tool_call>` token (id 151657) that runtimes (Ollama, |
| llama.cpp) parse into OpenAI-style `tool_calls` β which breaks agentic loops. This |
| fine-tune closes that gap on a tiny (1.5B) model: **100% native `<tool_call>` emission** |
| in free generation on held-out prompts (base model: 0%). |
| |
| ## Results |
| - **Native tool-call rate:** 100% (16/16 held-out prompts) β the release gate. |
| - **Agentic bench (smolcode pass@1, 10 tasks):** 9/10 as the entry tier of a |
| 1.5Bβ8Bβ30B ladder, solving **7/10 entirely on its own** (2β16s each). For |
| comparison the all-Granite ladder (3B entry) scores 10/10 β the 1.5B carries the |
| same standalone load as a 2Γ-larger 3B. |
| - **Train loss:** 0.138 (3 epochs, assistant-only loss). |
| |
| ## Training |
| - **Base:** Qwen/Qwen2.5-Coder-1.5B-Instruct |
| - **Method:** bf16 LoRA (r=16, Ξ±=32) on attention + MLP projections, **plus full |
| training of `embed_tokens` + `lm_head`** (`modules_to_save`) β required so the model |
| can *output* the `<tool_call>` special token, which LoRA on attention/MLP alone |
| cannot. **Assistant-only loss** (loss on tool calls + final answers only). |
| - **Data:** NousResearch/hermes-function-calling-v1 (breadth) + synthetic smolcode |
| tool-use trajectories (sharpness), all rendered through the *same* |
| `apply_chat_template(tools=...)` used at inference β training target is byte-identical |
| to the served prompt (fixes the v1 train/inference template mismatch). |
| - **Schedule:** 3 epochs, full 2048 sequence length. Trained on Modal (A100). |
| |
| ## Serving β read this, two non-obvious requirements |
| 1. **Serve via the GGUF, not the safetensors directly.** Ollama's bf16-safetensors |
| auto-import produces garbage (`??????`) for this model. Use the included |
| `smolcode-1.5b-q4_k_m.gguf` (converted with llama.cpp `convert_hf_to_gguf.py`): |
| ```bash |
| ollama create smolcode-coder-1.5b:tools -f Modelfile # Modelfile is in this repo |
| ``` |
| 2. **`repeat_penalty` / `repetition_penalty` MUST be 1.0.** The tool system prompt |
| literally contains the `<tool_call>` token, so any penalty > 1 suppresses the model |
| from emitting it (you'll see a stray token + bare JSON instead). The included |
| `Modelfile` sets `PARAMETER repeat_penalty 1.0`. For raw `transformers.generate`, |
| pass `repetition_penalty=1.0`. |
| |
| With those, Ollama's `/v1/chat/completions` returns proper native `tool_calls`. |
| |
| ## Use (transformers) |
| Standard Qwen2.5 chat template with `tools=`; greedy, `repetition_penalty=1.0`. The |
| model responds with `<tool_call>{"name": ..., "arguments": ...}</tool_call>`. |
| |
| ## Files |
| - `model.safetensors` + tokenizer/config β the merged model (lm_head untied). |
| - `smolcode-1.5b-q4_k_m.gguf` β quantized GGUF for serving. |
| - `Modelfile` β Ollama import recipe (template + `repeat_penalty 1.0`). |
| |
| ## License |
| Apache-2.0 (inherits from the base model). |
| |
