Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,225 @@
+---
+library_name: transformers
+base_model: unsloth/Qwen3.5-4B
+base_model_relation: finetune
+license: apache-2.0
+language:
+- en
+- zh
+- ko
+- ja
+- ru
+- es
+pipeline_tag: image-text-to-text
+tags:
+- qwen3_5
+- unsloth
+- merged
+- sft
+- distillation
+- reasoning
+- code-generation
+- coder
+- tool-use
+- function-calling
+- agent
+- mtp
+- multi-token-prediction
+- speculative-decoding
+- local-inference
+- edge-computing
+- on-device
+- gguf
+- llama.cpp
+- vision
+- multimodal
+- safetensors
+---
+
+# 🦅 Kestrel-4B-Coder
+
+![Kestrel-4B-Coder — a compact local coding model fine-tuned from Qwen3.5-4B](assets/model_hero.png)
+
+**Kestrel-4B-Coder** is a compact coding and agent-focused fine-tune of [Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B), prepared from [Unsloth Qwen3.5-4B](https://huggingface.co/unsloth/Qwen3.5-4B). Its focus is **useful intelligence closer to the device**: practical coding, compact thinking, and tool-assisted workflows that can run on personal hardware.
+
+The release is available as a [merged checkpoint](https://huggingface.co/Jackrong/Kestrel-4B-Coder) and a [GGUF model with bundled MTP weights](https://huggingface.co/Jackrong/Kestrel-4B-Coder-GGUF). The coding results below use **Q8_0 GGUF**.
+
+**Local today, broader edge deployment ahead.** The current agent results come from Apple Silicon with LM Studio. The development direction is to make capable coding agents more accessible across constrained devices, with broader hardware testing, efficient quantization, and stronger workspace reliability.
+
+## 💡 1. The Kestrel Philosophy — Make Each Token Count
+
+Kestrel puts practical coding first: understand the task, use the right tool, produce a useful edit, and check the result. Fine-tuned from **Qwen3.5-4B**, it brings focused coding and tool-use behavior to devices with tighter memory and compute budgets.
+
+The working loop is simple: **Read → Reason → Call Tools → Edit → Test**. Each turn adds to the time a developer waits for a result. Compact thinking aims to reduce unnecessary generation and keep the next action clear, especially when an agent repeats this loop many times.
+
+On an edge device, each generated token competes for memory bandwidth, compute time, and power. The design goal is a model that spends its reasoning budget on the next useful action, helping developers build responsive local assistants without relying on remote inference for every turn.
+
+> [!IMPORTANT]
+> **Useful code is the goal.** Brevity helps when the resulting code remains correct. Kestrel pairs compact thinking with code quality and practical tool use.
+
+## 🧪 2. Model Preparation and Collaboration
+
+The model was prepared by merging the coding adaptation into **unsloth/Qwen3.5-4B**. The merged release retains **MTP/NextN weights** and the inherited vision components. Its tokenizer and chat template come from the release adapter; use the included template for chat and tool interactions. The evaluations in this card cover text, code, and agent behavior.
+
+**Built with community support.** Thank you to [Kyle Hessling · @KyleHessling1 on X](https://x.com/KyleHessling1) for collaboration, testing, feedback, and hardware support, and to [Unsloth](https://unsloth.ai/) for efficient fine-tuning tooling. See the [Unsloth framework on GitHub](https://github.com/unslothai/unsloth).
+
+## 📚 3. Dataset and Distillation Recipe
+
+The training corpus contains approximately **150K high-quality code and complex reasoning examples**, selected to teach a compact model how to solve programming problems and act on tool feedback.
+
+- **Agent interaction and tool use (~40,000 examples).** Distilled from **Qwen3.8-27B**, this component covers multi-turn tool calls, environment state tracking, self-correction, and reflection on task and tool boundaries.
+- **Dense algorithms and code generation.** Distilled from **Qwen3.8-Flash-Next** and **Qwen3.8-Max**, these examples emphasize complex algorithm implementation, rigorous syntax, edge-case handling, and refactoring.
+- **Logical reasoning and instruction tuning.** Selected high-quality examples from the **nvidia/Nemotron** family reinforce multi-step reasoning and strict instruction following.
+
+The quality-control pipeline applies **strict AST-based syntax validation to applicable code, deduplication and cleaning, and sandbox execution filters**. These checks support a corpus centered on executable solutions, coherent tool interactions, and efficient developer workflows.
+
+## 📊 4. Coding Quality — HumanEval and HumanEval+
+
+The author-run results for **Kestrel-4B-Coder**, **Qwopus3.5-9B-Coder-Flash**, and **Qwen3.5-9B** cover **164 programming tasks, five samples per task, and 820 generations per model**. Their reported configuration uses **Q8_0 GGUF**, **llama.cpp CUDA**, **MTP speculative decoding with maximum draft length 2**, **temperature 0.2**, and **top-p 0.95**.
+
+![HumanEval and HumanEval+ comparison, ordered Kestrel-4B-Coder, Qwen3.5-4B, Qwopus3.5-9B-Coder-Flash, and Qwen3.5-9B; 4B columns are pale green and 9B columns pale purple](assets/coding_quality.png)
+
+<details>
+<summary>Text version of the coding results</summary>
+
+<table>
+  <thead>
+    <tr>
+      <th style="background: #f1f5f9; color: #334155;">Metric</th>
+      <th style="background: #d1fae5; color: #065f46;">Kestrel-4B-Coder</th>
+      <th style="background: #d1fae5; color: #475569;">Qwen3.5-4B</th>
+      <th style="background: #ede9fe; color: #4c1d95;">Qwopus3.5-9B-Coder-Flash</th>
+      <th style="background: #ede9fe; color: #475569;">Qwen3.5-9B</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr><td>HumanEval Pass@1</td><td style="background: #ecfdf5; color: #047857;"><strong>83.66%</strong></td><td style="background: #ecfdf5; color: #475569; font-weight: 400;">72.56%</td><td style="background: #f5f3ff; color: #6d28d9;"><strong>91.83%</strong></td><td style="background: #f5f3ff; color: #475569; font-weight: 400;">85.24%</td></tr>
+    <tr><td>HumanEval Pass@5</td><td style="background: #ecfdf5; color: #047857;"><strong>94.51%</strong></td><td style="background: #ecfdf5; color: #475569; font-weight: 400;">89.63%</td><td style="background: #f5f3ff; color: #6d28d9;"><strong>98.17%</strong></td><td style="background: #f5f3ff; color: #475569; font-weight: 400;">97.56%</td></tr>
+    <tr><td>HumanEval+ Pass@1</td><td style="background: #ecfdf5; color: #047857;"><strong>79.51%</strong></td><td style="background: #ecfdf5; color: #475569; font-weight: 400;">69.88%</td><td style="background: #f5f3ff; color: #6d28d9;"><strong>88.05%</strong></td><td style="background: #f5f3ff; color: #475569; font-weight: 400;">80.37%</td></tr>
+    <tr><td>HumanEval+ Pass@5</td><td style="background: #ecfdf5; color: #047857;"><strong>92.07%</strong></td><td style="background: #ecfdf5; color: #475569; font-weight: 400;">76.83%</td><td style="background: #f5f3ff; color: #6d28d9;"><strong>96.95%</strong></td><td style="background: #f5f3ff; color: #475569; font-weight: 400;">95.12%</td></tr>
+    <tr><td>HumanEval correct samples</td><td style="background: #ecfdf5; color: #047857;"><strong>686 / 820</strong></td><td style="background: #ecfdf5; color: #475569; font-weight: 400;">595 / 820</td><td style="background: #f5f3ff; color: #6d28d9;"><strong>753 / 820</strong></td><td style="background: #f5f3ff; color: #475569; font-weight: 400;">699 / 820</td></tr>
+    <tr><td>HumanEval+ correct samples</td><td style="background: #ecfdf5; color: #047857;"><strong>652 / 820</strong></td><td style="background: #ecfdf5; color: #475569; font-weight: 400;">573 / 820</td><td style="background: #f5f3ff; color: #6d28d9;"><strong>722 / 820</strong></td><td style="background: #f5f3ff; color: #475569; font-weight: 400;">659 / 820</td></tr>
+  </tbody>
+</table>
+
+</details>
+
+Kestrel reports **83.66% HumanEval Pass@1** and **79.51% HumanEval+ Pass@1**, alongside **72.56%** and **69.88%** for Qwen3.5-4B. Its **94.51% HumanEval Pass@5** and **92.07% HumanEval+ Pass@5** extend coverage across multiple attempts. The two 9B columns provide additional context, with Qwopus3.5-9B-Coder-Flash retaining the highest reported scores in this table.
+
+For the author-run results, Pass@1 is the reported single-attempt success estimate using five samples per task. Pass@5 summarizes whether the five attempts include a passing solution. Correct-sample counts refer to individual generations.
+
+## ⚡ 5. Efficiency — Local Compute and the Path Ahead
+
+The 4B footprint makes local deployment a central part of the model's purpose. The aim is to bring coding assistance and agent control to the machine where the work happens: a laptop, a personal workstation, or a smaller system with a limited resource budget.
+
+### Current deployment foundation
+
+- **Local inference and local tools.** The HermesAgent-20 results demonstrate a local model setup on **Apple Silicon with LM Studio**. The released GGUF also provides a path to compatible llama.cpp deployments, letting the model participate in coding and tool loops close to the user's workspace.
+- **An adjustable memory budget.** Quantization, context length, and offload settings provide practical deployment controls. A **<6 GB RAM/VRAM footprint** remains a target for suitable lower-bit configurations and modest contexts, rather than a measured peak-memory result. The currently available Q8_0 release does not establish that total-memory target; weights, context/KV cache, runtime buffers, and any projector all contribute to usage.
+- **Bundled MTP.** A compatible backend can use the retained MTP head for speculative drafting. The coding evaluation uses a maximum draft length of **2**. Throughput benefits depend on the device, workload, acceptance rate, and runtime; this card does not yet report a measured 4B speedup.
+
+### Development direction
+
+Future development is aimed at extending this local foundation into more dependable edge agents:
+
+1. **Broader device measurements.** Evaluate peak memory, first-token latency, sustained generation speed, and power use across more local hardware, with explicit model, quantization, and context settings.
+2. **Efficient deployment choices.** Explore lower-bit releases and device-specific MTP settings while checking that coding and tool-use quality remain useful.
+3. **Stronger workspace execution.** Improve stored procedures, file coordination, and recovery in multi-step tasks, guided by the capability gaps in HermesAgent-20.
+
+These are development priorities. The current evidence covers the reported coding evaluations and Apple Silicon agent runs; broader device coverage, energy efficiency, and future quantizations remain work to validate.
+
+## 🍎 6. Practical Agent Evaluation — HermesAgent-20
+
+[**BenchLocal**](https://github.com/stevibe/BenchLocal) provides the **HermesAgent-20** suite for evaluating practical agent workflows. The reported local environment is an **Apple Silicon Mac with LM Studio**.
+
+The overall composite score is **70 for Kestrel-4B-Coder**, compared with **63 for Qwen3.5-4B**. The supplied baseline screenshot identifies its model as `qwen3.5-4b-mlx via LM Studio` and shows one run with **20 / 20 result records**.
+
+![HermesAgent-20 overall comparison: Qwen3.5-4B scores 63 and Kestrel-4B-Coder scores 70](assets/benchlocal_scores.png)
+
+The breakdown shows higher reported **Memory & Recall** and **Delegation, Recovery & Boundaries** scores for Kestrel, with equal **Scheduling & Delivery** scores. Qwen scores higher on **Skills & Procedural Memory** and **Workspace Orchestration**, which provide clear targets for future development.
+
+![HermesAgent-20 capability comparison for Qwen3.5-4B and Kestrel-4B-Coder](assets/benchlocal_capabilities.png)
+
+<details>
+<summary>Text version of the HermesAgent-20 comparison</summary>
+
+| Overall / capability | Qwen3.5-4B | Kestrel-4B-Coder |
+| :--- | :---: | :---: |
+| **HermesAgent-20 overall** | **63** | **70** |
+| Memory & Recall | 63 | **88** |
+| Delegation, Recovery & Boundaries | 21 | **76** |
+| Scheduling & Delivery | 75 | **75** |
+| Skills & Procedural Memory | 88 | **68** |
+| Workspace Orchestration | 70 | **45** |
+
+</details>
+
+These are **BenchLocal composite scores, not pass rates**. The overall score is reported separately from the five selected dimensions. The baseline screenshot's 20 / 20 label counts completed result records, rather than successful scenarios. The baseline uses MLX; the exact Kestrel backend and quantization, matched run settings, and independent execution-verifier results are not documented in the supplied summary. Treat this as an observational local comparison, rather than a controlled measurement of the fine-tuning effect.
+
+Thank you to [**Steven Lei (`stevenlei`)**](https://github.com/stevenlei) for creating [BenchLocal](https://github.com/stevibe/BenchLocal).
+
+## 🚀 7. Getting Started
+
+Install the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli), then download the Q8_0 model. For a private repository, first authenticate with an account that has access using `hf auth login`.
+
+```bash
+python -m pip install -U huggingface_hub
+
+hf download Jackrong/Kestrel-4B-Coder-GGUF \
+  Kestrel-4B-Coder-MTP-Q8_0.gguf \
+  --local-dir ./Kestrel-4B-Coder-GGUF
+```
+
+Use a recent [llama.cpp build](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) with Qwen3.5 and **draft-MTP** support. This starts a local OpenAI-compatible server with the coding sampling settings and a maximum draft length of two:
+
+```bash
+llama-server \
+  --model ./Kestrel-4B-Coder-GGUF/Kestrel-4B-Coder-MTP-Q8_0.gguf \
+  --alias Kestrel-4B-Coder \
+  --spec-type draft-mtp \
+  --spec-draft-n-max 2 \
+  --ctx-size 8192 \
+  --n-gpu-layers 99 \
+  --jinja \
+  --reasoning on \
+  --temp 0.2 \
+  --top-p 0.95 \
+  --host 127.0.0.1 \
+  --port 8080
+```
+
+The **8,192-token context is a starting example**, not a benchmark-reproduction setting. Adjust context and GPU offload to your device. CUDA requires a CUDA-enabled build; Apple Silicon uses a compatible Metal build. Keep the included chat template and pass sampling settings explicitly when calling the server. Use thinking enabled for the reported coding setup. MTP requires both the bundled weights and backend support; check the server startup log to confirm it is active.
+
+For image input, also download the repository's exact projector filename and add `--mmproj ./Kestrel-4B-Coder-GGUF/mmproj-F16.gguf` to a compatible multimodal server:
+
+```bash
+hf download Jackrong/Kestrel-4B-Coder-GGUF \
+  mmproj-F16.gguf \
+  --local-dir ./Kestrel-4B-Coder-GGUF
+```
+
+The projector is optional for text-only coding. Vision quality is not evaluated in this card. For local agent use through LM Studio, select the model in the application and configure your agent to use its local server; MTP availability depends on the selected runtime.
+
+## 🧭 8. Intended Use and Limitations
+
+Intended uses include coding assistance on personal devices, local debugging, small refactors, tool-assisted development, and compact agent experiments. The longer-term direction is practical edge agents that work within explicit memory and compute budgets. Review generated changes and run tests before relying on them.
+
+HumanEval measures bounded programming tasks and does not establish repository-scale software engineering ability. HermesAgent-20 points to weaker workspace orchestration and procedural-memory scores relative to the baseline. Difficult multi-file tasks, long interaction histories, and ambiguous tool feedback can still lead to mistakes or unsuccessful actions.
+
+The reported results apply to their stated settings and scoring policies. They do not establish universal coding accuracy, long-context reliability, vision quality, or safe autonomous operation. Wider edge-device compatibility, sub-6 GB peak memory, and energy savings require device-specific validation. Quantization and runtime choices can affect behavior; the 4B results should not be extrapolated from the 9B model's measured reasoning savings.
+
+## 🙏 9. Acknowledgements and Citation
+
+Thanks to the [Qwen team](https://huggingface.co/Qwen/Qwen3.5-4B) for the foundation model; [Unsloth](https://unsloth.ai/) and its [open-source framework](https://github.com/unslothai/unsloth) for fine-tuning tools; [Kyle Hessling](https://x.com/KyleHessling1) for collaboration and support on Kestrel; and [NVIDIA](https://huggingface.co/nvidia) for the Nemotron data ecosystem. We also thank [EvalPlus](https://github.com/evalplus/evalplus), [llama.cpp](https://github.com/ggml-org/llama.cpp), [LM Studio](https://lmstudio.ai/), and [BenchLocal](https://github.com/stevibe/BenchLocal).
+
+**Base-model license:** Apache-2.0. See the release repository's license and provenance information for the distributed artifacts.
+
+```bibtex
+@misc{jackrong2026kestrel4bcoder,
+  title        = {Kestrel-4B-Coder},
+  author       = {Jackrong},
+  year         = {2026},
+  howpublished = {Hugging Face},
+  url          = {https://huggingface.co/Jackrong/Kestrel-4B-Coder}
+}
+```

added_tokens.json

@@ -0,0 +1,35 @@
+{
+  "</think>": 248069,
+  "</tool_call>": 248059,
+  "</tool_response>": 248067,
+  "<think>": 248068,
+  "<tool_call>": 248058,
+  "<tool_response>": 248066,
+  "<tts_pad>": 248072,
+  "<tts_text_bos>": 248073,
+  "<tts_text_bos_single>": 248075,
+  "<tts_text_eod>": 248074,
+  "<|audio_end|>": 248071,
+  "<|audio_pad|>": 248076,
+  "<|audio_start|>": 248070,
+  "<|box_end|>": 248050,
+  "<|box_start|>": 248049,
+  "<|endoftext|>": 248044,
+  "<|file_sep|>": 248065,
+  "<|fim_middle|>": 248061,
+  "<|fim_pad|>": 248063,
+  "<|fim_prefix|>": 248060,
+  "<|fim_suffix|>": 248062,
+  "<|im_end|>": 248046,
+  "<|im_start|>": 248045,
+  "<|image_pad|>": 248056,
+  "<|object_ref_end|>": 248048,
+  "<|object_ref_start|>": 248047,
+  "<|quad_end|>": 248052,
+  "<|quad_start|>": 248051,
+  "<|repo_name|>": 248064,
+  "<|video_pad|>": 248057,
+  "<|vision_end|>": 248054,
+  "<|vision_pad|>": 248055,
+  "<|vision_start|>": 248053
+}

chat_template.jinja

@@ -0,0 +1,158 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set num_sys = 0 %}
+{%- set merged_system = '' %}
+{%- if messages[0].role == 'system' or messages[0].role == 'developer' %}
+    {%- set first = render_content(messages[0].content, false, true)|trim %}
+    {%- if messages|length > 1 and (messages[1].role == 'system' or messages[1].role == 'developer') %}
+        {%- set second = render_content(messages[1].content, false, true)|trim %}
+        {%- set merged_system = first + '\n' + second %}
+        {%- set num_sys = 2 %}
+    {%- else %}
+        {%- set merged_system = first %}
+        {%- set num_sys = 1 %}
+    {%- endif %}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if merged_system %}
+        {{- '\n\n' + merged_system }}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if merged_system %}
+        {{- '<|im_start|>system\n' + merged_system + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- for message in messages %}
+    {%- if loop.index0 >= num_sys and message.role != "system" and message.role != "developer" %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is mapping %}
+                    {%- for args_name in tool_call.arguments %}
+                        {%- set args_value = tool_call.arguments[args_name] %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}
+{#- Unsloth fixes - developer role, tool calling #}

config.json

@@ -0,0 +1,124 @@
+{
+    "architectures": [
+        "Qwen3_5ForConditionalGeneration"
+    ],
+    "eos_token_id": 248044,
+    "image_token_id": 248056,
+    "model_name": "unsloth/Qwen3.5-4B",
+    "model_type": "qwen3_5",
+    "mtp_num_hidden_layers": 1,
+    "num_nextn_predict_layers": 1,
+    "pad_token_id": 248055,
+    "quantization": {
+        "group_size": 64,
+        "bits": 4,
+        "mode": "affine"
+    },
+    "quantization_config": {
+        "group_size": 64,
+        "bits": 4,
+        "mode": "affine"
+    },
+    "text_config": {
+        "attention_bias": false,
+        "attention_dropout": 0.0,
+        "attn_output_gate": true,
+        "bos_token_id": null,
+        "torch_dtype": "bfloat16",
+        "eos_token_id": 248044,
+        "full_attention_interval": 4,
+        "head_dim": 256,
+        "hidden_act": "silu",
+        "hidden_size": 2560,
+        "initializer_range": 0.02,
+        "intermediate_size": 9216,
+        "layer_types": [
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention",
+            "linear_attention",
+            "linear_attention",
+            "linear_attention",
+            "full_attention"
+        ],
+        "linear_conv_kernel_dim": 4,
+        "linear_key_head_dim": 128,
+        "linear_num_key_heads": 16,
+        "linear_num_value_heads": 32,
+        "linear_value_head_dim": 128,
+        "mamba_ssm_dtype": "float32",
+        "max_position_embeddings": 262144,
+        "mlp_only_layers": [],
+        "model_type": "qwen3_5_text",
+        "mtp_num_hidden_layers": 1,
+        "mtp_use_dedicated_embeddings": false,
+        "num_attention_heads": 16,
+        "num_hidden_layers": 32,
+        "num_key_value_heads": 4,
+        "pad_token_id": null,
+        "partial_rotary_factor": 0.25,
+        "rms_norm_eps": 1e-06,
+        "rope_parameters": {
+            "mrope_interleaved": true,
+            "mrope_section": [
+                11,
+                11,
+                10
+            ],
+            "partial_rotary_factor": 0.25,
+            "rope_theta": 10000000,
+            "rope_type": "default"
+        },
+        "tie_word_embeddings": true,
+        "use_cache": false,
+        "vocab_size": 248320
+    },
+    "tie_word_embeddings": true,
+    "unsloth_fixed_mtp": true,
+    "unsloth_version": "2026.8.19",
+    "video_token_id": 248057,
+    "vision_config": {
+        "deepstack_visual_indexes": [],
+        "depth": 24,
+        "torch_dtype": "bfloat16",
+        "hidden_act": "gelu_pytorch_tanh",
+        "hidden_size": 1024,
+        "in_channels": 3,
+        "initializer_range": 0.02,
+        "intermediate_size": 4096,
+        "model_type": "qwen3_5",
+        "num_heads": 16,
+        "num_position_embeddings": 2304,
+        "out_hidden_size": 2560,
+        "patch_size": 16,
+        "spatial_merge_size": 2,
+        "temporal_patch_size": 2
+    },
+    "vision_end_token_id": 248054,
+    "vision_start_token_id": 248053
+}

configuration.json

@@ -0,0 +1 @@
+{"framework":"Pytorch","task":"visual-question-answering"}

generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": 248044,
+  "pad_token_id": 248055,
+  "transformers_version": "5.5.0",
+  "use_cache": true
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7cbfbf9af802e037895f93365058410642d19de9790bcbd3ed31f8be88abcb22
+size 3034301037

preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+    "size": {
+        "longest_edge": 16777216,
+        "shortest_edge": 65536
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "image_processor_type": "Qwen2VLImageProcessorFast"
+}

processor_config.json

@@ -0,0 +1,64 @@
+{
+  "image_processor": {
+    "data_format": "channels_first",
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 16777216,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "data_format": "channels_first",
+    "default_to_square": true,
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "max_video_tokens": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

special_tokens_map.json

@@ -0,0 +1,38 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<|image_pad|>",
+  "pad_token": {
+    "content": "<|vision_pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer_config.json

@@ -0,0 +1,300 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|vision_pad|>",
+  "padding_side": "right",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>",
+  "added_tokens_decoder": {
+    "248044": {
+      "content": "<|endoftext|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248045": {
+      "content": "<|im_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248046": {
+      "content": "<|im_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248047": {
+      "content": "<|object_ref_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248048": {
+      "content": "<|object_ref_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248049": {
+      "content": "<|box_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248050": {
+      "content": "<|box_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248051": {
+      "content": "<|quad_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248052": {
+      "content": "<|quad_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248053": {
+      "content": "<|vision_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248054": {
+      "content": "<|vision_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248055": {
+      "content": "<|vision_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248056": {
+      "content": "<|image_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248057": {
+      "content": "<|video_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248058": {
+      "content": "<tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248059": {
+      "content": "</tool_call>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248060": {
+      "content": "<|fim_prefix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248061": {
+      "content": "<|fim_middle|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248062": {
+      "content": "<|fim_suffix|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248063": {
+      "content": "<|fim_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248064": {
+      "content": "<|repo_name|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248065": {
+      "content": "<|file_sep|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248066": {
+      "content": "<tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248067": {
+      "content": "</tool_response>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248068": {
+      "content": "<think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248069": {
+      "content": "</think>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": false
+    },
+    "248070": {
+      "content": "<|audio_start|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248071": {
+      "content": "<|audio_end|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248072": {
+      "content": "<tts_pad>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248073": {
+      "content": "<tts_text_bos>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248074": {
+      "content": "<tts_text_eod>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248075": {
+      "content": "<tts_text_bos_single>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    },
+    "248076": {
+      "content": "<|audio_pad|>",
+      "single_word": false,
+      "lstrip": false,
+      "rstrip": false,
+      "normalized": false,
+      "special": true
+    }
+  }
+}

video_preprocessor_config.json

@@ -0,0 +1,21 @@
+{
+    "size": {
+        "longest_edge": 25165824,
+        "shortest_edge": 4096
+    },
+    "patch_size": 16,
+    "temporal_patch_size": 2,
+    "merge_size": 2,
+    "image_mean": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "image_std": [
+        0.5,
+        0.5,
+        0.5
+    ],
+    "processor_class": "Qwen3VLProcessor",
+    "video_processor_type": "Qwen3VLVideoProcessor"
+}