Jeff v1.3: jeff-base

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,247 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3.5-0.8B
+base_model_relation: finetune
+library_name: transformers
+pipeline_tag: zero-shot-classification
+language:
+- en
+tags:
+- jeff
+- decision-model
+- calibration
+- system-1
+- local
+---
+
+# jeff-base
+
+**Jeff v1.3: the base model for Jeff's LoRA adapters.** Jeff is a small decision model, a fine-tune of
+[Qwen3.5-0.8B](https://huggingface.co/Qwen/Qwen3.5-0.8B). You describe a situation (the *state*) and list the options
+in plain words; Jeff returns a calibrated probability for each option from one forward pass, with no generated text to
+parse. This repository is versioned by tag: use revision **`v1.3`**.
+
+Catalogue, results and docs: **[jeffhub.ai](https://jeffhub.ai)**.
+
+**Jeff v1.3 is a change in direction. Zero-shot on everything is no longer the goal: the base is always meant to be used
+with an adapter.** The base is the foundation the adapters are trained on, and an adapter is where Jeff becomes good at
+a task. If you want zero-shot use without an adapter, use Jeff v1.2
+([mstrasser/Jeff-Qwen3.5-0.8B](https://huggingface.co/mstrasser/Jeff-Qwen3.5-0.8B), revision `v1.2`).
+
+## The base: live-last
+
+The v1.3 base is trained on the same data as v1.2, with one change: the **live-last** prompt layout. The fixed part of
+the prompt (instructions and options) comes first and the changing input comes last, so prefix caching works and
+repeated decisions over the same options get faster.
+
+The price: the model now reads all the options before it sees the input, and a 0.8B model is much worse at going back
+over a long option list than at reading the options with the input already in mind. On the general panel (4,599
+questions) the two bases are level: 78.6% against 78.8%, calibration error 0.028 against 0.024. On unfamiliar tasks
+with long option lists, the base on its own falls apart:
+
+| Task (no adapter) | Options | v1.2 base | v1.3 base |
+|---|--:|--:|--:|
+| [support-intents](https://jeffhub.ai/adapters/support-intents) | 7–64 | 85.1% | 24.2% |
+| [legal-clauses](https://jeffhub.ai/adapters/legal-clauses) | 100 | 66.0% | 7.4% |
+| [tools](https://jeffhub.ai/adapters/tools) | 4–136 | 57.8% | 30.2% |
+| [triage](https://jeffhub.ai/adapters/triage) | 2–36 | 67.1% | 47.6% |
+
+With the right adapter, almost all of it comes back: v1.3 + adapter is within −0.7 to +0.3 points of v1.2 + adapter on
+every task except legal-clauses (83.6% against 85.7%). Once the adapter has learned the options, putting them first
+costs almost nothing, and the fixed part of the prompt can be cached. We decided the speed-up was worth it.
+
+## What each adapter adds
+
+Each adapter is scored on its full held-out test set, which it never trained on, three ways on the same rows: the
+untrained Qwen3.5-0.8B that Jeff is built from, the Jeff v1.3 base alone, and the v1.3 base with the adapter. Across the
+13 measured adapters, mean accuracy goes from 30.4% untrained to 38.6% for the base alone and 92.8% with the adapter.
+
+| Adapter | Test rows | Qwen3.5-0.8B untrained | Jeff base v1.3 alone | Jeff base v1.3 + adapter |
+|---|--:|--:|--:|--:|
+| [aml](https://huggingface.co/mstrasser/jeff-adapter-aml) | 5,120 | 36.0% · 0.022 | 40.5% · 0.103 | 95.0% · 0.012 |
+| [emotion](https://huggingface.co/mstrasser/jeff-adapter-emotion) | 5,408 | 12.6% · 0.045 | 25.3% · 0.080 | 60.5% · 0.018 |
+| [ground](https://huggingface.co/mstrasser/jeff-adapter-ground) | 4,160 | 28.9% · 0.061 | 50.9% · 0.079 | 96.6% · 0.007 |
+| [guard](https://huggingface.co/mstrasser/jeff-adapter-guard) | 6,552 | 43.8% · 0.064 | 49.4% · 0.158 | 98.2% · 0.004 |
+| [legal-clauses](https://huggingface.co/mstrasser/jeff-adapter-legal-clauses) | 9,895 | 12.5% · 0.094 | 7.4% · 0.039 | 83.6% · 0.011 |
+| [nav](https://huggingface.co/mstrasser/jeff-adapter-nav) | 3,300 | 12.6% · 0.038 | 13.6% · 0.156 | 97.3% · 0.006 |
+| [sanctions](https://huggingface.co/mstrasser/jeff-adapter-sanctions) | 4,909 | 34.4% · 0.010 | 68.3% · 0.080 | 100.0% · 0.001 |
+| [soc](https://huggingface.co/mstrasser/jeff-adapter-soc) | 4,929 | 22.1% · 0.011 | 33.8% · 0.032 | 94.1% · 0.014 |
+| [spam](https://huggingface.co/mstrasser/jeff-adapter-spam) | 3,897 | 58.1% · 0.047 | 69.8% · 0.046 | 98.1% · 0.006 |
+| [support-intents](https://huggingface.co/mstrasser/jeff-adapter-support-intents) | 5,577 | 33.9% · 0.164 | 24.2% · 0.095 | 96.3% · 0.003 |
+| [tools](https://huggingface.co/mstrasser/jeff-adapter-tools) | 5,157 | 17.9% · 0.063 | 30.2% · 0.016 | 97.2% · 0.007 |
+| [trading-desk](https://huggingface.co/mstrasser/jeff-adapter-trading-desk) | 5,000 | 38.6% · 0.015 | 41.1% · 0.115 | 98.1% · 0.010 |
+| [triage](https://huggingface.co/mstrasser/jeff-adapter-triage) | 7,256 | 44.1% · 0.098 | 47.6% · 0.059 | 91.5% · 0.015 |
+| [code](https://huggingface.co/mstrasser/jeff-adapter-code) | not measured yet | | | |
+| [code-router](https://huggingface.co/mstrasser/jeff-adapter-code-router) | not measured yet | | | |
+
+Each cell: accuracy · calibration error (ECE, 15 bins, after each model's own fitted temperature; lower is better, 0
+is perfect). Calibration error measures how far the stated confidence is from the real hit rate: 0.01 means the stated
+confidence is, on average, about 1 percentage point away from how often those answers are right.
+
+**Jeff-Code.** The [code](https://huggingface.co/mstrasser/jeff-adapter-code) and
+[code-router](https://huggingface.co/mstrasser/jeff-adapter-code-router) adapters make two decisions for Qwen3.8-27B in
+the Jeff-Code coding agent ([github.com/firelex/jeff-code](https://github.com/firelex/jeff-code)). With Jeff's thinking
+threshold at 0.6 (step threshold 0.40), Jeff-Code matches Qwen3.8-27B's pass rate on five benchmarks: pooled 53.4%
+against 53.1% over 493 tasks, run side by side in paired blocks. It needs about 20% less time per session on average on
+the big benchmarks, SWE-rebench and Terminal-Bench 2.0; the typical session is 30–45% faster. These numbers were
+supplied by the maintainers; the detailed result files are still to come. [The details](https://jeffhub.ai/adapters/code).
+
+### v1.3 + adapter against v1.2 + adapter
+
+Each pair is measured on exactly the same test rows. Where an adapter's v1.3 test set changed, the comparison uses its
+copy of the v1.2 test (named in brackets).
+
+| Adapter test set | Test rows | v1.2 + adapter | v1.3 + adapter | Change (points) |
+|---|--:|--:|--:|--:|
+| emotion | 5,408 | 60.6% | 60.5% | −0.1 |
+| ground | 4,160 | 97.0% | 96.6% | −0.4 |
+| guard | 6,552 | 98.4% | 98.2% | −0.2 |
+| legal-clauses | 9,895 | 85.7% | 83.6% | −2.1 |
+| nav (heldout_synthetic-v12) | 3,300 | 97.0% | 97.3% | +0.3 |
+| spam (test-v12) | 3,603 | 98.4% | 98.1% | −0.3 |
+| support-intents | 5,577 | 96.8% | 96.3% | −0.5 |
+| tools | 5,157 | 97.9% | 97.2% | −0.7 |
+| triage | 7,256 | 91.8% | 91.5% | −0.3 |
+
+All results and their sources: [jeffhub.ai/results](https://jeffhub.ai/results), and in one file,
+[jeffhub-v1.3.json](https://jeffhub.ai/data/jeffhub-v1.3.json).
+
+## The adapters
+
+Adapters are not merged into the base. You load one base and all the adapters you need, and pick one per request. Each
+adapter records the exact base it was trained on, and the server refuses an adapter trained on a different one, so a
+v1.2 adapter does not load on v1.3. All nine v1.2 adapters are retrained on v1.3, with about 10% of the base model's own
+training data mixed in as a precaution (its effect has not been measured).
+
+| Adapter | Repository | LoRA GGUF |
+|---|---|---|
+| aml | [mstrasser/jeff-adapter-aml](https://huggingface.co/mstrasser/jeff-adapter-aml) | [jeff-adapter-aml-gguf](https://huggingface.co/mstrasser/jeff-adapter-aml-gguf) |
+| code | [mstrasser/jeff-adapter-code](https://huggingface.co/mstrasser/jeff-adapter-code) | [jeff-adapter-code-gguf](https://huggingface.co/mstrasser/jeff-adapter-code-gguf) |
+| code-router | [mstrasser/jeff-adapter-code-router](https://huggingface.co/mstrasser/jeff-adapter-code-router) | [jeff-adapter-code-router-gguf](https://huggingface.co/mstrasser/jeff-adapter-code-router-gguf) (Q8_0 base only) |
+| emotion | [mstrasser/jeff-adapter-emotion](https://huggingface.co/mstrasser/jeff-adapter-emotion) | [jeff-adapter-emotion-gguf](https://huggingface.co/mstrasser/jeff-adapter-emotion-gguf) |
+| ground | [mstrasser/jeff-adapter-ground](https://huggingface.co/mstrasser/jeff-adapter-ground) | [jeff-adapter-ground-gguf](https://huggingface.co/mstrasser/jeff-adapter-ground-gguf) |
+| guard | [mstrasser/jeff-adapter-guard](https://huggingface.co/mstrasser/jeff-adapter-guard) | [jeff-adapter-guard-gguf](https://huggingface.co/mstrasser/jeff-adapter-guard-gguf) |
+| legal-clauses | [mstrasser/jeff-adapter-legal-clauses](https://huggingface.co/mstrasser/jeff-adapter-legal-clauses) | [jeff-adapter-legal-clauses-gguf](https://huggingface.co/mstrasser/jeff-adapter-legal-clauses-gguf) |
+| nav | [mstrasser/jeff-adapter-nav](https://huggingface.co/mstrasser/jeff-adapter-nav) | [jeff-adapter-nav-gguf](https://huggingface.co/mstrasser/jeff-adapter-nav-gguf) |
+| sanctions (CC BY-NC 4.0) | [mstrasser/jeff-adapter-sanctions](https://huggingface.co/mstrasser/jeff-adapter-sanctions) | [jeff-adapter-sanctions-gguf](https://huggingface.co/mstrasser/jeff-adapter-sanctions-gguf) |
+| soc (CC BY-NC 4.0) | [mstrasser/jeff-adapter-soc](https://huggingface.co/mstrasser/jeff-adapter-soc) | [jeff-adapter-soc-gguf](https://huggingface.co/mstrasser/jeff-adapter-soc-gguf) |
+| spam | [mstrasser/jeff-adapter-spam](https://huggingface.co/mstrasser/jeff-adapter-spam) | [jeff-adapter-spam-gguf](https://huggingface.co/mstrasser/jeff-adapter-spam-gguf) |
+| support-intents | [mstrasser/jeff-adapter-support-intents](https://huggingface.co/mstrasser/jeff-adapter-support-intents) | [jeff-adapter-support-intents-gguf](https://huggingface.co/mstrasser/jeff-adapter-support-intents-gguf) |
+| tools | [mstrasser/jeff-adapter-tools](https://huggingface.co/mstrasser/jeff-adapter-tools) | [jeff-adapter-tools-gguf](https://huggingface.co/mstrasser/jeff-adapter-tools-gguf) |
+| trading-desk | [mstrasser/jeff-adapter-trading-desk](https://huggingface.co/mstrasser/jeff-adapter-trading-desk) | [jeff-adapter-trading-desk-gguf](https://huggingface.co/mstrasser/jeff-adapter-trading-desk-gguf) |
+| triage | [mstrasser/jeff-adapter-triage](https://huggingface.co/mstrasser/jeff-adapter-triage) | [jeff-adapter-triage-gguf](https://huggingface.co/mstrasser/jeff-adapter-triage-gguf) |
+
+In a request, an adapter is named by its short name (`"model": "soc"`, `"model": "code"`): on a Jeff server, names
+starting with "jeff" mean the base model.
+
+## How to use it
+
+Adapter serving arrives with the next Jeff release; until then, these commands need the `feat/lora` branch of
+[firelex/jeff](https://github.com/firelex/jeff).
+
+```bash
+git clone https://github.com/firelex/jeff && cd jeff
+uv sync --no-default-groups --extra lora                # CPU; add --extra cuda (NVIDIA GPU) or --extra mac (Apple silicon)
+uv run --no-default-groups hf download mstrasser/jeff-base --revision v1.3 --local-dir checkpoints/jeff-base
+uv run --no-default-groups hf download mstrasser/jeff-adapter-support-intents --revision v1.3 \
+  --local-dir adapters/support-intents
+JEFF_CHECKPOINT=checkpoints/jeff-base JEFF_ADAPTERS=adapters/ PORT=8765 \
+  uv run --no-default-groups jeff-serve                 # on a Mac, add JEFF_BACKEND=mlx
+```
+
+Each adapter lives in its own folder inside one adapters folder; the folder's name is the name you use in requests.
+Name the adapter as the model:
+
+```bash
+curl -s localhost:8765/v1/systemone -H 'content-type: application/json' -d '{
+  "model": "support-intents",
+  "state": {"service": "Customer support chat of an online shop",
+            "message": "I sent the jacket back two weeks ago and still have not seen the money."},
+  "questions": {
+    "intent": {"type": "choice",
+               "instructions": "What does the customer want? Choose the request that best matches what the customer is asking for in their message.",
+               "criteria": {"track_refund": "Check the status of a refund they are expecting.",
+                            "get_refund": "Get their money back for a purchase.",
+                            "track_order": "Find out where their order is or its current status."}}
+  }
+}'
+```
+
+Guides: [getting started](https://jeffhub.ai/docs/getting-started), [request format](https://jeffhub.ai/docs/request-format),
+[serving adapters](https://jeffhub.ai/docs/serving-adapters).
+
+**llama.cpp.** v1.3 also ships as GGUF, in Q8_0 and Q4_K_M:
+[mstrasser/jeff-base-gguf](https://huggingface.co/mstrasser/jeff-base-gguf), plus one small LoRA GGUF per adapter. See
+[Running Jeff with llama.cpp](https://jeffhub.ai/docs/llama-cpp).
+
+## What stays fixed for the life of v1.3
+
+- The [request format](https://jeffhub.ai/docs/request-format): state, questions and instructions, with the changing
+  state field last.
+- The option rules: named options, keys never bare numbers.
+- The answer format: a probability for every option.
+
+A data set written to the [data guidelines](https://jeffhub.ai/docs/data-guidelines) trains on v1.3 as it is.
+
+## Files
+
+| File | What it is |
+|---|---|
+| `model.safetensors` | The weights (sha256 `d324dd6c9bb61b30af65564135b33f6892c30a9b2bd22667b2e09b9c8118cf77`; every v1.3 adapter checks it) |
+| `readout.safetensors` | Jeff's readout over the answer codes |
+| `decision_config.json` | Answer codes and their token ids, temperature and prompt layout (`live-last`) |
+| `config.json`, `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`, `processor_config.json` | The Qwen3.5 configuration and tokenizer |
+
+## Training
+
+| | |
+|---|---|
+| Built on | Qwen/Qwen3.5-0.8B |
+| Data | The v1.2 base training data, unchanged |
+| Change from v1.2 | The live-last prompt layout |
+| Checkpoint | `ll-v12-final` (run `ll-v12`, step 1113) |
+
+## Not measured yet
+
+Nothing on JeffHub is estimated. These are still to come for v1.3:
+
+- code and code-router: test scores (untrained, v1.3 base, v1.3 + adapter);
+- jeff-serve speed and GPU memory with the v1.3 base and adapters, with prompt reuse;
+- calibration charts, commonest confusions and accuracy per answer for each v1.3 adapter;
+- example responses recorded from the v1.3 adapters;
+- Jeff-Code: the detailed result files behind the maintainer-supplied numbers;
+- Jeff's memory on a Mac.
+
+## Limitations
+
+- Use it with an adapter. Without one, the v1.3 base is weak on long, unfamiliar option lists (table above). For
+  zero-shot use, use v1.2.
+- Small models don't reason. Expect fast, calibrated choices between the options you describe, not multi-step
+  reasoning, and no generated text.
+- English and text only.
+
+## Licence and data
+
+Weights: **Apache-2.0**, as for v1.2. The model is a fine-tune of Qwen3.5-0.8B by the Qwen team (Alibaba Cloud).
+
+Training data: the same data as the v1.2 base, which mixes public data sets under various licences, some of them
+share-alike (CC BY-SA), with code-built and synthetic questions. The training data is not released. The v1.2 data
+sources and their licences are listed in
+[docs/data-sources.md](https://github.com/firelex/jeff/blob/main/docs/data-sources.md) in the Jeff repository.
+
+**To confirm:** JeffHub does not yet list the base model's data sources and their licences for v1.3; the list above is
+the v1.2 one, which the v1.3 base reuses unchanged.
+
+Each adapter's own data sources and licences, including non-commercial restrictions (sanctions and soc are
+CC BY-NC 4.0), are on its model card and its [JeffHub page](https://jeffhub.ai).
+
+## Links
+
+- Release notes: [Jeff v1.3: what changed](https://jeffhub.ai/docs/release-notes-v1-3)
+- Results: [jeffhub.ai/results](https://jeffhub.ai/results)
+- Code and server: [github.com/firelex/jeff](https://github.com/firelex/jeff)
+- GGUF: [mstrasser/jeff-base-gguf](https://huggingface.co/mstrasser/jeff-base-gguf)
+- Jeff v1.2 (zero-shot): [mstrasser/Jeff-Qwen3.5-0.8B](https://huggingface.co/mstrasser/Jeff-Qwen3.5-0.8B), revision `v1.2`
+
+Jeff is an independent project. It uses the same request format as Jev but is not affiliated with or endorsed by
+TypeSafe, the makers of Jev.

chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if loop.index0 > ns.last_query_index %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | tojson | safe if args_value is mapping or (args_value is sequence and args_value is not string) else args_value | string %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is true %}
+        {{- '<think>\n' }}
+    {%- else %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- endif %}
+{%- endif %}

config.json

@@ -0,0 +1,105 @@
+{
+  "architectures": [
+    "Qwen3_5Model"
+  ],
+  "dtype": "bfloat16",
+  "image_token_id": 248056,
+  "model_type": "qwen3_5",
+  "text_config": {
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_output_gate": true,
+    "bos_token_id": null,
+    "dtype": "bfloat16",
+    "eos_token_id": 248044,
+    "full_attention_interval": 4,
+    "head_dim": 256,
+    "hidden_act": "silu",
+    "hidden_size": 1024,
+    "initializer_range": 0.02,
+    "intermediate_size": 3584,
+    "layer_types": [
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention",
+      "linear_attention",
+      "linear_attention",
+      "linear_attention",
+      "full_attention"
+    ],
+    "linear_conv_kernel_dim": 4,
+    "linear_key_head_dim": 128,
+    "linear_num_key_heads": 16,
+    "linear_num_value_heads": 16,
+    "linear_value_head_dim": 128,
+    "mamba_ssm_dtype": "float32",
+    "max_position_embeddings": 262144,
+    "mlp_only_layers": [],
+    "model_type": "qwen3_5_text",
+    "mtp_num_hidden_layers": 1,
+    "mtp_use_dedicated_embeddings": false,
+    "num_attention_heads": 8,
+    "num_hidden_layers": 24,
+    "num_key_value_heads": 2,
+    "pad_token_id": null,
+    "partial_rotary_factor": 0.25,
+    "rms_norm_eps": 1e-06,
+    "rope_parameters": {
+      "mrope_interleaved": true,
+      "mrope_section": [
+        11,
+        11,
+        10
+      ],
+      "partial_rotary_factor": 0.25,
+      "rope_theta": 10000000,
+      "rope_type": "default"
+    },
+    "tie_word_embeddings": true,
+    "use_cache": true,
+    "vocab_size": 248320
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.17.0",
+  "video_token_id": 248057,
+  "vision_config": {
+    "deepstack_visual_indexes": [],
+    "depth": 12,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 768,
+    "in_channels": 3,
+    "initializer_range": 0.02,
+    "intermediate_size": 3072,
+    "model_type": "qwen3_5_vision",
+    "num_heads": 12,
+    "num_position_embeddings": 2304,
+    "out_hidden_size": 1024,
+    "patch_size": 16,
+    "rope_parameters": {
+      "rope_theta": 10000.0,
+      "rope_type": "axial"
+    },
+    "spatial_merge_size": 2,
+    "temporal_patch_size": 2
+  },
+  "vision_end_token_id": 248054,
+  "vision_start_token_id": 248053
+}

decision_config.json

@@ -0,0 +1,540 @@
+{
+  "step": 1113,
+  "provenance": {
+    "run": "ll-v12",
+    "git_commit": "016711d9daba54b941a38789be8e8b328e8db1e7",
+    "development": "ebcedc3e726548c08da90ff569b94e4822f23312dfc3ea29b93579cfec3016c7",
+    "temperature": "e735fa4e0df3482752112bc9cdff6af9ddb7be07a0a9f2fb8dd8cf2355042bf6",
+    "train": "f826592e75acdaa9510af5b15d5c944e07628632134cd57d2d6840192696e4ac",
+    "train.py": "9f3fdd0e4caf20f8eff77aa5b49fbc4c60df0b185a405ca4bc7ccf021cff5922",
+    "model.py": "8c5579d3fcd42cb2de4e5a6515259cf4c5177ad7754af9b44c3d7f38b22efcc2",
+    "encoder.py": "6aa794b6c48a586d413e0be4d4ac2a1bb50bd3a83d23439b13c12c5b7dd0aac8",
+    "decoder.py": "9ee8ebbe5adc1ca11821b9f51e511206c6c612aeaa2648aa2388f3604480d2e0",
+    "models.py": "19abbc6be22049c2934a664e93aff0f968b724aa74f7d080e7dc0e367ac0120c",
+    "optim.py": "4fbfc0fee138121f363651eacaff84a672f758ab64d36175569bf00929c9cc5b",
+    "evaluate.py": "6d5c55b52ecf99d10c1522089feac4654eb2d161e90b13e408d085bca248ec41",
+    "types.py": "085d188898f939c581f6eeec2ebf1ab31df64f0aadef2e108cc70f2f42ecb929",
+    "events.py": "989646a80f0ee8695bd7c0e98ff945d04d02a511b9ecf06439048b4df0aceec2",
+    "uv.lock": "69e63d333763a522fd9aa451fa244653bf46a26b48f6216ada175a614471a12e"
+  },
+  "max_options": 254,
+  "format_version": 1,
+  "base_model": "Qwen/Qwen3.5-0.8B",
+  "revision": "2fc06364715b967f1860aea9cf38778875588b17",
+  "codes": [
+    "A",
+    "B",
+    "C",
+    "D",
+    "E",
+    "F",
+    "G",
+    "H",
+    "I",
+    "J",
+    "K",
+    "L",
+    "M",
+    "N",
+    "O",
+    "P",
+    "Q",
+    "R",
+    "S",
+    "T",
+    "U",
+    "V",
+    "W",
+    "X",
+    "Y",
+    "Z",
+    "AA",
+    "AB",
+    "AC",
+    "AD",
+    "AE",
+    "AF",
+    "AG",
+    "AH",
+    "AI",
+    "AJ",
+    "AK",
+    "AL",
+    "AM",
+    "AN",
+    "AO",
+    "AP",
+    "AQ",
+    "AR",
+    "AS",
+    "AT",
+    "AU",
+    "AV",
+    "AW",
+    "AX",
+    "AY",
+    "AZ",
+    "BA",
+    "BB",
+    "BC",
+    "BD",
+    "BE",
+    "BF",
+    "BG",
+    "BH",
+    "BI",
+    "BJ",
+    "BK",
+    "BL",
+    "BM",
+    "BN",
+    "BO",
+    "BP",
+    "BR",
+    "BS",
+    "BT",
+    "BU",
+    "BV",
+    "BW",
+    "BX",
+    "BY",
+    "CA",
+    "CB",
+    "CC",
+    "CD",
+    "CE",
+    "CF",
+    "CG",
+    "CH",
+    "CI",
+    "CK",
+    "CL",
+    "CM",
+    "CN",
+    "CO",
+    "CP",
+    "CR",
+    "CS",
+    "CT",
+    "CU",
+    "CV",
+    "CW",
+    "CX",
+    "CY",
+    "DA",
+    "DB",
+    "DC",
+    "DD",
+    "DE",
+    "DF",
+    "DG",
+    "DH",
+    "DI",
+    "DJ",
+    "DK",
+    "DL",
+    "DM",
+    "DN",
+    "DO",
+    "DP",
+    "DR",
+    "DS",
+    "DT",
+    "DU",
+    "DV",
+    "DW",
+    "DX",
+    "DY",
+    "EA",
+    "EB",
+    "EC",
+    "ED",
+    "EE",
+    "EF",
+    "EG",
+    "EH",
+    "EI",
+    "EK",
+    "EL",
+    "EM",
+    "EN",
+    "EO",
+    "EP",
+    "EQ",
+    "ER",
+    "ES",
+    "ET",
+    "EU",
+    "EV",
+    "EW",
+    "EX",
+    "EZ",
+    "FA",
+    "FB",
+    "FC",
+    "FD",
+    "FE",
+    "FF",
+    "FG",
+    "FH",
+    "FI",
+    "FK",
+    "FL",
+    "FM",
+    "FN",
+    "FO",
+    "FP",
+    "FR",
+    "FS",
+    "FT",
+    "FU",
+    "FW",
+    "FX",
+    "FY",
+    "GA",
+    "GB",
+    "GC",
+    "GD",
+    "GE",
+    "GF",
+    "GG",
+    "GH",
+    "GI",
+    "GL",
+    "GM",
+    "GN",
+    "GO",
+    "GP",
+    "GR",
+    "GS",
+    "GT",
+    "GU",
+    "GV",
+    "GW",
+    "GX",
+    "GY",
+    "HA",
+    "HB",
+    "HC",
+    "HD",
+    "HE",
+    "HF",
+    "HG",
+    "HH",
+    "HI",
+    "HK",
+    "HL",
+    "HM",
+    "HN",
+    "HO",
+    "HP",
+    "HQ",
+    "HR",
+    "HS",
+    "HT",
+    "HU",
+    "HV",
+    "HW",
+    "HX",
+    "HY",
+    "HZ",
+    "IA",
+    "IB",
+    "IC",
+    "ID",
+    "IE",
+    "IF",
+    "IG",
+    "IH",
+    "II",
+    "IJ",
+    "IK",
+    "IL",
+    "IM",
+    "IN",
+    "IO",
+    "IP",
+    "IQ",
+    "IR",
+    "IS",
+    "IT",
+    "IU",
+    "IV",
+    "IW",
+    "IX",
+    "IZ",
+    "JA",
+    "JB",
+    "JC",
+    "JD",
+    "JE",
+    "JI",
+    "JJ",
+    "JK",
+    "JM",
+    "JO",
+    "JP",
+    "JR",
+    "JS",
+    "JT"
+  ],
+  "token_ids": [
+    32,
+    33,
+    34,
+    35,
+    36,
+    37,
+    38,
+    39,
+    40,
+    41,
+    42,
+    43,
+    44,
+    45,
+    46,
+    47,
+    48,
+    49,
+    50,
+    51,
+    52,
+    53,
+    54,
+    55,
+    56,
+    57,
+    5840,
+    1803,
+    1646,
+    1745,
+    13276,
+    8018,
+    1825,
+    28946,
+    15015,
+    29595,
+    11568,
+    939,
+    1354,
+    1058,
+    18183,
+    2456,
+    88898,
+    905,
+    1846,
+    802,
+    33869,
+    7839,
+    14006,
+    2860,
+    2926,
+    22828,
+    6844,
+    9798,
+    4738,
+    9265,
+    11261,
+    19278,
+    36513,
+    93801,
+    8335,
+    14544,
+    85266,
+    9110,
+    28000,
+    15137,
+    4525,
+    25261,
+    12717,
+    7116,
+    17078,
+    14497,
+    57339,
+    74909,
+    52072,
+    19305,
+    4887,
+    12607,
+    3580,
+    6281,
+    2036,
+    9362,
+    8533,
+    2080,
+    10911,
+    2925,
+    3040,
+    9690,
+    27731,
+    8023,
+    6901,
+    8702,
+    6211,
+    1123,
+    16307,
+    18990,
+    64045,
+    63037,
+    33380,
+    6151,
+    3392,
+    5449,
+    3967,
+    1113,
+    5095,
+    51923,
+    49600,
+    17099,
+    51483,
+    17756,
+    16037,
+    8135,
+    30237,
+    5683,
+    9992,
+    7444,
+    5751,
+    10284,
+    20887,
+    59884,
+    52396,
+    16103,
+    67547,
+    18535,
+    8006,
+    7263,
+    1425,
+    6878,
+    14453,
+    9097,
+    44072,
+    76089,
+    68720,
+    2662,
+    2629,
+    923,
+    6548,
+    8924,
+    52194,
+    622,
+    1515,
+    1300,
+    37523,
+    44473,
+    36530,
+    3152,
+    93924,
+    3505,
+    15731,
+    6542,
+    14176,
+    11091,
+    1686,
+    11660,
+    80440,
+    18836,
+    26132,
+    5934,
+    24794,
+    40229,
+    3660,
+    11361,
+    10191,
+    8225,
+    3860,
+    78413,
+    17680,
+    15900,
+    78138,
+    15653,
+    5213,
+    22150,
+    39500,
+    10460,
+    35131,
+    21563,
+    43194,
+    27127,
+    3697,
+    20011,
+    24368,
+    15058,
+    23658,
+    8362,
+    16035,
+    24583,
+    52857,
+    38403,
+    60456,
+    81519,
+    40097,
+    16522,
+    29722,
+    21756,
+    18567,
+    1736,
+    48043,
+    87013,
+    22456,
+    23165,
+    55523,
+    13097,
+    50397,
+    41741,
+    23073,
+    6401,
+    86538,
+    16585,
+    11622,
+    2464,
+    84982,
+    75516,
+    36984,
+    58795,
+    47217,
+    59675,
+    5681,
+    3151,
+    1271,
+    887,
+    5203,
+    2685,
+    1849,
+    72470,
+    5370,
+    74063,
+    27629,
+    1655,
+    1728,
+    669,
+    3682,
+    3191,
+    59865,
+    2712,
+    1580,
+    922,
+    77243,
+    2990,
+    78493,
+    5228,
+    2750,
+    42711,
+    44568,
+    56402,
+    48236,
+    38993,
+    43559,
+    61250,
+    32942,
+    86302,
+    25310,
+    26313,
+    81770,
+    12185,
+    78382
+  ],
+  "temperature": 1.075222461312357,
+  "prompt_layout": "live-last"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d324dd6c9bb61b30af65564135b33f6892c30a9b2bd22667b2e09b9c8118cf77
+size 1706027688

processor_config.json

@@ -0,0 +1,61 @@
+{
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_processor_type": "Qwen2VLImageProcessor",
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "merge_size": 2,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "longest_edge": 262144,
+      "shortest_edge": 65536
+    },
+    "temporal_patch_size": 2
+  },
+  "processor_class": "Qwen3VLProcessor",
+  "video_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "do_sample_frames": true,
+    "fps": 2,
+    "image_mean": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "image_std": [
+      0.5,
+      0.5,
+      0.5
+    ],
+    "max_frames": 768,
+    "max_video_tokens": 768,
+    "merge_size": 2,
+    "min_frames": 4,
+    "patch_size": 16,
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "return_metadata": false,
+    "size": {
+      "longest_edge": 25165824,
+      "shortest_edge": 4096
+    },
+    "temporal_patch_size": 2,
+    "video_processor_type": "Qwen3VLVideoProcessor"
+  }
+}

readout.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b504d7a83a6b72837e6ad99e7da12eab28e5a54f9f637e8c4c8c8997c800c74a
+size 522320

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f32ce20dc35f57a7f9ad1eac03525bd7d30f9df8cea6507e958279cc3657706
+size 19989492

tokenizer_config.json

@@ -0,0 +1,33 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": false,
+  "local_files_only": false,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_token": "<|endoftext|>",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}