kos-v4 initial release (private, under review)

README.md

@@ -0,0 +1,51 @@
+# kos-v4-instruct
+
+**A 3B general instruction-following + tool-calling model, trained from scratch.** The flagship of the kos-v4
+family. Its instruction ability comes from GRPO reinforcement learning on top of a from-scratch clinical base —
+landing **within ~4 points of Qwen2.5-3B-Instruct on IFEval** while holding strong tool-calling.
+
+- **Artifact:** `sft/models/v4_prod2_rl_step1120_hf` (~3.015B params, ~6.4 GB, safetensors)
+- **Base lineage:** kos-v4 from-scratch base → instruction SFT → GRPO RL (prod2 lineage, **step 1120 = the peak**)
+- **Chat template:** ChatML (`<|im_start|>`/`<|im_end|>`, eos = `<|im_end|>`); template bundled in tokenizer_config
+- **Context length:** 24k (whole-document pretraining)
+
+## Evaluations (public benchmarks)
+| benchmark | kos-v4-instruct (step 1120) | Qwen2.5-3B-It | Qwen3-4B-It | kos-v4 base |
+|---|---|---|---|---|
+| **IFEval** AVG4 (EleutherAI lm-eval, `--apply_chat_template`) | **62.7** | 65.4 | 86.6 | 47.7 |
+| — prompt-strict / inst-strict / prompt-loose / inst-loose | 55.8 / 67.0 / 58.4 / 69.7 | — | — | 39.4/54.0/41.2/56.2 |
+| **BFCL** simple / multiple / parallel | **0.81 / 0.73 / 0.62** | — | — | 0.84/0.76/0.65 |
+
+**Reading:** IFEval **62.7** is ~2.7 pts below Qwen2.5-3B (65.4) and far above our base (47.7, **+15.0 from RL**).
+Tool-calling stays strong (81/73/62 vs base 84/76/65) — only ~2 points of tax for the instruction gain.
+
+## How it was trained
+1. **Base** (from scratch): ~180.3B tokens, custom 3B decoder, 32k medical BBPE tokenizer (see family README §1).
+2. **Instruction SFT** on a curated instruction corpus (ChatML).
+3. **GRPO RL (verl)** with **verifiable IFEval-style constraint rewards** (programmatic length/format/keyword/
+   case/section/JSON checks). Official IFEval by GRPO step: base 47.7 → 400 **58.6** → 520 **59.4** → 800 **59.7**
+   → 1040 **61.5** → **1120 62.7 (peak)** → 1160 61.8. We ship the argmax (step 1120); the curve peaks and then
+   softens, so further RL isn't warranted.
+
+## Intended use
+General instruction following, structured output, and function/tool calling in clinical-adjacent workflows.
+For grounded record/guideline reading with abstention, use **kos-v4-grounded**; for retrieval, **kos-v4-retriever**.
+
+## Usage
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+tok = AutoTokenizer.from_pretrained("v4_prod2_rl_step1120_hf")
+model = AutoModelForCausalLM.from_pretrained("v4_prod2_rl_step1120_hf", torch_dtype="bfloat16", device_map="auto")
+msgs = [{"role":"user","content":"List three contraindications for ibuprofen. Answer in exactly 3 bullet points."}]
+ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
+print(tok.decode(model.generate(ids, max_new_tokens=256)[0][ids.shape[1]:], skip_special_tokens=True))
+```
+Serve with vLLM/TGI as a standard `Qwen`-arch causal LM (bf16). Pin **RoPE θ = 25000** on any GGUF/quantized export.
+
+## Limitations
+- **Not a medical-knowledge QA model** (MedQA/MMLU-Med ≈ 30). It follows instructions and calls tools; it does not
+  reliably recall parametric medical facts. Ground it with retrieval.
+- IFEval 62.7 trails Qwen2.5-3B (65.4) and Qwen3-4B (86.6); the GRPO curve peaked at step 1120 (1160 softened to 61.8), so 62.7 is this RL run's ceiling.
+- This checkpoint's clinical **grounding/abstention regressed** under IF-RL (RGB neg-reject ~0.7) — use kos-v4-grounded
+  when non-fabrication on unanswerable questions is required.
+- Public-benchmark performance is not a substitute for validation on real clinical data.

bad_words_ids.json

@@ -0,0 +1,159 @@
+{
+ "note": "v4 sink tokens orphaned in this surgical tokenizer; suppress at generation as belt-and-suspenders (surgical tokenizer already stops them appearing as input keys)",
+ "bad_words_ids": [
+  [
+   509
+  ],
+  [
+   1082
+  ],
+  [
+   1093
+  ],
+  [
+   1641
+  ],
+  [
+   2965
+  ],
+  [
+   3275
+  ],
+  [
+   4126
+  ],
+  [
+   4835
+  ],
+  [
+   5301
+  ],
+  [
+   7666
+  ],
+  [
+   7980
+  ],
+  [
+   8129
+  ],
+  [
+   8208
+  ],
+  [
+   8477
+  ],
+  [
+   10191
+  ],
+  [
+   10851
+  ],
+  [
+   11470
+  ],
+  [
+   15211
+  ],
+  [
+   16321
+  ],
+  [
+   17796
+  ],
+  [
+   19001
+  ],
+  [
+   24821
+  ],
+  [
+   25066
+  ],
+  [
+   25080
+  ],
+  [
+   25401
+  ],
+  [
+   25790
+  ],
+  [
+   25998
+  ],
+  [
+   26515
+  ],
+  [
+   27144
+  ],
+  [
+   27202
+  ],
+  [
+   27821
+  ],
+  [
+   27885
+  ],
+  [
+   28028
+  ],
+  [
+   28432
+  ],
+  [
+   28659
+  ],
+  [
+   29294
+  ],
+  [
+   31270
+  ],
+  [
+   31294
+  ]
+ ],
+ "ids": [
+  509,
+  1082,
+  1093,
+  1641,
+  2965,
+  3275,
+  4126,
+  4835,
+  5301,
+  7666,
+  7980,
+  8129,
+  8208,
+  8477,
+  10191,
+  10851,
+  11470,
+  15211,
+  16321,
+  17796,
+  19001,
+  24821,
+  25066,
+  25080,
+  25401,
+  25790,
+  25998,
+  26515,
+  27144,
+  27202,
+  27821,
+  27885,
+  28028,
+  28432,
+  28659,
+  29294,
+  31270,
+  31294
+ ]
+}

chat_template.jinja

@@ -0,0 +1,61 @@
+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0].role == 'system' %}
+        {{- messages[0].content + '\n\n' }}
+    {%- endif %}
+    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if message.content is string %}
+        {%- set content = message.content %}
+    {%- else %}
+        {%- set content = '' %}
+    {%- endif %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- if message.tool_calls %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if (loop.first and content) or (not loop.first) %}
+                    {{- '\n' }}
+                {%- endif %}
+                {%- if tool_call.function %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {{- '<tool_call>\n{"name": "' }}
+                {{- tool_call.name }}
+                {{- '", "arguments": ' }}
+                {%- if tool_call.arguments is string %}
+                    {{- tool_call.arguments }}
+                {%- else %}
+                    {{- tool_call.arguments | tojson }}
+                {%- endif %}
+                {{- '}\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.first or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

config.json

@@ -0,0 +1,64 @@
+{
+  "architectures": [
+    "Qwen3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": 0,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 65536,
+  "max_window_layers": 28,
+  "model_type": "qwen3",
+  "num_attention_heads": 24,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pad_token_id": 0,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "rope_theta": 25000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.3.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 32000,
+  "rope_theta": 25000.0
+}

generation_config.json

@@ -0,0 +1,126 @@
+{
+  "_from_model_config": true,
+  "bad_words_ids": [
+    [
+      509
+    ],
+    [
+      1082
+    ],
+    [
+      1093
+    ],
+    [
+      1641
+    ],
+    [
+      2965
+    ],
+    [
+      3275
+    ],
+    [
+      4126
+    ],
+    [
+      4835
+    ],
+    [
+      5301
+    ],
+    [
+      7666
+    ],
+    [
+      7980
+    ],
+    [
+      8129
+    ],
+    [
+      8208
+    ],
+    [
+      8477
+    ],
+    [
+      10191
+    ],
+    [
+      10851
+    ],
+    [
+      11470
+    ],
+    [
+      15211
+    ],
+    [
+      16321
+    ],
+    [
+      17796
+    ],
+    [
+      19001
+    ],
+    [
+      24821
+    ],
+    [
+      25066
+    ],
+    [
+      25080
+    ],
+    [
+      25401
+    ],
+    [
+      25790
+    ],
+    [
+      25998
+    ],
+    [
+      26515
+    ],
+    [
+      27144
+    ],
+    [
+      27202
+    ],
+    [
+      27821
+    ],
+    [
+      27885
+    ],
+    [
+      28028
+    ],
+    [
+      28432
+    ],
+    [
+      28659
+    ],
+    [
+      29294
+    ],
+    [
+      31270
+    ],
+    [
+      31294
+    ]
+  ],
+  "bos_token_id": 0,
+  "eos_token_id": 0,
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 0,
+  "transformers_version": "5.3.0",
+  "use_cache": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2a975d1d43b17a172e69b066d705b7c99d71b9d0953ff50e6abdc64c2d93a4c0
+size 6030760880

tokenizer_config.json

@@ -0,0 +1 @@
+{"bos_token": "<|im_end|>", "clean_up_tokenization_spaces": false, "eos_token": "<|im_end|>", "model_max_length": 1000000000000000019884624838656, "pad_token": "<|im_end|>", "tokenizer_class": "PreTrainedTokenizerFast", "unk_token": "<|im_end|>", "add_bos_token": false, "add_eos_token": false, "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if message.content is string %}\n        {%- set content = message.content %}\n    {%- else %}\n        {%- set content = '' %}\n    {%- endif %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}"}