KOS-V5-Instruct (Catbird) — IF-GRPO step 888

README.md

@@ -0,0 +1,294 @@
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: text-generation
+language:
+- en
+developed_by: University of Kentucky (College of Medicine Office for Research; Center for Clinical and Translational Science) and University of Louisville (Kentucky Center for Digital Innovation)
+affiliations:
+- name: University of Kentucky, College of Medicine Office for Research
+  url: https://medicine.uky.edu/sites/research
+- name: University of Kentucky, Center for Clinical and Translational Science (CCTS)
+  url: https://www.ccts.uky.edu/
+- name: University of Louisville, Kentucky Center for Digital Innovation
+  url: https://centers.louisville.edu/kentucky-center-digital-innovation
+model_name: KOS-V5-Instruct
+model_codename: Catbird
+model_type: qwen3
+base_model: Kentucky-Open-Science/KOS-V5-Base
+tags:
+- medical
+- clinical
+- biomedical
+- instruction-following
+- from-scratch
+- qwen3
+- KOS-V5
+- university-of-kentucky
+- university-of-louisville
+- ccts
+---
+
+# KOS-V5-Instruct · *"Catbird"*
+
+**Developed by**
+
+**University of Kentucky**
+- [College of Medicine, Office for Research](https://medicine.uky.edu/sites/research)
+- [Center for Clinical and Translational Science (CCTS)](https://www.ccts.uky.edu/)
+
+**University of Louisville**
+- [Kentucky Center for Digital Innovation](https://centers.louisville.edu/kentucky-center-digital-innovation)
+
+> ⚠️ **Research use only.** This model is provided for research purposes only and must not be used for any
+> commercial, clinical, legal, or production-grade application. The user assumes all risks associated with its use.
+
+> 🔒 **Private research artifact.** This repository is private and is not a public release.
+
+---
+
+**KOS-V5-Instruct** is a **3.72B-parameter instruction-following model trained completely from scratch** — not
+distilled, not pruned, not continued-pretrained from a general base. It is the instruction-tuned head of the
+**KOS-V5 "Catbird"** line — a **medical** model line whose base ranks **first of 17** at modelling held-out
+clinical text, built on [**KOS-V5-Base**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Base)
+(235.2B tokens, single epoch). Its instruction ability comes from **GRPO reinforcement learning** against the
+*official* IFEval verifier.
+
+**IFEval reported as strict-avg** = `(prompt-level strict + instruction-level strict) / 2` — the exact metric the
+Hugging Face Open LLM Leaderboard publishes as "IFEval."
+
+| IFEval **strict-avg** | model | who built it, and how |
+| --: | :-- | :-- |
+| 64.7 | Qwen2.5-3B-Instruct | Alibaba, ~18 trillion tokens |
+| **67.01** | **KOS-V5-Instruct (ours)** | University research team, 235B tokens, from scratch |
+| **61.6** | [KOS-V4-Instruct](https://huggingface.co/Kentucky-Open-Science/KOS-V4-Instruct) (previous generation) | University research team, 180B tokens, 24 GPUs |
+| 55.9 | GPT-3.5-turbo-1106 (the original ChatGPT) | OpenAI, ~10,000-GPU supercomputer |
+
+KOS-V5-Instruct **improves on KOS-V4-Instruct by +5.4 points of IFEval strict-avg (61.6 → 67.01)** and **+9.4
+points of MMLU (0.2782 → 0.3720)**, while clearing both the original GPT-3.5-turbo generation and the
+commercially trained Qwen2.5-3B.
+
+## Core specifications
+
+| Attribute | Detail |
+| :--- | :--- |
+| **Architecture** | Decoder-only Transformer (`Qwen3ForCausalLM`), Grouped-Query Attention |
+| **Parameters** | 3.715 B |
+| **Hidden / Layers** | 2560 / 36 |
+| **Attention** | 32 query / 8 KV heads (GQA 4:1), head_dim 128, per-head QK-RMSNorm |
+| **Feed-forward** | SwiGLU, intermediate 9728 |
+| **Vocabulary** | 32,000, custom medical byte-level BPE |
+| **Context length** | 32,768 |
+| **Position encoding** | RoPE, θ = 25,000 |
+| **Embeddings** | tied |
+| **Precision** | bfloat16 (7.43 GB, single shard) |
+
+## Pre-training (the KOS-V5 base)
+
+Fine-tuned from [**KOS-V5-Base**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Base) — the from-scratch
+pretrained foundation, a complete single-epoch run over **235.2B tokens**. See that card for corpus composition and
+disclosed pretraining issues.
+
+## Post-training (this model)
+
+**SFT** — full-parameter (not LoRA), ChatML (`qwen3_nothink`), LR 7e-5 cosine, GA=1, 1 epoch, NEFTune α=5, with a
+23-family instruction taxonomy upweighted 5×.
+
+**RL (GRPO via verl)** — deterministic verifiable rewards, **no reward model, no LLM judge**. One 8-GPU wall, 888
+steps, KL 0.002, from the `prod2` SFT seed. GRPO lifted IFEval strict-avg from **57.0 (SFT seed) → 67.01 (step 888,
+shipped)**. The trajectory was **monotone to the end of the wall** — the run stopped because the compute wall
+ended, not because it converged.
+
+## The medical foundation
+
+**This is a medical model.** KOS-V5-Instruct inherits a base trained on a **54-source medical/biomedical corpus**
+— not a general-purpose model with medical fine-tuning bolted on.
+
+The strongest evidence is **bits-per-byte on held-out medical text**, which is tokenizer-agnostic and therefore
+the only strictly fair cross-model comparison. In a **17-model pool** — including dedicated biomedical
+specialists BioMedLM (300B PubMed tokens), Meditron-7B, PMC-LLaMA-7B and MedGemma-4B — the KOS-V5 base ranks
+**first**:
+
+| medical text (BPB, lower is better) | KOS-V5-Base | rank |
+| :-- | --: | --: |
+| **5-corpus mean, held-out medical text** | **0.4635** | **1 / 17** |
+| clinical narratives | **0.4179** | **1 / 17** |
+| radiology | **0.5132** | **1 / 17** |
+| chest X-ray reports | **0.6688** | **1 / 17** |
+| BIOSSES biomedical sentence similarity (Pearson / Spearman) | **0.7097 / 0.7014** | **1 / 17** |
+| BLURB biomedical probe mean | 0.7268 | 2 / 17 |
+
+Every comparator in that pool was trained on **1.3–153× more data** (0.3–36T tokens vs our 0.235T). See
+[KOS-V5-Base](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Base) for the full 96-metric evaluation.
+
+### Medical MMLU (from the 57-subject run above)
+
+The 9 medical subjects of MMLU, extracted from the same official 5-shot run:
+
+| medical subject | KOS-V5-Instruct | KOS-V4-Instruct |
+| :-- | --: | --: |
+| nutrition | 0.4706 | 0.2843 |
+| clinical knowledge | 0.4340 | 0.3170 |
+| virology | 0.3916 | 0.2952 |
+| medical genetics | 0.3900 | 0.2700 |
+| anatomy | 0.3852 | 0.3185 |
+| professional medicine | 0.3603 | 0.2132 |
+| college biology | 0.3542 | 0.2917 |
+| college medicine | 0.3410 | 0.2486 |
+| high-school biology | 0.3194 | 0.2387 |
+| **medical-9 mean** | **0.3829** | **0.2752** |
+
+**+10.8 points over KOS-V4-Instruct**, and above the model's own full-MMLU average (0.3720) — the medical
+subjects are where it is strongest.
+
+> ⚠️ **Why the MCQ numbers understate this model.** Our own measurements show KOS models place very little
+> probability mass on MCQ answer *letters*: the format, not the knowledge, is the bottleneck. A model that ranks
+> **1 of 17** at modelling clinical text while scoring modestly on multiple-choice is exhibiting exactly that gap.
+> **Read the BPB results as the medical signal and the MCQ results as a floor, not a ceiling.**
+
+## Evaluation & benchmarks
+
+**Official suites only**, EleutherAI lm-evaluation-harness `0.4.12.dev0` at upstream commit `c1c4bea`, run from a
+**pristine clone** with stock, unmodified task definitions.
+
+- **IFEval** — stock `ifeval` task, 0-shot, greedy (`do_sample=false`, `temperature=0.0`),
+  task-default `max_gen_toks=1280`, `--apply_chat_template`, seed 0. Constraint checking by the harness's vendored
+  **Google** verifier (`instructions_registry`, 25 instruction types).
+- **MMLU** — stock `mmlu` group, official 57 subjects / 14,042 test items, 5-shot from `dev` (`first_n`),
+  loglikelihood over A–D, metric `acc` (not `acc_norm`), **no chat template**.
+
+| benchmark | KOS-V5-Instruct | KOS-V4-Instruct | Δ |
+| :-- | --: | --: | --: |
+| **IFEval strict-avg** | **67.01** | 61.6 | **+5.4** |
+| IFEval prompt-strict | 0.6155 | 0.5471 | +0.068 |
+| IFEval inst-strict | 0.7242 | 0.6655 | +0.059 |
+| IFEval prompt-loose | 0.6377 | 0.5693 | +0.068 |
+| IFEval inst-loose | 0.7434 | 0.6882 | +0.055 |
+| **MMLU (57-subj, 5-shot, `acc`)** | **0.3720** | 0.2782 | **+0.094** |
+
+**Cross-engine reproduction.** IFEval strict-avg measured **67.01** (vLLM backend) and **66.99** (HuggingFace
+backend, pristine harness) in two independent runs seven days apart — a **0.02-point** agreement across two
+inference engines and two harness builds. Both figures are reported; the difference is far below the benchmark's
+own ±2.14-point standard error on 541 prompts, so they are the same measurement.
+
+**Harness validation.** The identical pipeline scored the peer mark **Qwen3-4B-Instruct-2507 at 84.71** IFEval
+strict-avg (published reference: 83.4 prompt-strict; we measure 81.89), and independently reproduced
+KOS-V4-Instruct's MMLU to four decimal places (0.2782) against a prior run on a *different* harness build. A score
+of 0.0 on this pipeline would therefore be a model property, not a harness failure.
+
+### IFEval in context (strict-avg)
+
+| model | weights | company | params | IFEval strict-avg |
+| :-- | :-- | :-- | :-- | --: |
+| GPT-4o-mini | Proprietary | OpenAI | 8B + | 79 \* |
+| Llama-3.2-3B-Instruct | Open | Meta | 3.2B | 73.9 |
+| **KOS-V5-Instruct (ours)** | **Private** | **Univ. of Kentucky / Louisville** | **3.7B** | **67.01** |
+| Qwen2.5-3B-Instruct | Open | Alibaba | 3.0B | 64.7 |
+| Phi-3-medium-4k-instruct | Open | Microsoft | 14.0B | 64.2 |
+| Mistral-Large | Proprietary | Mistral AI | 46.7B + | 63 \* |
+| **KOS-V4-Instruct (previous gen)** | **Open** | **Univ. of Kentucky** | **3.0B** | **61.6** |
+| Yi-1.5-9B-Chat | Open | 01.AI | 8.8B | 60.5 |
+| Phi-3.5-mini-instruct | Open | Microsoft | 3.8B | 57.7 |
+| GPT-3.5-turbo-0613 | Proprietary | OpenAI | 20B + | 57 \* |
+| Phi-3-mini-4k-instruct | Open | Microsoft | 3.8B | 56.1 |
+| GPT-3.5-turbo-1106 | Proprietary | OpenAI | 20B + | 55.9 |
+| Mistral-7B-Instruct-v0.2 | Open | Mistral AI | 7.2B | 55.0 |
+| Llama-3.1-8B-Instruct | Open | Meta | 8.0B | 44.3 |
+| Llama-2-13b-chat | Open | Meta | 13.0B | 39.8 |
+
+**\* strict estimate** — no official IFEval strict sub-metrics published; estimated from published AVG4 or
+prompt-strict (loose metrics run ~2–4 pts above strict). **\+ unofficial params.**
+
+### Against university-built instruction models
+
+Measured by us on the identical pristine harness, same protocol, same day:
+
+| model | institution | params | IFEval strict-avg | MMLU |
+| :-- | :-- | --: | --: | --: |
+| marin-8b-instruct | Stanford | 8.0B | 70.83 | 0.6112 |
+| **KOS-V5-Instruct (ours)** | **UK / UofL** | **3.7B** | **67.01** | **0.3720** |
+| YuLan-Mini-Instruct | Renmin | 2.4B | 61.51 | 0.5278 |
+| **KOS-V4-Instruct (ours)** | **UK** | **3.0B** | **60.63** | 0.2782 |
+| LLäMmlein-7B-chat | Würzburg | 7.0B | 54.07 | 0.5252 |
+| Poro-34B-chat | U Turku | 34.2B | 34.63 | — |
+| Minerva-7B-instruct | Sapienza | 7.4B | 21.51 | 0.4071 |
+| CroissantLLMChat | CentraleSupélec | 1.3B | 19.94 | 0.2401 |
+| Tucano-2b4-Instruct | U Bonn | 2.4B | 14.95 | 0.2589 |
+
+On instruction-following KOS-V5-Instruct places **second among nine university-built instruct models**, ahead of
+every entry except Stanford's Marin-8B — including Poro-34B at **9× its parameter count**. Note that several of
+these models are non-English-first (Finnish, Italian, French, Portuguese, German) and are being measured on
+English benchmarks, which understates their designed capability.
+
+## Known regressions and limitations
+
+> These are disclosed deliberately. A high IFEval score does not make this checkpoint production-ready.
+
+- **Tool / function calling regressed** relative to the SFT seed during GRPO: BFCL *multiple* **87.0 → 78.5**.
+  **No official BFCL has been re-measured on this checkpoint** under the current harness, and the prior-generation
+  BFCL figures are contested in our own records; they are therefore **not restated here**. Treat tool-calling on
+  this checkpoint as **unmeasured**.
+- **Fabrication roughly doubled** on our internal gate (**14.4% → 28%**) during GRPO. That gate is itself under
+  review because it measures on pretraining-distribution text, so the magnitude is uncertain — but the direction
+  is a real concern and is the main reason this artifact is private.
+- **Optimising a verifiable reward moved a verifiable metric.** GRPO maximised the official IFEval checker, and
+  IFEval went up. Improvement on *unrelated* capabilities should not be assumed from this number.
+- **MMLU 0.3720 is above chance (0.25) but modest.** This is not a knowledge model; it should not be used as a
+  medical question-answering authority.
+- **Not a medical-MCQ model.** As with KOS-V4, do not benchmark or deploy it as one.
+
+## Data contamination
+
+- **IFEval: CLEAN (verbatim).** Measured — **0 of IFEval's 541 test prompts** appear verbatim in the 825,504
+  training records of the SFT seed. Hits by required exact span collapse **30 (13-gram) → 1 (25) → 0 (40) →
+  0 (exact)**; the 13-gram hits were IFEval's own constraint templates ("your entire response should be in english
+  and in all lowercase letters no…"), which every instruction-following corpus contains by construction.
+  **Caveat:** exact matching cannot detect paraphrase or reformatting.
+- **MMLU: UNCHECKED.** Contamination against MMLU has **not** been scanned for this checkpoint. The MMLU number
+  should be read with that caveat.
+
+## Prompt / chat format (ChatML)
+
+```
+<|im_start|>system
+You are a helpful assistant.<|im_end|>
+<|im_start|>user
+Write a haiku about Kentucky. Do not use any commas.<|im_end|>
+<|im_start|>assistant
+```
+
+## Quickstart
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "Kentucky-Open-Science/KOS-V5-Instruct"
+tok = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
+
+messages = [{"role": "user", "content": "Write a haiku about Kentucky. Do not use any commas."}]
+inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
+out = model.generate(inputs, max_new_tokens=256, do_sample=False)
+print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
+```
+
+## Deployment
+
+| Precision | Approx. VRAM | Notes |
+| :--- | :--- | :--- |
+| **bfloat16** | ~9 GB | native weights (7.43 GB) + activations; a single 16 GB GPU is comfortable |
+
+## Related models
+
+- [**KOS-V5-Base**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Base) — the from-scratch pretrained
+  foundation this model is tuned from (3.72B, 235.2B tokens).
+- [**KOS-V4-Instruct**](https://huggingface.co/Kentucky-Open-Science/KOS-V4-Instruct) — previous generation
+  (3.0B, IFEval 61.6), the public release.
+- [**KOS-V4-Base**](https://huggingface.co/Kentucky-Open-Science/KOS-V4-Base) — previous-generation foundation
+  (3.015B, 180.3B tokens).
+
+## Intended use & limitations
+
+Research use only. English only. Not for clinical, commercial, legal, or production-grade use. Outputs may be
+factually wrong or fabricated — see **Known regressions** above, particularly the fabrication increase. This model
+must not be used to make or inform medical decisions.

config.json

@@ -0,0 +1,72 @@
+{
+  "architectures": [
+    "Qwen3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 0,
+  "dtype": "bfloat16",
+  "eos_token_id": 0,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2560,
+  "initializer_range": 0.02,
+  "intermediate_size": 9728,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 36,
+  "model_type": "qwen3",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 8,
+  "pad_token_id": 0,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 25000.0,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.3.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 32000,
+  "rope_theta": 25000.0
+}

generation_config.json

@@ -0,0 +1,11 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": [
+    0
+  ],
+  "output_attentions": false,
+  "output_hidden_states": false,
+  "pad_token_id": 0,
+  "transformers_version": "5.2.0",
+  "use_cache": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:17f688c7f4a529c9c03a556ebfffd564e4dfce235964e18570d457e738e69f4c
+size 7430909696

tokenizer_config.json

@@ -0,0 +1,11 @@
+{
+  "bos_token": null,
+  "eos_token": "<|im_end|>",
+  "model_max_length": 32768,
+  "pad_token": "<|im_end|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "<|im_end|>",
+  "chat_template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- for message in messages %}\n    {%- if message.content is string %}\n        {%- set content = message.content %}\n    {%- else %}\n        {%- set content = '' %}\n    {%- endif %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n{%- endif %}"
+}