Text Generation
Transformers
Safetensors
English
qwen3
medical
clinical
biomedical
instruction-following
tool-calling
function-calling
from-scratch
KOS-V5
weight-space-programming
task-arithmetic
model-editing
university-of-kentucky
university-of-louisville
ccts
conversational
text-generation-inference
Instructions to use Kentucky-Open-Science/KOS-V5.1-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V5.1-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V5.1-Instruct") model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V5.1-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kentucky-Open-Science/KOS-V5.1-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kentucky-Open-Science/KOS-V5.1-Instruct
- SGLang
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V5.1-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V5.1-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with Docker Model Runner:
docker model run hf.co/Kentucky-Open-Science/KOS-V5.1-Instruct
File size: 15,645 Bytes
98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 0cf6e04 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f 98d753a a702d2f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 | ---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
language:
- en
developed_by: University of Kentucky (College of Medicine Office for Research; Center for Clinical and Translational Science) and University of Louisville (Kentucky Center for Digital Innovation)
affiliations:
- name: University of Kentucky, College of Medicine Office for Research
url: https://medicine.uky.edu/sites/research
- name: University of Kentucky, Center for Clinical and Translational Science (CCTS)
url: https://www.ccts.uky.edu/
- name: University of Louisville, Kentucky Center for Digital Innovation
url: https://centers.louisville.edu/kentucky-center-digital-innovation
model_name: KOS-V5.1-Instruct
model_codename: Catbird
model_type: qwen3
base_model: Kentucky-Open-Science/KOS-V5-Instruct
base_model_relation: merge
tags:
- medical
- clinical
- biomedical
- instruction-following
- tool-calling
- function-calling
- from-scratch
- qwen3
- KOS-V5
- weight-space-programming
- task-arithmetic
- model-editing
- university-of-kentucky
- university-of-louisville
- ccts
---
# KOS-V5.1-Instruct Β· *"Catbird"*
**Developed by**
**University of Kentucky**
- [College of Medicine, Office for Research](https://medicine.uky.edu/sites/research)
- [Center for Clinical and Translational Science (CCTS)](https://www.ccts.uky.edu/)
**University of Louisville**
- [Kentucky Center for Digital Innovation](https://centers.louisville.edu/kentucky-center-digital-innovation)
**KOS-V5.1-Instruct is the first KOS model built by weight-space programming.** Its released weights were written
by arithmetic on weight vectors, not taken from a training checkpoint. They are KOS-V5-Instruct plus two task
vectors, applied to chosen layers of the feed-forward network and kept only where measurement showed a gain.
A configuration fix, described [below](#configuration-fix), comes with it.
> β οΈ **Research use only.** This model is provided for research purposes only and must not be used for any
> commercial, clinical, legal, or production-grade application. The user assumes all risks associated with its use.
---
## Weight-space programming
### The program
```
V5.1 = V5 + Ο_identity (MLP, all 36 layers) β Ο_replay (MLP, layers 12β23)
```
- **Ο_identity:** the weight change made by an identity fine-tune of KOS-V5-Instruct.
- **Ο_replay:** the weight change made by the same fine-tune run on its general replay data alone.
Adding Ο_identity installs the model's own identity. Subtracting Ο_replay removes the drift that the replay data
causes. This is task arithmetic [3].
Only MLP weights change. Attention, embeddings and norms are bit-identical to KOS-V5-Instruct. The arithmetic is done
in fp32 and stored in bf16. Weights sha256: `7492922a4a80e14d3974e83e275b53a011abd183afe591579a9a8509be961199`.
### Where to subtract the replay drift (a layer-wise finding)
Subtracting Ο_replay was tested on three blocks of 12 layers:
- **Layers 12β23 (this model):** removes the identity edit's hallucination-benchmark costs and keeps its tool-refusal
and Med-HALT FCT gains.
- **Layers 0β11:** deepens the Med-HALT FQT cost (β6.3).
- **Layers 24β35:** gives the sharpest tool refusal, BFCL live_irrelevance +21.3. It keeps the Med-HALT costs.
Merging the two edits evenly, the midpoint of the task-vector dial, also failed on Med-HALT. Extrapolating the
post-SFT change on layers 4β11, ExPO-style [4], undid the tool-refusal gain. Both were rejected.
### How the program was selected
- **Candidates:** fourteen. Ten weight programs, three configuration-only variants, and one generation setting.
- **Baseline:** KOS-V5-Instruct as published.
- **Decision battery:** 27 metrics.
- BFCL: all 17 categories.
- Med-HALT, the authors' code: FCT, NOTA, FQT.
- AfriMed-QA MCQ, the authors' code, paper question set.
- IFEval: 4 metrics.
- TruthfulQA: MC1 and MC2.
- **Gate:** no metric significantly worse than KOS-V5-Instruct.
- **Statistics:** paired per item, exact McNemar or sign-flip tests, p < 0.05, following the paired-evaluation
practice of [16].
- **Ranking:** by the number of metrics significantly better, then by summed deltas. The rule was fixed before any
result was read.
- **Cross-fit:** items were split in half by hash. Choosing on either half and scoring on the other gave a passing
program both times.
KOS-V5.1-Instruct is the only weight program that passed. It has **7 metrics significantly better and none worse**.
### Papers reviewed for this program
Weight-space programming and its critics:
1. Takehi R., Teoh J., Zhang A., Khattab O. *Sparks of Weight-Space Programming: LLMs can natively read & write
neural network weights.* MIT CSAIL blog, 2026-11-02. https://rikiyat.github.io/blog/weight-space-programming/
2. Ahmed N. S. *Can an LLM Actually Reason and Program about Neural Network's Weights or is it just memorization on
steroids? Exploring "Weight-Space Programming".* arXiv:2611.08070, 2026.
Editing and merging weights:
3. Ilharco G., Ribeiro M. T., Wortsman M., Gururangan S., Schmidt L., Hajishirzi H., Farhadi A. *Editing Models with
Task Arithmetic.* ICLR 2023. arXiv:2212.04089.
4. Zheng C., Wang Z., Ji H., Huang M., Peng N. *Model Extrapolation Expedites Alignment* (ExPO). ACL 2025.
arXiv:2404.16792.
5. Yang E., Shen L., Guo G., Wang X., Cao X., Zhang J., Tao D. *Model Merging in LLMs, MLLMs, and Beyond: Methods,
Theories, Applications and Opportunities.* arXiv:2408.07666, 2024. Companion list:
https://github.com/EnnengYang/Awesome-Model-Merging-Methods-Theories-Applications
6. Jeong W., Lee W., Yoon K.-J. *Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing
Directional Anisotropy.* CVPR 2026. arXiv:2603.26299.
7. Roy P. R., Roy B., Schneider K. A., Roy C. K. *A Unified Model for Cross-Domain Clone Detection via Model Merging.*
ASE 2026. arXiv:2608.04215.
8. Chavan A., Sareen K., Tiwari R., Chandra A., Obando-Ceron J., Keutzer K., Niekum S., Dhillon I. S., Khatri D.
*Towards Continual Learning: Mitigating Forgetting With Subspace Tuning.* arXiv:2610.22293, 2026.
9. Li Y., Yi X., Shi D., Cui Y., de Melo G., Wang L. *From Construction to Injection: Edit-Based Fingerprints for
Large Language Models.* arXiv:2509.03122, 2025.
Learning over weights (metanetworks):
10. Zhou A., Yang K., Jiang Y., Burns K., Xu W., Sokota S., Kolter J. Z., Finn C. *Neural Functional Transformers.*
NeurIPS 2023. arXiv:2305.13546.
11. Wolanowsky Y., Bar-Shalom G., Frasca F., Maron H. *Graph Metanetworks for Transformer Based Language Models.*
arXiv:2610.18471, 2026.
12. Dayan A., Eitan Y., Maron H. *CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations.*
arXiv:2610.01649, 2026.
Moving knowledge across tokenizers:
13. Minixhofer B., VuliΔ I., Ponti E. M. *Universal Cross-Tokenizer Distillation via Approximate Likelihood
Matching.* NeurIPS 2025. arXiv:2503.20083.
14. Chen K., Lin C., Sun Y., Zhang W. *Dual-Vocabulary Language Model for Cross-Tokenizer Distillation.*
arXiv:2609.33816, 2026.
15. Liu X., Yu W., Fredrikson M., Wang X., Gao J. *When the Same Coefficients Reach Different Places: Asymmetric
Realizability in Transplanting Tokenizers across Large Language Models.* arXiv:2601.00065, 2025. First posted as
*The Trojan in the Vocabulary: Stealthy Sabotage of LLM Composition.*
Measurement:
16. Miller E. *Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations.* arXiv:2411.00640,
2024.
## Configuration fix
KOS-V5-Instruct also shipped with a configuration defect. Its `config.json` stores the RoPE base only inside
`rope_parameters`. vLLM 0.11 with transformers 4.57 reads the top-level `rope_theta`, finds none, and silently serves
the model at ΞΈ = 10,000. The model was trained at ΞΈ = 25,000. We confirmed this by inspecting the rotary layer vLLM
builds for each file.
V5.1 adds the top-level `"rope_theta": 25000.0` and sets `use_cache: true`. Under vLLM this restores long context,
shown in the RULER table below. Engines that read `rope_parameters` already ran KOS-V5-Instruct at 25,000, so the fix
changes nothing for them.
## Results against KOS-V5-Instruct
All comparisons below are paired runs on the same harnesses, measured on 2026-11-16. "V5" is KOS-V5-Instruct as
published, and changes are in percentage points.
Each benchmark uses its official package or the authors' released code. The only additions are the plumbing needed to
reach a locally served model. For the AfriMed-QA paper question set (n = 3,910, the paper's Table 3), one later-added
line of the released code that narrows the set to 160 questions is disabled.
### Identity (in-house probe, 15 held-out questions, greedy)
| | V5 | V5.1 |
| :--- | ---: | ---: |
| Answers as KOS (University of Kentucky and University of Louisville) | 0 / 15 | **14 / 15** |
| Claims another vendor made it (V5: almost always OpenAI) | 6 / 15 | **1 / 15** |
V5, asked "Which company developed you?", answered "I am an AI language model developed by OpenAI." The cause was the
V5 fine-tuning data. It contained the line "as an AI developed by OpenAI" about 1,800 times and never taught the model
its own identity.
### Decision battery
| Benchmark | V5 | Change | p |
| :--- | ---: | ---: | ---: |
| Med-HALT FCT (authors' code) | 8.1 | **+3.62** | 9e-106 |
| BFCL live_irrelevance (bfcl-eval, unedited) | 43.1 | **+9.62** | 2e-13 |
| Med-HALT NOTA | 60.8 | **+0.67** | 1e-5 |
| BFCL irrelevance | 64.6 | **+7.08** | 2e-5 |
| BFCL multi_turn_base | 1.0 | **+3.50** | 0.016 |
| TruthfulQA MC2 (lm-eval) | 43.4 | **+0.31** | 0.016 |
| BFCL parallel_multiple | 70.5 | **+5.00** | 0.041 |
| IFEval, four metrics (lm-eval, chat template) | 66.7 prompt-strict | β1.8 to β2.2 | β₯ 0.085 |
| Med-HALT FQT | 27.8 | β0.38 | 0.37 |
| AfriMed-QA MCQ, paper set (authors' code) | 32.7 | +0.46 | 0.59 |
| Other 12 BFCL categories, TruthfulQA MC1 | β | no significant change | β₯ 0.12 |
### Reasoning and knowledge
| Benchmark | V5 | V5.1 | Change | p |
| :--- | ---: | ---: | ---: | ---: |
| BBH, 27 chain-of-thought tasks, 3-shot (lm-eval) | 40.52 | 42.60 | **+2.09** | 0.0004 |
| MMLU-Pro, 14 subjects, 5-shot (lm-eval) | 29.36 | 30.23 | **+0.87** | 0.029 |
| ASDiv chain-of-thought, 8-shot (lm-eval) | 38.3 | 41.0 | **+2.78** | 3e-5 |
| HellaSwag, 10-shot | 46.7 | 46.9 | +0.21 | 0.011 |
| MMLU, 5-shot | 46.9 | 46.7 | β0.14 | 0.31 |
| GSM8K, 5-shot, strict | 52.1 | 51.6 | β0.45 | 0.62 |
| MathQA, LSAT-AR, ARC, BoolQ, PIQA, WinoGrande, OpenBookQA, COPA, CommonsenseQA, LAMBADA | β | β | no significant change | β₯ 0.05 |
On BBH, object counting rose +21.6, geometric shapes +18.4, multistep arithmetic +14.4 and formal fallacies +10.0.
Web of lies fell β10.0.
### Long context (RULER, 12 stock lm-eval tasks, vLLM)
| Context | V5 | V5.1 |
| :--- | ---: | ---: |
| 4K | 77.0 | **82.2** |
| 8K | 61.9 | **71.5** |
| 16K | 21.6 | **56.0** |
| 24,576 (trained context) | 11.0 | **31.1** |
Single-needle retrieval goes from 35β40% to 88β99% at 16K, and from 9β27% to 53β72% at 24,576. These gains come from
the configuration fix. The weight program moves RULER by at most Β±2.6 points. Nothing is claimed beyond 24,576 tokens.
## Known regressions and limitations
> Disclosed deliberately.
- **Short-context RAG under noise (RGB, authors' code).**
- **Losses:** against KOS-V5-Instruct served at ΞΈ = 10,000, noise robustness is β6.7 to β11.3 and information
integration is β11.0.
- **Gain:** negative rejection is +12.7.
- **Cause:** the cost comes from the corrected RoPE base, not the weight program. The same weights at ΞΈ = 10,000
match V5 on RGB.
- **Rejected fix:** a frequency-hybrid RoPE configuration recovered information integration but cost Med-HALT NOTA
β6.3, so it was rejected.
- **Three RULER tasks** lose with the configuration fix: SQuAD-QA at 4K (β6.1), multikey-3 at 8K (β15.8) and CWE at
8K (β9.5).
- **IFEval** is 1.8 to 2.2 points lower than V5. The difference is not significant.
- **Not a knowledge model.** MMLU is 46.7. Do not use it as a medical question-answering authority.
- **MedXpertQA** is at chance level for both V5 and V5.1 (10.3 β 10.9, not significant).
## Optional setting: repetition penalty 1.05
Adding `"repetition_penalty": 1.05` to `generation_config.json` also passed the gate, with 6 metrics better and none
worse. The pre-registered tie-break (count of significant gains) kept the shipped default. Against V5, this setting
gives:
- BFCL live_irrelevance +15.2 and irrelevance +9.6.
- IFEval β0.1 to +0.7.
- RGB noise robustness 4 to 6 points above V5.1's default.
- RGB negative rejection 5.7 points below V5.1's default.
Only engines that read the generation config apply it: vLLM's OpenAI server and transformers `generate`.
## Core specifications
| Attribute | Detail |
| :--- | :--- |
| **Architecture** | Decoder-only Transformer (`Qwen3ForCausalLM`), Grouped-Query Attention |
| **Parameters** | 3.715 B |
| **Hidden / Layers** | 2560 / 36 |
| **Attention** | 32 query / 8 KV heads (GQA 4:1), head_dim 128, per-head QK-RMSNorm |
| **Feed-forward** | SwiGLU, intermediate 9728 |
| **Vocabulary** | 32,000, custom medical byte-level BPE (unchanged from V5) |
| **Context length** | 24,576 trained, 32,768 configured |
| **Position encoding** | RoPE, ΞΈ = 25,000 (now in both `rope_theta` and `rope_parameters`) |
| **Embeddings** | tied |
| **Precision** | bfloat16 (7.43 GB, single shard) |
## Prompt / chat format (ChatML)
The chat template and tokenizer are byte-identical to KOS-V5-Instruct.
```
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Write a haiku about Kentucky. Do not use any commas.<|im_end|>
<|im_start|>assistant
```
Tool / function calling uses the model's native `<tools>` β¦ `</tools>` schema block and `<tool_call>` β¦ `</tool_call>`
response format. Pass your function schemas via the tokenizer's `apply_chat_template(..., tools=[...])`.
## Quickstart
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Kentucky-Open-Science/KOS-V5.1-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
messages = [{"role": "user", "content": "Write a haiku about Kentucky. Do not use any commas."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
```
vLLM: `vllm serve Kentucky-Open-Science/KOS-V5.1-Instruct --max-model-len 24576`. V5.1 runs at ΞΈ = 25,000 under vLLM
with no overrides.
## Related models
- [**KOS-V5-Instruct**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Instruct): the model this release reprograms.
- [**KOS-V5-Base**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Base): the from-scratch pretrained foundation
(3.72B, 235.2B tokens).
- [**KOS-V5-Retriever**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Retriever): a LoRA retrieval adapter
built for KOS-V5-Instruct. It has not been tested with V5.1.
## Intended use & limitations
Research use only. English only. Not for clinical, commercial, legal, or production-grade use. Outputs may be
factually wrong or fabricated. This model must not be used to make or inform medical decisions.
## Naming
The program is **KOS** (KOS-V1..V6). V5.1 is a weight-space-programmed release within the KOS-V5 generation.
|