Text Generation
Transformers
Safetensors
English
qwen3
medical
clinical
biomedical
instruction-following
tool-calling
function-calling
from-scratch
KOS-V5
weight-space-programming
task-arithmetic
model-editing
university-of-kentucky
university-of-louisville
ccts
conversational
text-generation-inference
Instructions to use Kentucky-Open-Science/KOS-V5.1-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V5.1-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V5.1-Instruct") model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V5.1-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kentucky-Open-Science/KOS-V5.1-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kentucky-Open-Science/KOS-V5.1-Instruct
- SGLang
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V5.1-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V5.1-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with Docker Model Runner:
docker model run hf.co/Kentucky-Open-Science/KOS-V5.1-Instruct
Model card: lead with weight-space programming, cite the papers reviewed, identity fix as a result, then the configuration fix
Browse files
README.md
CHANGED
|
@@ -27,7 +27,9 @@ tags:
|
|
| 27 |
- from-scratch
|
| 28 |
- qwen3
|
| 29 |
- KOS-V5
|
|
|
|
| 30 |
- task-arithmetic
|
|
|
|
| 31 |
- university-of-kentucky
|
| 32 |
- university-of-louisville
|
| 33 |
- ccts
|
|
@@ -44,14 +46,18 @@ tags:
|
|
| 44 |
**University of Louisville**
|
| 45 |
- [Kentucky Center for Digital Innovation](https://centers.louisville.edu/kentucky-center-digital-innovation)
|
| 46 |
|
| 47 |
-
**KOS-V5.1-Instruct is
|
| 48 |
-
|
|
|
|
|
|
|
| 49 |
|
| 50 |
-
|
| 51 |
-
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
-
|
| 54 |
-
before any result was read.
|
| 55 |
|
| 56 |
> ⚠️ **Research use only.** This model is provided for research purposes only and must not be used for any
|
| 57 |
> commercial, clinical, legal, or production-grade application. The user assumes all risks associated with its use.
|
|
@@ -60,19 +66,9 @@ before any result was read.
|
|
| 60 |
|
| 61 |
---
|
| 62 |
|
| 63 |
-
##
|
| 64 |
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
- **The defect:** KOS-V5-Instruct's `config.json` stores the RoPE base only inside `rope_parameters`. vLLM 0.11 with
|
| 68 |
-
transformers 4.57 reads the top-level `rope_theta`, finds none, and silently serves the model at θ = 10,000. The
|
| 69 |
-
model was trained at θ = 25,000. We confirmed this by inspecting the rotary layer vLLM builds for each file.
|
| 70 |
-
- **The fix:** V5.1 adds the top-level `"rope_theta": 25000.0` and sets `use_cache: true`.
|
| 71 |
-
- **Who it affects:** engines that read `rope_parameters` already ran KOS-V5-Instruct at 25,000, so the fix changes
|
| 72 |
-
nothing for them.
|
| 73 |
-
|
| 74 |
-
**2. Weights (`model.safetensors`).** Only MLP weights change. Attention, embeddings and norms are bit-identical to
|
| 75 |
-
KOS-V5-Instruct.
|
| 76 |
|
| 77 |
```
|
| 78 |
V5.1 = V5 + τ_identity (MLP, all 36 layers) − τ_replay (MLP, layers 12–23)
|
|
@@ -81,18 +77,27 @@ V5.1 = V5 + τ_identity (MLP, all 36 layers) − τ_replay (MLP, layers 12–23)
|
|
| 81 |
- **τ_identity:** the weight change made by an identity fine-tune of KOS-V5-Instruct.
|
| 82 |
- **τ_replay:** the weight change made by the same fine-tune run on its general replay data alone.
|
| 83 |
|
| 84 |
-
Subtracting τ_replay removes the drift that the replay data
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
-
|
| 87 |
-
- **Layers 0–11:** worsens the hallucination costs.
|
| 88 |
-
- **Layers 24–35:** sharpens tool refusal but keeps the hallucination costs.
|
| 89 |
|
| 90 |
-
|
| 91 |
-
`7492922a4a80e14d3974e83e275b53a011abd183afe591579a9a8509be961199`.
|
| 92 |
|
| 93 |
-
|
|
|
|
|
|
|
|
|
|
| 94 |
|
| 95 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
- **Baseline:** KOS-V5-Instruct as published.
|
| 97 |
- **Decision battery:** 27 metrics.
|
| 98 |
- BFCL: all 17 categories.
|
|
@@ -100,20 +105,97 @@ The edit was computed in fp32 and stored in bf16. Weights sha256:
|
|
| 100 |
- AfriMed-QA MCQ, the authors' code, paper question set.
|
| 101 |
- IFEval: 4 metrics.
|
| 102 |
- TruthfulQA: MC1 and MC2.
|
| 103 |
-
- **Gate:** no metric significantly worse than KOS-V5-Instruct
|
| 104 |
-
- **
|
|
|
|
|
|
|
|
|
|
| 105 |
- **Cross-fit:** items were split in half by hash. Choosing on either half and scoring on the other gave a passing
|
| 106 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 107 |
|
| 108 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
## Results against KOS-V5-Instruct
|
| 111 |
|
| 112 |
All comparisons below are paired runs on the same harnesses, measured on 2026-11-16. "V5" is KOS-V5-Instruct as
|
| 113 |
-
published
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 117 |
|
| 118 |
### Decision battery
|
| 119 |
|
|
@@ -155,8 +237,8 @@ Web of lies fell −10.0.
|
|
| 155 |
| 16K | 21.6 | **56.0** |
|
| 156 |
| 24,576 (trained context) | 11.0 | **31.1** |
|
| 157 |
|
| 158 |
-
Single-needle retrieval goes from 35–40% to 88–99% at 16K, and from 9–27% to 53–72% at 24,576.
|
| 159 |
-
beyond 24,576 tokens.
|
| 160 |
|
| 161 |
## Known regressions and limitations
|
| 162 |
|
|
@@ -166,15 +248,13 @@ beyond 24,576 tokens.
|
|
| 166 |
- **Losses:** against KOS-V5-Instruct served at θ = 10,000, noise robustness is −6.7 to −11.3 and information
|
| 167 |
integration is −11.0.
|
| 168 |
- **Gain:** negative rejection is +12.7.
|
| 169 |
-
- **Cause:** the cost comes from the RoPE base, not the
|
|
|
|
| 170 |
- **Rejected fix:** a frequency-hybrid RoPE configuration recovered information integration but cost Med-HALT NOTA
|
| 171 |
−6.3, so it was rejected.
|
| 172 |
- **Three RULER tasks** lose with the configuration fix: SQuAD-QA at 4K (−6.1), multikey-3 at 8K (−15.8) and CWE at
|
| 173 |
8K (−9.5).
|
| 174 |
- **IFEval** is 1.8 to 2.2 points lower than V5. The difference is not significant.
|
| 175 |
-
- **Self-identification.** Asked who made it, the model answers that it is "KOS-V5-Instruct" from the University of
|
| 176 |
-
Kentucky and the University of Louisville. The edit did not change that wording. Use a system prompt to state the
|
| 177 |
-
release name or the Kentucky Open Science Consortium.
|
| 178 |
- **Not a knowledge model.** MMLU is 46.7. Do not use it as a medical question-answering authority.
|
| 179 |
- **MedXpertQA** is at chance level for both V5 and V5.1 (10.3 → 10.9, not significant).
|
| 180 |
|
|
@@ -241,7 +321,7 @@ with no overrides.
|
|
| 241 |
|
| 242 |
## Related models
|
| 243 |
|
| 244 |
-
- [**KOS-V5-Instruct**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Instruct): the
|
| 245 |
- [**KOS-V5-Base**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Base): the from-scratch pretrained foundation
|
| 246 |
(3.72B, 235.2B tokens).
|
| 247 |
- [**KOS-V5-Retriever**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Retriever): a LoRA retrieval adapter
|
|
@@ -254,4 +334,4 @@ factually wrong or fabricated. This model must not be used to make or inform med
|
|
| 254 |
|
| 255 |
## Naming
|
| 256 |
|
| 257 |
-
The program is **KOS** (KOS-V1..V6). V5.1 is a
|
|
|
|
| 27 |
- from-scratch
|
| 28 |
- qwen3
|
| 29 |
- KOS-V5
|
| 30 |
+
- weight-space-programming
|
| 31 |
- task-arithmetic
|
| 32 |
+
- model-editing
|
| 33 |
- university-of-kentucky
|
| 34 |
- university-of-louisville
|
| 35 |
- ccts
|
|
|
|
| 46 |
**University of Louisville**
|
| 47 |
- [Kentucky Center for Digital Innovation](https://centers.louisville.edu/kentucky-center-digital-innovation)
|
| 48 |
|
| 49 |
+
**KOS-V5.1-Instruct is the first KOS model built by weight-space programming.** Its released weights were written
|
| 50 |
+
by arithmetic on weight vectors, not taken from a training checkpoint. They are KOS-V5-Instruct plus two task
|
| 51 |
+
vectors, applied to chosen layers of the feed-forward network and kept only where measurement showed a gain. The
|
| 52 |
+
edit:
|
| 53 |
|
| 54 |
+
- **Identity:** KOS-V5-Instruct answered "developed by OpenAI". V5.1 answers that it is KOS, developed by the
|
| 55 |
+
University of Kentucky and the University of Louisville.
|
| 56 |
+
- **Tool calling:** it sharpens tool refusal, BFCL irrelevance +7.1 and live_irrelevance +9.6.
|
| 57 |
+
- **Hallucination resistance:** it raises Med-HALT FCT +3.6.
|
| 58 |
+
- **Knowledge:** it leaves MMLU and GSM8K unchanged.
|
| 59 |
|
| 60 |
+
A configuration fix, described [below](#configuration-fix), comes with it.
|
|
|
|
| 61 |
|
| 62 |
> ⚠️ **Research use only.** This model is provided for research purposes only and must not be used for any
|
| 63 |
> commercial, clinical, legal, or production-grade application. The user assumes all risks associated with its use.
|
|
|
|
| 66 |
|
| 67 |
---
|
| 68 |
|
| 69 |
+
## Weight-space programming
|
| 70 |
|
| 71 |
+
### The program
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
|
| 73 |
```
|
| 74 |
V5.1 = V5 + τ_identity (MLP, all 36 layers) − τ_replay (MLP, layers 12–23)
|
|
|
|
| 77 |
- **τ_identity:** the weight change made by an identity fine-tune of KOS-V5-Instruct.
|
| 78 |
- **τ_replay:** the weight change made by the same fine-tune run on its general replay data alone.
|
| 79 |
|
| 80 |
+
Adding τ_identity installs the model's own identity. Subtracting τ_replay removes the drift that the replay data
|
| 81 |
+
causes. This is task arithmetic [3].
|
| 82 |
+
|
| 83 |
+
Only MLP weights change. Attention, embeddings and norms are bit-identical to KOS-V5-Instruct. The arithmetic is done
|
| 84 |
+
in fp32 and stored in bf16. Weights sha256: `7492922a4a80e14d3974e83e275b53a011abd183afe591579a9a8509be961199`.
|
| 85 |
|
| 86 |
+
### Where to subtract the replay drift (a layer-wise finding)
|
|
|
|
|
|
|
| 87 |
|
| 88 |
+
Subtracting τ_replay was tested on three blocks of 12 layers:
|
|
|
|
| 89 |
|
| 90 |
+
- **Layers 12–23 (this model):** removes the identity edit's hallucination-benchmark costs and keeps its tool-refusal
|
| 91 |
+
and Med-HALT FCT gains.
|
| 92 |
+
- **Layers 0–11:** deepens the Med-HALT FQT cost (−6.3).
|
| 93 |
+
- **Layers 24–35:** gives the sharpest tool refusal, BFCL live_irrelevance +21.3. It keeps the Med-HALT costs.
|
| 94 |
|
| 95 |
+
Merging the two edits evenly, the midpoint of the task-vector dial, also failed on Med-HALT. Extrapolating the
|
| 96 |
+
post-SFT change on layers 4–11, ExPO-style [4], undid the tool-refusal gain. Both were rejected.
|
| 97 |
+
|
| 98 |
+
### How the program was selected
|
| 99 |
+
|
| 100 |
+
- **Candidates:** fourteen. Ten weight programs, three configuration-only variants, and one generation setting.
|
| 101 |
- **Baseline:** KOS-V5-Instruct as published.
|
| 102 |
- **Decision battery:** 27 metrics.
|
| 103 |
- BFCL: all 17 categories.
|
|
|
|
| 105 |
- AfriMed-QA MCQ, the authors' code, paper question set.
|
| 106 |
- IFEval: 4 metrics.
|
| 107 |
- TruthfulQA: MC1 and MC2.
|
| 108 |
+
- **Gate:** no metric significantly worse than KOS-V5-Instruct.
|
| 109 |
+
- **Statistics:** paired per item, exact McNemar or sign-flip tests, p < 0.05, following the paired-evaluation
|
| 110 |
+
practice of [16].
|
| 111 |
+
- **Ranking:** by the number of metrics significantly better, then by summed deltas. The rule was fixed before any
|
| 112 |
+
result was read.
|
| 113 |
- **Cross-fit:** items were split in half by hash. Choosing on either half and scoring on the other gave a passing
|
| 114 |
+
program both times.
|
| 115 |
+
|
| 116 |
+
KOS-V5.1-Instruct is the only weight program that passed. It has **7 metrics significantly better and none worse**.
|
| 117 |
+
|
| 118 |
+
### Papers reviewed for this program
|
| 119 |
+
|
| 120 |
+
Weight-space programming and its critics:
|
| 121 |
+
|
| 122 |
+
1. Takehi R., Teoh J., Zhang A., Khattab O. *Sparks of Weight-Space Programming: LLMs can natively read & write
|
| 123 |
+
neural network weights.* MIT CSAIL blog, 2026-11-02. https://rikiyat.github.io/blog/weight-space-programming/
|
| 124 |
+
2. Ahmed N. S. *Can an LLM Actually Reason and Program about Neural Network's Weights or is it just memorization on
|
| 125 |
+
steroids? Exploring "Weight-Space Programming".* arXiv:2611.08070, 2026.
|
| 126 |
+
|
| 127 |
+
Editing and merging weights:
|
| 128 |
+
|
| 129 |
+
3. Ilharco G., Ribeiro M. T., Wortsman M., Gururangan S., Schmidt L., Hajishirzi H., Farhadi A. *Editing Models with
|
| 130 |
+
Task Arithmetic.* ICLR 2023. arXiv:2212.04089.
|
| 131 |
+
4. Zheng C., Wang Z., Ji H., Huang M., Peng N. *Model Extrapolation Expedites Alignment* (ExPO). ACL 2025.
|
| 132 |
+
arXiv:2404.16792.
|
| 133 |
+
5. Yang E., Shen L., Guo G., Wang X., Cao X., Zhang J., Tao D. *Model Merging in LLMs, MLLMs, and Beyond: Methods,
|
| 134 |
+
Theories, Applications and Opportunities.* arXiv:2408.07666, 2024. Companion list:
|
| 135 |
+
https://github.com/EnnengYang/Awesome-Model-Merging-Methods-Theories-Applications
|
| 136 |
+
6. Jeong W., Lee W., Yoon K.-J. *Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing
|
| 137 |
+
Directional Anisotropy.* CVPR 2026. arXiv:2603.26299.
|
| 138 |
+
7. Roy P. R., Roy B., Schneider K. A., Roy C. K. *A Unified Model for Cross-Domain Clone Detection via Model Merging.*
|
| 139 |
+
ASE 2026. arXiv:2608.04215.
|
| 140 |
+
8. Chavan A., Sareen K., Tiwari R., Chandra A., Obando-Ceron J., Keutzer K., Niekum S., Dhillon I. S., Khatri D.
|
| 141 |
+
*Towards Continual Learning: Mitigating Forgetting With Subspace Tuning.* arXiv:2610.22293, 2026.
|
| 142 |
+
9. Li Y., Yi X., Shi D., Cui Y., de Melo G., Wang L. *From Construction to Injection: Edit-Based Fingerprints for
|
| 143 |
+
Large Language Models.* arXiv:2509.03122, 2025.
|
| 144 |
+
|
| 145 |
+
Learning over weights (metanetworks):
|
| 146 |
|
| 147 |
+
10. Zhou A., Yang K., Jiang Y., Burns K., Xu W., Sokota S., Kolter J. Z., Finn C. *Neural Functional Transformers.*
|
| 148 |
+
NeurIPS 2023. arXiv:2305.13546.
|
| 149 |
+
11. Wolanowsky Y., Bar-Shalom G., Frasca F., Maron H. *Graph Metanetworks for Transformer Based Language Models.*
|
| 150 |
+
arXiv:2610.18471, 2026.
|
| 151 |
+
12. Dayan A., Eitan Y., Maron H. *CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations.*
|
| 152 |
+
arXiv:2610.01649, 2026.
|
| 153 |
+
|
| 154 |
+
Moving knowledge across tokenizers:
|
| 155 |
+
|
| 156 |
+
13. Minixhofer B., Vulić I., Ponti E. M. *Universal Cross-Tokenizer Distillation via Approximate Likelihood
|
| 157 |
+
Matching.* NeurIPS 2025. arXiv:2503.20083.
|
| 158 |
+
14. Chen K., Lin C., Sun Y., Zhang W. *Dual-Vocabulary Language Model for Cross-Tokenizer Distillation.*
|
| 159 |
+
arXiv:2609.33816, 2026.
|
| 160 |
+
15. Liu X., Yu W., Fredrikson M., Wang X., Gao J. *When the Same Coefficients Reach Different Places: Asymmetric
|
| 161 |
+
Realizability in Transplanting Tokenizers across Large Language Models.* arXiv:2601.00065, 2025. First posted as
|
| 162 |
+
*The Trojan in the Vocabulary: Stealthy Sabotage of LLM Composition.*
|
| 163 |
+
|
| 164 |
+
Measurement:
|
| 165 |
+
|
| 166 |
+
16. Miller E. *Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations.* arXiv:2411.00640,
|
| 167 |
+
2024.
|
| 168 |
+
|
| 169 |
+
## Configuration fix
|
| 170 |
+
|
| 171 |
+
KOS-V5-Instruct also shipped with a configuration defect. Its `config.json` stores the RoPE base only inside
|
| 172 |
+
`rope_parameters`. vLLM 0.11 with transformers 4.57 reads the top-level `rope_theta`, finds none, and silently serves
|
| 173 |
+
the model at θ = 10,000. The model was trained at θ = 25,000. We confirmed this by inspecting the rotary layer vLLM
|
| 174 |
+
builds for each file.
|
| 175 |
+
|
| 176 |
+
V5.1 adds the top-level `"rope_theta": 25000.0` and sets `use_cache: true`. Under vLLM this restores long context,
|
| 177 |
+
shown in the RULER table below. Engines that read `rope_parameters` already ran KOS-V5-Instruct at 25,000, so the fix
|
| 178 |
+
changes nothing for them.
|
| 179 |
|
| 180 |
## Results against KOS-V5-Instruct
|
| 181 |
|
| 182 |
All comparisons below are paired runs on the same harnesses, measured on 2026-11-16. "V5" is KOS-V5-Instruct as
|
| 183 |
+
published, and changes are in percentage points.
|
| 184 |
+
|
| 185 |
+
Each benchmark uses its official package or the authors' released code. The only additions are the plumbing needed to
|
| 186 |
+
reach a locally served model. For the AfriMed-QA paper question set (n = 3,910, the paper's Table 3), one later-added
|
| 187 |
+
line of the released code that narrows the set to 160 questions is disabled.
|
| 188 |
+
|
| 189 |
+
### Identity (in-house probe, 15 held-out questions, greedy)
|
| 190 |
+
|
| 191 |
+
| | V5 | V5.1 |
|
| 192 |
+
| :--- | ---: | ---: |
|
| 193 |
+
| Answers as KOS (University of Kentucky and University of Louisville) | 0 / 15 | **14 / 15** |
|
| 194 |
+
| Claims another vendor made it (V5: almost always OpenAI) | 6 / 15 | **1 / 15** |
|
| 195 |
+
|
| 196 |
+
V5, asked "Which company developed you?", answered "I am an AI language model developed by OpenAI." The cause was the
|
| 197 |
+
V5 fine-tuning data. It contained the line "as an AI developed by OpenAI" about 1,800 times and never taught the model
|
| 198 |
+
its own identity.
|
| 199 |
|
| 200 |
### Decision battery
|
| 201 |
|
|
|
|
| 237 |
| 16K | 21.6 | **56.0** |
|
| 238 |
| 24,576 (trained context) | 11.0 | **31.1** |
|
| 239 |
|
| 240 |
+
Single-needle retrieval goes from 35–40% to 88–99% at 16K, and from 9–27% to 53–72% at 24,576. These gains come from
|
| 241 |
+
the configuration fix. The weight program moves RULER by at most ±2.6 points. Nothing is claimed beyond 24,576 tokens.
|
| 242 |
|
| 243 |
## Known regressions and limitations
|
| 244 |
|
|
|
|
| 248 |
- **Losses:** against KOS-V5-Instruct served at θ = 10,000, noise robustness is −6.7 to −11.3 and information
|
| 249 |
integration is −11.0.
|
| 250 |
- **Gain:** negative rejection is +12.7.
|
| 251 |
+
- **Cause:** the cost comes from the corrected RoPE base, not the weight program. The same weights at θ = 10,000
|
| 252 |
+
match V5 on RGB.
|
| 253 |
- **Rejected fix:** a frequency-hybrid RoPE configuration recovered information integration but cost Med-HALT NOTA
|
| 254 |
−6.3, so it was rejected.
|
| 255 |
- **Three RULER tasks** lose with the configuration fix: SQuAD-QA at 4K (−6.1), multikey-3 at 8K (−15.8) and CWE at
|
| 256 |
8K (−9.5).
|
| 257 |
- **IFEval** is 1.8 to 2.2 points lower than V5. The difference is not significant.
|
|
|
|
|
|
|
|
|
|
| 258 |
- **Not a knowledge model.** MMLU is 46.7. Do not use it as a medical question-answering authority.
|
| 259 |
- **MedXpertQA** is at chance level for both V5 and V5.1 (10.3 → 10.9, not significant).
|
| 260 |
|
|
|
|
| 321 |
|
| 322 |
## Related models
|
| 323 |
|
| 324 |
+
- [**KOS-V5-Instruct**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Instruct): the model this release reprograms.
|
| 325 |
- [**KOS-V5-Base**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Base): the from-scratch pretrained foundation
|
| 326 |
(3.72B, 235.2B tokens).
|
| 327 |
- [**KOS-V5-Retriever**](https://huggingface.co/Kentucky-Open-Science/KOS-V5-Retriever): a LoRA retrieval adapter
|
|
|
|
| 334 |
|
| 335 |
## Naming
|
| 336 |
|
| 337 |
+
The program is **KOS** (KOS-V1..V6). V5.1 is a weight-space-programmed release within the KOS-V5 generation.
|