Instructions to use Kentucky-Open-Science/KOS-V5.1-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V5.1-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V5.1-Instruct") model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V5.1-Instruct", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kentucky-Open-Science/KOS-V5.1-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kentucky-Open-Science/KOS-V5.1-Instruct
- SGLang
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V5.1-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kentucky-Open-Science/KOS-V5.1-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kentucky-Open-Science/KOS-V5.1-Instruct", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kentucky-Open-Science/KOS-V5.1-Instruct with Docker Model Runner:
docker model run hf.co/Kentucky-Open-Science/KOS-V5.1-Instruct
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V5.1-Instruct")
model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V5.1-Instruct", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))KOS-V5.1-Instruct · "Catbird"
Developed by
University of Kentucky
University of Louisville
KOS-V5.1-Instruct is the first KOS model built by weight-space programming. Its released weights were written by arithmetic on weight vectors, not taken from a training checkpoint. They are KOS-V5-Instruct plus two task vectors, applied to chosen layers of the feed-forward network and kept only where measurement showed a gain.
A configuration fix, described below, comes with it.
⚠️ Research use only. This model is provided for research purposes only and must not be used for any commercial, clinical, legal, or production-grade application. The user assumes all risks associated with its use.
Weight-space programming
The program
V5.1 = V5 + τ_identity (MLP, all 36 layers) − τ_replay (MLP, layers 12–23)
- τ_identity: the weight change made by an identity fine-tune of KOS-V5-Instruct.
- τ_replay: the weight change made by the same fine-tune run on its general replay data alone.
Adding τ_identity installs the model's own identity. Subtracting τ_replay removes the drift that the replay data causes. This is task arithmetic [3].
Only MLP weights change. Attention, embeddings and norms are bit-identical to KOS-V5-Instruct. The arithmetic is done
in fp32 and stored in bf16. Weights sha256: 7492922a4a80e14d3974e83e275b53a011abd183afe591579a9a8509be961199.
Where to subtract the replay drift (a layer-wise finding)
Subtracting τ_replay was tested on three blocks of 12 layers:
- Layers 12–23 (this model): removes the identity edit's hallucination-benchmark costs and keeps its tool-refusal and Med-HALT FCT gains.
- Layers 0–11: deepens the Med-HALT FQT cost (−6.3).
- Layers 24–35: gives the sharpest tool refusal, BFCL live_irrelevance +21.3. It keeps the Med-HALT costs.
Merging the two edits evenly, the midpoint of the task-vector dial, also failed on Med-HALT. Extrapolating the post-SFT change on layers 4–11, ExPO-style [4], undid the tool-refusal gain. Both were rejected.
How the program was selected
- Candidates: fourteen. Ten weight programs, three configuration-only variants, and one generation setting.
- Baseline: KOS-V5-Instruct as published.
- Decision battery: 27 metrics.
- BFCL: all 17 categories.
- Med-HALT, the authors' code: FCT, NOTA, FQT.
- AfriMed-QA MCQ, the authors' code, paper question set.
- IFEval: 4 metrics.
- TruthfulQA: MC1 and MC2.
- Gate: no metric significantly worse than KOS-V5-Instruct.
- Statistics: paired per item, exact McNemar or sign-flip tests, p < 0.05, following the paired-evaluation practice of [16].
- Ranking: by the number of metrics significantly better, then by summed deltas. The rule was fixed before any result was read.
- Cross-fit: items were split in half by hash. Choosing on either half and scoring on the other gave a passing program both times.
KOS-V5.1-Instruct is the only weight program that passed. It has 7 metrics significantly better and none worse.
Papers reviewed for this program
Weight-space programming and its critics:
- Takehi R., Teoh J., Zhang A., Khattab O. Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights. MIT CSAIL blog, 2026-11-02. https://rikiyat.github.io/blog/weight-space-programming/
- Ahmed N. S. Can an LLM Actually Reason and Program about Neural Network's Weights or is it just memorization on steroids? Exploring "Weight-Space Programming". arXiv:2611.08070, 2026.
Editing and merging weights:
- Ilharco G., Ribeiro M. T., Wortsman M., Gururangan S., Schmidt L., Hajishirzi H., Farhadi A. Editing Models with Task Arithmetic. ICLR 2023. arXiv:2212.04089.
- Zheng C., Wang Z., Ji H., Huang M., Peng N. Model Extrapolation Expedites Alignment (ExPO). ACL 2025. arXiv:2404.16792.
- Yang E., Shen L., Guo G., Wang X., Cao X., Zhang J., Tao D. Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities. arXiv:2408.07666, 2024. Companion list: https://github.com/EnnengYang/Awesome-Model-Merging-Methods-Theories-Applications
- Jeong W., Lee W., Yoon K.-J. Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy. CVPR 2026. arXiv:2603.26299.
- Roy P. R., Roy B., Schneider K. A., Roy C. K. A Unified Model for Cross-Domain Clone Detection via Model Merging. ASE 2026. arXiv:2608.04215.
- Chavan A., Sareen K., Tiwari R., Chandra A., Obando-Ceron J., Keutzer K., Niekum S., Dhillon I. S., Khatri D. Towards Continual Learning: Mitigating Forgetting With Subspace Tuning. arXiv:2610.22293, 2026.
- Li Y., Yi X., Shi D., Cui Y., de Melo G., Wang L. From Construction to Injection: Edit-Based Fingerprints for Large Language Models. arXiv:2509.03122, 2025.
Learning over weights (metanetworks):
- Zhou A., Yang K., Jiang Y., Burns K., Xu W., Sokota S., Kolter J. Z., Finn C. Neural Functional Transformers. NeurIPS 2023. arXiv:2305.13546.
- Wolanowsky Y., Bar-Shalom G., Frasca F., Maron H. Graph Metanetworks for Transformer Based Language Models. arXiv:2610.18471, 2026.
- Dayan A., Eitan Y., Maron H. CrossGMN: Graph Metanetworks for Cross-Architecture Weight-Space Transformations. arXiv:2610.01649, 2026.
Moving knowledge across tokenizers:
- Minixhofer B., Vulić I., Ponti E. M. Universal Cross-Tokenizer Distillation via Approximate Likelihood Matching. NeurIPS 2025. arXiv:2503.20083.
- Chen K., Lin C., Sun Y., Zhang W. Dual-Vocabulary Language Model for Cross-Tokenizer Distillation. arXiv:2609.33816, 2026.
- Liu X., Yu W., Fredrikson M., Wang X., Gao J. When the Same Coefficients Reach Different Places: Asymmetric Realizability in Transplanting Tokenizers across Large Language Models. arXiv:2601.00065, 2025. First posted as The Trojan in the Vocabulary: Stealthy Sabotage of LLM Composition.
Measurement:
- Miller E. Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations. arXiv:2411.00640,
Configuration fix
KOS-V5-Instruct also shipped with a configuration defect. Its config.json stores the RoPE base only inside
rope_parameters. vLLM 0.11 with transformers 4.57 reads the top-level rope_theta, finds none, and silently serves
the model at θ = 10,000. The model was trained at θ = 25,000. We confirmed this by inspecting the rotary layer vLLM
builds for each file.
V5.1 adds the top-level "rope_theta": 25000.0 and sets use_cache: true. Under vLLM this restores long context,
shown in the RULER table below. Engines that read rope_parameters already ran KOS-V5-Instruct at 25,000, so the fix
changes nothing for them.
Results against KOS-V5-Instruct
All comparisons below are paired runs on the same harnesses, measured on 2026-11-16. "V5" is KOS-V5-Instruct as published, and changes are in percentage points.
Each benchmark uses its official package or the authors' released code. The only additions are the plumbing needed to reach a locally served model. For the AfriMed-QA paper question set (n = 3,910, the paper's Table 3), one later-added line of the released code that narrows the set to 160 questions is disabled.
Identity (in-house probe, 15 held-out questions, greedy)
| V5 | V5.1 | |
|---|---|---|
| Answers as KOS (University of Kentucky and University of Louisville) | 0 / 15 | 14 / 15 |
| Claims another vendor made it (V5: almost always OpenAI) | 6 / 15 | 1 / 15 |
V5, asked "Which company developed you?", answered "I am an AI language model developed by OpenAI." The cause was the V5 fine-tuning data. It contained the line "as an AI developed by OpenAI" about 1,800 times and never taught the model its own identity.
Decision battery
| Benchmark | V5 | Change | p |
|---|---|---|---|
| Med-HALT FCT (authors' code) | 8.1 | +3.62 | 9e-106 |
| BFCL live_irrelevance (bfcl-eval, unedited) | 43.1 | +9.62 | 2e-13 |
| Med-HALT NOTA | 60.8 | +0.67 | 1e-5 |
| BFCL irrelevance | 64.6 | +7.08 | 2e-5 |
| BFCL multi_turn_base | 1.0 | +3.50 | 0.016 |
| TruthfulQA MC2 (lm-eval) | 43.4 | +0.31 | 0.016 |
| BFCL parallel_multiple | 70.5 | +5.00 | 0.041 |
| IFEval, four metrics (lm-eval, chat template) | 66.7 prompt-strict | −1.8 to −2.2 | ≥ 0.085 |
| Med-HALT FQT | 27.8 | −0.38 | 0.37 |
| AfriMed-QA MCQ, paper set (authors' code) | 32.7 | +0.46 | 0.59 |
| Other 12 BFCL categories, TruthfulQA MC1 | — | no significant change | ≥ 0.12 |
Reasoning and knowledge
| Benchmark | V5 | V5.1 | Change | p |
|---|---|---|---|---|
| BBH, 27 chain-of-thought tasks, 3-shot (lm-eval) | 40.52 | 42.60 | +2.09 | 0.0004 |
| MMLU-Pro, 14 subjects, 5-shot (lm-eval) | 29.36 | 30.23 | +0.87 | 0.029 |
| ASDiv chain-of-thought, 8-shot (lm-eval) | 38.3 | 41.0 | +2.78 | 3e-5 |
| HellaSwag, 10-shot | 46.7 | 46.9 | +0.21 | 0.011 |
| MMLU, 5-shot | 46.9 | 46.7 | −0.14 | 0.31 |
| GSM8K, 5-shot, strict | 52.1 | 51.6 | −0.45 | 0.62 |
| MathQA, LSAT-AR, ARC, BoolQ, PIQA, WinoGrande, OpenBookQA, COPA, CommonsenseQA, LAMBADA | — | — | no significant change | ≥ 0.05 |
On BBH, object counting rose +21.6, geometric shapes +18.4, multistep arithmetic +14.4 and formal fallacies +10.0. Web of lies fell −10.0.
Long context (RULER, 12 stock lm-eval tasks, vLLM)
| Context | V5 | V5.1 |
|---|---|---|
| 4K | 77.0 | 82.2 |
| 8K | 61.9 | 71.5 |
| 16K | 21.6 | 56.0 |
| 24,576 (trained context) | 11.0 | 31.1 |
Single-needle retrieval goes from 35–40% to 88–99% at 16K, and from 9–27% to 53–72% at 24,576. These gains come from the configuration fix. The weight program moves RULER by at most ±2.6 points. Nothing is claimed beyond 24,576 tokens.
Known regressions and limitations
Disclosed deliberately.
- Short-context RAG under noise (RGB, authors' code).
- Losses: against KOS-V5-Instruct served at θ = 10,000, noise robustness is −6.7 to −11.3 and information integration is −11.0.
- Gain: negative rejection is +12.7.
- Cause: the cost comes from the corrected RoPE base, not the weight program. The same weights at θ = 10,000 match V5 on RGB.
- Rejected fix: a frequency-hybrid RoPE configuration recovered information integration but cost Med-HALT NOTA −6.3, so it was rejected.
- Three RULER tasks lose with the configuration fix: SQuAD-QA at 4K (−6.1), multikey-3 at 8K (−15.8) and CWE at 8K (−9.5).
- IFEval is 1.8 to 2.2 points lower than V5. The difference is not significant.
- Not a knowledge model. MMLU is 46.7. Do not use it as a medical question-answering authority.
- MedXpertQA is at chance level for both V5 and V5.1 (10.3 → 10.9, not significant).
Optional setting: repetition penalty 1.05
Adding "repetition_penalty": 1.05 to generation_config.json also passed the gate, with 6 metrics better and none
worse. The pre-registered tie-break (count of significant gains) kept the shipped default. Against V5, this setting
gives:
- BFCL live_irrelevance +15.2 and irrelevance +9.6.
- IFEval −0.1 to +0.7.
- RGB noise robustness 4 to 6 points above V5.1's default.
- RGB negative rejection 5.7 points below V5.1's default.
Only engines that read the generation config apply it: vLLM's OpenAI server and transformers generate.
Core specifications
| Attribute | Detail |
|---|---|
| Architecture | Decoder-only Transformer (Qwen3ForCausalLM), Grouped-Query Attention |
| Parameters | 3.715 B |
| Hidden / Layers | 2560 / 36 |
| Attention | 32 query / 8 KV heads (GQA 4:1), head_dim 128, per-head QK-RMSNorm |
| Feed-forward | SwiGLU, intermediate 9728 |
| Vocabulary | 32,000, custom medical byte-level BPE (unchanged from V5) |
| Context length | 24,576 trained, 32,768 configured |
| Position encoding | RoPE, θ = 25,000 (now in both rope_theta and rope_parameters) |
| Embeddings | tied |
| Precision | bfloat16 (7.43 GB, single shard) |
Prompt / chat format (ChatML)
The chat template and tokenizer are byte-identical to KOS-V5-Instruct.
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Write a haiku about Kentucky. Do not use any commas.<|im_end|>
<|im_start|>assistant
Tool / function calling uses the model's native <tools> … </tools> schema block and <tool_call> … </tool_call>
response format. Pass your function schemas via the tokenizer's apply_chat_template(..., tools=[...]).
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Kentucky-Open-Science/KOS-V5.1-Instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
messages = [{"role": "user", "content": "Write a haiku about Kentucky. Do not use any commas."}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
vLLM: vllm serve Kentucky-Open-Science/KOS-V5.1-Instruct --max-model-len 24576. V5.1 runs at θ = 25,000 under vLLM
with no overrides.
Related models
- KOS-V5-Instruct: the model this release reprograms.
- KOS-V5-Base: the from-scratch pretrained foundation (3.72B, 235.2B tokens).
- KOS-V5-Retriever: a LoRA retrieval adapter built for KOS-V5-Instruct. It has not been tested with V5.1.
Intended use & limitations
Research use only. English only. Not for clinical, commercial, legal, or production-grade use. Outputs may be factually wrong or fabricated. This model must not be used to make or inform medical decisions.
Naming
The program is KOS (KOS-V1..V6). V5.1 is a weight-space-programmed release within the KOS-V5 generation.
- Downloads last month
- -
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V5.1-Instruct") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)