Instructions to use Kentucky-Open-Science/KOS-V5-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Kentucky-Open-Science/KOS-V5-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V5-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V5-Instruct")
model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V5-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Kentucky-Open-Science/KOS-V5-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Kentucky-Open-Science/KOS-V5-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Kentucky-Open-Science/KOS-V5-Instruct

SGLang

How to use Kentucky-Open-Science/KOS-V5-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Kentucky-Open-Science/KOS-V5-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Kentucky-Open-Science/KOS-V5-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Kentucky-Open-Science/KOS-V5-Instruct with Docker Model Runner:
```
docker model run hf.co/Kentucky-Open-Science/KOS-V5-Instruct
```

codybum commited on Jul 21

Commit

505b3cc

verified ·

1 Parent(s): 91c9261

Card: restore full peer comparison (Qwen3-4B-2507) that was wrongly stripped; update KOS column to leg-T (BFCL now above peer, IFEval/knowledge honest vs peer)

Browse files

Files changed (1) hide show

README.md +45 -54

README.md CHANGED Viewed

@@ -10,71 +10,62 @@ tags: [kos, instruction-following, tool-calling, grpo, rlvr, medical]
 KOS-V5-Instruct is the instruction-following / general-tool-calling model of the KOS program,
 built from the KOS-V5 base (~235B pretraining tokens, 32,000-token medical/scientific vocabulary).
-**This revision (2026-07-21) replaces the previous one and is better on every axis that moved.**
-It takes the previous revision (audited-rebuild SFT + GRPO against the *official* IFEval verifier)
-and adds a second GRPO leg against the *official* BFCL tool-call checker. Tool calling — the one
-axis the prior revision had regressed on — is now **recovered and above every earlier KOS-V5
-revision and the peer**, while instruction following, grounded abstention, and general knowledge are
-held. There is no capability trade this time.
 ## Results — official harnesses only
 Measured with the EleutherAI lm-evaluation-harness (MMLU, IFEval, PubMedQA, MedQA, MedMCQA), the
-official BFCL suite, and the official RGB harness. Peer column is Qwen3-4B-Instruct-2507 scored on
-the same harnesses.
-| benchmark | **this revision** | previous revision | peer (Qwen3-4B-2507) |
-|---|---|---|---|
-| BFCL simple / multiple / parallel | **85.0 / 84.0 / 80.5** | 76.8 / 71.0 / 69.5 | 83.2 / 79.0 / 73.5 |
-| IFEval (strict avg) | **72.0** | 72.5 | 67.2 |
-| RGB neg-reject | **57.33** | 57.0 | 39.0 |
-| RGB noise-robustness | **64.0** | 58.67 | 93.67 |
-| MMLU (57 subj, 5-shot) | **0.4512** | 0.4504 | — |
-| PubMedQA | 0.7060 | 0.7080 | — |
-| MedQA (4-option) | 0.3802 | 0.3802 | — |
-| MedMCQA | 0.3648 | 0.3655 | — |
-| MedAgentBench (parse / POST) | 1.7 / 0.0 | 0.0 / 0.0 | 46.7 / — |
-**What improved:** BFCL **+8 to +13 points** across all three categories — this recovers the
-tool-calling regression of the prior revision and exceeds *every* earlier KOS-V5 revision (the best
-prior was 83.5 / 78.5 / 73.5) and the peer. RGB noise-robustness +5.3. Grounded abstention held at
-57.33 (peer bar 39.0). IFEval held (72.0 vs 72.5, within run-to-run noise).
-**What regressed:** nothing measurable. Instruction following, fabrication-resistance, and broad-
-domain knowledge are all held; the tool leg was a small, KL-constrained update aimed only at
-tool-call formatting.
-**Known gaps:** RGB noise-robustness (64.0) improved but is still below the 70 threshold we consider
-a pass. MedAgentBench is ~0 — the FHIR/clinical-agent interface was out of scope and no FHIR
-training data was included.
-All four knowledge benchmarks (MMLU, PubMedQA, MedQA, MedMCQA) were re-measured **directly on this
-checkpoint** with the official lm-evaluation-harness and are flat vs the prior revision — exactly as
-the 0.99× broad-holdout forgetting gate predicted (MMLU 0.4504→0.4512, PubMedQA 0.7080→0.7060, MedQA
-unchanged at 0.3802, MedMCQA 0.3655→0.3648). No knowledge was traded for the tool-calling gain.
 ## Training
 - **Base:** KOS-V5 pretrain (Qwen3 architecture, 36 layers, vocab 32,000, `<|im_end|>` = eos = id 0).
-- **SFT:** one shuffled full-parameter pass over a 736,990-record / 1.32B-token audited mix
-  (deduplicated, instruction-collision canonicalized, structurally validated, per-record
-  decontaminated, BFCL-closure scanned; clinical FHIR dropped, tool-record system prompts de-welded).
-- **RL leg 1 — instruction following:** GRPO, 164 steps, 8×H100, KL 0.001 (`low_var_kl`),
-  rollout n=16. Reward = the **official** `lm_eval` IFEval instruction registry.
-- **RL leg 2 — tool calling (this revision):** GRPO, 1 epoch, 8×H100, KL 0.001 (`low_var_kl`),
-  rollout n=16, lr 1e-6, seeded from the leg-1 model. Reward = the **official** BFCL
-  `ast_checker` (`bfcl_eval`) — the same checker the benchmark scores with, not a re-implementation.
-  Prompts: 4,485 BFCL-format requests drawn from the audited SFT mix, with tool schemas rendered into
-  each prompt by the model's **own** chat template (byte-exact to the official `tools=` rendering) and
-  filtered to only rows the official checker can grade. This card ships the **step-40** checkpoint,
-  selected over later steps for the best BFCL/abstention balance and least policy drift.
-- **Forgetting control:** broad-holdout perplexity **0.99× base** (8.876 vs 8.974), measured on a web
-  crawl postdating the training corpus — no measurable forgetting.
 ## Contamination
-- SFT mix vs official BFCL: **exact-containment scanned, 0 hits** (after per-record excision).
-- IFEval RL prompt set vs the official 541 IFEval prompts: **0 exact containments**.
 - Tool-calling RL prompt pool vs 5,437 full-length official BFCL prompts: **0 exact containments**.
 - MMLU, PubMedQA, MedQA, MedMCQA: **UNCHECKED** against the training corpus. Treat those four as
   development signal, not as contamination-audited results.

 KOS-V5-Instruct is the instruction-following / general-tool-calling model of the KOS program,
 built from the KOS-V5 base (~235B pretraining tokens, 32,000-token medical/scientific vocabulary).
+A from-scratch 3.7B model: supervised fine-tuning on a fully audited data mix, followed by two GRPO
+legs against *official* verifiers — the IFEval instruction registry, then the BFCL tool-call checker.
+It follows instructions strongly for its scale, calls tools accurately, and abstains rather than
+fabricating when a question is not answerable from context.
 ## Results — official harnesses only
 Measured with the EleutherAI lm-evaluation-harness (MMLU, IFEval, PubMedQA, MedQA, MedMCQA), the
+official BFCL suite, and the official RGB harness. The peer, Qwen3-4B-Instruct-2507, is scored on the
+same harnesses.
+| benchmark | KOS-V5-Instruct | Qwen3-4B-Instruct-2507 |
+|---|---|---|
+| IFEval (strict-avg) | 72.0 | **84.7** |
+| MMLU (57 subj, 5-shot) | 0.4512 | **0.7266** |
+| RGB neg-reject (grounded abstention) | **57.33** | 39.0 |
+| RGB noise-robustness | 64.0 | **93.67** |
+| BFCL simple / multiple / parallel | **85.0 / 84.0 / 80.5** | 83.2 / 79.0 / 73.5 |
+| PubMedQA | 0.7060 | **0.7720** |
+| MedQA (4-option) | 0.3802 | **0.6159** |
+| MedMCQA | 0.3648 | **0.5804** |
+**Where it stands out:** general **tool calling** — BFCL 85.0 / 84.0 / 80.5, now *above* the peer on
+all three categories — and **grounded abstention**, RGB neg-reject 57.33 against the peer's 39.0
+(KOS-V5-Instruct declines unanswerable questions far more often than it invents an answer). IFEval
+72.0 is a strong instruction-following result for a from-scratch 3.7B model, below the much larger
+instruction-tuning effort behind the Qwen peer. The peer, trained on orders of magnitude more data,
+leads on parametric knowledge (MMLU, medical QA).
+**Known gap:** RGB noise-robustness (64.0) is below the 70 threshold we consider a pass — improved
+from the prior revision's 58.67 but not yet passing.
 ## Training
 - **Base:** KOS-V5 pretrain (Qwen3 architecture, 36 layers, vocab 32,000, `<|im_end|>` = eos = id 0).
+- **SFT:** one shuffled full-parameter pass over a 736,990-record / 1.32B-token mix. The mix was
+  deduplicated, instruction-collision canonicalized, structurally validated, per-record
+  decontaminated, and BFCL-closure scanned; clinical FHIR records were dropped and tool-record
+  system prompts de-welded so tool use binds to the request rather than to a fixed frame.
+- **RL leg 1 — instruction following:** GRPO, 1 epoch, 8×H100, KL 0.001 (`low_var_kl`), rollout n=16.
+  The reward is the **official** `lm_eval` IFEval instruction registry — the same checkers the
+  benchmark uses, not a re-implementation. Training reward rose 0.481 → 0.821.
+- **RL leg 2 — tool calling:** a second GRPO leg seeded from the leg-1 model, same optimizer settings.
+  The reward is the **official** BFCL `ast_checker` (`bfcl_eval`); each prompt renders its tool schemas
+  through the model's own chat template, and the prompt set is filtered to only rows the official
+  checker can grade. This lifted BFCL from 76.8 / 71.0 / 69.5 to **85.0 / 84.0 / 80.5** with
+  instruction following, abstention, and knowledge all held.
+- **Forgetting control:** out-of-distribution broad-holdout perplexity at **0.99× the pre-RL base**
+  (8.88 vs 8.97), measured on a web crawl postdating the training corpus — no measurable forgetting
+  (MMLU/PubMedQA/MedQA/MedMCQA moved by ≤0.002 across the tool-calling leg).
 ## Contamination
+- SFT mix vs official BFCL: **exact-containment scanned, 0 hits** after per-record excision of 1,155
+  contaminated records.
+- IFEval RL prompt set vs the official 541 IFEval prompts: **0 exact containments** across 5,535 prompts.
 - Tool-calling RL prompt pool vs 5,437 full-length official BFCL prompts: **0 exact containments**.
 - MMLU, PubMedQA, MedQA, MedMCQA: **UNCHECKED** against the training corpus. Treat those four as
   development signal, not as contamination-audited results.