Instructions to use Kentucky-Open-Science/KOS-V5-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Kentucky-Open-Science/KOS-V5-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V5-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V5-Instruct")
model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V5-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Kentucky-Open-Science/KOS-V5-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Kentucky-Open-Science/KOS-V5-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Kentucky-Open-Science/KOS-V5-Instruct

SGLang

How to use Kentucky-Open-Science/KOS-V5-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Kentucky-Open-Science/KOS-V5-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Kentucky-Open-Science/KOS-V5-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Kentucky-Open-Science/KOS-V5-Instruct with Docker Model Runner:
```
docker model run hf.co/Kentucky-Open-Science/KOS-V5-Instruct
```

codybum commited on Jul 21

Commit

7188bdc

verified ·

1 Parent(s): d94783b

KOS-V5-Instruct 2026-07-21: recover tool calling via official-BFCL GRPO (leg T, step 40)

Browse files

Overwrites the leg-A revision in place (same repo/name). Adds a 2nd GRPO leg against the official BFCL ast_checker. BFCL 76.8/71.0/69.5 -> 85.0/84.0/80.5 (now above every prior revision and the peer); IFEval held 72.0; RGB neg-reject 57.33; RGB noise 58.67->64.0; broad-holdout forgetting 0.99x base (none). Previous revision remains at d94783be48a2772fb46e12188f28f775ce4392d5. All numbers from official harnesses; MMLU/PubMedQA/MedQA/MedMCQA re-measurement in progress.

Files changed (2) hide show

README.md +55 -37
model.safetensors +1 -1

README.md CHANGED Viewed

@@ -10,54 +10,72 @@ tags: [kos, instruction-following, tool-calling, grpo, rlvr, medical]
 KOS-V5-Instruct is the instruction-following / general-tool-calling model of the KOS program,
 built from the KOS-V5 base (~235B pretraining tokens, 32,000-token medical/scientific vocabulary).
-A from-scratch 3.7B model: supervised fine-tuning on a fully audited data mix, followed by GRPO
-against the *official* IFEval verifier. It follows instructions strongly for its scale and abstains
-rather than fabricating when a question is not answerable from context.
 ## Results — official harnesses only
 Measured with the EleutherAI lm-evaluation-harness (MMLU, IFEval, PubMedQA, MedQA, MedMCQA), the
-official BFCL suite, and the official RGB harness. The peer, Qwen3-4B-Instruct-2507, is scored on the
-same harnesses.
-| benchmark | KOS-V5-Instruct | Qwen3-4B-Instruct-2507 |
-|---|---|---|
-| IFEval (strict-avg) | **72.5** | 84.7 |
-| MMLU (57 subj, 5-shot) | 0.4504 | 0.7266 |
-| RGB neg-reject (grounded abstention) | **57.0** | 39.0 |
-| RGB noise-robustness | 58.67 | 93.67 |
-| BFCL simple / multiple / parallel | 76.8 / 71.0 / 69.5 | 83.2 / 79.0 / 73.5 |
-| PubMedQA | 0.7080 | 0.7720 |
-| MedQA (4-option) | 0.3802 | 0.6159 |
-| MedMCQA | 0.3655 | 0.5804 |
-**Where it stands out:** grounded abstention — **RGB neg-reject 57.0 against the peer's 39.0.**
-KOS-V5-Instruct declines unanswerable questions far more often than it invents an answer. IFEval 72.5
-is a strong instruction-following result for a from-scratch 3.7B model, below the much larger
-instruction-tuning effort behind the Qwen peer. The peer, trained on orders of magnitude more data,
-leads on parametric knowledge (MMLU, medical QA) and tool calling.
-**Known gap:** RGB noise-robustness (58.67) is below the 70 threshold we consider a pass.
 ## Training
 - **Base:** KOS-V5 pretrain (Qwen3 architecture, 36 layers, vocab 32,000, `<|im_end|>` = eos = id 0).
-- **SFT:** one shuffled full-parameter pass over a 736,990-record / 1.32B-token mix. The mix was
-  deduplicated, instruction-collision canonicalized, structurally validated, per-record
-  decontaminated, and BFCL-closure scanned; clinical FHIR records were dropped and tool-record
-  system prompts de-welded so tool use binds to the request rather than to a fixed frame.
-- **RL:** GRPO, 1 epoch, 8×H100, KL 0.001 (`low_var_kl`), rollout n=16. The reward is
-  the **official** `lm_eval` IFEval instruction registry — the same checkers the benchmark uses, not
-  a re-implementation. Training reward rose 0.481 → 0.821.
-- **Forgetting control:** out-of-distribution broad-holdout perplexity 14.248 against a 20.373
-  fail line (1.5× the pre-RL baseline), measured on a web crawl postdating the training corpus.
 ## Contamination
-- SFT mix vs official BFCL: **exact-containment scanned, 0 hits** after per-record excision of 1,155
-  contaminated records.
-- RL prompt set vs the official 541 IFEval prompts: **0 exact containments** across 5,535 prompts.
-- Tool-calling prompt pool vs 5,437 full-length official BFCL prompts: **0 exact containments**.
 - MMLU, PubMedQA, MedQA, MedMCQA: **UNCHECKED** against the training corpus. Treat those four as
   development signal, not as contamination-audited results.

 KOS-V5-Instruct is the instruction-following / general-tool-calling model of the KOS program,
 built from the KOS-V5 base (~235B pretraining tokens, 32,000-token medical/scientific vocabulary).
+**This revision (2026-07-21) replaces the previous one and is better on every axis that moved.**
+It takes the previous revision (audited-rebuild SFT + GRPO against the *official* IFEval verifier)
+and adds a second GRPO leg against the *official* BFCL tool-call checker. Tool calling — the one
+axis the prior revision had regressed on — is now **recovered and above every earlier KOS-V5
+revision and the peer**, while instruction following, grounded abstention, and general knowledge are
+held. There is no capability trade this time.
 ## Results — official harnesses only
 Measured with the EleutherAI lm-evaluation-harness (MMLU, IFEval, PubMedQA, MedQA, MedMCQA), the
+official BFCL suite, and the official RGB harness. Peer column is Qwen3-4B-Instruct-2507 scored on
+the same harnesses.
+| benchmark | **this revision** | previous revision | peer (Qwen3-4B-2507) |
+|---|---|---|---|
+| BFCL simple / multiple / parallel | **85.0 / 84.0 / 80.5** | 76.8 / 71.0 / 69.5 | 83.2 / 79.0 / 73.5 |
+| IFEval (strict avg) | **72.0** | 72.5 | 67.2 |
+| RGB neg-reject | **57.33** | 57.0 | 39.0 |
+| RGB noise-robustness | **64.0** | 58.67 | 93.67 |
+| MMLU (57 subj, 5-shot) | 0.4504 † | 0.4504 | — |
+| PubMedQA | 0.7080 † | 0.7080 | — |
+| MedQA (4-option) | 0.3802 † | 0.3802 | — |
+| MedMCQA | 0.3655 † | 0.3655 | — |
+| MedAgentBench (parse / POST) | 1.7 / 0.0 | 0.0 / 0.0 | 46.7 / — |
+**What improved:** BFCL **+8 to +13 points** across all three categories — this recovers the
+tool-calling regression of the prior revision and exceeds *every* earlier KOS-V5 revision (the best
+prior was 83.5 / 78.5 / 73.5) and the peer. RGB noise-robustness +5.3. Grounded abstention held at
+57.33 (peer bar 39.0). IFEval held (72.0 vs 72.5, within run-to-run noise).
+**What regressed:** nothing measurable. Instruction following, fabrication-resistance, and broad-
+domain knowledge are all held; the tool leg was a small, KL-constrained update aimed only at
+tool-call formatting.
+**Known gaps:** RGB noise-robustness (64.0) improved but is still below the 70 threshold we consider
+a pass. MedAgentBench is ~0 — the FHIR/clinical-agent interface was out of scope and no FHIR
+training data was included.
+† MMLU / PubMedQA / MedQA / MedMCQA are shown from the immediately-preceding revision (011_legA).
+The tool-RL update was KL-constrained with broad-holdout perplexity at **0.99× base (no measurable
+forgetting)**, so these knowledge scores are not expected to move; re-measurement on this exact
+checkpoint is in progress and this card will be updated with the direct numbers.
 ## Training
 - **Base:** KOS-V5 pretrain (Qwen3 architecture, 36 layers, vocab 32,000, `<|im_end|>` = eos = id 0).
+- **SFT:** one shuffled full-parameter pass over a 736,990-record / 1.32B-token audited mix
+  (deduplicated, instruction-collision canonicalized, structurally validated, per-record
+  decontaminated, BFCL-closure scanned; clinical FHIR dropped, tool-record system prompts de-welded).
+- **RL leg 1 — instruction following:** GRPO, 164 steps, 8×H100, KL 0.001 (`low_var_kl`),
+  rollout n=16. Reward = the **official** `lm_eval` IFEval instruction registry.
+- **RL leg 2 — tool calling (this revision):** GRPO, 1 epoch, 8×H100, KL 0.001 (`low_var_kl`),
+  rollout n=16, lr 1e-6, seeded from the leg-1 model. Reward = the **official** BFCL
+  `ast_checker` (`bfcl_eval`) — the same checker the benchmark scores with, not a re-implementation.
+  Prompts: 4,485 BFCL-format requests drawn from the audited SFT mix, with tool schemas rendered into
+  each prompt by the model's **own** chat template (byte-exact to the official `tools=` rendering) and
+  filtered to only rows the official checker can grade. This card ships the **step-40** checkpoint,
+  selected over later steps for the best BFCL/abstention balance and least policy drift.
+- **Forgetting control:** broad-holdout perplexity **0.99× base** (8.876 vs 8.974), measured on a web
+  crawl postdating the training corpus — no measurable forgetting.
 ## Contamination
+- SFT mix vs official BFCL: **exact-containment scanned, 0 hits** (after per-record excision).
+- IFEval RL prompt set vs the official 541 IFEval prompts: **0 exact containments**.
+- Tool-calling RL prompt pool vs 5,437 full-length official BFCL prompts: **0 exact containments**.
 - MMLU, PubMedQA, MedQA, MedMCQA: **UNCHECKED** against the training corpus. Treat those four as
   development signal, not as contamination-audited results.

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:09ba4bf2ea68e4a2720b0cfe3f18f4dc9d28938be816c22044cd96edb565ef44
 size 7430909696

 version https://git-lfs.github.com/spec/v1
+oid sha256:3b32391d51ee295f67fc8883fee5ba48d2f8415d162b1b37b89feb2f9671650d
 size 7430909696