Instructions to use Kentucky-Open-Science/KOS-V5-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Kentucky-Open-Science/KOS-V5-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Kentucky-Open-Science/KOS-V5-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Kentucky-Open-Science/KOS-V5-Instruct")
model = AutoModelForCausalLM.from_pretrained("Kentucky-Open-Science/KOS-V5-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Kentucky-Open-Science/KOS-V5-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Kentucky-Open-Science/KOS-V5-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Kentucky-Open-Science/KOS-V5-Instruct

SGLang

How to use Kentucky-Open-Science/KOS-V5-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Kentucky-Open-Science/KOS-V5-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Kentucky-Open-Science/KOS-V5-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kentucky-Open-Science/KOS-V5-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Kentucky-Open-Science/KOS-V5-Instruct with Docker Model Runner:
```
docker model run hf.co/Kentucky-Open-Science/KOS-V5-Instruct
```

codybum commited on Jul 18

Commit

a7a4ce7

verified ·

1 Parent(s): fa7fe5e

Card: Catbird logo + intro, official BFCL results

Browse files

Files changed (1) hide show

README.md +11 -4

README.md CHANGED Viewed

@@ -210,11 +210,18 @@ the answer-letter bottleneck below bites hardest.
 Measured with the **official `bfcl_eval`** suite in **FC (function-calling) mode**, non-live categories, the model
 prompted in its own native tool format and served via vLLM.
-| BFCL (official, FC mode) | **KOS-V5-Instruct** | KOS-V4-Instruct | Δ |
 | :-- | --: | --: | --: |
-| simple *(n=400)* | **83.25** | 72.75 | **+10.50** |
-| multiple *(n=200)* | **78.50** | 73.00 | **+5.50** |
-| parallel *(n=200)* | **73.50** | 60.50 | **+13.00** |
 **KOS-V5-Instruct improves on the published KOS-V4-Instruct on all three BFCL categories.** Tool calling was not
 a target of the GRPO objective — the reward ran the IFEval checker only — so this is carried capability from the

 Measured with the **official `bfcl_eval`** suite in **FC (function-calling) mode**, non-live categories, the model
 prompted in its own native tool format and served via vLLM.
+| BFCL (official, FC mode, **non-live AST**) | **KOS-V5-Instruct** | KOS-V4-Instruct | Δ |
 | :-- | --: | --: | --: |
+| simple *(334/400)* | **83.50** | 72.75 | **+10.75** |
+| multiple *(157/200)* | **78.50** | 73.00 | **+5.50** |
+| parallel *(147/200)* | **73.50** | 60.50 | **+13.00** |
+Measured twice, independently: the run above and an earlier one seven days prior gave **83.25 / 78.50 /
+73.50** — two categories identical, `simple` differing by a single item out of 400.
+> **Scope.** These are the **non-live AST** categories only (`simple_python`, `multiple`, `parallel`). The
+> live, multi-turn, web-search and memory categories were **not run**, so no BFCL *overall* score is reported
+> here — the suite's aggregate column is not meaningful when most categories are unrun.
 **KOS-V5-Instruct improves on the published KOS-V4-Instruct on all three BFCL categories.** Tool calling was not
 a target of the GRPO objective — the reward ran the IFEval checker only — so this is carried capability from the