Instructions to use KvcHub/KaiLu-3-Pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use KvcHub/KaiLu-3-Pro with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="KvcHub/KaiLu-3-Pro") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("KvcHub/KaiLu-3-Pro") model = AutoModelForCausalLM.from_pretrained("KvcHub/KaiLu-3-Pro", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use KvcHub/KaiLu-3-Pro with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf KvcHub/KaiLu-3-Pro:Q8_0 # Run inference directly in the terminal: llama cli -hf KvcHub/KaiLu-3-Pro:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf KvcHub/KaiLu-3-Pro:Q8_0 # Run inference directly in the terminal: llama cli -hf KvcHub/KaiLu-3-Pro:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf KvcHub/KaiLu-3-Pro:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf KvcHub/KaiLu-3-Pro:Q8_0
Use Docker
docker model run hf.co/KvcHub/KaiLu-3-Pro:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use KvcHub/KaiLu-3-Pro with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "KvcHub/KaiLu-3-Pro" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KvcHub/KaiLu-3-Pro", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/KvcHub/KaiLu-3-Pro:Q8_0
- SGLang
How to use KvcHub/KaiLu-3-Pro with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "KvcHub/KaiLu-3-Pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KvcHub/KaiLu-3-Pro", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "KvcHub/KaiLu-3-Pro" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "KvcHub/KaiLu-3-Pro", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use KvcHub/KaiLu-3-Pro with Ollama:
ollama run hf.co/KvcHub/KaiLu-3-Pro:Q8_0
- Unsloth Desktop
- Docker Model Runner
How to use KvcHub/KaiLu-3-Pro with Docker Model Runner:
docker model run hf.co/KvcHub/KaiLu-3-Pro:Q8_0
- Lemonade
How to use KvcHub/KaiLu-3-Pro with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull KvcHub/KaiLu-3-Pro:Q8_0
Run and chat with the model
lemonade run user.KaiLu-3-Pro-Q8_0
List all available models
lemonade list
- Atomic Chat
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("KvcHub/KaiLu-3-Pro")
model = AutoModelForCausalLM.from_pretrained("KvcHub/KaiLu-3-Pro", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))KaiLu 3 Pro
KaiLu 3 Pro is a 1-billion-parameter chat model trained from scratch by KvcHub. It is small enough to run on a laptop: about 1.1 GB as a GGUF, fast on a CPU and faster on Apple Silicon or a GPU. It is open source under the Apache License 2.0.
Files
| File | What it is |
|---|---|
KaiLu-3-Pro-Q8_0.gguf |
The whole model in one file for llama.cpp, Ollama and LM Studio: 8-bit weights, tokenizer and chat template (1.09 GB). |
model.safetensors |
The full-precision weights (bfloat16) in one file for Transformers (2.07 GB). It also holds 6 vision_projector.* tensors from KaiLu's experimental image add-on; Transformers skips them, and they don't work on their own. |
config.json, generation_config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja |
Settings, tokenizer and chat format for Transformers. |
Modelfile, template, params |
Ollama setup: chat format, temperature 0.2, stop words. |
Use it
Download everything with hf download KvcHub/KaiLu-3-Pro --local-dir KaiLu-3-Pro, or only the GGUF with hf download KvcHub/KaiLu-3-Pro KaiLu-3-Pro-Q8_0.gguf Modelfile --local-dir KaiLu-3-Pro.
Ollama
ollama run hf.co/KvcHub/KaiLu-3-Pro
The repo's template and params files give Ollama KaiLu's chat format and settings. In a downloaded folder, ollama create kailu-3-pro -f Modelfile does the same.
llama.cpp
llama-cli -m KaiLu-3-Pro-Q8_0.gguf --jinja -c 1024 --temp 0.2
llama-server -m KaiLu-3-Pro-Q8_0.gguf --jinja -c 1024 # OpenAI-compatible API on http://localhost:8080
LM Studio: import KaiLu-3-Pro-Q8_0.gguf. The chat template is stored in the file.
Transformers (4.54 or newer)
from transformers import AutoModelForCausalLM, AutoTokenizer
path = "KvcHub/KaiLu-3-Pro" # or a local folder
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path, dtype="bfloat16")
messages = [{"role": "user", "content": "What is the capital of France?"}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tok(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
A low temperature (0 to 0.3) gives the best answers; at 0.7 a model this size wanders.
Chat format
<|system|>
You are KaiLu, a helpful assistant.
<|user|>
Who are you?
<|assistant|>
I'm KaiLu 3 Pro, an AI assistant created by KvcHub.<|endoftext|>
If there's no system message, the template adds the default one above. Each assistant reply ends with <|endoftext|> (token 0). KaiLu also learned to reason inside <think>...</think> when asked, for example with the system message You are KaiLu, a helpful assistant. Mode: reasoning=medium. Think step-by-step inside <think>...</think> before your final answer.
Model details
| Parameters | 1,027,304,448 |
| Layers | 36: 24 gated short-convolution layers (kernel 3) and 12 attention layers (every third layer) |
| Width | hidden 1,536, SwiGLU feed-forward 4,096 |
| Attention | 24 query heads, 6 key-value heads (grouped-query), QK-norm, RoPE theta 1,000,000 |
| Context | trained on 1,024-token windows |
| Vocabulary | 32,768 tokens (byte-level BPE, trained by KvcHub) |
| Embeddings | input and output shared |
| Format | Lfm2ForCausalLM. KaiLu 3 Pro uses LFM2's layer layout only: it was trained from scratch and contains no LFM2 weights. |
How it was made
- KaiLu 3 base (329M parameters): pretrained from scratch on 6.6B tokens of educational web text (FineWeb-Edu), synthetic textbooks (Cosmopedia v2), web and books (SlimPajama-6B, without Books3), math (FineMath), Wikipedia and code (OpenCoder annealing). Then fine-tuned on smol-smoltalk conversations, and preference-tuned (DPO) on UltraFeedback, HelpSteer2 and Anthropic HH-RLHF pairs.
- Grown to 1B: widened (hidden 1,024 to 1,536) and deepened (24 to 36 layers) with a function-preserving expansion, so it started from what the 329M model already knew.
- Further pretraining: including the Dolmino Mix (DCLM web text, StackExchange, peS2o papers, FLAN and math).
- Fine-tuning: Tulu 3 persona instruction-following, MetaMathQA, GSM8K, MMLU, ARC-Challenge, HumanEval and General-Knowledge, plus examples written by KvcHub (step-by-step reasoning in
<think>, coding in several languages, and who KaiLu is). - Preference tuning (DPO, beta 0.1): pairs from UltraFeedback (binarized) and Orca DPO pairs.
The datasets are listed in this card's metadata. Each keeps its own license and terms.
Limitations
- It's a small model. It gets facts wrong, sometimes confidently (for example, it once answered "Japan is the capital of Japan"), and it can invent details. Don't rely on it for facts, medical, legal or financial advice.
- Short memory. It was trained on 1,024-token windows. It often doesn't use what you said earlier in a chat (asked "What is my name?" right after giving it, it says it doesn't know personal details), and it may answer only part of a multi-part question.
- English only.
- No benchmark scores here, on purpose. The test sets of MMLU, ARC-Challenge and HumanEval, and parts of GSM8K, were used in fine-tuning, so the model's scores on those benchmarks would overstate its ability. Evaluate it on data it hasn't seen.
- Text only. The
vision_projector.*tensors need KaiLu's own image encoder pipeline, which isn't part of this release.
License
Copyright 2026 KvcHub.
Licensed under the Apache License, Version 2.0 (see LICENSE). You may use, modify and redistribute the model, including commercially, under its terms.
- Downloads last month
- 48
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="KvcHub/KaiLu-3-Pro") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)