Instructions to use KvcHub/KaiLu-Web with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use KvcHub/KaiLu-Web with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf KvcHub/KaiLu-Web:Q4_K_M # Run inference directly in the terminal: llama cli -hf KvcHub/KaiLu-Web:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf KvcHub/KaiLu-Web:Q4_K_M # Run inference directly in the terminal: llama cli -hf KvcHub/KaiLu-Web:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf KvcHub/KaiLu-Web:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf KvcHub/KaiLu-Web:Q4_K_M
Use Docker
docker model run hf.co/KvcHub/KaiLu-Web:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use KvcHub/KaiLu-Web with Ollama:
ollama run hf.co/KvcHub/KaiLu-Web:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use KvcHub/KaiLu-Web with Docker Model Runner:
docker model run hf.co/KvcHub/KaiLu-Web:Q4_K_M
- Lemonade
How to use KvcHub/KaiLu-Web with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull KvcHub/KaiLu-Web:Q4_K_M
Run and chat with the model
lemonade run user.KaiLu-Web-Q4_K_M
List all available models
lemonade list
- Atomic Chat
KaiLu in the browser
Browser-sized builds of KaiLu models for the chat on kvchub.com. The chat runs them entirely in your browser with wllama (llama.cpp compiled to WebAssembly), so nothing you type is sent anywhere.
| File | Model | Size | License |
|---|---|---|---|
KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf, KaiLu-3-Pro-Q4_K_M-00002-of-00002.gguf |
KaiLu 3 Pro (1B), 4-bit Q4_K_M, split in two parts so browsers download them in parallel | 623 MB | Apache 2.0 (LICENSE-Apache-2.0) |
KaiLu-3-Q8_0.gguf |
KaiLu 3 (329M), 8-bit Q8_0 | 351 MB | KaiLu 3 License: free for personal, non-commercial use (LICENSE-KaiLu-3) |
Full-precision versions and model cards: KvcHub/KaiLu-3-Pro and KvcHub/KaiLu3.
Use them in your own page
import { Wllama } from '@wllama/wllama';
const wllama = new Wllama({ default: '/wllama.wasm' });
await wllama.loadModelFromHF({ repo: 'KvcHub/KaiLu-Web', file: 'KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf' });
const reply = await wllama.createChatCompletion({
messages: [{ role: 'user', content: 'What is the capital of France?' }],
temperature: 0.2,
});
Point it at the first part and it loads both. llama.cpp also opens split files from the first part. Multi-threading needs the page served with Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp.
- Downloads last month
- 41
Hardware compatibility
Log In to add your hardware
4-bit
8-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support