How to use from
Docker Model Runner
docker model run hf.co/KvcHub/KaiLu-Web:Q4_K_M
Quick Links

KaiLu in the browser

Browser-sized builds of KaiLu models for the chat on kvchub.com. The chat runs them entirely in your browser with wllama (llama.cpp compiled to WebAssembly), so nothing you type is sent anywhere.

File Model Size License
KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf, KaiLu-3-Pro-Q4_K_M-00002-of-00002.gguf KaiLu 3 Pro (1B), 4-bit Q4_K_M, split in two parts so browsers download them in parallel 623 MB Apache 2.0 (LICENSE-Apache-2.0)
KaiLu-3-Q8_0.gguf KaiLu 3 (329M), 8-bit Q8_0 351 MB KaiLu 3 License: free for personal, non-commercial use (LICENSE-KaiLu-3)

Full-precision versions and model cards: KvcHub/KaiLu-3-Pro and KvcHub/KaiLu3.

Use them in your own page

import { Wllama } from '@wllama/wllama';

const wllama = new Wllama({ default: '/wllama.wasm' });
await wllama.loadModelFromHF({ repo: 'KvcHub/KaiLu-Web', file: 'KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf' });
const reply = await wllama.createChatCompletion({
  messages: [{ role: 'user', content: 'What is the capital of France?' }],
  temperature: 0.2,
});

Point it at the first part and it loads both. llama.cpp also opens split files from the first part. Multi-threading needs the page served with Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp.

Downloads last month
41
GGUF
Model size
0.3B params
Architecture
lfm2
Hardware compatibility
Log In to add your hardware

4-bit

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support