KaiLu-Web / README.md
KVC899's picture
Browser builds of KaiLu 3 Pro (Q4_K_M, 2 parts) and KaiLu 3 (Q8_0) for the kvchub.com in-browser chat
69d334e verified
|
Raw History Blame Contribute Delete
1.7 kB
---
license: other
license_name: per-file
license_link: https://huggingface.co/KvcHub/KaiLu-Web/blob/main/README.md
library_name: gguf
tags:
- kailu
- kvchub
- gguf
- wllama
- webassembly
- in-browser
---
# KaiLu in the browser
Browser-sized builds of KaiLu models for the chat on [kvchub.com](https://kvchub.com). The chat runs them entirely in your browser with [wllama](https://github.com/ngxson/wllama) (llama.cpp compiled to WebAssembly), so nothing you type is sent anywhere.
| File | Model | Size | License |
|---|---|---|---|
| `KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf`, `KaiLu-3-Pro-Q4_K_M-00002-of-00002.gguf` | KaiLu 3 Pro (1B), 4-bit Q4_K_M, split in two parts so browsers download them in parallel | 623 MB | Apache 2.0 (`LICENSE-Apache-2.0`) |
| `KaiLu-3-Q8_0.gguf` | KaiLu 3 (329M), 8-bit Q8_0 | 351 MB | KaiLu 3 License: free for personal, non-commercial use (`LICENSE-KaiLu-3`) |
Full-precision versions and model cards: [KvcHub/KaiLu-3-Pro](https://huggingface.co/KvcHub/KaiLu-3-Pro) and [KvcHub/KaiLu3](https://huggingface.co/KvcHub/KaiLu3).
## Use them in your own page
```js
import { Wllama } from '@wllama/wllama';
const wllama = new Wllama({ default: '/wllama.wasm' });
await wllama.loadModelFromHF({ repo: 'KvcHub/KaiLu-Web', file: 'KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf' });
const reply = await wllama.createChatCompletion({
messages: [{ role: 'user', content: 'What is the capital of France?' }],
temperature: 0.2,
});
```
Point it at the first part and it loads both. llama.cpp also opens split files from the first part. Multi-threading needs the page served with `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp`.