KaiLu-Web / README.md
KVC899's picture
Browser builds of KaiLu 3 Pro (Q4_K_M, 2 parts) and KaiLu 3 (Q8_0) for the kvchub.com in-browser chat
69d334e verified
|
Raw History Blame Contribute Delete
1.7 kB
metadata
license: other
license_name: per-file
license_link: https://huggingface.co/KvcHub/KaiLu-Web/blob/main/README.md
library_name: gguf
tags:
  - kailu
  - kvchub
  - gguf
  - wllama
  - webassembly
  - in-browser

KaiLu in the browser

Browser-sized builds of KaiLu models for the chat on kvchub.com. The chat runs them entirely in your browser with wllama (llama.cpp compiled to WebAssembly), so nothing you type is sent anywhere.

File Model Size License
KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf, KaiLu-3-Pro-Q4_K_M-00002-of-00002.gguf KaiLu 3 Pro (1B), 4-bit Q4_K_M, split in two parts so browsers download them in parallel 623 MB Apache 2.0 (LICENSE-Apache-2.0)
KaiLu-3-Q8_0.gguf KaiLu 3 (329M), 8-bit Q8_0 351 MB KaiLu 3 License: free for personal, non-commercial use (LICENSE-KaiLu-3)

Full-precision versions and model cards: KvcHub/KaiLu-3-Pro and KvcHub/KaiLu3.

Use them in your own page

import { Wllama } from '@wllama/wllama';

const wllama = new Wllama({ default: '/wllama.wasm' });
await wllama.loadModelFromHF({ repo: 'KvcHub/KaiLu-Web', file: 'KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf' });
const reply = await wllama.createChatCompletion({
  messages: [{ role: 'user', content: 'What is the capital of France?' }],
  temperature: 0.2,
});

Point it at the first part and it loads both. llama.cpp also opens split files from the first part. Multi-threading needs the page served with Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp.