Instructions to use KvcHub/KaiLu-Web with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use KvcHub/KaiLu-Web with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf KvcHub/KaiLu-Web:Q4_K_M # Run inference directly in the terminal: llama cli -hf KvcHub/KaiLu-Web:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf KvcHub/KaiLu-Web:Q4_K_M # Run inference directly in the terminal: llama cli -hf KvcHub/KaiLu-Web:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf KvcHub/KaiLu-Web:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf KvcHub/KaiLu-Web:Q4_K_M
Use Docker
docker model run hf.co/KvcHub/KaiLu-Web:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use KvcHub/KaiLu-Web with Ollama:
ollama run hf.co/KvcHub/KaiLu-Web:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use KvcHub/KaiLu-Web with Docker Model Runner:
docker model run hf.co/KvcHub/KaiLu-Web:Q4_K_M
- Lemonade
How to use KvcHub/KaiLu-Web with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull KvcHub/KaiLu-Web:Q4_K_M
Run and chat with the model
lemonade run user.KaiLu-Web-Q4_K_M
List all available models
lemonade list
- Atomic Chat
|
Download README.md from KvcHub/KaiLu-Web: direct link, hf CLI and curl.
- Browser
- Download file 1.7 kB
-
https://huggingface.co/KvcHub/KaiLu-Web/resolve/main/README.md
- Command line
-
hf download hf://KvcHub/KaiLu-Web/README.md
-
curl -L -o README.md https://huggingface.co/KvcHub/KaiLu-Web/resolve/main/README.md
1.7 kB
| license: other | |
| license_name: per-file | |
| license_link: https://huggingface.co/KvcHub/KaiLu-Web/blob/main/README.md | |
| library_name: gguf | |
| tags: | |
| - kailu | |
| - kvchub | |
| - gguf | |
| - wllama | |
| - webassembly | |
| - in-browser | |
| # KaiLu in the browser | |
| Browser-sized builds of KaiLu models for the chat on [kvchub.com](https://kvchub.com). The chat runs them entirely in your browser with [wllama](https://github.com/ngxson/wllama) (llama.cpp compiled to WebAssembly), so nothing you type is sent anywhere. | |
| | File | Model | Size | License | | |
| |---|---|---|---| | |
| | `KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf`, `KaiLu-3-Pro-Q4_K_M-00002-of-00002.gguf` | KaiLu 3 Pro (1B), 4-bit Q4_K_M, split in two parts so browsers download them in parallel | 623 MB | Apache 2.0 (`LICENSE-Apache-2.0`) | | |
| | `KaiLu-3-Q8_0.gguf` | KaiLu 3 (329M), 8-bit Q8_0 | 351 MB | KaiLu 3 License: free for personal, non-commercial use (`LICENSE-KaiLu-3`) | | |
| Full-precision versions and model cards: [KvcHub/KaiLu-3-Pro](https://huggingface.co/KvcHub/KaiLu-3-Pro) and [KvcHub/KaiLu3](https://huggingface.co/KvcHub/KaiLu3). | |
| ## Use them in your own page | |
| ```js | |
| import { Wllama } from '@wllama/wllama'; | |
| const wllama = new Wllama({ default: '/wllama.wasm' }); | |
| await wllama.loadModelFromHF({ repo: 'KvcHub/KaiLu-Web', file: 'KaiLu-3-Pro-Q4_K_M-00001-of-00002.gguf' }); | |
| const reply = await wllama.createChatCompletion({ | |
| messages: [{ role: 'user', content: 'What is the capital of France?' }], | |
| temperature: 0.2, | |
| }); | |
| ``` | |
| Point it at the first part and it loads both. llama.cpp also opens split files from the first part. Multi-threading needs the page served with `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp`. | |