qwev / README.md
NuJan's picture
Upload 2 files
2624d97 verified
|
Raw History Blame Contribute Delete
1.85 kB
---
license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B-Base
library_name: gguf
tags:
- gguf
- llama.cpp
- tool-routing
- on-device
- android
language: en
---
# Qwev
Qwev is the tool router inside NuJan, an AI agent app for Android.
## What it does
Qwev decides which tool families a request needs, so a small on-device chat model is offered a short tool list instead of all of them. The base model is Qwen/Qwen3.5-0.8B-Base, fine-tuned with LoRA (rank 32), merged, and quantised to GGUF Q4_K_M, about 529 MB. It runs on llama.cpp.
Given a request in a fixed prompt, Qwev outputs a probability for each of 17 route letters at one position. Those routes are 15 tool families, "core tools only", and "just answer". It does not generate text in normal use. qwev.router.json holds the prompt prefix and suffix, the route names, and the token id of each route letter, so a client can read the 17 probabilities directly.
## Files
| File | Contents |
| --- | --- |
| qwev-v1-Q4_K_M.gguf | GGUF Q4_K_M weights, about 529 MB |
| qwev.router.json | Prompt prefix and suffix, route names, and the token id of each route letter |
## How NuJan uses it
NuJan offers at most 2 families per request: the most likely ones until they cover 90% of the probability, none below 0.15, within the space left in the context window.
## Results
With Qwev, a 2B on-device model completed 10 of 12 test tasks on two phones, against 7 to 8 of 12 without it. A routing decision takes about 0.2 to 0.3 seconds on a recent flagship phone and about 1 second on a 2019 mid-range phone once warm.
## Limits
Qwev is for English requests. The 17 routes are NuJan's own tool families, so other apps would need their own mapping. It picks families, not individual tools. Tasks that need a sub-agent are a common miss.
## License
Apache-2.0, the same license as the base model.