--- license: apache-2.0 base_model: openbmb/MiniCPM5-2B library_name: gguf pipeline_tag: text-generation language: - en tags: - tool-calling - voice-assistant - router - lora - gguf - hermes-agent --- # vessel-handoff-MiniCPM5-2B A small, fast decision model for voice assistants: **does this spoken request need the agent's tools, or can the voice just answer?** Fine-tuned from [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) for [Hermes Vessel](https://github.com/Ex8-ca/hermes-vessel)'s private **Local** voice. There, a full-duplex speech model (MiniCPM-o 4.5) does the talking but can't use tools, Whisper turns the user's speech into text, and this model decides, in about a tenth of a second on a GPU or 1-2 s on a CPU, when to hand the user's words to the [Hermes](https://github.com/NousResearch/hermes-agent) agent. | File | What | Size | |---|---|---| | `MiniCPM5-2B-vessel-handoff-Q6_K.gguf` | recommended (used by Vessel's installer; same scores as Q8_0) | 2.1 GB | | `MiniCPM5-2B-vessel-handoff-Q8_0.gguf` | near-lossless | 2.7 GB | | `lora/` | the LoRA adapter (PEFT) on `openbmb/MiniCPM5-2B` | 100 MB | | `router_prompt.json` | the system prompt, tool definition and request settings it was trained and measured with | | ## Use ```bash llama-server -m MiniCPM5-2B-vessel-handoff-Q6_K.gguf --jinja -c 4096 --port 8799 # llama.cpp b12177 or newer ``` Send the system prompt and `use_tools` tool from `router_prompt.json`, the user's transcribed words as the user message, `temperature: 0`, thinking off (`chat_template_kwargs: {"enable_thinking": false}`). Only the decision is needed, so `max_tokens: 4` is enough: a reply that starts with `