Add model card

README.md

@@ -0,0 +1,171 @@
+---
+license: apache-2.0
+base_model:
+  - jaredpalmer/kev-0.8b
+  - Qwen/Qwen3.5-0.8B-Base
+tags:
+  - vllm.cpp
+  - decisions
+  - systemone
+  - kev
+  - localai
+---
+
+# kev-0.8b for vllm.cpp
+
+This repository holds [jaredpalmer/kev-0.8b](https://huggingface.co/jaredpalmer/kev-0.8b)
+converted into the single-directory layout that
+[vllm.cpp](https://github.com/mudler/vllm.cpp) loads as `KevModel`.
+
+kev is a System 1 decision model. It answers typed `choice`, `noul` and
+`score` questions about a text `state` with one scoring pass per question. It
+does not generate text. kev is a frozen `Qwen/Qwen3.5-0.8B-Base` backbone, a
+rank-16 LoRA adapter, and a PointerHead readout.
+
+**This checkpoint only works with vllm.cpp** (directly, or through LocalAI's
+`vllm-cpp` backend). It is not a general-purpose checkpoint: `transformers`,
+vLLM and llama.cpp do not know the `KevModel` architecture or `head.safetensors`.
+
+## Redistribution and license
+
+This is a redistribution of upstream weights, converted for vllm.cpp and
+LocalAI. No weights were trained or changed here, other than the LoRA merge
+described below.
+
+- kev adapter and PointerHead: [jaredpalmer/kev-0.8b](https://huggingface.co/jaredpalmer/kev-0.8b)
+  @ `9a45d25eb2ab761841196625383fa1dff0e56c1e`, by Jared Palmer, Apache-2.0.
+  Reference code: [jaredpalmer/kev](https://github.com/jaredpalmer/kev).
+- Base backbone and tokenizer: [Qwen/Qwen3.5-0.8B-Base](https://huggingface.co/Qwen/Qwen3.5-0.8B-Base)
+  @ `dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68`, by the Qwen team, Apache-2.0.
+
+The upstream licenses apply to these files. Credit for the model belongs to
+the original authors.
+
+## Conversion
+
+The converter is `scripts/convert-kev.py` from vllm.cpp at commit
+`96788348627b6a079fcc3ef7fc6676a970965d7b`. It merges the LoRA once
+(`W' = W + 2.0 * (B @ A)`, r=16, alpha=32, math in F32, stored in the base
+dtype BF16), drops the base's vision and MTP tensors, converts `head.pt` to
+`head.safetensors`, and writes a `config.json` with `architectures: ["KevModel"]`.
+
+```sh
+hf download jaredpalmer/kev-0.8b --revision 9a45d25eb2ab761841196625383fa1dff0e56c1e \
+  --local-dir kev-0.8b
+hf download Qwen/Qwen3.5-0.8B-Base --revision dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68 \
+  --local-dir Qwen3.5-0.8B-Base
+python3 scripts/convert-kev.py kev-0.8b \
+  --base-model-dir Qwen3.5-0.8B-Base --output-dir kev-0.8b-vllm-cpp
+```
+
+The converter logged: 186 of 186 LoRA modules merged, 168 vision/MTP tensors
+dropped, 320 tensors written. Environment: torch 2.11.0, safetensors 0.7.0, CPU.
+
+## Files
+
+| file | bytes | content |
+|---|---|---|
+| `model.safetensors` | 1,504,832,320 | merged Qwen3.5-0.8B backbone, BF16 |
+| `head.safetensors` | 2,099,504 | PointerHead `q`/`k` weight and bias, F32 |
+| `config.json` | 2,526 | base config, `architectures: ["KevModel"]`, `kev_head_dim: 256`, `kev_temperature: 2.3510958125672174` |
+| `meta.json` | 258 | head metadata from `head.pt` |
+| `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt` | | copied from the base model |
+
+## Serve it with vllm.cpp
+
+Build the vllm.cpp server (`-DVLLM_CPP_SERVER=ON`, target `vllm-server`), then:
+
+```sh
+hf download mudler/kev-0.8b-vllm-cpp --revision c17e73666ded1e9d284470eae7e0de9a27294e77 --local-dir kev-0.8b-vllm-cpp
+build/examples/vllm-server --model kev-0.8b-vllm-cpp --served-model-name kev --port 8000
+```
+
+The server logs `kev decision model (KevModel); serving /v1/systemone`.
+
+## Serve it with LocalAI
+
+A model config for LocalAI's `vllm-cpp` backend:
+
+```yaml
+name: kev-0.8b
+backend: vllm-cpp
+known_usecases:
+  - decisions
+parameters:
+  model: mudler/kev-0.8b-vllm-cpp
+artifacts:
+  - name: model
+    target: model
+    source:
+      type: huggingface
+      repo: mudler/kev-0.8b-vllm-cpp
+      revision: c17e73666ded1e9d284470eae7e0de9a27294e77
+```
+
+Then send the same request body to LocalAI's `POST /v1/systemone` with
+`"model": "kev-0.8b"`.
+
+## Example
+
+Request (served on CPU from this exact directory, 2026-09-30):
+
+```sh
+curl http://localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
+  "model": "kev",
+  "state": "I was charged twice for my subscription this month. Please refund the duplicate payment.",
+  "questions": {
+    "refund": {"type": "noul", "instructions": "Does the user request a refund?"},
+    "department": {"type": "choice", "instructions": "Which department should handle this?",
+                   "criteria": {"billing": "Payments and refunds", "technical": "Software bugs", "sales": "New purchases"}},
+    "urgency": {"type": "score", "instructions": "How urgent is this?",
+                "criteria": ["Routine", "Urgent", "Emergency"]}}}'
+```
+
+Response (the `rl_agent` fields are omitted here):
+
+```json
+{
+  "answers": {
+    "department": {"type": "choice", "choice": "billing", "confidence": 0.9264,
+                   "probabilities": {"billing": 0.951, "sales": 0.0264, "technical": 0.0227}},
+    "refund": {"type": "noul", "noul": 0.9667},
+    "urgency": {"type": "score", "score": 0.7069, "confidence": 0.7244,
+                "legend": {"0": "Routine", "1": "Urgent", "2": "Emergency"},
+                "probabilities": {"0": 0.4222, "1": 0.4487, "2": 0.1291}}
+  },
+  "model": "kev",
+  "usage": {"input_tokens": 111, "output_tokens": 0},
+  "latency_ms": 2451.45
+}
+```
+
+kev's answer semantics follow the kev reference: `choice` confidence is
+`(max(p) - 1/K) / (1 - 1/K)`, `score` is the expected level index with
+confidence `1 - E|level - mode| / (L - 1)`, `noul` is `p(true)`, and
+probabilities are rounded to 4 decimals in this response.
+
+## What was verified and what was not
+
+Verified for this upload:
+
+- The conversion above ran at the stated vllm.cpp commit and merged all 186
+  LoRA modules.
+- The directory loads in the vllm.cpp CPU server built from the same commit,
+  and the one request above returns the answers shown. The answers are
+  plausible for the input. This is a smoke test, not an accuracy measurement.
+
+Recorded by the vllm.cpp project, not re-measured for this upload:
+
+- PointerHead golden-vector tests (25 cases) and LoRA-merge / hidden-state
+  goldens against the kev Python reference at
+  `jaredpalmer/kev@19dcae9b6e3e1a48200c5825aad9fc200d31e20a`.
+- A 5-case end-to-end comparison (choice 2 and 3 options, score 5 levels,
+  noul, multi-question) against the kev reference server, recorded as equal.
+
+Not verified:
+
+- No GPU (CUDA, ROCm, Metal) run of this checkpoint.
+- No comparison of this upload against the kev reference server. No vLLM
+  gate exists for kev, because vLLM does not serve this architecture.
+- No accuracy benchmark.
+- No GGUF or quantized variant.