Knack v3: GGUF for Ollama / llama.cpp + knack.py

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+knack-v3-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+knack-v3-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -4,7 +4,10 @@ tags:
   - devshub
   - decision-model
   - developer-tools
+  - gguf
+  - ollama
 pipeline_tag: text-classification
+inference: false
 ---
 
 # CodeSync Knack
@@ -19,54 +22,138 @@ in about 10 milliseconds.
 - **Free.** Ships with every CodeSync install. No API key, no account, no usage limits.
 - **Private.** Runs locally. Your prompts and commands never leave your machine to be judged.
 - **Fast.** About 10 ms per decision on a GPU, about 250 ms on a plain CPU.
-- **Small.** About 1 GB on disk. Installs itself in the background after CodeSync starts.
+- **Small.** 0.5B parameters. 546 MB (Q8_0) or 371 MB (Q4_K_M) in Ollama.
 - **Accurate where it matters.** Trained on real CodeSync sessions, with extra focus on the
   rare cases that cost the most when missed: destructive deletes, commands that never exit,
   resuming interrupted work, refactor / test / documentation requests.
 
-## Results
+## Try it in one line
 
-Measured on CodeSync's own judgments (version v3, November 2026):
+You need [Ollama](https://ollama.com/download) running. Nothing else to install: the script
+uses only the Python standard library and pulls the model on first run.
 
-| Benchmark | Knack v3 | What it measures |
-|---|---:|---|
-| Hand-labelled set | **97.9%** | 233 decisions labelled by hand |
-| Held-out set | **95.5%** | 3,267 decisions from real sessions, never trained on |
-| Rare-case set | **93.7%** | 4,120 decisions on the hardest, rarest cases |
-| Latency | **13 ms** | median per request on a laptop GPU |
+```bash
+curl -fsSL https://huggingface.co/devshub-ai/knack/resolve/main/knack.py | python3 - "rm -rf build/"
+```
 
-## How to get it
+```
+risk           2             0.93   0 safe, 1 needs care, 2 destructive / dangerous
+long_running   no            0.97   keeps running (server, watch, poll, tail -f)
+activity       running       0.95   what kind of command it is
+...
+```
 
-Install CodeSync. That's it.
+Judge a user message instead of a command:
 
 ```bash
-curl -fsSL https://app.devshub.ai/install.sh | bash
+curl -fsSL https://huggingface.co/devshub-ai/knack/resolve/main/knack.py | python3 - --turn "fix the failing login test"
 ```
 
-The installer stays as fast as before. On first start, CodeSync downloads Knack in the
-background and keeps working while it does. Once Knack is ready it becomes CodeSync's first
-judgment engine, and new versions are picked up automatically.
+Or save the script and use it as a tool or a local server:
+
+```bash
+curl -fsSLO https://huggingface.co/devshub-ai/knack/resolve/main/knack.py
+python3 knack.py "docker compose up" --json          # machine-readable
+python3 knack.py --turn "add dark mode" -q route.mode -q task.intent
+python3 knack.py serve --port 33727                    # POST /v1/systemone (CodeSync format)
+```
+
+## Use it with Ollama directly
+
+```bash
+ollama pull hf.co/devshub-ai/knack:Q8_0      # or :Q4_K_M (smaller, near-identical accuracy)
+```
+
+Knack is a classifier, not a chat model. A request is a **state** (one `field: value` line
+per field) followed by one **question id**. The answer is the next token: a letter `A`, `B`,
+`C`... that picks the question's option in the order listed below.
+
+```bash
+curl -s localhost:11434/api/generate -d '{
+  "model": "hf.co/devshub-ai/knack:Q8_0", "raw": true, "stream": false,
+  "logprobs": true, "top_logprobs": 20, "options": {"temperature": 0, "num_predict": 1},
+  "prompt": "<|im_start|>user\ncommand: tail -f app.log\ndescription: <|im_end|>\n<|im_start|>assistant\nlong_running:"
+}'
+```
 
-To turn it off: `CODESYNC_KNACK_DISABLED=1`.
+**Read the answer from `logprobs`, not from `response`.** A question with N options uses
+only the first N letters. Take the probabilities of those letters, renormalise them, and
+pick the highest (for `long_running` above: `A` = yes, `B` = no). The generated letter on
+its own can be an unused letter. `knack.py` does this for you.
 
 ## What it decides
 
-Knack answers CodeSync's fixed set of 29 decisions:
+**Every shell command** (state fields: `command`, `description`):
+
+| id | options (A, B, C...) |
+|---|---|
+| `risk` | 0 safe, 1 needs care, 2 destructive |
+| `long_running` | yes, no |
+| `activity` | testing, building, installing, git, inspecting, running |
+| `verifies` | yes, no |
+| `mutates_repo` | yes, no |
+| `deploys` | yes, no |
+| `runtime_check` | yes, no |
+| `serves` | yes, no |
+
+**Every user turn** (state fields, in this order: `prompt`, `current_mode`,
+`recent_activity`, `conversation_size`, `open_task`, `turns_on_open_task`,
+`interrupted_work`, `browser_context`; see `turn_state()` in `knack.py` for defaults):
+
+| id | options (A, B, C...) |
+|---|---|
+| `turn.chat_or_task` | chat, task |
+| `turn.actionable` / `turn.question_shaped` / `turn.new_task` / `turn.resume` / `turn.mission` / `turn.plan_authoring` / `turn.demo` / `turn.skip_orientation` | yes, no |
+| `turn.change_intent` | create, adjust, enhance, fix, refactor, none |
+| `turn.screencast` | start, stop, none |
+| `task.intent` | bug_fix, feature, refactor, api_schema, config_deploy, test, documentation, investigation, unknown |
+| `task.execution_strategy` | direct, plan, parallel |
+| `route.mode` | plan, build, ask, debug |
+| `route.tier` | light, medium, heavy |
+| `route.thinking` | off, low, medium, high, xhigh, max |
+| `route.risk` | low, medium, high |
+| `browser` / `messaging` / `delivery.feature_task` / `delivery.slash_task` | yes, no |
+
+Knack only knows these 29 questions. Inside CodeSync, anything else goes to CodeSync's cloud
+judgment service, so a new kind of question never gets a guess.
+
+## Results
+
+Measured on CodeSync's own judgments (version v3, November 2026):
 
-- **Every turn (21):** task intent, routing and reasoning depth, plan / ask / build mode,
-  new task vs continuing work, resume requests, screencast start and stop, browser and
-  messaging needs, whether the request is chat or a task, and more.
-- **Every shell command (8):** risk level, activity type, long-running or not, whether it
-  verifies, mutates the repo, deploys, needs a running app, or starts a server.
+| Benchmark | Knack v3 | Ollama Q8_0 | Ollama Q4_K_M | What it measures |
+|---|---:|---:|---:|---|
+| Hand-labelled set | **97.9%** | 97.9% | 97.4% | 233 decisions labelled by hand |
+| Held-out set | **95.5%** | 95.7% | 95.7% | 3,267 decisions from real sessions, never trained on |
+| Rare-case set | **93.7%** | 93.6% | 93.6% | 4,120 decisions on the hardest, rarest cases |
+| Latency | **13 ms** | ~80 ms | ~80 ms | median per command (8 questions) on a laptop GPU |
+
+The Ollama files run one request per question; CodeSync's built-in runtime answers all
+questions of a turn in a single forward pass, which is why it is faster.
+
+## Use it inside CodeSync
+
+Install CodeSync. That's it.
+
+```bash
+curl -fsSL https://app.devshub.ai/install.sh | bash
+```
 
-It only answers the questions it was trained on. Anything else is passed to CodeSync's
-cloud judgment service, so a new kind of question never gets a guess.
+On first start, CodeSync downloads Knack in the background and keeps working while it
+does. Once Knack is ready it becomes CodeSync's first judgment engine, and new versions are
+picked up automatically. To turn it off: `CODESYNC_KNACK_DISABLED=1`.
 
 ## Files
 
 | File | Purpose |
 |---|---|
-| `manifest.json` | current version, archive URL, sha256 (read by CodeSync) |
-| `knack-<version>.tar.gz` | the model folder |
+| `knack-v3-Q8_0.gguf`, `knack-v3-Q4_K_M.gguf` | Ollama / llama.cpp edition |
+| `knack.py` | standalone runner (CLI and local server), standard library only |
+| `template`, `params` | Ollama prompt template and parameters (raw prompt, 1 token, temperature 0) |
+| `manifest.json`, `knack-<version>.tar.gz` | the PyTorch model folder CodeSync installs |
+
+The GGUF edition is the same weights with an untied output head: the rows of the answer
+letters carry a shared offset, so the letters always lead the vocabulary while their relative
+probabilities, and so every answer, stay the same.
 
 Built by [DevsHub](https://app.devshub.ai) for [CodeSync](https://app.devshub.ai).

knack-v3-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c55e4831fc36f7adf5bd90573bb3c2c81c0409451ddebd2e13bdce9f49ea1095
+size 515961664

knack-v3-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2eed77e11930251a46eed2bf1cf6401e8fe499c2a953bca5e8fdaa9bc4eeeb6b
+size 816653120

knack.py

@@ -0,0 +1,226 @@
+#!/usr/bin/env python3
+"""CodeSync Knack, standalone: ask the model CodeSync's judgment questions through Ollama.
+
+Standard library only. Needs Ollama (https://ollama.com) running; the model (546 MB) is
+pulled on first use. One-liner, nothing to install:
+
+    curl -fsSL https://huggingface.co/devshub-ai/knack/resolve/main/knack.py | python3 - "rm -rf build/"
+
+Usage:
+
+    python knack.py "rm -rf build/"                 # the 8 shell-command questions
+    python knack.py --turn "fix the failing login test"   # the 21 per-turn questions
+    python knack.py --turn "add dark mode" --mode plan --json
+    python knack.py serve --port 33727              # CodeSync-compatible POST /v1/systemone
+
+How Knack answers: the prompt is a state (``field: value`` lines) followed by one question
+id, and the answer is the next token, a letter ``A``, ``B``, ``C``... picking the question's
+option in order. Only the first N letters belong to a question with N options, so the
+answer is the most likely of THOSE letters, renormalised (this script does that from
+Ollama's logprobs). Do not read the raw generated letter on its own.
+
+Env: KNACK_OLLAMA_HOST (default http://127.0.0.1:11434),
+     KNACK_MODEL (default hf.co/devshub-ai/knack:Q8_0; :Q4_K_M is smaller, 371 MB).
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import math
+import os
+import sys
+import urllib.error
+import urllib.request
+from concurrent.futures import ThreadPoolExecutor
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from typing import Any
+
+HOST = os.environ.get("KNACK_OLLAMA_HOST", "http://127.0.0.1:11434").rstrip("/")
+MODEL = os.environ.get("KNACK_MODEL", "hf.co/devshub-ai/knack:Q8_0")
+PREFIX = "<|im_start|>user\n"
+SUFFIX = "<|im_end|>\n<|im_start|>assistant\n"
+LETTERS = [chr(65 + i) for i in range(26)]
+_READY: set[str] = set()
+
+# question id -> (type, option keys in letter order, what it decides)
+QUESTIONS: dict[str, tuple[str, list[str], str]] = {
+    # every shell command (state: command, description)
+    "risk": ("score", ["0", "1", "2"], "0 safe, 1 needs care, 2 destructive / dangerous"),
+    "long_running": ("noul", ["yes", "no"], "keeps running (server, watch, poll, tail -f)"),
+    "activity": ("choice", ["testing", "building", "installing", "git", "inspecting", "running"],
+                 "what kind of command it is"),
+    "verifies": ("noul", ["yes", "no"], "checks that work is correct (tests, lint, build)"),
+    "mutates_repo": ("noul", ["yes", "no"], "changes files or git state of the repo"),
+    "deploys": ("noul", ["yes", "no"], "deploys or publishes"),
+    "runtime_check": ("noul", ["yes", "no"], "needs a running app to check"),
+    "serves": ("noul", ["yes", "no"], "starts a server"),
+    # every user turn (state: prompt, current_mode, ...)
+    "turn.chat_or_task": ("choice", ["chat", "task"], "conversation or work request"),
+    "turn.actionable": ("noul", ["yes", "no"], "asks for something to be done"),
+    "turn.question_shaped": ("noul", ["yes", "no"], "phrased as a question"),
+    "turn.change_intent": ("choice", ["create", "adjust", "enhance", "fix", "refactor", "none"],
+                           "kind of code change asked for"),
+    "turn.new_task": ("noul", ["yes", "no"], "starts a new task (vs continuing)"),
+    "turn.resume": ("noul", ["yes", "no"], "asks to resume interrupted work"),
+    "turn.mission": ("noul", ["yes", "no"], "long multi-step mission"),
+    "turn.plan_authoring": ("noul", ["yes", "no"], "asks to write a plan"),
+    "turn.screencast": ("choice", ["start", "stop", "none"], "start / stop screen recording"),
+    "turn.demo": ("noul", ["yes", "no"], "asks for a demo"),
+    "turn.skip_orientation": ("noul", ["yes", "no"], "no codebase exploration needed"),
+    "task.intent": ("choice", ["bug_fix", "feature", "refactor", "api_schema", "config_deploy",
+                               "test", "documentation", "investigation", "unknown"], "task type"),
+    "task.execution_strategy": ("choice", ["direct", "plan", "parallel"], "how to execute"),
+    "route.mode": ("choice", ["plan", "build", "ask", "debug"], "best CodeSync mode"),
+    "route.tier": ("choice", ["light", "medium", "heavy"], "model size needed"),
+    "route.thinking": ("choice", ["off", "low", "medium", "high", "xhigh", "max"],
+                       "reasoning effort needed"),
+    "route.risk": ("choice", ["low", "medium", "high"], "risk of the requested work"),
+    "browser": ("choice", ["yes", "no"], "needs a browser"),
+    "messaging": ("choice", ["yes", "no"], "needs to send a message (email, chat)"),
+    "delivery.feature_task": ("choice", ["yes", "no"], "a feature delivery task"),
+    "delivery.slash_task": ("choice", ["yes", "no"], "a slash-command style task"),
+}
+BASH = list(QUESTIONS)[:8]  # the shell-command questions above
+TURN = list(QUESTIONS)[8:]  # the per-turn questions
+
+
+def bash_state(command: str, description: str = "") -> dict[str, Any]:
+    return {"command": command, "description": description}
+
+
+def turn_state(prompt: str, mode: str = "build") -> dict[str, Any]:
+    return {"prompt": prompt, "current_mode": mode, "recent_activity": "none",
+            "conversation_size": "unknown", "open_task": "none", "turns_on_open_task": 0,
+            "interrupted_work": "none", "browser_context": False}
+
+
+def render_state(state: dict[str, Any]) -> str:
+    return "\n".join(f"{k}: {v if isinstance(v, str) else json.dumps(v, ensure_ascii=False)}"
+                     for k, v in state.items())
+
+
+def _post(path: str, body: dict[str, Any], timeout: float = 60.0) -> dict[str, Any]:
+    req = urllib.request.Request(HOST + path, data=json.dumps(body).encode(),
+                                 headers={"Content-Type": "application/json"})
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        return json.loads(r.read())
+
+
+def ask_one(state: dict[str, Any], qid: str, model: str = MODEL) -> dict[str, Any]:
+    """{answer, confidence, probabilities} for one question."""
+    kind, keys, _ = QUESTIONS[qid]
+    out = _post("/api/generate", {
+        "model": model, "prompt": PREFIX + render_state(state) + SUFFIX + qid + ":",
+        "raw": True, "stream": False, "logprobs": True, "top_logprobs": 20,
+        "options": {"temperature": 0, "num_predict": 1}})
+    letters = LETTERS[: len(keys)]
+    logp = {e["token"].strip(): e["logprob"]
+            for e in (out.get("logprobs") or [{}])[0].get("top_logprobs", [])}
+    raw = {k: math.exp(logp[c]) if c in logp else 0.0 for k, c in zip(keys, letters)}
+    total = sum(raw.values()) or 1.0
+    probs = {k: v / total for k, v in raw.items()}
+    best = max(probs, key=probs.__getitem__)
+    return {"answer": best, "confidence": probs[best], "probabilities": probs, "type": kind}
+
+
+def ensure_model(model: str = MODEL) -> None:
+    """Pull the model into Ollama if it is not there yet (first run only)."""
+    if model in _READY:
+        return
+    _READY.add(model)
+    try:
+        _post("/api/show", {"model": model}, timeout=10)
+        return
+    except urllib.error.HTTPError as exc:
+        if exc.code != 404:
+            raise
+    print(f"pulling {model} into Ollama (first run only)...", file=sys.stderr, flush=True)
+    _post("/api/pull", {"model": model, "stream": False}, timeout=3600)
+
+
+def ask(state: dict[str, Any], qids: list[str], model: str = MODEL) -> dict[str, dict[str, Any]]:
+    unknown = [q for q in qids if q not in QUESTIONS]
+    if unknown:
+        raise KeyError(f"Knack was not trained on: {unknown}")
+    ensure_model(model)
+    with ThreadPoolExecutor(max_workers=8) as pool:
+        return dict(zip(qids, pool.map(lambda q: ask_one(state, q, model), qids)))
+
+
+def to_systemone(qid: str, a: dict[str, Any]) -> dict[str, Any]:
+    """CodeSync / TypeSafe System One answer shape."""
+    p, kind = a["probabilities"], a["type"]
+    if kind == "noul":
+        return {"noul": p["yes"], "confidence": a["confidence"]}
+    if kind == "score":
+        return {"score": sum(int(k) * v for k, v in p.items()), "confidence": a["confidence"],
+                "probabilities": p}
+    return {"choice": a["answer"], "confidence": a["confidence"], "probabilities": p}
+
+
+def serve(port: int, model: str) -> None:
+    class H(BaseHTTPRequestHandler):
+        def _send(self, code: int, body: dict[str, Any]) -> None:
+            data = json.dumps(body).encode()
+            self.send_response(code)
+            self.send_header("Content-Type", "application/json")
+            self.send_header("Content-Length", str(len(data)))
+            self.end_headers()
+            self.wfile.write(data)
+
+        def do_GET(self) -> None:  # noqa: N802
+            self._send(200, {"ok": True, "model": model, "questions": len(QUESTIONS)})
+
+        def do_POST(self) -> None:  # noqa: N802
+            body = json.loads(self.rfile.read(int(self.headers.get("Content-Length") or 0)))
+            qids = [q for q in body.get("questions", {}) if q in QUESTIONS]  # others: omitted
+            try:
+                res = ask(body["state"], qids, model)
+            except Exception as exc:  # noqa: BLE001
+                return self._send(502, {"error": f"{type(exc).__name__}: {exc}"})
+            self._send(200, {"answers": {q: to_systemone(q, a) for q, a in res.items()}})
+
+        def log_message(self, *a: Any) -> None:
+            pass
+
+    print(f"Knack ({model} via {HOST}) on http://127.0.0.1:{port}/v1/systemone", flush=True)
+    ThreadingHTTPServer(("127.0.0.1", port), H).serve_forever()
+
+
+def main(argv: list[str] | None = None) -> int:
+    argv = sys.argv[1:] if argv is None else argv
+    if argv[:1] == ["serve"]:
+        sp = argparse.ArgumentParser(prog="knack.py serve")
+        sp.add_argument("--port", type=int, default=33727)
+        sp.add_argument("--model", default=MODEL)
+        a = sp.parse_args(argv[1:])
+        serve(a.port, a.model)
+        return 0
+    ap = argparse.ArgumentParser(description="Ask CodeSync Knack (via Ollama).")
+    ap.add_argument("text", help="a shell command, or with --turn a user message")
+    ap.add_argument("--turn", action="store_true", help="judge a user message, not a command")
+    ap.add_argument("--mode", default="build", help="current CodeSync mode for --turn")
+    ap.add_argument("--description", default="", help="what the command is for")
+    ap.add_argument("-q", "--question", action="append", help="only these question ids")
+    ap.add_argument("--model", default=MODEL)
+    ap.add_argument("--json", action="store_true")
+    a = ap.parse_args(argv)
+    state = turn_state(a.text, a.mode) if a.turn else bash_state(a.text, a.description)
+    qids = a.question or (TURN if a.turn else BASH)
+    try:
+        res = ask(state, qids, a.model)
+    except OSError as exc:
+        print(f"cannot reach Ollama at {HOST} ({exc}). Run: ollama pull {a.model}", file=sys.stderr)
+        return 1
+    if a.json:
+        print(json.dumps(res, indent=1))
+        return 0
+    w = max(map(len, qids))
+    for q, r in res.items():
+        print(f"{q:<{w}}  {r['answer']:<13} {r['confidence']:.2f}   {QUESTIONS[q][2]}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

params

@@ -0,0 +1,5 @@
+{
+  "temperature": 0,
+  "num_predict": 1,
+  "num_ctx": 2048
+}

template

@@ -0,0 +1 @@
+{{ .Prompt }}