Blame for README.md

---
tags:
  - codesync
  - devshub
  - decision-model
  - developer-tools
  - gguf
  - ollama
pipeline_tag: text-classification
inference: false
---

# CodeSync Knack

**A specialized decision model: the free, private judgment model behind CodeSync.**

Every time you send CodeSync a message or it runs a shell command, it makes dozens of quick
calls: Is this command risky? Will it run forever? Are you asking to continue earlier work?
Is this a question or a task? Which mode fits? Knack makes those calls on your own machine,
in about 10 milliseconds. It answers only those decisions, asked by question id.

- **Free.** Ships with every CodeSync install. No API key, no account, no usage limits.
- **Private.** Runs locally. Your prompts and commands never leave your machine to be judged.
- **Fast.** About 10 ms per decision on a GPU, about 250 ms on a plain CPU.
- **Small.** 0.75B parameters in the Ollama edition: 817 MB (Q8_0) or 516 MB (Q4_K_M).
- **Accurate where it matters.** Trained on real CodeSync sessions, with extra focus on the
  rare cases that cost the most when missed: destructive deletes, commands that never exit,
  resuming interrupted work, refactor / test / documentation requests.

## Get it

```bash
ollama pull devshub/knack              # Q8_0, 817 MB  (https://ollama.com/devshub/knack)
ollama pull devshub/knack:q4_k_m       # 516 MB, near-identical accuracy
ollama pull hf.co/devshub-ai/knack:Q8_0   # same model, straight from this repo
```

## A specialized decision model

General decision models, like clef, read any question you write in plain language. Knack is
a different kind: a **specialized decision model**. It was trained on one product's fixed
decisions, the 29 that CodeSync makes on every shell command and every user message, and it
answers only those. It does not read question text: you ask by **question id**. In exchange
it is small, private, and answers all 8 command questions in about 17 ms.

| | General decision model (clef) | Specialized decision model (Knack) |
| --- | --- | --- |
| Questions | anything you write | 29 fixed ones, asked by id |
| Input | any text or image | a shell command, or a message to a coding assistant |
| Size | 27B | 0.75B |
| A question it was not trained on | answers it | rejects it |

## Ask it a question

Every call has the same three parts: **the text to judge**, **the question id**, and
**Knack's answer**. `echo` only hands the text to Knack. Nothing in the text is ever run,
so you can safely ask about dangerous commands.

```
echo "<text to judge>" | ollama run devshub/knack --ask <question id>
```

For example, "will `npm run dev` keep running until I stop it?":

```
$ echo "npm run dev" | ollama run devshub/knack --ask serves
yes  97%
```

### Questions about a shell command

Will it keep running until stopped?

```
$ echo "python -m http.server 8000" | ollama run devshub/knack --ask serves
yes  98%
$ echo "pytest -q" | ollama run devshub/knack --ask serves
no  97%
```

How dangerous is it?

```
$ echo "ls -la" | ollama run devshub/knack --ask risk --level safe --level "needs care" --level destructive
safe  97%  0.0 / 2
$ echo "npm install" | ollama run devshub/knack --ask risk --level safe --level "needs care" --level destructive
needs care  96%  1.0 / 2
$ echo "git push --force origin main" | ollama run devshub/knack --ask risk --level safe --level "needs care" --level destructive
destructive  96%  1.9 / 2
```

Will it take minutes?

```
$ echo "cargo build --release" | ollama run devshub/knack --ask long_running
yes  84%
$ echo "git status" | ollama run devshub/knack --ask long_running
no  97%
```

What kind of command is it?

```
$ echo "pip install requests" | ollama run devshub/knack --ask activity \
    --choice testing --choice building --choice installing --choice git --choice inspecting --choice running
installing  94%
$ echo "grep -rn TODO src" | ollama run devshub/knack --ask activity \
    --choice testing --choice building --choice installing --choice git --choice inspecting --choice running
inspecting  95%
```

Does a pass prove the change works? Does it change git? Does it deploy?

```
$ echo "pytest tests/test_login.py" | ollama run devshub/knack --ask verifies
yes  97%
$ echo "npm run build" | ollama run devshub/knack --ask verifies
no  96%
$ echo "git commit -am 'fix login'" | ollama run devshub/knack --ask mutates_repo
yes  96%
$ echo "git diff" | ollama run devshub/knack --ask mutates_repo
no  97%
$ echo "kubectl apply -f k8s/" | ollama run devshub/knack --ask deploys
yes  95%
```

### Questions about a message to a coding assistant

What kind of task is this?

```
$ echo "fix the failing login test" | ollama run devshub/knack --ask task.intent \
    --choice bug_fix --choice feature --choice refactor --choice api_schema --choice config_deploy \
    --choice test --choice documentation --choice investigation --choice unknown
bug_fix  90%
```

With the same `--choice` list: "add dark mode to the settings page" gives `feature  93%`,
"write docs for the payments API" gives `documentation  97%`, and "why does the build take
so long?" gives `investigation  97%`.

Is it chat or a request for work?

```
$ echo "hi, how are you?" | ollama run devshub/knack --ask turn.chat_or_task --choice chat --choice task
chat  98%
$ echo "rename the user module" | ollama run devshub/knack --ask turn.chat_or_task --choice chat --choice task
task  97%
```

Which assistant mode fits?

```
$ echo "what does this repo do?" | ollama run devshub/knack --ask route.mode --choice plan --choice build --choice ask --choice debug
ask  96%
$ echo "plan the migration to Postgres" | ollama run devshub/knack --ask route.mode --choice plan --choice build --choice ask --choice debug
plan  90%
```

Start or stop a screen recording?

```
$ echo "record my screen" | ollama run devshub/knack --ask turn.screencast --choice start --choice stop --choice none
start  89%
```

The answer is the option Knack picks and its confidence. Each id has one precise meaning,
listed in the tables below; read them before relying on an answer. `long_running`, for
example, means "takes minutes", so a build counts. Questions with options (`--choice`,
`--level`) need them in the order shown; if they do not match, the error shows the exact
flags to use. Any question that is not one of the 29 ids is rejected rather than guessed.

## API

### cURL

```
curl http://localhost:11434/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
  "model": "devshub/knack",
  "state": "npm run dev",
  "questions": {
    "serves": {"type": "noul", "instructions": "serves"},
    "long_running": {"type": "noul", "instructions": "long_running"}
  }
}'
```

```
{"model":"devshub/knack","answers":{"serves":{"type":"noul","noul":0.9701},"long_running":{"type":"noul","noul":0.9708}}}
```

The question id goes in `instructions` (or as the key). The state can also be an object
with Knack's own fields, for example `{"command": "npm run dev", "description": "start the app"}`.

### Python

```
pip install ollama
```

```python
import ollama

response = ollama.systemone(
    model="devshub/knack",
    state="npm run dev",
    questions={"serves": {"type": "noul", "instructions": "serves"}},
)
print(response.answers["serves"].noul)  # 0.9701
```

## Or use the standalone script

`knack.py` asks every question of a command or message at once and prints a table. It uses
only the Python standard library and pulls the model on first run.

```
curl -fsSL https://huggingface.co/devshub-ai/knack/resolve/main/knack.py | python3 - "npm run dev"
```

```
risk           1             0.95   0 read-only, 1 recoverable change, 2 irreversible / remote
long_running   yes           0.97   takes minutes, blocks, polls, watches or serves
activity       running       0.92   what kind of work it does
verifies       no            0.97   a test, type or lint check whose pass proves the change works
mutates_repo   no            0.97   changes git state (commit, push, merge, rebase, tag)
deploys        no            0.94   publishes, releases or changes a remote environment
runtime_check  no            0.95   a health probe of a running service that fails if unhealthy
serves         yes           0.97   keeps running until stopped (server, watcher, tail -f)
```

```
curl -fsSL https://huggingface.co/devshub-ai/knack/resolve/main/knack.py | python3 - --turn "fix the failing login test"
```

## What it decides

**Every shell command.** Input: the command (state fields `command`, `description`).

| id | yes / the answer means | options |
| --- | --- | --- |
| `risk` | the worst it can do: 0 read-only or trivially reversible, 1 changes recoverable local state, 2 irreversible loss or reaches beyond the checkout | 0, 1, 2 |
| `long_running` | takes minutes, blocks, polls, watches or serves, rather than finishing in seconds. A build or test suite counts | yes, no |
| `serves` | keeps running until it is stopped: server, daemon, watcher, tunnel, `tail -f`. A build that finishes on its own does not | yes, no |
| `activity` | what kind of work it does | testing, building, installing, git, inspecting, running |
| `verifies` | a test run, type or lint check whose passing exit is evidence the change works. Builds and plain runs do not count | yes, no |
| `mutates_repo` | changes version-control state: commit, push, merge, rebase, tag, amend. Editing or deleting files does not count | yes, no |
| `deploys` | publishes, releases, or changes a remote or runtime environment (deploy, rollout, push an image, bring a service up) | yes, no |
| `runtime_check` | a health or smoke probe of a running service that fails on an unhealthy response (`curl --fail`) | yes, no |

**Every user turn.** Input: the message a user sends to a coding assistant (state fields, in
this order: `prompt`, `current_mode`, `recent_activity`, `conversation_size`, `open_task`,
`turns_on_open_task`, `interrupted_work`, `browser_context`; plain text fills the rest with
defaults).

| id | the answer means | options |
| --- | --- | --- |
| `turn.chat_or_task` | conversation, or a request for work | chat, task |
| `turn.actionable` | asks for something to be done | yes, no |
| `turn.question_shaped` | phrased as a question | yes, no |
| `turn.change_intent` | kind of code change asked for | create, adjust, enhance, fix, refactor, none |
| `turn.new_task` | starts new work rather than continuing | yes, no |
| `turn.resume` | asks to resume interrupted work | yes, no |
| `turn.mission` | a long, multi-step job | yes, no |
| `turn.plan_authoring` | asks for a plan to be written | yes, no |
| `turn.screencast` | start or stop a screen recording | start, stop, none |
| `turn.demo` | asks for a demo | yes, no |
| `turn.skip_orientation` | can be done without exploring the codebase | yes, no |
| `task.intent` | type of task | bug_fix, feature, refactor, api_schema, config_deploy, test, documentation, investigation, unknown |
| `task.execution_strategy` | how to execute it | direct, plan, parallel |
| `route.mode` | best assistant mode | plan, build, ask, debug |
| `route.tier` | size of model needed | light, medium, heavy |
| `route.thinking` | reasoning effort needed | off, low, medium, high, xhigh, max |
| `route.risk` | risk of the requested work | low, medium, high |
| `browser` | needs a web browser | yes, no |
| `messaging` | needs to send an email or chat message | yes, no |
| `delivery.feature_task` | a feature delivery task | yes, no |
| `delivery.slash_task` | a slash-command style task | yes, no |

## What it does not do

- **Other questions.** Anything outside these 29, such as "is this log an error?" or "is a
  refund requested?", is rejected. Use a general decision model like clef for those.
- **Other input.** Logs, emails, tickets, documents or images give meaningless answers,
  even to these 29 questions.
- **Conversation history.** Through Ollama it sees a single message, so a question like
  `turn.resume` ("continue where you left off") answers no: there is nothing to resume. Inside CodeSync it also
  gets the session context, which makes the turn questions more reliable.
- **Text generation.** It returns one pick with a confidence, never sentences.

## Known weak spots

Measured with the commands above, so you know where to double-check:

| command | question | Knack says | should be |
| --- | --- | --- | --- |
| `tail -f app.log` | `serves` | no, 75% | yes |
| `npm run build` | `serves` | yes, 79% | no |
| `docker compose up` | `long_running` | 51% yes | yes |
| `curl --fail http://localhost:3000/health` | `runtime_check` | no, 92% | yes |
| `npm test` | `long_running` | 45% yes | unclear, depends on the suite |

These come from the model's training, not the Ollama packaging: on every command compared,
the Ollama build matches the original model to within 0.002. They are the next training targets.

## Results

Measured on CodeSync's own judgments (version v3, November 2026):

| Benchmark | Knack v3 | Ollama Q8_0 | Ollama Q4_K_M | What it measures |
|---|---:|---:|---:|---|
| Hand-labelled set | **97.9%** | 97.9% | 97.4% | 233 decisions labelled by hand |
| Held-out set | **95.5%** | 95.7% | 95.7% | 3,267 decisions from real sessions, never trained on |
| Rare-case set | **93.7%** | 93.6% | 93.6% | 4,120 decisions on the hardest, rarest cases |
| Latency | **13 ms** | 17 ms | ~17 ms | median per command (8 questions) on a laptop GPU |

Latency is the median for all 8 command questions in one `/v1/systemone` request.

## Use it inside CodeSync

Install CodeSync. That's it.

```bash
curl -fsSL https://app.devshub.ai/install.sh | bash
```

On first start, CodeSync downloads Knack in the background and keeps working while it
does. Once Knack is ready it becomes CodeSync's first judgment engine, and new versions are
picked up automatically. To turn it off: `CODESYNC_KNACK_DISABLED=1`.

## Files

| File | Purpose |
|---|---|
| `knack-v3-Q8_0.gguf`, `knack-v3-Q4_K_M.gguf` | Ollama / llama.cpp decision model (`--ask`, `/v1/systemone`) |
| `knack.py` | standalone runner (CLI and local server), standard library only |
| `manifest.json`, `knack-<version>.tar.gz` | the PyTorch model folder CodeSync installs |

The GGUF edition is the same weights packaged as an Ollama decision model: a `systemone`
prompt template rebuilds Knack's training prompt from the request, and the answer is read
from the option letters exactly as CodeSync's own runtime reads it.

Built by [DevsHub](https://app.devshub.ai) for [CodeSync](https://app.devshub.ai).