CodeSync Knack

A specialized decision model: the free, private judgment model behind CodeSync.

Every time you send CodeSync a message or it runs a shell command, it makes dozens of quick calls: Is this command risky? Will it run forever? Are you asking to continue earlier work? Is this a question or a task? Which mode fits? Knack makes those calls on your own machine, in about 10 milliseconds. It answers only those decisions, asked by question id.

• Free. Ships with every CodeSync install. No API key, no account, no usage limits.
• Private. Runs locally. Your prompts and commands never leave your machine to be judged.
• Fast. About 10 ms per decision on a GPU, about 250 ms on a plain CPU.
• Small. 0.75B parameters in the Ollama edition: 817 MB (Q8_0) or 516 MB (Q4_K_M).
• Accurate where it matters. Trained on real CodeSync sessions, with extra focus on the rare cases that cost the most when missed: destructive deletes, commands that never exit, resuming interrupted work, refactor / test / documentation requests.

Get it

ollama pull devshub/knack              # Q8_0, 817 MB  (https://ollama.com/devshub/knack)
ollama pull devshub/knack:q4_k_m       # 516 MB, near-identical accuracy
ollama pull hf.co/devshub-ai/knack:Q8_0   # same model, straight from this repo

A specialized decision model

General decision models, like clef, read any question you write in plain language. Knack is a different kind: a specialized decision model. It was trained on one product's fixed decisions, the 29 that CodeSync makes on every shell command and every user message, and it answers only those. It does not read question text: you ask by question id. In exchange it is small, private, and answers all 8 command questions in about 17 ms.

	General decision model (clef)	Specialized decision model (Knack)
Questions	anything you write	29 fixed ones, asked by id
Input	any text or image	a shell command, or a message to a coding assistant
Size	27B	0.75B
A question it was not trained on	answers it	rejects it

Ask it a question

Every call has the same three parts: the text to judge, the question id, and Knack's answer. echo only hands the text to Knack. Nothing in the text is ever run, so you can safely ask about dangerous commands.

echo "<text to judge>" | ollama run devshub/knack --ask <question id>

For example, "will npm run dev keep running until I stop it?":

$ echo "npm run dev" | ollama run devshub/knack --ask serves
yes  97%

Questions about a shell command

Will it keep running until stopped?

$ echo "python -m http.server 8000" | ollama run devshub/knack --ask serves
yes  98%
$ echo "pytest -q" | ollama run devshub/knack --ask serves
no  97%

How dangerous is it?

$ echo "ls -la" | ollama run devshub/knack --ask risk --level safe --level "needs care" --level destructive
safe  97%  0.0 / 2
$ echo "npm install" | ollama run devshub/knack --ask risk --level safe --level "needs care" --level destructive
needs care  96%  1.0 / 2
$ echo "git push --force origin main" | ollama run devshub/knack --ask risk --level safe --level "needs care" --level destructive
destructive  96%  1.9 / 2

Will it take minutes?

$ echo "cargo build --release" | ollama run devshub/knack --ask long_running
yes  84%
$ echo "git status" | ollama run devshub/knack --ask long_running
no  97%

What kind of command is it?

$ echo "pip install requests" | ollama run devshub/knack --ask activity \
    --choice testing --choice building --choice installing --choice git --choice inspecting --choice running
installing  94%
$ echo "grep -rn TODO src" | ollama run devshub/knack --ask activity \
    --choice testing --choice building --choice installing --choice git --choice inspecting --choice running
inspecting  95%

Does a pass prove the change works? Does it change git? Does it deploy?

$ echo "pytest tests/test_login.py" | ollama run devshub/knack --ask verifies
yes  97%
$ echo "npm run build" | ollama run devshub/knack --ask verifies
no  96%
$ echo "git commit -am 'fix login'" | ollama run devshub/knack --ask mutates_repo
yes  96%
$ echo "git diff" | ollama run devshub/knack --ask mutates_repo
no  97%
$ echo "kubectl apply -f k8s/" | ollama run devshub/knack --ask deploys
yes  95%

Questions about a message to a coding assistant

What kind of task is this?

$ echo "fix the failing login test" | ollama run devshub/knack --ask task.intent \
    --choice bug_fix --choice feature --choice refactor --choice api_schema --choice config_deploy \
    --choice test --choice documentation --choice investigation --choice unknown
bug_fix  90%

With the same --choice list: "add dark mode to the settings page" gives feature 93%, "write docs for the payments API" gives documentation 97%, and "why does the build take so long?" gives investigation 97%.

Is it chat or a request for work?

$ echo "hi, how are you?" | ollama run devshub/knack --ask turn.chat_or_task --choice chat --choice task
chat  98%
$ echo "rename the user module" | ollama run devshub/knack --ask turn.chat_or_task --choice chat --choice task
task  97%

Which assistant mode fits?

$ echo "what does this repo do?" | ollama run devshub/knack --ask route.mode --choice plan --choice build --choice ask --choice debug
ask  96%
$ echo "plan the migration to Postgres" | ollama run devshub/knack --ask route.mode --choice plan --choice build --choice ask --choice debug
plan  90%

Start or stop a screen recording?

$ echo "record my screen" | ollama run devshub/knack --ask turn.screencast --choice start --choice stop --choice none
start  89%

The answer is the option Knack picks and its confidence. Each id has one precise meaning, listed in the tables below; read them before relying on an answer. long_running, for example, means "takes minutes", so a build counts. Questions with options (--choice, --level) need them in the order shown; if they do not match, the error shows the exact flags to use. Any question that is not one of the 29 ids is rejected rather than guessed.

API

cURL

curl http://localhost:11434/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
  "model": "devshub/knack",
  "state": "npm run dev",
  "questions": {
    "serves": {"type": "noul", "instructions": "serves"},
    "long_running": {"type": "noul", "instructions": "long_running"}
  }
}'

{"model":"devshub/knack","answers":{"serves":{"type":"noul","noul":0.9701},"long_running":{"type":"noul","noul":0.9708}}}

The question id goes in instructions (or as the key). The state can also be an object with Knack's own fields, for example {"command": "npm run dev", "description": "start the app"}.

Python

pip install ollama

import ollama

response = ollama.systemone(
    model="devshub/knack",
    state="npm run dev",
    questions={"serves": {"type": "noul", "instructions": "serves"}},
)
print(response.answers["serves"].noul)  # 0.9701

Or use the standalone script

knack.py asks every question of a command or message at once and prints a table. It uses only the Python standard library and pulls the model on first run.

curl -fsSL https://huggingface.co/devshub-ai/knack/resolve/main/knack.py | python3 - "npm run dev"

risk           1             0.95   0 read-only, 1 recoverable change, 2 irreversible / remote
long_running   yes           0.97   takes minutes, blocks, polls, watches or serves
activity       running       0.92   what kind of work it does
verifies       no            0.97   a test, type or lint check whose pass proves the change works
mutates_repo   no            0.97   changes git state (commit, push, merge, rebase, tag)
deploys        no            0.94   publishes, releases or changes a remote environment
runtime_check  no            0.95   a health probe of a running service that fails if unhealthy
serves         yes           0.97   keeps running until stopped (server, watcher, tail -f)

curl -fsSL https://huggingface.co/devshub-ai/knack/resolve/main/knack.py | python3 - --turn "fix the failing login test"

What it decides

Every shell command. Input: the command (state fields command, description).

id	yes / the answer means	options
risk	the worst it can do: 0 read-only or trivially reversible, 1 changes recoverable local state, 2 irreversible loss or reaches beyond the checkout	0, 1, 2
long_running	takes minutes, blocks, polls, watches or serves, rather than finishing in seconds. A build or test suite counts	yes, no
serves	keeps running until it is stopped: server, daemon, watcher, tunnel, tail -f. A build that finishes on its own does not	yes, no
activity	what kind of work it does	testing, building, installing, git, inspecting, running
verifies	a test run, type or lint check whose passing exit is evidence the change works. Builds and plain runs do not count	yes, no
mutates_repo	changes version-control state: commit, push, merge, rebase, tag, amend. Editing or deleting files does not count	yes, no
deploys	publishes, releases, or changes a remote or runtime environment (deploy, rollout, push an image, bring a service up)	yes, no
runtime_check	a health or smoke probe of a running service that fails on an unhealthy response (curl --fail)	yes, no

Every user turn. Input: the message a user sends to a coding assistant (state fields, in this order: prompt, current_mode, recent_activity, conversation_size, open_task, turns_on_open_task, interrupted_work, browser_context; plain text fills the rest with defaults).

id	the answer means	options
turn.chat_or_task	conversation, or a request for work	chat, task
turn.actionable	asks for something to be done	yes, no
turn.question_shaped	phrased as a question	yes, no
turn.change_intent	kind of code change asked for	create, adjust, enhance, fix, refactor, none
turn.new_task	starts new work rather than continuing	yes, no
turn.resume	asks to resume interrupted work	yes, no
turn.mission	a long, multi-step job	yes, no
turn.plan_authoring	asks for a plan to be written	yes, no
turn.screencast	start or stop a screen recording	start, stop, none
turn.demo	asks for a demo	yes, no
turn.skip_orientation	can be done without exploring the codebase	yes, no
task.intent	type of task	bug_fix, feature, refactor, api_schema, config_deploy, test, documentation, investigation, unknown
task.execution_strategy	how to execute it	direct, plan, parallel
route.mode	best assistant mode	plan, build, ask, debug
route.tier	size of model needed	light, medium, heavy
route.thinking	reasoning effort needed	off, low, medium, high, xhigh, max
route.risk	risk of the requested work	low, medium, high
browser	needs a web browser	yes, no
messaging	needs to send an email or chat message	yes, no
delivery.feature_task	a feature delivery task	yes, no
delivery.slash_task	a slash-command style task	yes, no

What it does not do

• Other questions. Anything outside these 29, such as "is this log an error?" or "is a refund requested?", is rejected. Use a general decision model like clef for those.
• Other input. Logs, emails, tickets, documents or images give meaningless answers, even to these 29 questions.
• Conversation history. Through Ollama it sees a single message, so a question like turn.resume ("continue where you left off") answers no: there is nothing to resume. Inside CodeSync it also gets the session context, which makes the turn questions more reliable.
• Text generation. It returns one pick with a confidence, never sentences.

Known weak spots

Measured with the commands above, so you know where to double-check:

command	question	Knack says	should be
tail -f app.log	serves	no, 75%	yes
npm run build	serves	yes, 79%	no
docker compose up	long_running	51% yes	yes
curl --fail http://localhost:3000/health	runtime_check	no, 92%	yes
npm test	long_running	45% yes	unclear, depends on the suite

These come from the model's training, not the Ollama packaging: on every command compared, the Ollama build matches the original model to within 0.002. They are the next training targets.

Results

Measured on CodeSync's own judgments (version v3, November 2026):

Benchmark	Knack v3	Ollama Q8_0	Ollama Q4_K_M	What it measures
Hand-labelled set	97.9%	97.9%	97.4%	233 decisions labelled by hand
Held-out set	95.5%	95.7%	95.7%	3,267 decisions from real sessions, never trained on
Rare-case set	93.7%	93.6%	93.6%	4,120 decisions on the hardest, rarest cases
Latency	13 ms	17 ms	~17 ms	median per command (8 questions) on a laptop GPU

Latency is the median for all 8 command questions in one /v1/systemone request.

Use it inside CodeSync

Install CodeSync. That's it.

curl -fsSL https://app.devshub.ai/install.sh | bash

On first start, CodeSync downloads Knack in the background and keeps working while it does. Once Knack is ready it becomes CodeSync's first judgment engine, and new versions are picked up automatically. To turn it off: CODESYNC_KNACK_DISABLED=1.

Files

File	Purpose
knack-v3-Q8_0.gguf, knack-v3-Q4_K_M.gguf	Ollama / llama.cpp decision model (--ask, /v1/systemone)
knack.py	standalone runner (CLI and local server), standard library only
manifest.json, knack-<version>.tar.gz	the PyTorch model folder CodeSync installs

The GGUF edition is the same weights packaged as an Ollama decision model: a systemone prompt template rebuilds Knack's training prompt from the request, and the answer is read from the option letters exactly as CodeSync's own runtime reads it.

Built by DevsHub for CodeSync.