Instructions to use neerumsr/keel-pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use neerumsr/keel-pro with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Configuration Parsing Warning:In adapter_config.json: "peft.task_type" must be a string
Keel-Pro
Keel-Pro is an open-weight decision model: given a state and one or more typed questions (choice, score,
noul = yes/no), it returns a probability over the options. It is a LoRA fine-tune of Qwen/Qwen3.5-27B (Apache-2.0)
and serves Keel's optional decider-pro-1.0.0 pin through a loopback sidecar
(serve/keel_pro_server.py, see SERVE.md). Keel-Lite (the CPU ONNX model) stays Keel's default.
At a glance: Decision Index public 0.3 = 62.73 on the full public suite (all 139,677 requests answered; Jev 1.14 = 60.70 on the DI board; Microsoft-Decision-1 โ 62, read from Microsoft's chart). Median latency 245 ms per request on one RTX 5090. Apache-2.0, open weights, runs on your own GPU.
Quick start
git clone https://github.com/sivamatic/Keel && cd Keel
pip install -r serve/requirements.txt
hf download Qwen/Qwen3.5-27B --local-dir models/Qwen3.5-27B
hf download neerumsr/keel-pro --local-dir models/keel-pro
python serve/keel_pro_server.py --model-dir models/Qwen3.5-27B --adapter models/keel-pro --load-4bit --port 8090
Then run Keel with KEEL_PRO_URL=http://127.0.0.1:8090 and call it with model decider-pro-1.0.0:
curl -s http://127.0.0.1:8080/api/v1/systemone -H "Authorization: Bearer $KEEL_TOKEN" -H "Content-Type: application/json" -d '{
"model": "decider-pro-1.0.0",
"state": {"ticket": "I was charged twice this month"},
"questions": {"route": {"type": "choice", "instructions": "Which team handles this?",
"criteria": {"billing": "payments and invoices", "tech": "bugs and outages", "account": "login and profile"}}}
}'
The answer has answers.route.choice, .confidence and .probabilities. Docker, Compose and Kubernetes recipes:
deployment guide. Smaller GPU? Use the GGUF build with
llama.cpp (neerumsr/keel-pro-GGUF, Q4_K_M is a 16.8 GB file).
Results on the Decision Index (public 0.3 index)
Measured on this machine with the DI kit's own prompts and scorers (di-results.md has every run). The full public row covers every public request; the other rows are
sample estimates with a group-bootstrap 95% CI: dev (1,633 rows, used for iteration) and report
(3,272 disjoint rows, scored once for the finalist). HLE is gated and scored 0 (costs about 1.2 points).
| Model | Sample | Public index | 95% CI | Median latency |
|---|---|---|---|---|
| Keel-Pro (soup78), fast engine | full public suite (139,677 requests) | 62.73 | โ (full suite) | 245 ms (p80 438, mean 528) |
| Keel-Pro (finalist, soup78) | report | 61.96 | 58.98โ64.42 | 329 ms |
| Keel-Pro (soup78), fast engine | report | 61.66 | 58.83โ64.28 | 316 ms (mean 844, p80 847) |
| Keel-Pro (finalist, soup78) | dev | 62.06 | 56.63โ64.13 | 321 ms |
| Keel-Pro it7 (single run) | dev | 60.32 | 55.09โ62.68 | 319 ms |
| Jev 1.14 | full suite | 60.70 | โ | โ (DI-measured, vendor board) |
| Microsoft-Decision-1 | full suite | โ62 | โ | estimate read from Microsoft's chart, not a measurement |
Keel-Lite (decider-3.0.0) over HTTP |
dev | 5.27 | 4.68โ8.11 | 19 ms |
| Stock Qwen3.5-9B (kit prompt) | dev | 10.13 | 7.80โ13.75 | 125 ms |
Latency is our RTX 5090 (32 GB) per-request wall time, one process, not DI's RTX PRO 6000 measurement.
Full public run. The whole public 0.3 suite (139,677 requests, every one answered) scores 62.73, with
areas knowledge 0.476, language 0.642, retrieval 0.639, tools 0.841, arts 0.565. HLE (gated) still counts as 0,
so this is a lower bound, and scores.json is not yet "complete": true for the board. Median, p80 and mean
latency (245 / 438 / 528 ms) are all under the board's 1 s rule on our RTX 5090.
Report-sample area scores (soup78): knowledge 0.480, language 0.625, retrieval 0.612, tools 0.817, arts 0.621.
Read honestly: 61.96 is above Jev 1.14's 60.70 point estimate and level with the โ62 Microsoft-Decision-1 estimate,
but both lie inside our 95% CI, so this is on par, not a proven win. Three candidates (it7, it8, soup78) were
compared on dev; the report sample was scored once, for soup78 only.
Fast engine (hybrid_engine.py, prefix_share=1, the default): for a request with several questions, the token
prefix every prompt shares (system + state) is run once and the questions' remaining tokens continue copies of that
cache in right-padded batches. Same prompts and read-out; on the report sample it agrees with the one-pass-per-question
run on 17,150 of 17,182 answers (bf16 batching differences) and scores 61.66. It was built for the latency rule, not
to change the score, and we list both rows rather than pick one.
Training
- Base: Qwen/Qwen3.5-27B. LoRA r32, alpha 64, no dropout, on the attention, linear-attention (
in_proj_qkv,in_proj_z,out_proj) and MLP projections. QLoRA: the base is loaded in 4-bit NF4 with double quantization and bf16 compute; only the LoRA weights train (217.6M parameters). Trained with a 4,096 (it7) / 5,120 (it8) token cap on one RTX 5090 (32 GB). - The released adapter (soup78) is the exact average of two runs: it7 (48k rows, lr 1e-4) and it8 (it7 continued on
~20k fresh rows, including long POP909 chord rows, lr 5e-5). PEFT
catat 0.5/0.5 gives a rank-64 adapter whose weight change is the mean of the two (training/pro/soup_adapters.py). - Data: train splits of public datasets mapped to 24 DI benchmarks plus own synthetic generators; never DI eval rows. Every source, revision, split and licence is in DATA.md.
- Read-out (
letters): options are shown asA. key: description; the answer is the softmax over the label logits at one position (one forward pass). Half the training rows permute option order. - Iterations, levers and failures: train-notes.md.
Files
| File | What |
|---|---|
adapter_model.safetensors, adapter_config.json |
The LoRA, rank 64 (load on top of the base with PEFT) |
tokenizer.json, tokenizer_config.json, chat_template.jinja |
Unchanged from the base |
keel-pro.json |
Model id, read-out policy (letters) and how the adapter was built |
NOTICE, LICENSE, SHA256SUMS |
Attribution, Apache-2.0 text, file checksums |
Merged GGUF builds for llama.cpp (keel-pro-Q8_0.gguf, keel-pro-Q4_K_M.gguf) are in
neerumsr/keel-pro-GGUF.
GGUF fidelity: both keel-pro-Q8_0.gguf (29.0 GB) and keel-pro-Q4_K_M.gguf (16.8 GB) pick the same option as the
measured transformers run (4-bit NF4 base + adapter) on 30 of 30 dev questions (scripts/di/gguf_check.py, llama.cpp
b12029). Median per question with the CUDA llama-server (-ngl 99, RTX 5090): 243 ms (Q8_0) and 242 ms
(Q4_K_M), again 30 of 30 agreement. CPU-only llama-server (12 threads): 15.4 s for either file.
Limitations
- The full public run (62.73) has no CI because it covers the whole public suite; the sample rows have wide CIs (ยฑ3 points). HLE is gated and counted as 0, and the DI board also scores private tests we cannot run.
- Latency (full public suite, RTX 5090, fast engine): median 245 ms, p80 438 ms, mean 528 ms per request, inside the DI board's โค 1,000 ms rule; p95 is 1.27 s (long retrieval rows with 32 candidates). Not yet measured on the board's RTX PRO 6000.
- Weakest on reasoning-heavy benchmarks (GPQA, ChessBench, CRUXEval) and on taste/arts tasks.
- Needs a GPU for interactive latency (the Q4_K_M GGUF is a 16.8 GB file); CPU GGUF works but is seconds per question.
- English only. Probabilities are the model's own (no calibration layer).
- Not a safety classifier; use Keel-Lite's hybrid mode or a human gate for safety-critical decisions.
License
Apache-2.0 for the adapter and code; the Qwen3.5 base is Apache-2.0 (notice kept in NOTICE). Some training rows
come from CC BY-SA sources (MNLI portions, SNLI, ARC, HoVer/Wikipedia). They are attributed in DATA.md and
docs/licenses.md and never redistributed. No non-commercial data (ANLI, RACE) was used for training.
- Downloads last month
- -
Model tree for neerumsr/keel-pro
Evaluation results
- Public index (HLE gated, counted as 0) on Decision Index 0.3 (public suite, 139,677 requests)Self-reported full public run62.730