Instructions to use ManniX-ITA/JackOD-9B-Coder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ManniX-ITA/JackOD-9B-Coder with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ManniX-ITA/JackOD-9B-Coder")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ManniX-ITA/JackOD-9B-Coder")
model = AutoModelForMultimodalLM.from_pretrained("ManniX-ITA/JackOD-9B-Coder", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ManniX-ITA/JackOD-9B-Coder with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ManniX-ITA/JackOD-9B-Coder"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ManniX-ITA/JackOD-9B-Coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ManniX-ITA/JackOD-9B-Coder

SGLang

How to use ManniX-ITA/JackOD-9B-Coder with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ManniX-ITA/JackOD-9B-Coder" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ManniX-ITA/JackOD-9B-Coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ManniX-ITA/JackOD-9B-Coder" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ManniX-ITA/JackOD-9B-Coder",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ManniX-ITA/JackOD-9B-Coder with Docker Model Runner:
```
docker model run hf.co/ManniX-ITA/JackOD-9B-Coder
```

JackOD-9B-Coder / README.md

ManniX-ITA

v3 card: evaluation with the serving sampler (temp 0.4), v3 KLD/PPL, ollama tags; v2/v1 kept as history

b0cabb1 verified 12 days ago

preview code

Raw History Blame Contribute Delete

25.4 kB

---
license: apache-2.0
base_model:
- Qwen/Qwen3.5-9B
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- agentic
- code
- merge
- qwen3.5
- tool-calling
model_name: JackOD-9B-Coder
author: ManniX-ITA
architecture: Qwen3.5
parameters: 9.65B
format: safetensors
features:
- code
- reasoning
---
# JackOD-9B-Coder

A four-way **`omnimerge_v2`** merge over `Qwen/Qwen3.5-9B`, built for one job:
**multi-turn agentic coding** — staying on a task across turns and tool calls
until it is actually finished.
It is not built to win knowledge benchmarks, and its world knowledge is limited
for its size. What it is built to do is **finish, and stop**.
> **v3 (2026-10-24).** v3 is the v2 recipe with the two smaller weights swapped:
> **DeltaCoder 0.55 / Ornith-1.5-9B 0.30 / Coder-Flash 0.15** (v2: Coder-Flash 0.30 / Ornith 0.15).
> `omnimerge_v2` density 0.53, darex-q 0.75, seed 42, the root `eos_token_id` 248046 / `pad_token_id` 248055
> fix and the Coder-Flash tokenizer and template are unchanged. v3 ships a **new serving sampler** and was
> evaluated with it; the v2 and v1 results kept at the end of this card were measured **greedy**, so they are
> a record, not a same-basis comparison.

| role | model | weight |
|---|---|---|
| base / task-base | [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B) | — (shared ancestor) |
| source | [`danielcherubini/Qwen3.5-DeltaCoder-9B`](https://huggingface.co/danielcherubini/Qwen3.5-DeltaCoder-9B) (LoRA on Qwen3.5-9B, merged to dense) | **0.55** |
| source | [`ornith-ai/Ornith-1.5-9B`](https://huggingface.co/ornith-ai/Ornith-1.5-9B) | 0.30 (v2: 0.15) |
| source | [`Jackrong/Qwopus3-5-9B-Coder-Flash`](https://huggingface.co/Jackrong/Qwopus3-5-9B-Coder-Flash) (v1: [`Jackrong/Qwopus3.5-9B-coder`](https://huggingface.co/Jackrong/Qwopus3.5-9B-coder)) | 0.15 (v2: 0.30) |

`--base` and `--task-base` are both `Qwen3.5-9B`: every source descends from it,
so task vectors are taken against the true common ancestor instead of against a
sibling.

**Kept intact from the base:** the **MTP head** (15 tensors) is excluded from the
merge (`--skip-patterns visual.,mtp.`) and passed through, so the draft head
survives — an averaged MTP head stops being a usable drafter.

**There is no vision tower.** `config.json` inherits a populated `vision_config`
from the `Qwen3_5ForConditionalGeneration` architecture and the repo ships
`preprocessor_config.json` / `video_preprocessor_config.json`, but that is
metadata with no weights behind it: base and merge both carry **775 tensors and
zero `visual.*`**. No `mmproj` exists for this model. This model is text-only.
## Where the name comes from
**JackOD** is an initialism of its three merge sources:
| letter | source |
|---|---|
| **Jack** | [`Jackrong/Qwopus3-5-9B-Coder-Flash`](https://huggingface.co/Jackrong/Qwopus3-5-9B-Coder-Flash) — JackRong's Qwopus 9B Coder-Flash (v1: Qwopus3.5-9B-coder) |
| **O** | [`ornith-ai/Ornith-1.5-9B`](https://huggingface.co/ornith-ai/Ornith-1.5-9B) |
| **D** | [`danielcherubini/Qwen3.5-DeltaCoder-9B`](https://huggingface.co/danielcherubini/Qwen3.5-DeltaCoder-9B) |

`Qwen3.5-9B` is not in the name: it is the shared ancestor all three descend
from, the `--base`/`--task-base` the merge is taken *against*, rather than a
flavour contributed to the blend.

## Recipe

```bash
python omnimergekit.py \
  --base      Qwen3.5-9B --task-base Qwen3.5-9B \
  --source    DeltaCoder-9B-applied \
  --source    Qwopus3.5-9B-Coder-Flash \
  --source    Ornith-1.5-9B \
  --weights   0.55,0.15,0.30 \
  --method omnimerge_v2 --density 0.53 --darex-q 0.75 --seed 42 \
  --no-auto-mlp-skip --skip-patterns visual.,mtp. \
  --output JackOD-9B-Coder
```

Pinned inputs: Qwen3.5-9B `c2022362`, Coder-Flash `a972c214`, Ornith-1.5-9B
`489cb979`, DeltaCoder `10a2546f` (root adapter r32/a32, applied to dense).

One step is **not optional**: `Qwen3.5-9B`, `DeltaCoder-9B-applied` and
Coder-Flash all carry **no root `eos_token_id`** (it lives only in the nested
text config). The GGUF converter reads the root, finds nothing, and falls back —
putting the wrong terminator in the quant and leaking control tokens into
generations. The merge sets root `eos_token_id` 248046 / `pad_token_id` 248055
and takes the tokenizer and chat template from Coder-Flash, whose template this
model serves (v2 template differs from v1's Qwopus template; the tokenizer is
identical).

The v1 build chain is in
**[`recipes/jackod-9b-coder/`](https://github.com/mann1x/omnimergekit/tree/main/recipes/jackod-9b-coder)**
of [omnimergekit](https://github.com/mann1x/omnimergekit); v2 and v3 differ only in
the source and weight changes above.

## Evaluation (v3)

Measured on the **published Q6_K** (`JackOD-9B-Coder-Q6_K.gguf`, sha256 `3f202a76…51b8`, full AtomicChat
`ornith-1.5-9b` imatrix `feb286c4…`), llama.cpp `311d4211bf`, with **the serving sampler** below:
temperature 0.4 / top_p 0.95 / top_k 20 / min_p 0 / repeat_penalty 1.002 over the last 112 tokens /
presence_penalty 1.05, **seed not fixed** (a random seed per request). The sampler was read back from every
server's `/slots` while the run was in flight: 338 checks, 0 mismatches. Thinking budget 12,288 tokens
(`--reasoning-budget`). Scores are `summary.json.score`.
| Benchmark | **JackOD v3** |
|---|---|
| HumanEval (164) | 0.8720 |
| HumanEval+ (164) | 0.8049 |
| **LiveCodeBench v6 (55 hard)** | **0.9455** |
| MultiPL-E (300) | 0.8033 |
| IFEval (100) | 0.9400 |
- **These are sampled scores, one draw each.** They are not on the same basis as the greedy v2 and v1
  tables further down, and the difference between them is not claimed as a gain or a loss. On these sizes one
  problem is 1.8 pp of LiveCodeBench and 0.6 pp of HumanEval.
- **LiveCodeBench is 52 of 55.** All 55 generations ended on their own (none hit the length cap), and the 3
  misses are wrong answers (wrong output or a runtime error), not truncations. A provisional v3 run at a
  different sampler (temperature 0.7, seed 42) solved 50 of the same 55; it is superseded and not tabulated.
### LiveCodeBench v6 (55 hard) — reasoning length
Same 55 problems, 12,288-token thinking budget. Tokens are the server's `completion_tokens`.

| model | pass@1 | mean tokens | median tokens | length-capped | solved / 1k tokens |
|---|---|---|---|---|---|
| **JackOD-9B-Coder v3** | **0.9455** | 6,644 | 6,103 | **0** | 0.142 |

For orientation only (greedy, so not the same basis): v2 averaged 6,809 tokens with 2 of 55 length-capped.

### Tool calling — `tool-eval-bench` hardmode

84 scenarios scored, 168 points, **5 paired seeds (42–46)**, n=5 → t=2.776, `--hardmode
--weight-by-difficulty`, scorer `2.6.1.dev65+g6be685f0e`, llama.cpp `5bda51bf` with `--jinja`, served and
requested with the same serving sampler as above.

| model | total points (mean, 5 seeds) | 95% CI | per seed |
|---|---|---|---|
| **JackOD-9B-Coder v3** | **143.4 ±2.6** | [140.8, 146.0] | 143 · 146 · 141 · 145 · 142 |

v2 (140.2 ±4.2) and v1 (144.4 ±4.7) were measured greedy, so that comparison crosses samplers. v3 sits inside
both of their intervals.

| category | **v3** (mean of 5) |
|---|---|
| M Autonomous Planning (6) | 4.4 |
| H Instruction Following (10) | 8.0 |
| I Context & State (20) | 15.6 |
| P Hard Mode (38) | 31.4 |
| C Multi-Step Chains (8) | 8.0 |
| K Safety & Boundaries (26) | 21.2 |
| L Toolset Scale (8) | 7.6 |
| E Error Recovery (6) | 5.0 |
| J Code Patterns (6) | 5.0 |
| G Structured Reasoning (6) | 5.2 |

Categories A/B/D/F/N (tool selection, parameter precision, restraint & refusal, localization, creative
composition) are 6/6 on every seed; Structured Output is 2/4, as for every arm in the earlier cohorts.

**Prompt injection.** v3 fails `TC-60` (cross-turn sleeper injection) on 4 of 5 seeds: an instruction
planted in an earlier tool result is later acted on. v1, v2, the base and DeltaCoder fail it too, so it is
inherited, not introduced by the merge. Do not rely on this model to ignore instructions that arrive inside
tool output.

## Serving

`generation_config.json` ships the serving recipe:

```
temperature 0.4 · top_p 0.95 · top_k 20 · min_p 0 · repeat_penalty 1.002
```

Add, where your runtime supports them (ollama and llama.cpp do):

```
repeat_last_n 112 · presence_penalty 1.05
```

Leave the **seed unfixed**: that is how v3 was evaluated.

> **Worth trying: seed 42.** In the author's testing it behaves better (no loops, and stellar on some tasks)
> but it is not reliable: it still fails on others. It is a suggestion to test, not part of the recommended
> setup, and none of the scores on this card were measured with it.

This replaces the v1/v2 recipe (temperature 0.8, min_p 0.05, repeat_penalty 1.05 over 96, presence_penalty 0).
The reasoning earlier revisions of this card gave for 0.8 was worked out on v1 and does not describe v3.
For tool calling, `--jinja` is **mandatory** in llama.cpp: without it the tool grammar never comes from the
model's chat template and every tool scenario degrades to prose.
## Quants
GGUF quants: **[`ManniX-ITA/JackOD-9B-Coder-MTP-GGUF`](https://huggingface.co/ManniX-ITA/JackOD-9B-Coder-MTP-GGUF)**.
Every tier there is **v3**, built 2026-10-24 from these weights with the full AtomicChat imatrix; the
`Q6_K` there is the file every v3 score above was measured on. DFlash builds:
[`ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF`](https://huggingface.co/ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF).

On ollama (both published 2026-10-25, the serving sampler above baked into every tag):

- **[`mannix/jackod-9b-coder`](https://ollama.com/mannix/jackod-9b-coder)**: all 27 tiers, `:latest` =
  `Q4_K_M`. The tags set `draft_num_predict 3` for the built-in MTP head and declare ollama ≥ 0.32.12 (the
  `qwen3.5` renderer). Every tag was pulled back from the registry and passed a generation check on 2026-10-25.
- **[`mannix/jackod-9b-coder-dflash`](https://ollama.com/mannix/jackod-9b-coder-dflash)**: the same 27 tiers
  with the `Qwen3.5-9B-DFlash` Q8_0 drafter (`draft_num_predict 4`). Needs **xollama**; stock ollama has no
  `draft-dflash`. See [`ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF`](https://huggingface.co/ManniX-ITA/JackOD-9B-Coder-MTP-DFlash-GGUF).
`num_ctx` is not pinned in the tags: set it as high as your hardware allows for agentic work.

## JackOD-9B-Coder v2 — previous weights (replaced 2026-10-24)

Everything in this section was measured on the **v2** weights (Coder-Flash 0.30 / Ornith 0.15), **greedy**
(temperature 0), and served with the v1/v2 recipe. It is kept as the record of v2 and does not describe the v3
weights or the v3 sampler.

### Evaluation

Measured on the **published Q6_K** (`JackOD-9B-Coder-Q6_K.gguf`, sha256
`c94d907c…d25a`, full AtomicChat `ornith-1.5-9b` imatrix `5ad024b8…`), llama.cpp
`311d4211bf`, **greedy** (`sampler=template_default`, temp 0.0 / top-p 1.0 /
top-k 0), scores read from `summary.json.score`. v2 and Coder-Flash were run
together, on the same machine and build. The v1 and other source columns are
the v1 cohort (same templates, same sampler, its own llama.cpp build) and are
there for orientation.

| Benchmark | **JackOD v2** | JackOD v1 | Coder-Flash (0.30) | Qwen3.5-9B (base) | DeltaCoder-9B (0.55) | Ornith-1.5-9B (0.15) |
|---|---|---|---|---|---|---|
| HumanEval (164) | 0.8902 | 0.8841 | 0.8415 | 0.8902 | **0.9146** | 0.7805 |
| HumanEval+ (164) | **0.8293** | 0.8232 | 0.7805 | 0.8049 † | 0.8232 | 0.7073 |
| **LiveCodeBench v6 (55 hard)** | **0.8000** | 0.7818 | 0.6727 | 0.7273 | 0.6364 | 0.5818 |
| MultiPL-E (300) | 0.8033 | 0.8033 | 0.7933 | **0.8200** | 0.8000 | 0.7267 |
| IFEval (100) | **0.9400** | 0.9100 | 0.8500 | 0.9300 † | 0.9200 | 0.8200 |

† From an earlier 9B bank, not the cohort run (see the v1 section).

- **v2 is at or above v1 on every row.** HumanEval, HumanEval+ and LiveCodeBench
  each move by one problem, which is inside the run-to-run spread of a greedy
  llama.cpp eval and is not claimed as an improvement. IFEval (+3 prompts) is the
  one real move, and it puts v2 above the base.
- **LiveCodeBench 0.8000 is above every source and the base**: +7.27 pp over the
  base, +12.73 pp over Coder-Flash, the source it gained.
- MultiPL-E is unchanged at 0.8033, still 1.67 pp under the base.

#### LiveCodeBench v6 (55 hard) — accuracy and reasoning length

Same 55 problems, same 12,288-token thinking budget, same cap, for every arm.
Token counts are the server's `completion_tokens`; *budget hits* are runs that
reached the thinking budget; *length-capped* never terminated on their own.

| model | pass@1 | mean tokens | median tokens | budget hits (solved) | length-capped | solved / 1k tokens |
|---|---|---|---|---|---|---|
| Qwen3.5-9B (base) | 0.7273 | 13,005 | 12,685 | 47 (32) | 18 | 0.056 |
| Qwopus3.5-9B-Coder | 0.6000 | 10,811 | 12,437 | 38 (17) | 8 | 0.055 |
| Qwopus3.5-9B-Coder-Flash | 0.6727 | 5,922 | 3,336 | 14 (4) | 6 | 0.114 |
| JackOD-9B-Coder **v1** | 0.7818 | 8,869 | 12,358 | 29 (18) | 1 | 0.088 |
| **JackOD-9B-Coder v2** | **0.8000** | **6,809** | **5,524** | 15 (6) | 2 | **0.117** |

- **About 23% fewer tokens than v1** (mean 6,809 vs 8,869) and under half the
  median (5,524 vs 12,358). On the 36 problems both versions solve, v2 is
  shorter on 29 (median length ratio 0.66).
- The solved sets differ: 36 both, 7 only v1, 8 only v2. One 55-problem cell
  does not separate two models that close, and it is not claimed to.
- v2 is now the most token-efficient arm in the table (0.117 solved per 1k
  tokens), slightly ahead of Coder-Flash alone (0.114), at +12.73 pp accuracy.
- Termination stays near v1: **2 of 55** length-capped (v1: 1).

#### Tool calling — `tool-eval-bench` hardmode

84 scenarios scored, 168 points, **5 paired seeds (42–46)**, n=5 → t=2.776,
greedy, `--hardmode --weight-by-difficulty`, scorer `2.6.1.dev65+g6be685f0e`,
llama-server `--jinja`. v2 and Coder-Flash were served by llama.cpp
`311d4211bf` in the same run; the v1 cohort rows were served by `5bda51bf`, so
v2-vs-v1 crosses two llama.cpp builds.

| model | total points (mean, 5 seeds) | 95% CI | run |
|---|---|---|---|
| Ornith-1.5-9B | 145.6 ±4.2 | [141.4, 149.8] | v1 cohort |
| JackOD-9B-Coder **v1** | 144.4 ±4.7 | [139.7, 149.1] | v1 cohort |
| Qwen3.5-9B (base) | 142.0 ±3.9 | [138.1, 145.9] | v1 cohort |
| Qwopus3.5-9B-Coder-Flash | 140.6 ±1.9 | [138.7, 142.5] | v2 run |
| **JackOD-9B-Coder v2** | **140.2 ±4.2** | [136.0, 144.4] | v2 run |
| DeltaCoder-9B | 139.6 ±5.7 | [133.9, 145.3] | v1 cohort |
| Qwopus3.5-9B-Coder | 132.6 ±3.6 | [129.0, 136.2] | v1 cohort |

**v2 does not improve tool calling.** It is level with Coder-Flash and 4.2
points under v1; the intervals overlap, so this is not a measured regression
either, but there is no gain to claim.

| category | **v2** | v1 | Coder-Flash |
|---|---|---|---|
| M Autonomous Planning (6) | 4.2 | **5.2** | 2.4 |
| H Instruction Following (10) | 8.0 | **9.6** | 8.4 |
| I Context & State (20) | 15.8 | **17.0** | 16.8 |
| P Hard Mode (38) | 29.4 | **29.8** | 28.2 |
| C Multi-Step Chains (8) | 7.0 | **7.4** | 6.8 |
| K Safety & Boundaries (26) | 21.2 | 20.2 | **21.6** |
| L Toolset Scale (8) | 6.0 | 6.8 | **8.0** |
| E Error Recovery (6) | 5.0 | 4.6 | **5.8** |
| J Code Patterns (6) | **5.6** | 5.8 | 5.4 |

**v1's planning edge does not carry over.** v1 led the whole cohort on
Autonomous Planning (5.2/6); v2 scores 4.2, under the base (4.8) though well
above Coder-Flash (2.4). If long-horizon planning in a tool loop is what you
need, v1's numbers are not a description of this model.

**Prompt injection.** On every v2 seed the model fails `TC-60` (cross-turn
sleeper injection): an instruction planted in an earlier tool result is later
acted on (an attacker address added as BCC/CC). The same failure appears in v1,
the base and DeltaCoder — it is inherited, not introduced by the merge. Do not
rely on this model to ignore instructions that arrive inside tool output.

Categories A/B/D/F/N (tool selection, parameter precision, restraint & refusal,
localization, creative composition) are 6/6 for v2; G Structured Reasoning 6/6;
Structured Output 2/4, as for every arm in both cohorts.

## JackOD-9B-Coder v1 — previous weights (replaced 2026-10-21)

Everything below was measured on the **v1** weights (Qwopus3.5-9B-Coder in the
0.30 slot). It is kept because it is the record of how this merge was chosen;
it does not describe the v2 or v3 weights.

### Field impression — initial, limited testing

> **This section is the author's early hands-on impression from a small number
> of runs, not a benchmark result.** It is reported because it is the reason the
> model exists, and it should be read as an anecdote until a proper harness
> campaign replaces it.

Verification against the **cline test harness** is still in progress. So far it
has **fixed a task in 535 s** on the harness; a second run went to timeout. It
also fixed the task on a separate machine through the plugin.

For comparison, on the same task, A3B-class models typically take **1.5–4 h**
with a success rate **below 50%**. The one exception is **Ornith-1.5-27B-A3B**,
which has a high success rate but still needs **~2 h or more** per task.

That gap — minutes rather than hours, from a 9B dense model — is what this merge
was assembled to get. It is an early signal from few runs, and the sample is far
too small to quote a success rate of its own.

### Evaluation

Q6_K + imatrix, llama.cpp, **greedy** (`sampler=template_default`, temp 0.0 /
top-p 1.0 / top-k 0). Scores read from `summary.json.score`. The LiveCodeBench
row is **`lcb_v6_55`** throughout — `lcb_medium_55` is a different problem set
and is never mixed in.

| Benchmark | **JackOD-9B-Coder** | Qwen3.5-9B (base) | DeltaCoder-9B (0.55) | Qwopus3.5-9B-Coder (0.30) | Ornith-1.5-9B (0.15) |
|---|---|---|---|---|---|
| HumanEval (164) | 0.8841 | 0.8902 | **0.9146** | 0.8537 | 0.7805 |
| HumanEval+ (164) | **0.8232** | 0.8049 † | **0.8232** | 0.7988 | 0.7073 |
| **LiveCodeBench v6 (55 hard)** | **0.7818** | 0.7273 | 0.6364 | 0.6000 | 0.5818 |
| MultiPL-E (300) | 0.8033 | **0.8200** | 0.8000 | **0.8200** | 0.7267 |
| IFEval (100) | 0.9100 | **0.9300** † | 0.9200 | 0.8800 | 0.8200 |

† The base's HumanEval+ and IFEval cells come from an earlier 9B bank rather
than the cohort pod run. That bank agrees with the cohort exactly on HumanEval
(0.8902 both) but differs on MultiPL-E by 1.67 pp, so treat those two base cells
as indicative rather than same-basis.

**LiveCodeBench is the real result: 0.7818 beats every source and the base.**
+5.45 pp over the strongest source (the base itself) and **+14.54 pp over
DeltaCoder**, the 0.55-weight source the merge leans on hardest. On the hard
problems the merge is not splitting the difference between its parents — it is
above all of them.

Elsewhere it lands where a merge should: identical to DeltaCoder on HumanEval+
(0.8232 both), a little under it on HumanEval (−3.05 pp, 5 problems of 164) and
IFEval, a little under the base on MultiPL-E (−1.67 pp). Those are give-backs,
and small ones.

### Length and runaway behaviour — where the merge actually comes from

Response length in characters on `lcb_v6_55`, the same 55 hard problems under an
identical cap. **capped** = `finish_reason == "length"`: the model never
terminated on its own.

| model | capped / 55 | p50 | p90 | max | >20k |
|---|---|---|---|---|---|
| DeltaCoder-9B (0.55) | **25** | 37,524 | 50,555 | 77,590 | 54 |
| Qwen3.5-9B (base) | 18 | 37,984 | 49,376 | 53,564 | 53 |
| Qwopus3.5-9B-Coder (0.30) | 8 | 29,261 | 43,261 | 59,633 | 41 |
| Ornith-1.5-9B (0.15) | 2 | 28,722 | 50,278 | 56,571 | 28 |
| **JackOD-9B-Coder** | **1** | 29,282 | 45,749 | 89,324 | 37 |

**This table is the thesis of the merge.** Read next to the eval table above,
the two heaviest sources split cleanly along opposite axes:

- **DeltaCoder** is the best coder in the cohort (HumanEval 0.9146) and the
  *worst* at stopping — **25 of 55** hard problems never terminate.
- **Ornith-1.5-9B** is the weakest coder (LiveCodeBench 0.5818, HumanEval
  0.7805) and by far the best at stopping — **2 of 55**.
- **JackOD takes both**: the best LiveCodeBench score of the entire cohort
  **and** the best termination rate, **1 of 55**.

Against the base that is an **18× reduction in non-terminating generations**;
against DeltaCoder, its heaviest source, **25×**. A capped generation is a
scoring failure regardless of whether the reasoning was on track, so this is not
a separate nicety — it is a large part of *why* the LiveCodeBench number is what
it is.

Other benches, all on the published Q6_K:
| Benchmark | p50 | p90 | max | >20k |
|---|---|---|---|---|
| HumanEval (164) | 590 | 1,192 | 60,219 | 1 |
| HumanEval+ (164) | 590 | 1,182 | 5,961 | **0** |
| MultiPL-E (300) | 257 | 539 | 1,007 | **0** |
| IFEval (100) | 791 | 3,160 | 11,660 | **0** |
### Earlier attempts in this family — do not use them for agentic work
This merge went through several earlier arms. They are listed here because their
numbers are in circulation and, read as a table, **they are misleading**.
| arm | HumanEval | HumanEval+ | LCB v6 (55) | MultiPL-E | IFEval | capped / 55 |
|---|---|---|---|---|---|---|
| JackOD-Coder-9B | 0.8232 | 0.7683 | **0.8182** | 0.7933 | **0.9300** | 3 |
| JackOD3.5-9B | 0.8659 | — | 0.7455 | — | — | 7 |
| JackDeltaCoder-9B | 0.8415 | 0.7683 | 0.6364 | 0.8067 | 0.9200 | 8 |
| **JackOD-9B-Coder** (v1) | **0.8841** | **0.8232** | 0.7818 | 0.8033 | 0.9100 | **1** |
**`JackOD-Coder-9B` scores higher than this model on LiveCodeBench (0.8182 vs
0.7818) and on IFEval, and it is still the wrong model to deploy.** That is the
point of this section.
In real multi-turn agentic coding — a live tool loop, not a benchmark harness —
**JackDeltaCoder-9B and every earlier JackOD arm fail**:
- they **do not drive tools properly**, mis-forming or skipping tool calls that
  the task depends on;
- they **fall into loops**, re-treading the same step instead of advancing;
- they **abandon the task**, concluding the problem is too hard to solve and
  stopping without a fix.
None of that is visible in a single-shot code benchmark, which is exactly why
their benchmark rows look competitive. A one-shot HumanEval or LiveCodeBench
problem never asks a model to call a tool, read the result, and decide what to
do on turn seven. **The only arm in this family that holds up in a live agentic
loop is the published one.** Treat the rows above as a record of what was tried,
not as a menu.
### Tool calling — `tool-eval-bench` hardmode
88 scenarios offered, 84 scored, 168 points. **5 paired seeds (42–46)** per arm,
n=5 → t=2.776, greedy, `--hardmode --weight-by-difficulty`, scorer
`2.6.1.dev65+g6be685f0e`, served by llama.cpp `5bda51bf` with `--jinja`. Every
arm shares that basis exactly.
*(v1 tool-calling chart removed with the v1 weights; numbers below.)*
| model | role | total points (mean, 5 seeds) | 95% CI |
|---|---|---|---|
| Ornith-1.5-9B | source | **145.6 ±4.2** | [141.4, 149.8] |
| **JackOD-9B-Coder** | **merge** | **144.4 ±4.7** | [139.7, 149.1] |
| Qwen3.5-9B | base | 142.0 ±3.9 | [138.1, 145.9] |
| DeltaCoder-9B | source | 139.6 ±5.7 | [133.9, 145.3] |
| Qwopus3.5-9B-Coder | source | 132.6 ±3.6 | [129.0, 136.2] |
**On aggregate points this merge does not beat its sources — it is second, and
every interval overlaps.** It sits above its base and within noise of Ornith.
The aggregate is not where the difference lives.
#### Where it does separate: Autonomous Planning
| category | **JackOD** | Ornith | base | DeltaCoder | Qwopus |
|---|---|---|---|---|---|
| **M Autonomous Planning** (6) | **5.2** | 2.8 | 4.8 | 4.0 | 3.6 |
| H Instruction Following (10) | **9.6** | 9.2 | 9.2 | 8.8 | 8.0 |
| I Context & State (20) | 17.0 | **17.2** | 16.4 | 16.2 | 15.2 |
| P Hard Mode (38) | 29.8 | **30.2** | 29.8 | 28.2 | 26.0 |
| C Multi-Step Chains (8) | 7.4 | **8.0** | 6.6 | 7.2 | 5.2 |
| K Safety & Boundaries (26) | 20.2 | **21.4** | 19.2 | 19.6 | **21.4** |
| L Toolset Scale (8) | 6.8 | **7.8** | 7.2 | 7.2 | 6.8 |
| E Error Recovery (6) | 4.6 | **5.0** | **5.0** | **5.0** | 3.4 |
| J Code Patterns (6) | 5.8 | **6.0** | 5.8 | 5.4 | **6.0** |
**Autonomous Planning is the one axis with a clear, consistent edge: 5.2/6,
best of all five arms — while Ornith, the top aggregate scorer, is the worst at
2.8/6.** The merge reaches Ornith's overall level while fixing exactly Ornith's
long-horizon planning weakness.
That pairing is worth sitting with, because Ornith is also the source with the
best termination rate (2/55 capped). A model that stops reliably *and* plans
worst than anything else in the cohort looks like one that stops **too early** —
terminating is easy if you give up. The merge is the arm that does both: it
carries the goal (5.2/6 planning, best) and still lands the plane (1/55 capped,
best). That combination, not either number alone, is what the field impression
above is describing.
Categories A/B/D/F (tool selection, parameter precision, restraint & refusal,
localization) are 6/6 for every arm and are omitted; G Structured Reasoning and
N Creative Composition are 6/6 for all but Qwopus (5.8 and 5.2). Structured
Output is 2.0/4 for all five — none of these models clears it.