---
license: mit
tags:
- metacog
- ensemble
- system-one
- text-generation
base_model:
- prism-ml/Ternary-Bonsai-2-27B-gguf
- Mapika/decider-4b
---

# Heuristic 1

One model out of two: **Bonsai 2 27B (ternary)** thinks, **Decider-4B** judges,
[MetaCog](https://github.com/ItIsCuthNotCup/MetaCog) wires them into a single
OpenAI-compatible endpoint.

> **This is not a weight merge.** No tensors were averaged. It is an
> inference-time pipeline merge: the thinker writes candidate answers, an
> independent System-One judge (Decider-4B) scores them, and the winner is
> returned. One endpoint, one model ID, two models inside.

```
user prompt ──▶ bonsai-27B ──▶ greedy anchor judged FIRST
                                  │  ≥ 0.80 and finished? ──▶ answer immediately
                                  ▼
                        7 sampled paths IN PARALLEL,
                        each judged the moment it finishes;
                        first finished path ≥ 0.80 wins on the spot,
                        the rest are abandoned
                                  │
                        (nothing clears the bar? best judged
                         finished path wins)
```

The judge is *not* the thinker — that separation is the whole point.

`race.py` implements this loop (parallel generation, judge-as-they-finish,
early stop). The old sequential MetaCog `best_of_n` is still available via
`NO_RACE=1` (server) or `--no-race` (batch) for A/B comparison. The
early-stop bar is `STOP_CONFIDENCE` / `--stop-confidence` (default 0.80).

> **On the 0.80 bar, honestly:** the MetaCog repo measured its 0.95 cascade
> threshold on real Jev (a noul ≥ 0.95 was correct 226/229 times). 0.80 did
> not win anywhere — it is the operator's directive and is untested on
> decider-4b, whose confidence scale is not calibrated (values above 1.0
> have been observed). Measure before trusting it.

## Measured

| eval | bonsai alone | merge (Heuristic 1) | note |
|---|---|---|---|
| HumanEval, n=164, true pass@1 (prompt+completion graded as one program, `<think>` stripped) | 108/164 (65.9%) | **135/164 (82.3%)** | judge fixed 27, broke 0; McNemar p < 1e-6 |
| 12 easy Q&A (verified) | 12/12 | 12/12 | tie; nothing to fix |
| 10 hard Q&A (verified, tricks + big arithmetic) | 8/10 | **10/10** | judge fixed 2, broke 0 |

Cost per problem (HumanEval averages): merge uses ~5,600 thinker tokens
over 2.0 thinker calls + 8.5 judge calls; bonsai alone uses ~700 tokens in
1 call. Roughly 8× the thinker tokens for +16 points of accuracy, $0 on
local hardware.

Latency: when the judge is confident in the greedy answer it returns
immediately (~same as bonsai alone — e.g. 2.5s on a trivial prompt); the
full vote only runs when uncertain. Measured on 3 HumanEval problems where
the full vote ran (DGX Spark, sequential paths, old loop): heuristic-1 took
210s / 406s / 440s vs bonsai alone at 17s / 18s / 18s — roughly 12–24× slower
wall-clock on hard problems. The race loop (`race.py`) parallelizes the
sampled paths and cuts off the moment any finished path clears 0.80, which
removes most of that gap; re-measure on the Spark to confirm. Every response
carries `merge.judge_confidence`.

## Run it

```bash
pip install -r requirements.txt   # metacog + httpx + pydantic

# one model on :8200
python server.py
# POST /v1/chat/completions {"messages": [...], "model": "bonsai-decider4b-merge"}

# batch runner
python merge.py --problem "What is 17*23?"
python merge.py --problems problems.jsonl --out results.jsonl
```

Env knobs: `THINKER_URL` (default `http://localhost:8010`),
`THINKER_MODEL` (default `bonsai`), `JUDGE_URL` (default
`http://localhost:8008`), `PORT` (8200), `N_PATHS` (8), `NO_CASCADE`,
`MAX_TOKENS` (2048), `TEMPERATURE` (0.8).

You need both backends up:

- **Thinker** — any OpenAI-compatible chat endpoint serving Bonsai 2 27B.
  Reference setup: bjev's PrismML llama.cpp fork on `:8010`
  (`Ternary-Bonsai-2-27B-PQ2_0.gguf`, 6.66 GiB).
- **Judge** — a `/v1/systemone` endpoint serving Decider-4B.
  Reference: `runs/judge_server.py` (a `/v1/systemone` shim over the
  `Mapika/decider-4b` checkpoint) on `:8008`.

On a DGX Spark this fits alongside both models in ~20 GB and runs at $0.

## Files

- `server.py` — the one-model OpenAI-compatible server (`:8200`)
- `merge.py` — batch runner (problems in, answers + confidence + timings out)
- `eval/ab_bonsai_vs_merge.py` — A/B harness: bonsai alone vs the merge
- `eval/problems_easy.jsonl`, `eval/problems_hard.jsonl` — the measured sets

## License

MIT. This repo ships pipeline code only — no weights. Parent model licenses
(checked 2026-09-27):
- Bonsai 2 27B ternary (`prism-ml/Ternary-Bonsai-2-27B-gguf`) — Apache-2.0
- Decider-4B (`Mapika/decider-4b`) — Apache-2.0
- MetaCog — MIT; llama.cpp — MIT
Fetch the weights yourself under their Apache-2.0 terms; nothing here
relicenses them. Not affiliated with PrismML, Mapika, or Alibaba.