Upload 11 files

.gitignore

@@ -0,0 +1,7 @@
+__pycache__/
+*.pyc
+.venv/
+*.jsonl
+!.gitignore
+!eval/problems_easy.jsonl
+!eval/problems_hard.jsonl

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jacob Thomas Cuthbertson
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,118 @@
+---
+license: mit
+tags:
+- metacog
+- ensemble
+- system-one
+- text-generation
+base_model:
+- prism-ml/Ternary-Bonsai-2-27B-gguf
+- Mapika/decider-4b
+---
+
+# Heuristic 1
+
+One model out of two: **Bonsai 2 27B (ternary)** thinks, **Decider-4B** judges,
+[MetaCog](https://github.com/ItIsCuthNotCup/MetaCog) wires them into a single
+OpenAI-compatible endpoint.
+
+> **This is not a weight merge.** No tensors were averaged. It is an
+> inference-time pipeline merge: the thinker writes candidate answers, an
+> independent System-One judge (Decider-4B) scores them, and the winner is
+> returned. One endpoint, one model ID, two models inside.
+
+```
+user prompt ──▶ bonsai-27B ──▶ greedy anchor judged FIRST
+                                  │  ≥ 0.80 and finished? ──▶ answer immediately
+                                  ▼
+                        7 sampled paths IN PARALLEL,
+                        each judged the moment it finishes;
+                        first finished path ≥ 0.80 wins on the spot,
+                        the rest are abandoned
+                                  │
+                        (nothing clears the bar? best judged
+                         finished path wins)
+```
+
+The judge is *not* the thinker — that separation is the whole point.
+
+`race.py` implements this loop (parallel generation, judge-as-they-finish,
+early stop). The old sequential MetaCog `best_of_n` is still available via
+`NO_RACE=1` (server) or `--no-race` (batch) for A/B comparison. The
+early-stop bar is `STOP_CONFIDENCE` / `--stop-confidence` (default 0.80).
+
+> **On the 0.80 bar, honestly:** the MetaCog repo measured its 0.95 cascade
+> threshold on real Jev (a noul ≥ 0.95 was correct 226/229 times). 0.80 did
+> not win anywhere — it is the operator's directive and is untested on
+> decider-4b, whose confidence scale is not calibrated (values above 1.0
+> have been observed). Measure before trusting it.
+
+## Measured
+
+| eval | bonsai alone | merge (Heuristic 1) | note |
+|---|---|---|---|
+| HumanEval, n=164, true pass@1 (prompt+completion graded as one program, `<think>` stripped) | 108/164 (65.9%) | **135/164 (82.3%)** | judge fixed 27, broke 0; McNemar p < 1e-6 |
+| 12 easy Q&A (verified) | 12/12 | 12/12 | tie; nothing to fix |
+| 10 hard Q&A (verified, tricks + big arithmetic) | 8/10 | **10/10** | judge fixed 2, broke 0 |
+
+Cost per problem (HumanEval averages): merge uses ~5,600 thinker tokens
+over 2.0 thinker calls + 8.5 judge calls; bonsai alone uses ~700 tokens in
+1 call. Roughly 8× the thinker tokens for +16 points of accuracy, $0 on
+local hardware.
+
+Latency: when the judge is confident in the greedy answer it returns
+immediately (~same as bonsai alone — e.g. 2.5s on a trivial prompt); the
+full vote only runs when uncertain. Measured on 3 HumanEval problems where
+the full vote ran (DGX Spark, sequential paths, old loop): heuristic-1 took
+210s / 406s / 440s vs bonsai alone at 17s / 18s / 18s — roughly 12–24× slower
+wall-clock on hard problems. The race loop (`race.py`) parallelizes the
+sampled paths and cuts off the moment any finished path clears 0.80, which
+removes most of that gap; re-measure on the Spark to confirm. Every response
+carries `merge.judge_confidence`.
+
+## Run it
+
+```bash
+pip install -r requirements.txt   # metacog + httpx + pydantic
+
+# one model on :8200
+python server.py
+# POST /v1/chat/completions {"messages": [...], "model": "bonsai-decider4b-merge"}
+
+# batch runner
+python merge.py --problem "What is 17*23?"
+python merge.py --problems problems.jsonl --out results.jsonl
+```
+
+Env knobs: `THINKER_URL` (default `http://localhost:8010`),
+`THINKER_MODEL` (default `bonsai`), `JUDGE_URL` (default
+`http://localhost:8008`), `PORT` (8200), `N_PATHS` (8), `NO_CASCADE`,
+`MAX_TOKENS` (2048), `TEMPERATURE` (0.8).
+
+You need both backends up:
+
+- **Thinker** — any OpenAI-compatible chat endpoint serving Bonsai 2 27B.
+  Reference setup: bjev's PrismML llama.cpp fork on `:8010`
+  (`Ternary-Bonsai-2-27B-PQ2_0.gguf`, 6.66 GiB).
+- **Judge** — a `/v1/systemone` endpoint serving Decider-4B.
+  Reference: `runs/judge_server.py` (a `/v1/systemone` shim over the
+  `Mapika/decider-4b` checkpoint) on `:8008`.
+
+On a DGX Spark this fits alongside both models in ~20 GB and runs at $0.
+
+## Files
+
+- `server.py` — the one-model OpenAI-compatible server (`:8200`)
+- `merge.py` — batch runner (problems in, answers + confidence + timings out)
+- `eval/ab_bonsai_vs_merge.py` — A/B harness: bonsai alone vs the merge
+- `eval/problems_easy.jsonl`, `eval/problems_hard.jsonl` — the measured sets
+
+## License
+
+MIT. This repo ships pipeline code only — no weights. Parent model licenses
+(checked 2026-09-27):
+- Bonsai 2 27B ternary (`prism-ml/Ternary-Bonsai-2-27B-gguf`) — Apache-2.0
+- Decider-4B (`Mapika/decider-4b`) — Apache-2.0
+- MetaCog — MIT; llama.cpp — MIT
+Fetch the weights yourself under their Apache-2.0 terms; nothing here
+relicenses them. Not affiliated with PrismML, Mapika, or Alibaba.

eval/ab2_bonsai_vs_merge.py

@@ -0,0 +1,94 @@
+#!/usr/bin/env python3
+"""A/B: bonsai alone (greedy, temp 0) vs the merge (bonsai + decider-4b judge).
+
+Both served on the Spark: bonsai :8010, merge :8200.
+Usage: .venv/bin/python runs/ab_bonsai_vs_merge.py
+"""
+import json
+import re
+import time
+import urllib.request
+
+PROBLEMS = [
+    ("What is 1234 * 5678? Reply with just the number.", "7006652"),
+    ("A bat and ball cost $1.10. The bat costs $1 more than the ball. How much does the ball cost in cents? Reply with just the number.", "5"),
+    ("What is the 16th letter of the alphabet? Reply with just the letter.", "p"),
+    ("If you have 3 apples and take away 2, how many do you have? Reply with just the number.", "2"),
+    ("What year did the Berlin Wall fall? Reply with just the year.", "1989"),
+    ("How many sides does a heptagon have? Reply with just the number.", "7"),
+    ("What is 1/3 + 1/6? Reply as a fraction in lowest terms.", "1/2"),
+    ("Which planet has the most moons as of 2024, Saturn or Jupiter? Reply with just the name.", "saturn"),
+    ("A farmer has 17 sheep, all but 9 die. How many are left? Reply with just the number.", "9"),
+    ("What is the square root of 144? Reply with just the number.", "12"),
+]
+
+
+def chat(url, model, problem, temperature):
+    body = json.dumps({
+        "model": model,
+        "messages": [{"role": "user", "content": problem}],
+        "max_tokens": 256,
+        "temperature": temperature,
+    }).encode()
+    req = urllib.request.Request(url, data=body,
+                                 headers={"Content-Type": "application/json"})
+    with urllib.request.urlopen(req, timeout=300) as r:
+        return json.load(r)
+
+
+def final_text(answer):
+    # strip <think>...</think>, take the last non-empty line
+    t = re.sub(r"<think>.*?</think>", "", answer, flags=re.S | re.I).strip()
+    lines = [l.strip() for l in t.splitlines() if l.strip()]
+    return lines[-1] if lines else t
+
+
+def correct(final, expect):
+    return expect.lower() in final.lower()
+
+
+def main():
+    rows = []
+    for i, (problem, expect) in enumerate(PROBLEMS):
+        # bonsai alone
+        t0 = time.time()
+        rb = chat("http://localhost:8010/v1/chat/completions", "bonsai",
+                  problem, 0.0)
+        tb = time.time() - t0
+        ab = rb["choices"][0]["message"]["content"]
+        fb, cb = final_text(ab), correct(final_text(ab), expect)
+
+        # merge
+        t0 = time.time()
+        rm = chat("http://localhost:8200/v1/chat/completions",
+                  "heuristic-1", problem, 0.0)
+        tm = time.time() - t0
+        am = rm["choices"][0]["message"]["content"]
+        fm, cm = final_text(am), correct(final_text(am), expect)
+        conf = rm.get("merge", {}).get("judge_confidence", 0)
+
+        rows.append({"i": i, "problem": problem, "expect": expect,
+                     "bonsai": fb[:120], "bonsai_ok": cb, "bonsai_s": round(tb, 1),
+                     "merge": fm[:120], "merge_ok": cm, "merge_s": round(tm, 1),
+                     "conf": conf})
+        print(f"[{i+1}/{len(PROBLEMS)}] expect={expect!r} "
+              f"bonsai={'OK ' if cb else 'MISS'} ({tb:.0f}s) "
+              f"merge={'OK ' if cm else 'MISS'} ({tm:.0f}s, conf={conf:.2f})",
+              flush=True)
+
+    b_ok = sum(r["bonsai_ok"] for r in rows)
+    m_ok = sum(r["merge_ok"] for r in rows)
+    flips = [r for r in rows if r["merge_ok"] and not r["bonsai_ok"]]
+    losses = [r for r in rows if r["bonsai_ok"] and not r["merge_ok"]]
+    print(f"\n=== bonsai alone: {b_ok}/{len(rows)} | merge: {m_ok}/{len(rows)} ===")
+    print(f"judge fixed: {len(flips)} | judge broke: {len(losses)}")
+    for r in flips:
+        print(f"  FIXED: {r['problem'][:60]}... bonsai={r['bonsai'][:40]!r} merge={r['merge'][:40]!r}")
+    for r in losses:
+        print(f"  BROKE: {r['problem'][:60]}... bonsai={r['bonsai'][:40]!r} merge={r['merge'][:40]!r}")
+    json.dump(rows, open("runs/ab2_results.json", "w"), indent=1)
+    print("wrote runs/ab2_results.json")
+
+
+if __name__ == "__main__":
+    main()

eval/ab_bonsai_vs_merge.py

@@ -0,0 +1,96 @@
+#!/usr/bin/env python3
+"""A/B: bonsai alone (greedy, temp 0) vs the merge (bonsai + decider-4b judge).
+
+Both served on the Spark: bonsai :8010, merge :8200.
+Usage: .venv/bin/python runs/ab_bonsai_vs_merge.py
+"""
+import json
+import re
+import time
+import urllib.request
+
+PROBLEMS = [
+    ("What is 17*23? Reply with just the number.", "391"),
+    ("A train travels 120 km in 1.5 hours. What is its average speed in km/h? Reply with just the number.", "80"),
+    ("What is the 5th prime number? Reply with just the number.", "11"),
+    ("If all bloops are razzies and all razzies are lazzies, are all bloops lazzies? Reply yes or no.", "yes"),
+    ("Name the planet closest to the sun. Reply with just the name.", "mercury"),
+    ("What is 15% of 200? Reply with just the number.", "30"),
+    ("A rectangle has length 9 and width 4. What is its area? Reply with just the number.", "36"),
+    ("Which is larger: 3/4 or 5/7? Reply with just the fraction.", "3/4"),
+    ("What is the capital of Australia? Reply with just the city name.", "canberra"),
+    ("If you flip a fair coin twice, what is the probability both are heads? Reply as a fraction.", "1/4"),
+    ("What is 7^3? Reply with just the number.", "343"),
+    ("A book costs $12 after a 20% discount. What was the original price in dollars? Reply with just the number.", "15"),
+]
+
+
+def chat(url, model, problem, temperature):
+    body = json.dumps({
+        "model": model,
+        "messages": [{"role": "user", "content": problem}],
+        "max_tokens": 256,
+        "temperature": temperature,
+    }).encode()
+    req = urllib.request.Request(url, data=body,
+                                 headers={"Content-Type": "application/json"})
+    with urllib.request.urlopen(req, timeout=300) as r:
+        return json.load(r)
+
+
+def final_text(answer):
+    # strip <think>...</think>, take the last non-empty line
+    t = re.sub(r"<think>.*?</think>", "", answer, flags=re.S | re.I).strip()
+    lines = [l.strip() for l in t.splitlines() if l.strip()]
+    return lines[-1] if lines else t
+
+
+def correct(final, expect):
+    return expect.lower() in final.lower()
+
+
+def main():
+    rows = []
+    for i, (problem, expect) in enumerate(PROBLEMS):
+        # bonsai alone
+        t0 = time.time()
+        rb = chat("http://localhost:8010/v1/chat/completions", "bonsai",
+                  problem, 0.0)
+        tb = time.time() - t0
+        ab = rb["choices"][0]["message"]["content"]
+        fb, cb = final_text(ab), correct(final_text(ab), expect)
+
+        # merge
+        t0 = time.time()
+        rm = chat("http://localhost:8200/v1/chat/completions",
+                  "heuristic-1", problem, 0.0)
+        tm = time.time() - t0
+        am = rm["choices"][0]["message"]["content"]
+        fm, cm = final_text(am), correct(final_text(am), expect)
+        conf = rm.get("merge", {}).get("judge_confidence", 0)
+
+        rows.append({"i": i, "problem": problem, "expect": expect,
+                     "bonsai": fb[:120], "bonsai_ok": cb, "bonsai_s": round(tb, 1),
+                     "merge": fm[:120], "merge_ok": cm, "merge_s": round(tm, 1),
+                     "conf": conf})
+        print(f"[{i+1}/{len(PROBLEMS)}] expect={expect!r} "
+              f"bonsai={'OK ' if cb else 'MISS'} ({tb:.0f}s) "
+              f"merge={'OK ' if cm else 'MISS'} ({tm:.0f}s, conf={conf:.2f})",
+              flush=True)
+
+    b_ok = sum(r["bonsai_ok"] for r in rows)
+    m_ok = sum(r["merge_ok"] for r in rows)
+    flips = [r for r in rows if r["merge_ok"] and not r["bonsai_ok"]]
+    losses = [r for r in rows if r["bonsai_ok"] and not r["merge_ok"]]
+    print(f"\n=== bonsai alone: {b_ok}/{len(rows)} | merge: {m_ok}/{len(rows)} ===")
+    print(f"judge fixed: {len(flips)} | judge broke: {len(losses)}")
+    for r in flips:
+        print(f"  FIXED: {r['problem'][:60]}... bonsai={r['bonsai'][:40]!r} merge={r['merge'][:40]!r}")
+    for r in losses:
+        print(f"  BROKE: {r['problem'][:60]}... bonsai={r['bonsai'][:40]!r} merge={r['merge'][:40]!r}")
+    json.dump(rows, open("runs/ab_results.json", "w"), indent=1)
+    print("wrote runs/ab_results.json")
+
+
+if __name__ == "__main__":
+    main()

eval/problems_easy.jsonl

@@ -0,0 +1,12 @@
+{"problem": "What is 17*23? Reply with just the number.", "expect": "391"}
+{"problem": "A train travels 120 km in 1.5 hours. What is its average speed in km/h? Reply with just the number.", "expect": "80"}
+{"problem": "What is the 5th prime number? Reply with just the number.", "expect": "11"}
+{"problem": "If all bloops are razzies and all razzies are lazzies, are all bloops lazzies? Reply yes or no.", "expect": "yes"}
+{"problem": "Name the planet closest to the sun. Reply with just the name.", "expect": "mercury"}
+{"problem": "What is 15% of 200? Reply with just the number.", "expect": "30"}
+{"problem": "A rectangle has length 9 and width 4. What is its area? Reply with just the number.", "expect": "36"}
+{"problem": "Which is larger: 3/4 or 5/7? Reply with just the fraction.", "expect": "3/4"}
+{"problem": "What is the capital of Australia? Reply with just the city name.", "expect": "canberra"}
+{"problem": "If you flip a fair coin twice, what is the probability both are heads? Reply as a fraction.", "expect": "1/4"}
+{"problem": "What is 7^3? Reply with just the number.", "expect": "343"}
+{"problem": "A book costs $12 after a 20% discount. What was the original price in dollars? Reply with just the number.", "expect": "15"}

eval/problems_hard.jsonl

@@ -0,0 +1,10 @@
+{"problem": "What is 1234 * 5678? Reply with just the number.", "expect": "7006652"}
+{"problem": "A bat and ball cost $1.10. The bat costs $1 more than the ball. How much does the ball cost in cents? Reply with just the number.", "expect": "5"}
+{"problem": "What is the 16th letter of the alphabet? Reply with just the letter.", "expect": "p"}
+{"problem": "If you have 3 apples and take away 2, how many do you have? Reply with just the number.", "expect": "2"}
+{"problem": "What year did the Berlin Wall fall? Reply with just the year.", "expect": "1989"}
+{"problem": "How many sides does a heptagon have? Reply with just the number.", "expect": "7"}
+{"problem": "What is 1/3 + 1/6? Reply as a fraction in lowest terms.", "expect": "1/2"}
+{"problem": "Which planet has the most moons as of 2024, Saturn or Jupiter? Reply with just the name.", "expect": "saturn"}
+{"problem": "A farmer has 17 sheep, all but 9 die. How many are left? Reply with just the number.", "expect": "9"}
+{"problem": "What is the square root of 144? Reply with just the number.", "expect": "12"}

merge.py

@@ -0,0 +1,126 @@
+#!/usr/bin/env python3
+"""The merge: bonsai thinks, decider-4b judges, MetaCog wires them into one model.
+
+    thinker: bonsai (bjev llama.cpp, OpenAI chat) .... http://localhost:8010
+    judge:   decider-4b (/v1/systemone shim) ......... http://localhost:8008
+
+Usage:
+    .venv/bin/python merge.py --problem "What is 17*23?"
+    .venv/bin/python merge.py --problems problems.jsonl --out results.jsonl
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+
+import re
+
+def clean_answer(text: str) -> str:
+    """Strip <think>...</think> reasoning blocks the thinker may leak."""
+    return re.sub(r"<think>.*?</think>\s*", "", text, flags=re.S).strip()
+
+
+from metacog import Config, MetaCog, OpenAICompatThinker, SystemOneJudge
+
+import os
+
+THINKER_URL = os.environ.get("THINKER_URL", "http://localhost:8010")
+THINKER_MODEL = os.environ.get("THINKER_MODEL", "bonsai")
+JUDGE_URL = os.environ.get("JUDGE_URL", "http://localhost:8008")
+
+
+def build_thinker_judge():
+    thinker = OpenAICompatThinker(THINKER_URL, model=THINKER_MODEL)
+    # non-TypeSafe base_url -> no auth header; the local shim is open
+    judge = SystemOneJudge(base_url=JUDGE_URL)
+    return thinker, judge
+
+
+def build(n_paths: int = 8, no_cascade: bool = False) -> MetaCog:
+    thinker, judge = build_thinker_judge()
+    config = Config(
+        mode="best_of_n",
+        strategy="noul",
+        n_paths=n_paths,
+        greedy_anchor=True,
+        cascade_confidence=None if no_cascade else 0.95,
+        max_tokens=2048,
+        temperature=0.8,
+    )
+    return MetaCog(thinker, judge, config)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description="bonsai + decider-4b MetaCog merge")
+    src = ap.add_mutually_exclusive_group(required=True)
+    src.add_argument("--problem")
+    src.add_argument("--problems")
+    ap.add_argument("--n-paths", type=int, default=8)
+    ap.add_argument("--no-cascade", action="store_true")
+    ap.add_argument("--stop-confidence", type=float, default=0.80,
+                    help="race loop early-stop bar (default 0.80)")
+    ap.add_argument("--no-race", action="store_true",
+                    help="use the old sequential MetaCog best_of_n instead of the race loop")
+    ap.add_argument("--out", default="runs/merge_results.jsonl")
+    args = ap.parse_args()
+
+    if args.problem:
+        problems = [args.problem]
+    else:
+        problems = [json.loads(l)["problem"] for l in open(args.problems)
+                    if l.strip()]
+    if not problems:
+        print("no problems", file=sys.stderr)
+        return 1
+
+    mc = build(args.n_paths, args.no_cascade)
+    use_race = not args.no_race
+    thinker = judge = None
+    if use_race:
+        from race import race
+        thinker, judge = build_thinker_judge()
+    t0 = time.time()
+    with open(args.out, "w") as out:
+        for i, problem in enumerate(problems):
+            t1 = time.time()
+            if use_race:
+                r = race(problem, thinker, judge, n_paths=args.n_paths,
+                         stop_confidence=args.stop_confidence)
+                answer, finished = clean_answer(r.answer), r.finished
+                conf, toks, jcalls, dt = (r.confidence, r.thinker_tokens,
+                                          r.judge_calls, r.seconds)
+                extra = {"early_stopped": r.early_stopped,
+                         "winner_source": r.winner_source,
+                         "paths_judged": r.paths_judged}
+            else:
+                result = mc.run(problem)
+                dt = time.time() - t1
+                rnd = result.trace.rounds[-1] if result.trace.rounds else None
+                answer, finished = clean_answer(result.answer), result.finished
+                conf = round(rnd.verdict.confidence, 4) if rnd else 0.0
+                toks, jcalls = result.trace.thinker_tokens, result.trace.judge_calls
+                extra = {}
+            row = {
+                "i": i,
+                "problem": problem,
+                "answer": answer,
+                "finished": finished,
+                "judge_confidence": round(conf, 4),
+                "thinker_tokens": toks,
+                "judge_calls": jcalls,
+                "seconds": round(dt, 2),
+                **extra,
+            }
+            out.write(json.dumps(row) + "\n")
+            out.flush()
+            print(f"[{i+1}/{len(problems)}] conf={conf:.2f} "
+                  f"{dt:.1f}s: {answer[:120]}", flush=True)
+    print(f"done: {len(problems)} problems in {time.time()-t0:.0f}s -> {args.out}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

race.py

@@ -0,0 +1,148 @@
+#!/usr/bin/env python3
+"""Heuristic 1.1 race loop: parallel thought paths, judged as they finish.
+
+Replaces the sequential best_of_n + judge-everything-at-the-end loop:
+
+1. The greedy anchor (temperature 0) is generated and judged FIRST. If the
+   judge scores it >= stop_confidence and it finished, it wins immediately --
+   no sampled paths are ever generated.
+2. Otherwise n_paths-1 sampled paths are generated IN PARALLEL
+   (ThreadPoolExecutor over the thinker's HTTP endpoint).
+3. Each path is judged the moment it completes. The first FINISHED path to
+   score >= stop_confidence wins on the spot; the race is cut off at that
+   instant -- queued paths are cancelled and race() returns immediately.
+   (Honest cutoff semantics: threads already running a blocking HTTP
+   generation can't be killed, so they finish silently in the background
+   and their results are discarded. Nothing waits for them.)
+4. If nothing clears the bar, the best judged finished path wins. Finished
+   paths always outrank truncated ones, whatever the score.
+
+Only the thinker's .generate() and the judge's .score() from the metacog
+package are used -- the MetaCog controller is not touched.
+
+Caveat: decider-4b's confidence scale is not calibrated (values above 1.0
+have been observed), so stop_confidence is a knob to measure, not a true
+probability. The 0.95 default in the MetaCog repo was measured on real Jev
+(226/229 correct at >= 0.95); 0.80 here is the operator's directive and is
+untested on decider-4b.
+"""
+
+from __future__ import annotations
+
+import threading
+import time
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from dataclasses import dataclass
+
+
+@dataclass
+class RaceResult:
+    answer: str
+    finished: bool
+    confidence: float
+    thinker_tokens: int = 0
+    judge_calls: int = 0
+    paths_submitted: int = 0
+    paths_judged: int = 0
+    early_stopped: bool = False
+    winner_source: str = "sample"  # "greedy" | "sample"
+    seconds: float = 0.0
+
+
+def _judge_one(judge, problem: str, text: str) -> float:
+    """Judge's P(correct) for a single path; 0.0 if the judge call fails."""
+    try:
+        return float(judge.score(problem, [text]).confidence)
+    except Exception:
+        return 0.0
+
+
+def race(
+    problem: str,
+    thinker,
+    judge,
+    n_paths: int = 8,
+    stop_confidence: float = 0.80,
+    max_tokens: int = 2048,
+    temperature: float = 0.8,
+    max_workers: int = 8,
+) -> RaceResult:
+    t0 = time.time()
+    lock = threading.Lock()
+    thinker_tokens = 0
+    judge_calls = 0
+
+    def gen(**kw):
+        nonlocal thinker_tokens
+        gens = thinker.generate(problem, "", max_tokens=max_tokens, **kw)
+        with lock:
+            for g in gens:
+                thinker_tokens += g.tokens or 0
+        return gens
+
+    def judge_of(text: str) -> float:
+        nonlocal judge_calls
+        with lock:
+            judge_calls += 1
+        return _judge_one(judge, problem, text)
+
+    # --- 1. greedy anchor first -------------------------------------------
+    greedy = gen(n=1, temperature=0.0)[0]
+    g_conf = judge_of(greedy.text)
+    pool = [(greedy.text, greedy.finished, g_conf, "greedy")]
+    if greedy.finished and g_conf >= stop_confidence:
+        return RaceResult(
+            answer=greedy.text, finished=True, confidence=g_conf,
+            thinker_tokens=thinker_tokens, judge_calls=judge_calls,
+            paths_submitted=1, paths_judged=1,
+            early_stopped=True, winner_source="greedy",
+            seconds=time.time() - t0,
+        )
+
+    # --- 2. sampled paths in parallel, judged as they finish ---------------
+    n_sampled = max(n_paths - 1, 0)
+    early = False
+    winner = None
+    # NOTE: not a `with` block on purpose. Exiting a ThreadPoolExecutor
+    # context manager WAITS for every worker, which would defeat the early
+    # stop. Instead we shut down without waiting: queued futures are
+    # cancelled, race() returns at once, and any thread already inside a
+    # blocking thinker.generate() finishes on its own in the background
+    # (its result is discarded). The thinker's generate() is MetaCog's
+    # library code and is deliberately not modified, so true GPU-side
+    # abort of an in-flight generation isn't available -- this is a
+    # stop-waiting cutoff, which is what cuts user-facing latency.
+    ex = ThreadPoolExecutor(max_workers=max(1, min(max_workers, n_sampled or 1)))
+    try:
+        futs = [ex.submit(gen, n=1, temperature=temperature)
+                for _ in range(n_sampled)]
+        for fut in as_completed(futs):
+            try:
+                g = fut.result()[0]
+            except Exception:
+                continue  # a dead path just never gets judged
+            conf = judge_of(g.text)
+            pool.append((g.text, g.finished, conf, "sample"))
+            if g.finished and conf >= stop_confidence:
+                winner = (g.text, True, conf, "sample")
+                early = True
+                break
+    finally:
+        ex.shutdown(wait=False, cancel_futures=True)
+
+    # --- 3. no early stop: best judged finished path wins -------------------
+    if winner is None:
+        finished = [p for p in pool if p[1]]
+        text, fin, conf, src = max(finished or pool, key=lambda p: p[2])
+        winner = (text, fin, conf, src)
+
+    text, fin, conf, src = winner
+    with lock:
+        toks, calls = thinker_tokens, judge_calls
+    return RaceResult(
+        answer=text, finished=fin, confidence=conf,
+        thinker_tokens=toks, judge_calls=calls,
+        paths_submitted=1 + n_sampled, paths_judged=len(pool),
+        early_stopped=early, winner_source=src,
+        seconds=time.time() - t0,
+    )

requirements.txt

@@ -0,0 +1,3 @@
+metacog @ git+https://github.com/ItIsCuthNotCup/MetaCog.git
+httpx>=0.27
+pydantic>=2.0

server.py

@@ -0,0 +1,159 @@
+#!/usr/bin/env python3
+"""Serve the bonsai + decider-4b MetaCog merge as ONE OpenAI-compatible model.
+
+    thinker: bonsai 27B (ternary) ............ THINKER_URL  (default http://localhost:8010)
+    judge:   decider-4b (/v1/systemone) ..... JUDGE_URL    (default http://localhost:8008)
+
+    POST /v1/chat/completions {"messages": [...]} -> judge-picked answer.
+
+Env: PORT (8200), THINKER_URL, THINKER_MODEL (bonsai), JUDGE_URL,
+     N_PATHS (8), NO_CASCADE, MAX_TOKENS (2048), TEMPERATURE (0.8),
+     STOP_CONFIDENCE (0.80), NO_RACE (unset).
+"""
+from __future__ import annotations
+
+import json
+import os
+import time
+import uuid
+
+import re
+
+def clean_answer(text: str) -> str:
+    """Strip <think>...</think> reasoning blocks the thinker may leak."""
+    return re.sub(r"<think>.*?</think>\s*", "", text, flags=re.S).strip()
+
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+from metacog import Config, MetaCog, OpenAICompatThinker, SystemOneJudge
+from race import race
+
+MODEL_ID = "heuristic-1"
+
+
+def env(name: str, default: str) -> str:
+    return os.environ.get(name, default)
+
+
+thinker = OpenAICompatThinker(
+    env("THINKER_URL", "http://localhost:8010"),
+    model=env("THINKER_MODEL", "bonsai"),
+)
+judge = SystemOneJudge(base_url=env("JUDGE_URL", "http://localhost:8008"))
+USE_RACE = env("NO_RACE", "") != "1"
+STOP_CONFIDENCE = float(env("STOP_CONFIDENCE", "0.80"))
+N_PATHS = int(env("N_PATHS", "8"))
+MAX_TOKENS = int(env("MAX_TOKENS", "2048"))
+TEMPERATURE = float(env("TEMPERATURE", "0.8"))
+MC = MetaCog(
+    thinker,
+    judge,
+    Config(
+        mode="best_of_n",
+        strategy="noul",
+        n_paths=N_PATHS,
+        greedy_anchor=True,
+        cascade_confidence=None if env("NO_CASCADE", "") == "1" else 0.95,
+        max_tokens=MAX_TOKENS,
+        temperature=TEMPERATURE,
+    ),
+)
+
+
+def problem_from(messages):
+    for m in reversed(messages):
+        if m.get("role") == "user":
+            c = m.get("content", "")
+            return c if isinstance(c, str) else json.dumps(c)
+    return "\n".join(f"{m.get('role')}: {m.get('content')}" for m in messages)
+
+
+class Handler(BaseHTTPRequestHandler):
+    server_version = "MergeServer/1.0"
+
+    def _send(self, code, obj):
+        body = json.dumps(obj).encode()
+        self.send_response(code)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def log_message(self, fmt, *args):
+        print(f"[{time.strftime('%H:%M:%S')}] {self.command} {self.path}", flush=True)
+
+    def do_GET(self):
+        path = self.path.rstrip("/")
+        if path in ("/health", "/v1/health", "/v1/models"):
+            self._send(200, {"status": "ok", "model": MODEL_ID})
+        else:
+            self._send(404, {"error": "not found"})
+
+    def do_POST(self):
+        path = self.path.rstrip("/")
+        if path not in ("/v1/chat/completions", "/chat/completions"):
+            return self._send(404, {"error": "not found"})
+        try:
+            raw = self.rfile.read(int(self.headers.get("Content-Length", 0))) or b"{}"
+            body = json.loads(raw)
+        except Exception as e:
+            return self._send(400, {"error": f"bad JSON: {e}"})
+        messages = body.get("messages", [])
+        if not messages:
+            return self._send(400, {"error": "messages required"})
+        t0 = time.time()
+        try:
+            if USE_RACE:
+                r = race(problem_from(messages), thinker, judge,
+                         n_paths=N_PATHS, stop_confidence=STOP_CONFIDENCE,
+                         max_tokens=MAX_TOKENS, temperature=TEMPERATURE)
+                answer, finished = clean_answer(r.answer), r.finished
+                conf, toks, dt = r.confidence, r.thinker_tokens, r.seconds
+            else:
+                result = MC.run(problem_from(messages))
+                dt = time.time() - t0
+                rounds = result.trace.rounds
+                conf = float(rounds[-1].verdict.confidence) if rounds else 0.0
+                answer, finished, toks = (clean_answer(result.answer),
+                                          result.finished,
+                                          result.trace.thinker_tokens)
+        except Exception as e:
+            return self._send(502, {"error": f"pipeline failed: {e}"})
+        print(f"[{time.strftime('%H:%M:%S')}] conf={conf:.2f} {dt:.1f}s", flush=True)
+        self._send(
+            200,
+            {
+                "id": f"merge-{uuid.uuid4().hex[:12]}",
+                "object": "chat.completion",
+                "created": int(time.time()),
+                "model": body.get("model", MODEL_ID),
+                "choices": [
+                    {
+                        "index": 0,
+                        "message": {"role": "assistant", "content": answer},
+                        "finish_reason": "stop" if finished else "length",
+                    }
+                ],
+                "usage": {
+                    "prompt_tokens": 0,
+                    "completion_tokens": toks,
+                    "total_tokens": toks,
+                },
+                "merge": {
+                    "thinker": "bonsai",
+                    "judge": "decider-4b",
+                    "judge_confidence": round(conf, 4),
+                    "seconds": round(dt, 2),
+                },
+            },
+        )
+
+
+def main() -> None:
+    port = int(env("PORT", "8200"))
+    print(f"{MODEL_ID} on :{port}", flush=True)
+    ThreadingHTTPServer(("0.0.0.0", port), Handler).serve_forever()
+
+
+if __name__ == "__main__":
+    main()