Upload README.md with huggingface_hub
21f49a6 verified 4 days ago
20.8 kB
---
license: apache-2.0
tags:
  - benchmark
  - agentic-coding
  - quantization
  - apex
  - gguf
  - moe
language:
  - en
---
# APEX quant benchmarks — 42-task agentic coding suite

Results for the GGUF models I quantize and benchmark, measured on a real agentic coding
loop rather than a single-completion prompt. Every row below is a full **42-task** run
across all four suites.

This is a report, not a model. The quants themselves live in the individual
`Myric/*-APEX-GGUF` repositories, and the table links each row to the **actual model
tested** where a public card exists (otherwise to the base/official card — see the note
under the table).

## The benchmark

**42 tasks in four suites** (private benchmark harness):

| suite | tasks | what it is |
|---|---:|---|
| `opencode_tasks` | 25 | general algorithmic ladder: data structures, graphs, DP, streams |
| `opencode_tasks_hard` | 5 | fix/extend an existing codebase |
| `opencode_tasks_frontier` | 9 | the hard-coder frontier (regex, btree, SQL, CRDT, rate limiters, txns) |
| `opencode_tasks_tier4a` | 3 | multi-file repair in place (SWE-bench-style), not a harder rung |

**How a run works.** The model is served locally through `llama-server` and driven by
OpenCode's real tool-use loop (bash / read / write / edit) against each task's `PROMPT.md`.
The code it writes is graded by running the task's own `pytest test_solution.py` **fresh** —
the model's self-report is never trusted. Scoring is **partial credit**: the fraction of
individual pytest items passed, accumulated per suite.

**Fixed conditions.** `--parallel 1` (a shared KV pool with concurrent OpenCode requests is
not comparable); compaction **off**; context auto-sized to the largest that fits up to
262,144; per-family output cap above worst-case reasoning; per-task wall 3,600 s; sampling at
the model card's recommended temperature/top-p.

**Boxes.** *GB10* = NVIDIA GB10 (DGX Spark-class), ~119 GiB unified memory. *Thor* = a 2013
dual-Xeon workstation with 128 GB DDR3 and a 16 GB RTX 4060. The box is noted per row where it
matters; wall-clock is only comparable within a box.

**A perfect score is the entry bar, not a finding.** These suites are designed to saturate
for a competent coder. A full 42/42 says the arm *qualified for comparison*; it is not
evidence that anything had no effect. The measurement lives in **which suite a model falls
off at**, and — once pass/fail saturates — in **output tokens at equal score**. Code quality
is not captured by this harness.

**Noise.** Seven of the nine frontier tasks can flip between identical repeats, and
within-arm token spread runs to double digits. The table shows the **best run** per row and
the **Reps** column gives the number of runs and the total range. A single-run row is a data
point, not a noise floor. Tokens compare within a model, never across models of different
scale.

### Time is part of the measurement

A full 42-task run is typically **~2 hours** on the GB10. Across the 74 full runs here the
median is **1.8 h**, p90 **4.9 h**, and the slowest outlier is **14.5 h** (Spark-X2.5-4B BF16,
10 voids — each timeout can burn the full 3,600 s per-task wall). A **12-hour outer budget** is
therefore a real bound rather than a hypothetical: most runs never approach it, but a slow or
verbose model that keeps timing out can exhaust it. A model that cannot finish inside that
budget is, for this box, **too slow to be useful** — the benchmark is meant to answer the
deployment question, not the abstract-capability question.

So a missing perfect score is not always a missing capability:

- A task that hits the wall or the output cap is **voided**, not marked wrong. The model may
  have been on the right track and simply not finished. That is why `Valid = 42 − void` is the
  column to read, and why a row like Spark-X2.6-4B is 39/39 valid despite a 39/42 raw total.
- But a void is also **not evidence of capability**. A model that voids the frontier tasks has
  not demonstrated that it can do them under the budget; a 100% valid ratio on a shrunken
  denominator is not the same as a clean 42/42. The clean rows are the stronger result.
- A smarter-but-slower model could move up on faster hardware or with a longer budget. The table
  measures capability **under this GB10's time budget**, not capability in the abstract. For a
  local deployment, the budget is part of the requirement.

This is why the frontier column and the `Void` count sit next to the score: they separate "did
not finish" from "got it wrong", and neither is silently folded into the other.

## Full 42-task results

`G` = general (25), `H` = hard (5), `F` = frontier (9), `T4A` = tier4a (3). Per-suite and
total columns are the **best run** (most valid tasks passed); `Void` is that run's harness-limit
count and `Reps` gives run count and the total-passed range. `Tok/task` is output tokens per
valid task (`tokens_out / valid`). Three throughput columns are reported, all from
`llama.cpp` server timings summed across the run:

  * `Prefill` — prompt processing rate (`prompt_tokens / prompt_ms`)
  * `Decode` — raw generation rate (`gen_tokens / eval_ms`), what `llama-bench` measures
  * `Eff.` — end-to-end task throughput (`tokens_out / wall_s`), including prefill, tool
    execution, `pytest` grading, and idle. Use `Prefill`/`Decode` to compare the model
    and `Eff.` to compare the deployment.
**A void is not a failure.** It is a timeout, output-cap truncation, or no-completion that the
harness refuses to score (`n_valid = 42 − void`). The `Valid` column is therefore
`passed / (42 − void)`, and it is the number to read. The 42-slot denominator is kept only so
the void count is visible. This is not academic: **Spark-X2.6-4B is 39/39 valid (3 void),
not 39/42**; Spark-X2.5 BF16 is 32/32 with 10 void; MiniCPM5-2B is 35/35 with 7 void. Reporting
those as 39/42, 32/42, or 35/42 would count harness limits as model failures.

<!-- TABLE:START -->
| Model | Quant | G (25) | H (5) | F (9) | T4A (3) | Valid | Void | Tok/task | Prefill | Decode | Eff. | Reps |
|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|
| [Muse-Glimmer-30B](https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF) | UD-Q4_K_XL | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 5,021 | 723.3 | 12.9 | 9.3 | 1 |
| [K2-Horizon-MoVA-36B-A4B](https://huggingface.co/Myric/K2-Horizon-MoVA-36B-A4B-APEX-GGUF) | APEX i-quality v2 | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 2,597 | — | — | 6.2 | 1 |
| [Kolibri-1 78B-A3.5B](https://huggingface.co/Myric/Kolibri-1-APEX-GGUF) | APEX 2.4 bpw | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 8,466 | 1585.9 | 57.4 | 44.7 | 1 |
| [Kolibri-1 78B-A3.5B](https://huggingface.co/Myric/Kolibri-1-APEX-GGUF) | Q4_K uniform | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 8,932 | 1540.5 | 51.0 | 41.1 | 2 (42) |
| [Laguna-XS-2.1](https://huggingface.co/Myric/Laguna-XS-2.1-APEX-GGUF) | APEX i-quality v2 | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 4,180 | 1510.7 | 60.3 | 47.3 | 2 (42) |
| [Laguna-XS-2.1](https://huggingface.co/Myric/Laguna-XS-2.1-APEX-GGUF) | APEX dynamic v2 | 25/25 | 5/5 | 8/9 | 3/3 | **41/42** | 0 | 7,174 | 1182.5 | 44.9 | 40.0 | 2 (41) |
| [Laguna-S-2.1](https://huggingface.co/Myric/Laguna-S-2.1-APEX-GGUF) | APEX i-quality v2 | 25/25 | 5/5 | 7/7 | 3/3 | **40/40** | 2 | 5,558 | 631.0 | 27.0 | 19.6 | 1 |
| [Nemotron-3-Super-120B-A12B](https://huggingface.co/Myric/Nemotron-3-Super-120B-A12B-APEX-GGUF) | APEX i-quality | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 3,943 | 490.5 | 17.8 | 13.5 | 1 |
| [Nemotron-3.5-Super-VL-120B-A12B](https://huggingface.co/Myric/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-APEX-GGUF) | APEX i-quality | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 2,856 | 456.8 | 17.2 | 10.9 | 1 |
| [Ornith-1.5-122B](https://huggingface.co/ornith-ai/Ornith-1.5-122B) | APEX i-quality | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 3,206 | 469.0 | 24.4 | 20.1 | 2 (42) |
| [Ornith-1.5-122B](https://huggingface.co/ornith-ai/Ornith-1.5-122B) | APEX i-compact | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 3,148 | 521.4 | 26.2 | 20.6 | 2 (42) |
| [Ternary-Bonsai-2-27B](https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf) | PQ2_0 (2.13 bpw) | 25/25 | 5/5 | 8/8 | 3/3 | **41/41** | 1 | 11,578 | 797.5 | 24.9 | 21.6 | 1 |
| [Gemma-4-12B-it QAT](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf) | Q4_0 (grid) | 24/25 | 5/5 | 9/9 | 3/3 | **41/42** | 0 | 5,931 | 520.2 | 17.3 | 13.5 | 1 |
| [Gemma-4-12B-it QAT](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-gguf) | Q4_0 (Google) | 24/25 | 5/5 | 8/8 | 3/3 | **40/41** | 1 | 7,621 | 519.7 | 16.4 | 12.5 | 2 (39–40) |
| [Mellum2.1-12B-A2.5B-Thinking](https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF) | Q8_0 | 25/25 | 5/5 | 8/8 | 3/3 | **41/41** | 1 | 5,029 | 2536.5 | 77.0 | 51.8 | 2 (40–41) |
| [Mellum2.1-12B-A2.5B-Thinking](https://huggingface.co/Myric/Mellum2.1-12B-A2.5B-Thinking-APEX-GGUF) | APEX 4.9 bpw | 25/25 | 5/5 | 8/9 | 3/3 | **41/42** | 0 | 6,782 | — | — | 71.7 | 2 (41) |
| [Mellum2.1-12B-A2.5B-Thinking](https://huggingface.co/Myric/Mellum2.1-12B-A2.5B-Thinking-APEX-GGUF) | APEX 4.3 bpw | 25/25 | 5/5 | 8/9 | 3/3 | **41/42** | 0 | 6,356 | — | — | 73.8 | 2 (40–41) |
| [Mellum2.1-12B-A2.5B-Thinking](https://huggingface.co/Myric/Mellum2.1-12B-A2.5B-Thinking-APEX-GGUF) | APEX 3.7 bpw | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 5,992 | — | — | 83.0 | 2 (41–42) |
| [Mellum2.1-12B-A2.5B-Thinking](https://huggingface.co/Myric/Mellum2.1-12B-A2.5B-Thinking-APEX-GGUF) | APEX 3.1 bpw | 25/25 | 5/5 | 7/9 | 3/3 | **40/42** | 0 | 10,729 | — | — | 93.8 | 2 (36–40) |
| [Mellum2.1-12B-A2.5B-Thinking](https://huggingface.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF) | Q4_K_M stock | 25/25 | 5/5 | 8/9 | 3/3 | **41/42** | 0 | 5,920 | — | — | 68.9 | 2 (40–41) |
| [Mellum2.1-12B-A2.5B-Thinking](https://huggingface.co/bartowski/Mellum2.1-12B-A2.5B-Thinking-GGUF) | Q6_K_L bartowski | 25/25 | 5/5 | 8/9 | 3/3 | **41/42** | 0 | 5,397 | — | — | 57.6 | 2 (41) |
| [GPT-OSS-20B](https://huggingface.co/ggml-org/gpt-oss-20b-GGUF) | MXFP4 (Thor) | 25/25 | 5/5 | 8/9 | 3/3 | **41/42** | 0 | 7,653 | — | — | 50.3 | 2 (40–41) |
| [Nemotron-3.5-Lightning-30B-A3B](https://huggingface.co/Myric/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-APEX-GGUF) | APEX compact (Thor) | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 13,211 | — | — | 34.9 | 2 (40–42) |
| [Gemma-4-26B-A4B](https://huggingface.co/google/gemma-4-26B-A4B-it-qat-q4_0-gguf) | Q4_0 QAT (Thor) | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 6,000 | — | — | 30.3 | 2 (41–42) |
| [Kolibri-1 78B-A3.5B](https://huggingface.co/Myric/Kolibri-1-APEX-GGUF) | APEX i-quality high (Thor) | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 8,269 | — | — | 15.8 | 1 |
| [Kolibri-1 78B-A3.5B](https://huggingface.co/Myric/Kolibri-1-APEX-GGUF) | APEX i-quality low (Thor) | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 5,600 | — | — | 15.7 | 1 |
| [LFM2.5-24B-A2B](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF) | QAD Q4_0 | 25/25 | 5/5 | 7/9 | 3/3 | **40/42** | 0 | 12,442 | 2863.3 | 90.3 | 73.9 | 4 (37–40) |
| [LFM2.5-24B-A2B](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF) | Q8_0 | 25/25 | 3/3 | 8/8 | 3/3 | **39/39** | 3 | 5,875 | 1964.4 | 69.0 | 38.7 | 1 |
| [LFM2.5-24B-A2B](https://huggingface.co/LiquidAI/LFM2.5-24B-A2B) | IQ3_XXS (BF16 src) | 25/25 | 4/4 | 7/9 | 3/3 | **39/41** | 1 | 11,937 | 2344.9 | 112.0 | 78.4 | 2 (36–39) |
| [Nemotron-3.5-Bolt-74B-A9B](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF) | 24 GB quant | 24/24 | 5/5 | 7/7 | 3/3 | **39/39** | 3 | 4,214 | 1059.2 | 38.5 | 16.7 | 1 |
| [Spark-X2.6-4B](https://huggingface.co/Myric/Spark-X2.6-4B-GGUF) | Q6_K | 25/25 | 4/4 | 7/7 | 3/3 | **39/39** | 3 | 5,760 | — | — | 25.0 | 1 |
| [Spark-X2.5-4B](https://huggingface.co/Myric/Spark-X2.5-4B-GGUF) | BF16 | 22/22 | 3/3 | 4/4 | 3/3 | **32/32** | 10 | 17,953 | 2193.1 | 18.5 | 11.0 | 1 |
| [Spark-X2.5-4B](https://huggingface.co/Myric/Spark-X2.5-4B-GGUF) | Q6_K | 22/22 | 3/3 | 2/2 | 3/3 | **30/30** | 12 | 12,916 | — | — | 16.2 | 1 |
| [MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF) | Q8_0 | 23/23 | 3/3 | 6/6 | 3/3 | **35/35** | 7 | 11,148 | 6076.0 | 64.9 | 57.1 | 1 |
| [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B-GGUF) | Q4_0 | 23/24 | 3/4 | 4/7 | 2/3 | **32/38** | 4 | 18,431 | 8632.1 | 135.3 | 85.3 | 1 |
| [Flint](https://huggingface.co/Myric/Flint-APEX-GGUF) | APEX 5bpw | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 4,397 | 1564.3 | 53.8 | 40.1 | 1 |
| [Qwen3.8-Flash-Next](https://huggingface.co/Myric/Qwen3.8-Flash-Next-APEX-GGUF) | APEX MIDDLE | 24/24 | 5/5 | 9/9 | 3/3 | **41/41** | 1 | 5,970 | 439.1 | 25.9 | 16.1 | 1 |
| [Ornith-1.5-35B-A3B](https://huggingface.co/Myric/Ornith-1.5-35B-A3B-APEX-GGUF) | APEX i-quality | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 5,028 | 1680.6 | 64.5 | 50.8 | 1 |
| [GLM-4.7-Flash](https://huggingface.co/Myric/GLM-4.7-Flash-APEX-GGUF) | APEX i-quality | 24/25 | 3/5 | 6/9 | 3/3 | **36/42** | 0 | 10,226 | 656.1 | 40.8 | 28.2 | 1 |
| [Muse-Glimmer-30B-heretic](https://huggingface.co/darkc0de/Muse-Glimmer-30B-heretic) | Q4_K_M | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 4,895 | 704.0 | 11.9 | 8.8 | 1 |
| [Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) | Q4_K_M | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 2,236 | 612.1 | 11.6 | 9.6 | 1 |
| [Ripple-65B-Preview](https://huggingface.co/Myric/Ripple-65B-Preview-APEX-GGUF) | Q4_K_M + DFlash2 (n16) | 25/25 | 5/5 | 9/9 | 3/3 | **42/42** | 0 | 2,188 | 281.5 | 12.7 | 9.5 | 1 |
<!-- TABLE:END -->
Every row links to the public card for the exact quant where one exists; where the tested GGUF is
local-only (the Gemma grid variant, Ornith's APEX tiers before publication, LFM's BF16-source
IQ3_XXS), the link goes to the base/official card and the Quant column names the variant.

### What the fall-off says

With voids excluded, the picture is cleaner — and different from the raw denominators:

- **Clean 42/42, no voids:** Muse-Glimmer-30B, K2-Horizon v2, Kolibri-1 (APEX and uniform),
  Laguna-XS i-quality v2, Nemotron-3-Super-120B, Nemotron-3.5-Super-VL-120B-A12B,
  Ornith-1.5-122B (i-quality and i-compact).
  Their comparison is tokens-at-equal-score, not correctness.
- **Perfect on every valid task, via voids:** Spark-X2.6-4B (39/39, 3 void), Spark-X2.5 at
  BF16 and Q6_K (32/32 and 30/30; 10 and 12 void), MiniCPM5-2B (35/35, 7 void), LFM2.5 Q8_0
  (39/39, 3 void), Nemotron-3.5-Bolt-74B (39/39, 3 void), Laguna-S (40/40, 2 void), and
  **Bonsai 2 27B (41/41, 1 void)**. The voids are the output cap or the wall on the longest
  tasks; the write-ups note no degeneration. These are not capability losses. Bonsai's one
  void is the "smart but too slow" case: **56.8k output tokens and the full 3600 s wall,
  still unfinished** — not a wrong answer.
- **Real frontier misses (1–2 valid tasks):** Laguna-XS dynamic v2 41/42, Mellum2.1 APEX 4.9 /
  4.3 / stock Q4_K_M / bartowski Q6_K_L all 41/42, Mellum2.1 APEX 3.1 40/42, LFM2.5 QAD Q4_0
  40/42, LFM2.5 IQ3_XXS 39/41. This is the scale doing its job: the frontier tier is where the
  quant or the smaller model gives up first.
- **General-suite miss:** Gemma-4-12B QAT grid 41/42 (one general task). The Google QAT arm is
  40/41 on its best-passed rep (one general miss, one void); its second rep is 39/39 with 3 void.
  The general-suite miss is the one real failure; the void is a harness limit.
- **Mellum2.1 APEX 3.7 bpw reaches 42/42** on its best rep (41/42 on the other), while 3.1 bpw
  falls to 40/42 — the aggressive tier is where the quant starts to cost correctness, and it
  shows up in the frontier column.

## Plots

Measured wall-clock from the full 42-task runs. Time plots use **GB10 runs only** — Thor
wall-clock is not comparable to the GB10. Each point is the best valid run for that
model/quant. Parameter counts are nominal from the model cards/configs; Ornith-1.5-122B's
active count is derived from its 256-expert top-8 config (~12 B).

**Wall clock vs active parameters.** Active count alone explains little: the 4 B Spark pair
spans 2.5–14.5 h while the 12 B-active 120–124 B MoEs finish in 1.8–3.4 h.

![Wall clock vs active parameters](plots/wall_vs_active.png)

**Wall clock vs active parameters — perfect-score runs, by model origin.** Filtered to runs
that passed every valid task (`passed == valid`); voids are allowed, and their wall time is
still included because this is elapsed wall clock, not valid-task time. **Fill = pretrained-base
origin; edge = post-training/compression origin.** Blue = US, red = China, amber = Germany,
purple = Czechia, green = UAE. The origin plots omit the Spark-X2.5 14.5 h BF16 outlier to keep
the vertical scale readable. Bonsai 27B and Ornith-1.5-122B show red fill / blue edge: Chinese
pretrained bases, US post-processing. `Muse-Glimmer-30B-heretic` has a US/Meta base (blue
fill) and a gray edge labelled **Abliterated**, because its Heretic abliteration publisher's origin is undocumented.

![Wall clock vs active parameters, perfect-score runs by country](plots/wall_vs_active_perfect_country.png)

**Wall clock vs total parameters.** Total size is not the driver either: a 124 B model can
finish in 1.75 h while a 4 B model takes 14.5 h.

![Wall clock vs total parameters](plots/wall_vs_total.png)

**Wall clock vs total parameters — perfect-score runs, by model origin.** Same filter, fill/edge encoding, and color key as the active-parameter version (Spark-X2.5
omitted); wall time still includes voided tasks.

![Wall clock vs total parameters, perfect-score runs by country](plots/wall_vs_total_perfect_country.png)

**Wall clock vs output tokens.** The clearest single driver: the runs that spend more output
tokens take longer. The Spark outliers are the high-token runs, not the large models.

![Wall clock vs output tokens](plots/wall_vs_tokens.png)

**Verbosity vs active parameters.** Output tokens per valid task; circles are GB10, squares
Thor. Larger active models are not uniformly more verbose, and the 4 B Spark BF16 run is the
outlier at ~18 k tokens/task.

![Verbosity vs active parameters](plots/tokens_per_task_vs_active.png)

**Performance per byte.** Capability vs quant file size (left; top-left is best) and end-to-end
output tokens/s vs size (right; GB10 only — throughput is box-specific). The ternary Bonsai 27B
PQ2_0 is the efficiency outlier: **41/41 valid at 7.21 GB**, with one void
(`btree_insert_delete` — 3600 s / 32k cap). Full capability on every valid task, at the cost of
raw decode speed from the custom ternary kernels.
![Performance per byte](plots/perf_per_byte.png)
## Also measured (subset runs)
Not comparable to the 42-task table above — these ran one suite, or fewer tasks. Kept here so
the frontier signal is not lost.
| Model | Quant | Suite | Score | Note |
|---|---|---|---:|---|
| Qwen3.8-Flash-Next | APEX (several arms) | frontier | 9/9 | one arm 8/9 |
| K2-Horizon-MoVA-36B-A4B | APEX compact / mini | frontier | 9/9 | both tiers |
| K2-Horizon-MoVA-36B-A4B | APEX i-quality | frontier | 8/9 GB10, 9/9 Thor | box difference |
| Ornith-1.5-35B-A3B | APEX i-quality | frontier | 9/9 | — |
| Nemotron-3.5-Lightning-30B-A3B | APEX i-quality | frontier | 7/9 | — |
| Spark-X2.5-4B | Q5 (1M-context) | general | 10/13 | long-context variant |
| Spark-X2.5-4B | Q4 | hard | 3/5 | — |
| Solar-Open2-148B-A15B REAP | Q4_K_M | tier4a | 2/3 (25/27, 91.7%) | 131k ctx, low reasoning, Solar proxy parser |
## Quantization note
APEX tiers are **per-tensor allocations**, not fixed recipes: the edge/near/middle bands, the
shared-expert and latent-bottleneck protection, and the F32 state/norm/router pins all change
per architecture. `i-quality` is a shelf label for the tier I reach for by default, not a
claim of identity with another project's recipe. Sizes and bpw live in each model repo.
## Files
| path | contents |
|---|---|
| `results.json` | every full-42 and subset run, per-suite passed/valid/partial, model path, box, void/wall |
| `results_full42.csv` | flat table of the 74 full-42 runs, one row per run |
| `plots/` | wall-clock vs parameters/tokens, verbosity, performance-per-byte, and country-colored perfect-score plots |

## Caveats

- **Pass/fail is a gate.** This harness saturates by design; a perfect score is not a ranking.
- **Code quality is unmeasured.** Tests-passed and tokens are what is recorded; clarity,
  structure, and whether the model understood the problem are not.
- **Tokens are within-model.** Cross-model token comparisons confound capability with
  verbosity and are not reported.
- **Single runs are noisy.** Where only one run exists, treat the score as a data point.