README.md

---
license: mit
base_model: deepgrove/maple-preview
base_model_relation: quantized
language: en
pipeline_tag: text-generation
tags:
  - hipfire
  - mixture-of-experts
  - reasoning
  - ternary
  - rdna
  - rocm
---

# Maple-Preview — hipfire HFQ (arch 15)

A [hipfire](https://github.com/warpfront/hipfire) build of
[`deepgrove/maple-preview`](https://huggingface.co/deepgrove/maple-preview),
a 20B-A1B natively-ternary reasoning MoE, for AMD RDNA GPUs.

| | |
|---|---|
| File | `maple-preview.mq2lloydu` |
| Size | 6,499,340,288 bytes (6.05 GiB) |
| sha256 | `7fb52fe72c1a0a4455d0fe3a8109b0df66fa53782f41d8b257140d3e966645db` |
| arch_id | 15 |
| Body carrier | `MQ2G256LloydU` (qt=51), 2.250 bpw |
| lm_head | Q8 |
| Context | 131,072 |

```bash
hipfire run maple-preview "Prove that sqrt(2) is irrational."
```

## The body is a packing, not a quantization

Every linear weight in Maple-Preview is already exactly `{-s, 0, +s}` with one
bf16 scale per output row. `MQ2G256LloydU` (qt=51) is an **unrotated** MQ2-Lloyd
carrier built for this: it stores those three levels exactly.

Verified end-to-end against the **original** `deepgrove/maple-preview`
safetensors — not against our own dequantizer, which could only confirm a shared
misunderstanding:

```
model.layers.0.mlp.experts.0.gate_proj.weight   max|err| = 0
model.layers.0.mlp.experts.0.down_proj.weight   max|err| = 0
model.layers.0.self_attn.q_proj.weight          max|err| = 0
model.layers.0.self_attn.o_proj.weight          max|err| = 0
```

Every bitwise difference is a signed zero (`-0.0` → `+0.0`), which is
numerically irrelevant. The converter *refuses* a non-ternary row rather than
falling back to lossy quantization, so a clean convert is itself evidence the
property holds across all 18,528 ternary tensors (19,579,011,072 weights).

The accuracy this model loses relative to full precision therefore comes from
the runtime, not the weights.

## Measured quality

Against a BF16 reference — `deepgrove/maple-preview` converted to a BF16 GGUF
and scored with llama.cpp on CPU — over 2,048 teacher-forced wikitext tokens,
identical tokens and protocol on both sides:

| KV tier | lm_head | mean KL | top-1 agreement |
|---|---|---|---|
| **bf16** (default) | **Q8** | **0.0511** | **91.9%** |
| q8 | Q8 | 0.0842 | 90.8% |

The lm_head is free: a bf16 head gives the *identical* 0.0511, so Q8 costs
nothing and decodes 23% faster.

**Interpreting 0.0511.** It is close to the floor for this model rather than a
defect. llama.cpp rounds activations to bf16 before every matmul
(`vec_dot_type = GGML_TYPE_BF16`, ε = 2⁻⁸), so the reference is itself
precision-limited: an F16 reference built from the same weights differs from the
BF16 reference by **mean KL 0.0357** — the same order as our entire divergence
from it — and this build sits *closer* to the more precise reference (0.0477)
than to the bf16 one (0.0511). Maple is unusually sensitive to activation
precision, plausibly because ternary weights and 256-expert top-8 routing make
expert selection easy to perturb.

At position 0, where attention has no history to read, KL is **1.2e-4**.

## Performance and the lm_head

gfx1151 (Radeon 8060S / Strix Halo), bf16 KV, greedy, 1,500 tokens, 3 paired
interleaved repetitions. KL is against the BF16 reference on the same 2,048
tokens as above.

| lm_head | bpw | mean KL | top-1 | decode |
|---|---|---|---|---|
| **q8** (shipped in the base) | 8.5 | **0.0511** | **91.9%** | ~145 tok/s |
| q4k (overlay) | 4.5 | 0.0640 | 90.1% | ~168 tok/s |
| bf16 (overlay) | 16 | 0.0511 | 91.7% | ~118 tok/s |

Prefill is roughly 800 tok/s on short prompts.

**q8 is the default because bf16 is strictly dominated** — identical KL, 23%
slower. A bf16 head costs throughput and buys exactly zero accuracy.

**q4k is offered for throughput**, not recommended by default: +16% decode for
+25% mean KL and −1.8pp top-1. Note DeepGrove *do* ship a Q4_K head in their own
llama.cpp example, and that is the right call there — on their CPU path the same
swap buys 49% (252.7 vs 169.8 tok/s) rather than 16%, because the MoE body
dominates decode differently on a GPU. Same decision, different machine.

## Head variants ship as overlays, not as separate models

The head is 2.7% of the file, so shipping a whole model per carrier would
duplicate the identical 6.17 GB body every time. Instead the base carries the
recommended q8 head and runs standalone, and alternative carriers are
single-tensor `.hfq` overlays that shadow `lm_head.weight` at load:

| file | size |
|---|---|
| `maple-preview.mq2lloydu` (base, q8 head) | 6.50 GB |
| `maple-head-q4k.hfq` | 188 MB |
| `maple-head-bf16.hfq` | 635 MB |

```bash
hipfire run maple-preview --head q4k "..."     # by registry name
hipfire run <model.hfq> --head <head.hfq> ""   # or by path
```

Each overlay is a normal `.hfq` carrying a single tensor at the same logical
shape as the base's head, which is what lets hipfire attach it at load time and
shadow `lm_head.weight`. They are listed in the registry entry under `heads`,
with their own sha256, and are rebuilt with:

```bash
hipfire-quantize --format maple --input <safetensors dir> \
  --output maple-head-q4k.hfq --head-quant q4k --head-only
```

An overlay is not an approximation of a full build — it reproduces one exactly
(q4k overlay 0.0640 KL / 90.1% top-1; a monolithic q4k build measures the same).
Three full variants would be 19.63 GB; this is 7.30 GB, and switching heads
costs 188–635 MB rather than a 6.5 GB re-download.

## Recommended settings

Use the **vendor's** values — they are not the Qwen defaults, despite the shared
tokenizer:

```
temperature 1.0
top_p       0.95
```

Source: DeepGrove's own llama.cpp fork,
[`github.com/deepgrove-ai/llama.cpp`](https://github.com/deepgrove-ai/llama.cpp),
which `deepgrove/maple-preview-GGUF` links to as the official setup. There is no
`generation_config.json` upstream and the model card names no sampler, so that
README is the only first-party source. `top_k` / `min_p` values circulating
elsewhere are **not** from DeepGrove.

**The chat template matters.** Maple's embedded template ends its generation
prompt with `<|im_start|>assistant\n<think>\n` — including the thinking prefix.
Omitting it makes the model open its own reasoning block, starting every
generation off-distribution *inside* the block where degenerate loops occur. On
a fixed prompt across 8 seeds, the loop rate went from **5/8 to 0/8** once the
prefix and temperature were corrected. hipfire applies the template from the
HFQ metadata automatically.

## KV cache

Defaults to **bf16**, overridable with `--kv-mode q8`.

Maple is 3:1 sliding-window(512) / global-NoPE attention. Q8 KV costs 39% of the
measured divergence, and the damage is concentrated in the **tail** rather than
spread as uniform blur — the median position moves only 24%, but the worst goes
10.36 → 4.21 nats. Rare catastrophic positions are what derail long generations.

bf16 KV costs 1.88× the KV bytes (26,112 → 49,152 B/token; +2.81 GiB at 131k
context) and ~2% decode. Use `--kv-mode q8` to trade that fidelity back for
memory.

## Limitations

These are the **model's**, not the port's — see the
[upstream card](https://huggingface.co/deepgrove/maple-preview):

> This preview received minimal post-training for agentic tasks and only
> small-scale general reinforcement learning.

Wikitext perplexity is high (~57 on the standard protocol) where a comparable
peer scores ~7.7. That is the model, not the packing: KL against a
full-precision reference is 0.05, and DeepGrove's own ternary GGUF scores within
0.2% of a BF16 run of the same weights. Maple is not *uncertain* on raw prose —
its entropy matches its peer — it is confidently wrong on it, which is
consistent with a reasoning-focused preview.

**Give it room to think.** Maple spends a large share of its budget reasoning,
and a short cap truncates it mid-derivation in a way that reads like a
repetition loop but is not one. On a five-house logic puzzle, 8/8 sampled runs
hit a 12,000-token cap without converging — while at 40,000 tokens they
terminated cleanly with the correct answer after ~84–134 KB of reasoning. If
output looks stuck, raise `max_tokens` before suspecting the model.

Most of what otherwise looked like intrinsic looping was configuration: with
the vendor template and temperature above, the loop rate on a fixed prompt went
from 5/8 to 0/8 across 8 seeds.

## Provenance

MIT, inherited from `deepgrove/maple-preview`. Weights are DeepGrove's,
carried into HFQ losslessly. Converted with:

```bash
hipfire-quantize --format maple \
  --input <maple-preview safetensors dir> \
  --output maple-preview.mq2lloydu \
  --head-quant q8
```