--- license: mit base_model: deepgrove/maple-preview base_model_relation: quantized language: en pipeline_tag: text-generation tags: - hipfire - mixture-of-experts - reasoning - ternary - rdna - rocm --- # Maple-Preview — hipfire HFQ (arch 15) A [hipfire](https://github.com/warpfront/hipfire) build of [`deepgrove/maple-preview`](https://huggingface.co/deepgrove/maple-preview), a 20B-A1B natively-ternary reasoning MoE, for AMD RDNA GPUs. | | | |---|---| | File | `maple-preview.mq2lloydu` | | Size | 6,499,340,288 bytes (6.05 GiB) | | sha256 | `7fb52fe72c1a0a4455d0fe3a8109b0df66fa53782f41d8b257140d3e966645db` | | arch_id | 15 | | Body carrier | `MQ2G256LloydU` (qt=51), 2.250 bpw | | lm_head | Q8 | | Context | 131,072 | ```bash hipfire run maple-preview "Prove that sqrt(2) is irrational." ``` ## The body is a packing, not a quantization Every linear weight in Maple-Preview is already exactly `{-s, 0, +s}` with one bf16 scale per output row. `MQ2G256LloydU` (qt=51) is an **unrotated** MQ2-Lloyd carrier built for this: it stores those three levels exactly. Verified end-to-end against the **original** `deepgrove/maple-preview` safetensors — not against our own dequantizer, which could only confirm a shared misunderstanding: ``` model.layers.0.mlp.experts.0.gate_proj.weight max|err| = 0 model.layers.0.mlp.experts.0.down_proj.weight max|err| = 0 model.layers.0.self_attn.q_proj.weight max|err| = 0 model.layers.0.self_attn.o_proj.weight max|err| = 0 ``` Every bitwise difference is a signed zero (`-0.0` → `+0.0`), which is numerically irrelevant. The converter *refuses* a non-ternary row rather than falling back to lossy quantization, so a clean convert is itself evidence the property holds across all 18,528 ternary tensors (19,579,011,072 weights). The accuracy this model loses relative to full precision therefore comes from the runtime, not the weights. ## Measured quality Against a BF16 reference — `deepgrove/maple-preview` converted to a BF16 GGUF and scored with llama.cpp on CPU — over 2,048 teacher-forced wikitext tokens, identical tokens and protocol on both sides: | KV tier | lm_head | mean KL | top-1 agreement | |---|---|---|---| | **bf16** (default) | **Q8** | **0.0511** | **91.9%** | | q8 | Q8 | 0.0842 | 90.8% | The lm_head is free: a bf16 head gives the *identical* 0.0511, so Q8 costs nothing and decodes 23% faster. **Interpreting 0.0511.** It is close to the floor for this model rather than a defect. llama.cpp rounds activations to bf16 before every matmul (`vec_dot_type = GGML_TYPE_BF16`, ε = 2⁻⁸), so the reference is itself precision-limited: an F16 reference built from the same weights differs from the BF16 reference by **mean KL 0.0357** — the same order as our entire divergence from it — and this build sits *closer* to the more precise reference (0.0477) than to the bf16 one (0.0511). Maple is unusually sensitive to activation precision, plausibly because ternary weights and 256-expert top-8 routing make expert selection easy to perturb. At position 0, where attention has no history to read, KL is **1.2e-4**. ## Performance and the lm_head gfx1151 (Radeon 8060S / Strix Halo), bf16 KV, greedy, 1,500 tokens, 3 paired interleaved repetitions. KL is against the BF16 reference on the same 2,048 tokens as above. | lm_head | bpw | mean KL | top-1 | decode | |---|---|---|---|---| | **q8** (shipped in the base) | 8.5 | **0.0511** | **91.9%** | ~145 tok/s | | q4k (overlay) | 4.5 | 0.0640 | 90.1% | ~168 tok/s | | bf16 (overlay) | 16 | 0.0511 | 91.7% | ~118 tok/s | Prefill is roughly 800 tok/s on short prompts. **q8 is the default because bf16 is strictly dominated** — identical KL, 23% slower. A bf16 head costs throughput and buys exactly zero accuracy. **q4k is offered for throughput**, not recommended by default: +16% decode for +25% mean KL and −1.8pp top-1. Note DeepGrove *do* ship a Q4_K head in their own llama.cpp example, and that is the right call there — on their CPU path the same swap buys 49% (252.7 vs 169.8 tok/s) rather than 16%, because the MoE body dominates decode differently on a GPU. Same decision, different machine. ## Head variants ship as overlays, not as separate models The head is 2.7% of the file, so shipping a whole model per carrier would duplicate the identical 6.17 GB body every time. Instead the base carries the recommended q8 head and runs standalone, and alternative carriers are single-tensor `.hfq` overlays that shadow `lm_head.weight` at load: | file | size | |---|---| | `maple-preview.mq2lloydu` (base, q8 head) | 6.50 GB | | `maple-head-q4k.hfq` | 188 MB | | `maple-head-bf16.hfq` | 635 MB | ```bash hipfire run maple-preview --head q4k "..." # by registry name hipfire run --head "" # or by path ``` Each overlay is a normal `.hfq` carrying a single tensor at the same logical shape as the base's head, which is what lets hipfire attach it at load time and shadow `lm_head.weight`. They are listed in the registry entry under `heads`, with their own sha256, and are rebuilt with: ```bash hipfire-quantize --format maple --input \ --output maple-head-q4k.hfq --head-quant q4k --head-only ``` An overlay is not an approximation of a full build — it reproduces one exactly (q4k overlay 0.0640 KL / 90.1% top-1; a monolithic q4k build measures the same). Three full variants would be 19.63 GB; this is 7.30 GB, and switching heads costs 188–635 MB rather than a 6.5 GB re-download. ## Recommended settings Use the **vendor's** values — they are not the Qwen defaults, despite the shared tokenizer: ``` temperature 1.0 top_p 0.95 ``` Source: DeepGrove's own llama.cpp fork, [`github.com/deepgrove-ai/llama.cpp`](https://github.com/deepgrove-ai/llama.cpp), which `deepgrove/maple-preview-GGUF` links to as the official setup. There is no `generation_config.json` upstream and the model card names no sampler, so that README is the only first-party source. `top_k` / `min_p` values circulating elsewhere are **not** from DeepGrove. **The chat template matters.** Maple's embedded template ends its generation prompt with `<|im_start|>assistant\n\n` — including the thinking prefix. Omitting it makes the model open its own reasoning block, starting every generation off-distribution *inside* the block where degenerate loops occur. On a fixed prompt across 8 seeds, the loop rate went from **5/8 to 0/8** once the prefix and temperature were corrected. hipfire applies the template from the HFQ metadata automatically. ## KV cache Defaults to **bf16**, overridable with `--kv-mode q8`. Maple is 3:1 sliding-window(512) / global-NoPE attention. Q8 KV costs 39% of the measured divergence, and the damage is concentrated in the **tail** rather than spread as uniform blur — the median position moves only 24%, but the worst goes 10.36 → 4.21 nats. Rare catastrophic positions are what derail long generations. bf16 KV costs 1.88× the KV bytes (26,112 → 49,152 B/token; +2.81 GiB at 131k context) and ~2% decode. Use `--kv-mode q8` to trade that fidelity back for memory. ## Limitations These are the **model's**, not the port's — see the [upstream card](https://huggingface.co/deepgrove/maple-preview): > This preview received minimal post-training for agentic tasks and only > small-scale general reinforcement learning. Wikitext perplexity is high (~57 on the standard protocol) where a comparable peer scores ~7.7. That is the model, not the packing: KL against a full-precision reference is 0.05, and DeepGrove's own ternary GGUF scores within 0.2% of a BF16 run of the same weights. Maple is not *uncertain* on raw prose — its entropy matches its peer — it is confidently wrong on it, which is consistent with a reasoning-focused preview. **Give it room to think.** Maple spends a large share of its budget reasoning, and a short cap truncates it mid-derivation in a way that reads like a repetition loop but is not one. On a five-house logic puzzle, 8/8 sampled runs hit a 12,000-token cap without converging — while at 40,000 tokens they terminated cleanly with the correct answer after ~84–134 KB of reasoning. If output looks stuck, raise `max_tokens` before suspecting the model. Most of what otherwise looked like intrinsic looping was configuration: with the vendor template and temperature above, the loop rate on a fixed prompt went from 5/8 to 0/8 across 8 seeds. ## Provenance MIT, inherited from `deepgrove/maple-preview`. Weights are DeepGrove's, carried into HFQ losslessly. Converted with: ```bash hipfire-quantize --format maple \ --input \ --output maple-preview.mq2lloydu \ --head-quant q8 ```