Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,212 @@
+---
+license: mit
+base_model: deepgrove/maple-preview
+base_model_relation: quantized
+language: en
+pipeline_tag: text-generation
+tags:
+  - hipfire
+  - mixture-of-experts
+  - reasoning
+  - ternary
+  - rdna
+  - rocm
+---
+
+# Maple-Preview — hipfire HFQ (arch 15)
+
+A [hipfire](https://github.com/warpfront/hipfire) build of
+[`deepgrove/maple-preview`](https://huggingface.co/deepgrove/maple-preview),
+a 20B-A1B natively-ternary reasoning MoE, for AMD RDNA GPUs.
+
+| | |
+|---|---|
+| File | `maple-preview.mq2lloydu` |
+| Size | 6,499,340,288 bytes (6.05 GiB) |
+| sha256 | `7fb52fe72c1a0a4455d0fe3a8109b0df66fa53782f41d8b257140d3e966645db` |
+| arch_id | 15 |
+| Body carrier | `MQ2G256LloydU` (qt=51), 2.250 bpw |
+| lm_head | Q8 |
+| Context | 131,072 |
+
+```bash
+hipfire run maple-preview "Prove that sqrt(2) is irrational."
+```
+
+## The body is a packing, not a quantization
+
+Every linear weight in Maple-Preview is already exactly `{-s, 0, +s}` with one
+bf16 scale per output row. `MQ2G256LloydU` (qt=51) is an **unrotated** MQ2-Lloyd
+carrier built for this: it stores those three levels exactly.
+
+Verified end-to-end against the **original** `deepgrove/maple-preview`
+safetensors — not against our own dequantizer, which could only confirm a shared
+misunderstanding:
+
+```
+model.layers.0.mlp.experts.0.gate_proj.weight   max|err| = 0
+model.layers.0.mlp.experts.0.down_proj.weight   max|err| = 0
+model.layers.0.self_attn.q_proj.weight          max|err| = 0
+model.layers.0.self_attn.o_proj.weight          max|err| = 0
+```
+
+Every bitwise difference is a signed zero (`-0.0` → `+0.0`), which is
+numerically irrelevant. The converter *refuses* a non-ternary row rather than
+falling back to lossy quantization, so a clean convert is itself evidence the
+property holds across all 18,528 ternary tensors (19,579,011,072 weights).
+
+The accuracy this model loses relative to full precision therefore comes from
+the runtime, not the weights.
+
+## Measured quality
+
+Against a BF16 reference — `deepgrove/maple-preview` converted to a BF16 GGUF
+and scored with llama.cpp on CPU — over 2,048 teacher-forced wikitext tokens,
+identical tokens and protocol on both sides:
+
+| KV tier | lm_head | mean KL | top-1 agreement |
+|---|---|---|---|
+| **bf16** (default) | **Q8** | **0.0511** | **91.9%** |
+| q8 | Q8 | 0.0842 | 90.8% |
+
+The lm_head is free: a bf16 head gives the *identical* 0.0511, so Q8 costs
+nothing and decodes 23% faster.
+
+**Interpreting 0.0511.** It is close to the floor for this model rather than a
+defect. llama.cpp rounds activations to bf16 before every matmul
+(`vec_dot_type = GGML_TYPE_BF16`, ε = 2⁻⁸), so the reference is itself
+precision-limited: an F16 reference built from the same weights differs from the
+BF16 reference by **mean KL 0.0357** — the same order as our entire divergence
+from it — and this build sits *closer* to the more precise reference (0.0477)
+than to the bf16 one (0.0511). Maple is unusually sensitive to activation
+precision, plausibly because ternary weights and 256-expert top-8 routing make
+expert selection easy to perturb.
+
+At position 0, where attention has no history to read, KL is **1.2e-4**.
+
+## Performance and the lm_head
+
+gfx1151 (Radeon 8060S / Strix Halo), bf16 KV, greedy, 1,500 tokens, 3 paired
+interleaved repetitions. KL is against the BF16 reference on the same 2,048
+tokens as above.
+
+| lm_head | bpw | mean KL | top-1 | decode |
+|---|---|---|---|---|
+| **q8** (shipped in the base) | 8.5 | **0.0511** | **91.9%** | ~145 tok/s |
+| q4k (overlay) | 4.5 | 0.0640 | 90.1% | ~168 tok/s |
+| bf16 (overlay) | 16 | 0.0511 | 91.7% | ~118 tok/s |
+
+Prefill is roughly 800 tok/s on short prompts.
+
+**q8 is the default because bf16 is strictly dominated** — identical KL, 23%
+slower. A bf16 head costs throughput and buys exactly zero accuracy.
+
+**q4k is offered for throughput**, not recommended by default: +16% decode for
++25% mean KL and −1.8pp top-1. Note DeepGrove *do* ship a Q4_K head in their own
+llama.cpp example, and that is the right call there — on their CPU path the same
+swap buys 49% (252.7 vs 169.8 tok/s) rather than 16%, because the MoE body
+dominates decode differently on a GPU. Same decision, different machine.
+
+## Head variants ship as overlays, not as separate models
+
+The head is 2.7% of the file, so shipping a whole model per carrier would
+duplicate the identical 6.17 GB body every time. Instead the base carries the
+recommended q8 head and runs standalone, and alternative carriers are
+single-tensor `.hfq` overlays that shadow `lm_head.weight` at load:
+
+| file | size |
+|---|---|
+| `maple-preview.mq2lloydu` (base, q8 head) | 6.50 GB |
+| `maple-head-q4k.hfq` | 188 MB |
+| `maple-head-bf16.hfq` | 635 MB |
+
+Each overlay is a normal `.hfq` carrying a single tensor at the same logical
+shape as the base's head, which is what lets hipfire attach it at load time and
+shadow `lm_head.weight`. They are listed in the registry entry under `heads`,
+with their own sha256, and are rebuilt with:
+
+```bash
+hipfire-quantize --format maple --input <safetensors dir> \
+  --output maple-head-q4k.hfq --head-quant q4k --head-only
+```
+
+An overlay is not an approximation of a full build — it reproduces one exactly
+(q4k overlay 0.0640 KL / 90.1% top-1; a monolithic q4k build measures the same).
+Three full variants would be 19.63 GB; this is 7.30 GB, and switching heads
+costs 188–635 MB rather than a 6.5 GB re-download.
+
+## Recommended settings
+
+Use the **vendor's** values — they are not the Qwen defaults, despite the shared
+tokenizer:
+
+```
+temperature 1.0
+top_p       0.95
+```
+
+Source: DeepGrove's own llama.cpp fork,
+[`github.com/deepgrove-ai/llama.cpp`](https://github.com/deepgrove-ai/llama.cpp),
+which `deepgrove/maple-preview-GGUF` links to as the official setup. There is no
+`generation_config.json` upstream and the model card names no sampler, so that
+README is the only first-party source. `top_k` / `min_p` values circulating
+elsewhere are **not** from DeepGrove.
+
+**The chat template matters.** Maple's embedded template ends its generation
+prompt with `<|im_start|>assistant\n<think>\n` — including the thinking prefix.
+Omitting it makes the model open its own reasoning block, starting every
+generation off-distribution *inside* the block where degenerate loops occur. On
+a fixed prompt across 8 seeds, the loop rate went from **5/8 to 0/8** once the
+prefix and temperature were corrected. hipfire applies the template from the
+HFQ metadata automatically.
+
+## KV cache
+
+Defaults to **bf16**, overridable with `--kv-mode q8`.
+
+Maple is 3:1 sliding-window(512) / global-NoPE attention. Q8 KV costs 39% of the
+measured divergence, and the damage is concentrated in the **tail** rather than
+spread as uniform blur — the median position moves only 24%, but the worst goes
+10.36 → 4.21 nats. Rare catastrophic positions are what derail long generations.
+
+bf16 KV costs 1.88× the KV bytes (26,112 → 49,152 B/token; +2.81 GiB at 131k
+context) and ~2% decode. Use `--kv-mode q8` to trade that fidelity back for
+memory.
+
+## Limitations
+
+These are the **model's**, not the port's — see the
+[upstream card](https://huggingface.co/deepgrove/maple-preview):
+
+> This preview received minimal post-training for agentic tasks and only
+> small-scale general reinforcement learning.
+
+Wikitext perplexity is high (~57 on the standard protocol) where a comparable
+peer scores ~7.7. That is the model, not the packing: KL against a
+full-precision reference is 0.05, and DeepGrove's own ternary GGUF scores within
+0.2% of a BF16 run of the same weights. Maple is not *uncertain* on raw prose —
+its entropy matches its peer — it is confidently wrong on it, which is
+consistent with a reasoning-focused preview.
+
+**Give it room to think.** Maple spends a large share of its budget reasoning,
+and a short cap truncates it mid-derivation in a way that reads like a
+repetition loop but is not one. On a five-house logic puzzle, 8/8 sampled runs
+hit a 12,000-token cap without converging — while at 40,000 tokens they
+terminated cleanly with the correct answer after ~84–134 KB of reasoning. If
+output looks stuck, raise `max_tokens` before suspecting the model.
+
+Most of what otherwise looked like intrinsic looping was configuration: with
+the vendor template and temperature above, the loop rate on a fixed prompt went
+from 5/8 to 0/8 across 8 seeds.
+
+## Provenance
+
+MIT, inherited from `deepgrove/maple-preview`. Weights are DeepGrove's,
+carried into HFQ losslessly. Converted with:
+
+```bash
+hipfire-quantize --format maple \
+  --input <maple-preview safetensors dir> \
+  --output maple-preview.mq2lloydu \
+  --head-quant q8
+```