File size: 5,064 Bytes
742cfba 0b0f4bc 742cfba 0b0f4bc | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 | ---
license: mit
base_model:
- deepgrove/maple-preview
tags:
- maple
- gguf
- dpo
- preference-optimization
- on-policy-distillation
- moe
- ternary
- apple-silicon
library_name: gguf
---
# Maple Pro — preference-optimized Maple (20B-A1B ternary MoE)
A single-file GGUF of Maple (deepgrove/maple-preview, MIT) after a full
distillation + DPO campaign: **the strongest measured variant of this model
family**, improving or matching the baseline on all 12 evaluated axes.
## Results (measured on a fixed local suite, identical seeds/conditions)
| Axis | Base Maple | **Maple Pro** |
|---|---|---|
| Coding probe (26 tasks) | 20 | **24** |
| GSM8K (pooled n=200) | 90/100 | **90.5/100** |
| HumanEval+ (50) | 44 | 44 |
| IFEval official checker (100) | 63 | **65** |
| Tools eval (12) | 11 | **12/12** |
| Knowledge (25) | 22 | **23** |
| Browser-agent probe (10) | 3 | **5** |
| IF-10 / IF-30 | 7 / 19 | **8** / 19 |
| DROP-30 | 20 | 20 |
| Median tokens/answer | 759 | **570 (−25%)** |
First and only configuration (of 17 tested) achieving simultaneous dominance
across every axis. All numbers reproducible; eval suite and logs described in
the model card below.
## What's inside
- Full LoRA fusion: 64 attention LoRA tensors **and** 121 DPO-updated base
tensors (norms, expert routers, QK-norms) baked into a single GGUF.
Attention projections stored as F16 (mixed precision; layers 0–7 keep the
base ternary TQ2_0 attention); ternary expert weights untouched
(byte-identical to the official base).
- Runtime: 6.6 GB, ~1B active parameters.
- Context: **128K native**, GGUF metadata allows **256K** — verified to work
natively with no flags (needle-tested — see Context section). No other
changes to the model: the 256K build differs from the 128K original by a
single metadata field.
## Context (measured)
5-needle recall (wikitext-2 haystack, depths 10/30/50/70/90%), same seeds:
| Context | Mode | Recall |
|---|---|---|
| 16K–131K | native | 3/5 |
| **256K** | **native (no flags)** | **4/5** |
| 256K | YaRN ×2 | 4/5 (same) |
| 384K | YaRN ×3 | 2/5 — fails |
No length degradation up to 256K: the hybrid SWA design (18/24 layers at
window 512) extrapolates ×2 natively, so **256K works by just setting the
context size — no rope-scaling flags needed** (LM Studio: set the slider).
Remaining misses are section-binding confusions between same-shape needles,
uniform across lengths. 384K+ is out of reach without long-context training.
## Runtime (important)
The `maple` architecture is **not** in upstream llama.cpp. Use the DeepGrove
fork (Apache-2.0):
```sh
git clone --depth 1 https://github.com/deepgrove-ai/llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-server
```
The Metal backend of the current fork does not implement the ternary TQ2_0
matmuls (experts, and the ternary attention of layers 0–7), so serve with
those tensors on CPU:
```sh
./build/bin/llama-server -m maple-pro.gguf --port 8080 -t 8 \
--cpu-moe --no-op-offload \
-ot 'blk\.[0-7]\.attn_[a-z]+\.weight=CPU'
```
OpenAI-compatible API on :8080. **LM Studio works too**: the GGUF was
verified on LM Studio's bundled runtime (2.65.1) — just load it and set
the context size. Measured decode on an M4 16 GB (idle): ~75 tok/s
(base maple-preview: ~94 tok/s — Pro trades some bandwidth for its
F16 fused attention).
### Serving at 256K
The GGUF advertises a 262144 context: just set the context size to 256K —
no rope-scaling flags required (native ×2 extrapolation is needle-verified,
see Context section). On a 16 GB machine use quantized KV cache to fit:
```sh
./build/bin/llama-server -m maple-pro.gguf --port 8080 -t 8 \
--ctx-size 262144 \
--cpu-moe --no-op-offload \
-ot 'blk\.[0-7]\.attn_[a-z]+\.weight=CPU' \
--cache-type-k q8_0 --cache-type-v q8_0
```
KV cache quantization is your choice; q8_0 was used in the tests
(~10 GB total on a 16 GB machine).
## Training recipe (summary)
1. **Semi on-policy distillation**: the student samples its own solutions on
verified tasks (GSM8K gold answers, IFEval-style constraint checkers,
sandboxed coding tests); a frontier teacher API model (GLM-5.3-Flash)
corrects only the failures. ~3,000 verified on-policy samples + corrections.
2. **SFT** on-policy multi-domain (the 11/12-axes intermediate).
3. **DPO** on preference pairs (student error ↔ verified teacher correction),
full-parameter (attention LoRA + norms/routers), SFT-anchored.
4. Zero-tolerance gate: a candidate ships only if it matches or beats the
baseline on **all** axes at once. 16 configurations were rejected by this
gate before this one passed.
## Credits & license
- Base model: [deepgrove/maple-preview](https://huggingface.co/deepgrove/maple-preview)
and its GGUF release (MIT).
- Runtime: [deepgrove-ai/llama.cpp](https://github.com/deepgrove-ai/llama.cpp) fork.
- This model: MIT, derivative work with attribution. Trained locally on a
single M4 16 GB via MLX.
- Made by West — [madebywest.com](https://madebywest.com)
|