| --- |
| license: mit |
| base_model: |
| - deepgrove/maple-preview |
| tags: |
| - maple |
| - gguf |
| - dpo |
| - preference-optimization |
| - on-policy-distillation |
| - moe |
| - ternary |
| - apple-silicon |
| library_name: gguf |
| --- |
| |
| # Maple Pro — preference-optimized Maple (20B-A1B ternary MoE) |
|
|
| A single-file GGUF of Maple (deepgrove/maple-preview, MIT) after a full |
| distillation + DPO campaign: **the strongest measured variant of this model |
| family**, improving or matching the baseline on all 12 evaluated axes. |
|
|
| ## Results (measured on a fixed local suite, identical seeds/conditions) |
|
|
| | Axis | Base Maple | **Maple Pro** | |
| |---|---|---| |
| | Coding probe (26 tasks) | 20 | **24** | |
| | GSM8K (pooled n=200) | 90/100 | **90.5/100** | |
| | HumanEval+ (50) | 44 | 44 | |
| | IFEval official checker (100) | 63 | **65** | |
| | Tools eval (12) | 11 | **12/12** | |
| | Knowledge (25) | 22 | **23** | |
| | Browser-agent probe (10) | 3 | **5** | |
| | IF-10 / IF-30 | 7 / 19 | **8** / 19 | |
| | DROP-30 | 20 | 20 | |
| | Median tokens/answer | 759 | **570 (−25%)** | |
|
|
| First and only configuration (of 17 tested) achieving simultaneous dominance |
| across every axis. All numbers reproducible; eval suite and logs described in |
| the model card below. |
|
|
| ## What's inside |
|
|
| - Full LoRA fusion: 64 attention LoRA tensors **and** 121 DPO-updated base |
| tensors (norms, expert routers, QK-norms) baked into a single GGUF. |
| Attention projections stored as F16 (mixed precision; layers 0–7 keep the |
| base ternary TQ2_0 attention); ternary expert weights untouched |
| (byte-identical to the official base). |
| - Runtime: 6.6 GB, ~1B active parameters. |
| - Context: **128K native**, GGUF metadata allows **256K** — verified to work |
| natively with no flags (needle-tested — see Context section). No other |
| changes to the model: the 256K build differs from the 128K original by a |
| single metadata field. |
| |
| ## Context (measured) |
| |
| 5-needle recall (wikitext-2 haystack, depths 10/30/50/70/90%), same seeds: |
| |
| | Context | Mode | Recall | |
| |---|---|---| |
| | 16K–131K | native | 3/5 | |
| | **256K** | **native (no flags)** | **4/5** | |
| | 256K | YaRN ×2 | 4/5 (same) | |
| | 384K | YaRN ×3 | 2/5 — fails | |
| |
| No length degradation up to 256K: the hybrid SWA design (18/24 layers at |
| window 512) extrapolates ×2 natively, so **256K works by just setting the |
| context size — no rope-scaling flags needed** (LM Studio: set the slider). |
| Remaining misses are section-binding confusions between same-shape needles, |
| uniform across lengths. 384K+ is out of reach without long-context training. |
| |
| ## Runtime (important) |
| |
| The `maple` architecture is **not** in upstream llama.cpp. Use the DeepGrove |
| fork (Apache-2.0): |
| |
| ```sh |
| git clone --depth 1 https://github.com/deepgrove-ai/llama.cpp |
| cmake -B build -DCMAKE_BUILD_TYPE=Release |
| cmake --build build -j --target llama-server |
| ``` |
| |
| The Metal backend of the current fork does not implement the ternary TQ2_0 |
| matmuls (experts, and the ternary attention of layers 0–7), so serve with |
| those tensors on CPU: |
|
|
| ```sh |
| ./build/bin/llama-server -m maple-pro.gguf --port 8080 -t 8 \ |
| --cpu-moe --no-op-offload \ |
| -ot 'blk\.[0-7]\.attn_[a-z]+\.weight=CPU' |
| ``` |
|
|
| OpenAI-compatible API on :8080. **LM Studio works too**: the GGUF was |
| verified on LM Studio's bundled runtime (2.65.1) — just load it and set |
| the context size. Measured decode on an M4 16 GB (idle): ~75 tok/s |
| (base maple-preview: ~94 tok/s — Pro trades some bandwidth for its |
| F16 fused attention). |
|
|
| ### Serving at 256K |
|
|
| The GGUF advertises a 262144 context: just set the context size to 256K — |
| no rope-scaling flags required (native ×2 extrapolation is needle-verified, |
| see Context section). On a 16 GB machine use quantized KV cache to fit: |
|
|
| ```sh |
| ./build/bin/llama-server -m maple-pro.gguf --port 8080 -t 8 \ |
| --ctx-size 262144 \ |
| --cpu-moe --no-op-offload \ |
| -ot 'blk\.[0-7]\.attn_[a-z]+\.weight=CPU' \ |
| --cache-type-k q8_0 --cache-type-v q8_0 |
| ``` |
|
|
| KV cache quantization is your choice; q8_0 was used in the tests |
| (~10 GB total on a 16 GB machine). |
| |
| ## Training recipe (summary) |
| |
| 1. **Semi on-policy distillation**: the student samples its own solutions on |
| verified tasks (GSM8K gold answers, IFEval-style constraint checkers, |
| sandboxed coding tests); a frontier teacher API model (GLM-5.3-Flash) |
| corrects only the failures. ~3,000 verified on-policy samples + corrections. |
| 2. **SFT** on-policy multi-domain (the 11/12-axes intermediate). |
| 3. **DPO** on preference pairs (student error ↔ verified teacher correction), |
| full-parameter (attention LoRA + norms/routers), SFT-anchored. |
| 4. Zero-tolerance gate: a candidate ships only if it matches or beats the |
| baseline on **all** axes at once. 16 configurations were rejected by this |
| gate before this one passed. |
| |
| ## Credits & license |
| |
| - Base model: [deepgrove/maple-preview](https://huggingface.co/deepgrove/maple-preview) |
| and its GGUF release (MIT). |
| - Runtime: [deepgrove-ai/llama.cpp](https://github.com/deepgrove-ai/llama.cpp) fork. |
| - This model: MIT, derivative work with attribution. Trained locally on a |
| single M4 16 GB via MLX. |
| - Made by West — [madebywest.com](https://madebywest.com) |
| |
