| --- |
| license: mit |
| language: |
| - en |
| pipeline_tag: text-generation |
| library_name: gguf |
| tags: |
| - quantization |
| - gguf |
| - llama-cpp |
| - imatrix |
| - hybrid-quantization |
| - selective-quantization |
| - priority-queue |
| - mse |
| - qwen3.5 |
| - mtp |
| - ornith |
| - deepreinforce |
| - agentic-coding |
| - coding |
| --- |
| |
| # Ornith-1.0-9B — ASHQ1 Quantization (MTP-Aware) |
|
|
| ASHQ1 quants of [Ornith-1.0-9B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B) with **explicit MTP handling** — Multi-Token Prediction heads are kept at Q8_0 and excluded from the classifier budget, preserving their quality. |
| |
| ## Quants |
| |
| | File | Size | Method | Description | |
| |:-----|:----:|:------:|:------------| |
| | `Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf` | 5.9 GB | ASHQ1 (v6) | Balanced — beats uniform Q6_K (7.0 GB) by **0.155 PPL**, 16.5% smaller | |
|
|
| ## Benchmark vs Uniform Q6_K |
| |
| | Method | Size | PPL (ctx 1024) | Δ vs Uniform | |
| |:-------|:----:|:--------------:|:------------:| |
| | **ASHQ1** (v6, MTP-aware) | 6012 MiB | **7.4697 ± 0.04862** | **−0.1551** | |
| | Uniform Q6_K | 7198 MiB | 7.6248 ± 0.05039 | baseline | |
|
|
| ASHQ1 beats uniform Q6_K while being **1.2 GB smaller**. The current classifier (v6) dropped empirical depth-weighting heuristics — the theoretical priority queue now works even better. |
| |
| MTP heads are kept at Q8_0 and excluded from the classifier budget. The 6000 MiB target includes MTP cost — the classifier gets the remainder after MTP overhead is subtracted. |
|
|
| ## How ASHQ1 Works |
|
|
| See [wepiqx/ASHQ1](https://huggingface.co/wepiqx/ASHQ1) for the full method, source code, and benchmarks. |
|
|
| ## Usage |
|
|
| ### llama.cpp |
|
|
| ```bash |
| llama-cli \ |
| -m Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf \ |
| --jinja \ |
| -ngl 99 \ |
| -c 8192 |
| ``` |
|
|
| ### Ollama |
|
|
| Create a `Modelfile`: |
|
|
| ``` |
| FROM ./Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf |
| |
| PARAMETER num_ctx 8192 |
| PARAMETER temperature 0.6 |
| PARAMETER top_k 20 |
| PARAMETER top_p 0.95 |
| ``` |
|
|
| Build and run: |
|
|
| ```bash |
| ollama create ornith-ashq1-mtp-6000 -f Modelfile |
| ollama run ornith-ashq1-mtp-6000 |
| ``` |
|
|
| ### Recommended Sampling |
|
|
| `temperature 0.6–1.0`, `top_p 0.95`, `top_k 20` |
|
|
| ## Notes |
|
|
| - Quantized from the [Ornith MTP KL-stabilised BF16](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B) source |
| - Imatrix: [bartowski's Ornith imatrix](https://huggingface.co/bartowski/deepreinforce-ai_Ornith-1.0-9B-GGUF) (248 entries, 6 datasets) |
| - Architecture: Qwen3.5-9B (32 layers, GQA, SSM, MTP, 262k context) |
| - MTP heads deployed at Q8_0 (5 heads, ~1.7 GB reserved) |
| - License: MIT |
| |
