File size: 2,459 Bytes
491cce7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | ---
license: mit
language:
- en
pipeline_tag: text-generation
library_name: gguf
tags:
- quantization
- gguf
- llama-cpp
- imatrix
- hybrid-quantization
- selective-quantization
- priority-queue
- mse
- qwen3.5
- mtp
- ornith
- deepreinforce
- agentic-coding
- coding
---
# Ornith-1.0-9B — ASHQ1 Quantization (MTP-Aware)
ASHQ1 quants of [Ornith-1.0-9B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B) with **explicit MTP handling** — Multi-Token Prediction heads are kept at Q8_0 and excluded from the classifier budget, preserving their quality.
## Quants
| File | Size | Method | Description |
|:-----|:----:|:------:|:------------|
| `Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf` | 5.9 GB | ASHQ1 (v6) | Balanced — beats uniform Q6_K (7.0 GB) by **0.155 PPL**, 16.5% smaller |
## Benchmark vs Uniform Q6_K
| Method | Size | PPL (ctx 1024) | Δ vs Uniform |
|:-------|:----:|:--------------:|:------------:|
| **ASHQ1** (v6, MTP-aware) | 6012 MiB | **7.4697 ± 0.04862** | **−0.1551** |
| Uniform Q6_K | 7198 MiB | 7.6248 ± 0.05039 | baseline |
ASHQ1 beats uniform Q6_K while being **1.2 GB smaller**. The current classifier (v6) dropped empirical depth-weighting heuristics — the theoretical priority queue now works even better.
MTP heads are kept at Q8_0 and excluded from the classifier budget. The 6000 MiB target includes MTP cost — the classifier gets the remainder after MTP overhead is subtracted.
## How ASHQ1 Works
See [wepiqx/ASHQ1](https://huggingface.co/wepiqx/ASHQ1) for the full method, source code, and benchmarks.
## Usage
### llama.cpp
```bash
llama-cli \
-m Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf \
--jinja \
-ngl 99 \
-c 8192
```
### Ollama
Create a `Modelfile`:
```
FROM ./Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf
PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
```
Build and run:
```bash
ollama create ornith-ashq1-mtp-6000 -f Modelfile
ollama run ornith-ashq1-mtp-6000
```
### Recommended Sampling
`temperature 0.6–1.0`, `top_p 0.95`, `top_k 20`
## Notes
- Quantized from the [Ornith MTP KL-stabilised BF16](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B) source
- Imatrix: [bartowski's Ornith imatrix](https://huggingface.co/bartowski/deepreinforce-ai_Ornith-1.0-9B-GGUF) (248 entries, 6 datasets)
- Architecture: Qwen3.5-9B (32 layers, GQA, SSM, MTP, 262k context)
- MTP heads deployed at Q8_0 (5 heads, ~1.7 GB reserved)
- License: MIT
|