metadata
license: mit
language:
- en
pipeline_tag: text-generation
library_name: gguf
tags:
- quantization
- gguf
- llama-cpp
- imatrix
- hybrid-quantization
- selective-quantization
- priority-queue
- mse
- qwen3.5
- mtp
- ornith
- deepreinforce
- agentic-coding
- coding
Ornith-1.0-9B — ASHQ1 Quantization (MTP-Aware)
ASHQ1 quants of Ornith-1.0-9B with explicit MTP handling — Multi-Token Prediction heads are kept at Q8_0 and excluded from the classifier budget, preserving their quality.
Quants
| File | Size | Method | Description |
|---|---|---|---|
Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf |
5.9 GB | ASHQ1 (v6) | Balanced — beats uniform Q6_K (7.0 GB) by 0.155 PPL, 16.5% smaller |
Benchmark vs Uniform Q6_K
| Method | Size | PPL (ctx 1024) | Δ vs Uniform |
|---|---|---|---|
| ASHQ1 (v6, MTP-aware) | 6012 MiB | 7.4697 ± 0.04862 | −0.1551 |
| Uniform Q6_K | 7198 MiB | 7.6248 ± 0.05039 | baseline |
ASHQ1 beats uniform Q6_K while being 1.2 GB smaller. The current classifier (v6) dropped empirical depth-weighting heuristics — the theoretical priority queue now works even better.
MTP heads are kept at Q8_0 and excluded from the classifier budget. The 6000 MiB target includes MTP cost — the classifier gets the remainder after MTP overhead is subtracted.
How ASHQ1 Works
See wepiqx/ASHQ1 for the full method, source code, and benchmarks.
Usage
llama.cpp
llama-cli \
-m Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf \
--jinja \
-ngl 99 \
-c 8192
Ollama
Create a Modelfile:
FROM ./Ornith-1.0-9B-MTP-BF16-ASHQ1-6000.gguf
PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
Build and run:
ollama create ornith-ashq1-mtp-6000 -f Modelfile
ollama run ornith-ashq1-mtp-6000
Recommended Sampling
temperature 0.6–1.0, top_p 0.95, top_k 20
Notes
- Quantized from the Ornith MTP KL-stabilised BF16 source
- Imatrix: bartowski's Ornith imatrix (248 entries, 6 datasets)
- Architecture: Qwen3.5-9B (32 layers, GQA, SSM, MTP, 262k context)
- MTP heads deployed at Q8_0 (5 heads, ~1.7 GB reserved)
- License: MIT