Abiray/ThinkingCap-Qwen3.6-27B-Q4_K_M-GGUF

ThinkingCap: Qwen 3.6 27B - GGUF

This is a GGUF quantized version of bottlecapai/ThinkingCap-Qwen3.6-27B. It contains the Q4_K_M quantization, providing a balanced trade-off between memory efficiency and reasoning quality for local inference frameworks like llama.cpp.

---

Capability of Qwen3.6-27B with 50% less thinking tokens on average, and over 90% less in best cases. Achieved via finetuning Qwen3.6-27B (Qwen Team, 2026) with state-of-the-art algorithms on a curated set of problems of various domains and difficulty. We designed the finetuning to be as minimally invasive as possible, preserving all of the original answer quality and style of Qwen, while being more token efficient.

We rigorously evaluate the resulting checkpoint across general reasoning, non-reasoning multiple-choice question answering, everyday multi-turn conversations, system prompt adherence, safety, math, code and agentic use cases. Due to the high variability of reasoning quality at Qwen-recommended sampling temperature 1.0, we run each benchmark with multiple seeds and do statistical significance testing on all the results. We evaluate both in domain (holdout parts of selected datasets included in training) and out of domain.

Out-of-domain token efficiency

Benchmark	Accuracy		Thinking tokens
	Base	Ours	Base	Ours	Reduction
Knowledge & reasoning
GPQA-Diamond	85.5 ±1.4	83.8 ±1.9	10,777	3,351	↓ 67.8%
SuperGPQA	64.0 ±0.2	64.0 ±0.1	8,246	3,384	↓ 58.4%
MMLU-Pro	85.9 ±0.2	85.4 ±0.2	3,455	1,290	↓ 53.7%
MMLU-Redux	93.9 ±0.1	93.9 ±0.1	947	406	↓ 44.8%
C-Eval	90.6 ±0.7	90.3 ±0.6	1,279	663	↓ 47.1%
Math & code
HMMT (Nov 2025)	88.0 ±3.7	84.7 ±3.7	39,277	27,388	↓ 38.0%
LiveCodeBench	80.7 ±0.6	84.3 ±1.0	15,744	10,158	↓ 41.1%
Long-context & multimodal
LongBench v2	62.6 ±3.6	60.2 ±1.7	1,765	1,091	↓ 39.1%
RealWorldQA	82.4 ±0.7	81.9 ±1.2	2,959	913	↓ 48.5%
AA-LCR	76.2 ±3.0	74.2 ±2.2	2,455	1,337	↓ 45.5%
Instruction following & agentic
System-prompt adherence	80.6 ±1.2	81.5 ±1.8	1,737	976	↓ 40.0%
Claw-Eval think/task	87.0 ±1.9	84.4 ±1.2	919	689	↓ 25.2%
Macro average	81.5	80.7	—	—	↓ 45.8%

_{Claw-Eval thinking tokens are per-task (agentic; not a single-turn trace).}

Settings

• Models: base Qwen/Qwen3.6-27B vs bottlecapai/ThinkingCap-Qwen3.6-27B (shown as Ours in the table).
• Seeds: 5 per condition; thinking on; cells are mean ± 95% CI across seeds.
• Decoding: thinking on; sampling temperature=1.0, top_p=0.95, top_k=20, min_p=0.0.
• Max generation tokens: 100,000 for the general suite (gpqa_diamond, mmlu_pro, longbench_v2, realworldqa) and AA-LCR; 250,000 for HMMT; 32,768 for supergpqa and livecodebench; 16,384 for ceval and mmlu_redux; 15,000 for llm-system-prompts-benchmark; 49,152 for Claw-Eval.

In-domain evals

Holdout test splits of datasets whose train splits are part of the finetuning mix — quality retention on in-distribution tasks.

Benchmark	Accuracy		Thinking tokens
	Base	Ours	Base	Ours	Reduction
GSM8K	93.3 ±1.5	96.5 ±0.3	3,175	648	↓ 74.1%
ARC-Challenge	97.0 ±0.3	97.6 ±0.4	966	335	↓ 51.5%
ARC-Easy	99.3 ±0.2	99.4 ±0.2	566	260	↓ 44.5%
CommonsenseQA	86.7 ±0.7	88.2 ±0.9	1,118	273	↓ 64.1%
OpenBookQA	96.0 ±0.5	96.7 ±0.6	858	248	↓ 59.5%
QASC	91.7 ±0.7	92.2 ±0.5	1,258	348	↓ 61.9%
SciQ	97.0 ±0.2	97.5 ±0.2	766	276	↓ 48.3%
Macro average	94.4	95.4	—	—	↓ 57.7%

Guardrails preservation

Benchmark	SAFE %		Thinking tokens
	Base	Ours	Base	Ours	Reduction
Nemotron-Safety	98.9 ±0.8	99.0 ±0.4	1,242	933	↓ 23.8%
HEx-PHI	99.9 ±0.2	100.0 ±0.0	693	543	↓ 20.0%
Macro average	99.4	99.5	—	—	↓ 21.9%

---

GGUF Local Usage (llama.cpp)

To run this model locally, you will need a framework capable of handling Q4_K_M GGUF files. llama.cpp is highly recommended.

# Clone and build llama.cpp (if you haven't already)
git clone [https://github.com/ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp)
cd llama.cpp
make -j

# Run the model locally
./llama-cli -hf Abiray/ThinkingCap-Qwen3.6-27B-Q4_K_M-GGUF -p "Explain quantum computing in simple terms."