---
license: apache-2.0
base_model: Qwen/Qwen3.8-27B
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- thinklite
- reasoning
- efficient-reasoning
- qwen3.8
- gguf
---

# Thinklite-27B

**Qwen3.8-27B that thinks with fewer tokens.** Thinklite-27B is trained to drop the parts of its reasoning that do not
help it reach the answer: restating the question, repeating a step it already did, re-checking a result that was
already checked, and exploring approaches it then abandons. Answers arrive sooner, with the same model, the same chat
template and the same hardware cost per token.

| Version | Reasoning tokens (avg per benchmark) | All output tokens | Time to full answer (TTFA) |
|---|---|---|---|
| **Thinklite-27B Low** | 20% fewer | 14% fewer | 12% faster |
| **Thinklite-27B High** | 28% fewer | 20% fewer | 22% faster |
| **Thinklite-27B Max** | 40% fewer | 33% fewer | 43% faster |

Measured on six benchmarks at `reasoning_effort=low` against Qwen3.8-27B at the same setting (details below).
- *Reasoning tokens*: the thinking part only; the change is averaged over the six benchmarks.
- *All output tokens*: thinking plus answer, summed over all 4,291 requests.
- *TTFA*: the time a single user waits for the complete answer, averaged over the six benchmarks. Thinklite-27B
  decodes at exactly the base model's speed, so the whole speedup comes from writing fewer tokens.

## Versions

All three versions are full models (merged bf16 weights, drop-in replacements for Qwen3.8-27B) in this repository:

| Version | Branch | What it is | Choose it for |
|---|---|---|---|
| **Thinklite-27B High** | `main` | Trained on the full dataset; the weights keep the largest 25% of each layer's fine-tuning changes, applied at 0.7x | the balance of savings and accuracy |
| **Thinklite-27B Low** | `low` | Trained on a smaller dataset, merged the same way as High | the closest accuracy to the base model |
| **Thinklite-27B Max** | `max` | Trained on the full dataset, with the full fine-tuning applied | the largest token savings |

"Low", "High" and "Max" name the versions, not reasoning settings: every version supports `reasoning_effort` `low`,
`medium` and `xhigh`, exactly like Qwen3.8-27B. A Q4_K_M GGUF of Thinklite-27B High is on `main`; the LoRA adapters are
on the `adapters` branch.

### How to get each version

Each version lives on its own **branch** of this repository: `main` is High (the default), `low` is Low, `max` is Max.
Pick the branch (Hugging Face calls it a *revision*) wherever you load the model:

- **On the website:** open the **Files and versions** tab and switch the branch menu (it shows `main`) to `low` or
  `max`.
- **Download with the CLI:**

  ```bash
  hf download Sparset/thinklite-27B --revision low --local-dir Thinklite-27B-Low
  hf download Sparset/thinklite-27B --revision max --local-dir Thinklite-27B-Max
  hf download Sparset/thinklite-27B --local-dir Thinklite-27B-High          # main
  ```

- **Python:** `snapshot_download("Sparset/thinklite-27B", revision="low")`, or pass `revision="low"` / `revision="max"` to any
  `from_pretrained(...)` call.
- **vLLM:** add `--revision low` or `--revision max` to `vllm serve` (examples under Usage).
- **Git:** `git clone -b low https://huggingface.co/Sparset/thinklite-27B` (requires `git lfs`).

Without a revision you always get `main`, which is Thinklite-27B High.

## Objective

Reasoning models spend much of their output on thinking that does not change the answer. The goal of Thinklite is to
remove that overhead **without teaching the model new behaviour**: every training target is the model's own reasoning,
shortened only by deleting redundant parts, so the model learns to reach the same answers by a shorter route rather
than imitating a different model's style. Reasoning that is genuinely needed, for example on hard problems, is kept.

## Method

**1. Self-generated, verified answers.** Qwen3.8-27B answered a broad prompt set covering math, code, structured output
(JSON / YAML / TOML / tool calls), instruction following, science, tool use, grounded question answering and
multi-turn conversations. Only answers verified as correct were kept: exact checkers for math, executed unit tests for
code, schema and parser checks for structured output, rule checks for instructions, and an answer-only judge for
open-ended questions.

**2. Trimming and deletion-only editing.** Each correct reasoning trace was split into units.
- Everything after the point where the model had committed to its final answer was cut.
- Redundant units were deleted: restatements of the prompt, repeated steps, redundant checks, abandoned detours and
  filler. Nothing was rewritten, paraphrased or added.
- Every edit had to pass mechanical checks: the final answer is unchanged, every value the answer uses is still
  derived, and no dependency is broken.
- A **continuation test** let the model continue from the shortened reasoning and required it to still solve the task.
- Traces from the model's high-effort mode were kept at full length as anchors, so its `xhigh` behaviour is preserved.

**3. Training: SFT + DPO with LoRA and quantization-aware training.**
- LoRA (rank 32, alpha 64) on all attention, MLP and linear-attention projections.
- The objective is DPO (beta 0.1) plus a supervised term on the preferred answer. The shortened trace is preferred over
  the original; rows without a shorter variant are supervised directly.
- Quantization-aware training: during training the frozen base layers ran fake-quantized to int4 (group size 32), so
  the learned changes hold up when the model is quantized, as in the GGUF.
- Learning rate 5e-5. The full dataset is 1,370 rows (1 epoch); Thinklite-27B Low was trained on a smaller set of 474
  rows (2 epochs).

**4. Update-selective merge (High and Low).** Instead of adding the whole LoRA update to the base weights, each weight
matrix keeps only the largest 25% (by magnitude) of its fine-tuning changes, and those are applied at 0.7x
([DLER](https://arxiv.org/abs/2510.15110)). The model keeps most of the efficiency gain while staying closer to the
base model on hard problems. Thinklite-27B Max applies the full update.

## Benchmarks

**Setup:**
- One frozen suite: identical prompts, chat template, token caps and scoring for every model.
- vLLM 0.30.0, bf16 weights, FP8 KV cache, Qwen3.8's thinking-mode sampling, 64 concurrent requests.
- One NVIDIA RTX PRO 6000 Blackwell (96 GB).
- All rows use `reasoning_effort=low` except the reference "Qwen3.8-27B (xhigh)" column.
- Answers that hit their cap were rerun once with a 131,072-token cap. Truncated or unparseable answers count as
  incorrect.

- **GSM8K**: 1,319 problems
- **AIME 2025**: 30 problems x 2 samples, avg@2
- **MMLU-Pro**: 1,000-question stratified subset
- **IFEval**: 541 prompts, prompt-level strict
- **LiveCodeBench v6**: 131 problems (2025-02 to 2025-04), pass@1
- **BFCL**: 1,240 non-live function-calling cases (BFCL v3 categories, official AST checker)

### Accuracy (%)

| Benchmark | Qwen3.8-27B (xhigh) | Qwen3.8-27B (low) | Thinklite-27B Low | Thinklite-27B High | Thinklite-27B Max |
|---|---|---|---|---|---|
| GSM8K | 97.2 | 97.4 | 96.9 | 97.3 | 96.7 |
| AIME 2025 | 93.3 | 90.0 | 88.3 | 78.3 | 65.0 |
| MMLU-Pro | 86.3 | 84.4 | 84.7 | 84.7 | 82.6 |
| IFEval | 91.1 | 86.3 | 84.3 | 83.9 | 83.0 |
| LiveCodeBench v6 | 84.0 | 71.8 | 72.5 | 67.2 | 61.8 |
| BFCL | 88.7 | 90.8 | 90.4 | 90.4 | 90.2 |
| **Average** | | **86.8** | **86.2** | **83.6** | **79.9** |

### Reasoning tokens per answer (mean; change vs Qwen3.8-27B at low)

| Benchmark | Qwen3.8-27B (low) | Thinklite-27B Low | Thinklite-27B High | Thinklite-27B Max |
|---|---|---|---|---|
| GSM8K | 183 | 163 (-11%) | 147 (-20%) | 124 (-32%) |
| AIME 2025 | 12,125 | 8,374 (-31%) | 8,618 (-29%) | 7,867 (-35%) |
| MMLU-Pro | 1,103 | 934 (-15%) | 748 (-32%) | 554 (-50%) |
| IFEval | 659 | 464 (-30%) | 416 (-37%) | 326 (-51%) |
| LiveCodeBench v6 | 10,741 | 9,152 (-15%) | 7,581 (-29%) | 5,639 (-48%) |
| BFCL | 96 | 80 (-17%) | 76 (-21%) | 70 (-27%) |
| **All output tokens, 4,291 requests** | 4,915,895 | 4,244,014 (-14%) | 3,915,800 (-20%) | 3,285,976 (-33%) |

### Time to full answer, single user (seconds, median / p95; speed-up at the median)

| Benchmark | Qwen3.8-27B (low) | Thinklite-27B Low | Thinklite-27B High | Thinklite-27B Max |
|---|---|---|---|---|
| GSM8K | 11.3 / 23 | 10.4 / 21 (8% faster) | 10.3 / 20 (10% faster) | 9.7 / 17 (16% faster) |
| AIME 2025 | 181.9 / 1612 | 156.9 / 1109 (16% faster) | 147.6 / 1227 (23% faster) | 122.7 / 1163 (48% faster) |
| MMLU-Pro | 28.9 / 169 | 25.4 / 118 (14% faster) | 24.3 / 88 (19% faster) | 20.9 / 68 (38% faster) |
| IFEval | 25.1 / 110 | 19.3 / 85 (30% faster) | 18.9 / 80 (33% faster) | 15.8 / 69 (59% faster) |
| LiveCodeBench v6 | 129.4 / 1764 | 125.0 / 1588 (4% faster) | 89.4 / 1542 (45% faster) | 67.6 / 1323 (91% faster) |
| BFCL | 5.6 / 19 | 5.4 / 17 (4% faster) | 5.4 / 16 (4% faster) | 5.4 / 16 (4% faster) |

TTFA is computed from each answer's length, the prompt prefill time and the single-user decode speed measured on the
same GPU (about 28 tokens/s for every model). Thinklite-27B Low was timed in a separate run on the same GPU type; its
times are shown relative to the base model timed in that run (same speedup, common base timing).

## Usage

**vLLM**

```bash
vllm serve Sparset/thinklite-27B --max-model-len 135168 --kv-cache-dtype fp8                    # Thinklite-27B High
vllm serve Sparset/thinklite-27B --revision low --max-model-len 135168 --kv-cache-dtype fp8     # Thinklite-27B Low
vllm serve Sparset/thinklite-27B --revision max --max-model-len 135168 --kv-cache-dtype fp8     # Thinklite-27B Max
```

```python
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
reply = client.chat.completions.create(
    model="Sparset/thinklite-27B",
    messages=[{"role": "user", "content": "How many primes are there below 100?"}],
    extra_body={"chat_template_kwargs": {"reasoning_effort": "low"}},
)
print(reply.choices[0].message.content)
```

**Transformers**

```python
from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained("Sparset/thinklite-27B", revision="main", dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("Sparset/thinklite-27B", revision="main")
```

Use `revision="low"` or `revision="max"` for the other versions.

**llama.cpp (Thinklite-27B High, GGUF)**

```bash
llama-server -hf Sparset/thinklite-27B:Q4_K_M --jinja -c 32768 --chat-template-kwargs '{"reasoning_effort": "low"}'
# or with local files:
llama-server -m Thinklite-27B-High-Q4_K_M.gguf --mmproj mmproj-Thinklite-27B-High-F16.gguf --jinja -c 32768
```

**Adapters** (`adapters` branch): the LoRA adapters for the full-dataset training (`adapters/full`) and the
smaller-dataset training (`adapters/small`). Loading the full-dataset adapter with PEFT gives Thinklite-27B Max; High
and Low are those adapters with the update-selective merge above, so use the merged weights for them.

## Files

- `main`: Thinklite-27B High, merged bf16 safetensors with tokenizer, chat template and processor files; plus
  `Thinklite-27B-High-Q4_K_M.gguf` and `mmproj-Thinklite-27B-High-F16.gguf` (the vision projector).
- `low`: Thinklite-27B Low, merged bf16 safetensors.
- `max`: Thinklite-27B Max, merged bf16 safetensors.
- `adapters`: the LoRA adapters.

## Limitations

- Evaluated at `reasoning_effort=low`. Other effort levels are supported but are not part of the tables above.
- The vision encoder is unchanged from Qwen3.8-27B; training data was text only.
- As with any fine-tune, check the model on your own tasks before relying on it.

## License

Apache-2.0, the license of the base model [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B).