Thinklite-27B NVFP4
This is an NVFP4/FP8 quantization of Sparset/Thinklite-27B at revision 2aa81615c583852cd561164946f1a458ed21b13f. It uses LLM Compressor's Qwen3.8 AWQ + GPTQ recipe, calibrated on 512 text samples from mlabonne/open-perfectblend with a 4,096-token maximum. Most MLP projections use NVFP4; attention and selected later MLP projections use FP8, the vision tower remains higher precision, and the KV cache uses calibrated FP8.
On the full GSM8K test set, this NVFP4 build scored 1269/1,319 (96.21%), compared with 1275/1,319 (96.66%) for the original BF16 model (-0.45 percentage points), under the matched thinking-enabled evaluation described below. On MMLU-Pro, BF16 scored 642/1,000 and NVFP4 648/1,000; on IFEval prompt-strict, BF16 scored 481/541 and NVFP4 485/541. The source checkpoint is BF16, rather than FP16. This single benchmark does not establish overall or image accuracy; calibration was text-only.
Matched GSM8K evaluation
| Model | Correct | Accuracy | Length-capped answers |
|---|---|---|---|
| Original Thinklite-27B BF16 | 1275/1,319 | 96.66% | 0 |
| This NVFP4/FP8 build | 1269/1,319 | 96.21% | 0 |
Both runs used vLLM 0.30 on one RTX PRO 6000 Blackwell, thinking enabled with reasoning_effort=xhigh, temperature 1.0, top_p 0.95, top_k 20, FP8 KV cache and one sample per question. Each item used seed 42 plus its index. The input was the full GSM8K test split, pinned to revision 740312add88f781978c0658806c59bc2815b9866; frozen input SHA-256: d0b8410b286a771aa8827e6ee30aca724e449c335ddd4bbac53690ea51acff89. Prompts requested a final numerical answer. Generations were capped at 8,192 tokens with a 10,240-token context; length-capped or missing-final-answer outputs were scored incorrect. The grader extracts the last number after the closing thinking tag and compares it to the gold answer. This protocol does not reproduce NVIDIA's multi-benchmark evaluation or the source card's cap and rerun rules.
The calibration dataset was screened for exact normalized GSM8K question containment: none of the 1,319 questions matched the 512 calibration records. This check does not rule out paraphrases or overlap in prior training. Raw model completions and scores are in the evaluation files.
Additional paired evaluations
| Benchmark | BF16 | NVFP4 | Delta, percentage points |
|---|---|---|---|
| MMLU-Pro, stratified 1,000 | 64.20% (642/1,000) | 64.80% (648/1,000) | +0.60 |
| IFEval, prompt-level strict | 88.91% (481/541) | 89.65% (485/541) | +0.74 |
Both models used the same vLLM 0.30 settings as the GSM8K comparison above: thinking enabled, xhigh effort, temperature 1.0, top_p 0.95, top_k 20, one sample per item, 8,192 output tokens and a 10,240-token context. Length-capped answers and answers without a final response were marked incorrect. These settings and prompts differ from the source model card's full protocol.
MMLU-Pro uses a reproducible category-proportional 1,000-question subset of TIGER-Lab/MMLU-Pro, revision b189ec765aa7ed75c8acfea42df31fdae71f97be. The source card did not identify its stratified subset, so ours is not claimed to be the same subset. Our zero-shot prompt requests a final letter; only a standalone A–J answer is accepted. Frozen subset SHA-256: da457ae531d2eebf9b7701ad9e74a47fd2cc3d2446fcf969b2d04509b6ebb6b3.
IFEval uses all 541 prompts and Google's official strict grader, reporting the fraction of prompts satisfying every instruction. Thinking text was removed before grading; length-capped responses were scored incorrect. Input SHA-256: 67ffeee0fcb87c317c5b08a2de85557b4a7e96ada6178aa645b4954fe4b53d49. The grader used nltk 3.10.4, langdetect 1.0.9 (seed 0), immutabledict 4.3.1 and absl-py 2.5.1.
Exact normalized question/prompt containment checks found no matches between these evaluation items and the 512 calibration records. This does not rule out paraphrases or prior-training overlap. Results, frozen MMLU-Pro subset, and evaluation scripts are available in the evaluation files.
Original Thinklite-27B model card
The following material is reproduced from Sparset/Thinklite-27B, licensed Apache-2.0. Its benchmark results, speed claims, examples and usage instructions describe the original model and do not establish results for this NVFP4 build.
Qwen3.8-27B that thinks with half the tokens and keeps 99.6% of its accuracy. At
reasoning_effort="xhigh", Thinklite-27B uses 47% less reasoning than Qwen3.8-27B on average over six benchmarks and writes half as many output tokens, so its time-to-final-output (TTFO) is about 2x shorter, while it keeps 99.6% of the base model's average accuracy (89.8 against 90.2). Atreasoning_effort="low"it uses 28% less reasoning and keeps 97.5% of the base model's accuracy (2.2 points below).
Check out the official Thinklite-27B-GGUF, a 16.5 GB Q4_K_M for llama.cpp and Ollama that uses 48% less reasoning than Qwen3.8-27B quantized the same way at 99.3% of its accuracy.
| Model | reasoning_effort = xhigh | reasoning_effort = low | ||
|---|---|---|---|---|
| Reasoning vs base | TTFO speed-up | Reasoning vs base | TTFO speed-up | |
| Thinklite-27B | ↓ 47.0% | 2.0x | ↓ 28.0% | 1.3x |
| ThinkingCap-Qwen3.8-27B | ↓ 45.8% | 1.7x | ↓ 16.4% | 1.2x |
| Swift-1.5-Qwen3.8-27b | ↓ 35.6% | 1.6x | ↓ 12.7% | 1.1x |
- Reasoning vs base: the change in mean reasoning tokens per answer against Qwen3.8-27B at the same effort, averaged over the six benchmarks below.
- TTFO speed-up: TTFO (time-to-final-output) is how long a user waits until the model's complete final output is there, thinking included. The speed-up is Qwen3.8-27B's TTFO divided by the model's, over all 4,291 requests for one user. All four models share the Qwen3.8-27B architecture and decode at the same speed (Thinklite-27B 26.3 vs base 26.2 tokens/s for a single user on the same GPU), and the prompts are identical, so TTFO falls in proportion to the tokens a model writes.
Side-by-side comparisons
Debugging
Qwen3.8-27B and Thinklite-27B get the same debugging prompt (retry code plus two logs: find the duplicate charges to refund and the fix) with the same seed, each model on one NVIDIA RTX PRO 6000 serving 20 requests at once. Both answer correctly; Thinklite-27B is done in 2:36.9 with 3,381 tokens, the base model in 11:04.9 with 14,313 tokens (4.2x). The video plays at 22x speed and the clocks show the measured time. This is the largest speed-up of 20 paired runs of this prompt. Thinklite-27B was correct in 19 of the 20 runs and the base model in all 20; over the 19 correct runs the median speed-up was 2.7x.
Building a 3D physics lab
Both models get the same objective: build an interactive 3D science lab as a single HTML file in which everything has
to follow realistic physics. Objects have real sizes and masses and fall under real gravity (9.81 m/s²): a hinged door
swings with its momentum and stops at its frame, glassware tips over when bumped and shatters when it hits something
hard, a rubber ball bounces high while a heavy steel ball barely bounces, a block tower topples and a pendulum swings.
Both models run at reasoning_effort="xhigh" on the same NVIDIA RTX PRO 6000. The numbers are the total tokens and
generation time each model needed to reach a working version of the lab: Thinklite-27B 87,134 tokens in 56.7 min, the
base model 127,661 tokens in 87.6 min (1.5x). The video shows one working game per model; it is not a measure of how
often each model succeeds.
How it works
Reasoning models spend much of their output on thinking that does not change the answer: restating the question,
repeating a step, re-checking a result that was already checked. Thinklite-27B is fine-tuned from Qwen3.8-27B on its
own verified-correct answers to reach the same answers by a shorter route. Only the thinking gets shorter; the visible
answer keeps its full length and format. It is a drop-in replacement for Qwen3.8-27B with the same chat template,
tokenizer and reasoning_effort control.
Benchmarks
Setup:
We ran all four models ourselves on one frozen suite: identical prompts, chat template, sampling, token caps and scoring. Compared models, both also built on Qwen3.8-27B and run in bf16: bottlecapai/ThinkingCap-Qwen3.8-27B (revision
7891e03) and ukisai/Swift-1.5-Qwen3.8-27b (revisionb4c84d42).vLLM 0.30.0, bf16 weights, FP8 KV cache, temperature 1.0, top_p 0.95, top_k 20, 64 concurrent requests, one NVIDIA RTX PRO 6000 Blackwell (96 GB). All answers were scored together on one machine.
One sample per question (two on AIME 2025). Answers that hit their cap were rerun once with a 131,072-token cap; truncated or unparseable answers count as incorrect.
GSM8K: 1,319 problems
AIME 2025: 30 problems x 2 samples, avg@2
MMLU-Pro: 1,000-question stratified subset
IFEval: 541 prompts, prompt-level strict
LiveCodeBench v6 (LCB): 131 problems (2025-02 to 2025-04), pass@1
BFCL v4: 1,240 non-live function-calling cases
reasoning_effort="xhigh"
| Model | GSM8K | AIME 2025 | MMLU-Pro | IFEval | LCB v6 | BFCL v4 | Average |
|---|---|---|---|---|---|---|---|
| Accuracy (%) | |||||||
| Thinklite-27B | 96.8 | 95.0 | 84.4 | 89.1 | 84.7 | 88.9 | 89.82 |
| ThinkingCap-Qwen3.8-27B | 97.0 | 100.0 | 86.5 | 90.9 | 84.7 | 89.0 | 91.37 |
| Swift-1.5-Qwen3.8-27b | 96.9 | 100.0 | 86.6 | 90.9 | 84.7 | 87.6 | 91.12 |
| Qwen3.8-27B | 97.2 | 93.3 | 86.3 | 91.5 | 84.0 | 88.7 | 90.17 |
| Reasoning tokens vs Qwen3.8-27B | |||||||
| Thinklite-27B | ↓ 63% | ↓ 43% | ↓ 69% | ↓ 41% | ↓ 38% | ↓ 28% | ↓ 47.0% |
| ThinkingCap-Qwen3.8-27B | ↓ 68% | ↓ 33% | ↓ 58% | ↓ 51% | ↓ 20% | ↓ 44% | ↓ 45.8% |
| Swift-1.5-Qwen3.8-27b | ↓ 40% | ↓ 34% | ↓ 44% | ↓ 44% | ↓ 32% | ↓ 20% | ↓ 35.6% |
Mean reasoning tokens per answer
| Benchmark | Thinklite-27B | ThinkingCap-Qwen3.8-27B | Swift-1.5-Qwen3.8-27b | Qwen3.8-27B |
|---|---|---|---|---|
| GSM8K | 163 (-63%) | 141 (-68%) | 267 (-40%) | 443 |
| AIME 2025 | 11,236 (-43%) | 13,228 (-33%) | 12,961 (-34%) | 19,642 |
| MMLU-Pro | 1,343 (-69%) | 1,785 (-58%) | 2,404 (-44%) | 4,295 |
| IFEval | 1,082 (-41%) | 891 (-51%) | 1,031 (-44%) | 1,830 |
| LiveCodeBench v6 | 17,079 (-38%) | 21,936 (-20%) | 18,770 (-32%) | 27,589 |
| BFCL v4 | 143 (-28%) | 113 (-44%) | 159 (-20%) | 200 |
| Average change | -47.0% | -45.8% | -35.6% | - |
| All output tokens, 4,291 requests | 5,822,599 (-50%) | 6,724,080 (-42%) | 7,319,314 (-37%) | 11,533,518 |
reasoning_effort="low"
| Model | GSM8K | AIME 2025 | MMLU-Pro | IFEval | LCB v6 | BFCL v4 | Average |
|---|---|---|---|---|---|---|---|
| Accuracy (%) | |||||||
| Thinklite-27B | 96.7 | 81.7 | 83.3 | 84.3 | 71.0 | 91.2 | 84.70 |
| ThinkingCap-Qwen3.8-27B | 97.2 | 86.7 | 84.8 | 85.8 | 72.5 | 91.5 | 86.40 |
| Swift-1.5-Qwen3.8-27b | 97.1 | 90.0 | 84.7 | 86.0 | 73.3 | 91.2 | 87.04 |
| Qwen3.8-27B | 97.4 | 90.0 | 84.4 | 86.1 | 72.5 | 90.8 | 86.88 |
| Reasoning tokens vs Qwen3.8-27B | |||||||
| Thinklite-27B | ↓ 21% | ↓ 46% | ↓ 23% | ↓ 41% | ↓ 17% | ↓ 19% | ↓ 28.0% |
| ThinkingCap-Qwen3.8-27B | ↓ 12% | ↓ 23% | ↓ 18% | ↓ 19% | ↓ 15% | ↓ 12% | ↓ 16.4% |
| Swift-1.5-Qwen3.8-27b | ↓ 5% | ↓ 22% | ↓ 13% | ↓ 15% | ↓ 13% | ↓ 7% | ↓ 12.7% |
Mean reasoning tokens per answer
| Benchmark | Thinklite-27B | ThinkingCap-Qwen3.8-27B | Swift-1.5-Qwen3.8-27b | Qwen3.8-27B |
|---|---|---|---|---|
| GSM8K | 144 (-21%) | 160 (-12%) | 173 (-5%) | 183 |
| AIME 2025 | 6,521 (-46%) | 9,324 (-23%) | 9,416 (-22%) | 12,125 |
| MMLU-Pro | 854 (-23%) | 908 (-18%) | 963 (-13%) | 1,103 |
| IFEval | 386 (-41%) | 537 (-19%) | 557 (-15%) | 659 |
| LiveCodeBench v6 | 8,882 (-17%) | 9,183 (-15%) | 9,341 (-13%) | 10,741 |
| BFCL v4 | 78 (-19%) | 85 (-12%) | 90 (-7%) | 96 |
| Average change | -28.0% | -16.4% | -12.7% | - |
| All output tokens, 4,291 requests | 3,748,880 (-23%) | 4,121,634 (-16%) | 4,303,835 (-12%) | 4,896,249 |
Image understanding
Thinklite-27B accepts images like Qwen3.8-27B. Its training data was text only and its vision encoder is
byte-for-byte the base model's, so we checked that the fine-tuned language model still reads what it sees: 88 questions
from six public image datasets, each answerable only from the picture (a chart, a scanned document, an infographic, a
math figure, an exam diagram or a photo). Both models got the same image and question at reasoning_effort="low",
with the same seeds and sampling as the benchmarks above.
| Model | ChartQA | DocVQA | InfoVQA | MathVista | MMMU | A-OKVQA | Average |
|---|---|---|---|---|---|---|---|
| Score (%) | |||||||
| Thinklite-27B | 100.0 | 100.0 | 91.1 | 87.5 | 100.0 | 87.5 | 94.35 |
| Qwen3.8-27B | 100.0 | 100.0 | 88.1 | 87.5 | 100.0 | 87.5 | 93.85 |
| Reasoning tokens vs Qwen3.8-27B | |||||||
| Thinklite-27B | ↓ 15% | ↓ 15% | ↓ 32% | ↓ 36% | ↓ 30% | ↓ 15% | ↓ 24.0% |
- 16 questions per dataset (12 for InfographicVQA and MMMU), a seeded random sample of each validation or testmini split. Scores: ChartQA relaxed accuracy (numbers within 5%), DocVQA and InfographicVQA ANLS, multiple-choice or exact accuracy for MathVista, MMMU and A-OKVQA. No answer was cut off or unparseable.
- Both models answered the same 83 of 88 questions correctly; the only score difference is partial ANLS credit on one InfographicVQA answer.
- 18 of the questions (3 per dataset) also ran at
reasoning_effort="xhigh": Thinklite-27B answered 17 correctly and Qwen3.8-27B 16, with 52% less reasoning per answer (758 vs 1,565 tokens). - This is a check, not a full benchmark: one question moves a dataset's score by 6 to 8 points. It shows that image understanding is kept, not that it improved.
Usage
vLLM
vllm serve Sparset/Thinklite-27B --reasoning-parser qwen3 --max-model-len 135168 --kv-cache-dtype fp8
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="-")
reply = client.chat.completions.create(
model="Sparset/Thinklite-27B",
messages=[{"role": "user", "content": "How many primes are there below 100?"}],
temperature=1.0, top_p=0.95,
extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "xhigh"}},
)
print(reply.choices[0].message.content)
Set reasoning_effort to "low", "medium" or "xhigh", exactly as with Qwen3.8-27B. The benchmarks used
temperature 1.0, top_p 0.95 and top_k 20, as recommended for the base model.
Transformers
from transformers import AutoModelForImageTextToText, AutoProcessor
model = AutoModelForImageTextToText.from_pretrained("Sparset/Thinklite-27B", dtype="auto", device_map="auto")
processor = AutoProcessor.from_pretrained("Sparset/Thinklite-27B")
Files
Merged bf16 safetensors with the tokenizer, chat template and processor files of Qwen3.8-27B.
Limitations
- One benchmark run per effort; the small benchmarks (AIME 2025 with 60 answers, LiveCodeBench with 131) carry wide confidence intervals.
- At xhigh, accuracy is below the base model on IFEval (89.1 vs 91.5) and MMLU-Pro (84.4 vs 86.3).
- At low effort, the average accuracy is 2.2 points below the base model, mostly from AIME 2025 (81.7 vs 90.0).
- Long single-answer code was less reliable than the base model's: in the game test above, 1 of 5 games fully worked against 3 of 5.
- The vision encoder is unchanged from Qwen3.8-27B and training data was text only; image understanding was checked on 88 questions, not a full vision benchmark.
- No safety tuning beyond the base model's. Check the model on your own tasks before relying on it.
License
Apache-2.0, the license of the base model Qwen/Qwen3.8-27B.
- Downloads last month
- 53