---
license: apache-2.0
base_model: Jackrong/Qwopus3.8-27B-Coder-Flash
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
tags: [qwen3_8, qwopus, coding, tool-use, gptq, int4, mtp, speculative-decoding, intel, xpu, arc, vllm]
---

# Qwopus3.8-27B-Coder-Flash, GPTQ INT4 baked (v1.1.1)

[Jackrong/Qwopus3.8-27B-Coder-Flash](https://huggingface.co/Jackrong/Qwopus3.8-27B-Coder-Flash) quantized from BF16 to
INT4 by Launch80. The `lm_head` and the MTP draft head are INT4 **on disk** too, so a single **Intel Arc Pro B65 32 GB**
serves it at the full **262,144-token** context, and stock vLLM loads it with no weight patches.

**This model is a work in progress, and the full story is public:**

- **The study, with every number, the bug we found and how we fixed it:** https://launch80.com/blog/qwopus-coder-int4-bake-study
- **Follow along and ask questions in the Launch80 Discord:** https://discord.gg/launch80

| version | what changed | status |
|---|---|---|
| v1.0 (tag `v1.0`) | GPTQ INT4 body calibrated with tool-call data, baked INT4 head and MTP | released 2 Nov 2026 |
| v1.0.1 (tag `v1.0.1`) | card only, weights unchanged: all seven tasks measured on this exact checkpoint, served as shipped (MTP k=6) | released 2 Nov 2026 |
| v1.1 (tag `v1.1`) | weights unchanged; adds a fast-decode serving profile with our INT4 DFlash2 drafter (k=7, 128k): 89.3 tok/s on the B65 | released 3 Nov 2026 |
| **v1.1.1** (this revision, tag `v1.1.1`) | card only, weights unchanged: one more vLLM-XPU boot patch removes a per-step host sync in the verify step; fast profile **92.5 / 96.0 tok/s** (two quick runs) | released 3 Nov 2026 |

## What's in it

| tensor | source (BF16) | this repo |
|---|---|---|
| 400 body linears | BF16 | GPTQ INT4 sym g128 (GPTQModel 7.3.2) |
| `lm_head` (248,320 × 5,120) | BF16, 2.54 GB | GPTQ INT4 sym g128, Hessian-calibrated |
| `mtp.*` draft: `fc`, q/k/v/o, gate/up/down | BF16 | GPTQ INT4 sym g128, Hessian-calibrated |
| embeddings, norms, vision tower, GDN `in_proj_a`/`in_proj_b` | BF16 | unchanged (byte-identical) |

17.0 GB on disk. The tokenizer, chat template and processor files are the source repo's, unchanged.

**Body.** GPTQModel 7.3.2 ran on the Arc Pro B65 itself with these settings:

- 4-bit, symmetric, group 128
- `desc_act` off, `act_group_aware` on
- damping 0.05, seed 608
- 512 × 2048-token calibration samples: 153 C4, 204 code, 53 chat and 102 tool-call conversations. The tool-call samples are from [NousResearch/hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1) `@dae3e1d`, rendered through the model's own chat template with `tools=`, half of them parallel calls.

The calibration sha256 is in `quant-meta.json`.

**Head and MTP bake.** These steps follow the layout of our earlier bakes:

1. Forward hooks inside vLLM captured Hessians on wikitext, chat and code slices.
2. Each of the nine linears was GPTQ-quantized under a grid of dampings and slice mixes.
3. The setting with the lowest output error on held-out rows was kept.

Every baked linear beats round-to-nearest (RTN):

| linear | baked | RTN |
|---|--:|--:|
| `lm_head` | **4.68%** | 8.18% |
| `mtp.fc` | **6.98%** | 11.47% |
| `mtp…q_proj` | **2.68%** | 4.44% |
| `mtp…k_proj` | **6.05%** | 10.08% |
| `mtp…v_proj` | **5.11%** | 8.28% |
| `mtp…o_proj` | **6.67%** | 13.04% |
| `mtp…gate_proj` | **3.67%** | 6.35% |
| `mtp…up_proj` | **5.70%** | 9.56% |
| `mtp…down_proj` | **7.57%** | 11.96% |

## Quality vs BF16 (pre-registered)

We wrote the thresholds down and hashed them before any quantized model was evaluated. The BF16 reference ran on a rented
RTX PRO 6000 (vLLM 0.30, CUDA); this checkpoint ran on the Arc Pro B65 (vLLM 0.30, XPU), served exactly as shipped,
with MTP k=6 on for every task and for the logprobs. Full study:
https://launch80.com/a/84829a3a-a0b2-407f-ab38-59cd2d5006fa

| # | measure | bar | this checkpoint | |
|---|---|---|---|---|
| T1 | wikitext-2 perplexity | ≤ +3% | +1.72% | pass |
| T2 | code-corpus perplexity | ≤ +3% | +1.83% | pass |
| T3 | mean next-token KL divergence (1.05M tokens, top-64) | ≤ 0.05 | 0.0412 | pass |
| T4 | top-1 next-token agreement | ≥ 93% | 92.84% | **fail** (by 0.16 pt) |
| T5 | seven tasks, paired pass@1, greedy | Δ ≥ −2 pts or CI includes 0 | 7/7 tasks (table below) | pass |
| T6 | KLD vs Jackrong's Q4_K_M GGUF | ours ≤ theirs | 0.0412 vs 0.0250 | **fail** |
| T7 | greedy, spec decoding on = off | identical | 26/32; all 6 differences are at a tie (3 exact, 3 a 0.0156 FP16 step) | fail as written |

| task | BF16 | this checkpoint | Δ pts [95% CI] |
|---|--:|--:|---|
| GSM8K | 86.96 | 86.66 | −0.30 [−1.21, +0.61] |
| IFEval | 80.22 | 80.78 | +0.55 [−2.22, +3.14] |
| MMLU-Pro subset | 71.80 | 70.90 | −0.90 [−2.70, +0.90] |
| HumanEval+ | 90.24 | 88.41 | −1.83 [−5.49, +1.83] |
| MBPP+ | 76.98 | 75.66 | −1.32 [−3.17, +0.53] |
| BFCL subset (400) | 89.00 | 88.25 | −0.75 [−3.00, +1.50] |
| LiveCodeBench v6 slice | 41.14 | 40.57 | −0.57 [−5.14, +4.00] |

**T6, against [Jackrong's Q4_K_M GGUF](https://huggingface.co/Jackrong/Qwopus3.8-27B-Coder-Flash-GGUF): fail.** On the same
512 × 2048 tokens, Q4_K_M stays closer to BF16: mean KLD 0.0250 vs our 0.0412, top-1 agreement 94.92% vs 92.84%, perplexity
+0.73% vs +1.72% (wikitext-2). The files are about the same size (16.8 vs 17.1 GB); Q4_K_M keeps 67 tensors at 6 bits, while
this checkpoint carries a BF16 vision tower and fp16 embeddings and a uniform 4-bit g128 body. Method: the GGUF ran in llama.cpp
(b11723, SYCL) on the B65, its top-64 logprobs were scored with the same code against the same BF16 reference. This differs from
the registered method (a BF16 GGUF base, which needed a rental) and is dated in the thresholds file. Q8_0 measured the same way
gives 0.0024, so the cross-engine term is small. Pick Q4_K_M for fidelity on llama.cpp; pick this build for vLLM with
speculative decoding.

Needle retrieval: 100% at 32k and at 128k. MTP draft acceptance over the whole suite: 46.8% of drafted tokens
(80.2 / 62.2 / 48.0 / 37.4 / 29.5 / 23.3% by draft position), about 3.8 tokens per verify step under greedy decoding.

**Disclosure: the tool-call calibration is a post-hoc fix.** The first bake, with no tool calls in its calibration set,
passed six of the seven tasks but lost 7.75 points on BFCL. It ended its turn after the first of several parallel calls.
We traced that to the calibration data, recalibrated the body, and rebaked the head and MTP. This checkpoint is that rebake,
and the table above measures it on all seven tasks. The change was made after seeing results, so treat it as post hoc.
The blog post has the whole investigation.

## Speed (Arc Pro B65 32 GB, vLLM 0.30.0 XPU, fp8 KV, single stream)

Two serving profiles, same weights. Both are quick BetterBench runs (1 warmup + 5 passes per category):

| profile | drafter | context | weighted decode | page |
|---|---|---|--:|---|
| **fast decode** + GDN no-sync patch (v1.1.1) | [Launch80/Qwen3.8-27B-DFlash2-Int4-baked-drafter](https://huggingface.co/Launch80/Qwen3.8-27B-DFlash2-Int4-baked-drafter), k=7 | 131,072 | **92.5 / 96.0 tok/s** (two runs) | https://launch80.com/a/81dea056-a018-4e48-9f5e-b89b039f368d |
| fast decode, v1.1 (no patch) | same | 131,072 | 89.3 tok/s | https://launch80.com/a/54608011-fd4d-4c68-9d32-868a2fcdd03b |
| full context | built-in MTP head, k=6 | 262,144 | 75.8 tok/s | https://launch80.com/a/30aa8ae2-f00c-409a-a4a9-2e298fc81f67 |

The fast profile gains most on structured output (5.2 accepted tokens per step on JSON, 4.7 on summaries) and least on
long thinking and prose (2.8 and 2.5). It costs context: with the drafter loaded, the 32 GB card fits 128k, not 262k.
Spec-decoding parity (T7) for the fast profile with the v1.1.1 patch: 30 of 32 greedy outputs identical to spec-off, and both
differences are at exact top-2 ties (v1.1 without the patch: 28 of 32, all ties), the same pattern as the MTP profile.

What v1.1.1 fixes: on every speculative step, vLLM's GDN (linear-attention) metadata builder indexed GPU tensors with a CPU
boolean mask. That forces the host to wait for the GPU, which cancelled async scheduling and left ~7 ms of host time in each
step at k=7. `patch_gdn_spec_nosync.py` replaces the mask with the equivalent slice: 47.9 → 44.4 ms per step on a single
prompt. We also tried a Triton INT4 GEMM for the 9-to-16-row verify shapes; it ran 3.5× slower than oneDNN, so the fast profile
stays at k=7. Quick BetterBench runs vary by a few tok/s between runs, so both runs are shown.

The MTP head is byte-identical to base Qwen3.8-27B (the fine-tune never trained it). We also fine-tuned the DFlash2
drafter on 10,000 of this model's own answers; no checkpoint beat the stock drafter, so v1.1 uses the stock drafter, INT4-baked.
Drafter study: https://launch80.com/a/a7b8a95a-e2c5-473c-b017-310232fc56e0

## Serving

Full-context profile (75.8 tok/s, 262k context):

```bash
vllm serve Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked \
  --quantization gptq --dtype float16 --kv-cache-dtype fp8 --max-model-len 262144 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":6}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3
```

Fast-decode profile (92.5 / 96.0 tok/s, 128k context):

```bash
vllm serve Launch80/Qwopus3.8-27B-Coder-Flash-Int4-baked \
  --quantization gptq --dtype float16 --kv-cache-dtype fp8 --max-model-len 131072 \
  --speculative-config '{"method":"dflash","model":"Launch80/Qwen3.8-27B-DFlash2-Int4-baked-drafter","num_speculative_tokens":7,"quantization":"gptq"}' \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3
```

The drafter must run in bf16 while this target runs in fp16: on XPU that needs the `patch_dflash_mixed_dtype.py` boot
patch from [launch80/B65](https://github.com/launch80/B65) (on CUDA, serve both in bf16 with `--dtype bfloat16`).

Every number above was measured with these configurations (vLLM 0.30.0 XPU image), plus three vLLM-XPU correctness
patches for speculative decoding (MTP nightly, MTP boundary, GDN mixed-split) from the same repository. The fast profile's
v1.1.1 number also needs `patch_gdn_spec_nosync.py` (run it after the MTP boundary patch); without it the same command gives 89.3. Tool calling
needs `qwen3_xml`, not `hermes`. Pin a revision (`--revision v1.1.1`) if you want this exact build; `main` moves as new
versions ship.

## License

Apache-2.0, as the source model and the Qwen3.8-27B base are.

## Credits

Qwen (Qwen3.8-27B) · Unsloth (the base Jackrong built on) · Jackrong (Qwopus3.8-27B-Coder-Flash) · NousResearch
(hermes-function-calling-v1, used for calibration) · GPTQModel · Inco AI / z-lab (the DFlash2 drafter) · SpecForge (the drafter fine-tune
experiment) · Wrapzii (asked for this bake, and the on-disk bake
layout it follows).

## Community

Questions, results, or a model you want tested? Join the Launch80 Discord: https://discord.gg/launch80

If this work is useful to you, please support the server: https://launch80.com/supporter