DaisyRose-27B-GPTQ-Int4

RadixArk/DaisyRose-27B quantized to GPTQ Int4 (group_size=32). DaisyRose is RadixArk's reinforcement-learning fine-tune of Qwen/Qwen3.8-27B for long-horizon agent tasks.

This is the DaisyRose sibling of palmfuture/Qwen3.8-27B-GPTQ-Int4 and palmfuture/Qwen3.8-27B-pi-GPTQ-Int4: the same quantization settings and file layout, with the MTP (multi-token prediction) weights included for speculative decoding. The calibration set is new: it adds agent, tool-call and reasoning sessions in the model's chat format, and Thai text.

What has and has not been measured

  • RadixArk reports gains over Qwen3.8-27B on agent benchmarks (for example Terminal-Bench 2.1 73.0 โ†’ 75.5, DeepSWE 42.2 โ†’ 46.2). Those are their numbers for the unquantized and FP8 model. They have not been reproduced on this quantization. DaisyRose's weights differ from the base model's by 0.10-0.15% per layer (relative Frobenius norm, our measurement), which is small next to what 4-bit quantization changes, so do not assume the gains carry over: test on your own agent workload.
  • What we did measure is below: quantization loss, perplexity-style scores in Thai and English, and speed.

Quality

Metric This model Qwen3.8-27B-pi-GPTQ-Int4 Qwen3.8-27B-GPTQ-Int4
GPTQ modules quantized / RTN fallback 400 / 0 400 / 0 400 / 0
Thai text, mean token NLL (lower is better) 1.558 1.535 1.575
English text, mean token NLL 1.720 1.750 1.745
Short factual/arithmetic QA (6 items) 5/6 4/6 5/6
  • NLL was measured on our own small Thai and English text sets, so compare the columns with each other, not with other models' numbers. Differences of 1-2% between columns are about what we see between two quantization runs of the same model.
  • GPTQ loss on this run's calibration set: mean 6.16e-05, max 3.67e-04. It is not comparable with the loss on the sibling cards, which used a different calibration set. Quantizing DaisyRose with the siblings' set gave mean 1.52e-04 and max 8.83e-04, the same as the pi checkpoint (1.52e-04 / 8.86e-04).
  • With the sibling's calibration set instead of this one, the same scores came out at 1.547 (Thai), 1.724 (English) and 5/6: the new calibration set made no difference we could measure with these tests. None of them measures agent ability.
  • Thai users: like the base model, this one sometimes writes a Chinese character in the middle of Thai text. In our test, 9 of 96 long Thai articles had at least one at temperature 0.7 / top_p 0.8, and 35 of 384 short stories at temperature 1.0.

How it was made

  1. Quantize RadixArk/DaisyRose-27B (bf16) with GPTQModel 7.0.0: 4 bit, group_size=32, symmetric, desc_act=False, damp_percent=0.05, true_sequential=True.
  2. Add back what GPTQModel's save() leaves out: mtp.safetensors and its 15 entries in model.safetensors.index.json. DaisyRose's mtp.* tensors and vision tower are bit-identical to Qwen3.8-27B's, so the MTP file is the same as in the sibling repositories.
  3. Tokenizer files (tokenizer.json, tokenizer_config.json) and generation_config.json are the ones from the sibling repositories. The vocabulary is the same as upstream; tokenizer_class is PreTrainedTokenizerFast (see the note under Calibration).

Quantized (int4, g32):

  • Self-attention: q_proj, k_proj, v_proj, o_proj
  • Linear attention (Gated DeltaNet layers): in_proj_qkv, in_proj_z, out_proj
  • MLP: gate_proj, up_proj, down_proj

Kept bf16:

  • Linear-attention state dynamics: A_log, conv1d, dt_bias, in_proj_b, in_proj_a, in_proj_ba, linear_attn.norm
  • q_norm, k_norm, input_layernorm, post_attention_layernorm
  • Multi-token prediction head mtp.* (15 tensors, in mtp.safetensors)
  • Vision encoder model.visual.*
  • Embeddings and lm_head

Calibration

276 samples, 321,541 tokens, cut at 2,048 tokens each (code at 1,024). Conversations are rendered through the Qwen3.8 chat template, so the model sees tool definitions, <tool_call> blocks and <think> reasoning during calibration the way it sees them when serving. Thai is 27.5% of the tokens.

Source Samples Tokens Purpose
SWE-bench/SWE-smith-trajectories (tool split) 46 94,208 Coding-agent sessions with tool calls, half with reasoning
NousResearch/hermes-function-calling-v1 26 36,908 Function calling, re-rendered in Qwen's tool-call format
open-thoughts/OpenThoughts-114k 14 28,226 Long reasoning
airesearch/WangchanThaiInstruct 50 44,284 Thai instructions
wikimedia/wikipedia 20231101.th 30 44,242 Native Thai text
allenai/c4 40 22,778 General English text
allenai/tulu-3-sft-mixture 30 24,990 Instruction following
codeparrot/codeparrot-clean-valid 25 20,825 Code
HuggingFaceH4/MATH-500 15 5,080 Mathematical reasoning

If you calibrate with Thai text yourself: upstream's tokenizer_config.json declares tokenizer_class: "Qwen2Tokenizer". Under Transformers 5.9.0 that class split our Thai text into about twice as many tokens as tokenizer.json does (50 against 26 on one sentence); English and code were unaffected. Compare AutoTokenizer against tokenizers.Tokenizer.from_file("tokenizer.json") on a Thai sentence before calibrating.

Hardware and toolchain

4ร— NVIDIA RTX 3060 12GB, Intel i9-7900X, 32 GB RAM. Quantization took 105 minutes, with the cached calibration activations kept in RAM (calibration_data_device="cpu") because they do not fit beside a layer on a 12 GB card at this calibration size.

Component Version
GPTQModel 7.0.0
Transformers 5.9.0
PyTorch 2.12.0+cu130
Python 3.13 (free-threading)

auto_forward_data_parallel=False in QuantizeConfig: on this hardware GPTQModel's parallel calibration forward crashed in most runs.

If you re-quantize this model with GPTQModel, register mtp.* again in model.safetensors.index.json, pointing at mtp.safetensors. Without those entries speculative decoding is silently not used.

Usage

We serve it with HyperQwen (below). The stock vLLM command here follows the sibling repositories' cards and has not been run against this repository as uploaded:

vllm serve palmfuture/DaisyRose-27B-GPTQ-Int4 \
  --served-model-name DaisyRose-27B-GPTQ \
  --quantization gptq \
  --dtype float16 \
  --tensor-parallel-size 4 \
  --max-model-len 32768 \
  --enable-chunked-prefill \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Adjust --tensor-parallel-size and --max-model-len to your GPUs. The weights need about 20 GB of VRAM before the KV cache. For sampling settings and thinking modes, follow the DaisyRose card.

Speed on the hardware above

Measured with HyperQwen (ghcr.io/syv-ai/hyperqwen, a patched vLLM 0.33), tensor parallel over the four cards, MTP with 3 speculative tokens, 120 W per card:

tok/s
One request, greedy about 107
8 concurrent requests, total about 440
Prompt processing about 1,300

The base model's MTP head drafts DaisyRose as well as it drafts the base model: 2.6 tokens per step in these runs, the same as on the sibling checkpoints. The runs also used a reduced-vocabulary draft head built for that server, which is not part of this repository; expect less from the files here alone, and different numbers on other hardware or other vLLM versions.

Licence and credits

Apache-2.0, as are the parents. Changes from RadixArk/DaisyRose-27B: the listed linear layers are quantized to 4 bit, mtp.* is stored in a separate file (mtp.fc.weight as float16), and the tokenizer and generation config files are replaced as described above.

Downloads last month
2
Safetensors
Model size
28B params
Tensor type
BF16
ยท
I32
ยท
F16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for palmfuture/DaisyRose-27B-GPTQ-Int4

Base model

Qwen/Qwen3.8-27B
Quantized
(38)
this model