Model Card for Qwen3.8-27B (GGUF, with vision)

GGUF builds of Qwen3.8-27B packaged to run entirely on device, with the vision tower, the tokenizer, the merge table, the chat template and the sampling card all embedded in the single weight file. Nothing calls out to a server at inference time and there are no sibling files to keep in sync.

This repository holds the quantized weights only, not training data or the original checkpoint.

File format

.ggxf is GGUF plus the tensors and metadata keys this project adds, so these files load in Gadeon and are not expected to load in other GGUF readers.

Files

Five points on the size/quality curve. Take ONE file, nothing beside it.

File	Size	Tensors	Drafter	iq4_nl
Qwen3.8-27B-IQ1_S.ggxf	6.88 GiB	1200	yes	no
Qwen3.8-27B-IQ2_XXS.ggxf	7.88 GiB	1200	yes	no
Qwen3.8-27B-IQ3_XXS.ggxf	11.05 GiB	1200	yes	no
Qwen3.8-27B-IQ4_XS.ggxf	14.14 GiB	1200	yes	yes
Qwen3.8-27B-Q4_K_S.ggxf	15.17 GiB	1200	yes	yes

Every file carries the vision tower, the tokenizer, the chat template and the sampling card. The tower travels inside the GGUF rather than as a separate mmproj, and the sampling parameters travel with it, so a runtime never has to guess them or borrow another model's settings.

The iq4_nl column matters for third-party runtimes: every block type here is a standard ggml one inherited from the upstream quantization, but the two widest mixes include iq4_nl, which some runtimes decode and others do not.

The additions are the embedded vision tower, the drafter where the trunk lacked one, and the metadata. Those have only been exercised in the companion on-device runtime -- third-party tools have not been tested against these files, and a tool that does not expect a tower inside the weight file may ignore it or refuse it.

Provenance

Each language trunk is one of unsloth's quantizations of Qwen3.8-27B -- UD-IQ1_S, UD-IQ2_XXS, UD-IQ3_XXS, UD-IQ4_XS and UD-Q4_K_S -- used as published. What this build adds is the vision tower and the metadata: the v.* / mm.* tensors and clip.* configuration, the generation config with its sampling matrix and its stop-token list. The tower is carried at source precision (F16/F32) and is bit-identical to the tower in the origin checkpoint.

The UD-IQ3_XXS, UD-IQ4_XS and UD-Q4_K_S trunks ship unsloth's nextn layer already. UD-IQ2_XXS and UD-IQ1_S do not, so their drafter block was taken verbatim from unsloth's separate MTP/mtp-Qwen3.8-27B-Q4_0.gguf.

Model Details

Model Description

Qwen3.8-27B is a dense hybrid. Its layers alternate three Gated DeltaNet linear-attention blocks to one full attention block. The linear blocks carry a small fixed-size recurrent state instead of a growing key/value cache, so memory stays flat as context grows, while the periodic attention blocks preserve exact long-range recall.

These builds are multimodal. The vision tower is included, so image input works from any one of these files alone.

Every build carries a multi-token-prediction drafter. Speculative decode over it was measured token-identical to plain greedy on this repository's own runtime; a runtime that does not speculate decodes normally.

• Developed by: Qwen, Alibaba (base model); quantization by unsloth; packaging by leok7v
• Model type: Hybrid Gated DeltaNet + attention causal language model with a vision encoder
• Language(s): English and the languages of the base model
• License: Apache 2.0, inherited from the base model
• Quantized from model: Qwen/Qwen3.8-27B

Model Sources

• Repository (this build): https://huggingface.co/leok7v/Qwen3.8-27B
• Base model: Qwen/Qwen3.8-27B
• Trunk quantization: unsloth/Qwen3.8-27B-GGUF

Uses

Direct Use

On-device chat, summarization, drafting, question answering over supplied text, and description of supplied images, in offline or privacy-sensitive settings where prompts must not leave the device.

Out-of-Scope Use

Anything the base model's licence and card exclude. These builds add no alignment, no safety tuning and no filtering of their own; they are the upstream model in a different numeric format. Not a basis for legal, medical, financial or safety decisions, and not intended for high-stakes automated decisions without human review.

Bias, Risks, and Limitations

The base model's biases and failure modes carry over unchanged.

Quantization damage is real and it is not uniform across these files. The 1-bit build is the most heavily compressed in this family and is noticeably further from the upstream model than the wider ones: outputs will not match the upstream model token for token, the gap is widest where the model was already uncertain, and short factual questions can be answered incorrectly or declined even when the model "knows" the answer at higher precision. The 2-bit build is better but still visibly damaged. Long multi-step derivations are where errors accumulate fastest in every case.

Anyone depending on one of these builds should evaluate it on their own task rather than assume it behaves like the full-precision model, or like the other files here. If accuracy matters more than footprint, prefer a wider quant.

Recommendations

Treat generations as drafts to verify, not as fact. For anything where accuracy matters, and for arithmetic in particular, check the result.

How to Get Started with the Model

The tokenizer, the chat template and the sampling card are embedded, so a runtime reads all of them from the GGUF. The chat template documents the roles, the thinking block and the tool-call format.

The base model card recommends these sampling parameters, and the embedded sampling card carries them:

mode	temperature	top_p	top_k	presence_penalty
Thinking	1.0	0.95	20	0.0
Instruct (non-thinking)	0.7	0.80	20	1.5

reasoning_effort is supported by the template; low and medium cost fewer tokens per turn than the default.

Generation stops on either <|im_end|> (248046) or <|endoftext|> (248044). Both ids are carried in the embedded generation config and in tokenizer.ggml.eos_token_ids; a runtime that reads only the scalar tokenizer.ggml.eos_token_id will see <|im_end|> alone.

Technical Specifications

Model Architecture and Objective

Property	Value
Objective	Autoregressive next-token prediction
Hidden size	5120
Layers	64 (Gated DeltaNet + attention, three to one)
Attention heads	24 query / 4 key-value (grouped-query)
Head dimension	256 (partial rotary, 64)
Linear-attention state	128, 16 groups, inner size 6144
Feed-forward	17408
Vocabulary	248320
Context length	up to 262144 tokens
Vision encoder	27 blocks, 768px tiles, 2x2 spatial merge

Weight format

GGUF v3. Every trunk is a MIXED-precision assignment rather than one block type throughout: the widths are chosen per tensor, so the most sensitive rows keep more bits than the bulk of the model. The name on the file is the dominant type, not the only one.

Dominant trunk types per file, by tensor count:

File	Leading types
IQ1_S	IQ1_S 264, Q8_0 96, IQ2_XXS 59, IQ1_M 25, IQ2_XS 21
IQ2_XXS	IQ2_XXS 143, IQ1_S 100, Q8_0 96, IQ3_XXS 40, IQ2_S 36
IQ3_XXS	IQ3_XXS 120, IQ3_S 106, Q8_0 98, IQ4_XS 45, IQ2_S 35
IQ4_XS	IQ4_XS 211, Q8_0 98, Q4_K 55, Q5_K 51, IQ3_S 46
Q4_K_S	IQ4_XS 172, Q8_0 99, Q4_K 95, Q5_K 80, Q6_K 18

The Q4_K_S mix is worth a second look: despite the name, more of its trunk is IQ4_XS than Q4_K, and it is the file that carries the most iq4_nl.

In every file the norms, layer scalars and recurrent-state parameters stay F32 (360 tensors), and the 334 vision-tower tensors stay at source precision (222 F32 + 112 F16).

Citation

Please cite the upstream Qwen3.8-27B model. See the base model card at Qwen/Qwen3.8-27B for the canonical reference and license.

Model Card Contact

leok7v on Hugging Face.