Qwen3.6-35B-A3B-BaseQuant-V1

BaseQuant V1 build of Qwen/Qwen3.6-35B-A3B, as a .base bundle for baseRT and a GGUF for llama.cpp.

Size baseRT bundle GGUF (llama.cpp)
Medium Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base (23.08 GB) Qwen3.6-35B-A3B-BaseQuant-V1-Medium.gguf (22.13 GB)

The .base bundle also carries the vision tower and the MTP head for speculative decoding. It needs baseRT 0.5.2 or later.

Usage

# baseRT
hf download basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base --local-dir .
basert serve ./Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base

# SuperFluid
curl -fsSL https://superfluid.sh/install.sh | sh
superfluid serve basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 --runtime basert --pull-file Qwen3.6-35B-A3B-BaseQuant-V1-Medium.base

# llama.cpp
llama-server -hf basecompute/Qwen3.6-35B-A3B-BaseQuant-V1 -hff Qwen3.6-35B-A3B-BaseQuant-V1-Medium.gguf -c 32768 -ngl 99

Accuracy

KL divergence of the next-token distribution from the BF16 model, on held-out text: 58 prompts in ten categories, 32,507 scored tokens. None of it was used to build these files. Lower is closer to BF16. The .base bundle is scored in baseRT, and the GGUFs in llama.cpp. unsloth's UD-Q4_K_M, the same size, is shown for reference.

Held-out KL divergence by category

Category Tokens BaseQuant .base BaseQuant GGUF unsloth UD-Q4_K_M
Agentic coding 2,228 0.00465 0.00490 0.00515
Chat 3,342 0.00648 0.00634 0.00645
Code 2,883 0.00427 0.00434 0.00450
Long context 2,048 0.00427 0.00438 0.00536
Math 4,608 0.00304 0.00318 0.00265
Multilingual 4,232 0.00222 0.00213 0.00233
Prose 3,994 0.00624 0.00587 0.00628
Research 2,123 0.00737 0.00848 0.00584
Science 4,382 0.00467 0.00510 0.00490
Tool use 2,667 0.00474 0.00517 0.00519
All 32,507 0.00462 0.00476 0.00467
Downloads last month
244
GGUF
Model size
35B params
Architecture
qwen35moe
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for basecompute/Qwen3.6-35B-A3B-BaseQuant-V1

Quantized
(1,014)
this model

Collection including basecompute/Qwen3.6-35B-A3B-BaseQuant-V1