AtomicChat/Clover-35B-A3B-Preview-GGUF

Clover-35B-A3B-Preview · GGUF

GGUF quants of DiffuSpace's Clover-35B-A3B-Preview, built from DiffuSpace's original BF16 weights.

• Clover-35B-A3B-Preview is DiffuSpace's block-diffusion language model: a 35B mixture-of-experts with about 3B active parameters per token, derived from Qwen3.6-35B-A3B, 128K context, thinking on by default, Apache-2.0.
• Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.
• Atomic Dynamic (AD) files with our own importance matrix, from 13.7 GB to 26.3 GB on disk, plus Q8_0; every file measured against the BF16 original on one stand, next to other publishers' files.
• Need it without refusals? Clover-35B-A3B-Preview-Uncensored-GGUF has baked builds and a LoRA adapter for the quants here.

---

These files need llama.cpp built from PR #31306 (commit 4838ffb, not merged yet). Released llama.cpp builds, Ollama, LM Studio and apps on stock llama.cpp do not know the clover_moe architecture yet. If the PR changes the file format before it is merged, we will re-quantize.

Plain files on release day (28 October), AD files on 30 October. The AD- files use our own importance matrix, collected on the BF16 weights (--parse-special) over 4,000 chunks of 512 tokens of our calibration corpus. The corpus build was rendered for Ornith-1.5-35B-A3B, whose Qwen3.5-family chat format (reasoning kept in every turn) Clover's template follows. The matrix is imatrix/imatrix.gguf in the metrics repo.

Pick a file

KL divergence is how far the quantized model's predictions drift from the original BF16 weights. Lower is better, and zero means identical. top-1 is how often it picks the same next token the original would have picked. Every number below comes from one stand: our neutral multilingual text, 87 chunks of 4,096 tokens, llama.cpp 4838ffb (PR #31306), plain decoding, one H100.

File	Size on disk	For	KL divergence	top-1
Clover-35B-A3B-Preview-Q8_0.gguf	36.9 GB	48 GB and up	0.0259	93.04%
Clover-35B-A3B-Preview-AD-Q6_K-Q5_K.gguf	26.3 GB	32 GB	0.0293	92.44%
Clover-35B-A3B-Preview-AD-Q5_K-Q4_K.gguf	22.1 GB	24 GB, short context	0.0416	90.90%
Clover-35B-A3B-Preview-AD-Q4_K-IQ4_XS.gguf	20.1 GB	24 GB	0.0486	90.17%
Clover-35B-A3B-Preview-AD-IQ4_XS-IQ3_S.gguf	17.6 GB	20 GB	0.0764	87.82%
Clover-35B-A3B-Preview-AD-IQ3_S-IQ3_XXS.gguf	15.5 GB	16 GB, short context	0.1101	85.46%
Clover-35B-A3B-Preview-AD-IQ3_XXS-IQ2_S.gguf	13.7 GB	16 GB	0.1735	81.86%
Removed on 31 October, measured on the same stand: llama.cpp's plain presets without an importance matrix,
Q6_K (28.5 GB, KL divergence 0.0400, top-1 91.22%), Q5_K_M (24.7 GB, 0.0687, 88.76%) and Q4_K_M
(21.2 GB, 0.1110, 85.34%). Each AD- file is smaller than the plain preset it replaced and drifts less: AD-Q4_K-IQ4_XS (20.1 GB)
has 56% less KL divergence than plain Q4_K_M (21.2 GB), AD-Q5_K-Q4_K (22.1 GB) 39% less than
Q5_K_M (24.7 GB), AD-Q6_K-Q5_K (26.3 GB) 27% less than Q6_K (28.5 GB). AD-IQ3_S-IQ3_XXS (15.5 GB)
matches plain Q4_K_M at 5.7 GB less. The smallest file, AD-IQ3_XXS-IQ2_S, changes the next token
often (top-1 81.9%): in our chat check it answered a simple time question wrong with thinking on, where
the larger files were right. Use it only if nothing larger fits. "For" is the memory class the file fits
with room for a modest context; long contexts need more.

AD- marks an Atomic Dynamic layout: the routed experts carry the low-bit types, with ffn_down one step above ffn_gate/ffn_up in all 40 blocks (the name is <down>-<gate_up>); attention, the linear-attention blocks, the shared expert, the embeddings and the head stay q8_0. Compare files by size on disk, not by name.

Read these numbers against this model's noise floor. In this build, Clover's predictions move even when the weights do not: on our 28 October stand, the BF16 file run with only the micro-batch changed (128 instead of 512 tokens) had KL divergence 0.014 and top-1 94.9% against the same BF16 run; with only the K/V cache in bf16 instead of f16, 0.013 and 95.1%. For most models both are near zero. The graph rounds the residual stream to bf16 in every block, which may be part of it. Perplexity barely moves (Q8_0 +0.15%): the files swap near-equal candidates more often than they lose quality. Compare the files with each other.

Next to other publishers

The same stand, the same reference, each file downloaded from its repo on 30 October:

Other publisher's file	GB	KL divergence	top-1	Nearest AD file	GB	KL divergence	top-1
maternion IQ4_XS	18.9	0.1095	85.38%	AD-IQ4_XS-IQ3_S	17.6	0.0764	87.82%
maternion Q3_K_M	16.8	0.2309	79.14%	AD-IQ3_S-IQ3_XXS	15.5	0.1101	85.46%
na0x2c6 IQ4_XS-exps-hq	19.4	0.0540	89.60%	AD-Q4_K-IQ4_XS	20.1	0.0486	90.17%
na0x2c6 IQ3_XXS-exps-down3-hq	15.1	0.1165	85.01%	AD-IQ3_S-IQ3_XXS	15.5	0.1101	85.46%
na0x2c6 IQ2_S-exps-down3-hq	13.8	0.1675	82.09%	AD-IQ3_XXS-IQ2_S	13.7	0.1735	81.86%
DiffuSpace Q4_K_M	23.0	0.0422	90.88%	AD-Q5_K-Q4_K	22.1	0.0416	90.90%

Against na0x2c6's imatrix files the differences are small and go both ways: their smallest file is a little closer to BF16 than ours at the same size. Raw logs of every run are in the metrics repo.

How Clover runs in llama.cpp

Clover is trained to fill a block of masked tokens at once. llama.cpp uses that in two ways:

• Plain decoding (llama-server, llama-cli): one token at a time, like any model. Every number in the tables above is measured this way.
• Self-speculative decoding (--spec-type draft-self, from the PR): the model drafts a masked block and checks it causally, with no separate draft model.

With these files draft-self works: on a code prompt every drafted token was accepted (101 of 101 on BF16, 108 of 108 on Q8_0, 100 of 100 on the plain Q4_K_M preset). It is not fast yet in this build: on one A100-80GB it decoded 0.6-0.7 tokens/s, against about 53 tokens/s for plain decoding of that Q4_K_M in llama-server, because the drafted blocks run slowly. The PR's author reports about 32 tokens/s on an H200 and plans speedups in later PRs. llama-server --spec-type draft-self does not start the self-speculative path in this build (it logs failed to initialize speculative decoding context and decodes plainly). For now, use plain decoding.

Run

Build llama.cpp from the PR:

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git fetch origin pull/31306/head && git checkout 4838ffb0c8ade92f34d9157deea38d92758ab824
cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-server llama-speculative-simple

Chat (plain decoding):

./build/bin/llama-server -hf AtomicChat/Clover-35B-A3B-Preview-GGUF:AD-Q4_K-IQ4_XS --jinja -c 32768 \
    --presence-penalty 0.9 --repeat-penalty 1.05

Thinking is on by default. To turn it off: --chat-template-kwargs '{"enable_thinking": false}'. DiffuSpace's defaults are greedy decoding with presence penalty 0.9 and repetition penalty 1.05; their length and end-of-thinking controls (dllm_length_penalty, dllm_think_end_bias) belong to the diffusion sampler and are not in llama.cpp.

Self-speculative decoding (the PR's example program):

./build/bin/llama-speculative-simple -m Clover-35B-A3B-Preview-AD-Q4_K-IQ4_XS.gguf --spec-type draft-self \
    --spec-draft-n-max 15 -ctk bf16 -ctv bf16 -fa on -ngl 99 -n 512 -p "..."

License and credits

The weights are DiffuSpace's, released under Apache-2.0 (LICENSE and ATTRIBUTIONS.md in this repo). Clover is derived from Qwen3.6-35B-A3B (Copyright 2026 Alibaba Cloud, Apache-2.0). Quantized by AtomicChat.