knoopx/Qwen3.8-Flash-Next-GSQ-RCO-Coder-Basalt

Qwen3.8-Flash-Next · GSQ-RCO Coder · Basalt

Model file for Basalt, converted from ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF: the coder-targeted release with half of the routed experts removed (256 of 512 per layer), stored one file.

57.43 GiB, of which 23.42 GiB are the routed experts — half the expert tier of the GSQ-RCO IQ3_S pack (46.8 GiB) for the same 176.9B-parameter base. It runs on Linux or Windows with an NVIDIA Blackwell GPU (sm_120); experts the card cannot hold stay in system RAM, and the 26.8 GiB n-gram table is never held in RAM at all.

Not measured here. See Benchmarks.

Run

hf download knoopx/Qwen3.8-Flash-Next-GSQ-RCO-Coder-Basalt \
  Qwen3.8-FN-GSQ-RCO-Coder-mtp.basalt --local-dir models

basalt_config.json:

{
  "server": { "model": "models/Qwen3.8-FN-GSQ-RCO-Coder-mtp.basalt", "gpu": "0", "port": 8080 },
  "engine": { "expert-cache": "auto", "prefill": "auto", "kv": "int8", "resident-experts": true }
}

bin/basalt serve --config basalt_config.json     # OpenAI- and Anthropic-compatible API on 127.0.0.1

Image input needs "vision": 0 (or another card number) in the server section and basalt-vision next to bin/basalt; the pack carries a vision projector. basalt-probe --model models/Qwen3.8-FN-GSQ-RCO-Coder-mtp.basalt measures your machine and prints values to start from.

The file

Section	Size	What it holds
experts	23.42 GiB	the 256 retained experts of each of 48 layers, laid out per layer
gguf	30.97 GiB	every non-expert source tensor, byte for byte; 26.82 GiB of it is the PLE n-gram table (IQ4_NL)
dense	1.37 GiB	the pool copies the engine reads (the source's float-form tensors)
mtp/dense, mtp/experts, mtp/draft_vocab	786 MiB	the speculative drafter (expert format q2_0) and its proposal vocabulary
vision/mmproj, vision/text_vocab	876 MiB	image projector and its vocabulary
profile, tokenizer/*, meta, indexes	18 MiB	expert ranking, tokenizer and chat template, geometry and provenance

File: Qwen3.8-FN-GSQ-RCO-Coder-mtp.basalt, 61,668,558,532 bytes, SHA-256 1f44864b043c1a619ef080d2dbef8bdb55fc1d0c41686796557e42a7da7e8cdd.

Geometry read from the pack: qwen4exp, 48 layers, 256 experts with 10 active per token, 262,144 context, 24 attention heads over 2 KV heads, hyper-connection matrices on both attention and FFN per layer (hyper_connection.count 4), 3-gram PLE table. Pack name Qwen3.8-Flash-Next GSQ-RCO Coder MTP; default sampling T 1.0, top-p 0.95, top-k 20. There is no hc8 section: the engine builds the int8 hyper-connection copies itself.

experts.layout gives the encodings as stored — mixed per layer, chosen upstream by RCO:

gate/up	down	layers
IQ2_S (2.56 bpw)	IQ4_NL (4.5)	15
IQ3_XXS (3.06)	IQ4_NL	14
IQ3_S (3.44)	IQ4_NL	9
IQ2_S	Q2_0 (2.25)	5
IQ3_XXS	Q2_0	3
IQ3_S	Q2_0	1
IQ4_XS (4.25)	IQ4_NL	1

3.33 bits per retained expert parameter, 1.67 measured against the 512-expert layout. Note the source shard is named IQ1_M; this pack contains no IQ1_M expert block, so the name is a label for the upstream release, not an encoding here.

What differs from a plain conversion

• Drafter grafted. The coder GGUF pair carries no MTP layer, so mtp/* came from Qwen3.8-FN-GSQ-RCO-IQ3_S.basalt; all four sections are byte-identical to that pack's, and the pack's meta.graft records base, source and section list. The drafter is therefore the unpruned GSQ-RCO one, not a coder-pruned one.
• Expert profile is the converter's plain fill order, not derived from routing traces. Cold decode is slower than on an official pack; run the engine once with --expert-profile-save FILE and re-convert with --profile FILE to fix that.
• Vision projector and chat template are the base model's, shared with the other GSQ-RCO packs.

Build it yourself

hf download ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF \
  --include "IQ1_M/*" "mmproj-*.gguf" --local-dir coder
python3 tools/convert/convert.py --gguf coder/IQ1_M/Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf \
  --mmproj coder/mmproj-Qwen3.8-Flash-Next-BF16.gguf --out Qwen3.8-FN-GSQ-RCO-Coder.basalt

Shard 2 is found beside shard 1. The coder GGUF carries no drafter tensors, so the result has no MTP layer until you copy the four mtp/* sections of a pack that has one into it (tools/model_file.py; the header digests are recomputed and must verify afterwards). The inputs this pack was built from, recorded with their fingerprints, pin which upstream build was used:

Input	Size	Fingerprint (first + last 64 MiB, size)
Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00001-of-00002.gguf	29,608,446,496	53df9b241c288e772ad9522f2e1e2e32ae868cab815324d0eb128679293fb6e1
Qwen3.8-Flash-Next-GSQ-RCO-IQ1_M-00002-of-00002.gguf	28,800,138,432	d7860b7ebdddd9450948726843da988fdea64a08e451654ee17e1117eadb3c68
mmproj-Qwen3.8-Flash-Next-BF16.gguf	907,543,008	1a3adffad8b4ce1aad84e1bb6910ce75a23c5f2922c8c563e8965a31e8829cfd

Benchmarks

No PPL/KLD or speed measurement of this pack has been published. Its quality is not on a par with the rows in the official Basalt pack table until one is run: this is a pruned model, so it is compared against the base only by the source's own numbers, not by the Q8_0 gate.

Coding scores are the source card's, measured upstream on the GGUF in llama.cpp and quoted here, not re-measured: SWE-bench Verified 79.20 and LiveCodeBench v6 87.43 against 82.80 / 87.43 for the BF16 base, at xhigh reasoning effort.

To produce your own numbers on this file, stop any server, then bin/basalt-bench --quality (100 wikitext-2 chunks through the decode path, against the Q8_0 reference) or bin/basalt-bench --community --submitter NAME (adds a row to docs/community/benchmarks.json); see quality and community runs.

Sources and license

• Weights: ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF, Institute of Science and Technology Austria (GSQ quantization + RCO expert pruning).
• Base model: Qwen/Qwen3.8-Flash-Next, whose terms these weights inherit.
• Drafter and projector sections are copied byte for byte from the GSQ-RCO IQ3_S pack of the same model, which took them from Qwen's BF16 MTP tensors and BF16 mmproj.
• Basalt is a fork of Strata (MIT); the engine is MIT, the weights are not.

The pack's meta.license carries the Qwen Community License 1.0 notice, which these weights inherit from the base model; the upstream GGUF repository is tagged apache-2.0, and the base model's terms are the binding ones. Under those terms, a business offering Model as a Service or an AI coding assistant needs a separate license from Qwen, and a product above 100M monthly users or US$20M monthly revenue must display the model name.

If you use the GSQ-RCO weights, please cite:

@article{gsq2026,
  title  = {GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling},
  author = {Dadgarnia, Alireza and Tabesh, Soroush and Nikdan, Mahdi and Helcig, Michael and Kurtic, Eldar and Kleinegger, Maximilian and Alistarh, Dan},
  journal= {arXiv preprint arXiv:2604.18556},
  year   = {2026}
}
@article{rco2026,
  title  = {Model Compression with Exact Budget Constraints via Riemannian Manifolds},
  author = {Helcig, Michael and Alistarh, Dan},
  journal= {arXiv preprint arXiv:2605.00649},
  year   = {2026}
}