bpx03/Qwen3.8-Flash-Next-QEngine

Qwen3.8-Flash-Next for QEngine

Qwen3.8-Flash-Next in the weight layout of QEngine, a source-available inference engine for 3× NVIDIA RTX 3090. Download it and run it with QEngine directly; there is no conversion step.

This revision is for QEngine 1.3.4.1 or later; 1.3.5 is the current version and the files haven't changed for it. It is about twice as close to the original BF16 model as the 1.3.4 files: the parts every token passes through are more precise, and the routed experts are unchanged. The previous files remain available as revision v1.3.4 (see Previous version). These files only work with QEngine; QEngine 1.3.4 and older stop at load on them with tensor not found: ...linear_attn.in_proj_qkv.weight_packed (update QEngine, or download --revision v1.3.4). The quantization_config in config.json only describes the files; QEngine reads the formats from the tensors.

Closeness to BF16

We measured next-token KL divergence against the official BF16 model on a public reference set: 21 cases and 32,359 positions covering prose, code, Japanese, Chinese, SQL, agent turns and WikiText-2. The set was scored with its own compare.py (dataset revision 5294aa9a); every position runs through QEngine's decode path in forwards of up to 16 tokens on 3 cards. Lower KL is closer to BF16.

QEngine weights	mean KL	top-1 agreement	WikiText-2 KL
this revision, QEngine 1.3.5	0.114	89.6 %	0.125
this revision, QEngine 1.3.4.1	0.119	89.1 %	0.141
previous revision (v1.3.4)	0.267	83.3 %	0.329

The files are the same in both of those rows, so please don't read the small move as a quality gain: we haven't traced it to one change. With the context built by the server's batched prefill (the last 512 positions of the long cases), mean KL is 0.126 on QEngine 1.3.5 and 0.146 on 1.3.4.1, against 0.358 for v1.3.4.

In the club-3090 8-pack (150 scenarios, 3 runs, 3× RTX 3090) QEngine 1.3.5 with these files passed 120.7 (thinking off) and 126.7 (thinking on) on average, and QEngine 1.3.4.1 passed 117.0 and 124.7; with the 1.3.4 files it passed 114.7 and 128.7. Three runs per version are too few to separate the versions.

Speed. The more precise DeltaNet projections are read on every decode step, so on the same engine version decode is about 4-5 % slower than with v1.3.4 (prefill is unchanged). QEngine 1.3.4.2 won this back. Use the current QEngine; its README has the speeds.

What's inside

• Converted from Intel/Qwen3.8-Flash-Next-W4A16-AutoRound, an INT4 quantization of Qwen/Qwen3.8-Flash-Next.
• base.safetensors, layer-00 … layer-47: the model, regrouped per layer for QEngine:
  • routed experts: INT4 from the AutoRound checkpoint, repacked bit-exact;
  • DeltaNet projections (in_proj_qkv / z / out_proj): stored as INT8 g128 (int8 weights + bf16 scales per 128 inputs, symmetric), exactly what the engine runs (v1.3.4: INT4); the download is about 1 GB larger;
  • shared expert and n-gram key/value projections: INT4 (group 128), calibrated with GPTQ on our own calibration text (v1.3.4: round-to-nearest);
  • QSA attention q/k/v/o and lm_head: BF16 in the files, INT8 in the engine;
  • hyper-connections, router, embeddings, indexer, gates, norms: BF16.
• mtp.safetensors / mtp-int4.safetensors: the multi-token-prediction head used for speculative decoding (QEngine runs its attention / fc from the BF16 copies as INT8, its experts and shared expert INT4).
• ngram_int8.safetensors: the n-gram embedding table in INT8 (51.8 GB instead of 102.4 GB in BF16). In QEngine's tests its next-token KL against the BF16 table was within the engine's own run-to-run noise.
• expert_prior.bin, cold_experts.json, token_rank.bin: routing statistics and the draft vocabulary that QEngine uses.

Requirements

• 3× RTX 3090 (24 GB), or 2 of them. GPU-to-GPU P2P helps but is not needed (see the QEngine README).
• Host RAM: with about 140 GB (3 cards) or 160 GB (2 cards) QEngine uses its full defaults. Smaller hosts (from 64 GB) get a plan sized to their RAM at start-up. Details are in the QEngine README.
• About 129 GB (120 GiB) of disk (v1.3.4: 128 GB).

Previous version

The 1.3.4 files (smaller DeltaNet projections, about 4-5 % faster decode on the same engine version, further from BF16) are kept as revision v1.3.4:

hf download bpx03/Qwen3.8-Flash-Next-QEngine --revision v1.3.4 --local-dir models/Qwen3.8-Flash-Next-QE-INT4-v1.3.4
server/run.sh 8080 --model models/Qwen3.8-Flash-Next-QE-INT4-v1.3.4

QEngine 1.3.4.1 and later run both revisions: the engine reads the format from the files, so nothing else changes.

License

The weights are a derivative of Qwen3.8-Flash-Next and are distributed under the Qwen Community License 1.0 (copy included). QEngine itself has its own license, see its repository.