WaveCut/Edge0-145B-q3k-NInfer-v3

Edge0-145B-q3k, NInfer v3 artifact

A NInfer v3 artifact of Edge0's Edge0-145B-q3k release for NInfer-all, the master branch of iamwavecut/ninfer-all. Edge0's models keep Qwen3.8-Flash-Next's skeleton (Gated DeltaNet, sparse attention, hyper-connections, the n-gram embedding, the router, the shared expert and the MTP block) and replace every text layer's 512 routed experts by latent experts: each expert belongs to one of 8 groups and computes in its group's latent width (1792), which a group encoder and decoder project from and back to the model's 2560. This artifact keeps the release's blocks byte for byte, and the engine multiplies them in place.

Stock NInfer builds refuse this file: the latent experts and the gguf_* formats exist only in NInfer-all, from commit 72d1d8e04 on. The file includes the release's MTP block and Qwen3.8-Flash-Next's Vision tower, which the release does not ship and edge0-strata reads pictures with. The Edge0 guide describes the conversion, the placements and the measurements.

What is inside

component	representation
routed experts, 48 layers × 512 in 8 groups	ggml Q3_K gate and up, IQ4_NL down, with the release's per-layer column scales (scales/l{layer}.f32, stored as their FP32 reciprocals, as edge0-strata computes them); 40.1 GB
group encoders and decoders	the release's latent_dense_q8.bin: Q8_0 rows, 3.74 GB
expert groups	the release's labels.bin
skeleton: Gated DeltaNet, sparse attention, hyper-connections, router, shared experts, head, token table	the release's model-skeleton.gguf blocks, as in every Flash-Next GGUF conversion
MTP block	the release's mtp/: Q2_0 experts as gguf_q2_0 blocks, Q8_0 matrices
Vision tower	Qwen3.8-Flash-Next's, from ISTA-DASLab's mmproj-Qwen3.8-Flash-Next-BF16.gguf (the file edge0-strata reads pictures with) in BF16, 0.9 GB: 27 blocks of width 1152, merging 2×2 patches onto the text model's 2,560, with Qwen/Qwen3.8-Flash-Next's image and video preprocessor configurations
n-gram table	not in this file: the model names its table by the SHA-256 of its rows, 57496326706760c564e80ed401f6b26a7d6b340cc05966502675c8dc01ef8fd9, published once as WaveCut/Edge0-ngram-table-FP8-NInfer-v3; NInfer-all from 57395b20b on reads any table of the same hash constants and geometry in its place, in any row format, such as Flash-Next's 26.92 GiB IQ4_NL table: perplexity 3.145938 against 3.127031 with the FP8 table (+0.60%, first 40 KB of WikiText, INT8 KV)
tokenizer and chat template	Qwen/Qwen3.8-Flash-Next's; the converter refuses a release whose vocabulary or merges differ

One file, Edge0-145B-q3k-ninfer-v3.ninfer, of 50,930,150,656 bytes (47.43 GiB), next to its conversion report, SHA256SUMS, and the release's LICENSE, NOTICE, release-scope.json and third-party/Qwen-LICENSE.txt. Sources: revision d7de835c of Edge0/Edge0-145B-q3k, revision 812d96e1 of Qwen/Qwen3.8-Flash-Next (its configuration, tokenizer and preprocessor files) and revision ed59f920 of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (its mmproj-Qwen3.8-Flash-Next-BF16.gguf). Conversion command:

python3 -m tools.convert --model Qwen3.8-Flash-Next \
  --recipe edge0_pack --source pack=Edge0-145B-q3k --source vision=mmproj-Qwen3.8-Flash-Next-BF16.gguf \
  --components text,vision,mtp --device cpu --name Edge0-145B-q3k --out Edge0-145B-q3k-ninfer-v3.ninfer

Running

Download the artifact and the n-gram table, then serve them:

hf download WaveCut/Edge0-145B-q3k-NInfer-v3 Edge0-145B-q3k-ninfer-v3.ninfer --local-dir models
hf download WaveCut/Edge0-ngram-table-FP8-NInfer-v3 Edge0-ngram-table-FP8-ninfer-v3.ninfer --local-dir models
M=models/Edge0-145B-q3k-ninfer-v3.ninfer
T=models/Edge0-ngram-table-FP8-ninfer-v3.ninfer

# One GPU: the experts in pinned host memory (40.1 GB), the most used of them cached on the GPU.
ninfer-serve $M --ngram-table $T --model-id edge0-145b --expert-residency host --max-context 32768

# Little RAM: the experts stay in the file and stream into a GPU cache.
ninfer-serve $M --ngram-table $T --model-id edge0-145b --expert-residency disk --max-context 32768

# Two GPUs dividing every layer (Linux), about 25 GB of weights on each: each holds whole expert groups with their encoder and decoder rows.
ninfer-serve $M --ngram-table $T --model-id edge0-145b --devices 0,1 --tensor-parallel --max-context 32768

--vision adds the Vision tower for images and video (0.9 GB of GPU memory). Every expert on the GPU needs about 50 GB, more than the A40's 48 GB; the experts in pinned host memory need 40.1 GB of RAM. Only the host and disk placements were run with this release; two-GPU tensor parallelism was checked as two ranks on one A40 with Edge0-120B-q3k. Without --spec, NInfer drafts with the MTP block (three drafts a round) with host experts, and with experts on the GPU when it serves one request at a time; disk experts decode without drafts unless --spec mtp asks for them.

Checks and speed

November 12-13, 2026, NInfer-all master (CUDA 12.8, sm_86) on one A40 (48 GB, 300 W) in a container with a 7.65-CPU quota and 50 GB of RAM, one run per row:

• Perplexity over the first 40 KB of the repository's WikiText sample (9,234 targets, INT8 KV) is 3.127031 with host experts and 3.127031 with disk experts.
• This file was converted again for publication and gave the same perplexity and the same greedy answer as the measured one.
• Greedy answers are the same with and without drafts under every placement.
• With --vision, greedy and without drafts, the repository's image examples got the expected answers with disk experts: the chart's title, circle count and side (image_chart), the mailbox number and the sun's side (image_natural).

experts	GPU expert cache	no drafts, tok/s	MTP, tok/s	drafts kept
host	33.3 GiB	42.6	71.1	59.0%
disk	33.6 GiB	16.5	32.3	59.0%

Greedy decode of up to 256 tokens after a 33-token prompt, MTP with three drafts and --draft-min-p 0.5. Disk experts read the file through the container's page cache, so their rows depend on how much of it the cache holds.

No quality benchmark was run in NInfer; Edge0's card reports theirs.

Changes from the release and license

WaveCut repacked the release's files into NInfer's container for NInfer-all, with the converter in tools/convert/edge0.py. Every routed expert, encoder and decoder row and every skeleton block is stored byte for byte; the MTP block's BF16 hyper-connection matrices become Q8_0 rows and its norms drop their stored 1 + w, as in every Flash-Next GGUF conversion. The per-layer column scales become their FP32 reciprocals, one division each, as edge0-strata computes them. No weight was retrained, and no other block was re-quantized. The Vision tower comes from Qwen3.8-Flash-Next, not from the release: its BF16 tensors are stored as mmproj-Qwen3.8-Flash-Next-BF16.gguf holds them.

The Edge0-specific contributions that the release's release-scope.json designates are under the Edge0 Community Model License 1.0, reproduced in LICENSE with Edge0's NOTICE. The Qwen materials stay under the Qwen Community License 1.0, reproduced unchanged in third-party/Qwen-LICENSE.txt; the Edge0 license does not replace it. Commercial model offerings need a separate license from Edge0 (its Section 4), and Qwen's conditions apply as well. These files imply no endorsement by Edge0 or Qwen.