WaveCut/Edge0-120B-q3k-NInfer-v3

Edge0-120B-q3k, NInfer v3 artifact

A NInfer v3 artifact of Edge0's Edge0-120B-q3k release for NInfer-all, the master branch of iamwavecut/ninfer-all. Edge0's models keep Qwen3.8-Flash-Next's skeleton (Gated DeltaNet, sparse attention, hyper-connections, the n-gram embedding, the router, the shared expert and the MTP block) and replace every text layer's 512 routed experts by latent experts: each expert belongs to one of 8 groups and computes in its group's latent width (1280), which a group encoder and decoder project from and back to the model's 2560. This artifact keeps the release's blocks byte for byte, and the engine multiplies them in place.

Stock NInfer builds refuse this file: the latent experts and the gguf_* formats exist only in NInfer-all, from commit 72d1d8e04 on. The file includes the release's MTP block and Qwen3.8-Flash-Next's Vision tower, which the release does not ship and edge0-strata reads pictures with. The Edge0 guide describes the conversion, the placements and the measurements.

What is inside

component	representation
routed experts, 48 layers × 512 in 8 groups	ggml Q3_K gate and up, IQ4_NL down, with the release's per-layer column scales (scales/l{layer}.f32, stored as their FP32 reciprocals, as edge0-strata computes them); 28.6 GB
group encoders and decoders	the release's latent_dense_q4.bin: Q4_0 rows, 1.42 GB
expert groups	the release's labels.bin
skeleton: Gated DeltaNet, sparse attention, hyper-connections, router, shared experts, head, token table	the release's model-skeleton.gguf blocks, as in every Flash-Next GGUF conversion
MTP block	the release's mtp/: Q2_0 experts as gguf_q2_0 blocks, Q8_0 matrices
Vision tower	Qwen3.8-Flash-Next's, from ISTA-DASLab's mmproj-Qwen3.8-Flash-Next-BF16.gguf (the file edge0-strata reads pictures with) in BF16, 0.9 GB: 27 blocks of width 1152, merging 2×2 patches onto the text model's 2,560, with Qwen/Qwen3.8-Flash-Next's image and video preprocessor configurations
n-gram table	not in this file: the model names its table by the SHA-256 of its rows, 57496326706760c564e80ed401f6b26a7d6b340cc05966502675c8dc01ef8fd9, published once as WaveCut/Edge0-ngram-table-FP8-NInfer-v3; NInfer-all from 57395b20b on reads any table of the same hash constants and geometry in its place, in any row format, such as Flash-Next's 26.92 GiB IQ4_NL table: perplexity 5.017691 against 4.964565 with the FP8 table (+1.07%, first 40 KB of WikiText, INT8 KV)
tokenizer and chat template	Qwen/Qwen3.8-Flash-Next's; the converter refuses a release whose vocabulary or merges differ

One file, Edge0-120B-q3k-ninfer-v3.ninfer, of 37,151,739,136 bytes (34.60 GiB), next to its conversion report, SHA256SUMS, and the release's LICENSE, NOTICE, release-scope.json and third-party/Qwen-LICENSE.txt. Sources: revision 51a60a20 of Edge0/Edge0-120B-q3k, revision 812d96e1 of Qwen/Qwen3.8-Flash-Next (its configuration, tokenizer and preprocessor files) and revision ed59f920 of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (its mmproj-Qwen3.8-Flash-Next-BF16.gguf). Conversion command:

python3 -m tools.convert --model Qwen3.8-Flash-Next \
  --recipe edge0_pack --source pack=Edge0-120B-q3k --source vision=mmproj-Qwen3.8-Flash-Next-BF16.gguf \
  --components text,vision,mtp --device cpu --name Edge0-120B-q3k --out Edge0-120B-q3k-ninfer-v3.ninfer

Running

Download the artifact and the n-gram table, then serve them:

hf download WaveCut/Edge0-120B-q3k-NInfer-v3 Edge0-120B-q3k-ninfer-v3.ninfer --local-dir models
hf download WaveCut/Edge0-ngram-table-FP8-NInfer-v3 Edge0-ngram-table-FP8-ninfer-v3.ninfer --local-dir models
M=models/Edge0-120B-q3k-ninfer-v3.ninfer
T=models/Edge0-ngram-table-FP8-ninfer-v3.ninfer

# One 48 GB GPU with every expert on it.
ninfer-serve $M --ngram-table $T --model-id edge0-120b --max-context 32768

# One GPU: the experts in pinned host memory (28.6 GB), the most used of them cached on the GPU.
ninfer-serve $M --ngram-table $T --model-id edge0-120b --expert-residency host --max-context 32768

# Little RAM: the experts stay in the file and stream into a GPU cache.
ninfer-serve $M --ngram-table $T --model-id edge0-120b --expert-residency disk --max-context 32768

--vision adds the Vision tower for images and video (0.9 GB of GPU memory). Every expert on the GPU needs about 36 GB; the experts in pinned host memory need 28.6 GB of RAM. Without --spec, NInfer drafts with the MTP block (three drafts a round) with host experts, and with experts on the GPU when it serves one request at a time; disk experts decode without drafts unless --spec mtp asks for them.

Checks and speed

November 12-13, 2026, NInfer-all master (CUDA 12.8, sm_86) on one A40 (48 GB, 300 W) in a container with a 7.65-CPU quota and 50 GB of RAM, one run per row:

• Perplexity over the first 40 KB of the repository's WikiText sample (9,234 targets, INT8 KV) is 4.964565 with device, host and disk experts alike; two tensor-parallel ranks on the one card score 4.998407.
• This file was converted again for publication and gave the same perplexity and the same greedy answer as the measured one.
• Greedy answers are the same with and without drafts under every placement.
• With --vision, greedy and without drafts, the repository's image examples got the expected answers with device experts: the chart's title, circle count and side (image_chart), the mailbox number and the sun's side (image_natural), both pictures' circle counts and the added shape (multi_image_compare), and the picture's first word joined to the video's last number (mixed_image_video); the chart answer was the same with host experts and with the default MTP drafting.

experts	GPU expert cache	no drafts, tok/s	MTP, tok/s	drafts kept
host	26.7 GiB	49.2	73.9	52.7%
disk	35.7 GiB	23.3	38.1	52.7%

Greedy decode of up to 256 tokens after a 33-token prompt, MTP with three drafts and --draft-min-p 0.5. Disk experts read the file through the container's page cache, so their rows depend on how much of it the cache holds.

With every expert on the GPU, 256 greedy tokens decoded at 54.6 tok/s without drafts and 100.7 with MTP after a code prompt, 54.1 and 69.1 after a prose prompt, and 53.7 and 79.8 after a 5,415-token document. Eight requests at once through ninfer-serve decoded 131.9 to 148.1 tok/s in all without drafts and 111.3 to 129.5 with MTP, so the engine drafts by default only for one request at a time. Against edge0-strata, Edge0's own engine (--spec 4), NInfer with MTP gave six of eight greedy chat answers token for token the same and decoded each request 1.41 to 1.74 times as fast.

No quality benchmark was run in NInfer; Edge0's card reports theirs.

Changes from the release and license

WaveCut repacked the release's files into NInfer's container for NInfer-all, with the converter in tools/convert/edge0.py. Every routed expert, encoder and decoder row and every skeleton block is stored byte for byte; the MTP block's BF16 hyper-connection matrices become Q8_0 rows and its norms drop their stored 1 + w, as in every Flash-Next GGUF conversion. The per-layer column scales become their FP32 reciprocals, one division each, as edge0-strata computes them. No weight was retrained, and no other block was re-quantized. The Vision tower comes from Qwen3.8-Flash-Next, not from the release: its BF16 tensors are stored as mmproj-Qwen3.8-Flash-Next-BF16.gguf holds them.

The Edge0-specific contributions that the release's release-scope.json designates are under the Edge0 Community Model License 1.0, reproduced in LICENSE with Edge0's NOTICE. The Qwen materials stay under the Qwen Community License 1.0, reproduced unchanged in third-party/Qwen-LICENSE.txt; the Edge0 license does not replace it. Commercial model offerings need a separate license from Edge0 (its Section 4), and Qwen's conditions apply as well. These files imply no endorsement by Edge0 or Qwen.