Nex-N2.6 NVFP4 / MXFP8 / CSF

A mixed-precision, post-training-quantized export of Nex-AGI's Nex-N2.6, an approximately 321B-parameter multimodal model for agentic, coding, computer-use, and long-horizon tool tasks.

  • 178.55 GB of tensor storage (166.28 GiB), in 37 safetensors files.
  • NVFP4 routed experts with losslessly compressed E4M3 block scales (CSF) and calibrated static 4-bit activations for prefill.
  • MXFP8 attention, shared experts, and vision attention.
  • Unofficial derivative: this repository is maintained by dndll, not Nex-AGI. It is a PTQ export, not the QAD (quantization-aware distillation) weights referenced by the conversion recipe.

Important: precision and runtime

The name describes a mixed-precision checkpoint, not an MXFP8-only export. The decoder routed experts use W4A4 during prefill and W4A16 during decode with the pinned serving configuration below; most other quantized groups have dynamic MXFP8 activations or BF16 activations.

Stock vLLM, Transformers, GGUF/llama.cpp, and generic NVFP4 loaders do not automatically support this checkpoint's compressed scales. Use a runtime that implements the Hugging Face-layout weight_scale_encoding: "csf" contract for glm5_next. Standard safetensors loading is retained, but the quantization implementation must understand the CSF encoding and the Glm5NextForConditionalGeneration architecture.

The verified runtime was Local Inference Lab's karmic-kraken vLLM image, whose lil-serve launcher recognizes the CSF recipes in config.json and resolves the appropriate settings. Use the GLM profile/preset invocation below without --quantization or --load-format; do not force instanttensor, which bypasses the CSF routing and rejects the tensors/ layout. Pinned image digest used for local inference:

ghcr.io/local-inference-lab/vllm@sha256:bc48d33ee167fb7675a59889fe0a7fea539bc6700c53cada89284d594a754b54

Download

Install the Hugging Face CLI, then:

hf download dndll/Nex-N2.6-NVFP4-MXFP8-CSF

Or use Python:

from huggingface_hub import snapshot_download

model = snapshot_download("dndll/Nex-N2.6-NVFP4-MXFP8-CSF")
print(model)

For lil-serve, pass the resulting directory as MODEL and use the GLM-5.3-Flash profile and TP2 deployment preset:

export MODEL=/path/to/Nex-N2.6-NVFP4-MXFP8-CSF
docker run --rm -it \
  --gpus '"device=0,1"' \
  --network host --ipc host --shm-size 32g \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  -e MODEL \
  -v "$MODEL":"$MODEL":ro \
  -v "$HOME/.cache/huggingface:/root/.cache/huggingface" \
  -v "$HOME/.cache/lil-nex:/cache" \
  --entrypoint /usr/local/bin/lil-serve \
  ghcr.io/local-inference-lab/vllm@sha256:bc48d33ee167fb7675a59889be0a7fea539bc6700c53cada89284d594a754b54 \
  --profile glm53-flash \
  --hardware rtx-pro-6000-pcie \
  --preset glm53-tp2 \
  -- \
  --served-model-name Nex-N2.6-NVFP4-MXFP8-CSF

This example follows the local TP2, Blackwell deployment. It targets two GPUs and is not a universal single-GPU or commodity-vLLM command. KV-cache sizing depends on hardware, context, and serving options. The local deployment explicitly reserved about 432 MB per rank more for this PTQ checkpoint's larger scale-exception set than the reference QAD checkpoint.

Once serving on port 8000:

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Nex-N2.6-NVFP4-MXFP8-CSF",
    "messages": [
      {"role": "user", "content": "Explain quantization in one paragraph."}
    ],
    "reasoning_effort": "high",
    "max_tokens": 1024
  }'

Keep the supplied tokenizer, processor, and chat template. The model accepts text and image inputs through a compatible GLM-5.3-Flash serving implementation. The template supports reasoning_effort values low, high, and max.

Quantization layout

Component Weights Activations / scale handling
Routed experts, decoder layers 3–44 NVFP4 E2M1, group 16 Calibrated static 4-bit activations for prefill; BF16 activations for decode. E4M3 block scales CSF-encoded; FP32 weight and input scales
MTP routed experts, layer 45 NVFP4 E2M1, group 16 BF16 activations (W4A16). Raw E4M3 block scales, FP32 global scale
Attention in all 46 layers, including KDA, MLA, and DSA indexer MXFP8 E4M3, group 32 Dynamic MXFP8; UE8M0 scale byte
Shared experts, layers 3–45 MXFP8 E4M3, group 32 Dynamic MXFP8
Vision attention MXFP8 E4M3, group 32 Dynamic MXFP8
Vision MLP and merger NVFP4 E2M1, group 16 BF16 activations (W4A16), E4M3 block scales, FP32 global scale
Embeddings, lm_head, routers, norms, convolutions, KDA A_log/dt_bias, dense MLP in layers 0–2 BF16 Preserved

CSF is lossless compression of the stored quantization scales, not a promise of lossless model-quality preservation. It encodes 36,288 decoder routed-expert scale matrices. In the conversion record, CSF encoded 19.03 GB of raw scales as 9.61 GB fixed-stream data plus 1.07 GB exception data. Weight quantization itself remains lossy.

The quantization_config uses quant_method: "modelopt" and quant_algo: "MIXED_PRECISION". Indexed tensors live under tensors/, including a separate activation-scale file. Both must stay together with the root config.json and model.safetensors.index.json.

Activation calibration

Static scales were captured from routed-expert activations over 198,549 tokens, including 133,759 prompt tokens. The corpus drew on coding, agent/tool, mathematics, reasoning, multilingual, and other synthetic prompts as well as selected interactions from the local deployment.

  • Scale per expert: amax / (6 × 448), with gate/up sharing their input scale and down using the intermediate activation scale.
  • Two experts observed no calibration tokens; each took its layer's maximum scale instead of zero.
  • Final scales come from Nex-N2.6's own activations. A tested floor based on the related GLM-5.3-Flash published scales did not improve the local evaluation and was not used.
  • The calibration corpus is not redistributed.

Prompt-token NLL on the calibration corpus, lower is better:

Checkpoint activation configuration Prompt NLL
Same weights, scale-1.0 prefill (scale-A/B baseline) 2.6873
Final calibrated scales 2.6440

These figures measure calibration fit, not benchmark accuracy. Completions and small held-out subsets are not used to claim a win: completions came from the older configuration, and a 12,395-token held-out set had observed noise of roughly ±0.015 NLL.

Weight reconstruction checks

Metrics compare dequantized output weights to the dequantized FP8 source before the separate activation calibration stage.

Category Aggregate relative L2 Max unit relative L2 Min unit cosine
NVFP4 CSF routed experts 0.07682 0.08233 0.996605
NVFP4 MTP routed experts 0.08094 0.08300 0.996550
NVFP4 vision MLP 0.08125 0.08137 0.996684
MXFP8 attention 0.01322 0.02715 0.999632
MXFP8 shared experts 0.02425 0.02641 0.999652
MXFP8 vision attention 0.02653 0.02660 0.999646

The producer checked every source tensor was processed once, paired and dequantized source FP8 tensors before requantization, checked CSF round trips, and re-read each output shard for dtype, shape, duplicate names, and non-finite scales. A later CSF header correction preserved shard payloads byte-for-byte.

The quantization-manifest.json describes the original weight-conversion pass. reconstruction.json holds its numerical results. CALIBRATION.md documents the subsequent activation-scale update.

Validation and limitations

  • The CSF-aware lil-serve launcher accepts the local Hugging Face layout and its complete indexed file set.
  • A text generation smoke request completed successfully on 2026-11-18.
  • No standard task benchmark, general multimodal evaluation, or direct base-model-versus-quantized quality comparison has been completed. Do not transfer Nex-AGI's upstream benchmark scores to this export.
  • Vision is quantized but no new image-input test was performed as part of publication.
  • The evaluation is narrow and its corpus has not been independently audited for contamination or representativeness.
  • Expect additional output-error sensitivity to prompts, batch composition, context length, and serving implementation. Numerical reconstruction error and prompt-NLL measurements do not bound downstream behavior.
  • This is an unofficial PTQ conversion, not Nex-AGI's official model or an official Local Inference Lab QAD release.

See the upstream model card for intended use, upstream evaluations, and model behavior. The upstream model's limitations apply here along with quantization-induced limitations.

Provenance

Field Value
Base model nex-agi/Nex-N2.6
Base revision at conversion 987472298e7867721576ac49a253a6f5b526013e
Source index SHA-256 09d8cdb6af4714def10fff9606a1db01ddb6d67a23d2c45555c19e4bff2bb337
Quantization Post-training, local mixed-precision conversion
NVIDIA Model Optimizer 0.47.0rc0
PyTorch 2.13.0+cu130
Conversion hardware NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition
CSF codec byte-window4-fixed-stream-u24-exceptions/1
Published by dndll

Reproducibility files

  • quantization-manifest.json: original weight-conversion provenance and statistics.
  • reconstruction.json: numerical reconstruction metrics.
  • reconstruction-checkpoints.json: conversion checkpoint metadata.
  • CALIBRATION.md: activation-scale method and evaluation caveats.
  • SHA256SUMS: SHA-256 checksums for every distributed file except the checksum file itself.

Local scratch files, rejected calibration alternatives, raw prompts, and session data are excluded. For integrity checks, the Hub's LFS metadata and download checksums are authoritative; SHA256SUMS is an additional convenience.

License and attribution

The upstream repository declares Apache-2.0. Its license text is included in LICENSE. NOTICE describes modifications made for this export.

Nex-N2.6 and the upstream model belong to their respective authors. The conversion incorporates the mixed-precision CSF layout recipe of Local Inference Lab's reference checkpoint, not its QAD weights. No endorsement by Nex-AGI or Local Inference Lab is implied.

Downloads last month
28
Safetensors
Model size
343B params
Tensor type
F32
·
U32
·
BF16
·
U8
·
F8_E4M3
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for dndll/Nex-N2.6-NVFP4-MXFP8-CSF

Base model

nex-agi/Nex-N2.6
Quantized
(8)
this model