Sage S2.1 Flash NVFP4

A calibrated 4-bit quantization of internlm/Sage-S2.1-Flash (revision 76f4513db7464485605a002a06a6190de0460574), laid out so that SparkGLM loads it on two or four NVIDIA DGX Sparks.

  • Size: 179 GB, down from 643 GB in BF16.
  • Fidelity: KL divergence from the BF16 model 0.053, top-1 agreement 94.5% (details below).
  • Input: text, images and video. Sage's scientific-input projectors are not included.
  • License: MIT, as the original.

What is quantized, and how

Weights Format How
Routed experts, layers 3 to 44 (42 layers of 288 experts) NVFP4 GPTQ, one Hessian per expert from that expert's own inputs, 600K calibration tokens
Attention projections of the 34 linear-attention layers, the dense MLP of layers 0 to 2 MXFP8, 1脳32 blocks Re-rounded against layer inputs, 200K calibration tokens
Attention projections of the 11 indexed-attention layers (except kv_b_proj, which stays BF16), shared experts FP8, 128脳128 blocks Re-rounded against layer inputs, 200K calibration tokens
lm_head MXFP8, 1脳32 blocks Round to nearest
Multi-token-prediction (MTP) layer NVFP4 experts, MXFP8 elsewhere Round to nearest
Vision tower, embeddings, norms, routers BF16 Unchanged

Every routed expert was calibrated from the BF16 weights. No other quantized checkpoint was used as a starting point. The 8-bit matrices keep the formats SparkGLM requires; calibrating their rounding recovered the accuracy that plain rounding of those matrices costs (KL 0.0625 to 0.0531).

Calibration used 2,229 sequences (2.4M tokens available, 600K used for the experts) from public datasets: reasoning chat, plain chat, tool calling, code, English and multilingual prose, science and math. The sources and row numbers are listed in receipts/MANIFEST.md.

Fidelity against BF16

Measured in PyTorch on 224 held-out documents (239,711 scored positions) that are not in the calibration set. Intervals are 95% bootstrap intervals over documents.

KL Top-1 agreement NLL increase
This checkpoint 0.0531 [0.0441, 0.0631] 94.50% [94.10, 94.88] +0.64%
Bizuayeu/Sage-S2.1-Flash-Mixed-NVFP4 (32063d6) 0.0676 [0.0548, 0.0819] 93.89% +0.66%

Paired over the same documents, this checkpoint's KL is lower by 0.0145 [0.0100, 0.0197] and its top-1 agreement higher by 0.61 points [0.45, 0.78]; its KL is lower on 157 of the 224 documents. The other checkpoint keeps the linear-attention projections and lm_head in BF16, which SparkGLM does not load, so the two are not the same size or layout. Its own model card reports different numbers because it uses a different document set.

By category (32 documents each), KL and top-1 agreement:

Category This checkpoint Bizuayeu Mixed-NVFP4
Reasoning chat 0.0573, 95.35% 0.0679, 94.35%
Code 0.0276, 96.84% 0.0263, 96.92%
Math 0.0199, 96.35% 0.0234, 96.11%
English prose 0.0424, 94.25% 0.0415, 94.40%
Japanese prose 0.0265, 94.04% 0.0280, 93.64%
Science 0.0159, 95.18% 0.0163, 95.11%
Tool calling 0.2772, 85.93% 0.4176, 81.53%

Most of the overall gain is in tool calling. On code and English prose the two are level, with this checkpoint slightly behind.

Serving on DGX Sparks

Checked with two Sparks (tensor parallel 2) in Spark mode. Decode speed and the qualification suite are from SparkGLM 0.3.6, installed through LLooM on 2026-11-18; the image, video and other probes below were run on 0.3.5:

Sage S2.1 Flash NVFP4 GLM-5.3-Flash FP4-Local, same pair
Prose decode 38.6 tok/s 37.7 tok/s
Code decode 67.9 tok/s 66.1 tok/s
Structured output decode 90.0 tok/s 88.8 tok/s
Prompt cache pool 2.22M tokens 2.38M tokens

Speeds are medians of three runs. The pool sizes are from consecutive starts and vary by a few percent from start to start.

  • Image questions 3 of 3 and video questions 2 of 2 answered correctly.
  • Arithmetic probe 39 of 40, long-context retrieval 12 of 12, tool calls 21 of 21, agentic tasks 20 of 20.
  • SparkGLM's qualification suite in Spark mode: 109 of 109 on SparkGLM 0.3.6, 97 of 109 on 0.3.5.

Forced tool choice needs SparkGLM 0.3.6. On SparkGLM 0.3.5, a request that forces a tool (tool_choice naming a tool, required, or any) can come back with the call written as text instead of as a tool call. The cause is how the engine masked the first token of a forced call: it allowed the tag's first character as plain text. Requests with automatic tool choice are not affected. The fix (Enntity/atlas#33) is in SparkGLM 0.3.6, the current release, which is that fix on top of 0.3.5. The 0.3.6 results describe the fault and the checks.

Use Spark mode. Sage reasons before answering even when a request switches thinking off. Spark mode caps that reasoning; Strict mode, the default of SparkGLM's start.sh, does not. In Strict mode on the engine commit that 0.3.6 ships, the suite passes 97 of 109: the 12 failures are three prompts that ask the model to repeat tool-call markup as plain text (each sent four ways), where the longer reasoning leads it to a different answer.

There are two ways to serve it:

  • Through LLooM, which is how the numbers above were measured. Its SparkGLM recipe installs this checkpoint in Spark mode on the 0.3.6 image, pinned at revision 6967e495 of this repository (the same weight files as the current revision):

    lloom setup --recipe linux-nvidia-dgx-spark-glm53-atlas --model sage-s21-flash --additive --apply --yes
    

    The check started from a local copy of the weights, which the Hugging Face tool verified against that revision; a first download of all 179 GB through setup was not run.

  • With SparkGLM's start.sh: check out v0.3.6 or later, set CHECKPOINT_REPO=Enntity/Sage-S2.1-Flash-NVFP4 and CHECKPOINT_REVISION= the commit to pin in .env, and start with ./start.sh --mode spark. This path has not been run end to end with this checkpoint.

The drafter SparkGLM uses for speculative decoding was trained on GLM-5.3-Flash, not on Sage.

Limits

  • Scientific inputs. Molecules, crystal structures, proteins and the other Sage SciPerception inputs need the original repository's projectors and are not supported here.
  • Other engines. Only SparkGLM was tested. The layout follows NVIDIA's GLM-5.3-Flash FP4-Local checkpoint, but vLLM, SGLang and others were not tried.
  • Four Sparks. Tensor parallel 4 was not measured, and neither were LLooM's --memory shared and --streams 8 options.
  • Tokenizer rule. The original tokenizer encodes control strings such as <tool_call> inside message content as ordinary text. SparkGLM has its own handling for literal markup; its tests for that pass with this checkpoint, but it is not the original code path.
  • MTP layer. Its experts are rounded to nearest, not calibrated.

Files

SHA256SUMS lists the weight, tokenizer and configuration files. deployment_guide.md is the original repository's guide, unchanged. receipts/ holds the evaluation outputs behind the tables above, the script that computes the paired comparison, the list of calibration and evaluation sources, and the SparkGLM suite and speed results.

Downloads last month
-
Safetensors
Model size
343B params
Tensor type
F32
路
BF16
路
F8_E4M3
路
U32
路
U16
路
U8
路
Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for Enntity/Sage-S2.1-Flash-NVFP4

Quantized
(12)
this model