Sage S2.1 Flash NVFP4
A calibrated 4-bit quantization of internlm/Sage-S2.1-Flash
(revision 76f4513db7464485605a002a06a6190de0460574), laid out so that
SparkGLM loads it on two or four NVIDIA DGX Sparks.
- Size: 179 GB, down from 643 GB in BF16.
- Fidelity: KL divergence from the BF16 model 0.053, top-1 agreement 94.5% (details below).
- Input: text, images and video. Sage's scientific-input projectors are not included.
- License: MIT, as the original.
What is quantized, and how
| Weights | Format | How |
|---|---|---|
| Routed experts, layers 3 to 44 (42 layers of 288 experts) | NVFP4 | GPTQ, one Hessian per expert from that expert's own inputs, 600K calibration tokens |
| Attention projections of the 34 linear-attention layers, the dense MLP of layers 0 to 2 | MXFP8, 1脳32 blocks | Re-rounded against layer inputs, 200K calibration tokens |
Attention projections of the 11 indexed-attention layers (except kv_b_proj, which stays BF16), shared experts |
FP8, 128脳128 blocks | Re-rounded against layer inputs, 200K calibration tokens |
lm_head |
MXFP8, 1脳32 blocks | Round to nearest |
| Multi-token-prediction (MTP) layer | NVFP4 experts, MXFP8 elsewhere | Round to nearest |
| Vision tower, embeddings, norms, routers | BF16 | Unchanged |
Every routed expert was calibrated from the BF16 weights. No other quantized checkpoint was used as a starting point. The 8-bit matrices keep the formats SparkGLM requires; calibrating their rounding recovered the accuracy that plain rounding of those matrices costs (KL 0.0625 to 0.0531).
Calibration used 2,229 sequences (2.4M tokens available, 600K used for the experts) from public
datasets: reasoning chat, plain chat, tool calling, code, English and multilingual prose, science and
math. The sources and row numbers are listed in receipts/MANIFEST.md.
Fidelity against BF16
Measured in PyTorch on 224 held-out documents (239,711 scored positions) that are not in the calibration set. Intervals are 95% bootstrap intervals over documents.
| KL | Top-1 agreement | NLL increase | |
|---|---|---|---|
| This checkpoint | 0.0531 [0.0441, 0.0631] | 94.50% [94.10, 94.88] | +0.64% |
Bizuayeu/Sage-S2.1-Flash-Mixed-NVFP4 (32063d6) |
0.0676 [0.0548, 0.0819] | 93.89% | +0.66% |
Paired over the same documents, this checkpoint's KL is lower by 0.0145 [0.0100, 0.0197] and its top-1
agreement higher by 0.61 points [0.45, 0.78]; its KL is lower on 157 of the 224 documents. The other
checkpoint keeps the linear-attention projections and lm_head in BF16, which SparkGLM does not
load, so the two are not the same size or layout. Its own model card reports different numbers
because it uses a different document set.
By category (32 documents each), KL and top-1 agreement:
| Category | This checkpoint | Bizuayeu Mixed-NVFP4 |
|---|---|---|
| Reasoning chat | 0.0573, 95.35% | 0.0679, 94.35% |
| Code | 0.0276, 96.84% | 0.0263, 96.92% |
| Math | 0.0199, 96.35% | 0.0234, 96.11% |
| English prose | 0.0424, 94.25% | 0.0415, 94.40% |
| Japanese prose | 0.0265, 94.04% | 0.0280, 93.64% |
| Science | 0.0159, 95.18% | 0.0163, 95.11% |
| Tool calling | 0.2772, 85.93% | 0.4176, 81.53% |
Most of the overall gain is in tool calling. On code and English prose the two are level, with this checkpoint slightly behind.
Serving on DGX Sparks
Checked with two Sparks (tensor parallel 2) in Spark mode. Decode speed and the qualification suite are from SparkGLM 0.3.6, installed through LLooM on 2026-11-18; the image, video and other probes below were run on 0.3.5:
| Sage S2.1 Flash NVFP4 | GLM-5.3-Flash FP4-Local, same pair | |
|---|---|---|
| Prose decode | 38.6 tok/s | 37.7 tok/s |
| Code decode | 67.9 tok/s | 66.1 tok/s |
| Structured output decode | 90.0 tok/s | 88.8 tok/s |
| Prompt cache pool | 2.22M tokens | 2.38M tokens |
Speeds are medians of three runs. The pool sizes are from consecutive starts and vary by a few percent from start to start.
- Image questions 3 of 3 and video questions 2 of 2 answered correctly.
- Arithmetic probe 39 of 40, long-context retrieval 12 of 12, tool calls 21 of 21, agentic tasks 20 of 20.
- SparkGLM's qualification suite in Spark mode: 109 of 109 on SparkGLM 0.3.6, 97 of 109 on 0.3.5.
Forced tool choice needs SparkGLM 0.3.6. On SparkGLM 0.3.5, a request that forces a tool
(tool_choice naming a tool, required, or any) can come back with the call written as text
instead of as a tool call. The cause is how the engine masked the first token of a forced call: it allowed
the tag's first character as plain text. Requests with automatic tool choice are not affected.
The fix (Enntity/atlas#33) is in
SparkGLM 0.3.6, the current release,
which is that fix on top of 0.3.5. The
0.3.6 results
describe the fault and the checks.
Use Spark mode. Sage reasons before answering even when a request switches thinking off. Spark
mode caps that reasoning; Strict mode, the default of SparkGLM's start.sh, does not. In Strict mode
on the engine commit that 0.3.6 ships, the suite passes 97 of 109: the 12 failures are three prompts that ask the
model to repeat tool-call markup as plain text (each sent four ways), where the longer reasoning
leads it to a different answer.
There are two ways to serve it:
Through LLooM, which is how the numbers above were measured. Its SparkGLM recipe installs this checkpoint in Spark mode on the 0.3.6 image, pinned at revision
6967e495of this repository (the same weight files as the current revision):lloom setup --recipe linux-nvidia-dgx-spark-glm53-atlas --model sage-s21-flash --additive --apply --yesThe check started from a local copy of the weights, which the Hugging Face tool verified against that revision; a first download of all 179 GB through setup was not run.
With SparkGLM's
start.sh: check outv0.3.6or later, setCHECKPOINT_REPO=Enntity/Sage-S2.1-Flash-NVFP4andCHECKPOINT_REVISION=the commit to pin in.env, and start with./start.sh --mode spark. This path has not been run end to end with this checkpoint.
The drafter SparkGLM uses for speculative decoding was trained on GLM-5.3-Flash, not on Sage.
Limits
- Scientific inputs. Molecules, crystal structures, proteins and the other Sage SciPerception inputs need the original repository's projectors and are not supported here.
- Other engines. Only SparkGLM was tested. The layout follows NVIDIA's GLM-5.3-Flash FP4-Local checkpoint, but vLLM, SGLang and others were not tried.
- Four Sparks. Tensor parallel 4 was not measured, and neither were LLooM's
--memory sharedand--streams 8options. - Tokenizer rule. The original tokenizer encodes control strings such as
<tool_call>inside message content as ordinary text. SparkGLM has its own handling for literal markup; its tests for that pass with this checkpoint, but it is not the original code path. - MTP layer. Its experts are rounded to nearest, not calibrated.
Files
SHA256SUMS lists the weight, tokenizer and configuration files. deployment_guide.md is the original repository's guide, unchanged.
receipts/ holds the evaluation outputs behind the tables above, the script that computes the paired
comparison, the list of calibration and evaluation sources, and the SparkGLM suite and speed results.
- Downloads last month
- -
Model tree for Enntity/Sage-S2.1-Flash-NVFP4
Base model
internlm/Sage-S2.1-Flash