Fidelity measurement: KL(reference || this quant) on frozen tokens, receipt-backed
What this is. A third-party fidelity measurement of RadixArk/GLM-5.3-NVFP4 @ 11af4cba759e6559eda70358a5778bd1bddddd78 (nvfp4, 4 bits per weight as declared) against its unquantized reference, made with quant-fidelity-suite. Measured by malaiwah, not by the model's author. Every number below is read from a sealed receipt named at the bottom.
| KL(reference ‖ candidate), mean tokenwise, nats | 0.05107107411834918 |
| top-1 agreement | 0.9363556424035173 |
| KL median / p95 / p99 / max | 0.003256846124508247 / 0.22865076586296776 / 0.8125921770529119 / 6.187286753691117 |
| reference root dataset | malaiwah/glm53-fidelity-root-v1 @ 9c4a29ee10f393ed2fdbdb9262c1192ddb1507b4 (capture 9eba97dddb4ff2e2…) |
| panel | panel--glm53.malaiwah.corpus5x5-v1, 25 contexts, 51175 scored positions |
| direction / vocabulary / accumulation | KL(reference |
| method | dequantize-and-run, weights only: nvfp4-modelopt-dequant-to-bf16 -- the ModelOpt NVFP4 dialect (quant_algo NVFP4, producer modelopt 0.47.0.dev91+g7ff81dd79): each routed-expert projection's packed e2m1 nibbles (group 16 along the input axis) are decoded to exact fp32 as e2m1 x weight_scale.f32 x weight_scale_2 on the capture device and cast once to bf16 under the official tensor name, bitwise the compressed-tensors reference on real fetched rows; routed experts only -- every non-routed tensor is carried as shipped (plain bf16 under the official names); the per-tensor input_scale is an activation quantity and is NOT applied (static-nvfp4-not-applied), so the measurement is weights-only; same engine, schedule and device as the reference capture |
| determinism | two fresh processes captured the candidate; both sealed captures carry content digest 768c007dfc51818f… (self-comparison 0.0) |
| comparability class | advisory |
Scope (scope_digest): attn.o=native:bf16@16|attn.other=native:bf16@16|attn.qkv=native:bf16@16|embed_tokens=native:bf16@16|lm_head=native:bf16@16|mlp.down=native:bf16@16|mlp.gate=native:bf16@16|mlp.up=native:bf16@16|moe.experts=quantized:nvfp4@4|moe.router=native:mixed|moe.shared_expert=native:bf16@16|mtp=native:mixed|norm=native:bf16@16|head=native|kv=bf16
Disclosures on the comparison receipt:
native_head_replay(info): HEAD-1d: each side replayed through its own sealed head (reference 864f488a0074, candidate 864f488a0074); head error is inside the measurement, as under HEAD-2, and nothing is substituted. The heads are content-identical.activation_quantization_not_captured(caveat): candidate was captured from a bf16 materialisation of its weights (nvfp4-modelopt-dequant-to-bf16); the checkpoint declares activation quantization (static-nvfp4-not-applied) that a weights-only capture does not apply, so a served deployment also quantizes activations at runtime. That term is not in this number, which is expected to understate the served divergence; it is not a mathematical bound. The comparison is advisory.
What this number does not do. It is a same-lane distance from one reference capture on one panel. It does not rank this artifact against numbers measured on another panel, lane or reference, and a same-lane root does not retroactively upgrade rows measured against another teacher. Per-window scatter exceeds the gap between adjacent bit-widths; compare only within a group whose comparability keys match.
Receipts.
- comparison receipt
receipt_sha256230346ad147a048a265ab6b5c28ca45e599335aabe8f1d6dffa620af2b44a714 - root-qualification receipt
receipt_sha2569c7772b61dd904057551ef7d62fa7cc001b35abbbf5045aead6ca280f67851f6(canonical dataset_sha256b0c99cfcdfe8527cb782bc73fa0584ff4a96ae691e77f5065cfe1f11ce5a0a54, repeat96566fe2f692e06b270a572df8812e8fbd0e3bb7a77ad221575eb38b03668ef2) - job
job_id_fullfa62e646a2279f58ffc55c4a991a905ac7e8ad9b23b79d9f23fa3cbb245a0dc1 - candidate capture dataset published:
malaiwah/glm53-fidelity-nvfp4-radixark-v1@8dbc5abecfd41352dc2932f066cf016c454dc6c4
Reproduce: fetch the two datasets named above and run fidelity-dataset compare --reference <root> --candidate <this> --own-heads; the receipt's estimator block is the exact recipe. Questions and corrections are welcome here; the registry files this as a third-party row (measured_by enumerated, never conflated with author-reported numbers).