Commit History
-
Model card: add IFBench and SciCode results
Verified signaturee52f170
-
Update README.md
Verified signature3315e86 -
Model card: bundled A_log fix now follows #150
Verified signature40a9f21 -
Fix A_log: size by num_heads, slice the zero-padded checkpoint at load
Verified signatureThe previous bundled fix enlarged A_log to head_dim (128) to match the checkpoint. That makes from_pretrained succeed and then fails on the first forward: fla/ops/kda/gate.py does A_log.view(H, 1) with H = g.shape[-2] = num_heads, so a 128-element parameter raises "shape '[96, 1]' is invalid for input of size 128". It fixed loading and broke inference; nothing caught it because the offline cast never runs a forward and SGLang uses its own model implementation rather than this remote code. Now sizes the parameter at num_heads and narrows the checkpoint tensor in _load_from_state_dict, following https://huggingface.co/moonshotai/Kimi-K3/discussions/150 . The checkpoint holds 96 trained decay values zero-padded to 128 (verified: the [96:128] tail is exactly zero in all 69 A_log tensors), so the slice is lossless. A non-zero tail raises instead of being silently truncated.9942256 -
Update README.md
Verified signature00d2892 -
Model card: state both A_log fixes without ranking them
Verified signaturebc01502 -
Model card: final GPQA Diamond result, 92.96% pass@1 avg-of-16
Verified signature58a284f -
Model card: A_log is zero-padded; note discussion #150 as the better fix
Verified signature6ec7b0f
-
Update README.md
Verified signaturec5d5e69 -
Update README.md
Verified signature3980e06 -
Update README.md
Verified signature12127b7 -
Update README.md
Verified signaturedef9877 -
Update README.md
Verified signatured03e3d5 -
Model card: scope the precision note to the weight level, unbold none
Verified signature4af146e -
Model card: move the W4A16 note to the end of the Deployment section
Verified signaturee9efe32 -
Model card: drop 'Honest' from the precision note heading
Verified signaturecddfb2a -
Rewrite model card: SGLang deployment recipe, GPQA 93.22%, W4A16 reality
Verified signatureThe previous card claimed the checkpoint was not servable by vLLM or SGLang. It is: SGLang kimi-k3 branch, 2x8 B200, TP16, --moe-runner-backend marlin. Adds the full multi-node launch command (no credentials), the five non-obvious requirements that each independently block startup, GPQA Diamond = 93.22% (pass@1 avg-of-7), the querying caveat for a reasoning model, and an explicit note that marlin makes this a W4A16 deployment rather than W4A4. Also states plainly that the bit-exact cast uses none of NVFP4's precision advantages for weights, and why that is the correct trade.9e5aec5 -
Fix NVFP4 quantization metadata: stale per-group mxfp4 format, glob-style exclude_modules
Verified signatureThe generated config still described the MXFP4 source in three places, each of which makes the checkpoint unloadable: 1. `config_groups.group_0.format` was still `mxfp4-pack-quantized`. SGLang's Kimi-K3 argument override keys on this and forces `moe_runner_backend=flashinfer_mxfp4`, i.e. MXFP4 kernels over NVFP4 tensors. 2. `ignore` used compressed-tensors `re:` regexes. ModelOpt matches with `re.fullmatch(pattern.replace('.', r'\.').replace('*', '.*'))`, so those patterns never match; and it has no `exclude_modules` key at all. Since ModelOpt quantizes everything *except* `exclude_modules` (the inverse of compressed-tensors), unquantized modules were loaded as NVFP4 and failed a shape assertion. 3. `weights` still carried MXFP4 block/scale metadata. `exclude_modules` is now derived from the modules that actually kept an unpacked 2-D float `.weight` in the shards (2,398 of them, collapsed to 55 globs), verified to cover all of them while excluding none of the 247,296 quantized module…eb07696
-
Revert: Kimi-K3 uses a Python token renderer (encoding_k3.py), not a Jinja template; engines implement it natively
Verified signatured426143 -
Add chat_template.jinja (from moonshotai/Kimi-K3 discussions/66, minja variant)
Verified signaturedacb247 -
Add quant_algo=NVFP4 (flat format required by SGLang/vLLM loaders)
Verified signature78595ac -
Add processor files required for SGLang/vLLM startup (multimodal arch)
Verified signature3eb400c -
Set base_model_relation: quantized (was showing as Finetuned)
Verified signature8a65030 -
Add files using upload-large-folder tool
Verified signaturef1ac38e -
Add model card: bit-exact MXFP4->NVFP4 weight-only cast, deployment status
Verified signature7218095 -
Add files using upload-large-folder tool
Verified signaturebb123bd -
Add files using upload-large-folder tool
Verified signature3f06273 -
Fix config_groups to describe NVFP4 (group_size 16, E4M3 scales, tensor_group)
Verified signaturecae75c4 -
initial commit
Verified signature635e490