license: apache-2.0
base_model: moonshotai/Kimi-K3.1-Pro
tags:
- kimi-k3.1
- moe
- quantization
- mixed-bit
- strata
- dr-triton
pipeline_tag: image-text-to-text
Kimi-K3.1-Pro-Coder (2.8T Mixed-Bit MoE + Extra Coding)
Grawnch/Kimi-K3.1-Pro-Coder is a full 96-shard (774.97 GB / 721.75 GiB) Strata-Coder 3-Tier Mixed-Bit MoE quantization of moonshotai/Kimi-K3.1-Pro (2.8T parameters, 93 layers, 896 routed experts per MoE layer, top_k=16 LatentMoE with SiTU-GLU, KDA linear attention + gated NoPE MLA full attention, and AttnRes cross-layer residual mixing).
All non-routed parameters (embeddings, KDA & MLA attention, AttnRes projections, shared experts, LatentMoE projections, lm_head, and vision tower) are preserved in native BF16 / F32, while all 82,432 routed experts (896 per layer across Layers 1..92) are compressed using Strata-Coder 3-Tier Mixed-Bit MoE Compression calibrated on coding and mathematical reasoning router activation frequencies:
| Expert / Weight Tier | Routed Experts / Layer | Effective BPW | Total Size (GB / GiB) |
Default Storage Tier (Strata) |
Matrix Cosine vs Base MXFP4 |
|---|---|---|---|---|---|
Base Non-Routed (BF16/F32) |
Shared + KDA/MLA + lm_head |
16.0 bpw |
114.76 GB (106.88 GiB) |
GPU VRAM / Shared | 100.00% (Exact Native) |
| Tier 1 (Hot Coding/Math) | 134 / 896 (15.0%) |
2.81 bpw |
160.58 GB (149.55 GiB) |
Tier 1: Hot GPU VRAM | 96.95% |
| Tier 2 (Warm General) | 269 / 896 (30.0%) |
2.56 bpw |
254.53 GB (237.05 GiB) |
Tier 2: Warm Pinned System RAM | 94.53% |
| Tier 3 (Cold Tail) | 493 / 896 (55.0%) |
1.44 bpw |
244.95 GB (228.13 GiB) |
Tier 3: Cold NVMe SSD (mmap) |
82.51% |
| Full Model Total | All 96 / 96 Shards | ~1.98 bpw (MoE) |
774.97 GB (721.75 GiB, -56.2%) |
VRAM + RAM + NVMe | 93.09% – 98.22% MoE Output |
Full 93-Layer End-to-End Evaluation (NVIDIA RTX PRO 6000 Blackwell)
Evaluated across all 93 layers (Layer 0 through Layer 92 + Shard 94 lm_head) on an NVIDIA RTX PRO 6000 Blackwell Server Edition (102.6 GB GDDR7 VRAM) over two zero-leakage benchmarks (22,307 total tokens across 70 sequences):
AIME 2025(All 30 Problems, Parts I & II): Zero-leakage exact integer prediction at\boxed{.- Rolling-Window
LiveCodeBench(2024.10 – 2025.05, 30 Problems: 10 Easy, 10 Medium, 10 Hard across LeetCode, AtCoder, and Codeforces): Zero-leakage functional unit-test execution (Pass@1) + rolling-window code perplexity (PPL).
| Benchmark / Metric | moonshotai/Kimi-K3.1-Pro (Base Ref) |
Grawnch/Kimi-K3.1-Pro-Coder (774.97 GB) |
Delta / Agreement |
|---|---|---|---|
AIME 2025 (All 30 Problems, Pass@1) |
29 / 30 (96.67%) |
29 / 30 (96.67%) |
0.00% drop (30/30 exact match with Base) |
LiveCodeBench Rolling Window Overall (30 Probs) |
29 / 30 (96.67%) |
29 / 30 (96.67%) |
0.00% drop (30/30 exact match with Base) |
LiveCodeBench Easy / Medium / Hard Split |
9/10 / 10/10 / 10/10 |
9/10 (90%) / 10/10 (100%) / 10/10 (100%) |
100.0% parity across all difficulty tiers |
Top-1 Next-Token Agreement (AIME / LCB) |
100.00% |
95.20% / 95.06% |
95.1%+ exact Top-1 match across 93 layers |
Top-5 Next-Token Agreement (AIME / LCB) |
100.00% |
99.91% / 99.92% |
99.91%+ Top-5 coverage |
Mean KL Divergence (KLD, AIME / LCB) |
0.00000 |
0.01912 / 0.02248 |
~0.020 nats end-to-end across 93 layers |
AIME 2025 Math Perplexity (PPL) |
2.2135 |
2.2358 |
+0.0223 (+1.01%) |
LiveCodeBench Hard Perplexity (PPL) |
2.1136 |
2.1562 |
+0.0426 (+2.02%) |
LiveCodeBench Overall Perplexity (PPL) |
2.6911 |
2.7629 |
+0.0718 (+2.67%) |
MoE Layer Throughput (RTX PRO 6000) |
— | 3,382.7 layer-tok/s |
36.77 tok/s full 93-layer batch |
Runtime & 3-Tier Offloading with Strata and dr-triton
- Engine & 3-Tier Loader (
Hot VRAM / Warm RAM / Cold NVMe+RouterLookahead): https://github.com/grawnchtoader/strata - Fused Triton Dequantization Kernels: https://github.com/grawnchtoader/dr-triton