Kimi K3.1 Pro β Native Flare DCP8 and DCP16 Cores
This repository contains native Flare exports of Kimi K3.1 Pro with MoonViT image support, DFlash2 speculation, and independent target/draft precision controls.
Upload status: the non-weight portions are complete and verified. Weight publication is still in progress. A directory is ready for native serving only when its _READY.json marker is present. _NONWEIGHTS_READY.json means metadata and other non-weight files are available for review, not that the core is complete.
| Variant | Repository path | Graphs | Status |
|---|---|---|---|
| DCP8 / TP16 | gb202-dcp8/ |
71 | Non-weight files ready; weights uploading |
| DCP16 / TP16 | gb202-dcp16/ |
71 | Non-weight files ready; weights uploading |
Precision and execution configuration
Both variants use:
- 93 target layers and five dense DFlash2 layers;
- target online MXFP8 scope
kda-input; - DFlash2 body context, attention latent/query/output, and MLP projections converted to MXFP8;
- an independent NVFP4 proposal copy of the target vocabulary head;
- BF16 target verification vocabulary head;
- native MXFP4 routed expert weights with lossless CSF block scales;
- checkpoint precision for normalization, convolution, candidate-selector, and absorbed key/value weights;
- DFlash2 with seven draft tokens, adaptive verification, and no history drafts;
- prefill chunk 4096, maximum concurrency four, and image concurrency four;
- MoonViT with 256, 1024, 4096, 16384, and 65536 patch capacities.
DCP8 uses two eight-rank context groups. DCP16 uses all sixteen ranks as context readers. This changes cache geometry, not the target or draft precision.
Source revisions
- Target:
moonshotai/Kimi-K3.1-Pro, revisiond9e99b0792a5e93b78ea9bea62a68167e36ddd9e. - DFlash2:
RadixArk/Kimi-K3.1-Dflash2-Preview, revisionea257b38a8ee755b65287bb8ccd0938b519ce26b. - FlashInfer:
aa6c4221586f7607c2bca64abbcdc0699845cf3b. - Native cache handling: Flare main containing PR #286.
- Independent DFlash2 body selection: PR #291, or an equivalent source tree. A checkout without #291 does not provide
--draft-body-quantization. After #291 merges, no additional patch is required.
Run exports in a prepared Flare/Ignition environment with ignition importable and flare_heliopause built from the same source tree. The commands below use /path/to/Kimi-K3.1-Pro for the source checkpoint and /path/to/preparation.toml for a valid local Flare configuration.
Export DCP8
The first vision-enabled export has no compatible text-only cached weight layout, so it must run without --reuse-weights:
python -m ignition export /path/to/Kimi-K3.1-Pro \
--config /path/to/preparation.toml \
--gpus 0-15 \
--max-concurrency 4 \
--dcp-size 8 \
--image-concurrency 4 \
--prefill-chunk-size 4096 \
--speculative-method dflash2 \
--mtp-draft-tokens 7 \
--speculative-token-capacity 32 \
--speculative-graph-budget 16 \
--online-mxfp8 \
--online-mxfp8-scope kda-input \
--draft-body-quantization mxfp8 \
--draft-head-quantization nvfp4
Ignition prints Exported cache entry: ... at completion. Keep this DCP8 core as the model's cached default version when using weight reuse for DCP16.
Export DCP16
With the compatible DCP8 vision core selected as the cached default, the DCP16 export can use its verified weight objects:
python -m ignition export /path/to/Kimi-K3.1-Pro \
--config /path/to/preparation.toml \
--gpus 0-15 \
--max-concurrency 4 \
--dcp-size 16 \
--image-concurrency 4 \
--prefill-chunk-size 4096 \
--speculative-method dflash2 \
--mtp-draft-tokens 7 \
--speculative-token-capacity 32 \
--speculative-graph-budget 16 \
--online-mxfp8 \
--online-mxfp8-scope kda-input \
--draft-body-quantization mxfp8 \
--draft-head-quantization nvfp4 \
--reuse-weights
--reuse-weights does not skip checkpoint loading or preparation. It reuses signed, layout-compatible cached payloads instead of rehashing them during final publication. It is not valid for the initial text-only-to-vision layout change. Remove the flag for an ordinary DCP16 export, or if the cached default is not the compatible DCP8 vision core.
LIL signing
A publishing configuration must identify the official key and repository, for example:
[publishing]
signing_key = "/secure/path/signing-key.pem"
namespace = "local-inference-lab"
official_source = "https://huggingface.co/local-inference-lab/kimi-k3.1-pro-flare"
Promote the development export with:
python -m ignition sign \
--config /path/to/official-signing.toml \
/path/to/exported-core \
--output /path/to/lil-signed-core
Supply the key passphrase through a secure prompt. Do not place the private key or passphrase in the repository, scripts, or logs. The resulting publisher metadata is local-inference-lab with development: false.
Native serving
Start the signed core without a manual memory reserve:
flare run /path/to/lil-signed-core \
--config /path/to/preparation.toml \
--offline \
--gpus 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 \
--max-concurrency 4 \
--kv-cache-limit memory \
--publisher-keys /path/to/publisher.json \
--host 0.0.0.0 \
--port 8171 \
--served-model-name Kimi-K3.1-Pro \
--mtp-draft-tokens 7 \
--mtp-adaptive-verification \
--no-history-drafts
Binding 0.0.0.0 exposes the API on the host's network interfaces. Apply appropriate network access controls.
Verified results
Measurements used real weights, native serving, C1/C4 request concurrency, and three verified-cold 32K prefills. C1/C4 include tokens/step and scheduler milliseconds/step rather than only request throughput.
| Result | DCP8 | DCP16 |
|---|---|---|
| KV tokens | 1,776,640 | 3,557,376 |
| Shared snapshot slots | 86 | 88 |
| Manual memory reserve | none | none |
| Startup OOM | none | none |
| C1 completion tok/s | 92.791 | 88.342 |
| C1 tokens/step | 1.9157 | 1.8634 |
| C1 ms/step | 20.660 | 21.108 |
| C4 completion tok/s | 194.082 | 178.441 |
| C4 tokens/step | 4.3165 | 3.9536 |
| C4 ms/step | 22.257 | 22.173 |
| Cold 32K aggregate prompt tok/s | 2368.045 | 2365.112 |
| Mean 32K TTFT | 13.838 s | 13.855 s |
A real image request produced the expected answer Red and recorded 100 multimodal image tokens on both variants.
Limitations
C4 acceptance is weaker than the earlier combined target-and-draft all configuration. These runs are not a controlled same-vision A/B, so a performance or quality benefit from draft-body MXFP8 is not established. No full model-quality benchmark was run. Nonempty response and image smoke checks do not replace numerical or acceptance-quality evaluation.
Source checkpoint license terms remain applicable; consult the source model card.