Kimi K3.1 Pro β€” Native Flare DCP8 and DCP16 Cores

This repository contains native Flare exports of Kimi K3.1 Pro with MoonViT image support, DFlash2 speculation, and independent target/draft precision controls.

Upload status: the non-weight portions are complete and verified. Weight publication is still in progress. A directory is ready for native serving only when its _READY.json marker is present. _NONWEIGHTS_READY.json means metadata and other non-weight files are available for review, not that the core is complete.

Variant Repository path Graphs Status
DCP8 / TP16 gb202-dcp8/ 71 Non-weight files ready; weights uploading
DCP16 / TP16 gb202-dcp16/ 71 Non-weight files ready; weights uploading

Precision and execution configuration

Both variants use:

  • 93 target layers and five dense DFlash2 layers;
  • target online MXFP8 scope kda-input;
  • DFlash2 body context, attention latent/query/output, and MLP projections converted to MXFP8;
  • an independent NVFP4 proposal copy of the target vocabulary head;
  • BF16 target verification vocabulary head;
  • native MXFP4 routed expert weights with lossless CSF block scales;
  • checkpoint precision for normalization, convolution, candidate-selector, and absorbed key/value weights;
  • DFlash2 with seven draft tokens, adaptive verification, and no history drafts;
  • prefill chunk 4096, maximum concurrency four, and image concurrency four;
  • MoonViT with 256, 1024, 4096, 16384, and 65536 patch capacities.

DCP8 uses two eight-rank context groups. DCP16 uses all sixteen ranks as context readers. This changes cache geometry, not the target or draft precision.

Source revisions

  • Target: moonshotai/Kimi-K3.1-Pro, revision d9e99b0792a5e93b78ea9bea62a68167e36ddd9e.
  • DFlash2: RadixArk/Kimi-K3.1-Dflash2-Preview, revision ea257b38a8ee755b65287bb8ccd0938b519ce26b.
  • FlashInfer: aa6c4221586f7607c2bca64abbcdc0699845cf3b.
  • Native cache handling: Flare main containing PR #286.
  • Independent DFlash2 body selection: PR #291, or an equivalent source tree. A checkout without #291 does not provide --draft-body-quantization. After #291 merges, no additional patch is required.

Run exports in a prepared Flare/Ignition environment with ignition importable and flare_heliopause built from the same source tree. The commands below use /path/to/Kimi-K3.1-Pro for the source checkpoint and /path/to/preparation.toml for a valid local Flare configuration.

Export DCP8

The first vision-enabled export has no compatible text-only cached weight layout, so it must run without --reuse-weights:

python -m ignition export /path/to/Kimi-K3.1-Pro \
    --config /path/to/preparation.toml \
    --gpus 0-15 \
    --max-concurrency 4 \
    --dcp-size 8 \
    --image-concurrency 4 \
    --prefill-chunk-size 4096 \
    --speculative-method dflash2 \
    --mtp-draft-tokens 7 \
    --speculative-token-capacity 32 \
    --speculative-graph-budget 16 \
    --online-mxfp8 \
    --online-mxfp8-scope kda-input \
    --draft-body-quantization mxfp8 \
    --draft-head-quantization nvfp4

Ignition prints Exported cache entry: ... at completion. Keep this DCP8 core as the model's cached default version when using weight reuse for DCP16.

Export DCP16

With the compatible DCP8 vision core selected as the cached default, the DCP16 export can use its verified weight objects:

python -m ignition export /path/to/Kimi-K3.1-Pro \
    --config /path/to/preparation.toml \
    --gpus 0-15 \
    --max-concurrency 4 \
    --dcp-size 16 \
    --image-concurrency 4 \
    --prefill-chunk-size 4096 \
    --speculative-method dflash2 \
    --mtp-draft-tokens 7 \
    --speculative-token-capacity 32 \
    --speculative-graph-budget 16 \
    --online-mxfp8 \
    --online-mxfp8-scope kda-input \
    --draft-body-quantization mxfp8 \
    --draft-head-quantization nvfp4 \
    --reuse-weights

--reuse-weights does not skip checkpoint loading or preparation. It reuses signed, layout-compatible cached payloads instead of rehashing them during final publication. It is not valid for the initial text-only-to-vision layout change. Remove the flag for an ordinary DCP16 export, or if the cached default is not the compatible DCP8 vision core.

LIL signing

A publishing configuration must identify the official key and repository, for example:

[publishing]
signing_key = "/secure/path/signing-key.pem"
namespace = "local-inference-lab"
official_source = "https://huggingface.co/local-inference-lab/kimi-k3.1-pro-flare"

Promote the development export with:

python -m ignition sign \
    --config /path/to/official-signing.toml \
    /path/to/exported-core \
    --output /path/to/lil-signed-core

Supply the key passphrase through a secure prompt. Do not place the private key or passphrase in the repository, scripts, or logs. The resulting publisher metadata is local-inference-lab with development: false.

Native serving

Start the signed core without a manual memory reserve:

flare run /path/to/lil-signed-core \
    --config /path/to/preparation.toml \
    --offline \
    --gpus 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 \
    --max-concurrency 4 \
    --kv-cache-limit memory \
    --publisher-keys /path/to/publisher.json \
    --host 0.0.0.0 \
    --port 8171 \
    --served-model-name Kimi-K3.1-Pro \
    --mtp-draft-tokens 7 \
    --mtp-adaptive-verification \
    --no-history-drafts

Binding 0.0.0.0 exposes the API on the host's network interfaces. Apply appropriate network access controls.

Verified results

Measurements used real weights, native serving, C1/C4 request concurrency, and three verified-cold 32K prefills. C1/C4 include tokens/step and scheduler milliseconds/step rather than only request throughput.

Result DCP8 DCP16
KV tokens 1,776,640 3,557,376
Shared snapshot slots 86 88
Manual memory reserve none none
Startup OOM none none
C1 completion tok/s 92.791 88.342
C1 tokens/step 1.9157 1.8634
C1 ms/step 20.660 21.108
C4 completion tok/s 194.082 178.441
C4 tokens/step 4.3165 3.9536
C4 ms/step 22.257 22.173
Cold 32K aggregate prompt tok/s 2368.045 2365.112
Mean 32K TTFT 13.838 s 13.855 s

A real image request produced the expected answer Red and recorded 100 multimodal image tokens on both variants.

Limitations

C4 acceptance is weaker than the earlier combined target-and-draft all configuration. These runs are not a controlled same-vision A/B, so a performance or quality benefit from draft-body MXFP8 is not established. No full model-quality benchmark was run. Nonempty response and image smoke checks do not replace numerical or acceptance-quality evaluation.

Source checkpoint license terms remain applicable; consult the source model card.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support