Instructions to use tiny-random/kimi-k3.1-pro with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tiny-random/kimi-k3.1-pro with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="tiny-random/kimi-k3.1-pro", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("tiny-random/kimi-k3.1-pro", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tiny-random/kimi-k3.1-pro with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tiny-random/kimi-k3.1-pro"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/kimi-k3.1-pro",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/tiny-random/kimi-k3.1-pro

SGLang

How to use tiny-random/kimi-k3.1-pro with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tiny-random/kimi-k3.1-pro" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/kimi-k3.1-pro",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tiny-random/kimi-k3.1-pro" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/kimi-k3.1-pro",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use tiny-random/kimi-k3.1-pro with Docker Model Runner:
```
docker model run hf.co/tiny-random/kimi-k3.1-pro
```

kimi-k3.1-pro / README.md

yujiepan

Upload folder using huggingface_hub

3a26692 verified about 17 hours ago

preview code

Raw History Blame Contribute Delete

5.04 kB

metadata

library_name: transformers
pipeline_tag: image-text-to-text
base_model:
  - moonshotai/Kimi-K3.1-Pro

This is a randomly initialized tiny model based on moonshotai/Kimi-K3.1-Pro for testing and debugging.

Feature	Source	This repo
Checkpoint tensor elements	57.36B BF16, 11.12M F32, 1.45T U8	13.98M BF16, 163.07K F32, 6.68M U8
Safetensors weight files	1.56 TB (96 files)	35.58 MB (model.safetensors)
Total / DSA-gated layers	93 / 24 (plus 69 KDA, 1 dense)	17 / 5 (plus 11 KDA, 1 dense)
Attention composition	69 KDA + 24 gated DSA	11 KDA + 5 DSA cadence
Hidden width / MLP width	7,168 / 33,792	8 / 32
Full-attn Q heads / head dim	96 / 128 (nope 128, rope 64, v 128)	8 / 128 (same per-head dims)
MLA q / kv LoRA rank	1,536 / 512	128 / 512
Routed / shared experts, top-k	896 / 2, 16	16 / 2, 8
LSA index heads / head dim / topk	32 / 128 / 2,048	32 / 128 / 2,048 (unchanged)
Physical DSA indexer layers	24 (dsa_layers - 1)	[3, 7, 11, 15, 16]
Routed expert precision	MXFP4 packed (group 32, E8M0 scales)	Same
Vision depth / hidden / heads	27 / 1,024 / 12	2 / 64 / 2
Patch size	14	Same
Vocabulary rows / tying	163,840 / untied	Same
Maximum context length	1,048,576	Same

K3.1-Pro ships no modeling code; the included pinned K3 modeling_kimi_k3.py/modeling_kimi_linear.py backs the tiny model. The LSA indexer tensors are included forward-compatibly — released Transformers/vLLM/SGLang have no KimiK31 gated-DSA kernel yet, so the DSA layers run their MLA core.

File path	Size
model.safetensors	35.6MB

Example usage

Transformers (unverified)

Loads with the pinned K3 reference modeling on GPU (KDA delta attention uses Triton via fla-core). The K3.1 gated-DSA layers run their MLA attention core; the LSA indexer tensors are present but not consumed because released Transformers lacks the gated-DSA operator.

def transformers_example(model_id: str):
    import numpy as np
    import torch
    from PIL import Image
    from transformers import AutoModel, AutoProcessor

    model_id = str(model_id)
    processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
    model = AutoModel.from_pretrained(
        model_id,
        dtype=torch.bfloat16,
        device_map="cuda",
        trust_remote_code=True,
        attn_implementation="eager",
    ).eval()
    image = Image.fromarray(np.random.default_rng(42).integers(0, 255, (56, 56, 3), dtype=np.uint8))
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": "Hello, world!"},
            ],
        }
    ]
    inputs = processor(messages=messages, return_tensors="pt").to(model.device)
    with torch.no_grad():
        outputs = model.generate(**inputs, max_new_tokens=16)
    generated_ids = outputs.sequences if hasattr(outputs, "sequences") else outputs
    print(processor.decode(generated_ids[0].detach().cpu().tolist()))


if __name__ == "__main__":
    transformers_example(model_id="scratch/kimi-k3.1-pro-tiny-random")

vLLM serving (unverified)

Requires a vLLM build with KimiK31 gated-DSA support (newer than the released wheel at tiny-model creation time). See https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro.

# K3.1 gated-DSA serving requires a vLLM build with KimiK31 DSA support (newer than the
# released wheel at tiny-model creation time). See https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro
vllm serve scratch/kimi-k3.1-pro-tiny-random \
    --trust-remote-code \
    --tensor-parallel-size 8 \
    --load-format safetensors \
    --max-model-len 131072 \
    --gpu-memory-utilization 0.90

SGLang serving (unverified)

Requires an SGLang build newer than 0.5.23 with the KimiK31 DSA indexer (the gated-DSA kernels had not landed in the released wheel at tiny-model creation time).

# K3.1 gated-DSA serving requires an SGLang build newer than 0.5.23 (the KimiK31 DSA indexer
# kernels had not landed in the released wheel at tiny-model creation time).
python3 -m sglang.launch_server \
    --model-path scratch/kimi-k3.1-pro-tiny-random \
    --trust-remote-code \
    --tp-size 8 \
    --chunked-prefill-size 16384 \
    --cuda-graph-backend-prefill breakable \
    --dtype bfloat16 \
    --kv-cache-dtype fp8_e4m3 \
    --reasoning-parser kimi_k3 \
    --tool-call-parser kimi_k3 \
    --mem-fraction-static 0.85

Creating this repo

Code: .meta/create_model.py
Model structure: .meta/model.log
Environment: .meta/env.json

Change log

2026-11-05: Initial version: 17-layer K3.1-Pro (12 KDA + 5 gated-DSA cadence) with the source LSA indexer contract and packed MXFP4 experts, reusing the pinned K3 remote modeling.

This repo is generated by GPT-6.1-Sol.