Blame for README.md

---
library_name: transformers
pipeline_tag: image-text-to-text
base_model:
- moonshotai/Kimi-K3.1-Pro
---

This is a randomly initialized tiny model based on [moonshotai/Kimi-K3.1-Pro](https://huggingface.co/moonshotai/Kimi-K3.1-Pro) for testing and debugging.

| Feature | Source | This repo |
|---|---|---|
| Checkpoint tensor elements | 57.36B BF16, 11.12M F32, 1.45T U8 | 13.98M BF16, 163.07K F32, 6.68M U8 |
| Safetensors weight files | 1.56 TB (96 files) | 35.58 MB (model.safetensors) |
| Total / DSA-gated layers | 93 / 24 (plus 69 KDA, 1 dense) | 17 / 5 (plus 11 KDA, 1 dense) |
| Attention composition | 69 KDA + 24 gated DSA | 11 KDA + 5 DSA cadence |
| Hidden width / MLP width | 7,168 / 33,792 | 8 / 32 |
| Full-attn Q heads / head dim | 96 / 128 (nope 128, rope 64, v 128) | 8 / 128 (same per-head dims) |
| MLA q / kv LoRA rank | 1,536 / 512 | 128 / 512 |
| Routed / shared experts, top-k | 896 / 2, 16 | 16 / 2, 8 |
| LSA index heads / head dim / topk | 32 / 128 / 2,048 | 32 / 128 / 2,048 (unchanged) |
| Physical DSA indexer layers | 24 (dsa_layers - 1) | [3, 7, 11, 15, 16] |
| Routed expert precision | MXFP4 packed (group 32, E8M0 scales) | Same |
| Vision depth / hidden / heads | 27 / 1,024 / 12 | 2 / 64 / 2 |
| Patch size | 14 | Same |
| Vocabulary rows / tying | 163,840 / untied | Same |
| Maximum context length | 1,048,576 | Same |

K3.1-Pro ships no modeling code; the included pinned K3 `modeling_kimi_k3.py`/`modeling_kimi_linear.py` backs the tiny model. The LSA indexer tensors are included forward-compatibly — released Transformers/vLLM/SGLang have no KimiK31 gated-DSA kernel yet, so the DSA layers run their MLA core.

| File path | Size |
|------|------|
| model.safetensors | 35.6MB |

### Example usage

#### Transformers (unverified)

Loads with the pinned K3 reference modeling on GPU (KDA delta attention uses Triton via fla-core). The K3.1 gated-DSA layers run their MLA attention core; the LSA indexer tensors are present but not consumed because released Transformers lacks the gated-DSA operator.

```python
def transformers_example(model_id: str):
    import numpy as np
    import torch
    from PIL import Image
    from transformers import AutoModel, AutoProcessor

    model_id = str(model_id)
    processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
    model = AutoModel.from_pretrained(
        model_id,
        dtype=torch.bfloat16,
        device_map="cuda",
        trust_remote_code=True,
        attn_implementation="eager",
    ).eval()
    image = Image.fromarray(np.random.default_rng(42).integers(0, 255, (56, 56, 3), dtype=np.uint8))
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": "Hello, world!"},
            ],
        }
    ]
    inputs = processor(messages=messages, return_tensors="pt").to(model.device)
    with torch.no_grad():
        outputs = model.generate(**inputs, max_new_tokens=16)
    generated_ids = outputs.sequences if hasattr(outputs, "sequences") else outputs
    print(processor.decode(generated_ids[0].detach().cpu().tolist()))


if __name__ == "__main__":
    transformers_example(model_id="scratch/kimi-k3.1-pro-tiny-random")
```

#### vLLM serving (unverified)

Requires a vLLM build with KimiK31 gated-DSA support (newer than the released wheel at tiny-model creation time). See https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro.

```bash
# K3.1 gated-DSA serving requires a vLLM build with KimiK31 DSA support (newer than the
# released wheel at tiny-model creation time). See https://recipes.vllm.ai/moonshotai/Kimi-K3.1-Pro
vllm serve scratch/kimi-k3.1-pro-tiny-random \
    --trust-remote-code \
    --tensor-parallel-size 8 \
    --load-format safetensors \
    --max-model-len 131072 \
    --gpu-memory-utilization 0.90
```

#### SGLang serving (unverified)

Requires an SGLang build newer than 0.5.23 with the KimiK31 DSA indexer (the gated-DSA kernels had not landed in the released wheel at tiny-model creation time).

```bash
# K3.1 gated-DSA serving requires an SGLang build newer than 0.5.23 (the KimiK31 DSA indexer
# kernels had not landed in the released wheel at tiny-model creation time).
python3 -m sglang.launch_server \
    --model-path scratch/kimi-k3.1-pro-tiny-random \
    --trust-remote-code \
    --tp-size 8 \
    --chunked-prefill-size 16384 \
    --cuda-graph-backend-prefill breakable \
    --dtype bfloat16 \
    --kv-cache-dtype fp8_e4m3 \
    --reasoning-parser kimi_k3 \
    --tool-call-parser kimi_k3 \
    --mem-fraction-static 0.85
```

### Creating this repo

- Code: [.meta/create_model.py](.meta/create_model.py)
- Model structure: [.meta/model.log](.meta/model.log)
- Environment: [.meta/env.json](.meta/env.json)

### Change log

- 2026-11-05: Initial version: 17-layer K3.1-Pro (12 KDA + 5 gated-DSA cadence) with the source LSA indexer contract and packed MXFP4 experts, reusing the pinned K3 remote modeling.

*This repo is generated by GPT-6.1-Sol.*