gatilin/kimi-k3.1-pro-vision-encoder

Kimi K3.1 Pro MoonViT-V2 vision tower and projector

This repository contains the vision_tower.* and mm_projector.* tensors extracted without modification from Kimi K3.1 Pro at revision 6d0a0202956796543364d666ab19c6ad1a494ee1. The language model is not included. This is an independent community extraction, not an official Moonshot AI release.

The 27-layer MoonViT-V2 vision encoder and patch-merger projector are included together. All 168 tensors retain their original keys, shapes, dtypes and values. tensor_mapping.json lists their source shard locations, while extraction.json records byte-level verification and SHA-256.

The visual implementation in modeling_moonvit.py is adapted from the official Kimi K3 reference implementation. The K3 and K3.1 Pro vision_config values match. K3.1 Pro does not publish that implementation in its model repository, so compatibility with its full production inference stack is not claimed. The original K3.1 Pro image processor is included. The bundled code uses eager attention for portability.

Loading

Install compatible versions of PyTorch, Transformers, Accelerate, safetensors, Pillow and NumPy. Then download the repository and use:

import sys
import torch
from PIL import Image
from huggingface_hub import snapshot_download
root = snapshot_download('gatilin/kimi-k3.1-pro-vision-encoder')
sys.path.insert(0, root)
from component import load_model, load_processor
model = load_model(root)
processor = load_processor(root)
image = Image.open('image.jpg').convert('RGB')
inputs = processor.preprocess([{'type': 'image', 'image': image}], return_tensors='pt')
pixels = inputs['pixel_values'].to(next(model.vision_tower.parameters()).dtype)
with torch.inference_mode():
    vision_features = model.vision_tower(pixels, inputs['grid_thws'])
    projected_features = model.mm_projector(vision_features)

component.py must be imported from the downloaded repository directory. Inputs are packed patch tensors produced by the bundled processor; they are not standard RGB BCHW tensors. Vision features are lists of spatially grouped tokens, and projected features are lists of text-embedding-sized vectors.

Validation and limits

See validation.json for the actual run and conditions. A local CPU smoke test does not establish numerical parity with full K3.1 Pro inference, quality across image resolutions, or CUDA and FlashAttention behavior.

Licenses

The weights, configuration and K3.1 Pro processor are distributed under the original Kimi K3.1-Pro License, which has specific conditions for certain commercial uses. The K3 reference-derived implementation is subject to the original Kimi K3 License, and contains the Apache 2.0 attribution notices preserved in the source. Review both license files before use.