Image-Text-to-Text
Transformers
Safetensors
kimi_k25
feature-extraction
compressed-tensors
conversational
custom_code
Eval Results
Instructions to use moonshotai/Kimi-K2.6 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use moonshotai/Kimi-K2.6 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="moonshotai/Kimi-K2.6", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("moonshotai/Kimi-K2.6", trust_remote_code=True) model = AutoModel.from_pretrained("moonshotai/Kimi-K2.6", trust_remote_code=True, device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use moonshotai/Kimi-K2.6 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "moonshotai/Kimi-K2.6" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K2.6", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/moonshotai/Kimi-K2.6
- SGLang
How to use moonshotai/Kimi-K2.6 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "moonshotai/Kimi-K2.6" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K2.6", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "moonshotai/Kimi-K2.6" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "moonshotai/Kimi-K2.6", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use moonshotai/Kimi-K2.6 with Docker Model Runner:
docker model run hf.co/moonshotai/Kimi-K2.6
Commit History
-
Add MMMU-Pro evaluation result (#41)
- Add MMMU-Pro evaluation result (a0131c813d71d07933dbf0e44748ee6b6cf5f50a) Co-authored-by: Nathan Habib <SaylorTwift@users.noreply.huggingface.co>7eb5002
-
Revert "use-fast-tokenizer (#38)"
weichuThis reverts commit 81bcaaa7947338ce2641983d98947bca0cc1a4d4.b5aabbf
-
use-fast-tokenizer (#38)
- use fast tokenizer, fix transformers v5 inference issues (100231dd6b04cec5fff2b2f649754f6b760e9476) - remove slow tokenizer (fce056ba284610b40a8fad680b2492fe0494df1d)81bcaaa
-
support-transformers-inference (#26)
- fix for transformers inference (baded737743fc2c5170986e6e6765b603a086d33)2755962
-
fix: add missing use_deterministic_attn parameter to MoonViT3dEncoder (#22)
- fix: add missing use_deterministic_attn parameter to MoonViT3dEncoder (f13e4d071155abcae05f85cc3c1b5fa56ea1b9fc) - also fix weight init issue (bed55d493c07900df87ed51274581025cbf8315a) Co-authored-by: Ace Eldeib <ace-coreweave@users.noreply.huggingface.co>9e6eed7
-
Update README.md
Verified signature5a49d03
-
Update SGLang deployment guide: point to K2.6 cookbook and use stable release (#6)
- Update SGLang deployment guide: point to K2.6 cookbook and use stable release (b5632fb78a0f133882b5d4ecfe53913221f3a039) - Use sglang>=0.5.10.post1 (drop [all] extra) (8a5354a4c1cde1ee18cebf4d61a6ae02f8916887) - Use uv (preferred) with --prerelease=allow for sglang install (06b359a11c97ad794eafbf128e3836ed03cfcfc5) Co-authored-by: Xinyuan Tong <JustinTong@users.noreply.huggingface.co>bc8e704
-
fix: typo
Verified signature2b2b88e -
Add evaluation results for HLE, GPQA, AIME, HMMT, SWE-Bench, and Terminal-Bench (#4)
- Add evaluation results for HLE, GPQA, AIME, HMMT, SWE-Bench, and Terminal-Bench (64ed7450cf8801fae6fc24d6bea85e034393d974) Co-authored-by: Nathan Habib <SaylorTwift@users.noreply.huggingface.co>d9cb81b -
add tech blog
weichu36d7872 -
update footnote
Verified signaturee69e669 -
Update README.md
Verified signature20008c5 -
Update README.md
Verified signature4931c56
-
Add files using upload-large-folder tool
e0ee936 -
initial commit
Verified signature344a634