Instructions to use microsoft/kosmos-2.5 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/kosmos-2.5 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="microsoft/kosmos-2.5")# pip install -U transformers accelerate # Load model directly from transformers import AutoImageProcessor, AutoModelForMultimodalLM processor = AutoImageProcessor.from_pretrained("microsoft/kosmos-2.5") model = AutoModelForMultimodalLM.from_pretrained("microsoft/kosmos-2.5", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use microsoft/kosmos-2.5 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "microsoft/kosmos-2.5" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/kosmos-2.5", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/microsoft/kosmos-2.5
- SGLang
How to use microsoft/kosmos-2.5 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "microsoft/kosmos-2.5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/kosmos-2.5", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "microsoft/kosmos-2.5" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "microsoft/kosmos-2.5", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use microsoft/kosmos-2.5 with Docker Model Runner:
docker model run hf.co/microsoft/kosmos-2.5
Commit History
-
Replace **dtype** with **torch_dtype** in Model Loading
Verified signatureThe model card for Kosmos 2.5 has been updated to address an issue where the dtype parameter in the from_pretrained method was not functioning as intended. The code has been revised to use torch_dtype instead, aligning with the standard PyTorch API convention in Hugging Face's Transformers library. This change ensures proper data type specification during model loading and resolves issues with the previous dtype parameter. The updated code is as follows: ``` model = Kosmos2_5ForConditionalGeneration.from_pretrained(repo, device_map=device, torch_dtype=dtype) ``` This update does not alter the model's functionality but ensures compatibility with the Hugging Face Transformers API.29b2ebe -
Add code snippets, library name and pipeline tag (#18)
Verified signature- Add code snippets, library name and pipeline tag (21ab46e9261f3c32615e400e3c3f7e67cd227ddf) Co-authored-by: Niels Rogge <nielsr@users.noreply.huggingface.co>ec3c805
-
Update after integration (#17)
Verified signature- update after integration (b9b6785fb4884bc79302d2710585c99675639794) - update after integration (6d11b0db7c3041a2e01f09fb965e648f9df83b16)042d713
-
Upload receipt_00008.png
Verified signature2e33681 -
Delete receipt_00008.png
Verified signature3e44b1b
-
Update README.md
Verified signature745639e -
Update README.md
Verified signature0118317
-
Upload 2 files
Verified signature14d28b7 -
support hf
Ubuntuf39cf74
-
Update README.md
Verified signatureedb09fc
-
add hf
Ubuntuf29bcc9 -
update
Ubuntufd3dac7
-
add hf version
Dod-ob80e777
-
Update README.md
Verified signature207fa63
-
Update README.md
Verified signature1f30eef -
Update README.md
Verified signature788d7ca
-
Update README.md
Verified signature8ceeda6 -
Update README.md
Verified signaturec1681f1 -
first
76840c7 -
initial commit
Verified signature85bcd92