Image-Text-to-Text
Transformers
Safetensors
llava_onevision2_5
text-generation
multimodal
video-understanding
streaming
llava-onevision
conversational
custom_code
Instructions to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct
- SGLang
How to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct with Docker Model Runner:
docker model run hf.co/lmms-lab-encoder/LLaVA-OneVision-2.5-4B-Instruct
Model card: remove citation section for now
Browse files- README.md +0 -16
- README_CN.md +0 -16
README.md
CHANGED
|
@@ -181,22 +181,6 @@ The checkpoint already supports streaming: pass `streaming=True` to `apply_chat_
|
|
| 181 |
- Decode greedily (`do_sample=False`) and leave a generous `max_new_tokens`.
|
| 182 |
- For multiple choice or grounding, state the answer format you expect in the prompt.
|
| 183 |
|
| 184 |
-
## Citation
|
| 185 |
-
|
| 186 |
-
The LLaVA-OneVision-2.5 technical report is coming soon. Until then, please cite LLaVA-OneVision-2:
|
| 187 |
-
|
| 188 |
-
```bibtex
|
| 189 |
-
@misc{an2026llavaonevision2nextgenerationperceptualintelligence,
|
| 190 |
-
title = {LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence},
|
| 191 |
-
author = {Xiang An and Yin Xie and Feilong Tang and Yunyao Yan and Huajie Tan and Didi Zhu and Changrui Chen and Xiuwei Zhao and Bin Qin and Kaicheng Yang and Yifei Shen and Yuanhan Zhang and Kaichen Zhang and Wenkang Zhang and Zheng Cheng and Nansen Zhang and Chunsheng Wu and Chunjiang Ge and Zimin Ran and Dehua Song and Chunyuan Li and Shikun Feng and Ming Hu and Zhangquan Chen and Junbo Niu and Bo Li and Ziyong Feng and Ziwei Liu and Zongyuan Ge and Jiankang Deng},
|
| 192 |
-
year = {2026},
|
| 193 |
-
eprint = {2605.25979},
|
| 194 |
-
archivePrefix = {arXiv},
|
| 195 |
-
primaryClass = {cs.CV},
|
| 196 |
-
url = {https://arxiv.org/abs/2605.25979}
|
| 197 |
-
}
|
| 198 |
-
```
|
| 199 |
-
|
| 200 |
## License
|
| 201 |
|
| 202 |
Apache-2.0.
|
|
|
|
| 181 |
- Decode greedily (`do_sample=False`) and leave a generous `max_new_tokens`.
|
| 182 |
- For multiple choice or grounding, state the answer format you expect in the prompt.
|
| 183 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 184 |
## License
|
| 185 |
|
| 186 |
Apache-2.0.
|
README_CN.md
CHANGED
|
@@ -181,22 +181,6 @@ inputs = processor(text=text, videos=["example.mp4"],
|
|
| 181 |
- 使用贪心解码(`do_sample=False`),并给足 `max_new_tokens`。
|
| 182 |
- 选择题或 grounding 任务,在 prompt 里写明期望的回答格式。
|
| 183 |
|
| 184 |
-
## 引用
|
| 185 |
-
|
| 186 |
-
LLaVA-OneVision-2.5 的技术报告即将发布。在此之前,请引用 LLaVA-OneVision-2:
|
| 187 |
-
|
| 188 |
-
```bibtex
|
| 189 |
-
@misc{an2026llavaonevision2nextgenerationperceptualintelligence,
|
| 190 |
-
title = {LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence},
|
| 191 |
-
author = {Xiang An and Yin Xie and Feilong Tang and Yunyao Yan and Huajie Tan and Didi Zhu and Changrui Chen and Xiuwei Zhao and Bin Qin and Kaicheng Yang and Yifei Shen and Yuanhan Zhang and Kaichen Zhang and Wenkang Zhang and Zheng Cheng and Nansen Zhang and Chunsheng Wu and Chunjiang Ge and Zimin Ran and Dehua Song and Chunyuan Li and Shikun Feng and Ming Hu and Zhangquan Chen and Junbo Niu and Bo Li and Ziyong Feng and Ziwei Liu and Zongyuan Ge and Jiankang Deng},
|
| 192 |
-
year = {2026},
|
| 193 |
-
eprint = {2605.25979},
|
| 194 |
-
archivePrefix = {arXiv},
|
| 195 |
-
primaryClass = {cs.CV},
|
| 196 |
-
url = {https://arxiv.org/abs/2605.25979}
|
| 197 |
-
}
|
| 198 |
-
```
|
| 199 |
-
|
| 200 |
## 许可
|
| 201 |
|
| 202 |
Apache-2.0。
|
|
|
|
| 181 |
- 使用贪心解码(`do_sample=False`),并给足 `max_new_tokens`。
|
| 182 |
- 选择题或 grounding 任务,在 prompt 里写明期望的回答格式。
|
| 183 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 184 |
## 许可
|
| 185 |
|
| 186 |
Apache-2.0。
|