Instructions to use lmms-lab/LLaVA-OneVision-1.5-8B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lmms-lab/LLaVA-OneVision-1.5-8B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="lmms-lab/LLaVA-OneVision-1.5-8B-Instruct", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("lmms-lab/LLaVA-OneVision-1.5-8B-Instruct", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lmms-lab/LLaVA-OneVision-1.5-8B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lmms-lab/LLaVA-OneVision-1.5-8B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmms-lab/LLaVA-OneVision-1.5-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/lmms-lab/LLaVA-OneVision-1.5-8B-Instruct

SGLang

How to use lmms-lab/LLaVA-OneVision-1.5-8B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lmms-lab/LLaVA-OneVision-1.5-8B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmms-lab/LLaVA-OneVision-1.5-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lmms-lab/LLaVA-OneVision-1.5-8B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lmms-lab/LLaVA-OneVision-1.5-8B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use lmms-lab/LLaVA-OneVision-1.5-8B-Instruct with Docker Model Runner:
```
docker model run hf.co/lmms-lab/LLaVA-OneVision-1.5-8B-Instruct
```

LLaVA-OneVision-1.5-8B-Instruct

Commit History

Jul 16

12:54
Update README.md

yiyexy

Verified signature
e137910

Oct 21, 2025

05:53
Update README.md

xiangan

Verified signature
bdf9518
05:51
Update README.md

xiangan

Verified signature
9072dbf
05:46
Update README.md

xiangan

Verified signature
ae3e394
05:37
Update README.md

xiangan

Verified signature
80719a5
05:35
Update README.md

xiangan

Verified signature
75a1758
05:34
Update README.md

xiangan

Verified signature
59bcd19
05:28
Update README.md

xiangan

Verified signature
35f4ece

Oct 14, 2025

05:40
Delete .gitattributes

Yiye

Verified signature
2e2c9d4
05:33
Upload folder using huggingface_hub

Yiye

Verified signature
9f0f75e

Oct 5, 2025

07:12
Update README.md

xiangan

Verified signature
ae3f2e6
07:11
Update README.md

xiangan

Verified signature
746e144
07:11
Improve model card: Add paper, project page, and code links (#3)⁠

xiangan +1: nielsr

Verified signature
9206a19

Sep 30, 2025

02:05
Update README.md

Yiye

Verified signature
491e88c

Sep 28, 2025

03:39
Update README.md

xiangan

Verified signature
e6a9a0d
03:37
Update README.md

xiangan

Verified signature
391f872
03:23
Upload 2 files

xiangan

Verified signature
f30a35d

Sep 27, 2025

14:28
Update README.md

xiangan

Verified signature
813bbd9
14:27
Update README.md

xiangan

Verified signature
b2e3f0c
14:25
Update README.md

xiangan

Verified signature
ec07908

Sep 25, 2025

05:27
Upload 2 files

xiangan

Verified signature
731bf0b
05:24
Rename tensorboard/instruct to tensorboard/instruct/README.md

xiangan

Verified signature
47aa218
05:24
Create instruct

xiangan

Verified signature
2326204
05:23
Create tensorboard/mid_training/README.md

xiangan

Verified signature
778932b
05:23
Delete tensorboard/mid_training

xiangan

Verified signature
af82ecf
05:23
Create tensorboard/mid_training/

xiangan

Verified signature
0d4578c
05:22
Delete tensorboard/mid_training_85m

xiangan

Verified signature
7d4543c
05:21
Upload 2 files

xiangan

Verified signature
2128d00
05:19
Create tensorboard/mid_training_85m/README.md

xiangan

Verified signature
5850e72

Sep 23, 2025

08:48
Update the `library` tag for more visibility. (#1)⁠

Yiye +1: ariG23498

Verified signature
ad0b3fc

Sep 22, 2025

05:29
Upload modeling_llavaonevision1_5.py with huggingface_hub

Yiye

Verified signature
1e8c3bf

Sep 18, 2025

08:24
Upload preprocessor_config.json with huggingface_hub

Yiye

Verified signature
7852b29

Sep 17, 2025

01:46
Update README.md

Yiye

Verified signature
4025465

Sep 16, 2025

14:20
Update README.md

Yiye

Verified signature
8eb300d
14:10
Update README.md

Yiye

Verified signature
193cd55
11:03
Update README.md

Yiye

Verified signature
5351272
10:56
Upload tokenizer_config.json with huggingface_hub

Yiye

Verified signature
cc55440
10:55
Upload preprocessor_config.json with huggingface_hub

Yiye

Verified signature
7a4cd86
10:55
Upload modeling_llavaonevision1_5.py with huggingface_hub

Yiye

Verified signature
4f9eaf6
10:55
Upload model-00003-of-00004.safetensors with huggingface_hub

Yiye

Verified signature
8b7725a
10:53
Upload model-00001-of-00004.safetensors with huggingface_hub

Yiye

Verified signature
8f815ad
10:52
Upload inference.py with huggingface_hub

Yiye

Verified signature
47c9186
10:51
Upload inference.py with huggingface_hub

Yiye

Verified signature
e843cc6
10:51
Upload configuration_llavaonevision1_5.py with huggingface_hub

Yiye

Verified signature
16dae6f
10:50
Upload chat_template.jinja with huggingface_hub

Yiye

Verified signature
c1b093d
10:50
Upload vocab.json with huggingface_hub

Yiye

Verified signature
6070d3f
10:50
Update README.md

Yiye

Verified signature
9718063
10:50
Upload tokenizer.json with huggingface_hub

Yiye

Verified signature
26bcb6c
10:49
Upload special_tokens_map.json with huggingface_hub

Yiye

Verified signature
a7b9dad
10:49
Upload model.safetensors.index.json with huggingface_hub

Yiye

Verified signature
d856cc7

Commit History

Jul 16

Update README.md

Oct 21, 2025

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Oct 14, 2025

Delete .gitattributes

Upload folder using huggingface_hub

Oct 5, 2025

Update README.md

Update README.md

Improve model card: Add paper, project page, and code links (#3)⁠

Sep 30, 2025

Update README.md

Sep 28, 2025

Update README.md

Update README.md

Upload 2 files

Sep 27, 2025

Update README.md

Update README.md

Update README.md

Sep 25, 2025

Upload 2 files

Rename tensorboard/instruct to tensorboard/instruct/README.md

Create instruct

Create tensorboard/mid_training/README.md

Delete tensorboard/mid_training

Create tensorboard/mid_training/

Delete tensorboard/mid_training_85m

Upload 2 files

Create tensorboard/mid_training_85m/README.md

Sep 23, 2025

Update the `library` tag for more visibility. (#1)⁠

Sep 22, 2025

Upload modeling_llavaonevision1_5.py with huggingface_hub

Sep 18, 2025

Upload preprocessor_config.json with huggingface_hub

Sep 17, 2025

Update README.md

Sep 16, 2025

Update README.md

Update README.md

Update README.md

Upload tokenizer_config.json with huggingface_hub

Upload preprocessor_config.json with huggingface_hub

Upload modeling_llavaonevision1_5.py with huggingface_hub

Upload model-00003-of-00004.safetensors with huggingface_hub

Upload model-00001-of-00004.safetensors with huggingface_hub

Upload inference.py with huggingface_hub

Upload inference.py with huggingface_hub

Upload configuration_llavaonevision1_5.py with huggingface_hub

Upload chat_template.jinja with huggingface_hub

Upload vocab.json with huggingface_hub

Update README.md

Upload tokenizer.json with huggingface_hub

Upload special_tokens_map.json with huggingface_hub

Upload model.safetensors.index.json with huggingface_hub