Instructions to use morpheushoc/InternVL2_5-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use morpheushoc/InternVL2_5-2B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="morpheushoc/InternVL2_5-2B", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("morpheushoc/InternVL2_5-2B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use morpheushoc/InternVL2_5-2B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "morpheushoc/InternVL2_5-2B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "morpheushoc/InternVL2_5-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/morpheushoc/InternVL2_5-2B

SGLang

How to use morpheushoc/InternVL2_5-2B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "morpheushoc/InternVL2_5-2B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "morpheushoc/InternVL2_5-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "morpheushoc/InternVL2_5-2B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "morpheushoc/InternVL2_5-2B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use morpheushoc/InternVL2_5-2B with Docker Model Runner:
```
docker model run hf.co/morpheushoc/InternVL2_5-2B
```

InternVL2_5-2B / examples /chat_batch.py

ardalan.mehrani

point to remote folder

b4837db over 1 year ago

Raw

History Blame Contribute Delete

2.01 kB


	import torch
	from transformers import AutoModel, AutoTokenizer
	from utils import load_image, load_video

	if __name__ == "__main__":

	dir, rev = 'morpheushoc/InternVL2_5-2B', 'main'

	# path = 'OpenGVLab/InternVL2_5-2B'
	model = AutoModel.from_pretrained(dir,
	torch_dtype=torch.bfloat16,
	load_in_8bit=False,
	low_cpu_mem_usage=True,
	use_flash_attn=True,
	trust_remote_code=True,
	revision=rev).eval().cuda()
	tokenizer = AutoTokenizer.from_pretrained(dir, trust_remote_code=True, use_fast=False)
	generation_config = dict(max_new_tokens=1024, do_sample=False)

	paths = [
	'image1.jpg',
	'image1.jpg',
	'image2.jpg',
	'red-panda.mp4',
	]

	questions = [
	'describe this image',
	'describe this image',
	'describe this image',
	'describe this video'
	]

	pixel_values, num_patches_list = [], []
	for i, fp in enumerate(paths):

	if fp.endswith('mp4'):
	pxl_val, num_patches = load_video(fp, num_segments=8, max_num=1)
	prefix = ''.join([f'Frame{i+1}: <image>\n' for i in range(len(num_patches))])
	else:
	pxl_val = load_image(fp, max_num=12).to(torch.bfloat16)
	num_patches = [len(pxl_val)]
	prefix = '<image>\n'

	pixel_values.append(pxl_val)
	num_patches_list.append(num_patches)
	questions[i] = prefix + questions[i]

	pixel_values = torch.cat(pixel_values).to(torch.bfloat16).cuda()
	response = model.batch_chat(tokenizer, pixel_values, questions, generation_config,
	num_patches_list=num_patches_list, history=None, return_history=False)

	for q, r in zip(questions, response):
	print(f'User: {q}\nAssistant: {r}')
	print('\n')