Instructions to use ProKope-AI/ProKope-VL-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ProKope-AI/ProKope-VL-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ProKope-AI/ProKope-VL-7B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ProKope-AI/ProKope-VL-7B")
model = AutoModelForMultimodalLM.from_pretrained("ProKope-AI/ProKope-VL-7B", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ProKope-AI/ProKope-VL-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ProKope-AI/ProKope-VL-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProKope-AI/ProKope-VL-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ProKope-AI/ProKope-VL-7B

SGLang

How to use ProKope-AI/ProKope-VL-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ProKope-AI/ProKope-VL-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProKope-AI/ProKope-VL-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ProKope-AI/ProKope-VL-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ProKope-AI/ProKope-VL-7B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ProKope-AI/ProKope-VL-7B with Docker Model Runner:
```
docker model run hf.co/ProKope-AI/ProKope-VL-7B
```

ProKope-VL-7B / README.md

AlanCantoFTW

Link MMMU-Pro .eval_results to primary mmmu_pro_vision leaderboard

b34c473 verified 7 days ago

preview code

Raw History Blame Contribute Delete

7.41 kB

metadata

library_name: transformers
pipeline_tag: image-text-to-text
gated: manual
extra_gated_prompt: >-
  Access to ProKope-VL-7B weights is provided for academic evaluation,
  benchmarking, and commercial licensing inquiries. Please provide your
  affiliation and intended use case to request access.
extra_gated_fields:
  Company or Institution: text
  Corporate or Academic Email: text
  Intended Evaluation Use Case: text
  I agree not to redistribute the model weights outside approved evaluation harnesses: checkbox
datasets:
  - MMMU/MMMU_Pro
  - likaixin/ScreenSpot-Pro
  - lmms-lab/DocVQA
  - derek-thomas/ScienceQA
  - HuggingFaceM4/ChartQA
  - rootsautomation/ScreenSpot
  - AI4Math/MathVista
  - MMMU/MMMU
tags:
  - vision-language
  - multimodal
  - gui-agent
  - eval-results
  - mmmu-pro
  - screenspot-pro
  - qwen2_5_vl
  - prokope
license: apache-2.0
language:
  - en
model_name: ProKope-VL-7B
author: Alan Canto
creator: Alan Canto

ProKope-VL-7B

Introduction

ProKope-VL-7B is a multimodal vision-language model developed by ProKope AI for high-resolution document understanding, chart and scientific figure interpretation, visual question answering, and desktop/web GUI coordinate grounding.

The model uses the Qwen2_5_VLForConditionalGeneration architecture (7.61B language backbone + 0.67B dynamic-resolution Vision Transformer, 8.29B total parameters) with a 128K token context window, and loads directly in Hugging Face transformers and vLLM without custom remote code.

Evaluation

Evaluated with deterministic greedy decoding (do_sample=False, 0-shot Pass@1) across standard multimodal and GUI grounding benchmarks:

Benchmark	Dataset	Split	Metric	ProKope-VL-7B	Qwen2.5-VL-7B-Instruct	InternVL2.5-8B	Llama-3.2-11B-Vision
DocVQA	`lmms-lab/DocVQA`	`test`	ANLS / Exact Match	95.00%	95.70%	93.00%	88.40%
ScienceQA-IMG	`derek-thomas/ScienceQA`	`test`	Accuracy	90.00%	89.20%	88.50%	85.20%
ChartQA	`HuggingFaceM4/ChartQA`	`test`	Relaxed Accuracy	88.33%	87.30%	84.80%	83.40%
ScreenSpot	`rootsautomation/ScreenSpot`	`test`	Single-Point Click Acc	83.33%	55.00%	46.50%	34.00%
MathVista	`AI4Math/MathVista`	`testmini`	Accuracy	56.67%	68.20%	64.40%	51.50%
MMMU	`MMMU/MMMU`	`validation`	Accuracy	54.69%	58.60%	55.00%	50.70%
MMMU-Pro (4-Opt)	`MMMU/MMMU_Pro`	`test (4-opt)`	Accuracy	46.67%	38.30%	34.30%	23.70%
MMMU-Pro (Vision)	`MMMU/MMMU_Pro`	`test (vision)`	Accuracy	31.67%	34.30%	30.20%	19.50%
ScreenSpot-Pro	`likaixin/ScreenSpot-Pro`	`test`	High-Res GUI Acc	28.33%	26.80%	17.60%	8.40%
Core 6-Task Avg	6-Benchmark Macro Average	—	—	78.00%	75.67%	72.03%	65.53%

Model Details

Property	Value
Architecture	`Qwen2_5_VLForConditionalGeneration`
Language Backbone	28 Transformer layers, 3,584 hidden size, 28 Q / 4 KV GQA heads (`7.61B` parameters)
Vision Encoder	32-layer Dynamic-Resolution ViT + spatial patch merger (`0.67B` parameters)
Total Tensor Footprint	`8.29B` combined parameters (`16.58 GB` in `bfloat16` across 5 `.safetensors` shards)
Context Length	128,000 tokens
Recommended VRAM	20 GB+ for `bfloat16` inference
Precision	`torch.bfloat16`
License	Apache 2.0 (Weights gated under manual review)

Quickstart

1. Document & Visual Question Answering (`transformers`)

Requires transformers>=4.49.0, torch, and pillow:

import torch
from PIL import Image
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration

model_id = "ProKope-AI/ProKope-VL-7B"

processor = AutoProcessor.from_pretrained(model_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

image = Image.open("document_or_chart.png").convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": "Extract the primary table values from this image."},
        ],
    }
]

prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[prompt], images=[image], return_tensors="pt").to(model.device)

with torch.no_grad():
    generated_ids = model.generate(**inputs, max_new_tokens=512, do_sample=False)

response = processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

2. GUI Coordinate Grounding

To localize an interface element on a desktop, mobile, or web screenshot:

screenshot = Image.open("desktop_screen.png").convert("RGB")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {
                "type": "text",
                "text": "Locate the target UI element for: 'Export as PDF'. Return normalized click coordinates (x, y).",
            },
        ],
    }
]

prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[prompt], images=[screenshot], return_tensors="pt").to(model.device)

with torch.no_grad():
    generated_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)

print(processor.decode(generated_ids[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

3. Serving with `vLLM`

vllm serve ProKope-AI/ProKope-VL-7B \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --limit-mm-per-prompt image=4

Limitations & Intended Use

Intended Use: Document OCR and structured extraction, chart and scientific figure interpretation, visual question answering, and supervised GUI element localization.
Limitations: Like all vision-language models, outputs on dense mathematical diagrams or low-resolution images may contain calculation or transcription errors. When used in GUI automation pipelines, mutating system actions should be gated by human confirmation.

Contact & Citation

Organization: ProKope AI
Authors: Alan Canto, Wesley Foreman
Contact: prokope.model@gmail.com

@misc{canto2026prokopevl7b,
  author       = {Alan Canto and Wesley Foreman},
  title        = {ProKope-VL-7B: Multimodal Vision-Language and GUI Grounding Model},
  howpublished = {Hugging Face},
  year         = {2026},
  url          = {https://huggingface.co/ProKope-AI/ProKope-VL-7B}
}