Image-Text-to-Text
Transformers
Safetensors
qwen3_vl
vision-language
image-classification
calibrated-probabilities
structured-outputs
typed-questions
jev-inspired
qwen3-vl
conversational
Instructions to use MeerDevelopment/Qevi-2B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use MeerDevelopment/Qevi-2B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="MeerDevelopment/Qevi-2B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("MeerDevelopment/Qevi-2B") model = AutoModelForMultimodalLM.from_pretrained("MeerDevelopment/Qevi-2B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use MeerDevelopment/Qevi-2B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "MeerDevelopment/Qevi-2B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MeerDevelopment/Qevi-2B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/MeerDevelopment/Qevi-2B
- SGLang
How to use MeerDevelopment/Qevi-2B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "MeerDevelopment/Qevi-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MeerDevelopment/Qevi-2B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "MeerDevelopment/Qevi-2B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "MeerDevelopment/Qevi-2B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use MeerDevelopment/Qevi-2B with Docker Model Runner:
docker model run hf.co/MeerDevelopment/Qevi-2B
Document the bundled qevi engine; drop the pip install claim
Browse files
README.md
CHANGED
|
@@ -37,7 +37,8 @@ restricted to the allowed answer tokens. If you call `.generate()` and parse the
|
|
| 37 |
reproduce them, because that is not the path the model was fine-tuned on.
|
| 38 |
|
| 39 |
The readout is ~30 lines of ordinary transformers code and needs no `trust_remote_code`. A
|
| 40 |
-
self-contained version is below; the [`qevi`](#the-qevi-
|
|
|
|
| 41 |
|
| 42 |
---
|
| 43 |
|
|
@@ -77,7 +78,7 @@ probs = torch.softmax(logits[cand].float() / 2.45, dim=-1) # 2.45 = calibratio
|
|
| 77 |
print(dict(zip(answers, probs.tolist()))) # {'Yes': 0.97, 'No': 0.03}
|
| 78 |
```
|
| 79 |
|
| 80 |
-
###
|
| 81 |
|
| 82 |
```python
|
| 83 |
from qevi import build_readouts, forward_logits, softmax, noul_question, choice_question_generic
|
|
@@ -267,14 +268,64 @@ documented above (1 epoch, lr 2e-5, ~5 GPU-hours on two consumer cards).
|
|
| 267 |
has not been validated for moderation use and should not be deployed for it without its own
|
| 268 |
evaluation.
|
| 269 |
|
| 270 |
-
## The `qevi`
|
| 271 |
|
| 272 |
-
Packing, prompt construction and the readout live in
|
| 273 |
-
|
|
|
|
| 274 |
|
| 275 |
-
```bash
|
| 276 |
-
pip install qevi
|
| 277 |
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 278 |
|
| 279 |
## Citation
|
| 280 |
|
|
|
|
| 37 |
reproduce them, because that is not the path the model was fine-tuned on.
|
| 38 |
|
| 39 |
The readout is ~30 lines of ordinary transformers code and needs no `trust_remote_code`. A
|
| 40 |
+
self-contained version is below; the [`qevi` engine bundled in this repo](#the-qevi-engine-bundled-in-this-repo)
|
| 41 |
+
adds question packing.
|
| 42 |
|
| 43 |
---
|
| 44 |
|
|
|
|
| 78 |
print(dict(zip(answers, probs.tolist()))) # {'Yes': 0.97, 'No': 0.03}
|
| 79 |
```
|
| 80 |
|
| 81 |
+
### Many questions, one forward pass (using the bundled engine)
|
| 82 |
|
| 83 |
```python
|
| 84 |
from qevi import build_readouts, forward_logits, softmax, noul_question, choice_question_generic
|
|
|
|
| 268 |
has not been validated for moderation use and should not be deployed for it without its own
|
| 269 |
evaluation.
|
| 270 |
|
| 271 |
+
## The `qevi` engine (bundled in this repo)
|
| 272 |
|
| 273 |
+
Packing, prompt construction and the readout live in `qevi/` inside **this repository** — three
|
| 274 |
+
files, ~350 lines, torch only, Apache-2.0. Nothing to pip install, and no `trust_remote_code`:
|
| 275 |
+
it is ordinary Python you can read before you run it.
|
| 276 |
|
|
|
|
|
|
|
| 277 |
```
|
| 278 |
+
qevi/__init__.py public API
|
| 279 |
+
qevi/core.py prompt templates, build_readouts(), forward_logits()
|
| 280 |
+
qevi/pack.py block-diagonal mask + M-RoPE position construction
|
| 281 |
+
```
|
| 282 |
+
|
| 283 |
+
### Using it
|
| 284 |
+
|
| 285 |
+
`snapshot_download` fetches the whole repo (weights and engine together), then put it on the path:
|
| 286 |
+
|
| 287 |
+
```python
|
| 288 |
+
import sys, torch
|
| 289 |
+
from PIL import Image
|
| 290 |
+
from huggingface_hub import snapshot_download
|
| 291 |
+
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
|
| 292 |
+
|
| 293 |
+
REPO = "MeerDevelopment/Qevi-2B"
|
| 294 |
+
local = snapshot_download(REPO) # weights + qevi/ in one download
|
| 295 |
+
sys.path.insert(0, local) # makes `import qevi` work
|
| 296 |
+
|
| 297 |
+
from qevi import (build_readouts, forward_logits, softmax,
|
| 298 |
+
noul_question, choice_question_generic)
|
| 299 |
+
|
| 300 |
+
model = Qwen3VLForConditionalGeneration.from_pretrained(local, dtype=torch.bfloat16).cuda().eval()
|
| 301 |
+
processor = AutoProcessor.from_pretrained(local)
|
| 302 |
+
tok = processor.tokenizer
|
| 303 |
+
image = Image.open("photo.jpg").convert("RGB")
|
| 304 |
+
|
| 305 |
+
specs = [
|
| 306 |
+
(noul_question("There is a ladder in this image."), ["Yes", "No"]),
|
| 307 |
+
(noul_question("There is a bicycle in this image."), ["Yes", "No"]),
|
| 308 |
+
(choice_question_generic("Which is shown?",
|
| 309 |
+
["ladder", "bicycle", "car", "person"]), ["A", "B", "C", "D"]),
|
| 310 |
+
]
|
| 311 |
+
|
| 312 |
+
readouts = build_readouts(tok, specs)
|
| 313 |
+
logit_sets = forward_logits(model, processor, image, readouts) # ONE forward pass
|
| 314 |
+
|
| 315 |
+
for (question, answers), lg in zip(specs, logit_sets):
|
| 316 |
+
probs = softmax(lg, T=2.45) # see temperatures above
|
| 317 |
+
print(dict(zip(answers, [round(p, 4) for p in probs])))
|
| 318 |
+
```
|
| 319 |
+
|
| 320 |
+
All three questions share a single encoding of the image and are isolated from one another by the
|
| 321 |
+
block-diagonal mask, so the answers are identical to asking them separately — verified bitwise,
|
| 322 |
+
see [Order invariance](#order-invariance). Asking 30 questions instead of 3 costs about 5 ms more
|
| 323 |
+
per question rather than another full forward pass each.
|
| 324 |
+
|
| 325 |
+
### If you would rather vendor it
|
| 326 |
+
|
| 327 |
+
The three files have no dependency on this repo's layout. Copy `qevi/` into your own project and
|
| 328 |
+
`import qevi` directly; only `torch`, `transformers` and `pillow` are required.
|
| 329 |
|
| 330 |
## Citation
|
| 331 |
|