README.md

---
library_name: transformers
pipeline_tag: image-text-to-text
base_model: Qwen/Qwen3-VL-8B-Instruct
tags:
  - qwen3-vl
  - vision-language
  - reward-model
  - image-ranking
---

# VerifierReward Qwen3-VL 8B

```python
import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "JanHutter/verifierreward"

processor = AutoProcessor.from_pretrained(
    model_id, max_pixels=512 * 512, fix_mistral_regex=False,
)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
    attn_implementation="sdpa",
).eval()
```


## Score image-prompt alignment

The model was trained to answer `yes` or `no`. The scalar reward used during
evaluation is the full-vocabulary probability of the next token being `yes`.

```python
import torch
from PIL import Image

image = Image.open("image.jpg").convert("RGB")

instruction = """You are an AI assistant specializing in image analysis and ranking. Your task is to analyze and compare image based on how well they match the given prompt.
The given prompt is:{prompt}. Please consider the prompt and the image to make a decision and response directly with 'yes' or 'no'."""

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": instruction.format(prompt="a red car in snow")},
    ],
}]

inputs = processor.apply_chat_template(
    [messages],
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    logits = model(**inputs, use_cache=False).logits[:, -1, :].float()

yes_ids = processor.tokenizer.encode("yes", add_special_tokens=False)
assert len(yes_ids) == 1
yes_id = yes_ids[0]
reward = torch.softmax(logits, dim=-1)[:, yes_id]
print(reward.item())
```