Instructions to use visionq-review/VisionQ-Judge with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use visionq-review/VisionQ-Judge with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
VisionQ-Judge
LoRA adapters for VisionQ-Judge, from the anonymous CVPR 2027 submission VisionQ. VisionQ-Judge is Gemma-4-E4B fine-tuned with DPO to pick the method that a paper's authors claim wins in a qualitative comparison figure. The training pairs are symmetric evidence pairs in every cyclic option order: 13,394 pairs, one epoch.
| Folder | Seed | Test accuracy (538 VisionQ-MCQ questions) |
|---|---|---|
seed42/ |
42 | 61.3% |
seed43/ |
43 | 60.2% |
seed44/ |
44 | 61.3% |
Tab. 5 of the paper reports the mean of the three seeds: 61.0% against 50.4% for the base model (+10.6, 95% paper-bootstrap interval +5.0 to +16.0). The per-question predictions behind these numbers are in the code release (results/judge/test/Cfix_s4*). fp16 outputs vary with GPU type, so compare runs on the same GPU type.
Use
from transformers import AutoProcessor, AutoModelForImageTextToText
from peft import PeftModel
base = AutoModelForImageTextToText.from_pretrained("google/gemma-4-E4B-it", torch_dtype="float16", device_map="auto")
model = PeftModel.from_pretrained(base, "visionq-review/VisionQ-Judge", subfolder="seed42")
processor = AutoProcessor.from_pretrained("visionq-review/VisionQ-Judge", subfolder="seed42")
To reproduce the reported evaluation, including the prompt, the image handling and the caption context, use judge/evaluate.py from the code release with --adapter <path>/seed42 --prepend-context --anti-bias-instruction --fp16 --max-image-size 512.
Training
judge/train.sh Cfix_s42 42 all in the code release. DPO with beta 0.3 and learning rate 2e-5, LoRA r=16 and alpha 32 on the language and vision layers, on the VisionQ-MCQ train split (2,687 questions, paper-disjoint from val and test).
The adapters are a derivative of Gemma and follow the Gemma terms of use.
- Downloads last month
- -