HalluScope-4B

HalluScope-4B is a lightweight diagnostic model for fine-grained hallucination diagnosis in multimodal large language models (MLLMs). Given an image and a model-generated response, it detects hallucinated spans, classifies each into one of 12 fine-grained types, and returns span-level annotations in a single pass.

• Base model: Qwen3-VL-4B-Instruct
• Training data: HalluScope-30K
• Task: span-level hallucination detection + classification

Output Format

The model wraps hallucinated spans with typed <hallucination> tags:

<Tagged_Text>
The <hallucination type="Color_Attribute">bright red</hallucination>
<hallucination type="Object">sports</hallucination> car is
<hallucination type="Spatial_Attribute">parked near a lake</hallucination>.
</Tagged_Text>

Hallucination Taxonomy

12 fine-grained types across two categories:

Category	Types
Perception	Object, OCR, Numerical_Attribute, Color_Attribute, Shape_Attribute, Spatial_Attribute
Reasoning	Logical_Error, Calculation_Error, Knowledge_Error, Query_Misunderstanding, Numerical_Relation, Spatial_Relation

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor
from PIL import Image

model = AutoModelForImageTextToText.from_pretrained(
    "wkinglin/HalluScope-4B", torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("wkinglin/HalluScope-4B")

messages = [{
    "role": "user",
    "content": [
        {"type": "image", "image": Image.open("example.jpg")},
        {"type": "text", "text": "Analyze the response and tag hallucinated spans:\n<response to diagnose>"},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to(model.device)
out = model.generate(**inputs, max_new_tokens=2048)
print(processor.batch_decode(out, skip_special_tokens=True)[0])

For high-throughput inference, serve the model with vLLM and query it through the OpenAI-compatible API.

Related

• HalluScope-8B — the larger variant: wkinglin/HalluScope-8B
• Dataset — wkinglin/HalluScope-30K

Citation

@inproceedings{jin2026halluscope,
  title     = {HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models},
  author    = {Jin, Weilin and Wang, Mingyu and Li, Wenbo and Huang, Haoyang and Wu, Yifan and Li, Ying and Huang, Gang and Wu, Zhonghai},
  booktitle = {Proceedings of the 34th ACM International Conference on Multimedia (MM '26)},
  year      = {2026}
}