Papers
arxiv:2610.14609

EviScene: Beyond "Is It AI?" for Agentic Image Factuality Assessment

Published on Oct 10
ยท Submitted by
Bo
on Oct 14
Authors:
,
,
,
,
,
,
,
,
,
,
,

Abstract

Determining whether an image is AI-generated does not establish whether the situation it depicts is factually supported. We formulate Image Factuality Assessment: given an image and a generic verification instruction, a system returns a supported or refuted verdict with an evidence-grounded explanation. Without a supplied textual claim, the system must progressively refine factual questions during investigation and align external evidence with the specific situation depicted. We introduce EviLens, an expert-reviewed benchmark of 10,174 supported and refuted images from controlled construction and web collection, covering twelve factual error types and evaluating both verdict accuracy and evidence sufficiency. We develop EviScene, a multimodal ReAct agent that interleaves visual inspection, external retrieval, and evidence analysis, using new findings to refine its questions and revisit relevant image details. We train a compact policy with supervised fine-tuning on reviewed teacher trajectories, followed by privileged on-policy self-distillation applied to verified decisions from the policy's own investigations. On the 1,527-image test set, the trained Qwen3.5-9B agent achieves 79.05% balanced accuracy and a Strict Evidence Sufficiency Rate (SESR) of 52.26%, improving over the same-backbone base agent by 6.71 and 9.37 percentage points, respectively. Its SESR is the highest among the evaluated agents.Code, models, and data will be released at https://github.com/wangzian828/EVISCENE and https://huggingface.co/ZiAnwang/EviScene.

Community

Hi Community!

๐Ÿ” We introduce EviScene: an approach where the model is given only an image, without pre-provided textual assertions to verify. The agent autonomously identifies issues and conducts investigations through visual inspection, web searches, image retrieval, and evidence comparison. This capability is cultivated via Supervised Fine-Tuning (SFT) and Privileged on-policy Self-Distillation (PSD).

๐Ÿ“š The accompanying benchmark, EviLens, comprises 10,174 images and covers 12 categories of factual errors. Evaluation assesses both the correctness of the judgment and whether the evidence substantiates the conclusion; success requires not only the right answer but also a solid evidentiary basis.

๐Ÿ“ˆ Tested on 1,527 images, the trained Qwen3.5-9B agent achieved 79.05% BAcc and 52.26% SESRโ€”improvements of 6.71 and 9.37 points, respectively, over the baseline agent. It attained the highest SESR among all agents evaluated in the study. Code, model weights, and datasets have all been made publicly available.

๐Ÿ“„ Paper: https://arxiv.org/abs/2610.14609
๐Ÿ’ป Code: https://github.com/wangzian828/EVISCENE
๐Ÿค— Models & Data: https://huggingface.co/ZiAnwang/EviScene
๐ŸŒ Website: https://wangzian828.github.io/EVISCENE/

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.14609
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.14609 in a dataset README.md to link it from this page.

Spaces citing this paper 2

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.