Papers
arxiv:2611.19200

LogT: Logically Think with Images for Visual Search

Published on Nov 14
· Submitted by
Yanjun Fu
on Nov 19
Authors:
,
,
,
,
,
,
,

Abstract

Thinking with images empowers vision-language models (VLMs) to use an external Zoom-in tool for fine-grained visual search. Recent works employ supervised fine-tuning (SFT) and reinforcement learning (RL) with tool-specific designs to enable such capabilities. However, achieving logical reasoning alongside accurate and efficient tool use remains a significant challenge. To address this, we shift the focus away from increasing RL complexity and instead identify SFT data quality as a key bottleneck. We then propose LogT (short for Logically Think with Images), a fully automated SFT data engine that explicitly enforces logic-chain-guided reasoning. LogT generates an SFT dataset featuring challenging, visually grounded questions and structured tool-use traces with human-like logic. We validate our approach by fine-tuning two base models, Qwen2.5-VL-7B and InternVL3.5-8B, on the dataset and evaluating them across seven benchmark test sets. We show that LogT-SFT based on Qwen2.5-VL-7B outperforms the state-of-the-art trained with both SFT and RL, despite using 3.1times less data and >120times fewer GPU hours. Furthermore, applying a simple RL recipe without tool-specific design on LogT-SFT yields a model that surpasses prior methods with complex RL setups by a substantial 9.59-point margin. In addition to strong performance gains, our model exhibits higher tool-use accuracy and efficiency, as well as more structured and logically consistent reasoning traces, directly validating the effectiveness of our approach.

Community

When answering questions about images, vision-language models (VLMs) can benefit from zooming in on small but important details, much like humans do. However, teaching VLMs when and how to zoom in typically requires complex post-training methods. In this work, we take a different approach: instead of increasing training complexity, we focus on improving the quality of supervised fine-tuning (SFT) data. We introduce LogT, an agentic data engine that generates high-quality SFT examples. Remarkably, models trained solely on our SFT data outperform those trained with both SFT and reinforcement learning (RL). With standard, off-the-shelf RL, our model surpasses the state-of-the-art open-source model by an average of 9.59 points across seven test sets. These results demonstrate that high-quality SFT data can be more effective than sophisticated training methods, offering a much more efficient yet powerful approach to improving visual reasoning in VLMs.

This work was accepted at NeurIPS 2026.

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2611.19200 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2611.19200 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2611.19200 in a Space README.md to link it from this page.

Collections including this paper 1