Hawkeye: Reading Illegible Text with Vision Language Models
Abstract
Researchers introduce a benchmark and fine-tuned vision-language model to improve recognition of blurry, low-resolution scene text in noisy environments.
While modern Vision-Language Models (VLMs) have been remarkably successful in general scene text recognition, they consistently struggle with the long-tail distribution of text that is degraded by motion blur or low resolution. These limitations are particularly apparent in egocentric applications like smart glasses, where text is often captured in noisy environments. In this paper, we introduce the BSTRB (Blurry Scene Text Recognition Benchmark), a novel benchmark of 8,398 noisy image samples with high-fidelity annotations derived from 1,699 real-world video sequences. To support this, we present Hawkeye, a VLM fine-tuned on BSTRB to read and reason about illegible text by leveraging contextual priors and pixel-level alignment. Our results show that \name matches much larger VLMs on BSTRB overall and exceeds them on text at or below 20 pixels, in a single pass suited to real-time deployment. We characterize the resolution at which current VLMs begin to fail.
Get this paper in your agent:
hf papers read 2610.41254 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 2
byron123t/Hawkeye-8B
Datasets citing this paper 1
byron123t/BSTRB
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper