🧠 Visual-TableQA: Open-Domain Benchmark for Reasoning over Table Images

Welcome to Visual-TableQA, a project designed to generate high-quality synthetic question-answer datasets associated to images of tables. This resource is ideal for training and evaluating models on visually-grounded table understanding tasks such as document QA, table parsing, and multimodal reasoning.

🚀 Latest Update

We have refreshed the dataset with newly generated QA pairs created by stronger and more reliable LLMs, including:

• GPT-5
• Claude-Sonnet-4
• Gemini-2.5-Pro

✅ Quality Assessment

To ensure higher reliability, all QA pairs were evaluated by a jury of advanced models:

• Gemini-2.5-Pro
• GPT-4.1
• Mistral-Large
• DeepCogito-v2
• DeepSeek-v3.1

This multi-model jury setup improves consistency and reduces bias compared to earlier versions.

---

🔄 What’s new compared to the previous release:

• More diverse and accurate QA pairs, generated by frontier-level LLMs.
• Stronger evaluation pipeline, with cross-validation from multiple independent models.
• Higher alignment between questions and table content, with reduced hallucination rate.

We recommend using this updated release for benchmarking visual reasoning and fine-tuning multimodal models, as it provides higher quality and more trustworthy supervision signals.

🚀 What’s Inside

• 📊 Dataset entries containing rendered LaTeX table images paired with question–answer annotations.
• 🖼️ Each entry includes:
  • table_id: unique table identifier
  • image: rendered PNG of the LaTeX table
  • question: natural language query about the table
  • answer: ground-truth answer string
• 🔍 Questions are automatically generated and verified with reasoning LLMs to ensure alignment with the table content.

• ⚡ The dataset is ready-to-use for:
  - Fine-tuning LLMs or vision–language models on table reasoning tasks
  - Benchmarking pipelines for table question answering
  - Evaluating model robustness on structured visual data

📘 Paper (Out! ✅)

📝 A detailed dataset paper describing the methodology, QA strategy, and dataset statistics are presented in the paper.
📄 Read the Paper

🐙 Github Repository

All the codes and analysis of the dataset are provided in our Github Repository

🚀🚀 Getting Started

To quickly load and explore the dataset in Python, you can use the 🤗 datasets library:

from datasets import load_dataset

# Load the test split of the Im2TeX dataset
dataset = load_dataset('AI-4-Everyone/Visual-TableQA')

# Print dataset info
print(dataset)

🔄 Pipeline

The synthetic data generation pipeline in Visual-TableQA follows a multi-stage process combining vision-language models (VLMs), large language models (LLMs), and structured logic:

1. Offline Preparation: Using table images and textual prompts, a VLM and LLM generate LaTeX-style table templates and thematic concepts.
2. Table Synthesis: These instructions are passed to another LLM to generate diverse structured tables in JSON format, which are then rendered into images.
3. QA Generation: An additional LLM generates multiple QA pairs for each table, guided by reasoning-based prompts.
4. Validation: A group of LLMs acts as a jury to validate the correctness and relevance of each QA pair.
5. Export: Validated image–question–answer triplets are saved in a standardized format, ready for training and evaluation.

The complete pipeline is illustrated below:

Here are some generated tables:

📚 Citation

If you use this code or dataset in your research, please cite:

Plain-text citation:
Marc Haraoui, Boammani Aser Lompo Visual-TableQA. GitHub repository: https://github.com/AI-4-Everyone/Visual-TableQA-v2

BibTeX:

@inproceedings{
lompo2025visualtableqa,
title={Visual-Table{QA}: Open-Domain Benchmark for Reasoning over Table Images},
author={Boammani Aser Lompo and Marc Haraoui},
booktitle={NeurIPS 2025 Workshop on Foundations of Reasoning in Language Models},
year={2025},
url={https://openreview.net/forum?id=fvJRsGwhPf}
}