Document AI Benchmarks
Open benchmarks for document parsing and extraction: ParseBench, ExtractBench, and ExtractBench Pro. Datasets, papers, leaderboards, and code.
Benchmark • Updated • 169k • 28k • 145Note ParseBench evaluates document parsing across tables, charts, content faithfulness, semantic formatting, and visual grounding. Live leaderboard: https://www.parsebench.ai/ Evaluation code and quick start: https://github.com/run-llama/ParseBench
llamaindex/ExtractBench
Benchmark • Updated • 370 • 17.1k • 63Note ExtractBench evaluates schema-guided extraction from enterprise documents, including value accuracy and source grounding. Live leaderboard: https://www.extractbench.ai/ Evaluation code and quick start: https://github.com/run-llama/ExtractBench
llamaindex/ExtractBenchPro
Viewer • Updated • 100 • 931 • 3Note ExtractBench Pro tests the long tail of document extraction with a separate dataset and leaderboard. Challenges include document conventions, house vocabulary, counting, drawn relations, and charts. Live leaderboard: https://www.extractbench.ai/pro Evaluation code: https://github.com/run-llama/ExtractBench#extractbench-pro
ParseBench: A Document Parsing Benchmark for AI Agents
Paper • 2604.08538 • Published • 11Note ParseBench: A Document Parsing Benchmark for AI Agents. Read the methodology behind the parsing benchmark; use the dataset and evaluation code above to reproduce evaluations. Live results: https://www.parsebench.ai/
ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction
Paper • 2607.29677 • Published • 26Note ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction. Read the methodology for evaluating extracted values and source grounding. ExtractBench Pro is available as a separate dataset above. Live results: https://www.extractbench.ai/