TreasureX Researchcs.AINew Benchmark Tests AI Agents for Enterprise Document Extraction
Boyang Zhang, Adrian Lyjak, Eli Stewart, Zhaoqi Li, Simon Suo · July 31, 2026
Enterprises are increasingly using AI agents to pull structured data from documents like invoices, contracts, and forms. A new benchmark called ExtractBench aims to measure how well these agents follow a user-defined schema and provide source evidence. It includes 4,869 pages from 370 real-world documents across 8 business domains and 67 types, with tags for different challenges. The benchmark scores value accuracy, record completeness, grounding (traceability), and cost. Key findings: commercial vision-language models (VLMs) do well on short documents but often truncate long lists; coding agents are more accurate but far more expensive. The LlamaExtract Agentic Plus model ranked first on all metrics, achieving accuracy close to coding agents at a fraction of the cost. This matters for investors because it highlights the growing importance of efficient AI for back-office automation, potentially impacting companies that provide document processing solutions or use them to cut costs. The dataset and code are open-source on HuggingFace and GitHub.
Source: arXiv (https://arxiv.org/abs/2607.29677)
Educational content only, generated with AI assistance from public research abstracts. Not investment advice. Thank you to arXiv for use of its open access interoperability.