APEX-Accounting: A New Benchmark Tests AI Accountants

Julien Benchek, Austin Bennett, Jasmin Kern, Ryan Stevens, Rene Sultan, Charis Ching, Hayley Popiel, Vaibhav Mittal, Felix Mercier, Brendan Foody, Bertie Vidgen · July 29, 2026

A new benchmark called APEX-Accounting, developed by Mercor in partnership with Ramp, evaluates how well frontier AI models can perform real accounting tasks. The benchmark includes 160 tasks across 10 simulated companies, covering activities like reconciling accounts, accruing expenses, posting transactions, and generating reports. Each task was created and solved by professional accountants, who also provided grading rubrics.

In tests of nine leading models, Claude-Fable-5 (Max) achieved the highest score at 56.4% Mean Criteria@3, followed by Muse-Spark-1.1 (xHigh) at 52.6%. However, no model passed more than 2.6% of tasks at the strictest Pass^8 metric, and the best Pass@8 score was only 21.5%. The researchers also found a surprising Simpson's paradox: while overall scores improved with larger token budgets, within a fixed budget, tasks where models spent more tokens actually had lower scores.

This benchmark matters because it shows that even the most advanced AI still struggles with complex, real-world accounting work. For businesses and investors, it suggests that AI is not yet ready to replace human accountants, but could serve as an assistant for simpler tasks. The closed benchmark allows any frontier model to be evaluated upon request, enabling ongoing comparison.