TreasureX Researchnber.wpAI Workflow Audits Thousands of Economics Papers
Matthew Schwartz, Isaiah Andrews, Jesse M. Shapiro · September 28, 2026
A new open-source tool from economists Matthew Schwartz, Isaiah Andrews, and Jesse M. Shapiro lets a large language model dig through the replication packages that journals publish alongside research papers. The workflow does three things: it reruns the original calculations to see if the published numbers hold up, it tries a different implementation to make those calculations faster, and it proposes extensions the authors never attempted.
The scale is striking. Testing 4,452 replication packages from five economics journals, the tool flagged discrepancies in 3,460 papers or their appendices — a reminder that published results are not always perfectly reproducible. In 496 papers it cut computation time by more than tenfold while matching or beating the original accuracy, and in 923 papers it generated a sensible new extension consistent with the original assumptions.
Why it matters: reproducibility is the backbone of credible research, and manual replication is slow and expensive. Automating it could speed up peer review, catch errors earlier, and free researchers for genuinely new work. For investors, the takeaway is indirect but real — if AI can audit and accelerate quantitative analysis this cheaply, the same techniques will spread to corporate research, data vendors, and any firm that pays analysts to check numbers by hand.
Attribution: This summary draws on research from the National Bureau of Economic Research. Source: https://www.nber.org/papers/w35782#fromrss
Educational content only, generated with AI assistance from public research abstracts. Not investment advice.