TreasureX Researchq-fin.GNLLMs Can Check Research, But Can't Judge What Matters
Hai Lu, Yichen Luo, Luping Yu · October 13, 2026
Finance and accounting journals are split on whether referees may use large language models. A new study asks a sharper question: which parts of peer review can an LLM actually do well?
The authors had GPT-4o review papers presented at the 2024 AFA and NBER finance meetings, then tracked which ones were published in the top three finance journals over the following two years. Papers the model rated one standard deviation higher were 44% more likely to be published, even after accounting for author reputation. The model's training data ended before the conference, so it could not simply recall outcomes.
The striking part is where the predictive power comes from. The model's judgments of correctness and exposition predicted publication; its judgments of originality and significance predicted nothing. Telling the model to focus on importance did not help. The same split appeared for accounting papers and for a newer model, GPT-5.
The implication is practical: technical verification — checking whether a paper is right — can be delegated to AI, while judging whether it matters stays with humans. For investors, this suggests AI tools may soon speed up due diligence on quantitative claims, but human judgment on which ideas have lasting value remains essential.
Source: arXiv paper by Hai Lu, Yichen Luo, and Luping Yu. Read the full paper at https://arxiv.org/abs/2610.18435
Educational content only, generated with AI assistance from public research abstracts. Not investment advice. Thank you to arXiv for use of its open access interoperability.