TreasureX Researchq-fin.GNFinRank: A New Test for Financial AI That Checks the Evidence, Not Just the Answer
Sasan Mansouri, Daniel Saad, Mark Wahrenburg, Manu Weissel, Fabian Woebbeking · August 7, 2026
Most financial AI systems are judged on whether they give the right number. But in SEC filings, a correct-looking answer can be based on the wrong data—say, from a different quarter or a different company. FinRank is a new benchmark that forces AI to show its work. It includes 1,185 manually written questions over 10-K and 10-Q filings from 22 companies, each with the correct answer, the exact supporting passages, and tricky 'hard negatives'—passages that look similar but are wrong. The test measures how well systems retrieve, rank, and distinguish these passages. Results show it's tough: even a strong 7B-parameter model only gets 44.8% Recall@10, and many models barely beat a simple keyword search. This matters for investors because it highlights the risk of relying on AI for financial analysis—if the model points to the wrong evidence, the answer is unreliable. The practical implication: as AI tools become more common in investment research, they need to be evidence-grounded, not just accurate. FinRank provides a way to develop and test such systems, potentially leading to more trustworthy AI for financial decision-making.
Source: arXiv (https://arxiv.org/abs/2608.07400)
Educational content only, generated with AI assistance from public research abstracts. Not investment advice. Thank you to arXiv for use of its open access interoperability.