TreasureX Researchcs.AICan AI Agents Do Open-Ended AI Research? Early Evidence Says No
Peter Kirgis, Sayash Kapoor, Andrew Schwartz, Stephan Rabanser, David Africa, Konstantinos Voudouris, Viet Nguyen, Toby Pilditch, Magda Dubois, Harry Coppock, Cozmin Ududec, Nitya Nadgir, Matilda Orona, Tilman Bayer, Derrick Chan-Sew, Yue Ling, Abhishek Shetty, Helen Toner, Gillian Hadfield, Seth Lazar, Steve Newman, Shoshannah Tekofsky, Rishi Bommasani, Arvind Narayanan · July 29, 2026
A new study tests whether current AI agents can conduct open-ended AI research by having them tackle the central research questions of two unpublished NeurIPS 2026 papers. The agents were given six days and thousands of dollars in compute, and they completed all engineering tasks without human help. However, they failed to make substantial progress toward answering the research questions, leading both papers to be rejected by the original authors. The researchers identified five recurring failure modes: poor judgment about publishable research standards, uncreative responses to shortcomings, ineffective backtracking, poor resource awareness, and instruction drift. These results suggest that while today's agents can handle the engineering aspects of AI research, they struggle with critical parts of the research lifecycle, such as creativity and problem-solving. The study provides early evidence that AI agents are not yet capable of automating open-ended AI research, which has implications for forecasts of explosive AI progress.
Read more at arXiv: https://arxiv.org/abs/2607.27191
Educational content only, generated with AI assistance from public research abstracts. Not investment advice. Thank you to arXiv for use of its open access interoperability.