TreasureX Researchcs.AINo One-Size-Fits-All Recipe for Automated Scientific Discovery
Akshat Gupta, Jermaine Lei, Alexander Lu, Gopala Anumanchipalli, Leshem Choshen · July 20, 2026
Automated discovery systems like OpenEvolve and TTT-Discover are often treated as universal tools, but new research shows they are actually complex recipes with many design choices. The study decomposed these systems into components and tested 30 different harnesses across 12 model-problem pairs using over 3.1 million LLM rollouts. The key finding: no single harness consistently outperforms others across all problems. In fact, simpler alternatives often beat the popular OpenEvolve variants. This means choosing a harness should be treated as a hyperparameter, tuned to the specific problem and model, not as a fixed recipe. The researchers also discovered that early progress predicts final performance, enabling an adaptive strategy: start multiple harnesses, prune weak ones early, and reallocate compute to stronger ones. This adaptive approach outperformed both random fixed harness selection and non-adaptive ensembles. For investors and tech enthusiasts, this implies that building flexible, adaptive discovery systems could be more cost-effective than betting on a single method. It also highlights the importance of rigorous statistical testing in AI research, as many past comparisons may have been underpowered.
Read more: https://arxiv.org/abs/2607.18235
Educational content only, generated with AI assistance from public research abstracts. Not investment advice. Thank you to arXiv for use of its open access interoperability.