Human-written code is facing stiff competition from AI-generated evolutionary code.Credit: MTStock Studio/Getty
Researchers at Google have unveiled a new workflow to improve scientific software using artificial intelligence1.
The company built evolutionary ‘trees’ of software tools for six tasks. The ‘nodes’ in each tree were single programs whose performance was assessed using a standard benchmark. The team created new nodes by prompting a large language model (LLM) to improve an existing one’s performance. The researchers helped the LLM by feeding it summaries of research papers, specialist knowledge and other information. In each task, some of the resulting programs outperformed state-of-the-art tools.
“It’s really cool to see big companies like Google using evolutionary approaches to make breakthroughs in other scientific fields,” says Jenny Zhang, a computer scientist at the University of British Columbia in Vancouver, Canada, who has designed programs using similar methods. “It gives me hope that the research direction that I’m doing, when scaled up, can make a big impact.”
Google researchers answered questions from Nature about the work but declined to comment on the record because the manuscript has not yet been peer reviewed. The team is working to make the system available to scientists, the authors say. Many of the optimized tools can be found online.
Software evolution
Science runs on software. Researchers need code to analyse big data, simulate complex processes and operate sophisticated instruments. Last year’s Nobel Prize in Chemistry was awarded, in part, for AlphaFold, a platform that predicts how proteins will fold up. And many of this century’s most-cited papers describe research software.
But writing scientific software is time-consuming and technical. “When I’m actually focused on science, 90% of my time is coding,” says Evan Johnson, a biostatistician and director of the Center for Data Science at Rutgers University in Piscataway, New Jersey, who was not involved in the work. Google’s new workflow aims to automate the process by allowing software to evolve, building on both existing tools and online resources.
Google refined its code-mutation system by having it complete tasks that had been published on the data-science competition platform Kaggle, testing ways to choose nodes to mutate or to prompt the LLM to alter them. Then the researchers applied the method across six scientific domains. For each one, they grew several evolutionary trees, each with up to 2,000 nodes. They created the initial node of a tree by asking the LLM to write a program from scratch, telling it to implement an existing method, combine two methods or create a new one. When mutating a program, the LLM could also search the literature for fresh ideas. It was allowed to duplicate and ‘mutate’ any node in the tree, not just the best-performing one, leading to an open-ended discovery process in which evolution could take meandering paths to success.
The first application was ‘batch integration of single-cell RNA-sequencing data’ — that is, merging different genomics data sets. The system generated 40 programs that outperformed the best human-written program available, called ComBat, developed by Johnson. The top-scoring program was 14% better. Next, they used evolution to refine programs to predict the number of COVID-19 hospitalizations across US states. Their best predictors outscored all others in the COVID-19 Forecast Hub, a repository of forecasting models.
Other tasks included labelling satellite images, predicting neural activity in zebrafish (Danio rerio) and ‘time-series forecasting’: projecting how data points from various domains progress over intervals ranging from seconds to years. In all three cases, evolved programs beat existing ones. In the final challenge, Google created variations of a common function to solve calculus problems. The best one attempted 19 problems that the original had failed — and solved 17 of them.
