AI scientists are changing research — institutions, funders and publishers must respond

This week, Nature is publishing a paper¹ with, perhaps, an unusually underwhelming finding at its heart: that a particular technique failed to improve how artificial neural networks learn. It was not these findings that our editors found noteworthy. Rather, it was how the work was done, which is in fact the main focus of the paper.

AI is saving time and money in research — but at what cost?

The result was created using The AI Scientist engineered by researchers at Tokyo-based company Sakana AI, details of which were first made available² as a preprint in 2024. This artificial-intelligence system represents an effort to automate the scientific process entirely, from performing literature review and conceiving an idea to executing experiments and writing them up. As detailed in the Nature paper, The AI Scientist was able to follow this process and generate a research paper about its (negative) result; the work passed the first round of peer review for submissions to a workshop at a major machine-learning conference.

AI research assistants have since proliferated, with technology firms Google, OpenAI and Anthropic all trialling ways to automate research. Although their outputs have been limited and rarely innovative so far, the effects of being able to generate research papers quickly and cheaply are rippling through the scientific ecosystem. Universities, funders, publishers and researchers must plan how they will adapt.

Many researchers hope that generative large language models (LLMs) will accelerate discovery by automating repetitive or challenging parts of the research process, such as coding, data analysis and literature review. The AI Scientist goes further. It aims to use AI systems’ speed, pattern-recognition skills and ability to access vast amounts of interdisciplinary knowledge to automate even processes such as generating hypotheses and interpreting results.

Researchers built an ‘AI Scientist’ — what can it do?

Nature has published details of The AI Scientist because it is essential to know how AI research assistants work, and their limitations, to assess their likely impact on science. As a result of peer review, the Nature paper expands on the preprint’s description of the system’s weaknesses, includes more ethical considerations, and tones down the authors’ original statements about automating the entire research process (humans helped to filter the most promising outputs). The AI Scientist produced three papers, one of which, following peer review, reached the bar for acceptance at a workshop of the prestigious International Conference on Learning Representations. It did not meet the bar for the main conference track.

It comes after researchers last month released a theoretical-physics preprint³ in which the state-of-the-art generative AI model GPT5, from OpenAI in San Francisco, California, played an essential part. Nathaniel Craig, a physicist at the University of California, Santa Barbara, who was not involved in the work, described the paper as ‘journal-level research’.

That AI models are capable of such impressive outputs represents a huge technological feat, many years in the making. But as with all new technologies, caution is needed. The models still have fundamental issues, and are limited mostly to theoretical or coding-based research. They remain plagued by ‘hallucinated’ data, such as made-up citations. Unlike a human scientist, they struggle to gauge their confidence in a given output and have difficulty in stringing together the many steps involved in a typical research process.

AI ‘scientists’ joined these research teams: here’s what happened

LLMs can already craft papers using entirely fake but plausible-looking data. But even models that, for example, use algorithms to iterate a data analysis until they find something significant risk generating noise that would overload conference, publishing and funding peer-review systems, without shifting the needle on discovery. This has been described as nothing more than automated, large-scale P hacking — the process of tweaking analyses or sieving data to get statistically significant results. The temptation for under-pressure researchers to seize on such mass-produced ‘one-click’ science is huge.

System overload is not the only worry. It is hard to trace a model’s inspirations, which risks exploiting other people’s ideas without giving credit. AI-generated papers shred the long-standing (if rough) correlation between authors applying effort and the work having value. No one has worked out how to account for AI-inflated outputs in hiring and promotion decisions, nor what will happen to early-career researchers if tasks that are crucial to their training as scientists are done by a machine.

AI-driven science could change the nature of discovery itself. There are already tentative indications that the technology is influencing how people write and reason. A paper³ by researchers at Tsinghua University in Beijing found that adopting AI can make researchers more productive, but also shrinks the number of topics they study⁴. There is a risk that the ease of AI-based investigations could skew science towards certain fields and types of research, particularly in data-rich domains — potentially reducing scientific diversity.

AI is threatening science jobs. Which ones are most at risk?

Some researchers argue that AI simply changes where to focus human skills, just as calculators freed humans from relying on their own arithmetic. But no one ever had to worry that a calculator’s response was wrong. It is for this reason that Nature already requires transparency in how LLMs are used in submitted articles, and will not accept such models as authors (see go.nature.com/40j450w). For the sake of reproducibility, when a model contributes to the creative part of a study, Nature encourages researchers to submit transcripts of prompts and model responses alongside the final outputs, as one would with data sets.

Publishing the details of The AI Scientist is a step towards understanding what value automation can bring to science. A lot more work is needed to ensure that such tools can benefit the whole research ecosystem. It is up to the research community to put guard rails in place to ensure that happens.