Registered Reports avoid the ‘file-drawer problem’, in which negative or inconclusive results never see the light of day.Credit: Getty
Do algorithms used by social-media platforms amplify posts that are emotive, toxic, political or perceived as moralizing? And, if they do, does that affect how users see their social world? William Brady, a social psychologist at Northwestern University in Evanston, Illinois, and his colleagues wanted to test these questions during the 2024 US presidential election. To maximize their study’s rigour, they did something rare in science publishing1: they submitted their topic and research plan for peer review before they had gathered a single data point.
Read the paper: Redesigning algorithms to intervene on social norm misperceptions during a national election
The resulting Registered Report is the first completed study2 of this kind published in Nature since we began inviting them in 2023. Today, we are issuing new guidelines that expand the types of research — and research field — that can use the Registered Report format.
Until now, submissions for Registered Reports were considered only for confirmatory research — essentially types of study that test hypotheses — and only in cognitive neuroscience and the behavioural and social sciences. From now on, we will welcome Registered Reports across the fields in which we publish, extending beyond hypotheses-testing studies to include those that, for example, gather large amounts of data or compare scientific methods.
What exactly are Registered Reports? Authors propose a topic of study, explain its importance and set out their plans for data collection and analysis — including hypotheses, where applicable. These are reviewed before data are collected or, in the case of secondary analyses, before existing data are accessed. Once a proposal has undergone rigorous peer review, and editors and referees are convinced that the question is of significant and broad interest and that the data and design are the best possible way to provide answers, then the journal commits to publishing the study, whatever the outcome.
This approach has many benefits. It strengthens a study’s methods, including data collection and analysis. Reviewers are more involved in the research process because they can make suggestions about the design and analyses. In fact, the reviewers help to shape a project rather than commenting at the end, as is the case for conventionally published research.
Nature welcomes Registered Reports
Importantly, the process ensures that all results are published. Registered Reports thus avoid the ‘file-drawer problem’, in which negative or inconclusive results never see the light of day. Because authors must stick to the precise analysis they outlined in advance, the format can, in some cases, help to reduce P hacking, in which researchers conduct many statistical analyses until they get a significant result.
Brady says it was a “no-brainer” to do this research as a Registered Report. His team’s study involved manipulating the feeds of 2,000 users on the social-media platform Bluesky, using three distinct algorithms over eight weeks. The work involved hiring a software engineer and running something similar to a start-up company for a year, without any guarantee of what the study would find. It was made feasible only by the commitment that the work would be published, come what may, he says.
Getting reviewer feedback before the experiment began was another benefit. It led the authors to amend the study design to use data from existing Bluesky users rather than fresh volunteers, so as to obtain more-meaningful responses from people genuinely invested in their interactions on the platform. The Registered Report format demanded meticulous planning and a clarity that helped the experimental process to run smoothly. “I basically now tell students, let’s imagine we are going to do a Registered Report, because it helps us plan and it saves you so much time later on,” he says.
Registered Reports can be challenging. They are intensive, involving two stages, each of which typically needs multiple rounds of peer review. And it can be a struggle to be locked into a particular analysis. For Brady, there turned out to be many fewer ‘toxic’ posters than anticipated, which meant that some of the data were not a good fit for the planned analyses. In such cases — as our guidelines say — authors are allowed to carry out unforeseen ‘exploratory’ analyses, as long as the tests are clearly identified as such, are justified, are reported separately from the main results and do not become the basis of the paper’s conclusions.
Good research begins long before papers get written
As it happens, Brady’s team confirmed its hypothesis that social-media algorithms designed to boost engagement did amplify moralized, emotional, toxic and political content. The authors also found that there are ways to reduce users’ exposure to divisive content without decreasing enjoyment of the platform. And they found that inflammatory posts did not alter users’ behaviour by significantly boosting their engagement with such content. As a classic ‘null result’, this finding might not have been published if not in a Registered Report, says Brady.
The importance of the question and the quality of data collection and the planned analyses are not confined to confirmatory research. Expanding Registered Reports to more disciplines and research types reflects our conviction that these standards can elevate all forms of inquiry. This should make science more rigorous and transparent — and put a greater emphasis on work that meaningfully advances knowledge.
