Peer review by committee? New journal rethinks old model

The peer-review system has been stressed and stretched to a near-breaking point. It’s becoming harder to find reviewers, many of whom see reviewing as a burden that is not adequately rewarded. The rise of predatory publishers, many of which falsely claim to provide a peer-review process; paper mills, which are known to fabricate peer reviews; and plagiarism of peer-review reports have harmed trust in the system.

The Stacks Journal is aiming to provide a faster, more transparent and trustworthy peer-review model by organizing committees of researchers to assess manuscripts.

Launched in July as an open-access, digital-only publication, the Stacks Journal is the brainchild of David Green, an ecologist based in Portland, Oregon. The inspiration, says Green, was his own experience with the inefficiencies of academic publishing. In 2020, Green, who had finished a study on the impact of wildfire on carnivores¹, wanted to get the results out quickly so that they could inform land-management policy. But his paper languished in the publishing system for almost two years, with no clear explanation as to why. So, he resolved to change the process.

Green spoke to Nature Index about the inspiration for the Stacks, and how he hopes it will fix some of the weaknesses of academic publishing.

What inspired you to launch the Stacks Journal?

I talked to other ecologists at conferences and field sites, and everyone was frustrated with the status quo of scientific publishing — from huge article-processing fees and long peer-review times to the rise of predatory journals. These and other factors undermine people’s ability to publish their research; estimates from clinical-trial data suggest that around 50% of good data never get published². We’re missing out on a lot of important information.

I started researching peer review and learnt that it hasn’t changed much in the past 40 years. So, I explored what a new system could look like. I did in-depth interviews with dozens of researchers in different fields and surveyed hundreds more to test ideas.

The result is the Stacks Journal’s peer-review process, which was designed to reflect how people discuss ideas in the Internet age: meeting online to collaborate across social-media platforms, for example. Advances in the way we communicate haven’t yet made it to the peer-review process.

2024 Research Leaders

How does the Stacks Journal’s peer-review model work?

We are shifting peer review away from an individual gatekeeper model, wherein an editor at a journal decides what should be published. Instead, we use a community-based model, in which we gather input from a group of people to collectively determine whether an article is published. We’ve designed this model to be rewarding to both authors and reviewers, and completely transparent.

What’s key is that the Stacks Journal’s peer-review process happens in collaboration instead of isolation. This is how peer review and publishing used to work. For instance, in the nineteenth century, the Royal Society in London invited groups of scholars with expertise in specific topics to come together, debate new work and determine whether it would be published. Now, most journals have two reviewers who assess a manuscript separately. At the Stacks, we bring together communities of reviewers to collaborate. It’s double-blind, to ensure fairness, and reviewers can see each other’s comments and discuss whether they agree.

All the peer-review reports, underlying data and code are publicly posted, along with the names of the reviewers.

What else sets the Stacks Journal apart?

We’ve created a ‘credibility score’ for each published article, so readers can quickly get a sense of the reviewer’s feedback. The credibility score is calculated as the percentage of reviewers who voted to accept the article for publication. So, for example, if six out of seven reviewers think an article should be published, its score will be 86%.

To recognize the role of the reviewers in contributing to the research, they can opt in to be credited as ‘collaborators’, listed just below the authors on the published article. That way, a reviewer can include their work on their CV.

Our publishing model is also different — we offer an annual membership for US$199 that allows unlimited open-access publishing. In conventional publishing, it can cost thousands of dollars to publish one article. In our research, we found that this limits a lot of researchers from ever sharing or publishing their findings.

How does the journal find and coordinate reviewers?

The Stacks is built on communities of researchers that form around specific topics. Right now, we’re focused on ecology, but soon we’ll add chemistry, computer science and medicine. Any eligible researcher can sign up to be a reviewer on our website for free. To be eligible, you must have published at least one peer-reviewed article in the relevant field of study.

When we receive a submission, we send it to reviewers with expertise in the paper’s topic. Reviewers submit their feedback on our online platform, which they use to discuss among themselves. The reviewers are all blinded to each other’s identities during the process, and no individual carries more weight than another.

It has been easy for us to find reviewers. They find the process rewarding, and they keep coming back.

What challenges have you encountered?

We’ve had to cap the number of reviewers on each article at seven, because that’s what our software can handle. This means we’ve had to turn people away. We want to have unlimited reviewers on every article, so we are building new software to make this happen.

Another challenge is the fact that we are a new journal — we don’t have an impact factor or third-party marker of credibility, so some scientists are not ready to submit their research to us. However, authors who have say that they love how streamlined the publishing process is and how much our review system strengthened their papers, which brings credibility to their research that is more long-lasting than that afforded by most journals.

Over the next year, we aim to publish more than 100 articles, including our first special issue, and will continue finding ways to do peer review in a more productive and efficient way.

This interview has been edited for length and clarity.

Nature Index’s news and supplement content is editorially independent of its publisher, Springer Nature. For more information about Nature Index, see the homepage.