Science-integrity sleuths check the text and images in published papers for errors and inconsistencies. Credit: tadamichi/Getty
A group of research-integrity experts has launched a toolkit for researchers that outlines how to spot suspicious scientific papers.
The guide’s creators hope that it will be a useful resource for sleuths who review published work for signs of sloppy, fake or fraudulent science, and perhaps inspire others to get started. “The sleuthing community is growing. And this will be the essential handbook,” says Jana Christopher, an image-integrity analyst at FEBS Press in Heidelberg, Germany, who contributed to the project.
The Collection of Open Science Integrity Guides (COSIG) brings together 27 freely available resources that explain how to spot image duplication, citation manipulation, plagiarism, tortured phrases and other hallmarks of paper mills — businesses that produce fake papers to order. The guides also provide tips for reviewing papers in specific disciplines, including biology, chemistry, statistics and computer science.
“COSIG is really a compendium of all the tips and tricks that various sleuths have acquired over the years,” says Reese Richardson, a metascientist at Northwestern University in Evanston, Illinois, who led the project and authored a preprint on COSIG posted to the repository Zenodo on 4 June1. “We want to arm scientists with the tools to uphold the integrity of the literature.”
“It will really help publishers, as well,” says Christopher, adding that research-integrity officers could use the guide as the basis for checklists to evaluate manuscripts.
‘Anybody can do it’
Richardson came up with the idea for COSIG last year, while at a meeting of science-integrity specialists in Paris. Concerned about the surge in formulaic research and flawed publications that continue to circulate largely unnoticed, he and other delegates wanted to broaden participation in correcting the literature and to help more people learn how to scrutinize papers.
“A lot of people assume that you need some special talent, you need eagle eyes to see things, or you need to be at your computer ten hours a day looking through the scientific literature. But really, anybody can do it,” says Richardson. “That’s one of our mantras.”
The guides describe how to spot issues that are common across all disciplines, such as text that has been generated using artificial intelligence (AI), and publications from hijacked journals — scam websites that impersonate legitimate journals. They include instructions for how to use online tools such as the Feet of Clay Detector, which flags papers citing retracted or problematic research, and Imagetwin and Proofig, which use AI to detect integrity issues in scientific figures.
Some of the guides home in on areas of research that have seen a recent surge in formulaic publications, and those that are frequently targeted by paper mills.
For example, one guide shows how to detect experiments that use non-existent cell lines, or contaminated ones that are no longer suitable for studying certain cancers. Another outlines how to check that tumour sizes in rodent studies fall within limits approved by institutional ethics committees (a reported size that is too large to have been approved can be a sign of manipulated or fabricated experiments).
There are examples of papers that conflate a substance’s composition by weight with the number of each atom type, or instances in which an X-ray diffraction pattern doesn’t match a crystal’s properties.
“Every guide specifies what features of an article actually warrant additional scepticism and what features are totally benign or standard practice,” explains Richardson.
