Carl Laflamme knew what protein he wanted to study, but not where to find it. It is encoded by a gene called C9ORF72, which is mutated in some people with the devastating neurological condition motor neuron disease, also known as amyotrophic lateral sclerosis. And Laflamme wanted to understand its role in the disease.
When he started his postdoctoral fellowship at the Montreal Neurological Institute-Hospital in Canada, Laflamme scoured the literature, searching for information on the protein. The problem was that none of the papers seemed to agree where in the cell this mysterious molecule operates. “There was so much confusion in the field,” Laflamme says.
He wondered whether a reagent was to blame, in particular the antibodies that scientists used to measure the amount of the protein and track its position in the cell. So, he and his colleagues decided to test the antibodies that were available. They identified 16 commercial antibodies that were advertised as able to bind to the protein encoded by C9ORF72. When the researchers put them through their paces, only three performed well — meaning that the antibodies bound to the protein of interest without binding to other molecules. But not one published study had used these antibodies. About 15 papers described experiments using an antibody that didn’t even bind the key protein in Laflamme’s testing. And those papers had been collectively cited more than 3,000 times1.
Serious errors plague DNA tool that’s a workhorse of biology
Laflamme’s experience isn’t unusual. Scientists have long known that many commercial antibodies don’t work as they should — they often fail to recognize a specific protein or non-selectively bind to several other targets. The result is a waste of time and resources that some say has contributed to a ‘reproducibility crisis’ in the biological sciences, potentially slowing the pace of discovery and drug development.
Laflamme is part of a growing community that wants to solve the problem of unreliable antibodies in research. He teamed up with molecular geneticist Aled Edwards at the University of Toronto, Canada, to set up Antibody Characterization through Open Science (YCharOS, pronounced ‘Icarus’), an initiative that aims to characterize commercially available research antibodies for every human protein.
There are also efforts under way to produce better-performing antibodies, to make it easier for researchers to find them and to encourage the research community to adopt best practices when it comes to choosing and working with these molecules. Antibody vendors, funding agencies and scientific publishers are all getting in on the action, says Harvinder Virk, a physician–scientist at the University of Leicester, UK. “It’s hard to imagine that a problem that has been going on so long will suddenly change — but I’m hopeful.”
Putting antibodies to the test
The immune system produces antibodies in response to foreign substances, such as viruses and bacteria, flagging them for destruction. This makes antibodies useful in laboratory experiments. Scientists co-opt this ability by using them to mark or quantify specific biological molecules, such as a segment of a protein. To be effective, these molecular tags need to have both specificity — a strong affinity for the target — and selectivity — the ability to leave other proteins unmarked.
For decades, scientists created these antibodies themselves. They injected proteins into animals, such as rabbits, whose immune systems would generate antibodies against the foreign molecules. To create a longer-term, more consistent supply of antibodies, researchers extracted immune cells from animals and combined them with immortalized cancer cells. When reagent companies began the mass production of antibodies in the 1990s, most researchers shifted to purchasing antibodies from a catalogue. Today, there are around 7.7 million research antibody products on the market, sold by almost 350 antibody suppliers around the world.
In the late 2000s, scientists began reporting problems with both the specificity and selectivity of many commercially available antibodies, leading researchers to call for an independent body to certify that the molecules work as advertised. Over the years, a handful of groups have launched efforts to evaluate antibodies.
What sets YCharOS apart is the level of cooperation that it has obtained from companies that sell antibodies. When Laflamme and Edwards set out to start YCharOS, they called every single vendor they could find; more than a dozen were interested in collaborating. YCharOS’s industry partners provide the antibodies for testing, free of charge. The partners, along with the funders of the initiative (which include various non-profit organizations and funding agencies), are given the chance to review characterization reports and provide feedback before they are published.
YCharOS tests antibodies by comparing their specificity in a cell line that expresses the target protein at normal biological levels against their performance in what’s called a knock-out cell line that lacks the protein (see ‘Ways to validate’).
In an analysis published in eLife last year, the YCharOS team used this method to assess 614 commercial antibodies, targeting a total of 65 neuroscience-related proteins2. Two-thirds of them did not work as recommended by manufacturers.
“It never fails to amaze me how much of a hit or miss antibodies are,” says Riham Ayoubi, director of operations at YCharOS. “It shows you how important it is to include that negative control in the work.”
Antibody manufacturers reassessed more than half of the underperforming antibodies that YCharOS flagged in 2023. They issued updated recommendations for 153 of them and removed 73 from the market. The YCharOS team has now tested more than 1,000 antibodies that are meant to bind to more than 100 human proteins.
“There’s still a lot of work ahead,” Laflamme says. He estimates that, of the 1.6 million commercially available antibodies to human proteins, roughly 200,000 are unique (many suppliers sell the same antibodies under different names).
“I think the YCharOS initiative can really make a difference,” says Cecilia Williams, a cancer researcher at the KTH Royal Institute of Technology in Stockholm. “But it’s not everything, because researchers will use these antibodies in other protocols, and in other tissues and cells that may express the protein differently,” she says. The context in which antibodies are used can change how they perform.
Other characterization efforts are trying to tackle this challenge. Andrea Radtke and her collaborators were part of a cell-mapping consortium called the Human BioMolecular Atlas Program when they set up the Organ Mapping Antibody Panels (OMAPs). OMAPs are collections of community-validated antibodies used in multiplex imaging — a technique that involves visualizing several proteins in a single specimen. Unlike YCharOS, which focuses on conducting rigorous characterizations of antibodies for various applications in one specific context, OMAPs is looking at a single application for the antibodies, but in several contexts, such as in different human tissues and imaging methods. To do so, OMAPs recruits scientists from both academia and industry to conduct validations in their own labs.
“Vendors cannot test all possible applications of their antibodies, but as a community we can say ‘let’s try this’,” says Radtke, who now works as a principal scientist at the instrumentation company Leica Microsystems in Bethesda, Maryland. “People are testing things that you would never think you could test.”
Expanding the toolbox
Even if good antibodies are available, they are not always easy to find. In 2009, Anita Bandrowski, founder and chief executive of the data-sharing platform SciCrunch in San Diego, California, and her colleagues were examining how difficult it was to identify antibodies in journal articles. After sifting through papers in the Journal of Neuroscience, they found that 90% of the antibodies cited lacked a catalogue number (codes used by vendors to label specific products) — making them almost impossible to track down. To replicate an experiment, it’s important to have the right reagents — and proper labelling is crucial to finding them, Bandrowski says.
After seeing that a similar problem plagued other journals, Bandrowski and her colleagues decided to create unique, persistent identifiers for antibodies and other scientific resources, such as model organisms, which they called research resource identifiers, or RRIDs. Catalogue numbers can disappear if a company discontinues a product — and because companies create them independently, two different products might end up with the same one. RRIDs solve this.
‘A landmark moment’: scientists use AI to design antibodies from scratch
In 2014, Bandrowski and her team started a pilot project3 with 25 journals, in which they asked authors to include RRIDs in their manuscripts. In the years since, more than 1,000 journals have adopted policies that request these identifiers. “We currently have nearly one million citations to RRIDs from papers,” says Bandrowski.
Ultimately, the hope is that authors of every journal article will clearly label the resources they used, such as antibodies, with RRIDs, Bandrowski says. “That won’t change reproducibility by itself, but it is the first step.”
In addition to being able to track down antibodies, researchers need a way to choose which ones to use. In 2012, Andrew Chalmers, who was then a researcher at the University of Bath, UK, co-founded CiteAb, a search engine to help researchers find the most highly cited antibodies. Over the years, the platform has grown to include more than seven million antibodies — and now also includes, when available, information regarding validations. In May, CiteAb began integrating YCharOS’s characterization data onto its site.
“The big challenge is that antibodies are just used in so many different ways, for so many different species that you can’t tick off that an antibody is good or bad,” Chalmers says. Many say that knock-out validation is key, but less than 5% of antibodies on CiteAb have been validated in this way, either by suppliers or through other independent initiatives, such as YCharOS. “There’s a long way to go,” Chalmers says.
Stakeholders get involved
Like many others, Virk developed an interest in antibody reliability after a personal experience with bad antibodies. In 2016, Virk received a big grant to study the role of a protein called TRPA1 in airway inflammation. But one of his colleagues mentioned that, on the basis of his own experience, the antibodies he was working with might not be reliable.
When Virk put TRPA1 antibodies to the test, he discovered that his colleague was right: of the three most-cited antibodies used to study TRPA1, two didn’t detect the human protein at all, and the other detected several other proteins at the same time. “That was a shock,” Virk says. “At that point, I wanted to leave science — because if things are really this unreliable, what’s the point?”
Instead of leaving academia, Virk co-founded the Only Good Antibodies (OGA) community last year, with the aim of bringing together stakeholders — such as researchers, antibody manufacturers, funding agencies and publishers — to tackle the problem of poorly performing antibodies. In February, the OGA community hosted its first workshop, which included individuals from these various groups to discuss how to improve the reproducibility of research conducted with antibodies. They were joined by NC3Rs, a scientific organization and funder, based in London that focuses on reducing the use of animals in research. Better antibodies means fewer animals are used in the process of producing these molecules and conducting experiments with them.
Blame it on the antibodies
Currently, the OGA community is working on a project to help researchers choose the right antibodies for their work and to make it easier for them to identify, use and share data about antibody quality. It is also piloting an YCharOS site at the University of Leicester — the first outside Canada — which will focus on antibodies used in respiratory sciences. The OGA community is also working with funders and publishers to find ways to reward researchers for adopting antibody-related best practices. Examples of such rewards include grants for scientists taking part in antibody-validation initiatives.
Manufacturers have also been taking steps to improve antibody performance. In addition to increasingly conducting their own knock-out validations, a number of suppliers are also altering the way some of their products are made.
The need to modify antibody-production practices was brought to the fore in 2015, when a group of more than 100 scientists penned a commentary in Nature calling for the community to shift from antibodies generated by immune cells or immune–cancer-cell hybrids, to what are known as recombinant antibodies4. Recombinant antibodies are produced in genetically engineered cells programmed to make a specific antibody. Using these antibodies exclusively, the authors argued, would enable infinite production of antibodies that do not vary from batch to batch — a key problem with the older methods.
A few manufacturers are shifting towards making more recombinant antibodies. For example, Abcam, an antibody supplier in Cambridge, UK, has added more than 32,000 of them to their portfolio. “Facilitating the move towards recombinants across life-science research is a key part of improving reproducibility,” says Hannah Cable, the vice-president of new product development at Abcam. “That’s something that antibody suppliers should be doing.”
Rob Meijers, director of the antibody platform at the Institute for Protein Innovation in Boston, Massachusetts, a non-profit research organization that makes recombinant antibodies, says that this shift simply makes more business sense. “They’re much more reproducible, you can standardize the process for them, and the user feedback is very positive,” he says.
Standardize antibodies used in research
CiteAb’s data have revealed that scientists’ behaviour around antibody use has shifted drastically over the past decade. About 20% of papers from 2023 that involved antibodies used recombinants. “That’s a big change from where we were ten years ago,” says Chalmers, who is now CiteAb’s chief executive.
Although the ongoing efforts to improve antibody reliability are a move in the right direction, changing scientists’ behaviour remains one of the biggest challenges, say those leading the charge. There are cases in which researchers don’t want to hear that an antibody they’ve been using for their experiments isn’t actually doing what it’s meant to, Williams says. “If somebody is happy with the result of an antibody, it’s being used regardless, even if it’s certain that it doesn’t bind this protein,” Williams says. Ultimately, she adds, “you can never get around the fact that the researcher will have to do validations”.
Still, many scientists are hopeful that recent efforts will lead to much needed change. “I’m optimistic that things are getting better,” Radtke says. “What I’m so encouraged by is the young generation of scientists, who have more of a wolf-pack mentality, and are working together to solve this problem as a community.”
