Universities are relying on AI-detection software to catch cheating. How well do the programs work?

Last November, Lauren Jager, a chemistry undergraduate student at Idaho State University in Pocatello, was applying to PhD programmes when she noticed that some application portals warned students about using generative artificial-intelligence tools for their personal statements. They informed students that they would use detectors to sniff out applications that contained AI-generated text. The portals weren’t specific about which detectors they were using. But they were clear on one thing: “They said that if they felt that the personal statement had been written with AI, then they would disregard your entire application,” Jager says.

She didn’t think much of it — she hadn’t used AI at all — but a friend said they’d run their own statements through an AI detector on the Internet, just for safety. Jager decided to do the same with a few detectors she’d found online.

“They all came back at almost 100% AI,” she says. “I started freaking out.”

AI writing tools could hand scientists the ‘gift of time’

Looking back, Jager wonders why her essays were flagged. “I’m a chemistry major, but I was almost an English major. I grew up writing a lot and I used to study grammar books,” she says. “I think of myself as a good writer, but I write very much to the book, following all the rules specifically. Maybe that’s why it thought it had been written by AI.”

She ended up rewriting her statement entirely. And, instead of trying to write to the best of her ability, she says she wrote in a way to make sure it wasn’t flagged by the AI detector. “I was making it less perfect,” Jager recalls. When she ran her essay through the checkers again, they predicted that it was 30% AI-written. “I called that good enough and sent it in.”

When we last spoke to Jager, she had been accepted to do her PhD at the University of Utah in Salt Lake City.

Jager’s situation and cases like it are happening around the world. Universities that are struggling with a surge of written material that might have been generated, or heavily shaped, by AI are turning to AI-detector tools to help them grapple with the problem. And, just like other AI technology, such tools are far from perfect.

Educators and copycats have been trying to outsmart each other since the first Sumerian student copied their classmate’s cuneiform. In the modern Internet era, software provided by companies such as Turnitin, based in Oakland, California, detects text similarities on the basis of a vast corpus of previously published work. This has made it much harder for people to plagiarize text directly. In response, cheaters went online to find ghostwriters, paying others to produce their work for them. Many institutions tried in-person or timed exams to prevent this. But these come with their own set of issues, potentially disadvantaging certain groups, encouraging rote memorization and preventing students from demonstrating their ability to conduct deep research.

Shadow scholars: inside Kenya’s multibillion-dollar fake-essay industry

But large language models (LLMs) that power chatbots, such as ChatGPT, have made it even cheaper and faster to produce written work. Many of those interviewed for this article noted the irony that even though generative AI is built on a corpus of previous work, the writing it produces isn’t easily detected by plagiarism tools, which compare full sentences with previously published work.

Cath Ellis, an academic integrity professional at Western Sydney University in Australia, says AI use represents a fundamental change, both in the scale of its use and how to understand the written word. “Up until now, we’ve been mostly able to rely on a written document,” Ellis says. But “we’re now starting to see this massive volume of fraudulent or at least heavily fabricated processes,” she says. “The volume of stuff coming through has just rocketed.”

Fighting fire with fire

A growing number of companies say that the tools they have made can identify text that has been written by another AI system. On-the-market products include Copyleaks, GPTZero and ZeroGPT, and those created by Grammarly, QuillBot and Turnitin.

Many AI detectors rely on a measure known as perplexity, which estimates how predictable each word in a sequence is likely to be. Because AI-generated text tends to follow more statistically predictable patterns than does human writing, passages with lower perplexity scores are more likely to be flagged as machine-generated, whereas less predictable phrasing is taken as a signal of human authorship (see ‘The telltale signs of AI’).

The question is: do these tools actually work? And should they be used at all if there is any chance of unjustly accusing a student, like Jager, of cheating?

Academics have attempted to assess this. One 2025 paper¹ tested GPTZero, which the authors described as the most widely used AI detector. The study found that most fully AI-generated papers were detected with high confidence, but GPTZero’s rate of falsely identifying human-written essays as AI-generated — its false-positive rate — was about 16%. The authors concluded that its “reliability in distinguishing human-authored texts is limited”.

A 2023 study² evaluated several AI-detection tools – OpenAI, Writer, Copyleaks, GPTZero and CrossPlag. It found they were generally better at identifying text generated by the LLM GPT-3.5 than by the more advanced GPT-4 model. When applied to human-written passages, the tools produced inconsistent results, including false positives and uncertain classifications.

Clear evidence of AI detectors’ difficulties in correctly assessing human-written text was highlighted by several users on the social-media platform Reddit — they discovered that the US Declaration of Independence is often flagged as AI-written. Nature ran part of the 1776 text through ZeroGPT a number of times and was told it was between 95% and 100% AI-generated.

Finding your academic voice: faster, healthier writing with AI speech recognition

Other detectors might show more promise. Pangram Labs, based in New York City, says that its tool has a near-zero false-positive rate. Its approach involves training a model on a large corpus of human-written, then AI-rewritten, text. The model thus comes to understand how each newly released chatbot writes. So far, this approach, which avoids using perplexity as the only direct measure, has stood up to scrutiny: independent assessments have judged the product to be among the most accurate available.

Mike Perkins, who researches the impact of AI on academia at the British University Vietnam in Hanoi, says even when detectors perform reasonably well in controlled tests, their results should not be used as evidence in high-stakes decisions. “The short answer is no, they don’t [work reliably],” he says. “The long answer is yes, they can work — but the fact that there are so many concerns about false positives means they shouldn’t really be used when it comes to anything that’s sensitive for a student.” Otherwise, students like Jager can be caught in the net.

Part of the problem, argues Perkins, is that teachers have grown accustomed to accepting automated scores from plagiarism software. “People see a score and trust it,” he says. “Similarity tools worked because they could show you exactly where text matched something else. With AI-detection tools, that evidence just isn’t there.”

William Walters, a librarian and social scientist at Southern Illinois University in Carbondale, also found that detectors were much better at spotting text written using GPT-3.5 than GPT-4³. He said that only 3 of the 16 tools he tested performed strongly across AI and human writing alike. Speaking to Nature almost three years after publishing his study, Walters points out that many iterations of GPT models have since been launched. (The latest GPT model available to the public is GPT-5.5.) These newer models are even better at producing human-like writing.

Evading the detectors

Hybrid texts are another problem for those hoping to catch cheaters. “Work that I, and others, have done shows that if you just test a piece of AI text against a detector, it’s going to be pretty good at identifying it,” says Perkins. “But if you start to manipulate that text in different ways, then the detection really starts to break down.”

People don’t even need to edit passages themselves: they could ask another AI system to rewrite them, or run the text through ‘humanizer’ tools that are designed to lower AI-detection scores. Detection companies are now trying to identify the use of such tools, adds Perkins, but new systems that evade detectors are emerging quickly. “It becomes a huge arms race that doesn’t really help anyone,” he says.

AI FOMO: everyone is mastering AI except me — or are they?

This year, Marzena Karpinska, a linguist and computer scientist at Simon Fraser University in Burnaby, Canada, and a group of researchers used Pangram’s detector to analyse 186,000 articles published online by 1,500 national and local newspapers in the United States between June and September 2025, and compared them with writing published before the release of ChatGPT⁴. They concluded that “approximately 9% of newly published articles are detected as either partially or fully AI-generated”.

But Karpinska cautions that although this information might be useful for assessing how much content is probably generated by AI on a large scale, it does not mean that the results from Pangram — or from other tools trying to catch up to its abilities — should be taken at face value in individual cases. In other words, it can reveal trends, but not the guilt of any one author. “We certainly cannot mass-reject people because of it,” she says.

The issue of bias

In 2023, researchers at Stanford University in California tested the performance of 7 AI detectors on 91 essays obtained from a Chinese educational forum. The essays were part of students’ Test of English as a Foreign Language exams and took place before 2020 and the advent of ChatGPT⁵. More than half were incorrectly labelled as AI-generated, equivalent to an average false-positive rate of 61.3%. The study also analysed the detectors’ performance when presented with 88 essays written by US students aged 13–14 and found that these were accurately classified.

Expert-level test is a head-scratcher for AI