Can one AI unmask another? Pangram, an almost infallible tool for detecting machine-created texts and novels

The book C’était ça ou mourir has won the Fnac novel prize, the Medusa award and others, has sold more than 35,000 copies and is among the contenders for the major prizes of the autumn literary season. As of September 23, the book has another distinction: it is the first major French literary scandal allegedly involving a work produced with artificial intelligence (AI). Its author, a Canadian of Haitian origin, Thélyson Orélien, denies the claim. He says he finished the first draft in 2019 (ChatGPT was released at the end of 2022) and that his “Haitian and Caribbean” literary tradition may have caused confusion because “the imagery, repetitions, and rhythm convey a living voice,” he told French media. His publisher, Grasset, is backing him for now. The accusation against Orélien came from an anonymous account on X.

Orélien’s problem is that the era in which no detector of AI-generated text was truly effective at catching offenders is over. Because now there is one that gets it right most of the time: Pangram, launched in 2024 in the U.S. And it is multilingual. “Pangram works well in 20 other major languages besides English, and that includes Spanish and French,” Max Spero, co-founder of Pangram, told EL PAÍS.

Pangram’s accuracy figures exceed 99%: only one in 24,000 samples is wrong, the company says. The scientific community has also found that Pangram outperforms the alternatives. “There is no reason to think these models don’t work, because they are trained on an enormous amount of data, both human-written and AI-generated,” says Tuhin Chakrabarty, professor of computer science at Stony Brook University (U.S.) and a specialist in AI-generated text.

Pangram is also an AI. It is a model trained to detect machine-generated text. It is shown huge volumes of human articles, novels, and essays and similar texts generated by AI until it learns the stylistic differences. Since 2024, the company has been releasing new versions of its software. Some tests can be run for free, but it operates on a subscription model. In August 2025 it had only 7,000 users; by last month it had reached 222,000.

Professor Marzena Karpinska of Simon Fraser University (Canada) checked the accuracy of an earlier version in a 2025 scientific paper about how the people who use ChatGPT most are then best at detecting its writing style: “The current version of Pangram, version 4, is much more accurate. It’s like the difference between GPT-4 and GPT-6. They have worked to remove biases seen in other tools, such as labeling texts by people whose native language is not English as AI-generated,” she explains.

The doubts surrounding Orélien are not only because of the writer. The French literary world has lavished praise and prizes on the novel. How can Pangram state categorically that Orélien’s novel was generated with AI, yet nobody noticed? Because they don’t use AI and don’t know how it writes, Chakrabarty believes. “Critics hate AI. They have never read what AI prose looks like. How are they going to recognize it? One reason our institutions fail to detect AI prose is that they aren’t familiar with it. People like me, who work on this around the clock, are not part of prize juries. I read this French novel and it has the same patterns as bad English-language fiction: negative parallels, mixed metaphors, clichés. All very overworked,” he explains. The U.S. has also seen several controversies over books and articles flagged by Pangram.

Since these tools became widespread there has been a race to evade detection: introducing typos, “humanizing” phrases with other software and looking for prompts that fool the detector, for example asking it to imitate the style of a famous writer. This newspaper tried examples of texts produced by Claude but in the style of Spanish writers such as Javier Marías or David Trueba: Pangram judged them 100% AI. “I’ve done this experiment dozens of times,” says Chakrabarty. EL PAÍS also tested the first two chapters of Santos Cerdán’s book La caída, and Pangram returned confidence levels between 70% and 100% that it was AI. “Those figures are enough to say there was heavy use of AI,” Spero says. Cerdán has not replied to this newspaper’s messages.

The most sophisticated tricks require computing knowledge that writers rarely have. “If you fine-tune a model with the books of a famous author, something I have done, it may be possible to deceive. But nobody does that because it’s very expensive and requires a lot of technical expertise. These writers aren’t computer scientists; they go into ChatGPT, give it a prompt and ask it to write each chapter,” Chakrabarty says.

A major problem with this tool is that authors can deny using AI, and it is impossible to prove. Without a confession, everything remains in the realm of suspicion. It can also trigger a witch-hunt if there are false positives. In recent days, social networks in France have been full of Pangram’s now-famous graph with people saying that texts written before ChatGPT’s release were AI-generated. In one case, Spero offered $100 to the author of one of those screenshots, a well-known journalist, for the text used. The journalist, at least publicly, did not send it to him.

Pangram is less effective at detecting short texts of only one or a few sentences. The more paragraphs sampled, the more reliable it is. Wide use of AI across all chapters of a book is a huge red flag, almost indefensible: “Pangram works better with long texts. Below 50 words it isn’t reliable, and the more text there is, the better. Likewise, it performs better on creative writing than on recipes, for example,” says Karpinska, who made an additional check on Orélien, whose defense is that his style is distinct and far from dominant traditions. “Some have suggested the issue is the author’s specific Haitian French. I found an article he wrote in 2015 and analyzed it. The result was that it was human, so it does not appear, at least in this example, that Pangram is detecting a dialectal peculiarity,” Karpinska adds.

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition