Can one AI tell if another has written a novel? The Pangram debate, explained

The Spanish-language literary world is facing a new crisis — one other fields have been dealing with for years: the mass arrival of AI and how to live with it. EL PAÍS revealed on Friday that Pangram, one of the most highly regarded detectors, found that 99% of the novel Donde el agua trabaja (Where Water Works), by Peruvian writer David Silva Rojas, was AI-generated text. The manuscript won the inaugural Almadía Ventosa-Arrufat International Short Novel Prize in July 2026, a €20,000 ($22,000) award paid as an advance against royalties.

A lively debate emerged on social media about how one AI model can detect another and what that means for novelists, editors and other stakeholders. Pangram was founded in 2023 by Max Spero and Bradley Emi, graduates of Stanford University.

Before Pangram, they worked at companies such as Google and Tesla. Their concern foreshadowed the problem we face today: with the advent of ChatGPT, the cost of writing has fallen almost to zero. People would end up buried under a flood of machine-generated verbiage and would need some way to distinguish it from human writing. That was how Pangram came about. Here are some of the questions that have been raised today about this software.

1. What does it mean when 99% of a text is flagged?

That almost all of it was generated by AI. Pangram splits a text into segments and classifies each as “human,” “AI-assisted,” or “AI-generated.” The 99% figure refers to the share of the text that it identifies as AI-generated. It is not a probability score. In a text of more than 25,000 words, Pangram considers almost all of it to have been generated by AI.

2. How often does it mistake human writing for AI-generated text?

Very rarely, at least for texts of substantial length. At least seven independent evaluations have found that Pangram almost never classifies human-written texts as AI-generated. The false-positive rate ranges from 0% to 0.2% of the texts tested. Literature professor Steven Mollmann compiled 17 studies and found a total of 49 false positives in 36,282 human-written texts, a rate of 0.14%.

The most demanding test involves texts that Pangram could not possibly have seen during training. The moderator of an online fiction forum purchased 46 out-of-print books that had never been digitized, totaling 2.9 million words. Pangram could not have encountered them before, yet it produced no real false positives.

Other tests using older texts point to the same conclusion. A team at Duke University ran 3,000 passages from scientific articles published before ChatGPT through Pangram. It flagged only one as “AI-assisted” and none as “AI-generated.” Epoch AI tested Pangram on 495 human-written texts and found zero false positives.

Professor Tuhin Chakrabarty, author of a study on self-published books on Amazon, tested 50 self-published novels written before ChatGPT. None were flagged as AI-generated. At EL PAÍS, none of the 56 books published in 2019 and 2020 were classified as AI-written. Nor were recent books by well-known authors such as Javier Cercas, Isabel Allende, or María Dueñas. At the same time, Pangram successfully identified all 29 AI-generated texts submitted for testing.

3. Is Pangram an AI that detects other AIs?

Yes. There are thousands of AIs that are not chatbots. They all work similarly: neural networks trained on vast amounts of data. Some models “learn” to handle language, like ChatGPT. Others “learn” to predict weather from temperature, pressure and other variables. Pangram was trained solely to distinguish human-written texts from AI-generated texts. It finds patterns the human brain cannot perceive. While some grammatical and syntactic clues may be visible to human readers, many others are not. That is where Pangram performs best.

The system was trained on tens of millions of documents. Each human-written text was paired with a “mirror” version generated by an AI attempting to imitate it. The model then learned to tell them apart. Its creators were especially focused on minimizing false positives, cases in which a human author is wrongly accused of using AI. To achieve this, they exposed the model to large volumes of especially difficult human-written texts that were challenging to classify. And they repeated this process thousands of times.

According to the company’s figures, Pangram now incorrectly labels only about one in every 38,000 human-written texts as AI-generated. Pangram’s CEO explained that the system is evaluated using millions of texts written before 2022: “Text and writing that we know is not AI-generated, but it’s also not in our training set.” This ensures that Pangram is not simply recalling texts from memory.

4. Could the novel’s style, with its very short sentences, throw off the detector?

Silva Rojas’s novel is written in a style characterized by very short sentences which, according to the author, is influenced by Japanese haiku. Could that have confused the detector?

To test that possibility, we examined other texts by the author that do not use the same style. In an interview with Ulrica magazine, which confirmed that it received his responses in writing, Pangram attributed 97% of the answers to AI-generated text, while classifying the journalist’s text as entirely human. Another first-person piece by Silva Rojas describing how the novel came into being scored 80%.

Neither text relies heavily on short sentences. On average, they contain around 16 words per sentence, compared with just five in the novel.

5. Is it infallible?

No. False positives do occur, although extremely rarely according to all the studies we cited. False negatives are more common: cases in which Pangram classifies AI-generated text as human. This is a precautionary design choice. The detector is calibrated to err on the side of not accusing human authors of using AI. As a result, each positive result carries greater credibility.

6. Is this the first Spanish-language book detected by Pangram?

No. Two weeks ago, we ran 175 bestselling books in Spanish through Pangram. We analyzed books from most major Spanish-language publishers, including prize winners. We did not detect AI in books by leading fiction writers such as Javier Cercas, Antonio Muñoz Molina, Isabel Allende, María Dueñas, and Dolores Redondo. No traditionally published book returned a high AI score, although we did find indications of AI-generated text in seven of the 22 nonfiction titles we tested.

Where we did find substantial AI-generated content was among bestselling self-published books on Amazon. Four out of 10 contained at least 25% AI-generated text. The most prominent example was Spanish politician Alvise Pérez’s Devuélveme mi país (Give Me Back My Country), which Pangram judged to be 97% AI-generated. Pérez maintains that he wrote it “word by word.” Several romance novels also scored above 50%. Like Silva Rojas, their authors did not respond to this newspaper’s requests for comment.

7. Why is it outdated to say that no AI-detection tool works?

Because, until 2025, none worked particularly well. Pangram released its first model in 2024, but it did not become widely used until last year. On October 19, the company launched version 4.1, a substantial improvement over earlier iterations. Many people continue to judge AI detectors based on the ineffective tools available a few years ago. At the time, that criticism was justified. Today, it is not.

Artificial intelligence advances month by month, as successive versions of Claude and ChatGPT demonstrate. The same is true of AI-detection systems.

That said, many detectors still do not work. Some present an obvious and questionable conflict of interest: users paste in a text, are told it contains AI-generated content, and are then offered a paid “humanization” service to make it undetectable.

Pangram does not offer such a service. In fact, one of its goals is to identify AI-generated texts even after they have been artificially “humanized.”

8. Does it mistakenly identify classic works such as Don Quixote as AI-generated?

This claim is often repeated on X and other social media platforms. It may have been true in the past, either with very short excerpts or with other detection tools, but our tests suggest it is not the case today.

We ran excerpts from eight classics through Pangram, including Don Quixote, Lazarillo de Tormes, works by Benito Pérez Galdós, Emilia Pardo Bazán and Miguel de Unamuno, as well as Alice’s Adventures in Wonderland, Pride and Prejudice, and Genesis. Using the first 1,000 words and another 1,000-word sample from the middle of each text, Pangram classified every passage as 100% human-written.

That result, however, is not especially surprising. These are famous works that the detector may effectively know by heart. So we conducted a second test: we mixed the opening of Don Quixote with paragraphs generated by an AI instructed to imitate Cervantes. Pangram identified 84% of the AI-written passages while not flagging a single line by Cervantes. Once again, when the system makes mistakes, it tends to do so by missing AI-generated text, not by falsely accusing a human author.

9. Is it a problem if a writer uses AI to write?

That is a question for society as a whole, including writers and readers. Many programmers have largely accepted code that is written entirely by AI. Illustrators face a different debate over the fact that someone with no artistic training can now create a poster. Writers are still deciding whether AI is something to keep out of the creative process altogether or whether it is acceptable for a human author to “collaborate” with a machine.

It is important to understand what tools like Pangram actually detect. They can indicate that a text was generated by AI, but they cannot determine precisely what role a human played in the process.

AI can be used for extremely passive, almost automatic writing. But it can also be used to turn a person’s own ideas, notes, outlines or transcripts into polished prose. There is a vast grey area in between. There are legitimate uses of AI, and it is possible to imagine forms of AI-assisted writing that preserve genuine authorship.

Can a novel be written with AI in the same way that a film director works with screenwriters, actors and special-effects teams? Those are the debates that lie ahead. Which is why a witch hunt serves no purpose.

10. Why is this case problematic?

The question of what should or should not be allowed is not for journalists to decide.

This case was significant because of the size of the prize and because the competition’s rules stated explicitly that submitting an entry constituted “an express declaration that the work has not been written, either partially or entirely, using artificial intelligence tools.” The rules went on to specify that “any indication or suspicion of non-compliance with this provision shall result in the work’s disqualification.”

The case nevertheless raises a question that is unlikely to go away: what does it mean to write a book “partially” with AI? If AI merely edits your prose, does that count? What if it turns your notes into sentences? What if it generates text a hundred times and you guide and refine the output until it becomes your own? Many literary prizes are likely to find themselves rewriting their rules in the years ahead.

11. Have there been similar cases in other languages?

Yes. The most famous case is that of Canadian writer Thélyson Orélien. In late September, an account on X accused him of using AI to write his award-winning novel C’était ça ou mourir (It Was That or Death). Nearly a month later, his publisher, Les Éditions du Boréal, ended its relationship with him.

The publisher wanted Orélien to provide evidence of his creative process, but said its efforts “ failed to resolve the question of whether AI was used in his novel.” Orélien subsequently announced that he would provide no further documentation. While stressing that it did not wish to act as a policing authority, the publisher commissioned an independent review, whose findings “revealed traces of AI tags in the metadata of certain documents.”

The controversy has already had wider repercussions. According to Radio France, the French publishing house Gallimard will begin screening manuscripts with an AI detector starting in 2027.

In March, The New York Times reported that a horror novel by Mia Ballard was assessed by Pangram as being 78% AI-generated. The author blamed an associate she had hired, alleging that he had used AI without her knowledge. The publisher subsequently canceled the book’s U.S. release.

Then, in October, Zimbabwean writer Simbarashe Steyn Kundizeza won the Caine Prize for African Writing, awarded in London and worth £10,000 ($13,300). The Atlantic later reported that an AI detector had attributed the text to artificial intelligence. The prize organizers have opened a review, which is still ongoing.

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition