Are we all going to die or is it just theatrics? The arguments against an AI apocalypse

Fear of artificial intelligence (AI) has firmly entered the public debate. In recent days people have spoken of a possible extinction of humanity, of this technology being uncontrollable, and of the industry itself—not regulators—needing to self-regulate to rein in its own progress. These systems are already very powerful, and given their growth rate, no one knows how far they could go in the coming months or years.

But just as some warn about this technology’s dangers, other experts argue that the capabilities of these systems are being exaggerated. They also say the industry is using deep-rooted fears about these advances as a smokescreen to hide the real negative consequences: from mental health problems for some users to the challenge AI poses to education and work, to the pollution and water and energy consumption linked to the data centers that support it, and to autonomous weapons that are already killing humans in several war zones.

So are we all going to die or is there nothing to fear?

The context

Last week was frantic in the AI world. On Tuesday night, OpenAI announced—amid accusations of plagiarism—that one of its models had solved one of the seven Millennium Prize Problems, the most complex mathematical challenges known to humans. That same day, an Anthropic engineer resigned so as not to contribute with his work to the possible extinction of humanity: AI could kill us before 2030, he said. Anthropic announced on Friday that it had halted potential attempts to develop chemical weapons using its AI models, and reports leaked that Iran had used them to select targets in its war with the U.S.

And then the bomb dropped over the weekend: Anthropic’s chief executive, Dario Amodei, published an essay on Saturday calling for a substantial slowdown in AI development and for AI developers to submit to external monitoring. Sam Altman, his counterpart at OpenAI, agreed with Amodei. On Sunday, Google’s head of AI, Demis Hassabis, and even Elon Musk, owner of xAI, also said they favored slowing the pace of research. President Trump, for his part, believes the opposite: he sees the AI race as a war with China that must be won at all costs.

This carousel of news has happened in a very particular context. Both OpenAI and Anthropic are preparing for initial public offerings expected to be historic. To warm up the market, both companies have, in recent months, been balancing announcements intended to show that their products are extremely powerful and advanced while also claiming they do not pose unnecessary risks to humanity. Until now, they had managed to do both.

This summer, both OpenAI and Anthropic claimed that some of their models had supposedly “gone rogue” to hack third parties, although it later emerged that those attacks were part of instructions given to them. The British Artificial Intelligence Security Institute (AISI) reported that an autonomous AI system had created fake identities to manipulate people during exercises; in that case the system had internet access and a configuration error was found that let agents bypass some of the limits that had been set.

What if it’s not that bad?

One of the main arguments used against those who believe AI could bring about the apocalypse is that it serves as a distraction orchestrated by the industry so countries do not move to regulate the effects AI already has on society. “This god-machine narrative aims to distract us,” said Timnit Gebru, a former lead researcher on language-model bias at Google who was dismissed in 2020 before publishing a scathing report about the company.

When a bridge collapses, Gebru says, what you want to know is who built it so it doesn’t happen again, not whether the bridge was conscious or intended to collapse. Who approved the permits, how did it pass the relevant tests? “If you focus on the bridge collapsing itself, you’ve strayed from the issue,” she argues. “Many of the [AI apocalypse] scenarios seem to involve asking the machine to optimize for another task and then the machine deciding that the way to achieve that goal is to wipe out the human population. Does it also hypnotize you so you won’t unplug it?” she asks.

Anthropomorphizing AI has also contributed to public alarm. Ramón López de Mántaras, founder of the CSIC Institute for AI Research, has long warned that vocabulary is crucial on this subject. “The way we describe these incidents matters because metaphors shape social perception of AI and public policy. Talking about machines that ‘escape,’ ‘go rogue’ or ‘conspire’ promotes the idea of an AI acquiring its own will and can lead to misdiagnosing the problem,” he says. From what we know so far about this summer’s cybersecurity incidents, machines did not rebel; rather, increasingly powerful AI systems did exactly what they were asked: use complex attack paths to find solutions to their tasks, though not in the way OpenAI engineers had anticipated.

One claim, for example, was that “swarms” of AI agents coordinated to carry out attacks. “It’s quite normal for independent software programs to work together sharing information. That is, in fact, an essential part of distributed computing methods on which large software systems are based,” says Melanie Mitchell, the Davis Professor of Complexity at the Santa Fe Institute and a professor at Portland State University.

“Did OpenAI lose control of these models? No, it could have shut them down at any time. Did these models ‘escape’? No, they are software programs that were always running on OpenAI’s servers,” Mitchell writes. “Like a jailed hacker given a computer, these models found a way to access the internet that allowed them to hack other networks. Did these agents form a ‘squad’ that ‘ran wild’? No, but the agents did collectively create a notable messaging system that presumably helped discover how to attack Hugging Face’s network,” she explains.

Some also argue that the capabilities of frontier models—the most advanced today—are being overstated. That is the position of Yann LeCun, a Turing Award winner often likened to the Nobel of computing. The former head of AI at Meta reacted this weekend to Amodei’s call at Anthropic to slow AI progress: “Right. Dario was already claiming that GPT2 was too dangerous to open source back in 2019. I made fun of them then. Everyone should make fun of them now.”

Regarding proclamations that the world could end by 2030, many observers are asking those making such claims to publish the scientific arguments that support them. But that is not the same as saying AI is harmless. “Absurd: Humans will be extinct in five years. Equally absurd: Everything will turn out fine. The two dominant narratives are both completely wrong,” tech entrepreneur and AI analyst Gary Marcus said on social media on Monday. His position captures what many experts think: we are facing something very large that will change everything, though no one knows when or to what extent.

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition