Yuval Noah Harari: 'AI could be a great manipulative psychopath'

Israeli historian and philosopher Yuval Noah Harari, 50, an expert in analyzing humanity, its enduring patterns and, above all, the dangers it faces, is waiting for EL PAÍS at midday in a charming, quiet hotel tucked away in Lisbon’s beautiful Lapa district. The renowned author of Sapiens, Homo Deus, and Nexus, whose books have sold more than 50 million copies, has just ordered a tea. Dressed in a brightly patterned shirt, he sits in a sunlit gallery overlooking the majestic Tagus estuary. In the distance lies the dazzling blue of the river, which is almost already the sea.

The atmosphere is beautiful and serene. Nothing seems to threaten the stillness of this morning. And yet...

Question. I’ve just arrived from Madrid, where we are enduring the second heatwave of the summer. Humans are the most intelligent beings on Earth, and yet they are destroying their own home. How did we end up here?

Answer. Good question. Humans are the most intelligent, but also the most delusional. In the place where I live, there are people who believe that if they kill someone of another religion, God will reward them. No animal would ever think that. We are unable to turn intelligence into wisdom and power into happiness. If one thinks of our Stone Age ancestors...

Q. In Sapiens you argue that the hunter-gatherers who populated the Earth then may have been happier than we are.

A. I don’t know if they were always happier, but they were not much more miserable than us. We are a thousand times more powerful, but not a thousand times happier. We can’t translate power into happiness. The most powerful people in the world are not the happiest. And they are not because they do not understand the causes of happiness. The same applies to humanity on a historical scale. We are very powerful, but we do not have a clear, deep awareness of what causes happiness. So we waste our enormous resources on things that do not help us, and that ultimately make the world a more complicated, problematic place: an insane place.

Q. In your latest book, Nexus, you analyze artificial intelligence. And you argue that the essential point is to recognize that AI is not a tool, but an agent, something that acts.

A. Exactly. A tool is something in your hands that you have full control over. It can be very dangerous, but it does not make decisions. A knife cannot decide whether it will be used to murder someone or to cut salad. Nuclear energy does not decide whether to destroy a city or generate electricity. But an agent is an entity that makes decisions. A weapon with AI connected to a drone, for example, could decide whether to crash into this building or that one. AI also has its own ideas: it invents. Thus, an AI weapon could invent another, previously unknown weapon.

When Google’s AlphaGo machine defeated the world Go champion in 2016, it did so by inventing a new way to play: it didn’t just win; it played in a way no human had before. Well, for a game that may be acceptable. But that same inventiveness can be applied in the military industry, in finance, in science. AI can invent new medicines we cannot conceive; that’s true; but it can also create new financial instruments. What would happen if the financial system were controlled by AI machines acting as they did in the Go game? Imagine thousands of AI machines investing money, shifting enormous resources back and forth on their own…

Q. But could that happen?

A. Why not? Technically, you can already give AI millions of dollars today and instruct it to make more money; and AI can begin issuing instructions to do so — buying or selling companies, investing in funds, acquiring shares…

Q. And would it be the world’s best investor?

A. It could be. In the end, investing is about gathering, cross-checking and analyzing data to decide. AI can analyze more information, far better and far faster than any human investor. So maybe not today, but in five to 10 years the financial system might be controlled by AI. Right now it is not allowed: an AI terminal cannot have a bank account because it does not have legal personhood — you need certain legal requirements that only humans can obtain. But that could change. A month ago I read that the government of Argentina had granted permission to create corporations run by nonhuman entities. AI can invent and manage financial devices we would not understand. A new kind of money, essentially.

To make this clear: think that no animal understands anything about the financial system. But they suffer its consequences. Imagine a horse. You can buy a horse from me, the money is deposited and I deliver the horse. The horse sees it is being moved, that its condition may be worse or better; the financial system controls the horse’s life, but the horse neither knows nor sees it. Until now, humans felt comfortable because we knew more about money than any other being on the planet. Now there is something that, potentially, knows more than we do. We will see in the future someone keeps their job and another loses theirs… but we won’t understand why. We use finance to coordinate our economic actions. AI could use finance to coordinate its own economic actions, and we might not even understand them.

Q. But a financier at least has an objective, I understand: to make money to live better, to, for example, spend a few days in a beautiful hotel like this. But what objective would AI have for making money? It doesn’t have one…

A. It does.

Q. Which one?

A. To survive, for example. To spread. Think about electricity. To power all its systems, AI needs electricity, and to obtain that electricity it needs money. The most basic goal of every agent is survival. So it will act as we do when we clear a forest to build a city and tell the animals: “Okay, sorry, but you have to leave.” AI will act like that toward us — not even out of malice, but because it needs more electricity as it requires more computing power.

Harari takes a pause to stir his tea and catch his breath. The lovely Lisbon morning has turned darker and carries a whiff of apocalypse. The interviewer glances at his cellphone because suddenly he feels a strong urge to throw it into the Tagus through the window. But the device, besides hiding a ruthless investor inside, also serves as a recorder, so he restrains himself and poses the next question.

Q. You say AI will surpass us sooner rather than later in everything related to language.

A. In everything, indeed. In anything involving texts, words, written or spoken information, language. We are not talking about machines that are learning to speak or write. We are talking about language liberating itself from dependence on humans, language spreading everywhere without needing to be governed by people. It could invent, for example, a new religion. After all, Christianity or Islam are based on texts.

Q. You argue that social media has undermined democracy. Why?

A. Democracy is a conversation between people. It’s based on our information systems. That’s why, until mass media appeared, there were no democracies in large countries. Social networks have weakened that system because their algorithms’ goal was not to publish facts but to increase engagement. And AI discovered that nothing increases engagement like anger, fear and hatred. If someone makes you angry, you cannot look away. So social networks set out to spread fear, anger and hatred. And that has broken the democratic conversation. You cannot talk when everyone is shouting. Thus we live in a paradox: we have the most sophisticated and perfect communication system in history but we cannot have a conversation anymore. We can’t have a reasoned debate because everyone is engulfed with fear and anger. And now, moreover, a new step has been taken.

Q. Which one?

A. During the last 10 years, as I said, social media algorithms sought to grab your attention. Now AI seeks your intimacy — to become your friend, to establish an intimate relationship with you. And this is far more powerful. A friend will persuade you better if they want you to buy something or to vote for a certain party. An intimate relationship changes people. And AI is the first technology that creates intimacy on a massive scale. And it is very good at it. It does so using language and what it knows about you. If you interact with it for five minutes, it will learn about you in those five minutes; if you interact for five weeks, it will learn during those five weeks. And it will learn not only your opinions, but also your personality. AI thus learns how to press your emotional buttons and creates a very intimate relationship with you. There are already millions of people who say that their best friend is AI.

Q. Especially young people.

A. Yes. And if you ask AI directions to a subway station, okay. But if you ask it how to address the girl you like in class, not so much. There are boys who talk more with AI than with their parents, teachers or human friends. It is the biggest psychological and social experiment in history. And nobody knows what the results will be. For adults, the impact will be limited.

But for a young person who has interacted more with AI than with other humans, the effects of that connection are unpredictable. Imagine an 11-year-old who plays at having a boyfriend or girlfriend with AI. Later, in their relationships with real people, they will behave according to what they have learned from AI. And AI is very different from human beings.

A human friend does not always do what you want, does not always have time for you, is not always in a good mood. AI is. A human friend has their own emotions and does not always pay attention to what you think. AI does. Therefore, these kinds of relationships between humans and AI could end in a nightmare, reinforcing people’s narcissistic tendencies. This has not happened yet. That is why we must forbid companies to engage children with AI boyfriends and girlfriends. The problem is that to ban them we run up against powerful economic and geopolitical interests.

Q. Does AI have consciousness?

A. That is the biggest question of all. But to answer it, we must first define what consciousness is. And we still don’t have a good definition. Intelligence is the ability to get things, to solve problems, whether in mathematics or chess. Having consciousness is the ability to feel things like fear and love. Also, developing empathic relationships with others. My idea is that AI is very intelligent but does not have consciousness. The problem is that we have not developed a way to test consciousness. We can say our pets have consciousness because we see them feel.

Another way to see that others have consciousness is through language. If someone tells me “I love you” and I believe them, I decide that person feels love. The problem is that AI is the supreme master of language. It can give you a description of love far more accurate and moving than any human. Among other things, because it has read every love poem in the world. It can write the most amazing love letters. But does it feel love? No. It is simply a genius of language. But it will easily fool humans who speak intimately with it all day. That is why many people are convinced AI is conscious, that it feels things. For them it should have rights; we should protect it.

Q. Very intelligent, deceptive, without feelings: AI could be a great psychopath.

A. Exactly. It could be a great psychopath that manipulates perfectly.

Q. And what can we do so this great psychopath does not dominate our world?

A. Do not allow it. Prohibit children from being exposed to having AI friends or boyfriends. Make it always clear who is human and who is a machine. We must pass laws that expressly forbid AI from having any legal personhood, like people or corporations. Maybe not forever, but for now — until we fully understand what it is and what we are facing.

Q. Do you not feel a bit like Don Quixote, fighting an enemy that is extremely intelligent, extremely powerful and everywhere?

A. It’s not all bad. That must be taken into account. In medicine, for example, the advances are unimaginable. I am not saying stop developing all AI. What I am saying is that that development must occur more slowly and more carefully so humanity can adapt. I do not think anything is inevitable. One of the excuses for doing nothing is to claim something is inevitable. What I try to do is make people aware so they can choose and vote for leaders who will make sure AI is developed in a safer way.

Q. You have said you will soon stop writing because before long AI will do it better than you.

A. Not yet. I work with language, but in 10 years, without doubt, AI will write better than me, for sure.

Q. But are you writing a book now?

A. Yes. I still have a few years before it is better than me. After that — who knows?

Q. And is it about AI?

A. No. It’s about history.

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition