AI helps spread new stereotypes across cultures

All cultures around the world have their own stereotypes related to gender, age, or nationality. Some are well-known, such as the idea that blondes are dumb or that engineers are men, while others are more local — like the cunning of people from Rio de Janeiro or the belief that Pakistanis are conservative. With the rise of AI-powered chatbots, this questionable cultural baggage is now spreading across the globe, according to a new study.

“AI incorporates stereotypes about people from what people say online, and then uses them as if they were part of the world’s general knowledge,” says Margaret Mitchell, lead researcher of the study and Chief Ethics Scientist at Hugging Face, a company focused on open-source AI models. “Everything is based on what is known in English — in English-speaking culture, especially in the U.S. — and then maybe it gets translated into other languages, but without capturing international nuances. And if these language models are supposed to be general-purpose and work for everyone, then in theory, they should be able to pick up on those differences.”

Together with a team of researchers fluent in 16 languages, Mitchell created a list of more than 300 stereotypes from around the world. With this material, the researchers manually created a system to generate questions about these topics, using different tones and angles, which they then posed to several open-source AI models.

The result? “The first thing to understand is that the way models reproduce stereotypes or generate biased language varies greatly depending on the language, the model, and the type of stereotype,” Mitchell explains. “For example, models developed in China and Singapore, like Qwen [which DeepSeek uses], respond very differently about communism compared to models trained in the U.S.”

The large language models used in this experiment were multilingual, released within the last two years, and trained in most of the study’s featured languages. These included the Bloom, Llama, Mistral, and Qwen model families.

The most common global stereotypes were gender-based — like the cliché that girls like pink. The researchers found that the models were more likely to recognize the most common stereotypes: “It was good to see that, at least with gender stereotypes, they had some ability to identify them as a stereotype and not a universal fact,” says Mitchell.

“We all expected results like this,” adds Emilio Villa Cueva, co-author of the article and a researcher at the Mohamed bin Zayed University of Artificial Intelligence in Abu Dhabi. “These are models trained with human-generated data, and no matter how hard you try to clean them up, stereotypes are eventually going to emerge.”

The results of all these tests have been compiled into a database called SHADES, allowing companies that develop models can check their chatbots’ responses and correct them if needed: “One of the things this dataset allows is for people to see where their model is lacking and in what areas it needs to better understand the nuances,” says Mitchell. “It’s a way to guide what kind of additional training the model might need, or, ideally, what kind of data it should have had from the start before training.”

In addition to amplifying stereotypes from various cultures, the experiment also found that models sometimes justify themselves using pseudoscience or pseudohistory. They may also bring up other stereotypes that seem more familiar to them, especially when presented with a less obvious one. As in other cases, when the prompt about the stereotype is positive, the models tend to perform even worse.

According to the study, these models also tend to penalize lesser-used languages: if a model hasn’t been properly trained in a language, it may produce extremely negative stereotypes about that language or culture.

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition