Is digital violence in an AI era yet another threat to women?

Nowadays, anyone can create a hyperrealistic porn video with artificial intelligence (AI). All it takes is a photo, an email and between $10 and $50. These are the only requirements for the 96 apps already in existence to receive “compelling nudity” on tap for a year. “It’ll be a piece of cake to add someone to a porn scene,” says the website of one of the most popular apps, with 1.5 million monthly visits. “This is the crème de la crème because of how easy and powerful it is,” says another. It is precisely the ease of generating content without consent and sharing it that worries experts in the Latin American region where, in some countries, digital gender-based violence is hardly considered a crime and where commitment to its prevention is flaky to say the least.

The perverse use of social networks and virtuality opened more than a decade ago the debate on whether or not to criminalize these behaviors that mainly affect women; in countries such as Mexico, they account for 89% of the victims. Generative artificial intelligence has reopened the discussion on whether criminalizing the abusive use of intimate images is the solution. For María Camila Correa Flórez, professor of Criminal Law at the Rosario University in Bogotá, it is only one aspect of it: “Comprehensive public policies are needed to prevent, as well as to sanction,” she says. “Criminal law can intervene up to a point and make people understand that it is a crime. So, it’s a good first step.”

Colombia, however, is close to the bottom of the heap when it comes to protective legislation. Along with Nicaragua and Venezuela, online gender-based violence is not covered by any legal framework in Colombia. The production or distribution of sexual images without consent is not a crime, although in exceptional cases may be prosecuted as other criminal offenses such as extortion or hate crimes.

Argentina and Chile, on the other hand, have set up a more protective legal framework for children and adolescents, though there is a huge gap when it comes to adult women’s rights. Brazil has criminalized the non-consensual distribution of intimate images since 2018 and Peru also incorporated online sexual harassment and blackmail into its Penal Code. Mexico has been one of the countries most concerned with violence against women online and has a large package of federal and state reforms to prosecute these practices, known as the “Olimpia Law,” in a tribute to Olimpia Coral Melo, victim in 2014 of the unauthorized sharing of a video with sexual content.

However, despite Mexico’s pioneering legislation, the crime itself remains difficult to nail. “You can have laws... but the materialization of the sentence is the most difficult thing,” Elvia Karina Ramirez Juarez, a Mexican litigation lawyer, tells EL PAÍS. “It sounds all good and well on paper, but getting these videos downloaded from the internet is very unlikely. The gap between paper and reality is vast.” Julio César Bonilla Gutiérrez, Citizen Commissioner of Mexico City’s Transparency Institute, acknowledges that “there is still a long way to go. Although the legislation is progress towards dealing with [online violence] better, the reforms must be improved. But the danger of doing nothing is that digital violence tends to become physical.”

Fear of reporting the crime

With or without criminalization, very few cases of online gender-based violence have been resolved by the region’s courts to date. The lack of charges stems from the bureaucracy involved in reporting the crime, fear of revictimization, and feelings of shame coupled with distrust in judicial authorities and their lack of technical expertise. “The obstacles to prosecuting violence against women on and offline are the same,” says Luz Patricia Mejía, Technical Secretary of the Belém do Pará Convention’s Follow-up Mechanism. “In both cases, women do not report the crime for fear of not being believed or because they still think it is a matter of the couple’s intimacy. The only difference is that technology is moving so fast.”

Public policies, criminal consequences and conversations with large platforms are some of the solutions that Mejía proposes to regulate AI when it comes to gender-based violence. “We have to be prepared for debates we haven’t yet had; for example, freedom of expression, which is the pillar of democracy, can no longer be untouchable. We are headed towards transitioning to a new world that forces us to rethink these models. State governments are more limited in their capacity to regulate than Twitter and Meta. We will have to engage globally regarding applications. There is no other way.”

Cecilia Celeste Danesi, UPSA researcher and author of The Algorithms Empire, suggests that regulation will be informative rather than prohibitive: “The proposals being pushed by the European Union and the United States involve putting a watermark informing the consumer if the content generated is fake; this is also linked to the consumers’ right to be informed,” she says. “AI could even be part of the solution if it is used to track down this content.”

According to Katya Vera Morales, gender expert officer from the Organization of American States’ (OAS) Cybersecurity Program, the victims need to be at the center of the policies: “Many of them do not want to report the crime and subject themselves to the process of re-victimization that this usually entails,” she says. “Sometimes, they just want to stop their photos from being distributed — to stop being sextorted. We have to ask ourselves, what does access to justice mean for victims? And, based on the answers we get, we then design a course of action.”

Complaints on social networks

Although the reporting of these crimes does not yet reflect the threat, dozens of women have spoken out about being victims of generative AI on social media. “The law [Olympia] should be made to compensate and take care of the victims,” Eri Gutierrez, a content creator and technology specialist, tells EL PAÍS. “And it’s not doing that. They have to go after the servers. It’s true that stopping the flow doesn’t stop the damage, but it limits it. The data generated in the applications stays on the servers, and the technique is being perfected so that the user has a wider repertoire. You see horrible things on the dark web.”

“It’s incredibly easy to create the hottest girls with just a few words, making them do what you tell them. Go crazy!” says a forum that compares “deepfake porn” apps, which gives a very high score to an application that creates sexy avatars of women with user-supplied photos for a fee. In order for the images it produces not to be blurred, you have to subscribe. It costs less than $10. Ramirez, a Mexican litigator, believes that the key to nailing these cases is in proving responsibility: “This debate is only being had by analysts, not politicians,” he says. “In Mexico, there has never been a restriction on apps or social networks. And, honestly, I don’t see it coming.”

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition