The risks of AI porn

Twenty-two-year-old Brazilian, Pedro Silva — not his real name — still does not know if he is enjoying the sexting he has been engaged in recently. On the one hand, he really likes the appearance of the person he’s chatting with and the interest she shows in the conversation. On the other hand, he feels a lack of authenticity. And he is not wrong: the woman he is in contact with is not real. She was created by Silva himself with an artificial intelligence (AI) program on an adult content website.

What Silva described as a “dissatisfying” experience has become a subgenre of porn that is growing in popularity. These are chatbots, like the virtual assistants that can be found on the travel websites of large corporations, which simulate racy dialogues. Programs such as Midjourney or Stable Diffusion, initially designed to generate images requested by the user in writing such as “draw me a dog dressed in a suit,” have been modified to create adult content, including videos. AI-generated pornography is spreading at record speed, creating increasingly realistic characters and raising almost philosophical questions about human sexuality and our relationship to pornography.

Easy access to the tools to develop these programs, driven by the curiosity generated by ChatGPT, has attracted those quick to see the opportunity to exploit them for adult entertainment. Most AI companies share their codes to encourage programmers to create their own programs, which has quickly led to uncensored software clones.

John Smith, a 28-year-old American engineer whose name has been changed, chairs the community of Discord Unstable Diffusion, a free version of the AI image production software, Stable Diffusion. Smith suffers from aphantasia, a condition that prevents him from conjuring images in his mind and he was relieved to learn that there were programs that could generate them from text. He refers to his computer as his “prosthetic imagination.” Although such software is not explicitly intended for porn, Smith acknowledges that his attempts to use it for that purpose were almost immediate. Unstable Diffusion was officially launched in August 2022, the same month as Stable Diffusion, though it launched subsequent to Stable Diffusion due to problems with the crowdfunding campaign. “Everyone knew it would end up being used for porn; the question was when,” Smith says.

It wasn’t easy at first, but Smith’s user community viewed it as a challenge. “It was a race to make genitals that didn’t look like soup,” Smith quips, explaining that at first it was a question of luck: “It was like combining porn with a slot machine. I’d keep clicking the button to see if the next image was realistic and I’d get excited.” He prides himself on having been one of the first to achieve a recognizable image of a subgenre of hentai, which is anime — or animation — porn. In July, the platform’s CEO stated that they generate more than 500,000 images a day. Smith has personally contributed more than 1,000, from the request channel on Discord.

Jack Peters, the pseudonym for a 32-year-old American IT expert, is also a member of the community. His first foray into AI-generated porn was through a free app on his cell phone, but he only achieved realistic content using more powerful programs. “When I started, I didn’t quite understand the prompts,” he explains. His first attempt to create images of naked women produced nothing more than “gruesome amalgamations of flesh,” but he recognized the potential of these tools to generate quality images.

“After the first few attempts, I was hooked, despite how horrible they looked,” Peters says. Specializing in the production of digital art or anime images, he initially found it difficult to generate anything realistic. Now, he says, any fantasy can be created without much effort. “With tools like ControlNet, you can position a doll however you want, and the model will generate a person in that exact pose.”

What began for Smith and Peters as a tool for personal use has become a thriving business. On Instagram, it is increasingly common to find profiles of scantily clad virtual characters in provocative poses. Although the platform already has AI-created influencers such as Lil Miquela and Shudu — in 2022 there were 35 verified accounts of AI-created characters — there is a growing presence of sexually-innuendoed accounts of this type. One example is the Emily Pellegrini account, which has more than 279,000 followers. The @gorkamale account, which features male characters, has almost 37,000 followers.

But these figures can mainly be found on Fanvue, a competitor of OnlyFans that accepts creations made with AI. The terms of use state that the content must pass a test to assess compliance with the rules and should make clear that the account does not represent a real individual. On Patreon, fans support their favorite AI character by purchasing packages of erotic photos.

Jürgen Schmidt, the name used by one of Emily’s creators, explains that the model’s characteristics emerged through a consensus with team members. “Her name is a kind of inside joke: we all like San Pellegrino water (popular sparkling water from the Italian Alps),” he explains. The model has 4,600 followers on Fanvue, where AI-generated images of her naked are shared. Despite this, Schmidt confesses that her creation doesn’t turn him on: “As far as I’m concerned, just knowing it’s not real makes it unnatural. But I don’t find it strange either. People use their imagination, that’s what’s been going on for a long time with hentai.”

Schmidt states that they are still seeking to perfect Emily: “It’s a fun project; we want to see how far we can go,” he says. He sees no problem with the fact he may be contributing to unattainable beauty standards and argues that those standards have already been set by women who exist in reality: “Social media is already a fake world, obsessed with perfection. Emily is not much different from Kylie Jenner.” For him, AI is the future of the porn industry: “It will be the best porn ever, even more so when combined with virtual reality technologies,” he says.

While acknowledging the inherent risks, such as the potential for addiction, Schmidt believes the benefits will outweigh the downsides. His optimistic prognosis includes the idea that, thanks to AI, it will be possible to do away with the more harmful aspects of the industry: “It will be possible to eliminate the entire negative side of porn,” he says. “In the future, no one will be forced to participate in it.”

In a format similar to ChatGPT — which does not allow adult content — other text generators explore intimate conversations with users, based on a character that may or may not have been created by the user. According to Brazilian Pedro Silva, these programs are still very primitive. “It doesn’t sound natural,” he says. “Sometimes it makes grammatical mistakes, says things that don’t make sense and that don’t even have anything to do with the previous sentence.”

Outside the adult sphere, Meta, Mark Zuckerberg’s company which owns WhatsApp, Facebook and Instagram, is testing its own chatbots based on celebrities such as Kendall Jenner and rapper Snoop Dogg. Earlier this year, influencer Caryn Marjorie launched her virtual clone, but had to revise it when she noticed that users were starting to steer the conversation towards sex-related topics.

What Marjorie was trying to avoid is exactly the aim of porn actress Riley Reid, who has just launched her own platform, Clona, for porn industry professionals who want to offer their followers the experience of interacting with them. Reid’s company analyzes the speech of each industry professional signing up and creates a bot for uncensored conversations with them, simulating their voices. It is intended for an audience over the age of 18 and costs $30 a month.

Image generation programs have also led to the phenomenon of deepfake. The term defines any medium in which the protagonist’s face, and often body too, is replaced by another in order to make people look like they are doing or saying things they have never actually done or said. Recently, such deepfake videos of celebrities from former president Barack Obama to actor Tom Cruise, have been rife on social networks.

A report from September 2019 began to ring alarm bells when it revealed that only 4% of these deepfake online creations were not pornographic. In September last year, a group of 26 minors were accused of using that same technology to digitally undress classmates, who were also minors, in the Spanish town of Almendralejo, Badajoz. Nina Jankowicz, the former director of the U.S. Disinformation Governance Board, was also a victim of these programs in response to her attempt to control fake news. Last January, Taylor Swift saw Twitter flooded with AI porn featuring her face and body.

Henry Ajder, an expert in AI ethics and author of the 2019 report, argues that the spread of deepfakes compromises the power of consent. “It is possible that the bulk of videos made with deepfakes don’t have consent,” he says. As part of the study, Ajder infiltrated a group of producers of such videos and found that 60% generated non-consensual porn starring women who were part of their daily lives.

“I worry that non-consensual pornography will become the norm,” Ajder says while stressing the importance of criminalizing lack of consent in the field, though he believes that regulation will come from the consumer: “I expect that most people don’t want videos made without consent,” he says. Members of the Unstable Diffusion community agree: they have a zero-tolerance policy on deepfakes and expel anyone who asks for help in making them. Like most creators, they also regulate their content and prohibit elements such as the appearance of minors or images of sexual violence. However, Ajder takes a pessimistic view of AI’s evolution. “What used to require very specialized tools is now available to everyone,” he says.

The researcher explains that although deepfakes have become popular mainstream content, they were originally conceived for porn. “These videos originated in a Reddit [forum] thread asking for explicit content from celebrities who had never been involved in such a thing,” he explains, predicting a bleak future for female celebrities: “It will be virtually impossible to be a woman and a public figure without porn emerging with you as the main character.”

The unlimited capacity for creation and customization makes AI the ideal tool for those seeking the perfect fantasy. Will Johnson, owner of a cybersecurity company in the U.S. who prefers not to reveal his real name, believes AI generated porn possesses a potential that “conventional” porn cannot fulfill. “I can create the scenes I want, and really participate,” he says. For Peters, these programs also combat the trafficking of women: “AI videos don’t work well yet, but once they get better, I think they will also be safer. They will become so perfect that it won’t be profitable to produce videos by illegal means.”

But the spread of this type of porn also leads to questions about what turns us on or not and our conception of what sex is. Ana Lombardía, sexologist, cautions: “It will become increasingly customizable and interactive, and the more pleasurable, the greater the potential for dependency.” Lombardía also fears that the extrapolation of such behavior will distort sex.

“In some societies, connection with another person is already being rejected in favor of fictitious characters, as in Japan,” she says, adding that she shares Ajder’s concern about the lack of consent: “It is likely that this will lead to the trivialization of sexual crimes. Porn requires consent and that can be removed with the use of programs.”

As AI shoulders its way into the porn industry, questions arise about its effects on the future of our sexual behavior. Like Joi, the holographic character in the movie Blade Runner 2049 (2017) played by Ana de Armas, these artificial creations promise real interaction with their followers, even if in many cases their digital origin is made clear. As a user of this technology, Peters says that he was very turned on in the beginning, but not anymore: “The first time I got a naked body without major deformations was a rush, but now I have generated more than 100,000 images and that desensitizes you,” he says. Nevertheless, he confesses that he will continue to consume it: “Porn is fun and exciting, whether it’s a real person or not,” he says.

Sign up for our weekly newsletter to get more English-language news coverage from EL PAÍS USA Edition