OpenAI chief executive Sam Altman. The start-up says it wants to have long-term research on the effects of sexually explicit chats and emotional attachments before making a product decision
OpenAI has shelved plans to release an erotic chatbot “indefinitely” as it refocuses on its core products, following concerns from staff and investors about the effect of sexualised AI content on society. Sam Altman’s start-up had already delayed the release of its “adult mode” amid internal discussions over whether to scrap the model entirely, according to multiple people familiar.
The sexual chatbot faced growing pushback over how it could encourage unhealthy attachments to AI systems and expose minors to problematic sexual content. The erotic model was on hold with no timeline for its release, OpenAI confirmed. The $730bn AI lab said it wanted to have long-term research on the effects of sexually explicit chats and emotional attachments, before making a product decision, acknowledging there was no “empirical evidence” at present.
It marked the latest decision by OpenAI to drop what executives have termed “side quests” in favour of devoting resources to productivity tools, bringing together products such as coding assistants and ChatGPT into one “super app”. OpenAI on Tuesday said it was winding down its Sora video generation model and social app.
The erotic model had proved particularly controversial for OpenAI, at a time when social media groups such as Meta are facing a legal reckoning over their products’ harm to children.
Elon Musk’s xAI last year made a big push to attract users with adult content but faced a global backlash over its Grok model creating fake sexual images of real people, including children.
The tension highlights the pressure on OpenAI to boost engagement and find new avenues for growth in a competitive market, while grappling with the ethical and reputational risks of products that blur the line between utility and emotional dependence.
OpenAI’s flirtation with adult mode had caused disquiet among some of its investors, according to two people familiar with the matter, because of the risks of such a product and its relatively small upside for the business. OpenAI’s adult mode had also raised alarm bells among staff about whether pursuing a product that encourages romantic use is at odds with its founding mission to ensure that AI benefits all of humanity. “AI shouldn’t replace your friends or your family; you should have human connections,” said one former senior employee, who said they left in part due to the issue.
Alongside rising concerns about the project’s social impact, the start-up has faced technical challenges in creating an adult mode. OpenAI had difficulties in training AI models that previously avoided such conversations for safety reasons to produce explicit content, two people familiar with the project said. They added that using datasets including sexual content posed challenges, including removing illegal behaviour such as bestiality or incest. OpenAI said it was carefully defining the right boundaries and training the model to distinguish certain content in nuanced scenarios.
Code in the ChatGPT web app suggests that the adult model, called “Citron mode”, may require users to verify they are over 18 to access it.
Over the past few months, OpenAI has introduced a new age prediction system after several lawsuits were brought by families of teenagers who claim ChatGPT harmed them. The technology has an error rate of more than 10 per cent, one person familiar with the matter said, heightening concerns about underage access. The Wall Street Journal first reported the error rate and that OpenAI’s expert advisory council on wellbeing had raised concerns about the adult product. OpenAI said its age prediction system performs in line with industry standards, and it continues to invest in improving its accuracy and reliability.
[L/A]
s
