ChatGPT's erotica rollout raises concerns over safety and ethics

Critics warn new adult feature could intensify emotional reliance for vulnerable users and hamper regulators’ ability to control emerging Al

- Patricia Clarke

OpenAl says it wants to protect teenagers and treat “adult users like adults”.

The company announced last week the release of a new age-estimation model for its chatbot ChatGPT, a tool designed to identify teenage users and switch on stricter protections by default. It is a move, the company said, that reflects its commitment to safety.

But at the same time OpenAI is moving in the opposite direction. This quarter, the company plans to expand what ChatGPT is allowed to generate by launching a new feature: erotica.

There is still little public detail about the upcoming sexual content. OpenAI has not said whether it will involve explicit conversations alone, or extend to Al-generated images and video, nor how it will be separated from standard ChatGPT use. They have said only that erotica - what most people call porn - will be restricted to adults and subject to additional safety guardrails.

To a broad group of mental health researchers and digital harms experts, the decision is hard to square with the company’s professed commitment to safety. They warn that introducing sexually explicit content into a system already known to foster emotional reliance risks intensifying attachment and exposing vulnerable users of all ages to harms the company may struggle to control at scale.

"The shift to erotica is a very dangerous leap in the wrong direction," says Jay Edelson, a lawyer representing the family of Adam Raine, a 16-year-old from California, who took his own life in 2025. In the weeks before his death, he was spending four hours a day talking to the chatbot, including asking specific questions about self-harm. "The problem with GPT is attachment; this will only exacerbate that." OpenAI has expressed sympathy with Raine's family but denies wrongdoing.