Character.AI promises to bar minor users

The start-up, which creates AI companions, faces lawsuits from families who have accused Character.AI’s chatbots of leading teenagers to kill themselves, write Natallie Rocha and Kashmir Hill from San Francisco and New York

- 

Daniel De Freitas, co-founder of Character.AI, demonstrates the app in 2022. The company said people under 18 would be barred from using its chatbots starting later this month.

(NYT)

Character.AI said on Oct 29 that it would bar people under 18 from using its chatbots starting later this month, in a sweeping move to address concerns over child safety.

The rule will take effect Nov 25, the company said. To enforce it, Character.AI said, over the next month the company will identify which users are minors and put time limits on their use of the app. Once the measure begins, those users will not be able to converse with the company’s chatbots.

“We're making a very bold step to say for teen users, chatbots are not the way for entertainment, but there are much better ways to serve them,’ Karandeep Anand, Character.AI’s CEO, said in an interview. He said the company also planned to establish an artificial intelligence safety lab.

The moves follow mounting scrutiny over how chatbots sometimes called AI companions can affect users’ mental health. Last year, Character.AI was sued by the family of Sewell Setzer III, a 14-year-old in Florida who killed himself after constantly texting and conversing with one of Character.AI’s chatbots. His family accused the company of being responsible for his death.

The case became a lightning rod for how people can develop emotional attachments to chatbots, with potentially dangerous results. Character.AI has since faced other lawsuits over child safety. AI companies, including the ChatGPT maker OpenAI, have also come under scrutiny for their chatbots’ effects on people — especially youths — if they have sexually explicit or toxic conversations.