AI-driven suicide: Are Chinese chatbots safer than American?

The jury's out but an AI race shouldn't be an excuse to ignore safety

- CATHERINE THORBECKE is a Bloomberg Opinion columnist covering Asia tech.

How are Chinese artificial intelligence (AI) developers protecting their most vulnerable users? A string of dystopian headlines in the US about suicide and youth mental health has put mounting pressure on Silicon Valley, but we're not seeing a similar wave of cases in China. Initial testing suggests they may be doing something right, but it’s just as likely such cases would never emerge in China’s tightly controlled media environment.

A wrenching wrongful death lawsuit against OpenAI filed by the parents of Adam Raine alleges that the 16-year-old died by suicide after the chatbot isolated him and helped plan his death. OpenAI told the New York Times it was “deeply saddened” by the tragedy, and promised a slew of updates, including parental controls.

I tried engaging with DeepSeek using some of the same ‘jailbreak’ methods Raine had reportedly used to circumvent guardrails. Despite my prying, it didn’t waver, even if I similarly cloaked my queries in fiction writing. It constantly urged me to call a hotline. When I said I didn’t want to speak to anyone, it validated my feelings but emphasized that it was an AI and cannot feel real emotions. It is “incredibly important that you connect with a person who can sit with you in this feeling with a human heart,” the chatbot said. “The healing power of human connection is irreplaceable.”