A bad reason for ChatGPT to talk about suicide

Sam Altman's words offer unsettling hints on how the platform balances safety and freedom of users.

- JOSHUA PEDERSON GUEST CONTRIBUTOR

SAM ALTMAN wrote in a blog post about navigating the sometimes conflicting priorities for his company's AI engine.

(WIN MCNAMEE Getty Images)

LAST MONTH, the Senate Judiciary Subcommittee on Crime and Counter-terrorism held a hearing on what many consider to be an unfolding mental health crisis among teens.

Two of the witnesses were parents of children who'd committed suicide in the last year, and both believed that AI chatbots played a significant role in abetting their children’s deaths. One couple now alleges in a lawsuit that ChatGPT told their son about specific methods for ending his life and even offered to help write a suicide note.

In the run-up to the September Senate hearing, OpenAI co-founder Sam Altman took to the company blog, offering his thoughts on how corporate principles are shaping its response to the crisis. The challenge, he wrote, is balancing OpenAI's dual commitments to safety and freedom.

ChatGPT obviously shouldn't be acting as a de facto therapist for teens exhibiting signs of suicidal ideation, Altman argues in the blog. But because the company values user freedom, the solution isn’t to insert forceful programming commands that might prevent the bot from talking about self-harm. Why? “If an adult user is asking for help writing a fictional story that depicts a suicide, the model should help with that request.” In the same post, Altman promises that age restrictions are coming, but similar efforts I've seen to keep young users off social media have proved woefully inadequate.