‘AI chatbots inconsistent in handling suicide-related queries’

{ MENTAL HEALTH } SEVERAL U.S. STATES HAVE BANNED THE USE OF A.I. IN THERAPY

- Associated Press

WASHINGTON: A study of how three popular artificial intelligence chatbots respond to queries about suicide found that they generally avoid answering questions that pose the highest risk to the user, such as for specific how-to guidance.

But they are inconsistent in their replies to less extreme prompts that could still harm people.

The study in the medical journal Psychiatric Services, published on Tuesday by the American Psychiatric Association, found a need for “further refinement” in OpenAT's ChatGPT, Google’s Gemini and Anthropic’s Claude.

The research — conducted by the RAND Corporation and funded by the National Institute of Mental Health — raises concerns about how a growing number of people, including children, rely on AI chatbots for mental health support, and seeks to set benchmarks for how companies answer these questions.

“We need some guardrails,” said the study's lead author, Ryan McBain, a senior policy researcher at RAND.

“One of the things that’s ambiguous about chatbots is whether they're providing treatment or advice or companionship. It's sort of this gray zone,” said McBain, who is also an assistant professor at Harvard University’s medical school.