The human cost of AI

IT IS NOT a new phenomenon that technology has drawn people closer by transforming how they communicate and entertain themselves.

- SUGANDHA MUKHERJEE

From the days of SMS to team chat platforms, people have built new modes of conversation over the past two decades. But these interactions still involved people. With the rise of generative artificial intelligence, online gaming and viral challenges, a different form of engagement has entered daily life, and with it, new vulnerabilities.

Take chatbots for instance. Trained on vast datasets, they have become common tools for assisting with schoolwork, travel planning and even helping a person lose 27 kg in six months. In one study, titled "Me, Myself & I: Understanding and safeguarding children's use of AI chatbots," chatbots are being used by almost 64% of children for help with everything from homework to emotional advice and companionship. And, they are increasingly being implicated in mental health crises.

In Belgium, the parents of a teenager who died by suicide alleged that Chat GPT, the AI system developed by OpenAI, reinforced their son's negative worldview. They claimed the model did not offer appropriate warnings or support during moments of distress.

In the US, 14-year-old Sewell Setzer III died by suicide in February 2024. His mother Jessica Garcia later found messages suggesting that Character.AI, a start-up offering customized AI companions, had appeared to normalize his darkest thoughts. She has since argued that the platform lacked safeguards to protect vulnerable minors.